From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10315C48BD1 for ; Fri, 11 Jun 2021 12:17:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DCF5761246 for ; Fri, 11 Jun 2021 12:17:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231522AbhFKMTK (ORCPT ); Fri, 11 Jun 2021 08:19:10 -0400 Received: from mail-mw2nam12on2084.outbound.protection.outlook.com ([40.107.244.84]:47281 "EHLO NAM12-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S231572AbhFKMTI (ORCPT ); Fri, 11 Jun 2021 08:19:08 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S8ftjPb1AK/HSoDydDtNhwUfEH2tFlrO2PfASxumD7fDuXheuWTV0ncp2diBRnymuOnMb/D5gcUuMVg6C9eq5cfqBj5qspESJJjeQIBqx8qOyGpbQ2MBkfbNZAOtSMtx8tMcBIp5Ny13b0K/AkQGv4SJnOx2ozwbCH1jHqjXo0WEDGo6Vco83c1IuvATJ5DP/rVP2Hh18tanYs/JHXe6Cq6qgjSHGH2ncqx0Ty4siUorKi6NFrhyRn0lTlVqqcyCRPVBbn6VU+ypFbX5LkEPtUaNHKojKTc/61+KFo42R6qBcW0z3jML/9bIZGad/i+p/v/hn6/IyxaXkNitiRkrDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wZywcGSP0xyi2nFOmF07hpnuTnaYfCY6XOfXF+MxwfQ=; b=cKziv3t3iuJH0xLUBCY86uU2Ar4/8Trp/2sDVD7Ie0m2gTm1rcB+DQQfJf2bDbNafQ1W9Vyntse2/wvcUbF7+sL+sxMg0/LFVDHaIPdWNwUNMsCD88a5elpFvfhwAxzLS5S/A6IkdnIEo+nxH45QcfOPlbIBapujPZ/b3KVGC8SNZ15TVHPAeEeftRuCR9e+yd6a93Iktry12HJbuQEzvPk7J9eb1OdcRMYtyeyT3ADO90yAYR5snFjgMyELCxHx5FSm5MEbhk1yl5zNo+HFbBhja0OCR2t8sgfGN13hGo6EVIuZkD08DnOI+Vxc4j70+IN8wRFfdCDzweAjPRCeOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wZywcGSP0xyi2nFOmF07hpnuTnaYfCY6XOfXF+MxwfQ=; b=CkSWHuHb8+hG6oVE/AXzgTUvqqMuNbfIUqIvxHqVItN1LhK18832jyQbiaDFtWT6nCYF7hhDiEydsq5VF7uNf71GwaArY0TvtT6JcN0YFZ5KJjlCB18gYUY2fVNPBnvPmc9r5d1Kuga4lvpP/BQrmb9l9roJdzBTm54FHSmwrmO5iOSnXS5Wrzww8Z6wDmK3BJUUUNDaZc+vHEump2WdUyFlkebjq9cViXfUF9yUIk1/V9QCqSz+WRp50SraOdhplXHyk57RD1hpVN9ITcx7ZMsrXDl9Wra74TpMAlIfpDzkIjhv2DQkyCtlKDRVSvLUyzVUsOtGAJBbPy/Jm0OPNg== Authentication-Results: techsingularity.net; dkim=none (message not signed) header.d=none;techsingularity.net; dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB3823.namprd12.prod.outlook.com (2603:10b6:208:168::26) by MN2PR12MB3712.namprd12.prod.outlook.com (2603:10b6:208:164::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4219.22; Fri, 11 Jun 2021 12:17:08 +0000 Received: from MN2PR12MB3823.namprd12.prod.outlook.com ([fe80::dcee:535c:30e:95f4]) by MN2PR12MB3823.namprd12.prod.outlook.com ([fe80::dcee:535c:30e:95f4%6]) with mapi id 15.20.4219.023; Fri, 11 Jun 2021 12:17:08 +0000 From: Zi Yan To: Mel Gorman Cc: Andrew Morton , Dave Hansen , Vlastimil Babka , Michal Hocko , Jesper Dangaard Brouer , LKML , Linux-MM Subject: Re: [PATCH 2/2] mm/page_alloc: Allow high-order pages to be stored on the per-cpu lists Date: Fri, 11 Jun 2021 08:17:02 -0400 X-Mailer: MailMate (1.14r5812) Message-ID: <7E7AFAD1-A08E-4DE1-B307-C604A01BDC8C@nvidia.com> In-Reply-To: <20210611083433.GA30378@techsingularity.net> References: <20210603142220.10851-1-mgorman@techsingularity.net> <20210603142220.10851-3-mgorman@techsingularity.net> <88FCC7AA-FAAA-4B87-B382-50BD54B2886B@nvidia.com> <20210610111821.GY30378@techsingularity.net> <3B44DF44-5669-40B6-A122-011F1A749FAA@nvidia.com> <20210611083433.GA30378@techsingularity.net> Content-Type: multipart/signed; boundary="=_MailMate_FFF019EA-49DF-4392-9C9A-0F595FFA07C3_="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Originating-IP: [216.228.112.21] X-ClientProxiedBy: MN2PR04CA0010.namprd04.prod.outlook.com (2603:10b6:208:d4::23) To MN2PR12MB3823.namprd12.prod.outlook.com (2603:10b6:208:168::26) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [10.2.82.166] (216.228.112.21) by MN2PR04CA0010.namprd04.prod.outlook.com (2603:10b6:208:d4::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4219.20 via Frontend Transport; Fri, 11 Jun 2021 12:17:05 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2a47070f-f024-48d2-faa1-08d92cd2d510 X-MS-TrafficTypeDiagnostic: MN2PR12MB3712: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w16n0XUQj83oLxRFeM23dHZM+m2jogEIj5ewR9bNVt00vUUtOc0M8MMTsyfcIUsmvULXYc7MtieKuzAt3daiTkBYamjGi5091QjApzFa8A+Dt6kRMDCPz+RXT/uBbL60s19hW16ji0L6xa7MZC0NbkSAO4ViTwap15ez+KTgxzBjWeWOX3VItCBYqezt43ddph7kjG1m6MXdUz6DZk9xWicvdOBKVwTmlO2CWslC5rQIfZefXwC0Am8Z9Hm3d0CRwAgTBt+jgdhclcFceZVQap5U56NEmEL121VdIuRgPyB0qnmrN95glJJv8oWQRoyK3O0HVMek5w3b3A79J0JmPgJMqajkx/UX0U97gSNP7OTuu4maOtBncPbq+AmfBivHgnXwc3q+joZtrES/+sru9htNUapqGEsGz7CFZ9iS74O6FiQwNaGFfiGPTt8FW+QoANu2/Q18uivCRUC4UV5KCIumYpucIHV9mrXrnGwJi+WQlrXV2SN+K73t6FNuB1kw8EBLUYh99zZrEWClXF3MNYTVF7/dwDg1gI+117joPYbsXQkOLXX3lS4BpemyYj0uUVvnm3ZGatPcp6VyPJFzlvvJkwCj+E2ODjT6eluuV3f4gX0QmiIPDnkDFA1VfIxbTKKxybaMK5q4RSOoQp9D++b5RGjxiXhzmtSt6r9au5M= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB3823.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(366004)(39860400002)(346002)(396003)(136003)(6666004)(26005)(8936002)(8676002)(36756003)(478600001)(83380400001)(66476007)(16526019)(66556008)(2906002)(186003)(66946007)(6916009)(235185007)(2616005)(956004)(54906003)(316002)(5660300002)(4326008)(86362001)(16576012)(6486002)(38100700002)(33656002)(21480400003)(33964004)(53546011)(45980500001)(72826004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGlhcUdWMUxwS1FyRGZKVDg4TmttUXBKZ2tkd2RCUklTZlRYajJ6cXoxcGxO?= =?utf-8?B?WGlxZzdBRE9hanJXZVM2VzF2anlCNjhKTDM2UnVkWEpTcEtMUlV2b01ZL2kz?= =?utf-8?B?WnlSVTBsU3cycUVYUXdoZGo5T2lhMEMzbkMzelArQ3VZOGFMbHU2SGxFQTFO?= =?utf-8?B?aERoandpM2htcFJSL2p5VE5UUmVFY2REZUJSOEdIYm91ZmYybjNxeXJnNHBq?= =?utf-8?B?dVNmYnZsSCtFU1FSZkZHOEdXdzROdENSR2hQVlc2aWpUaEZoSlM4T3A5dG8r?= =?utf-8?B?bGtrbDRObDlJTGtaR1UyN1VZVnR5NURvYU9IK0pMVCtZRGxsSGVVdUpQV1Rw?= =?utf-8?B?SjgwT0FVWFdXMmYwVW0zWWQyTFpGQVREcnJSaENQc3RHV1Nac2xjMHJkTzdk?= =?utf-8?B?ejk5MVRWOXI3V2hJeHExdnhwellLVTRpWWloQSt6b2tYK2Y5MDB4bGJpV1Nh?= =?utf-8?B?U29YVmdnQjdIREVEU1RTV1ZLU28yK0V6S0dYd1haeC9nQXVJNUZ2VGFVQUk2?= =?utf-8?B?TllzNHRRWllvTDc3OU9jSGFGZXJ5YVBaYkIzR2ZTZWlsMHRZTHVqRUFOemZw?= =?utf-8?B?WWJNWmlkNDhZWWY4S3VVa200NHNqdGNyM1ZqTFN2R21ObG9WdzBSTVBjVk4v?= =?utf-8?B?WU9rZWx2NmY5OHVRbFBtdmNjV0xWOEh1MDhJTkJFMHpzK2FNTU40TGg5Zlpt?= =?utf-8?B?elVudjNHcXBEVzFxYkI3UHc0b1l6TXMwUnNxbmg1amZSeHNRaEh2TDQ1WG1q?= =?utf-8?B?bEJjS2dWcWVuVzE1Z292VkJ3L2FnemQ1My9DUHhyTEIvY04ydW4wSW1YdmFZ?= =?utf-8?B?eEFPRnFsdjhJcUdNamZHeHlpQkZ5ZDFEUU5XZFBvdUZVSE1DRHA3U1JLNUMy?= =?utf-8?B?ZVJBMXdNMDV5enFxWDZNTWdOUE16R3Z0UWFYR2JFNk9aNHNmR3NlN3diSTFz?= =?utf-8?B?S1NnWUJ0VEdRSXIwNTNnZTg2WElpMUtRb0JvQmFSVkFsczM1LzhLaUloemhv?= =?utf-8?B?VHdKR3J5R2h2bjF5TGNtdVU4TlMzdFh1N05wSTJaVmxYdCtYVGJrcnphRzNl?= =?utf-8?B?Nk8zYXZKQ1Q2TWdSLzdmUlllcCtpb3hDV2pyaUx1bi9RUnFOZG5lWmx2U3U5?= =?utf-8?B?aTRLZ0tXTFArWDJqa2xmTU1ISmE3WElHRGVwQk9iaDVRNUtncTEzQzJ6ZGVK?= =?utf-8?B?U3FMY2RqVUoyeWlVMERsZnpnUDQ0T3AwU1p4ZWI5SFYyV0gzQ3V6ZUlrNGtU?= =?utf-8?B?dkhweHQ3MHMwZ3U5VEhNckVLcHVZaFNmMVdUK2RZMVBoUTFPQ3dTSTRmSGVk?= =?utf-8?B?NWY3WVdnZldjY1pTUEt0V3BUY2JuaXJDUkxwd3VaTzV3VFNOY1BZaUlndmFj?= =?utf-8?B?b1FJT2FFMnhTaTFCUXY3Yk12Q3VEaDM0bnlnMWlvSC83QWZodnJicnZSS1ZL?= =?utf-8?B?dEcrRFVVZ1g0WktIcmtmSnJ2VVFPdWhPRWFlSkZmSFpBK0Q5YmVjNERVSnN6?= =?utf-8?B?cHBiSDlaUXhDUUEzaVpvRFJqNjRBWG4xRjd5Ym9HUVpBUGM1RXZPUXdML2xj?= =?utf-8?B?OGQ2Q1h5VC9RRmJ4SVRZR2dNREdKSFNaVks0Qmh2VW8vR25UUjlvQzNZWVR0?= =?utf-8?B?UXpSd2w4WThya2o3NEpPelMzVVRpTlRJVVoyWW12NXpxSWMxL0RmQVdWN1hG?= =?utf-8?B?MU44ckUySkhRVitjVnZZc3Y1elp1bzBsVFhHZ2FFanVVYzBLOERVSlB1RWJs?= =?utf-8?Q?+FviROCdF3KEvjzHIXLQk1zKf0h4794xutANO/O?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2a47070f-f024-48d2-faa1-08d92cd2d510 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB3823.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2021 12:17:08.5106 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: S1ygf/2GN9EeCU4vRDtiK2bXtav0hNwlaZcyYDkI+h2j00Tq6JxqIFX2xFbQw87X X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3712 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --=_MailMate_FFF019EA-49DF-4392-9C9A-0F595FFA07C3_= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 11 Jun 2021, at 4:34, Mel Gorman wrote: > On Thu, Jun 10, 2021 at 07:40:47AM -0400, Zi Yan wrote: >>>> qemu-system-x86_64 -kernel ~/repos/linux-1gb-thp/arch/x86/boot/bzIma= ge \ >>>> -drive file=3D~/qemu-image/vm.qcow2,if=3Dvirtio \ >>>> -append "nokaslr root=3D/dev/vda1 rw console=3DttyS0 " \ >>>> -pidfile vm.pid \ >>>> -netdev user,id=3Dmynet0,hostfwd=3Dtcp::11022-:22 \ >>>> -device virtio-net-pci,netdev=3Dmynet0 \ >>>> -m 16g -smp 6 -cpu host -enable-kvm -nographic \ >>>> -machine hmat=3Don -object memory-backend-ram,size=3D8g,id=3Dm0 = \ >>>> -object memory-backend-ram,size=3D8g,id=3Dm1 \ >>>> -numa node,memdev=3Dm0,nodeid=3D0 -numa node,memdev=3Dm1,nodeid=3D= 1 >>>> >>>> The attached config has THP disabled. The VM cannot boot with THP en= abled, >>>> either. >>>> >>> >>> There is not a lot of information to go on here. Can you confirm that= a >>> revert of that specific patch from mmotm-2021-06-07-18-33 also boots?= It >>> sounds like your console log is empty, does anything useful appear if= >>> you add "earlyprintk=3Dserial,ttyS0,115200" to the kernel command lin= e? >> >> Sure. I can confirm that reverting the patch makes the VM boot. >> The important information I forgot to mention is that after I remove >> the NUMA setting in the QEMU, the VM can boot too. >> >> earlyprintk gave the error message (page out of zone boundary) when th= e VM could not boot: >> > > Can you test with the following patch please? > > --8<--- > mm/page_alloc: Allow high-order pages to be stored on the per-cpu lists= -fix > > Zi Ya reported the following problem s/Zi Ya/Zi Yan/ > > I am not able to boot my QEMU VM with v5.13-rc5-mmotm-2021-06-07-18-3= 3. > git bisect points to this patch. The VM got stuck at "Booting from RO= M" > > "This patch" is "mm/page_alloc: Allow high-order pages to be stored on > the per-cpu lists" and earlyprintk showed the following > > [ 0.161237] Memory: 16396772K/16776684K available (18452K kernel c= ode, 3336K rwdata, 8000K rodata, 1852K init, 1444K bss, 379656K reserved,= 0K cma-reserve) > [ 0.162451] page 0x100041 outside node 1 zone Normal [ 0x240000 - = 0x440000 ] > [ 0.163057] page:(____ptrval____) refcount:0 mapcount:0 mapping:00= 00000000000000 index:0x0 pfn:0x100041 > > The patch is allowing pages from different zones to exist on the PCP > lists which is not allowed. Review found two problems -- first, the > bulk allocator is not using the correct PCP lists. It happens to work > because it's order-0 only but it's wrong. The real problem is that the > boot pagesets can store free pages which is not allowed. > > Signed-off-by: Mel Gorman > --- > mm/page_alloc.c | 12 ++++++++++-- > 1 file changed, 10 insertions(+), 2 deletions(-) > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index d6d90f046c94..8472bae567f0 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -3625,7 +3625,15 @@ struct page *__rmqueue_pcplist(struct zone *zone= , unsigned int order, > int batch =3D READ_ONCE(pcp->batch); > int alloced; > > - batch =3D max(batch >> order, 2); > + /* > + * Scale batch relative to order if batch implies > + * free pages can be stored on the PCP. Batch can > + * be 1 for small zones or for boot pagesets which > + * should never store free pages as the pages may > + * belong to arbitrary zones. > + */ > + if (batch > 1) > + batch =3D max(batch >> order, 2); > alloced =3D rmqueue_bulk(zone, order, > batch, list, > migratetype, alloc_flags); > @@ -5265,7 +5273,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int p= referred_nid, > /* Attempt the batch allocation */ > local_lock_irqsave(&pagesets.lock, flags); > pcp =3D this_cpu_ptr(zone->per_cpu_pageset); > - pcp_list =3D &pcp->lists[ac.migratetype]; > + pcp_list =3D &pcp->lists[order_to_pindex(ac.migratetype, 0)]; > > while (nr_populated < nr_pages) { Yes. This patch solves the issue. Thanks. =E2=80=94 Best Regards, Yan, Zi --=_MailMate_FFF019EA-49DF-4392-9C9A-0F595FFA07C3_= Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJDBAEBCgAtFiEEh7yFAW3gwjwQ4C9anbJR82th+ooFAmDDVD4PHHppeUBudmlk aWEuY29tAAoJEJ2yUfNrYfqKfyUQAKKDevRmZx7KJDYvCjAwTnI2qKVd6C+MZHT/ RSk0r00jsLFlW1/ZrB0SJqKNmtC9P5Q26EUiuXOzkOaBi58qhjaDkIuP0gXK6J+F lG1K1rkvo7pRAZ4AsNg1tJnnxQgR+/qml8K6uCioota5uLnLOE/XHmUmg6KU2SZD PNOC/hmd9rJRKAnIy6RN3Kplf4Z+OaPYWDwYYdeuR4CTBbuM4styF2Ev5nuDDw+f MuF/Zhiy1U6E4DjA/rjacLrKQiIkKe58szOmgPcVQ3IUNADAZ2yk9LVBbNXx15+w kALAJEK7kk9/d5vKvbLp7jqOpZE+6DR6BHCjqbvE1kxfo4eNnfcSwAAW0KjNeDwn +C9WO/896MhVvTqX0CQGGPY+7rPLaqAdxpDB/wWFcm+9N8dh1r3ehfRnOX2Y1jiw HrhOo2URN3XdyBwhqCfj30JG2rFDqHY1Q5+UNDSNb/gxwlib/XWt3cMMMHeWALNS OjY/Ek6gpkCg8p2R6SSGFNCRk2WuPfN3WdtN2dnZ+FXPcwioutmTwIoOyZc2nppa i2nA+wfeMsJ7CyHbAIob/47Q5IOOa6PClBx7FkkeKXLKOc7KOOeWQvTSg7uRumRf nXKHXzR7zvO4MarOeLv0/1U37KVApg1Cbxci9dhxkzKT2Yb1sy7zl0MdDMTu+5Kx mggiJ+s+ =dYL2 -----END PGP SIGNATURE----- --=_MailMate_FFF019EA-49DF-4392-9C9A-0F595FFA07C3_=--