From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE6D2C28D13 for ; Mon, 22 Aug 2022 15:06:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D9C76B008C; Mon, 22 Aug 2022 11:06:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 95E076B0092; Mon, 22 Aug 2022 11:06:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 789356B0093; Mon, 22 Aug 2022 11:06:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5361E6B0092 for ; Mon, 22 Aug 2022 11:06:41 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2F5361C1865 for ; Mon, 22 Aug 2022 15:06:41 +0000 (UTC) X-FDA: 79827555402.10.A67DA20 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf06.hostedemail.com (Postfix) with ESMTP id B3796180078 for ; Mon, 22 Aug 2022 15:06:40 +0000 (UTC) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27MEirI4017918; Mon, 22 Aug 2022 15:06:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2022-7-12; bh=81hLR2p41Wme7z5ElJv/3Isbxe8k0CndgU0ZZQSJKT4=; b=MRe2ww0n1l9ySQlcMCoCVHQUFzBrBcgaZ/CQHNQ78GnVNFnasivI0bzgA3NydoFese1t 0uiX/zM9NhO7h9dPK/2TvedxVMzfSAtw8lL7aRYqQJEuZ2KkWlaS5LFx6QZe5g7yphNF xQROQjChgv8RfpggZDrUcw/g8uCEfWNOWjLbjVYVPXpP7833q/PvUNVWFP0BW5vGxwYd SCFOu/qQctjOTayRt3wkMU7q8lVJaKzMHRTLMpTTeXMZ6YFL4OnWNMMBGRlYfLKqzWnc NFycY+C6akQVaECvj61UFGQSuafGfOnz1Cuai0oi1AFFvjPcgzPc7oamipLJ0BkiRHF5 3A== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3j4bkt049q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 22 Aug 2022 15:06:33 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 27ME4Nag031868; Mon, 22 Aug 2022 15:06:32 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2168.outbound.protection.outlook.com [104.47.55.168]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3j3mjd3986-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 22 Aug 2022 15:06:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bkQ2moZpeEFoSPZC73Z5wqyJPVfEVj2Feah3448FCQTln7f73509qTGz5Kqk6lP7dHXbx2RWIXxTemI/F7pUDbR0xYglaHbbHw5/L3zFY0eJoXBQ2gMqEUTvDG4ODU8YA9RsPZVllZ4voOeQZa5jt5H7dbeNwD/F/Ihy1Vw/Cr5ztkC3wSlSTzREZrZBb3MZpoZwjE0HFVECDgwusQMrMvI1FIPT7cxD+Us6pzWCo83S7kpqadfUpQYYBv5VWvSyI67XK6/ZbSGfpP/YPYCW3rfs7u09wwzmhZJmcwp0Z6gQYxpMqw40m2i7ynhd3wB3nULktYU4Dh2l5fXQ01ef9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=81hLR2p41Wme7z5ElJv/3Isbxe8k0CndgU0ZZQSJKT4=; b=FMglnfDH1pybDPpChqRVZ/bCqCP5WD8XKK9kjwdoPC893wO/V5kdZBwmiAOUFkGqXNAO12shkk/wTCq1jh4z46tIh4QX3LRt/rzXS47q2BUxjTK8VqFDrjksahiK7kLCdQVLCVomWU/daz5cPWdLwIz9hIbt0foIBp6/8OAyApayLbUWe7Mlz0l+fT1smHxHflecrMpjxXkzBo4qclkFN4ZtpbqKbIs4dfAI3MzxWKh/NhsF3eigQotZP41BBTaq7VSUDKiZxS7qcET/1zPyXan6rglDRpEDjKwlXkFzMaR1CM/37wWrZSw9SAQQIxVe7UP/aDKZjd0sl3ZYmwsjDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=81hLR2p41Wme7z5ElJv/3Isbxe8k0CndgU0ZZQSJKT4=; b=GfQ11S3AvMD1xDuTQ/W7EedppyofhchnT16WCSF51G6rFDAeQg+H7je5FSFnnnzxBSs1ryTgD2ORxMH617JvijCcdLO14IelMVoqTh0uGIYquaRuCRqT6U4GUyuADAsGVxyMovQBL1WtANWRpMgi65rFcwmmsszLJnpflX2iRYg= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by DM5PR10MB1433.namprd10.prod.outlook.com (2603:10b6:3:e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5546.21; Mon, 22 Aug 2022 15:06:30 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::3d34:ebb5:d9df:98b3]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::3d34:ebb5:d9df:98b3%5]) with mapi id 15.20.5546.023; Mon, 22 Aug 2022 15:06:30 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton CC: "Matthew Wilcox (Oracle)" , Liam Howlett , Vlastimil Babka Subject: [PATCH v13 48/70] sched: use maple tree iterator to walk VMAs Thread-Topic: [PATCH v13 48/70] sched: use maple tree iterator to walk VMAs Thread-Index: AQHYtjjACsPOJqyxnUW2LCMZZAy3mQ== Date: Mon, 22 Aug 2022 15:06:27 +0000 Message-ID: <20220822150128.1562046-49-Liam.Howlett@oracle.com> References: <20220822150128.1562046-1-Liam.Howlett@oracle.com> In-Reply-To: <20220822150128.1562046-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.35.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b672c62d-e798-44a7-9bb3-08da844fe4bb x-ms-traffictypediagnostic: DM5PR10MB1433:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: GVYfe6/Fya82G+OQTFpVSVF7swJ+XrVr22cZ8sRztWAzO4wa9D9fm8DR250J7G/m7o6sGv4ILeb1MKdsU/nNDxqyen3d+RNNgfEALU15z38WS6/f0JvX8ShKzdnhpxHSb4NhmbPVHAizYbholwARbdsonjvmD9IKipUcZZs+WW5d43jz3DeradEyPhux1CD+S8TloSS7YNbO/lKdcGff/8ySMTtk/6B0qa/8p+kBM1+FlJykTs93o48CNtc3TUjyBdd3VUpFvwh4KBj7+2QhC6+Vtnqf07OeMgmHxkbDO8o5FybVjOKYdIbHxGFUzXdV8M1gLAH/r/hGcqD3YSyWHysMksZye9UFzSqJWeTR3T0wtARXCUY5rjU4crdGarhLBXqMqNkqzVpOOpbUrQK1NIpigCopPvVXQm1GUqclJbB/4vN53PRPqUnbAd4wQD6kAJS8kBcFK5giSZ/3AVIOfb+wfx7OPeIw9bpc1UvgI/0Kx7+0Lnuv7Rxl/46Uyy5+/YolVNxbvO0B7res4qqubVL1TMy9UiQ4BtVQlCa5yFk+pJmk4YQ5xWf7Ef8twIB78lQn1HqbbvxbRmjmgTc2PTAvBb45E7IHgtcjkKUcGapsAKuNdTmlJlvUPAxnCYN2Qubaj2DOqxhQMfOdUnm4caLlRoyxm1Mt7SgXb686eiaKU05Gbomh0hfxHfnXxKyMALmELeHI0TPa32q54QT87ZAvIhENlskQ6eHvil0FiXiBmqm7Lxv2ZhYpLoFxLwcv x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(346002)(376002)(396003)(136003)(39860400002)(366004)(5660300002)(8936002)(38100700002)(122000001)(36756003)(41300700001)(44832011)(6486002)(6666004)(316002)(71200400001)(76116006)(91956017)(110136005)(8676002)(4326008)(64756008)(66446008)(66476007)(66556008)(66946007)(54906003)(478600001)(186003)(1076003)(6512007)(26005)(83380400001)(2616005)(6506007)(2906002)(38070700005)(86362001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?nSIJz5sXnt6AKIgOlpCv7bs4PqT+Et9NbcoRJy5FNqw/Dxv/SEzzzdm7Cu?= =?iso-8859-1?Q?XrBYuNY4tfpuG2zFqiojiUSXNIkOV/qODd7wJpYy303qnICDA1PWmnHtOO?= =?iso-8859-1?Q?fW1etYUuerHItA1LsnH9+bYJrrPEm7T0fkUZVKTgFIfrlBGMcaJBlsc3R9?= =?iso-8859-1?Q?lKvIdtD2mdn6iv7M5FQQ8afluzwWIsTk3sAIEjsPMhj+4skiMjSXf/+WyE?= =?iso-8859-1?Q?Cl0sb+1AW/lignNfFgeBP7AFhXGm0E6BlwRDYuB5aGoR/pitBUAh87RFOf?= =?iso-8859-1?Q?Sg/0VNW7blXNDjeJr3Q2TXPEtEan6VyyzakNVP1+1hs/iAbuRb4FloGMY/?= =?iso-8859-1?Q?cZ15si/T/wKjhUCMNrE30ZmwjrWWUAgJi4VY1MJnv1VnhDB/9qt2mQUAos?= =?iso-8859-1?Q?xHYeT/E/ogrGy/4eIRNwt46J9pYSNgLAWcIcA6tupQI+rwEeGtzktboV9V?= =?iso-8859-1?Q?NXQzJOKbJ11i7J6JiAv0IKJ18CizqqqQ/JEZbH4cgcFzJadgxNgyoZggc5?= =?iso-8859-1?Q?33IiPAYtBfPntQI6SGc1Cyt+ekj4K+gws1VvNgLFgy58y94DpNmmO+jFAn?= =?iso-8859-1?Q?02IyUHiURh0L5qrCnp6RDclqwrX64gkLrx+CiZLD3s/Hr2EXFT2bNBfXRD?= =?iso-8859-1?Q?ipsA/wpg1tU5SOngozcOEVpHx87j5cxEqxUhQ0fJp+uQifR4v+8nSHJrcv?= =?iso-8859-1?Q?BiFhOxNAG2TVxG+NjLvXpAq6D6SYv7doEZBXcgK70m1nfAR0oZfELGjKvJ?= =?iso-8859-1?Q?bNX7NcgEEBzQsoIR7t4qFPMLVAuSNkxChRE3Zo/3NTPjjx/eQtEPSfT3ta?= =?iso-8859-1?Q?ghXWWrnNAcpT1zMlTemZko2QD8R/21f00iIrkjTPEfhcQ0UVnmOS2zhzeI?= =?iso-8859-1?Q?tkz0l8FEI1nSEsiBM1vLN8GMfliIfYmGWHDgghVtPBBwaWpdfh1aHYsoJr?= =?iso-8859-1?Q?vh+croHByAVu2oqjuMUM1E0+1AuAVzfUvXI7kd4ioiOjiG0jIZwSmksr5H?= =?iso-8859-1?Q?t1+cPjO3QskM/Q5qJ6jyTTwMnXAHsY3l5kdsNDWzuHci/t8pdW7YbGmLgY?= =?iso-8859-1?Q?9M8+1L9ClJqG2lj9PmutCejk3bxgfeDaeFZ3LPwAJldfp5KwiILWD6PUbD?= =?iso-8859-1?Q?PDThIR2iQf/FOCOo8IlNVB1SIrjqBwJHA/pryEpKZ5q4FsoxwzYTd1QST7?= =?iso-8859-1?Q?sIkWuTRqSwkHmgSDUZSJrT0/o0mbrupzU3NLxWfz1OzIyCqSuJayp5oMTH?= =?iso-8859-1?Q?atgyQQP3P0pZKy6mmKE7m0OIvChjT1gNMSvNSg2Z9K43goJbNZYyoD9AAz?= =?iso-8859-1?Q?I2uyo2eneXWxVPPjO696qpt44PzOsgtDUF4n7AmWTBG8mbwlMqmaKZU9aj?= =?iso-8859-1?Q?X/nz2pLuoAQN6wDcHtsUgong4gjZIIqGkNEdy2wacPXHer/zmFT0UzdI07?= =?iso-8859-1?Q?CBLJp5yJhhY/w0ZDqSJC5Vm9cIyfC7a6VZ2UGXNiulMo647DWSvaHV5tr2?= =?iso-8859-1?Q?8aLTN4UrkBCS8I/Kw+ww8R2yNJL9JrA4lh2YV0HchGRjjSzn3fCCi0REOB?= =?iso-8859-1?Q?QH4F3xGqZW866O/XGthH9EWRhWLMFQV7Z+fFcgqidBgM8m8ivKc4qZB597?= =?iso-8859-1?Q?HTZYmg5bpFdod0gdjdtN5rtIq1wSCbyS6nIhbQ8s3IAvMEKvgcA/AZBw?= =?iso-8859-1?Q?=3D=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b672c62d-e798-44a7-9bb3-08da844fe4bb X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Aug 2022 15:06:27.6287 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BXjOwjWx9kQGQNojrNBEkzIkBx/5p22Ab8dkfjViubU9AJt8s/0zqyUX3z9oPqIclRGgQV7Iyi9sJOtpgmWtyw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR10MB1433 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-22_09,2022-08-22_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208220065 X-Proofpoint-ORIG-GUID: -tIZ9C3vTrClyC6BmhJ12BWCFX_EZaRq X-Proofpoint-GUID: -tIZ9C3vTrClyC6BmhJ12BWCFX_EZaRq ARC-Authentication-Results: i=2; imf06.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=MRe2ww0n; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=GfQ11S3A; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf06.hostedemail.com: domain of liam.howlett@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=liam.howlett@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1661180800; a=rsa-sha256; cv=pass; b=I55+OybNFEBaMWgeAkKTAac+bR0ag5BK7VHxHC/dcaHC2FJWlnokkX3eCSEsKwHGMBuMLW KtYfrtRWJ92QNLeArylwhCMiL6tiggcsvFDUa2PDBOskvjfvW0o1JdZnPgiDYidvTpJISJ RXvDmlIt3pSsjCrR+0MT4+1xVH9gTxg= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661180800; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=81hLR2p41Wme7z5ElJv/3Isbxe8k0CndgU0ZZQSJKT4=; b=L0dM3k7H3uf1WezpNy4Q5Y6peXRUydGMrInfPoyAj6VaiNjg+cC+CCj+j1L1c3Qh3grD2r LNwUSqTqpC5Ade3aoP/BDGKRr35fAEwboj56ofaACz7SVf3aOzNzm0sinhBGTBNQQTn7bx wCOlSZVTRU/bCpGcPmv9lQ9fk0sUfpY= X-Rspam-User: Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=MRe2ww0n; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=GfQ11S3A; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf06.hostedemail.com: domain of liam.howlett@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=liam.howlett@oracle.com X-Stat-Signature: xgx7cga64p94bhp6xyigpt7bjraqei5w X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B3796180078 X-HE-Tag: 1661180800-172764 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" The linked list is slower than walking the VMAs using the maple tree. We can't use the VMA iterator here because it doesn't support moving to an earlier position. Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Liam R. Howlett Acked-by: Vlastimil Babka --- kernel/sched/fair.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d642e9ff2829..c0c0e691d426 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2929,6 +2929,7 @@ static void task_numa_work(struct callback_head *work= ) struct task_struct *p =3D current; struct mm_struct *mm =3D p->mm; u64 runtime =3D p->se.sum_exec_runtime; + MA_STATE(mas, &mm->mm_mt, 0, 0); struct vm_area_struct *vma; unsigned long start, end; unsigned long nr_pte_updates =3D 0; @@ -2985,13 +2986,16 @@ static void task_numa_work(struct callback_head *wo= rk) =20 if (!mmap_read_trylock(mm)) return; - vma =3D find_vma(mm, start); + mas_set(&mas, start); + vma =3D mas_find(&mas, ULONG_MAX); if (!vma) { reset_ptenuma_scan(p); start =3D 0; - vma =3D mm->mmap; + mas_set(&mas, start); + vma =3D mas_find(&mas, ULONG_MAX); } - for (; vma; vma =3D vma->vm_next) { + + for (; vma; vma =3D mas_find(&mas, ULONG_MAX)) { if (!vma_migratable(vma) || !vma_policy_mof(vma) || is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) { continue; --=20 2.35.1