From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B739C54EED for ; Mon, 30 Jan 2023 12:53:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 238D16B007B; Mon, 30 Jan 2023 07:53:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E9206B007D; Mon, 30 Jan 2023 07:53:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B1446B007E; Mon, 30 Jan 2023 07:53:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F0A326B007B for ; Mon, 30 Jan 2023 07:53:55 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BF4881609EB for ; Mon, 30 Jan 2023 12:53:55 +0000 (UTC) X-FDA: 80411457630.20.B255C16 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by imf28.hostedemail.com (Postfix) with ESMTP id 8E755C0013 for ; Mon, 30 Jan 2023 12:53:53 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=i+DKUVyL; spf=pass (imf28.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675083233; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j0c9aamflhnbyccq2FhzqL4mmTM9eg1G9IkMY9G7LYg=; b=6BzclNTh9Dbk6DuXBzbJGQ9pzXESxVcIebVeGnpZmvF1T+T146mSk82HYAQUsSqsjF/IBT rUqfTIJeV5fGMRRHcYeiol22ajvnTvW+L3DfSNr2C7FhaKexEcP43c08aTgS8Y1mJfcHZ+ C+hVIzvJ4f37fq8muNL3k9klQ4jQ1xs= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=i+DKUVyL; spf=pass (imf28.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675083233; a=rsa-sha256; cv=none; b=PhQ+J7ZZT6YHI8GYwtMnSERxP1hPGANbTkuuOMAFAnPa8P3vxYFzDDRhsn644J5BzbA2nB NjfwOigRQkkalMlJUGpJr2H+IvrHzQu/o5qgaGgiHtOKUt79S2cKSxJ9g5exNLhDAwbnOD Fni9ZJvtvKQwZYkeUBOZQzdyS6NJhbI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675083233; x=1706619233; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BV7otbFxbnZltGO0oVLmlb9tBgsYOULHL9VhspoIUTo=; b=i+DKUVyLI1ynPqiPeXZRpbSzNyXYI8oN2w2/p92Gi75YjHZ1/taV2QI2 sV5WIlUS97/NwkCmKM8GD0VnKxYA0WuUqTw7N4JFNZNc40Sg5pYWFsihj xiuMcvZOCcAnA0LXsT6uJEJHsK6j5gTkbCXygn4rNeBp5rb0xtd29FVyo VA9Zv2RraCwINGC4EKEPpsXvAVWkdMUyh/cjOTkJiAssbGELLV5j+zYdo o8/7bnlYAZOHv5Qv2hFufOrHl5NvNv0MdAw4HQSWZ6+HOBfHA1ExgXEOn m+43FeEb9Ktve5gNbQGqW5E6J7+PZfTIxYoRY3d0YZ0yOb+0q/mrOHFBJ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10605"; a="328823824" X-IronPort-AV: E=Sophos;i="5.97,258,1669104000"; d="scan'208";a="328823824" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 04:53:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10605"; a="614015004" X-IronPort-AV: E=Sophos;i="5.97,258,1669104000"; d="scan'208";a="614015004" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by orsmga003.jf.intel.com with ESMTP; 30 Jan 2023 04:53:27 -0800 From: Yin Fengwei To: willy@infradead.org, david@redhat.com, linux-mm@kvack.org Cc: dave.hansen@intel.com, tim.c.chen@intel.com, ying.huang@intel.com, fengwei.yin@intel.com Subject: [RFC PATCH 5/5] filemap: batched update mm counter,rmap when map file folio Date: Mon, 30 Jan 2023 20:55:04 +0800 Message-Id: <20230130125504.2509710-6-fengwei.yin@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230130125504.2509710-1-fengwei.yin@intel.com> References: <20230130125504.2509710-1-fengwei.yin@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8E755C0013 X-Rspam-User: X-Stat-Signature: ewcihaikspi4bgsjd4i845sza53j7dog X-HE-Tag: 1675083233-686954 X-HE-Meta: U2FsdGVkX19Vb/4fCi8t4Fs2jYaz2iYcyn9HjyMc88dsQ3LdpfMwHAk34xVArNZ0tQWjwkQS81UVrhOBSGp5NfRSetHz0S7LGnN8M7zTcgzUv1jk/roweNw+WQcbz5SMSCFhTmgnILzT7JA4T9JKBrzceWLqqxD3XK8DslXzNrtgePtJwmeFZvhhMiVunKpht+Yut5kqi+gqw7nLt6OFwVU1Zm+QQJzcuOOZzemK/GK2GMd2G+ESIgeqlYdmmQI98DC6sdHnaY3pzVU5il4VigozzA4Ht17PaP9qWOCbFE7MUJ7q/sKcMIKU2HypqsW1f7nUMsvwu7Rf/aw96sJvFmtwp1eqS64Sz/Vf8GkeRuWKxmZPGDwzoSkIZb2h2mKdcZAcNK6YQKQYGsv7dLlM39qgmTM7BOVKia+mV/UNRXH8YVBgt4y+JV1tqOn5YxJDIxbDJJPOGPbULrIVPPTXUhgTOoSDmZWFDnOCjorT8Lb3IuF78OTEWLN7vvMp/OllfQrhKhsnvZ7ZoPQou2OvOKgYr1ycKX89Ez1yyyNHxTuGQgkYm9y0s2P03Xy3xefMnCnwTulWcWUZdEUtnul2pJMoHWZH8zJZR+hL2uj5Jo/so2wg79Gi1HiYSjtSa7yc5ACaV0MsKhb9w81KmFbmrKodwSzWJ04eAxgmueC0iCNJQNp0YYQUOtAFTeAvApOJj5m13sfsMJGYMdIHCyurKsxSceYDaaQU8lrnqtrGaKwmkIdB38jJjI8TBwFphiD6RYpTZ/O0rMyxmxC1orcfE296jVmtd5C7cvYvCME/jXVc11DXse+vbM09IOlFSmpENId4u1VkRne195uPdGqMU2q3GFiMODhYW2+7HqL4GWLa1tKzAUXhEML1ejoZPiIBWBXK7gK9WMSxOUdifa1EIylZ4UFxp4b4HA+ntCLQOdPBR/dyu60BaYwSghj0WHMDs5kaDWjFp6PsiWcLiia x/qGNRW6 QvTmaMtJnta8tOpT2a//SvCc8w/24d26lwufmXxbPql1esGTibKtj8ok9RzAx59Q+pvmj3KDLpm9g+eemmtuNBFbcR0qjBGtaX+R1Rmj+MqcZ0czDHtBmZRLBlXGI/cIdEnay8wREIP/g4RPUnMnm+NdqTH4pXhnAj8CVdrvVsprqWILgOAJ2hGK+rWZvRF9iKD+l95qlfKQk/D4KTrY/Hx6M7g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instead of update mm counter, rmap one by one, batched update brings some level performance gain. Signed-off-by: Yin Fengwei --- mm/filemap.c | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index fe0c226c8b1e..6d9438490025 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3360,28 +3360,52 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, struct vm_area_struct *vma = vmf->vma; struct file *file = vma->vm_file; unsigned int mmap_miss = READ_ONCE(file->f_ra.mmap_miss); - int ref_count = 0, count = 0; + int ref_count = 0, count = 0, maplen = 0; + struct page *pg = page; do { - if (PageHWPoison(page)) + if (PageHWPoison(page)) { + if (maplen) { + page_add_file_rmap_range(folio, pg, maplen, + vma, false); + add_mm_counter(vma->vm_mm, + mm_counter_file(pg), maplen); + } + pg = page + 1; + maplen = 0; continue; + } if (mmap_miss > 0) mmap_miss--; - if (!pte_none(*vmf->pte)) + if (!pte_none(*vmf->pte)) { + if (maplen) { + page_add_file_rmap_range(folio, pg, maplen, + vma, false); + add_mm_counter(vma->vm_mm, + mm_counter_file(pg), maplen); + } + pg = page + 1; + maplen = 0; continue; + } if (vmf->address == addr) ret = VM_FAULT_NOPAGE; ref_count++; + maplen++; - do_set_pte(vmf, page, addr); + do_set_pte_entry(vmf, page, addr); update_mmu_cache(vma, addr, vmf->pte); } while (vmf->pte++, page++, addr += PAGE_SIZE, ++count < len); + if (maplen) { + page_add_file_rmap_range(folio, pg, maplen, vma, false); + add_mm_counter(vma->vm_mm, mm_counter_file(pg), maplen); + } folio_ref_add(folio, ref_count); WRITE_ONCE(file->f_ra.mmap_miss, mmap_miss); -- 2.30.2