From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BACD7C00140 for ; Mon, 8 Aug 2022 19:35:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EBD8B94001B; Mon, 8 Aug 2022 15:35:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E45F8940018; Mon, 8 Aug 2022 15:35:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B85A094001B; Mon, 8 Aug 2022 15:35:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 83741940018 for ; Mon, 8 Aug 2022 15:35:33 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 6410F1A0AB5 for ; Mon, 8 Aug 2022 19:35:33 +0000 (UTC) X-FDA: 79777429746.10.A334C11 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP id F3F3B180145 for ; Mon, 8 Aug 2022 19:35:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=RwARIESPVJRvPjdBwRi4bFYQLWzQ2fXgslMpG4V504o=; b=NMM0eNj9D+5PD6yqkSBOcBx89h csMALxGWup2UgYZOBZteKiv3rXf9f2mDYglGaWHwIp0Qi80rJPifGCZjZSQhNNg9Llt+emlTidCRU wRIoePzg2ykG2QuW/SaPiFLRWrrxMX2SX7wAeL1i7lO2UFv2PtpieK7lwu7amn5d91oCr9KHa5XEb HpA7cc/UF/IJsTejJuMf5jnhXNzvWM5C7KSXyYSxJl+wqjAP4qA2+I+XPiU7dg69IiF2xECwJHRp2 xbop1QvyNEC2hKlEQpv33D6+PwEnDBzHx7/9nCudUxen51g43HgKwwDdxxzE1iHpnCkknCbqRZa5A 39DMHpfw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oL8XD-00EAzF-8a; Mon, 08 Aug 2022 19:35:31 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hughd@google.com Subject: [PATCH 39/59] swapfile: Convert unuse_pte_range() to use a folio Date: Mon, 8 Aug 2022 20:34:07 +0100 Message-Id: <20220808193430.3378317-40-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220808193430.3378317-1-willy@infradead.org> References: <20220808193430.3378317-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659987333; a=rsa-sha256; cv=none; b=aFEXB9lrgza6FqDr6OwqAiYcubvGGdW0zxA8Urs0oCCc/JVk3NcwHrISEK0Bz8TzvgpAlg 0v6IOMS/v64PSxCvxAd3QhQ8F7REqfFDdmmYoyT9dHzauRr9dLt5n4z7vioRLLMk1ksl5i qFkvVlnvZgmbWIOQXT48+wk51Zr4P0g= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NMM0eNj9; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659987333; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RwARIESPVJRvPjdBwRi4bFYQLWzQ2fXgslMpG4V504o=; b=55MZD02n7GQlf3neIzrOMs64I0tLInC8M4etkWtXje/YMjYXctvhJAFfL0+/BRC7i/b2pE atJUAHhvp97/e3K9GWVsUHsvr2nnC+STt67kb6dkmVZqeVX3uwIp3eaZS7rgm7tAmsYxIZ /oHzHTVXrfTIP7GUSr/GO/EArul5GtM= X-Rspamd-Server: rspam10 X-Stat-Signature: 1ema1cprohnqyja7eigrcujd9bogudbk Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NMM0eNj9; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspam-User: X-Rspamd-Queue-Id: F3F3B180145 X-HE-Tag: 1659987332-636235 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Delay fetching the precise page from the folio until we're in unuse_pte(). Saves many calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/swapfile.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index ce538c3f9161..9ee42a12cffc 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1754,8 +1754,9 @@ static inline int pte_same_as_swp(pte_t pte, pte_t swp_pte) * force COW, vm_page_prot omits write permission from any private vma. */ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd, - unsigned long addr, swp_entry_t entry, struct page *page) + unsigned long addr, swp_entry_t entry, struct folio *folio) { + struct page *page = folio_file_page(folio, swp_offset(entry)); struct page *swapcache; spinlock_t *ptl; pte_t *pte, new_pte; @@ -1827,17 +1828,18 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, unsigned int type) { - struct page *page; swp_entry_t entry; pte_t *pte; struct swap_info_struct *si; - unsigned long offset; int ret = 0; volatile unsigned char *swap_map; si = swap_info[type]; pte = pte_offset_map(pmd, addr); do { + struct folio *folio; + unsigned long offset; + if (!is_swap_pte(*pte)) continue; @@ -1848,8 +1850,9 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, offset = swp_offset(entry); pte_unmap(pte); swap_map = &si->swap_map[offset]; - page = lookup_swap_cache(entry, vma, addr); - if (!page) { + folio = swap_cache_get_folio(entry, vma, addr); + if (!folio) { + struct page *page; struct vm_fault vmf = { .vma = vma, .address = addr, @@ -1859,25 +1862,27 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, &vmf); + if (page) + folio = page_folio(page); } - if (!page) { + if (!folio) { if (*swap_map == 0 || *swap_map == SWAP_MAP_BAD) goto try_next; return -ENOMEM; } - lock_page(page); - wait_on_page_writeback(page); - ret = unuse_pte(vma, pmd, addr, entry, page); + folio_lock(folio); + folio_wait_writeback(folio); + ret = unuse_pte(vma, pmd, addr, entry, folio); if (ret < 0) { - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); goto out; } - try_to_free_swap(page); - unlock_page(page); - put_page(page); + folio_free_swap(folio); + folio_unlock(folio); + folio_put(folio); try_next: pte = pte_offset_map(pmd, addr); } while (pte++, addr += PAGE_SIZE, addr != end); -- 2.35.1