From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C496BC41535 for ; Fri, 22 Dec 2023 19:52:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A7618D000D; Fri, 22 Dec 2023 14:52:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 52F338D0001; Fri, 22 Dec 2023 14:52:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D04E8D000D; Fri, 22 Dec 2023 14:52:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 28BAE8D0001 for ; Fri, 22 Dec 2023 14:52:14 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 07E8580760 for ; Fri, 22 Dec 2023 19:52:14 +0000 (UTC) X-FDA: 81595500588.18.448605A Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf16.hostedemail.com (Postfix) with ESMTP id 3546D180011 for ; Fri, 22 Dec 2023 19:52:11 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=tdUfleNz; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703274732; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wF0SucCDjJdh7HZRNI84RgSHZWGk2eEUGYGdAuUxbgI=; b=b8Ai24FZX/9O3xzxcxbH+3j1UEsCLSU7s1TRqrlPnzLDsrrmT2qHs3vyVAd+vakiJXvFTk sbzrMJNx0/WC6Fel8OvdVZcYoSR/hB0qfAq94XKb8m/3hJs7xD5V1n758bzvhMusJ48iqb JcvPFAxUicr5AiLNm/Dj3Rmk4i9akgE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703274732; a=rsa-sha256; cv=none; b=C+L4QzIH7sjQ4W46R/x+hnu3bWS5RMrNMG4tZgtccs3DbkNN2peOJjjk8zj/39hb58qKOg 6fJLI1icDQAFyysor6x4a9hrfdS2uKEURqKDIpug2G/O3hlHbDqWPXIaf0fcI7/Z387ew3 td9bCBAR/WDxvQb4F1iAPtAzWzUZFSw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=tdUfleNz; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 33EFEB823C0; Fri, 22 Dec 2023 19:52:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 48A18C433CC; Fri, 22 Dec 2023 19:52:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1703274729; bh=9/goXFkgxBVSNEPhlQK8ZimkF0PGBTWUJgfmfSTrhhc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=tdUfleNzIZFzv+duf7Rv5BjnJJXD/ix8hHCOZDGsfcEXuMLvvm0JikroJIDhQUVOX bQ8+602rys/dcJVv4TpBVPr839CfHwNJHse7M4d5XfrQb06T9EbE4pc9DqVr3Pbv7I bNRx0DO6xGsj/pgzkzW3VXZdNFJ0X+FfuHBkK4tc= Date: Fri, 22 Dec 2023 11:52:08 -0800 From: Andrew Morton To: Chris Li Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Wei =?ISO-8859-1?Q?X?= =?UTF-8?Q?u=EF=BF=BC?= , Yu =?UTF-8?Q?Zhao=EF=BF=BC?= , Greg Thelen , Chun-Tse Shao , Suren =?UTF-8?Q?Baghdasaryan=EF=BF=BC?= , Yosry =?UTF-8?Q?Ahmed=EF=BF=BC?= , Brain Geffon , Minchan Kim , Michal Hocko , Mel Gorman , Huang Ying , Nhat Pham , Johannes Weiner , Kairui Song , Zhongkun He , Kemeng Shi , Barry Song Subject: Re: [PATCH] mm: swap: async free swap slot cache entries Message-Id: <20231222115208.ab4d2aeacdafa4158b14e532@linux-foundation.org> In-Reply-To: <20231221-async-free-v1-1-94b277992cb0@kernel.org> References: <20231221-async-free-v1-1-94b277992cb0@kernel.org> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 3546D180011 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 8tz34qrsad145mcfdfg8jxbbpn553u7b X-HE-Tag: 1703274731-192808 X-HE-Meta: U2FsdGVkX1+dIEm+JpoPCwgvLjhASISF7iOOYuCud7YcgMs6CIFQQGs5Z+BFAMKrIhbjJ2v6vxz7nvwuLUj+sJwmVOPWEP4AqZX5De2LzAmgXx4GZv7ZtEetLZK2bpG1wqELfK/oHHymxRhwOHml8r7OsM7l6WiIMLr1AU6iKS+yT2agaRkR8TzGRTws7CuH04eFdlgkEUvXqUCyYjQ1NmpP/NkAlJj51zLq/Wrmzu+F66uxN9lY98o6GYaMYHfmFbE/3WA2bFZhYb3sZKd9oQiy2AV3+KWETmNY+4RNrJFuqZHHN9jiYJCN9HXXcWW6w2oo6kiqVKrBz1sQxMekIq+zpLBlukbjBgmw92BCPctGkFL+G1Std/8+Y+JeacLd4WHV+1845dQW+Ds8G/mW53KHlMwwOkmOsDYIHUD+Iis2gSTmb6ZhPachtncef8pDfz0nYEzZtl0j1DzKJpFmdYB8BTX1000gg9JaCebQrh97eiyNBoC9nYmuafz9UJM2fiMPMiP5LPcSNS/l0y+/xcMLwdR/GFyKKmuUvRZa+mndrXtJw/YOq0/jYIeUquQ18xUelsSD8UwbWQaRDN9Cvkle09tDUXZuWX8/Yo98l1zl/UvjBOUgjCuEOzoGkapyv5EMSs/8hSnecd2H6zFku/w7jvfUaakpplt4EKThXbvictJUvKoBAH6pAicCJ8iv0tXQNnZSNvq4iGrZ2Qpm/44gC0dQ+J4fXHQWoMC1p48lDaNST/GABkMX8U0D0w7DZsdaCij7xVvnz5ZhBRQn+P+CyaSU3zGhVL7sS34ERp7itap8Vo1t+SmMLiwd02EJk1jmQcVYVrcFyOiK1KyCkYpjvNJjP98SCsHP7/FwwsQ0m25/be8+Dq0EdJ3Rhk8DAVbIVoBowEukMVZtAVkOF0dliA6dY3Nqd+nRua9s/GnuA2DJxl8Vwl4c4+u553LofM5YeIKcCrNin3fe04o HAeY1chU nSa2wHxQ5jL14M2LJJNFi9BzHSUUl7B6HcY7tll3enuROwxYI7y46KW6vzcgKg7KtVYMRO51PaqA4QVMo0tPLGOZkey/x33sPXsvO/uysX2fNhOG6MNXVzReXsIjRXYtQ/fS+Piqs3UgW4KqxjQbbEodOv94M3xxfXPE2+Kc95eyc8CPTORlsqFGVGlJh2gHsoMUfy8Cd9s6MLDe8KfwlHSuccg+L3UxGLCevEuyaTixO9rlkhKxWhUx/8qy16qEI2qp+6VzNiYCJrdBkZXbXB+i707bArV0m/sgllBGSLTClj5DW16qQoqMREg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 21 Dec 2023 22:25:39 -0800 Chris Li wrote: > We discovered that 1% swap page fault is 100us+ while 50% of > the swap fault is under 20us. > > Further investigation show that a large portion of the time > spent in the free_swap_slots() function for the long tail case. > > The percpu cache of swap slots is freed in a batch of 64 entries > inside free_swap_slots(). These cache entries are accumulated > from previous page faults, which may not be related to the current > process. > > Doing the batch free in the page fault handler causes longer > tail latencies and penalizes the current process. > > Move free_swap_slots() outside of the swapin page fault handler into an > async work queue to avoid such long tail latencies. This will require a larger amount of total work than the current scheme. So we're trading that off against better latency. Why is this a good tradeoff?