From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E4CEC3DA6E for ; Thu, 28 Dec 2023 15:34:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD1F48D0017; Thu, 28 Dec 2023 10:34:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A826B8D0012; Thu, 28 Dec 2023 10:34:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94A918D0017; Thu, 28 Dec 2023 10:34:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 826448D0012 for ; Thu, 28 Dec 2023 10:34:15 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4F7AE120897 for ; Thu, 28 Dec 2023 15:34:15 +0000 (UTC) X-FDA: 81616623270.01.2ADBE11 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by imf16.hostedemail.com (Postfix) with ESMTP id A7292180010 for ; Thu, 28 Dec 2023 15:34:12 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mKRP1x48; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703777652; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=75j1Tuw2sMYHrBdwhMsY/vjsJuT8QMsKkqQlgA1wbFg=; b=Tlslym77eirAy6cVE6rVI+Hj2IcyHfVxn54G5vXZHgxj/5kVEuBBvESfapM3KXgKz+Onm3 3Y+BXqTxl8YcqOfWZOFUgWVts8AS97VSe2JR5DfRmkV/xnlC7t68isrwLU90TvVW9mKljg +KouUAfZc8DXuR6grWSy77LKgydAmLE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mKRP1x48; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703777652; a=rsa-sha256; cv=none; b=EzGFJIpVy4Qh1S0LiV6u0Q01fAX71B9Cj3Yxj30NPL4dC9J+/qZ/zqfHsMSqF9vjChrZMf 949gBeeJZTbkAUVogMCa3BcssWhCUHpaEyIqRp5ognH4vSx4ZNZfKF5O2GkyXhSUtf2M4P qQ6/Wpqjkqo+p8S4icvGdo1Ro12KTyA= Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-555526a060aso1998017a12.1 for ; Thu, 28 Dec 2023 07:34:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1703777651; x=1704382451; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=75j1Tuw2sMYHrBdwhMsY/vjsJuT8QMsKkqQlgA1wbFg=; b=mKRP1x486lcyTaFpTWJE1/eWBjirREta9SAGXS+JzMqYMWZIh37AnApNxoAOhXiTH0 4Msro5agzuRaiNqfa1ckx2K7tk9pipZmE2DRCeFXfsZ5nI/zmDnADMxQ/DoE2/OV4c7y IZjibRa6Vc0lHITsiFbd0WEaB4O5DhGgcFuycXmf+yGYoUy83fwDFV8l4Qv9lxYv94DY rgHWytVPWFl6fhk6tPY4wIhVTk2MBESgwmIb04pnC3uaF2GwUz99EJ4SU2WkseaX/Lw9 Y4MFM15Gtmfd3AtCYP4K6rAspEUFZ5TkQOUkcN5hBD7psxZNYDMANoS6x9Bu6JQdGcTb WcGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703777651; x=1704382451; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=75j1Tuw2sMYHrBdwhMsY/vjsJuT8QMsKkqQlgA1wbFg=; b=SKohlsjHcUQfOannLCM0jCq2dNZR6TQ54SB8f6f/vugl+jaP0I5SncqSbdPgxZnVBc Qc5LlQULDnNfBoxX3uS8E3yLwndqPvv6RF72kdDuMcW+9pNbzoecC6ZXXNUkAJK4dhpM 83WyhXPw2Gf/6qP0t6bP3fSTdsEBqu+sXHEsYRBSwemjYH88nwffhPRuiSETuUuuDveB byUqQ5y+/Dkia+2BIuCdFnUGOlInpq5FYPrO1saH7BmuMKDEd0sTTMh8RC7PnH2crnYh QxpLs43itt2EYexHZPMcP3ClesRLP2XvROG1Xx/voC7Ofybvcj3NeZfSJK6oplJjKTkX RX+Q== X-Gm-Message-State: AOJu0YwEy3Kmaaekq6+hWtLZi8FFtCqwPg15IZikuH9fOLlK2ylgplTd /7JLmhD9co/6wcNO29FgLvvdQ3jQ+k8gedzgF00W+9Mf3BvG X-Google-Smtp-Source: AGHT+IEVdgeTgEcIj//cXTspyGF7Se6NK2vltYXoZjZd9YUDK9xPQVKpC7XTmsxWy7ooa1IF1ltop0GyQm2+l6XbOMA= X-Received: by 2002:a17:906:fb01:b0:a23:6143:61b5 with SMTP id lz1-20020a170906fb0100b00a23614361b5mr6356178ejb.129.1703777650946; Thu, 28 Dec 2023 07:34:10 -0800 (PST) MIME-Version: 1.0 References: <20231221-async-free-v1-1-94b277992cb0@kernel.org> In-Reply-To: <20231221-async-free-v1-1-94b277992cb0@kernel.org> From: Yosry Ahmed Date: Thu, 28 Dec 2023 07:33:34 -0800 Message-ID: Subject: Re: [PATCH] mm: swap: async free swap slot cache entries To: Chris Li Cc: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, =?UTF-8?B?V2VpIFh177+8?= , =?UTF-8?B?WXUgWmhhb++/vA==?= , Greg Thelen , Chun-Tse Shao , =?UTF-8?Q?Suren_Baghdasaryan=EF=BF=BC?= , Brain Geffon , Minchan Kim , Michal Hocko , Mel Gorman , Huang Ying , Nhat Pham , Johannes Weiner , Kairui Song , Zhongkun He , Kemeng Shi , Barry Song Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A7292180010 X-Stat-Signature: 7w8r3hw8dinkp7uotnjzoz5k9sfwinnc X-HE-Tag: 1703777652-935695 X-HE-Meta: U2FsdGVkX1+u8c5OLmeGEG5fYCYByRMyqJPSaUoeFdpnMVEv/GrjrUZbIVGTYsOaFj4xsJ0H1ACpNXaKWkEPZCAt7Me4XXRL1CawW7hwDnr08K+qH1HSER/0uz82ofzul/e/9cCEb1uKzKwCKeb41pLLaTcjj/ynIJRXaNcyhkOkR4smorIdNmGb02ekCHZhTTLbX6ixqagPW5BGq4Nt2xr34kpvj0Br1WfTabVUlVkVHjGxgksTnoejReEjrbkYMBtuji3+dJNIobXGdOv0jYwPB+K7SBxJ5i3/DUnJydW9q5+4+0++wJC6zgXJYA5oLWZgubV8dILWz6hjyeuTgHQpnIFObpcXpzIvFNSocO3fWxjRQoFchWHE2pbcqTDqJ7icWbs8PdHCjpCOk9Z3Ty9zRN11OASuV1Hn9WPoxbfdJLLzZ35t8ckvgfkIQ5tgTm0uNHCIXHPeOk2yK8oYodmtdzoBbLJziTITLLZaKdb78P09k7SeAVMOZxlGJPbWeTCQYjaV77UpeeVyw3WKz/0UpBR7X2nvMKBrmEHuhn+Cow081xk6OzwUtGDhg6dbQC+4IzB/VqPYseF7bdiTIpjQzQV96l+XCZV63CQIcqIR7ZWqL4LDbW0eEd2nIbLopiyKgvS4Pyny8YoO5HdPCM4N676Cc1vj3tMUBLMWlXSt68bPViY7oT3r/RDljmvxexdRrfh5v/zpJC+TVx3SfyZCj4AwKErbRhwyilwFXRt0ZaVIUFm9uIMha3Ckq5U73kHNgS44nRR9agivqgtwf/eai5bRlqrKd+lpShZUqHz/Vkz0dL/zWN3mn9YDY1tAABUd197ZgA8xQaNcokutswsgYYXlQ0CgSlS1VMgHnGfYzI6Vd8pXo+vfutGTakFWCQt+wG5urUjZWinMNwtN7hcC77m5ODtTTrOP8PGbYy51iPysXeCjKaCGH+VUoDlTq1QFd2n4SKSTfWfO+b7 QPQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Dec 21, 2023 at 10:25=E2=80=AFPM Chris Li wrote= : > > We discovered that 1% swap page fault is 100us+ while 50% of > the swap fault is under 20us. > > Further investigation show that a large portion of the time > spent in the free_swap_slots() function for the long tail case. > > The percpu cache of swap slots is freed in a batch of 64 entries > inside free_swap_slots(). These cache entries are accumulated > from previous page faults, which may not be related to the current > process. > > Doing the batch free in the page fault handler causes longer > tail latencies and penalizes the current process. > > Move free_swap_slots() outside of the swapin page fault handler into an > async work queue to avoid such long tail latencies. > > Testing: > > Chun-Tse did some benchmark in chromebook, showing that > zram_wait_metrics improve about 15% with 80% and 95% confidence. > > I recently ran some experiments on about 1000 Google production > machines. It shows swapin latency drops in the long tail > 100us - 500us bucket dramatically. > > platform (100-500us) (0-100us) > A 1.12% -> 0.36% 98.47% -> 99.22% > B 0.65% -> 0.15% 98.96% -> 99.46% > C 0.61% -> 0.23% 98.96% -> 99.38% I recall you mentioning that mem_cgroup_uncharge_swap() is the most expensive part of the batched freeing. If that's the case, I am curious what happens if we move that call outside of the batching (i.e. once the swap entry is no longer used and will be returned to the cache). This should amortize the cost of memcg uncharging and reduce the tail fault latency without extra work. Arguably, it could increase the average fault latency, but not necessarily in a significant way. Ying pointed out something similar if I understand correctly (and other operations that can be moved). Also, if we choose to follow this route, I think there we should flush the async worker in drain_slots_cache_cpu(), right?