From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C33BC4320A for ; Tue, 27 Jul 2021 01:01:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8327360F93 for ; Tue, 27 Jul 2021 01:01:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8327360F93 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=anarazel.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 0F5116B0036; Mon, 26 Jul 2021 21:01:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A5566B005D; Mon, 26 Jul 2021 21:01:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAE818D0001; Mon, 26 Jul 2021 21:01:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0062.hostedemail.com [216.40.44.62]) by kanga.kvack.org (Postfix) with ESMTP id CF8486B0036 for ; Mon, 26 Jul 2021 21:01:48 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 82E4418367790 for ; Tue, 27 Jul 2021 01:01:48 +0000 (UTC) X-FDA: 78406565496.05.F529892 Received: from new3-smtp.messagingengine.com (new3-smtp.messagingengine.com [66.111.4.229]) by imf05.hostedemail.com (Postfix) with ESMTP id F27F85028BF6 for ; Tue, 27 Jul 2021 01:01:47 +0000 (UTC) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 4564B580441; Mon, 26 Jul 2021 21:01:47 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Mon, 26 Jul 2021 21:01:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm2; bh=4GoP2lpEbphZOrM/qJqbnIpCwuz xJrI8rOm0CYBK67E=; b=K7+fIREOIDAFh6ouMeGqj4o4UBPlX88yq9vU1atI112 kvpsGuxqSzMOjzJxihs+SdKGc1tyzc9adbOUkbmK88ZudgdfwNHpEkfr59ibhOKk /nFDpGpplnQlTvC6r6U/3iCdiYYdyt5mOLXQ3X+WAzFzJ5t8yJxGC03uikSVl/xn lUG1vWA99D6vj0AFxIG2QEbep0cBljPVuAH1moI60xNJwqefd4bR3rS6F06qf5yc 0WFbFNX3AT0NWyorr5cGHR2yQzn1F0xMYFS3T4x/H7IhzdR+WwgV5AQAiEWnEVaG KfOD8cdoPtmPs55C35Qrc7Yu7gQrEVdT3IJtF5mbkmA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=4GoP2l pEbphZOrM/qJqbnIpCwuzxJrI8rOm0CYBK67E=; b=qJwNQmDsYMZetuodgigFCU DvrvTuxQ8xE3gzsT+C7iUxomQ4Qoh9SiXYY7RPEyWopwpdK2BL51aZeO4DCvy6Qd 0yffH5Vu4WlxgxJeJciaDds/mdyozdf/EFvVXoDtdjTLGpFwU5P0hJJWoZ3jxUa+ b90YhIz3irTxBBU3Uf6YO/lIbIUP3jOssjEUrgHIti/Lt/VCLyUj8+zs+TNyFiME XQYX51ZDyPjk7Gt29xpSulhW99pZw4WJvX9rQywAbjr11JS6JwD8Mz3+od29skK7 +fpcJFmqUFTOflQuJOSzxRXHU02itQObCxSP1/Zht7AVHQgeAsSfpLSKzruSY/HA == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrgeeigdeffecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpeffhffvuffkfhggtggujgesthdtredttddtvdenucfhrhhomheptehnughrvghs ucfhrhgvuhhnugcuoegrnhgurhgvshesrghnrghrrgiivghlrdguvgeqnecuggftrfgrth htvghrnhepudekhfekleeugeevteehleffffejgeelueduleeffeeutdelffeujeffhfeu ffdunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomheprg hnughrvghssegrnhgrrhgriigvlhdruggv X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 26 Jul 2021 21:01:45 -0400 (EDT) Date: Mon, 26 Jul 2021 18:01:44 -0700 From: Andres Freund To: Theodore Ts'o Cc: Matthew Wilcox , James Bottomley , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Linus Torvalds , Andrew Morton , "Darrick J. Wong" , Christoph Hellwig , Michael Larabel Subject: Re: Folios give an 80% performance win Message-ID: <20210727010144.na67murecket5h4b@alap3.anarazel.de> References: <20210715033704.692967-1-willy@infradead.org> <1e48f7edcb6d9a67e8b78823660939007e14bae1.camel@HansenPartnership.com> <17a9d8bf-cd52-4e6c-9b3e-2fbc1e4592d9@www.fastmail.com> <4c634d08-c658-44cf-ac92-92097eeb8532@www.fastmail.com> <20210724214413.fqsbjxhhodfzchs6@alap3.anarazel.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=anarazel.de header.s=fm2 header.b=K7+fIREO; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=qJwNQmDs; spf=pass (imf05.hostedemail.com: domain of andres@anarazel.de designates 66.111.4.229 as permitted sender) smtp.mailfrom=andres@anarazel.de; dmarc=none X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: F27F85028BF6 X-Stat-Signature: s8x6bfrz6pmwuxguc1sgqm7pb8j7qh1n X-HE-Tag: 1627347707-406843 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, On 2021-07-26 10:19:11 -0400, Theodore Ts'o wrote: > On Sat, Jul 24, 2021 at 02:44:13PM -0700, Andres Freund wrote: > > The phoronix test uses postgres with only one relevant setting adjusted > > (increasing the max connection count). That will end up using a buffer pool of > > 128MB, no huge pages, and importantly is configured to aim for not more than > > 1GB for postgres' journal, which will lead to constant checkpointing. The test > > also only runs for 15 seconds, which likely isn't even enough to "warm up" > > (the creation of the data set here will take longer than the run). > > > > Given that the dataset phoronix is using is about ~16GB of data (excluding > > WAL), and uses 256 concurrent clients running full tilt, using that limited > > postgres settings doesn't end up measuring something particularly interesting > > in my opinion. > I tend to use the phoronix test suite for my performance runs when > testing ext4 changes simply because it's convenient. Can you suggest > a better set configuration settings that I should perhaps use that > might give more "real world" numbers that you would find more > significant? It depends a bit on what you want to test, obviously... At the very least you should 'max_wal_size = 32GB' or such (it'll only use that much if enough WAL is generated within checkpoint timeout, which defaults to 5min). And unfortunately you're not going to get meaningful performance results for a read/write test within 10s, you need to run at least ~11min (so two checkpoints happen). With the default shared_buffers setting of 128MB you are going to simulate a much-larger-than-postgres's-memory workload, albeit one where the page cache *is* big enough on most current machines, unless you limit the size of the page cache considerably. Doing so can be useful to approximate a workload that would take much longer to initialize due to the size. I suggest *not* disabling autovacuum as currently done for performance testing - it's not something many real-world setups can afford to do, so benchmarking FS performance with it disabled doesn't seem like a good idea. FWIW, depending on what kind of thing you want to test, it'd not be hard to come up with a test that less time to initialize. E.g. an insert-only workload without an initial dataset or such. As long as you *do* initialize 16GB of data, I think it'd make sense to measure the time that takes. There's definitely been filesystem level performance changes of that, and it's often going to be more IO intensive. Greetings, Andres Freund