From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 371DAC433DF for ; Wed, 13 May 2020 20:32:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C20FD20659 for ; Wed, 13 May 2020 20:32:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="yHPT6i5J" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725977AbgEMUcE (ORCPT ); Wed, 13 May 2020 16:32:04 -0400 Received: from forward5-smtp.messagingengine.com ([66.111.4.239]:60655 "EHLO forward5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbgEMUcE (ORCPT ); Wed, 13 May 2020 16:32:04 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailforward.nyi.internal (Postfix) with ESMTP id 2DC4F194048A; Wed, 13 May 2020 16:32:03 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 13 May 2020 16:32:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=8/uT6Fyu1iz5eMYwmwTdiVaY95eH7ssrm/l9QXJ0X kc=; b=yHPT6i5JEuO123spClgY1e/ykbHQXZ2iW+mV7slkqCziGEpSxtbc8z2kC 3C3H7CRY6elioS4GJEc0FxGIXXzU3jFDtPz0L2UVDyNwQUmPqqFNajOe2/nueUux dkjjjELuK1zs5a6QVooAhmiPXd1iEy3N53xNWyH30hmxZB4+V/AT5oIJ3K3Da2V4 WVEeU+meGYOG5rY0P5b0Zu7KrPAc5SKac1CwZy8kr5dhKQsdxp/Ppwxe5PsVtnP8 cDzq82jrmF7nlIZVXTwHBtYVLpefzf5PkRm6+VVtkHozqFnklSBA7AAiZ0OeOt+a gvFTYcwIw6SWmkDK28MRjEd3oQarg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrleeggddugeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffvfhfhkffffgggjggtgfesthejredttdefjeenucfhrhhomheprfgvkhhk rgcugfhnsggvrhhguceophgvnhgsvghrghesihhkihdrfhhiqeenucggtffrrghtthgvrh hnpefhveegtdehhfdtfeeuleegfffhtdegkeefleehfeehieelgefgfffhgfehudefteen ucfkphepkeelrddvjedrfeefrddujeefnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphgvnhgsvghrghesihhkihdrfhhi X-ME-Proxy: Received: from [192.168.1.105] (89-27-33-173.bb.dnainternet.fi [89.27.33.173]) by mail.messagingengine.com (Postfix) with ESMTPA id 63489306631F; Wed, 13 May 2020 16:31:59 -0400 (EDT) Subject: Re: [PATCH RFC} io_uring: io_kiocb alloc cache To: Jens Axboe , Jann Horn Cc: io-uring , Xiaoguang Wang , joseph qi , Jiufei Xue , Pavel Begunkov , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Linux-MM References: <492bb956-a670-8730-a35f-1d878c27175f@kernel.dk> <20200513191919.GA10975@nero> From: Pekka Enberg Message-ID: Date: Wed, 13 May 2020 23:31:58 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: io-uring-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Hi Jens, On 5/13/20 1:20 PM, Pekka Enberg wrote: >> So I assume if someone does "perf record", they will see significant >> reduction in page allocator activity with Jens' patch. One possible way >> around that is forcing the page allocation order to be much higher. IOW, >> something like the following completely untested patch: On 5/13/20 11:09 PM, Jens Axboe wrote: > Now tested, I gave it a shot. This seems to bring performance to > basically what the io_uring patch does, so that's great! Again, just in > the microbenchmark test case, so freshly booted and just running the > case. Great, thanks for testing! On 5/13/20 11:09 PM, Jens Axboe wrote: > Will this patch introduce latencies or non-deterministic behavior for a > fragmented system? You have to talk to someone who is more up-to-date with how the page allocator operates today. But yeah, I assume people still want to avoid higher-order allocations as much as possible, because they make allocation harder when memory is fragmented. That said, perhaps it's not going to the page allocator as much as I thought, but the problem is that the per-CPU cache size is just to small for these allocations, forcing do_slab_free() to take the slow path often. Would be interesting to know if CONFIG_SLAB does better here because the per-CPU cache size is much larger IIRC. - Pekka