From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E1F9C43461 for ; Mon, 26 Apr 2021 16:15:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CAF2161175 for ; Mon, 26 Apr 2021 16:15:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234200AbhDZQPv (ORCPT ); Mon, 26 Apr 2021 12:15:51 -0400 Received: from verein.lst.de ([213.95.11.211]:41988 "EHLO verein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233736AbhDZQPu (ORCPT ); Mon, 26 Apr 2021 12:15:50 -0400 Received: by verein.lst.de (Postfix, from userid 2407) id C97DE68CFE; Mon, 26 Apr 2021 18:15:04 +0200 (CEST) Date: Mon, 26 Apr 2021 18:15:03 +0200 From: Christoph Hellwig To: Jens Axboe Cc: Christoph Hellwig , Jeffle Xu , Ming Lei , Damien Le Moal , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: switch block layer polling to a bio based model Message-ID: <20210426161503.GA30994@lst.de> References: <20210426134821.2191160-1-hch@lst.de> <2d229167-f56d-583b-569c-166c97ce2e71@kernel.dk> <20210426150638.GA24618@lst.de> <6b7e3ba0-aa09-b86d-8ea1-dc2e78c7529e@kernel.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6b7e3ba0-aa09-b86d-8ea1-dc2e78c7529e@kernel.dk> User-Agent: Mutt/1.5.17 (2007-11-01) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Mon, Apr 26, 2021 at 09:12:09AM -0600, Jens Axboe wrote: > Here's the series. It's not super clean (yet), but basically allows > users like io_uring to setup a bio cache, and pass that in through > iocb->ki_bi_cache. With that, we can recycle them instead of going > through free+alloc continually. If you look at profiles for high iops, > we're spending more time than desired doing just that. > > https://git.kernel.dk/cgit/linux-block/log/?h=io_uring-bio-cache So where do you spend the cycles? The do not memset the whole bio optimization is pretty obvious and is someting we should do independent of the allocator. The other thing that sucks is the mempool implementation, as it forces each allocation and free to do an indirect call. I think it might be worth to try to frontend it with a normal slab cache and only fall back to the mempool if that fails.