From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1A67C388F9 for ; Fri, 20 Nov 2020 02:25:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1FA5122282 for ; Fri, 20 Nov 2020 02:25:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BJ8jYlmc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726342AbgKTCYm (ORCPT ); Thu, 19 Nov 2020 21:24:42 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28396 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726100AbgKTCYm (ORCPT ); Thu, 19 Nov 2020 21:24:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605839081; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VQ9D3r+1A447JsLtjnYIsDZTTCNO7eTqqaIEx3Zq9b8=; b=BJ8jYlmcj7KHfYV4Fb3BEcE9DJ2JQ3olwxJDp35xBtmUe33hfgdZRBEnC8Wfc2UvSgA2Os rn9GCgEWDQjVqU0XntvX5CRXfn2tv/jBB8ihAFxXC4xH/isxjV6sOeZfx95uFVvbzp+GR2 LMh5PPzGtMGymkP714hKhcjijU9ntWE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-153-gmrJeu0QPQOmWuK204KoTg-1; Thu, 19 Nov 2020 21:24:39 -0500 X-MC-Unique: gmrJeu0QPQOmWuK204KoTg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 70B8B8143F3; Fri, 20 Nov 2020 02:24:37 +0000 (UTC) Received: from T590 (ovpn-13-9.pek2.redhat.com [10.72.13.9]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CB82760854; Fri, 20 Nov 2020 02:24:30 +0000 (UTC) Date: Fri, 20 Nov 2020 10:24:26 +0800 From: Ming Lei To: Matthew Wilcox Cc: Pavel Begunkov , linux-fsdevel@vger.kernel.org, Alexander Viro , Jens Axboe , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 1/2] iov_iter: optimise iov_iter_npages for bvec Message-ID: <20201120022426.GC333150@T590> References: <20201120012017.GJ29991@casper.infradead.org> <35d5db17-f6f6-ec32-944e-5ecddcbcb0f1@gmail.com> <20201120014904.GK29991@casper.infradead.org> <3dc0b17d-b907-d829-bfec-eab96a6f4c30@gmail.com> <20201120020610.GL29991@casper.infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201120020610.GL29991@casper.infradead.org> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Fri, Nov 20, 2020 at 02:06:10AM +0000, Matthew Wilcox wrote: > On Fri, Nov 20, 2020 at 01:56:22AM +0000, Pavel Begunkov wrote: > > On 20/11/2020 01:49, Matthew Wilcox wrote: > > > On Fri, Nov 20, 2020 at 01:39:05AM +0000, Pavel Begunkov wrote: > > >> On 20/11/2020 01:20, Matthew Wilcox wrote: > > >>> On Thu, Nov 19, 2020 at 11:24:38PM +0000, Pavel Begunkov wrote: > > >>>> The block layer spends quite a while in iov_iter_npages(), but for the > > >>>> bvec case the number of pages is already known and stored in > > >>>> iter->nr_segs, so it can be returned immediately as an optimisation > > >>> > > >>> Er ... no, it doesn't. nr_segs is the number of bvecs. Each bvec can > > >>> store up to 4GB of contiguous physical memory. > > >> > > >> Ah, really, missed min() with PAGE_SIZE in bvec_iter_len(), then it's a > > >> stupid statement. Thanks! > > >> > > >> Are there many users of that? All these iterators are a huge burden, > > >> just to count one 4KB page in bvec it takes 2% of CPU time for me. > > > > > > __bio_try_merge_page() will create multipage BIOs, and that's > > > called from a number of places including > > > bio_try_merge_hw_seg(), bio_add_page(), and __bio_iov_iter_get_pages() > > > > I get it that there are a lot of places, more interesting how often > > it's actually triggered and if that's performance critical for anybody. > > Not like I'm going to change it, just out of curiosity, but bvec.h > > can be nicely optimised without it. > > Typically when you're allocating pages for the page cache, they'll get > allocated in order and then you'll read or write them in order, so yes, > it ends up triggering quite a lot. There was once a bug in the page > allocator which caused them to get allocated in reverse order and it > was a noticable performance hit (this was 15-20 years ago). hugepage use cases can benefit much from this way too. Thanks, Ming