From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B055C10DAA for ; Sat, 12 Sep 2020 15:53:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F60E207EA for ; Sat, 12 Sep 2020 15:53:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="FjhCwl87" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725873AbgILPxn (ORCPT ); Sat, 12 Sep 2020 11:53:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725846AbgILPxm (ORCPT ); Sat, 12 Sep 2020 11:53:42 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E8DDC061573; Sat, 12 Sep 2020 08:53:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Qc5yghyJGBgDTTfHSsXLoAfmeYI1Ary/kXXv/3UfwB8=; b=FjhCwl87ORYo39n9Un3auXdBhI wu0axmdEZoPb8Uve5mievG0RLvAFghLys8WQSUAXqMzmzGOOmWnP4rrx+JIK3duQSf3f9mhwhMqe4 mtVp5FJV3qRcjAYl5eTRVrwShqp6/g4DQB3N03KdsMBZFoUt0HSyGVcig/YY+80LB+l0nqYgV9EJo 5uLd6kllCTqSnculuq/FIGavc/FuANsUxLDoZpPJ3yidQckO5aEwDcedMneIb8gVzZ5tipp7SOTVJ zdI4BWgWUm+rHCptwwPYbAApJBu31f1+M5o7FOd+gWG7RAjcv4LB9H3JcFYHpkrowBEbgbM+KWHu1 ah8RGPEw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kH7qE-0002Ct-Je; Sat, 12 Sep 2020 15:53:30 +0000 Date: Sat, 12 Sep 2020 16:53:30 +0100 From: Matthew Wilcox To: Amir Goldstein Cc: Michael Larabel , Linus Torvalds , Ted Ts'o , Andreas Dilger , Ext4 Developers List , Jan Kara , linux-fsdevel Subject: Re: Kernel Benchmarking Message-ID: <20200912155330.GC6583@casper.infradead.org> References: <8bb582d2-2841-94eb-8862-91d1225d5ebc@MichaelLarabel.com> <0cbc959e-1b8d-8d7e-1dc6-672cf5b3899a@MichaelLarabel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Sat, Sep 12, 2020 at 10:28:29AM +0300, Amir Goldstein wrote: > On Sat, Sep 12, 2020 at 1:40 AM Michael Larabel > wrote: > > > > On 9/11/20 5:07 PM, Linus Torvalds wrote: > > > On Fri, Sep 11, 2020 at 9:19 AM Linus Torvalds > > > wrote: > > >> Ok, it's probably simply that fairness is really bad for performance > > >> here in general, and that special case is just that - a special case, > > >> not the main issue. > > > Ahh. It turns out that I should have looked more at the fault path > > > after all. It was higher up in the profile, but I ignored it because I > > > found that lock-unlock-lock pattern lower down. > > > > > > The main contention point is actually filemap_fault(). Your apache > > > test accesses the 'test.html' file that is mmap'ed into memory, and > > > all the threads hammer on that one single file concurrently and that > > > seems to be the main page lock contention. > > > > > > Which is really sad - the page lock there isn't really all that > > > interesting, and the normal "read()" path doesn't even take it. But > > > faulting the page in does so because the page will have a long-term > > > existence in the page tables, and so there's a worry about racing with > > > truncate. Here's an idea (sorry, no patch, about to go out for the day) What if we cleared PageUptodate in the truncate path? And then filemap_fault() looks a lot more like generic_file_buffered_read() where we check PageUptodate, and only if it's clear do we take the page lock and call ->readpage. We'd need to recheck PageUptodate after installing the PTE and zap it ourselves, and we wouldn't be able to check page_mapped() in the truncate path any more, which would make me sad. But there's something to be said for making faults cheaper. Even the XFS model where we take the MMAPLOCK_SHARED isn't free -- it's just write-vs-write on a cacheline instead of a visible contention on the page lock.