From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8C17C2D0E0 for ; Tue, 15 Sep 2020 15:56:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 43DA7206B7 for ; Tue, 15 Sep 2020 15:56:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="io1LGxHC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 43DA7206B7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CFBF3900049; Tue, 15 Sep 2020 11:56:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C886690000E; Tue, 15 Sep 2020 11:56:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B7150900049; Tue, 15 Sep 2020 11:56:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0130.hostedemail.com [216.40.44.130]) by kanga.kvack.org (Postfix) with ESMTP id 9FF7890000E for ; Tue, 15 Sep 2020 11:56:09 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5CDA0362B for ; Tue, 15 Sep 2020 15:56:09 +0000 (UTC) X-FDA: 77265747258.03.leaf29_5a00d1527112 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 2F74E28A4E9 for ; Tue, 15 Sep 2020 15:56:09 +0000 (UTC) X-HE-Tag: leaf29_5a00d1527112 X-Filterd-Recvd-Size: 6039 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Sep 2020 15:56:08 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id j34so2201423pgi.7 for ; Tue, 15 Sep 2020 08:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=gu3jry4B0AmzKkFeYjUaKPbyQDrej3ixHxhQeqH0BLU=; b=io1LGxHCa+bzpTsA004NgcNc1/CPR3FyTOIzwxWhYkryYEBxZnX16fd1+ybJ1IqjKt JHTA9TnuQzWt8K+BCuyuSXUCv+gmCFfkCrXdpJEKNVjZ6gfh8ZhVcn9SqlTvYpw0pZ1P q0MJkdS2mVtbRcn/DRRlERCwlJAhZKyFoABcNJDlNfntWr4nebJ3ry1fwQYQBmy8De9B 1AWZAbkGyhhg+8v9QzPBCBOgsg8/dvzqzuOOFqvEoLoVmpL0YnA1oBfqcOMK6HddCrZo pdwJ32lfd+/AElGAnoxGAFw8+i4ljnn60UPn16TzcdxTSdsOEGLC23dk5COBcgnjhxHU peEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=gu3jry4B0AmzKkFeYjUaKPbyQDrej3ixHxhQeqH0BLU=; b=CZwj1KlWnoQtLjEol29b5Xo0pWAiIN+q456pKeNCGtKLLCadr2W80+JZO9PK0+Fvdk tHw7zIqV5rho0NB7Z3RDbwg8KyCZg6qiv9t5zPTnuKorR8p2JC0EXFIhfylBURmBEV0o R3um0ITOlq7benFBOp0Y5kr/3JIPLTFqznMipR6wehXPfvF8JIYsX9ecrwyhtruRAD9d kqF3EUFfW7UKnP9ZnFpuSNrJlbYT4o8RsyACSKxZiyWFD7LGVdeSJaUPLDhxIYlG9fPP AwFps6P2LfG1/q2WSQWwk/BTiRQKIbiAKdixdKn2iGuK2i/u+3l4IH03VW8zmZUNTfaU R7HA== X-Gm-Message-State: AOAM530ABNT2WXw5wHN/zkH9m/YWmYJXCY7+DPmOOL6HyvXPPHd7cg+n VO1D517tSJkDh4jf5s+tZFHFKw== X-Google-Smtp-Source: ABdhPJxzhG+emjG10bx4k2+p37+o/FSSbscV6LIByFWdsZd55jr6UgNYr33jGE//tnR4R9iVgA0rHA== X-Received: by 2002:a63:4822:: with SMTP id v34mr14875815pga.342.1600185367654; Tue, 15 Sep 2020 08:56:07 -0700 (PDT) Received: from ziepe.ca ([206.223.160.26]) by smtp.gmail.com with ESMTPSA id f5sm13458346pfj.212.2020.09.15.08.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2020 08:56:06 -0700 (PDT) Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1kIDJN-006U3b-68; Tue, 15 Sep 2020 12:56:05 -0300 Date: Tue, 15 Sep 2020 12:56:05 -0300 From: Jason Gunthorpe To: Peter Xu Cc: Leon Romanovsky , Linux-MM , Linux Kernel Mailing List , "Maya B . Gokhale" , Yang Shi , Marty Mcfadden , Kirill Shutemov , Oleg Nesterov , Jann Horn , Jan Kara , Kirill Tkhai , Andrea Arcangeli , Christoph Hellwig , Andrew Morton , Linus Torvalds Subject: Re: [PATCH 1/4] mm: Trial do_wp_page() simplification Message-ID: <20200915155605.GI1221970@ziepe.ca> References: <20200821234958.7896-1-peterx@redhat.com> <20200821234958.7896-2-peterx@redhat.com> <20200914143829.GA1424636@nvidia.com> <20200914183436.GD30881@xz-x1> <20200914211515.GA5901@xz-x1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200914211515.GA5901@xz-x1> X-Rspamd-Queue-Id: 2F74E28A4E9 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 14, 2020 at 05:15:15PM -0400, Peter Xu wrote: > On Mon, Sep 14, 2020 at 02:34:36PM -0400, Peter Xu wrote: > > On Mon, Sep 14, 2020 at 10:32:11AM -0700, Linus Torvalds wrote: > > > On Mon, Sep 14, 2020 at 7:38 AM Jason Gunthorpe wrote: > > > > > > > > I don't have a detailed explanation right now, but this patch appears > > > > to be causing a regression where RDMA subsystem tests fail. Tests > > > > return to normal when this patch is reverted. > > > > > > > > It kind of looks like the process is not seeing DMA'd data to a > > > > pin_user_pages()? > > > > > > I'm a nincompoop. I actually _talked_ to Hugh Dickins about this when > > > he raised concerns, and I dismissed his concerns with "but PAGE_PIN is > > > special". > > > > > > As usual, Hugh was right. Page pinning certainly _is_ special, but > > > it's not that different from the regular GUP code. > > > > > > But in the meantime, I have a lovely confirmation from the kernel test > > > robot, saying that commit 09854ba94c results in a > > > "vm-scalability.throughput 31.4% improvement", which was what I was > > > hoping for - the complexity wasn't just complexity, it was active > > > badness due to the page locking horrors. > > > > > > I think what we want to do is basically do the "early COW", but only > > > do it for FOLL_PIN (and not turn them into writes for anything but the > > > COW code). So basically redo the "enforced COW mechanism", but rather > > > than do it for everything, now do it only for FOLL_PIN, and only in > > > that COW path. > > > > > > Peter - any chance you can look at this? I'm still looking at the page > > > lock fairness performance regression, although I now think I have a > > > test patch for Phoronix to test out. > > > > Sure, I'll try to prepare something like that and share it shortly. > > Jason, would you please try the attached patch to see whether it unbreaks the > rdma test? Thanks! Hi Peter, My tester says the patch does not help (modified as Leon pointed to make it compile). He did another test where all forks were removed and the test program succeeds. Overall in our test suites we see failurs on tests that involve fork and success on tests that don't. So fork and COW seem very likely to be the issue. Thanks, Jason