From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752817AbbCHSgB (ORCPT ); Sun, 8 Mar 2015 14:36:01 -0400 Received: from mail-ig0-f174.google.com ([209.85.213.174]:40624 "EHLO mail-ig0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751843AbbCHSgA (ORCPT ); Sun, 8 Mar 2015 14:36:00 -0400 MIME-Version: 1.0 In-Reply-To: <20150308100223.GC15487@gmail.com> References: <1425741651-29152-1-git-send-email-mgorman@suse.de> <1425741651-29152-5-git-send-email-mgorman@suse.de> <20150307163657.GA9702@gmail.com> <20150308100223.GC15487@gmail.com> Date: Sun, 8 Mar 2015 11:35:59 -0700 X-Google-Sender-Auth: d0wk4UW4X8f90K9q08AeFK0Xn00 Message-ID: Subject: Re: [PATCH 4/4] mm: numa: Slow PTE scan rate if migration failures occur From: Linus Torvalds To: Ingo Molnar Cc: Mel Gorman , Dave Chinner , Andrew Morton , Aneesh Kumar , Linux Kernel Mailing List , Linux-MM , xfs@oss.sgi.com, ppc-dev Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Mar 8, 2015 at 3:02 AM, Ingo Molnar wrote: > > Well, there's a difference in what we write to the pte: > > #define _PAGE_BIT_NUMA (_PAGE_BIT_GLOBAL+1) > #define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL > > and our expectation was that the two should be equivalent methods from > the POV of the NUMA balancing code, right? Right. But yes, we might have screwed something up. In particular, there might be something that thinks it cares about the global bit, but doesn't notice that the present bit isn't set, so it considers the protnone mappings to be global and causes lots more tlb flushes etc. >> I don't like the old pmdp_set_numa() because it can drop dirty bits, >> so I think the old code was actively buggy. > > Could we, as a silly testing hack not to be applied, write a > hack-patch that re-introduces the racy way of setting the NUMA bit, to > confirm that it is indeed this difference that changes pte visibility > across CPUs enough to create so many more faults? So one of Mel's patches did that, but I don't know if Dave tested it. And thinking about it, it *may* be safe for huge-pages, if they always already have the dirty bit set to begin with. And I don't see how we could have a clean hugepage (apart from the special case of the zeropage, which is read-only, so races on teh dirty bit aren't an issue). So it might actually be that the non-atomic version is safe for hpages. And we could possibly get rid of the "atomic read-and-clear" even for the non-numa case. I'd rather do it for both cases than for just one of them. But: > As a second hack (not to be applied), could we change: > > #define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL > > to: > > #define _PAGE_BIT_PROTNONE (_PAGE_BIT_GLOBAL+1) > > to double check that the position of the bit does not matter? Agreed. We should definitely try that. Dave? Also, is there some sane way for me to actually see this behavior on a regular machine with just a single socket? Dave is apparently running in some fake-numa setup, I'm wondering if this is easy enough to reproduce that I could see it myself. Linus