From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751774Ab3LLUwp (ORCPT ); Thu, 12 Dec 2013 15:52:45 -0500 Received: from mail-qe0-f49.google.com ([209.85.128.49]:34080 "EHLO mail-qe0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751469Ab3LLUwl (ORCPT ); Thu, 12 Dec 2013 15:52:41 -0500 MIME-Version: 1.0 In-Reply-To: <20131212204950.GA6034@sgi.com> References: <20131212180050.GC134240@sgi.com> <20131212204950.GA6034@sgi.com> From: Andy Lutomirski Date: Thu, 12 Dec 2013 12:52:18 -0800 Message-ID: Subject: Re: [RFC PATCH 2/3] Add tunable to control THP behavior To: Alex Thorlton Cc: "linux-mm@kvack.org" , Andrew Morton , "Kirill A. Shutemov" , Benjamin Herrenschmidt , Rik van Riel , Wanpeng Li , Mel Gorman , Michel Lespinasse , Benjamin LaHaise , Oleg Nesterov , "Eric W. Biederman" , Al Viro , David Rientjes , Zhang Yanfei , Peter Zijlstra , Johannes Weiner , Michal Hocko , Jiang Liu , Cody P Schafer , Glauber Costa , Kamezawa Hiroyuki , Naoya Horiguchi , "linux-kernel@vger.kernel.org" Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Dec 12, 2013 at 12:49 PM, Alex Thorlton wrote: > > > Is there a setting that will turn off the must-be-the-same-node > > behavior? There are workloads where TLB matters more than cross-node > > traffic (or where all the pages are hopelessly shared between nodes, > > but hugepages are still useful). > > That's pretty much how THPs already behave in the kernel, so if you want > to allow THPs to be handed out to one node, but referenced from many > others, you'd just set the threshold to 1, and let the existing code > take over. > Right. I like that behavior for my workload. (Although I currently allocate huge pages -- when I wrote that code, THP interacted so badly with pagecache that it was a non-starter. I think it's fixed now, though.) > > As for the must-be-the-same-node behavior: I'd actually say it's more > like a "must have so much on one node" behavior, in that, if you set the > threshold to 16, for example, 16 4K pages must be faulted in on the same > node, in the same contiguous 2M chunk, before a THP will be created. > What happens after that THP is created is out of our control, it could > be referenced from anywhere. In that case, I guess I misunderstood your description. Are saying that, once any node accesses this many pages in the potential THP, then the whole THP will be mapped? --Andy From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-f171.google.com (mail-qc0-f171.google.com [209.85.216.171]) by kanga.kvack.org (Postfix) with ESMTP id 285806B0035 for ; Thu, 12 Dec 2013 15:52:40 -0500 (EST) Received: by mail-qc0-f171.google.com with SMTP id c9so792750qcz.30 for ; Thu, 12 Dec 2013 12:52:39 -0800 (PST) Received: from mail-qa0-f48.google.com (mail-qa0-f48.google.com [209.85.216.48]) by mx.google.com with ESMTPS id ko6si19939758qeb.123.2013.12.12.12.52.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 12 Dec 2013 12:52:39 -0800 (PST) Received: by mail-qa0-f48.google.com with SMTP id w5so121715qac.7 for ; Thu, 12 Dec 2013 12:52:38 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <20131212204950.GA6034@sgi.com> References: <20131212180050.GC134240@sgi.com> <20131212204950.GA6034@sgi.com> From: Andy Lutomirski Date: Thu, 12 Dec 2013 12:52:18 -0800 Message-ID: Subject: Re: [RFC PATCH 2/3] Add tunable to control THP behavior Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-linux-mm@kvack.org List-ID: To: Alex Thorlton Cc: "linux-mm@kvack.org" , Andrew Morton , "Kirill A. Shutemov" , Benjamin Herrenschmidt , Rik van Riel , Wanpeng Li , Mel Gorman , Michel Lespinasse , Benjamin LaHaise , Oleg Nesterov , "Eric W. Biederman" , Al Viro , David Rientjes , Zhang Yanfei , Peter Zijlstra , Johannes Weiner , Michal Hocko , Jiang Liu , Cody P Schafer , Glauber Costa , Kamezawa Hiroyuki , Naoya Horiguchi , "linux-kernel@vger.kernel.org" On Thu, Dec 12, 2013 at 12:49 PM, Alex Thorlton wrote: > > > Is there a setting that will turn off the must-be-the-same-node > > behavior? There are workloads where TLB matters more than cross-node > > traffic (or where all the pages are hopelessly shared between nodes, > > but hugepages are still useful). > > That's pretty much how THPs already behave in the kernel, so if you want > to allow THPs to be handed out to one node, but referenced from many > others, you'd just set the threshold to 1, and let the existing code > take over. > Right. I like that behavior for my workload. (Although I currently allocate huge pages -- when I wrote that code, THP interacted so badly with pagecache that it was a non-starter. I think it's fixed now, though.) > > As for the must-be-the-same-node behavior: I'd actually say it's more > like a "must have so much on one node" behavior, in that, if you set the > threshold to 16, for example, 16 4K pages must be faulted in on the same > node, in the same contiguous 2M chunk, before a THP will be created. > What happens after that THP is created is out of our control, it could > be referenced from anywhere. In that case, I guess I misunderstood your description. Are saying that, once any node accesses this many pages in the potential THP, then the whole THP will be mapped? --Andy -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org