From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754759AbdAaBDh (ORCPT ); Mon, 30 Jan 2017 20:03:37 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:35426 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754624AbdAaBDe (ORCPT ); Mon, 30 Jan 2017 20:03:34 -0500 Subject: Re: [RFC V2 02/12] mm: Isolate HugeTLB allocations away from CDM nodes To: Dave Hansen , Anshuman Khandual , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20170130033602.12275-1-khandual@linux.vnet.ibm.com> <20170130033602.12275-3-khandual@linux.vnet.ibm.com> <01671749-c649-e015-4f51-7acaa1fb5b80@intel.com> Cc: mhocko@suse.com, vbabka@suse.cz, mgorman@suse.de, minchan@kernel.org, aneesh.kumar@linux.vnet.ibm.com, bsingharora@gmail.com, srikar@linux.vnet.ibm.com, haren@linux.vnet.ibm.com, jglisse@redhat.com, dan.j.williams@intel.com From: Anshuman Khandual Date: Tue, 31 Jan 2017 06:33:12 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <01671749-c649-e015-4f51-7acaa1fb5b80@intel.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17013101-7323-0000-0000-000000B9B8B5 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17013101-7324-0000-0000-0000025AC877 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-01-30_11:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1701310009 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/30/2017 10:49 PM, Dave Hansen wrote: > On 01/29/2017 07:35 PM, Anshuman Khandual wrote: >> HugeTLB allocation/release/accounting currently spans across all the nodes >> under N_MEMORY node mask. Coherent memory nodes should not be part of these >> allocations. So use system_ram() call to fetch system RAM only nodes on the >> platform which can then be used for HugeTLB allocation purpose instead of >> N_MEMORY node mask. This isolates coherent device memory nodes from HugeTLB >> allocations. > > Does this end up making it impossible to use hugetlbfs to access device > memory? Right, thats the implementation at the moment. But going forward if we need to have HugeTLB pages on the CDM node, then we can implement through the sysfs interface from individual NUMA node paths instead of changing the generic HugeTLB path. I wrote this up in the cover letter but should also have mentioned in the comment section of this patch as well. Does this approach look okay ? "Now, we ensure complete HugeTLB allocation isolation from CDM nodes. Going forward if we need to support HugeTLB allocation on CDM nodes on targeted basis, then we would have to enable those allocations through the /sys/devices/system/node/nodeN/hugepages/hugepages-16384kB/nr_hugepages interface while still ensuring isolation from other generic sysctl and /sys/kernel/mm/hugepages/hugepages-16384kB/nr_hugepages interfaces."