From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 090D1C43461 for ; Wed, 16 Sep 2020 16:32:28 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 88FD920732 for ; Wed, 16 Sep 2020 16:32:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="W6iJl1d4"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="rlDLQX50" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 88FD920732 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KaFPQYmQi45n95r0guY/WLMMPHzaNC/4Iv3Qmttd1qY=; b=W6iJl1d4Gu4ppWl4HzB3UoCdD I0Ta4tLRjTXtkZmXCzRoVoE9FLIdyieDto0+vsF4M3kBLC9zUKy0DE3qX4j2jZPtZgex1uaetoqJ/ xmiTzh7c4iDC2Nm3XsmTgseXEgBMlL4quC836dS01yYZCWtGpXbh7iRvdVCpANrW8Uzv/v/eGFT1D 4Ew1bgI+6sJDL+1mPFFwfsXqVqnTAPI8+l11CjZHBeicHyoOjiCbpjdwiNn5ZZTcFfCciHo1gyjVm PQik9qyXn+RPhfSPvhj6hWdok9GzKgVh4lwVxkIUlUVA7Wr4Fz1Yi9Jlh1BG8PfiSj+0aIGkUVFY1 Ibnulw9zQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIaKd-0007B4-6L; Wed, 16 Sep 2020 16:30:55 +0000 Received: from userp2120.oracle.com ([156.151.31.85]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIaKa-00078W-3z for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 16:30:53 +0000 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 08GGOXv6051337; Wed, 16 Sep 2020 16:30:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=YKdl8gzDEpK1YpA3WFQ+M1n3JfBUrOCHTQf+cD4wugA=; b=rlDLQX50qseJ8UBRo7iVkY7W7X1/qWAPNDxWEm74Ee7+sG8hfbEGCE/Tb2PDzPwPrHgz 28VqUG3+zitRVNBuWdk200HkH6XoL+uomTkfSL2DvmrfVnK9+VLGOfgutpKDBm6lxEae JFU/O7i1nryMgvfcrbOTI/qJyjTJqPqS/VH6hmX26QvteLSlLrBQvR/iyHAyFnV+DXqE 02g+tQMiSselV4fxXlIK0SVjHO6rbOjnef7KK34ll7wseI6cfWbQrI6PQLGlX06497CR FEFYgQ3gQXpEUY511dRH2s6jyShaR8Zd2tfvNAG0eCMOQ4DZKWl/7HeVjw5nRFNFEDsu 6A== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 33j91dntjc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 16 Sep 2020 16:30:31 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 08GGQ2E4007875; Wed, 16 Sep 2020 16:30:30 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 33khpkpsd5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 16 Sep 2020 16:30:30 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 08GGUPg7008158; Wed, 16 Sep 2020 16:30:25 GMT Received: from [192.168.2.112] (/50.38.35.18) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 16 Sep 2020 16:30:25 +0000 Subject: Re: [PATCH] cma: make number of CMA areas dynamic, remove CONFIG_CMA_AREAS To: "Song Bao Hua (Barry Song)" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-mips@vger.kernel.org" References: <20200915205703.34572-1-mike.kravetz@oracle.com> <4b7b14c9eb6a42509f8324f7ed84b46f@hisilicon.com> From: Mike Kravetz Message-ID: <2bf99eee-29b1-4965-da7d-d4e341803440@oracle.com> Date: Wed, 16 Sep 2020 09:30:23 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <4b7b14c9eb6a42509f8324f7ed84b46f@hisilicon.com> Content-Language: en-US X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9746 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 adultscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2009160118 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9746 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 suspectscore=0 mlxlogscore=999 clxscore=1015 adultscore=0 lowpriorityscore=0 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2009160118 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_123052_331725_3794466E X-CRM114-Status: GOOD ( 31.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Aslan Bakirov , Joonsoo Kim , Rik van Riel , Michal Hocko , Andrew Morton , Will Deacon , Roman Gushchin , Mike Rapoport Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 9/16/20 2:14 AM, Song Bao Hua (Barry Song) wrote: >>> -----Original Message----- >>> From: Mike Kravetz [mailto:mike.kravetz@oracle.com] >>> Sent: Wednesday, September 16, 2020 8:57 AM >>> To: linux-mm@kvack.org; linux-kernel@vger.kernel.org; >>> linux-arm-kernel@lists.infradead.org; linux-mips@vger.kernel.org >>> Cc: Roman Gushchin ; Song Bao Hua (Barry Song) >>> ; Mike Rapoport ; Joonsoo >>> Kim ; Rik van Riel ; Aslan Bakirov >>> ; Michal Hocko ; Andrew Morton >>> ; Mike Kravetz >>> Subject: [PATCH] cma: make number of CMA areas dynamic, remove >>> CONFIG_CMA_AREAS >>> >>> The number of distinct CMA areas is limited by the constant >>> CONFIG_CMA_AREAS. In most environments, this was set to a default >>> value of 7. Not too long ago, support was added to allocate hugetlb >>> gigantic pages from CMA. More recent changes to make >> dma_alloc_coherent >>> NUMA-aware on arm64 added more potential users of CMA areas. Along >>> with the dma_alloc_coherent changes, the default value of CMA_AREAS >>> was bumped up to 19 if NUMA is enabled. >>> >>> It seems that the number of CMA users is likely to grow. Instead of >>> using a static array for cma areas, use a simple linked list. These >>> areas are used before normal memory allocators, so use the memblock >>> allocator. >>> >>> Acked-by: Roman Gushchin >>> Signed-off-by: Mike Kravetz >>> --- >>> rfc->v1 >>> - Made minor changes suggested by Song Bao Hua (Barry Song) >>> - Removed check for late calls to cma_init_reserved_mem that was part >>> of RFC. >>> - Added ACK from Roman Gushchin >>> - Still in need of arm testing >> >> Unfortunately, the test result on my arm64 board is negative, Linux can't boot >> after applying >> this patch. >> >> I guess we have to hold on this patch for a while till this is fixed. BTW, Mike, do >> you have >> a qemu-based arm64 numa system to debug? It is very easy to reproduce, we >> don't need to >> use hugetlb_cma and pernuma_cma. Just the default cma will make the boot >> hang. > > Hi Mike, > I spent some time on debugging the boot issue and sent a patch here: > https://lore.kernel.org/linux-mm/20200916085933.25220-1-song.bao.hua@hisilicon.com/ > All details and knic oops can be found there. > pls feel free to merge my patch into your v2 if you want. And we probably need ack from > arm maintainers. > > Also, +Will, > > Hi Will, the whole story is that Mike tried to remove the cma array with CONFIG_CMA_AREAS > and moved to use memblock_alloc() to allocate cma area, so that the number of cma areas > could be dynamic. It turns out it causes a kernel panic on arm64 during system boot as the > returned address from memblock_alloc is invalid before paging_init() is done on arm64. > Thank you! Based on your analysis, I am concerned that other architectures may also have issues. Andrew, I suggest we remove this patch from your tree. I will audit all architectures which enable CMA and look for similar issues there. Will then merge Barry's patch into a V2 with any other arch specific changes. -- Mike Kravetz _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel