From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 984FBCA9ECB for ; Thu, 31 Oct 2019 18:03:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6120E20650 for ; Thu, 31 Oct 2019 18:03:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="RLMQVSl1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6120E20650 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NfopiZLyJmFGu+61+C0dSB7OUtVZhmjfg3GPS2UYgO8=; b=RLMQVSl18ITh90 4S0thf1B5WRJKGjNb2DYlv9kpjgQO/ULGxlvs0pI8CaG5/hMPJ6IiqEz0W0eiucc6p4exgOOe5k9D OCGZlP2jMiwMZGytNBmhI0+pYX3kRdIvzf2b6iKqTpYMVYXloFPj20bg2ar2Crw7mP/fL9NJVVfkz 8wq3TH6A9FqhfZKQnxLWXdSCC3WCZfiQQCrpu/+q81ZknGx7r5YD3jGqFPE50q8Co1xjFjmYWs+yK BaU8xzn3hZxw6121dlx7M8r6nbHFau1yduUx9g2iHtXXe42crJjRjO+H5KgYywQpi90q78mRSNU06 lQGzMKkUw6nX7peffCsA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iQEmX-0002OC-DR; Thu, 31 Oct 2019 18:02:49 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iQEmU-0002NR-3Q; Thu, 31 Oct 2019 18:02:47 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8783A1FB; Thu, 31 Oct 2019 11:02:44 -0700 (PDT) Received: from arrakis.emea.arm.com (arrakis.cambridge.arm.com [10.1.197.42]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 86E343F6C4; Thu, 31 Oct 2019 11:02:42 -0700 (PDT) Date: Thu, 31 Oct 2019 18:02:40 +0000 From: Catalin Marinas To: Nicolas Saenz Julienne Subject: Re: [PATCH v6 3/4] arm64: use both ZONE_DMA and ZONE_DMA32 Message-ID: <20191031180240.GH39590@arrakis.emea.arm.com> References: <6703f8dab4a21fe4e1049f8f224502e1733bf72c.camel@suse.de> <9208de061fe2b9ee7b74206b3cd52cc116e43ac0.camel@suse.de> <1956a2c8f4911b2a7e2ba3c53506c0f06efb93f8.camel@suse.de> <20191031155145.GF39590@arrakis.emea.arm.com> <6fd539b82cbbb2ae307a67a76eb4c2ead0bd5d4a.camel@suse.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <6fd539b82cbbb2ae307a67a76eb4c2ead0bd5d4a.camel@suse.de> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191031_110246_231390_672D48DE X-CRM114-Status: GOOD ( 22.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: f.fainelli@gmail.com, mbrugger@suse.com, marc.zyngier@arm.com, Robin Murphy , Linux Kernel Mailing List , linux-mm@kvack.org, Rob Herring , Qian Cai , wahrenst@gmx.net, m.szyprowski@samsung.com, phill@raspberrypi.org, will@kernel.org, Christoph Hellwig , linux-arm-kernel@lists.infradead.org, linux-rpi-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Oct 31, 2019 at 05:04:34PM +0100, Nicolas Saenz Julienne wrote: > On Thu, 2019-10-31 at 15:51 +0000, Catalin Marinas wrote: > > (sorry, I've been away last week and only now caught up with emails) > > > > On Tue, Oct 22, 2019 at 01:23:32PM +0200, Nicolas Saenz Julienne wrote: > > > On Mon, 2019-10-21 at 16:36 -0400, Qian Cai wrote: > > > > I managed to get more information here, > > > > > > > > [ 0.000000] cma: dma_contiguous_reserve(limit c0000000) > > > > [ 0.000000] cma: dma_contiguous_reserve: reserving 64 MiB for global > > > > area > > > > [ 0.000000] cma: cma_declare_contiguous(size 0x0000000004000000, base > > > > 0x0000000000000000, limit 0x00000000c0000000 alignment 0x0000000000000000) > > > > [ 0.000000] cma: Failed to reserve 512 MiB > > > > > > > > Full dmesg: > > > > > > > > https://cailca.github.io/files/dmesg.txt > > > > > > OK I got it, reproduced it too. > > > > > > Here are the relevant logs: > > > > > > [ 0.000000] DMA [mem 0x00000000802f0000-0x00000000bfffffff] > > > [ 0.000000] DMA32 [mem 0x00000000c0000000-0x00000000ffffffff] > > > [ 0.000000] Normal [mem 0x0000000100000000-0x00000097fcffffff] > > > > > > As you can see ZONE_DMA spans from 0x00000000802f0000-0x00000000bfffffff > > > which > > > is slightly smaller than 1GB. > > > > > > [ 0.000000] crashkernel reserved: 0x000000009fe00000 - > > > 0x00000000bfe00000 (512 MB) > > > > > > Here crashkernel reserved 512M in ZONE_DMA. > > > > > > [ 0.000000] cma: Failed to reserve 512 MiB > > > > > > CMA tried to allocate 512M in ZONE_DMA which fails as there is no enough > > > space. > > > Makes sense. > > > > > > A fix could be moving crashkernel reservations after CMA and then if unable > > > to > > > fit in ZONE_DMA try ZONE_DMA32 before bailing out. Maybe it's a little over > > > the > > > top, yet although most devices will be fine with ZONE_DMA32, the RPi4 needs > > > crashkernel to be reserved in ZONE_DMA. > > > > Does RPi4 need CMA in ZONE_DMA? If not, I'd rather reserve the CMA from > > ZONE_DMA32. > > Yes, CMA is imperatively to be reserved in ZONE_DMA. > > > Even if you moved the crash kernel, someone else might complain that > > they had 2GB of CMA and it no longer works. > > I have yet to look into it, but I've been told that on x86/x64 they have a > 'high' flag to be set alongside with crashkernel that forces the allocation > into ZONE_DMA32. We could mimic this behavior for big servers that don't depend > on ZONE_DMA but need to reserve big chunks of memory. The 'high' flag actually talks about crashkernel reserved above 4G which is not really the case here. Since RPi4 is the odd one out, I'd rather have the default crashkernel and CMA in the ZONE_DMA32 (current mainline behaviour) and have the RPi4 use explicit size@offset parameters for crashkernel and cma. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel