From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 15AE4C433EF for ; Wed, 18 May 2022 15:35:30 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id CA71E8427F; Wed, 18 May 2022 17:35:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=konsulko.com header.i=@konsulko.com header.b="nXcNMHcv"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id ECA6284285; Wed, 18 May 2022 17:35:25 +0200 (CEST) Received: from mail-qv1-xf2c.google.com (mail-qv1-xf2c.google.com [IPv6:2607:f8b0:4864:20::f2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id C999684278 for ; Wed, 18 May 2022 17:35:22 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=trini@konsulko.com Received: by mail-qv1-xf2c.google.com with SMTP id c9so1086043qvx.8 for ; Wed, 18 May 2022 08:35:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=vjyCnA7p3ht5FWhMxLa25sxM65mRzZlfnPqRX2BoKHY=; b=nXcNMHcvAeu4JBzIeRzd2sjCjcBm6insfNw1vIAsq36jROrzhZQDg+3MvpgBGJ8D2v 10tZcUDXXLJ4En+qSq/Kf/N9asWH4L1vL8w+U2p4bC7aSERLDl1z9GO5a7mgNt/Tsdwv 6WQEV22YV21OV/ARDpnnj6QfsXgecAdRnTRcg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=vjyCnA7p3ht5FWhMxLa25sxM65mRzZlfnPqRX2BoKHY=; b=J0Iqq/YRe5aXLLVDkB0aDzNm0eQzkRaQg0gfYp5XNngn7JEqU74NCmbM+bhJ733Ml/ afRY19h3CG5tF1dorZmOwdNPEU3Km8cQa5kx0ufB6GmxBtiSEZpD7E6R40QzyPc5idsh EGHp2nJNqQYolNs71TtG9hU5YjqugL3wvz9bJOU+AZ2CpT9k/ZDxZBIbcMSpTseXRpGb SUs5O+E3pd09MMvQhT0y3dLFUjRkTxi/E3Y1vZz80u3g13kx6iciDjqJ2Jl5/AiOkI4N MbwqTuvlHlzYHl1rYD+B68P5dfqRDO1yO2f43gGzFFtD8h5I5iKCwGrQNv6NV6GTyeP5 EHUA== X-Gm-Message-State: AOAM530JaBUEu/wHRje6/l3qNxfhAvZLhu2UL0mHkm3Yfiuj4c/2x1kM SwcQxZ2xoRe5rin5pC0Ere+zxA== X-Google-Smtp-Source: ABdhPJzOuQuU8C4Z/EbQ8rk6L8xW6LxBPN2PwaD4dhg1u2nXXl8WAcl3bx3k4XM+SL0Pk6920ZALog== X-Received: by 2002:a05:6214:20ca:b0:45a:c100:7d18 with SMTP id 10-20020a05621420ca00b0045ac1007d18mr25034941qve.27.1652888121453; Wed, 18 May 2022 08:35:21 -0700 (PDT) Received: from bill-the-cat (2603-6081-7b01-4500-d95c-f120-8640-d523.res6.spectrum.com. [2603:6081:7b01:4500:d95c:f120:8640:d523]) by smtp.gmail.com with ESMTPSA id s62-20020ae9de41000000b0069fc13ce23esm1708483qkf.111.2022.05.18.08.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 May 2022 08:35:20 -0700 (PDT) Date: Wed, 18 May 2022 11:35:18 -0400 From: Tom Rini To: Pali =?iso-8859-1?Q?Roh=E1r?= Cc: Simon Glass , u-boot@lists.denx.de Subject: Re: Broken support for 4GB DDR on 32-bit platforms Message-ID: <20220518153518.GC3901321@bill-the-cat> References: <20220517155214.GS3901321@bill-the-cat> <20220517160016.ji4g3ly3rylhmfrp@pali> <20220517163843.GT3901321@bill-the-cat> <20220518105838.qzxtg7hi44dakxrl@pali> <20220518121655.GU3901321@bill-the-cat> <20220518121739.akw25qs67ochhian@pali> <20220518121936.GW3901321@bill-the-cat> <20220518131919.yzc66ieh6bn4gv4h@pali> <20220518151818.GA3901321@bill-the-cat> <20220518152709.syihdweyfzvo5ynh@pali> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="x+3N0iaIwLQJ9Oyk" Content-Disposition: inline In-Reply-To: <20220518152709.syihdweyfzvo5ynh@pali> X-Clacks-Overhead: GNU Terry Pratchett X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.5 at phobos.denx.de X-Virus-Status: Clean --x+3N0iaIwLQJ9Oyk Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 18, 2022 at 05:27:09PM +0200, Pali Roh=E1r wrote: > On Wednesday 18 May 2022 11:18:18 Tom Rini wrote: > > On Wed, May 18, 2022 at 03:19:19PM +0200, Pali Roh=E1r wrote: > > > On Wednesday 18 May 2022 08:19:36 Tom Rini wrote: > > > > On Wed, May 18, 2022 at 02:17:39PM +0200, Pali Roh=E1r wrote: > > > > > On Wednesday 18 May 2022 08:16:55 Tom Rini wrote: > > > > > > On Wed, May 18, 2022 at 12:58:38PM +0200, Pali Roh=E1r wrote: > > > > > > > On Tuesday 17 May 2022 12:38:43 Tom Rini wrote: > > > > > > > > On Tue, May 17, 2022 at 06:00:16PM +0200, Pali Roh=E1r wrot= e: > > > > > > > > > On Tuesday 17 May 2022 11:52:14 Tom Rini wrote: > > > > > > > > > > On Mon, May 16, 2022 at 11:56:51PM +0200, Pali Roh=E1r = wrote: > > > > > > > > > > > On Monday 16 May 2022 08:31:43 Tom Rini wrote: > > > > > > > > > > > > On Sat, May 14, 2022 at 01:00:06AM +0200, Pali Roh= =E1r wrote: > > > > > > > > > > > >=20 > > > > > > > > > > > > > Hello! I tried to enable support for 2GB+ of DDR = memory (with 4GB DDR3) > > > > > > > > > > > > > on powerpc P2020 board in 32-bit addressing mode = and U-Boot crashed > > > > > > > > > > > > > during startup. > > > > > > > > > > > > >=20 > > > > > > > > > > > > > I figured out that issue is not powerpc specific,= but rather generic to > > > > > > > > > > > > > all 32-bit platforms. U-Boot stores memory size i= nto phys_size_t type > > > > > > > > > > > > > (gd->ram_size) and last mapped memory address inc= reased by one byte into > > > > > > > > > > > > > phys_addr_t type (gd->ram_top). > > > > > > > > > > > > >=20 > > > > > > > > > > > > > Despite size 4GB fits into 32-bit addressing mode= , it does not fit into > > > > > > > > > > > > > above two variables, and it overflows to zero. U-= Boot then see zero RAM > > > > > > > > > > > > > size and crashes. > > > > > > > > > > > > >=20 > > > > > > > > > > > > > I tried to workaround this issue by changing both= phys_size_t and > > > > > > > > > > > > > phys_addr_t types to 64-bit. But it did not helpe= d because U-Boot on > > > > > > > > > > > > > many places cast gd->ram_size or gd->ram_top to u= long type, which is > > > > > > > > > > > > > 32-bit on 32-bit platforms. > > > > > > > > > > > > >=20 > > > > > > > > > > > > > Next I changed ulong parameters of board_get_usab= le_ram_top() function > > > > > > > > > > > > > to u64. > > > > > > > > > > > > >=20 > > > > > > > > > > > > > This still was not enough because config value CO= NFIG_MAX_MEM_MAPPED is > > > > > > > > > > > > > ignored on one important place -- in function get= _effective_memsize(). > > > > > > > > > > > > > This config value takes effect only when also CON= FIG_VERY_BIG_RAM is > > > > > > > > > > > > > set. > > > > > > > > > > > > >=20 > > > > > > > > > > > > > Finally With this change I was able to start U-Bo= ot with more than 2GB > > > > > > > > > > > > > of DDR memory inserted in SODIMM slot on P2020. > > > > > > > > > > > > >=20 > > > > > > > > > > > > > How to fix issues with gd->ram_size and gd->ram_t= op? That +1 byte is > > > > > > > > > > > > > really stupid limitation. Changing phys_size_t an= d phys_addr_t types > > > > > > > > > > > > > unconditionally to 64-bit? Or something else? > > > > > > > > > > > > >=20 > > > > > > > > > > > > > And what is the purpose of CONFIG_VERY_BIG_RAM co= nfig option? Why is > > > > > > > > > > > > > CONFIG_MAX_MEM_MAPPED check skipped in get_effect= ive_memsize() function, > > > > > > > > > > > > > but is not skipped on many more places? > > > > > > > > > > > >=20 > > > > > > > > > > > > So, there's two parts to this, as I recall it all. = First, even on 64bit > > > > > > > > > > > > platforms we contain ourselves to 32bit address spa= ce and even then > > > > > > > > > > > > something within the "old" 2GB window. We then set= a CONFIG option to > > > > > > > > > > > > not mess with the memory node in DT which has the r= eal value. Second, > > > > > > > > > > > > for 32bit platforms which can support 4GB memory, o= r more, some further > > > > > > > > > > > > games need to be played, typically I believe around= initializing the > > > > > > > > > > > > memory controller (I'm more confident of that for d= ra7xx_evm, which I > > > > > > > > > > > > don't have the big memory version of, just a small = memory one) so that > > > > > > > > > > > > Linux can do whatever needs doing to enable "36bit"= typically address > > > > > > > > > > > > support. Looking at the other P*36BIT* configs mig= ht give you some more > > > > > > > > > > > > clues about what to do on your platform, or at leas= t who might still be > > > > > > > > > > > > able to explain and test things on the PowerPC side. > > > > > > > > > > >=20 > > > > > > > > > > > I know about 36-bit addressing on e500v2 but I'm not = going to enable it > > > > > > > > > > > due to performance reasons (see Freescale AN4064 [1])= =2E So I want to > > > > > > > > > > > stick with 32-bit addressing for 2GB+ memory usage (a= round 3GB; it is > > > > > > > > > > > 4GB minus memory used by peripherals; which is still = more than 2GB). > > > > > > > > > > >=20 > > > > > > > > > > > And due to 32-bit type for phys_size_t, phys_addr_t a= nd casting these > > > > > > > > > > > types to ulong is an issue. Plus issue with CONFIG_VE= RY_BIG_RAM and > > > > > > > > > > > CONFIG_MAX_MEM_MAPPED as I already wrote. > > > > > > > > > >=20 > > > > > > > > > > I'm not seeing the problem, sorry. You run U-Boot in t= he normal 2GB > > > > > > > > >=20 > > > > > > > > > Ok, I will try to explain it again. > > > > > > > > >=20 > > > > > > > > > I want to run U-Boot in normal 2GB area. U-Boot normally = store detected > > > > > > > > > RAM size into the gd->ram_size structure. And here is the= issue. > > > > > > > > >=20 > > > > > > > > > 32-bit unsigned C type cannot represent maximal 32-bit ad= dressable size. > > > > > > > > > U-Boot tries to store 4GB value =3D 0x100000000 into gd->= ram_size which > > > > > > > > > overflows to 0x00000000. And U-Boot then crashes because = it expects that > > > > > > > > > can store some data at address "gd->ram_size - few_kb" as= it expects > > > > > > > > > that RAM is in the area [0, gd->ram_size). > > > > > > > > >=20 > > > > > > > > > It is more clear what is the problem? > > > > > > > > >=20 > > > > > > > > > If I theoretically find 3.9GB RAM module (but such probab= ly does not > > > > > > > > > exist) then U-Boot should work in normal 2GB area as numb= er 3.9GB can be > > > > > > > > > stored into 32-bit unsigned type. > > > > > > > >=20 > > > > > > > > Yes, you need to disable CONFIG_ARCH_FIXUP_FDT_MEMORY > > > > > > >=20 > > > > > > > CONFIG_ARCH_FIXUP_FDT_MEMORY affects booting OS. > > > > > > >=20 > > > > > > > The issue is that **u-boot** is crashing during its initializ= ation. > > > > > >=20 > > > > > > When configuring the controller for 4GB, but limiting U-Boot to= 2GB? > > > > >=20 > > > > > Yes! > > > >=20 > > > > Interesting. Maybe you just have to do the 36BIT games on that SoC= , to > > > > support what you're trying to do. If anyone at NXP knows something > > > > perhaps they'll chime in. > > >=20 > > > Why? It is issue in U-Boot and affects all 32-bit platforms not only = powerpc/NXP. > >=20 > > I disagree. It's working as intended with other 32bit platforms, unless >=20 > It really cannot work. See my previous emails with exact description. It > is mathematical limitation that you cannot store 4GB size of RAM into > U-Boot's gd structure. Maximal number which can be stored into ulong > type on 32-bit platforms is 2^32-1 which is less than 4GB. >=20 > So if U-Boot detects 4GB DDR module it crashes prior it tries to "limit" > usage just to e.g. 2GB. Yes, this is a semi-intentional limitation of our design history. We're still in the very old days of a 2GB:2GB address space split. > > it's stopped working on dra7xx_evm with >2GB memory (which I don't have, > > unfortunately). If you can't configure the memory controller to know > > that 4GB exists, but limit yourself to running within 2GB, that's a > > SoC-specific problem. >=20 > Have you really read what I wrote? It is not SoC-specific problem. Full > size of RAM (not mapped memory by U-Boot which may be less than full > size of RAM) is stored in gd->ram_size variable. And this variable is > cast to ulong type which cannot store 4GB. This is common U-Boot code, > not something SoC specific. >=20 > Plus CONFIG_MAX_MEM_MAPPED is ignored in some cases which I also > described in previous email. And code for ignoring is again **not** SoC > specific, but generic in common U-Boot code. I guess no, I'm not seeing everything you've wrote, and you're not seeing everything I wrote either. If you have a 32bit platform and more than 2GB of memory, you don't tell U-Boot that you have more than 2GB of memory because it's not going to work right. You lie, one way or another, and have U-Boot work inside a 2GB window, and do whatever you need so that the OS will see the correct amount and do the right thing. All of the options you've noted I believe are related to the kludges that have been required thus far to either enable 36bit support on some PowerPC platforms (which you've explained you don't want to do) or the 36bit support for some ARM platforms (of which I'm asking around to see if someone can re-test on current mainline). I don't _think_ we have an option today that would mirror the 1GB:3GB (peripheral:memory) split that was an option at some point for Linux, as how to start handling bigmem systems. What I keep reading what you want to do is to support 4GB of memory without doing some kludges. I have no idea how much work would be required to enable that, at this point. --=20 Tom --x+3N0iaIwLQJ9Oyk Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmKFEjMACgkQFHw5/5Y0 tyydiAwAulS5YIeWfzUhZYmRFeBiYLpcMAwlsPT7xpBqmOwCuUo4kE1ZfRtO02DP cAoYCAp7kScS7uW9eyiGnBLolMhtyPDLa4koWyICHXusG7a+UD2Ty/TJ+7tQqDcU LWX8fsLJV3raRKW2O7OPLdXTk6J046/GhU5pULLwr35i3AXhZA7vwAZbWsl+vrVc 7tEnqNRiWWEg7A5h11K2ZbgNfCEJevKe8L9IK+v3d7RDPb31OhYlbeemM+182fMF /ztGQwgCrCEm5GV/uOdCQpe+wXoq3Ipt+lKOdeYTKwHoJu87yO39T0zjb02AD6rR CoBhqtQT+HuSBHmyWEeNMqJr7VQkq1A7dXxvOkOXRfimIT5uHj3TWO9dFJN0I9Iv fWPtaR0M3pLo7uOB5qsBj/BZCyCqtaYcqwMN5V0mtJ3gQa8qTrXdvUubqjDo4Ppu r1XNvCIwboYqasFsnrFlwIF71+wu2wl43nk1Lb4KrpkXaB3LCW3Jd5BL326o/zz5 fAwewOZO =68Vg -----END PGP SIGNATURE----- --x+3N0iaIwLQJ9Oyk--