From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 392C7C433EF for ; Fri, 11 Feb 2022 17:11:08 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1A0B183BC1; Fri, 11 Feb 2022 18:11:06 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="lpK7PTDi"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 7140283BA5; Fri, 11 Feb 2022 18:11:03 +0100 (CET) Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4864C83BB3 for ; Fri, 11 Feb 2022 18:10:55 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=aford173@gmail.com Received: by mail-ej1-x636.google.com with SMTP id fy20so24116594ejc.0 for ; Fri, 11 Feb 2022 09:10:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=S64NwuEaY0+tF5Qbg43ST062Rs4qVdi3FKamRxT6z5s=; b=lpK7PTDiI0pZbFURFXsRCn1vTyYPH+aVugqzZLQ+38iRmWio4yEOhTWWQCLhJWplmr VMpSrAWXy/NII41g1W5D8AMfOUKr1W7kjjj7Y0NEvnHSuEhj52qHoOK/wGmtItnTrJWk yvo+eoC0hcsAeFRgJmGcnh3B0SKsf0Fvp+59qKvEuZbg76sk96ZaE+R1Md+1d7xeCi8o B0FrE78KYmS42KvlXuGXbItAUwESk16bgNwdGF3mOcqwAuwpIkMVzAzkVF2gtYrrFZeR 7VqRiq5wL8h0hMOmP1WGGUP84rZXUo0U0Z5TWM3SmLLMSHAZ2SAj5gx4E3DrOLppnJv8 LwCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=S64NwuEaY0+tF5Qbg43ST062Rs4qVdi3FKamRxT6z5s=; b=570W6/3+bf0WpQjJuIGHV+QXegNKxnaQJgI18pqmglloElS5FeO3iuEdoUEXEBSJ3p J5Rf7lkQdhKXrpexoXMnd9M/yeR/gtIa6JtPqkCc79Kyfp9XEcNweEseMk6Xz8mfnf/F ZfIrlJEcsnsPGvTQLcVKsT/gtRyb7MZfTH2VQgP9n45XaR+KXjrIy3MjZDQDVIwH27Uu J0yJ5kvRh/N1IhZMUW4eZGdawsB5/vH6KLorCbUtE+nVwNdpx/FlNb9jLw6/1zXKePyW 8cuVdF2ssolnhEbT2WSXg4TVqrvdwkFJrJRdGTGtzBYupzxGJmP/kh+fuiu7R2EBjAF+ o5+A== X-Gm-Message-State: AOAM532gLhgtliNwK3cauSJnjYRlmfsLV0rvVbO8LBcIdliFFum67O0R okvAaSoXGwJ2HHrLrcu6eQ3krkIFZpJWEkv1Kks= X-Google-Smtp-Source: ABdhPJwWgOgThZk+2xsFMZaP9wtOgZ2W05BqhWOLF8w3ocGE8YjVWc5g395wopdIWkkl/QtEeb3yWnIF2DKP3RJ0USA= X-Received: by 2002:a17:906:2c9:: with SMTP id 9mr2283868ejk.32.1644599454382; Fri, 11 Feb 2022 09:10:54 -0800 (PST) MIME-Version: 1.0 References: <20211014184811.482560-1-sjg@chromium.org> <20211014124803.v3.7.Id5595981cd99201c6a2d8b714254d775436a3483@changeid> <20220209123219.GZ7515@bill-the-cat> <20220210145750.GH2697206@bill-the-cat> <20220211161208.GF2697206@bill-the-cat> <20220211164453.GG2697206@bill-the-cat> In-Reply-To: <20220211164453.GG2697206@bill-the-cat> From: Adam Ford Date: Fri, 11 Feb 2022 11:10:43 -0600 Message-ID: Subject: Re: [PATCH v3 07/18] pxe: Move pxe_utils files To: Tom Rini Cc: Simon Glass , U-Boot Mailing List , Patrice Chotard , Artem Lapkin , Joe Hershberger , Heinrich Schuchardt , Peter Hoyes Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.5 at phobos.denx.de X-Virus-Status: Clean On Fri, Feb 11, 2022 at 10:44 AM Tom Rini wrote: > > On Fri, Feb 11, 2022 at 10:39:30AM -0600, Adam Ford wrote: > > On Fri, Feb 11, 2022 at 10:12 AM Tom Rini wrote: > > > > > > On Fri, Feb 11, 2022 at 09:50:32AM -0600, Adam Ford wrote: > > > > On Thu, Feb 10, 2022 at 8:57 AM Tom Rini wrote= : > > > > > > > > > > On Thu, Feb 10, 2022 at 07:56:52AM -0600, Adam Ford wrote: > > > > > > On Wed, Feb 9, 2022 at 11:16 AM Simon Glass = wrote: > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > On Wed, 9 Feb 2022 at 05:32, Tom Rini wr= ote: > > > > > > > > > > > > > > > > On Wed, Feb 09, 2022 at 05:40:03AM -0600, Adam Ford wrote: > > > > > > > > > On Thu, Oct 14, 2021 at 1:50 PM Simon Glass wrote: > > > > > > > > > > > > > > > > > > > > Move the header file into the main include/ directory s= o we can use it > > > > > > > > > > from the bootmethod code. Move the C file into boot/ si= nce it relates to > > > > > > > > > > booting. > > > > > > > > > > > > > > > > > > > +cc lokeshvutla@ti.com > > > > > > > > > > > > > > > > > > Simon, > > > > > > > > > > > > > > > > > > I can't explain why, but with git bisect, it appears this= patch breaks > > > > > > > > > my omap3_logic board (DM3730) by making it wrongly think = there is 4GB > > > > > > > > > of RAM, when in reality there is only 256MB. We have bot= h 256MB and > > > > > > > > > 512MB parts, and the automatic memory detection has alway= s 'just > > > > > > > > > worked' in the past. > > > > > > > > > > > > > > > > > > With this patch now, I see: > > > > > > > > > U-Boot 2022.01-rc1-00185-g262cfb5b15 (Feb 09 2022 - 05:23= :42 -0600) > > > > > > > > > > > > > > > > > > OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Cloc= k 1 GHz > > > > > > > > > Model: LogicPD Zoom DM3730 Torpedo + Wireless Development= Kit > > > > > > > > > DRAM: 4 GiB > > > > > > > > > > > > > > > > > > > > > > > > > > > With the previous commit, 8018b9af57b5 ("pxe: Tidy up the= is_pxe > > > > > > > > > global"), it properly detects the RAM and fully boots. > > > > > > > > > > > > > > > > > > U-Boot 2022.01-rc1-00184-g8018b9af57 (Feb 09 2022 - 05:21= :39 -0600) > > > > > > > > > > > > > > > > > > OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Cloc= k 1 GHz > > > > > > > > > Model: LogicPD Zoom DM3730 Torpedo + Wireless Development= Kit > > > > > > > > > DRAM: 256 MiB > > > > > > > > > NAND: 512 MiB > > > > > > > > > MMC: OMAP SD/MMC: 0 > > > > > > > > > Loading Environment from NAND... OK > > > > > > > > > OMAP die ID: 619e00029ff800000168300f1502501f > > > > > > > > > Net: eth0: ethernet@08000000 > > > > > > > > > Hit any key to stop autoboot: 0 > > > > > > > > > OMAP Logic # > > > > > > > > > > > > > > > > > > I have CONFIG_CMD_BOOTM, CONFIG_CMD_PXE and CONFIG_CMD_S= YSBOOT all > > > > > > > > > defined, so I am having a hard time understanding why thi= s would > > > > > > > > > change behavior or stomp on the the structure that knows = the memory > > > > > > > > > size. > > > > > > > > > > > > > > > > > > If I jump ahead to the current 'master' 531c0089457:("Mer= ge branch > > > > > > > > > '2022-02-08-TI-platform-updates') and revert this patch,= my board > > > > > > > > > boots correctly again, but I am struggling to understand = why. > > > > > > + Marek Beh=C3=BAn > > > > > > > > > > > > > > > > > > > > > > > > Do you have any suggestions for me to try? > > > > > > > > > > > > > > > > I would suggest objdump disassemble U-Boot before/after and= see what > > > > > > > > functions have changed. > > > > > > > > > > > > > > Keep an eye out for a BSS variable that is used before reloca= tion, perhaps? > > > > > > > > > > > > I am still investigating, but disabling LTO appears to fix the = issue > > > > > > for me. I'd like to keep LTO, so I'm going to attempt to focus= on the > > > > > > differences in the affected functions and how this patch makes = LTO > > > > > > behave differently. > > > > > > > > > > > > The disassembly of U-Boot is large, so it's going to take me a = bit of > > > > > > time to investigate. If someone has any LTO-related suggestion= s that > > > > > > I could try, I'd be open to try them too. > > > > > > > > > > Wait, the disassembly is large, or the differences between the > > > > > disassembly, before/after this change alone, are large? It's fee= ling > > > > > > > > I will be the first to admit thatI am not very good with the assemb= ly > > > > side of things, but this is what I did: > > > > > > > > git checkout master > > > > make CROSS_COMPILE=3Darm-linux-gnueabihf- -j8 > > > > arm-linux-gnueabihf-objdump -S u-boot > broken.dump > > > > git revert 262cfb5b15420a1aea465745a821e684b3dfa153 > > > > make CROSS_COMPILE=3Darm-linux-gnueabihf- -j8 > > > > arm-linux-gnueabihf-objdump -S u-boot > working.dump > > > > > > > > diff --side-by-side --suppress-common-lines broken.dump working.dum= p > > > > > broken-working.diff > > > > cat -n broken-working.diff > > > > > > > > The broken-working.diff file with common lines suppressed is 236256= lines long. > > > > > > OK, I just use '-d' and not '-S', which might help a little bit. But > > > you're probably going to still need to edit the dumps and just global= ly > > > change all of the addresses to 'XXXXXXXX' so that you'll end up > > > hopefully only seeing where functions were optimized differently. Bu= t > > > it might well end up being a bit trickier than that. > > > > It looks like none of the object files are showing any content with > > objdump when LTO is enabled. With a little google search, it appears > > we need lto-dump. I have some more meetings, but I'll try to spend > > some more time on it this weekend. > > > > > > > > > When I disable LTO for just pxe_utils.o and redo the same exercise, > > > > the diff file with common-lines removed is 266573 lines long. > > > > > > > > Maybe I am not using objdump correctly. I am not all that familiar > > > > with this code either, so I am not sure which variables should be i= n > > > > BSS. I did a search in both working and non-working dumps to look = for > > > > keyworks like BSS, but from what I can tell, both have similar > > > > functions: > > > > > > > > gd->mon_len =3D (ulong)&__bss_end - (ulong)_start; > > > > /* TODO: use (ulong)&__bss_end - (ulong)&__text_start; ? */ > > > > gd->mon_len =3D (ulong)&__bss_end - CONFIG_SYS_MONITOR_BASE; > > > > gd->mon_len =3D (ulong)&__bss_end - (ulong)_start; > > > > * reserve memory for U-Boot code, data & bss > > > > 8011051a : > > > > #if defined(CONFIG_SPL_BUILD) && defined(CONFIG_SPL_EARLY_BSS) > > > > CLEAR_BSS > > > > #if !defined(CONFIG_SPL_BUILD) || !defined(CONFIG_SPL_EARLY_BSS) > > > > CLEAR_BSS > > > > CLEAR_BSS > > > > > > > > When I grepped for mon_len, both sets of dumps looked nearly identi= cal: > > > > > > > > aford@aford-OptiPlex-7050:~/src/u-boot$ grep mon_len working.dump > > > > lmb_reserve(lmb, (phys_addr_t)(uintptr_t)_start, gd->mon_len); > > > > 80112724 : > > > > static int setup_mon_len(void) > > > > gd->mon_len =3D (ulong)&__bss_end - (ulong)_start; > > > > 80112726: 4903 ldr r1, [pc, #12] ; (80112734 ) > > > > 80112728: 4b03 ldr r3, [pc, #12] ; (80112738 ) > > > > gd->mon_len =3D (ulong)&__bss_end - CONFIG_SYS_MONITOR_BASE; > > > > gd->mon_len =3D (ulong)&__bss_end - (ulong)_start; > > > > gd->ram_top =3D board_get_usable_ram_top(gd->mon_len); > > > > gd->relocaddr -=3D gd->mon_len; > > > > gd->mon_len >> 10, gd->relocaddr); > > > > ip =3D mon_lengths[yleap]; > > > > > > > > > > > > aford@aford-OptiPlex-7050:~/src/u-boot$ grep mon_len broken.dump > > > > lmb_reserve(lmb, (phys_addr_t)(uintptr_t)_start, gd->mon_len); > > > > 80110398 : > > > > static int setup_mon_len(void) > > > > gd->mon_len =3D (ulong)&__bss_end - (ulong)_start; > > > > 8011039a: 4903 ldr r1, [pc, #12] ; (801103a8 ) > > > > 8011039c: 4b03 ldr r3, [pc, #12] ; (801103ac ) > > > > gd->mon_len =3D (ulong)&__bss_end - CONFIG_SYS_MONITOR_BASE; > > > > gd->mon_len =3D (ulong)&__bss_end - (ulong)_start; > > > > gd->ram_top =3D board_get_usable_ram_top(gd->mon_len); > > > > gd->relocaddr -=3D gd->mon_len; > > > > gd->mon_len >> 10, gd->relocaddr); > > > > ip =3D mon_lengths[yleap]; > > > > aford@aford-OptiPlex-7050:~/src/u-boot$ > > > > > > > > Since I think I narrowed it down to the pxe_utils.o file, I thought > > > > I'd do an objdump of both the working and non-working versions of > > > > pxe_utils.o and this is where it got interesting. > > > > > > > > With LTO building pxe_utils.o, the dump looks empty: > > > > > > > > arm-linux-gnueabihf-objdump -S boot/pxe_utils.o > pxe-notworking.du= mp > > > > cat pxe-notworking.dump > > > > > > > > boot/pxe_utils.o: file format elf32-littlearm > > > > > > > > ^-- no actual code dump > > > > If I take the working version of this same file without LTO enabled > > > > and do a dump, and it's 2291 lines long and full of functions. > > > > > > > > I tried adding some __used to the non-static function names, but th= at > > > > didn't appear to make any difference to the objdump of pxe_utils.o > > > > > > I feel like it can't be pxe_utils.o itself but rather how LTO is > > > behaving before/after that change and sorting the object files > > > differently. If modifying the dumps like I suggested above doesn't l= ead > > > > That's what I was thinking too. > > > > > to more clues, and it doesn't seem to matter what toolchain is used (= are > > > you using the gcc-11 from kernel.org that we use in docker and > > > buildman?), I'll try and look as well. > > > > I am using GCC 11, but I'm using the version that come with Ubuntu 21.1= 0: > > > > Thread model: posix > > Supported LTO compression algorithms: zlib zstd > > gcc version 11.2.0 (Ubuntu 11.2.0-5ubuntu1) > > OK. FWIW, if it's easier to build and test, I would suggest also trying > CFLAGS_REMOVE_xxx.o :=3D $(LTO_CFLAGS) > for all of the obj files under arch/arm/ and board/ and then if that > also works correctly, re-adding the flags a directory, then file, at a > time until you've narrowed it down. If I'm understanding this correctly, does this mean that you think it's the stuff that's calling the pxe_utils.o and not the pxe_utils.o itself? Doing the CFLAGS_REMOVE on the pxe_utils.o fixes the issue. adam > > -- > Tom