From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AE883C433F5 for ; Sat, 1 Oct 2022 23:24:22 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 0790A84C66; Sun, 2 Oct 2022 01:24:20 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=canonical.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.b="W59lyBuo"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 68BDC84C98; Sun, 2 Oct 2022 01:24:18 +0200 (CEST) Received: from smtp-relay-internal-0.canonical.com (smtp-relay-internal-0.canonical.com [185.125.188.122]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 8880B84A59 for ; Sun, 2 Oct 2022 01:24:14 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=canonical.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=heinrich.schuchardt@canonical.com Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 11EFD41473 for ; Sat, 1 Oct 2022 23:24:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1664666652; bh=RZXRuIQMo/N3H9AkuqCXWXbNpqIgQAYcJ+gqynaivnw=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=W59lyBuor/kI8tPmSNlnOXP65ziH0u4tLp5jcLa9BYME1Mt0vYvfhJrNlYjaNhjfv RZHM6ttr7k85cVTh6BgRRQXlYQF6w9fJMG42/IaRybu/AnWzNp9rWMRNRhVPIpjmdv fqKEoGUW196RjvoE4H9aiYVbmEmREeI07xfO0qzMEVDeXC0fONk+6kmi6pkdY5iRVL l9vybwJtf+S8qFUKXEwnylq37FYYmjJkz2ZAwejLI3sbBP+x2wfFBMtJgaXoxP2sb+ TknIkQoT+Ryshzm/ewXH7R96faTDiG+Rg3AYlye7+FKsspVkiv61L1Bqjcozi++SfG OIg+rAx0jozmg== Received: by mail-ed1-f69.google.com with SMTP id i17-20020a05640242d100b0044f18a5379aso6268818edc.21 for ; Sat, 01 Oct 2022 16:24:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date; bh=RZXRuIQMo/N3H9AkuqCXWXbNpqIgQAYcJ+gqynaivnw=; b=YxKUpPdSgsJkTY1rdk/qmyzri3rFrnD8hKpbxjoaywZPvQ0Lyox0r89CZxlJHmfJ9b 4H00fsw2P/8vK9qTleiZT5nSI3fcYVbCXRTZFBIhEU2XkSLdRhkX5e0vhTxfcC6tGqyo 8oG7O1HWVKWsYL0Xu4/cwAgyOJAik0Jo8X4/0N5+AcfDx+9bEOFHgBx5ItmRjbVXNSl2 cSuS5lwRb7LpxHrezU0QVdSPrGSYeK4p6wdOjzguMcPyKMFm6xaleKfhzN0uznJM9Rxg 9+xyC89Int4zGdcURhf9Luk8jSv+ieZ7T8Kq1zuGeXtYQgeCJTRKfXgikG59A+2dqmtR 8NVg== X-Gm-Message-State: ACrzQf0HA03KR4TTMdR5UV3Eg3oacfLUV+HWeWBfaCN4S130nu8P/l7p X27jTolXCFVA1xsD2IDHOUrxmRVySa9CpU2Piq8QATtngT6SFvLk+M78I3niql3iM8HZaUw7Van kcJeCiI1BGAUtudWAs4+7CawjGaVS7EY= X-Received: by 2002:a17:907:a07a:b0:77c:f98e:ab8c with SMTP id ia26-20020a170907a07a00b0077cf98eab8cmr11061684ejc.69.1664666650950; Sat, 01 Oct 2022 16:24:10 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5fvGSV2BbmdBpfrwshRbn/oF923rvHgWuMSKoulMRQdDleCI0+ePuUm8RTc2JPXyFRIY4mrA== X-Received: by 2002:a17:907:a07a:b0:77c:f98e:ab8c with SMTP id ia26-20020a170907a07a00b0077cf98eab8cmr11061667ejc.69.1664666650563; Sat, 01 Oct 2022 16:24:10 -0700 (PDT) Received: from [192.168.123.94] (ip-084-118-157-002.um23.pools.vodafone-ip.de. [84.118.157.2]) by smtp.gmail.com with ESMTPSA id ek12-20020a056402370c00b004587f9d3ce8sm3832827edb.56.2022.10.01.16.24.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 01 Oct 2022 16:24:09 -0700 (PDT) Message-ID: <61c6ba48-f0bc-db5b-3068-ff503e91bb9d@canonical.com> Date: Sun, 2 Oct 2022 01:24:08 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.1 Subject: Re: [PATCH] riscv: Fix build against binutils 2.38 To: Leo Liang , Alexandre Ghiti Cc: Rick Chen , Aurelien Jarno , u-boot@lists.denx.de, bmeng.cn@gmail.com, sjg@chromium.org, trini@konsulko.com References: <20220128134713.2322800-1-alexandre.ghiti@canonical.com> Content-Language: en-US From: Heinrich Schuchardt In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean On 3/9/22 07:31, Leo Liang wrote: > Hi Alex, > On Thu, Mar 03, 2022 at 11:06:18AM +0000, Leo Liang wrote: >> Hi Alex, >> On Tue, Mar 01, 2022 at 03:21:56AM +0000, Leo Liang wrote: >>> Hi Alex, >>> On Mon, Feb 21, 2022 at 05:42:41PM +0100, Alexandre Ghiti wrote: >>>> On Sat, Feb 19, 2022 at 9:52 AM Leo Liang wrote: >>>>> >>>>> Hi Alex, >>>>> On Thu, Feb 17, 2022 at 11:28:46AM +0100, Alexandre Ghiti wrote: >>>>>> Hi Leo, >>>>>> >>>>>> On Thu, Feb 17, 2022 at 10:25 AM Leo Liang wrote: >>>>>>> >>>>>>> Hi Alexandre, >>>>>>> On Fri, Jan 28, 2022 at 02:47:13PM +0100, Alexandre Ghiti wrote: >>>>>>>> The following description is copied from the equivalent patch for the >>>>>>>> Linux Kernel proposed by Aurelien Jarno: >>>>>>>> >>>>>>>> From version 2.38, binutils default to ISA spec version 20191213. This >>>>>>>> means that the csr read/write (csrr*/csrw*) instructions and fence.i >>>>>>>> instruction has separated from the `I` extension, become two standalone >>>>>>>> extensions: Zicsr and Zifencei. As the kernel uses those instruction, >>>>>>>> this causes the following build failure: >>>>>>>> >>>>>>>> arch/riscv/cpu/mtrap.S: Assembler messages: >>>>>>>> arch/riscv/cpu/mtrap.S:65: Error: unrecognized opcode `csrr a0,scause' >>>>>>>> arch/riscv/cpu/mtrap.S:66: Error: unrecognized opcode `csrr a1,sepc' >>>>>>>> arch/riscv/cpu/mtrap.S:67: Error: unrecognized opcode `csrr a2,stval' >>>>>>>> arch/riscv/cpu/mtrap.S:70: Error: unrecognized opcode `csrw sepc,a0' >>>>>>>> >>>>>>>> Signed-off-by: Alexandre Ghiti >>>>>>>> --- >>>>>>>> arch/riscv/Makefile | 11 ++++++++++- >>>>>>>> 1 file changed, 10 insertions(+), 1 deletion(-) >>>>>>> >>>>>>> This patch seems to fail CI somehow. >>>>>>> (https://source.denx.de/u-boot/custodians/u-boot-riscv/-/pipelines/11004) >>>>>>> >>>>>>> Could you take a look at it ? >>>>>> >>>>>> I have just tried on master (commit ab8903a24db1) and it failed for >>>>>> the same reason, so this is not related to this patch. Nevertheless, >>>>>> I'll try to bisect the problem :) >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Alex >>>>>> >>>>> >>>>> Thanks for putting the effort into it! >>>>> >>>>> AFAIK, this patch does nothing related to the error message "undefined reference to `__ashldi3'" from the failed CI. >>>>> >>>>> Nonetheless, I have tried a few times myself, >>>>> and found that CI could pass with ab8903a24db1 but cannot pass with this patch on my side. >>>>> https://source.denx.de/u-boot/custodians/u-boot-riscv/-/commits/staging >>>>> https://source.denx.de/u-boot/custodians/u-boot-riscv/-/pipelines >>>>> >>>> >>>> To me it is an issue with the toolchain: libgcc is missing those >>>> symbols. If I use an Ubuntu toolchain, it fails no matter which commit >>>> I am on (I tested as far as v2021.10). But if I use a toolchain from >>>> https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/, >>>> it works fine. >>>> >>>> What I don't understand is how you manage to have different build >>>> results with the same docker image: can you confirm that you use the >>>> same toolchains in the following builds? >>>> >>>> https://source.denx.de/u-boot/custodians/u-boot-riscv/-/jobs/393701 >>>> https://source.denx.de/u-boot/custodians/u-boot-riscv/-/jobs/393783 >>>> >>> >>> Sorry for the late reply. >>> I have checked the toolchain version of these two builds, >>> they are using the same toolchain[1] from Tom's docker image on docker hub[2]. >>> >>> Also the fail is reproducible using this docker image with the following commands: >>> >>> leo@host sudo docker run -it --name leo-test trini/u-boot-gitlab-ci-runner:focal-20220113-03Feb2022 /bin/bash >>> uboot@356268d27bf0:~$ git clone https://source.denx.de/u-boot/custodians/u-boot-riscv.git && cd u-boot-riscv >>> uboot@356268d27bf0:~/u-boot-riscv$ git checkout staging >>> uboot@356268d27bf0:~/u-boot-riscv$ export PATH=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin:$PATH >>> uboot@356268d27bf0:~/u-boot-riscv$ export CROSS_COMPILE=riscv64-linux- >>> uboot@356268d27bf0:~/u-boot-riscv$ make qemu-riscv32_spl_defconfig >>> uboot@356268d27bf0:~/u-boot-riscv$ make -j`nproc` V=1 >>> >>> ... >>> >>> riscv64-linux-ld.bfd -m elf32lriscv --gc-sections -static -pie -Bstatic --no-dynamic-linker -z notext --build-id=none -Ttext 0x81200000 -o u-boot -T u-boot.lds arch/riscv/cpu/start.o --whole-archive arch/riscv/cpu/built-in.o arch/riscv/cpu/generic/built-in.o arch/riscv/lib/built-in.o board/emulation/common/built-in.o board/emulation/qemu-riscv/built-in.o boot/built-in.o cmd/built-in.o common/built-in.o disk/built-in.o drivers/built-in.o drivers/usb/cdns3/built-in.o drivers/usb/common/built-in.o drivers/usb/dwc3/built-in.o drivers/usb/emul/built-in.o drivers/usb/eth/built-in.o drivers/usb/host/built-in.o drivers/usb/mtu3/built-in.o drivers/usb/musb-new/built-in.o drivers/usb/musb/built-in.o drivers/usb/phy/built-in.o drivers/usb/ulpi/built-in.o env/built-in.o fs/built-in.o lib/built-in.o net/built-in.o --no-whole-archive -L /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0 -lgcc -Map u-boot.map; true >>> riscv64-linux-ld.bfd: drivers/virtio/virtio-uclass.o: in function `virtio_uclass_child_pre_probe': >>> /home/uboot/u-boot-riscv/drivers/virtio/virtio-uclass.c:339: undefined reference to `__lshrdi3' >>> riscv64-linux-ld.bfd: /home/uboot/u-boot-riscv/drivers/virtio/virtio-uclass.c:311: undefined reference to `__ashldi3' >>> riscv64-linux-ld.bfd: /home/uboot/u-boot-riscv/drivers/virtio/virtio-uclass.c:310: undefined reference to `__ashldi3' >>> riscv64-linux-ld.bfd: drivers/nvme/nvme.o: in function `nvme_blk_rw': >>> /home/uboot/u-boot-riscv/drivers/nvme/nvme.c:776: undefined reference to `__lshrdi3' >>> riscv64-linux-ld.bfd: fs/ext4/ext4_common.o: in function `ext4fs_get_extent_block': >>> /home/uboot/u-boot-riscv/fs/ext4/ext4_common.c:1560: undefined reference to `__ashldi3' >>> riscv64-linux-ld.bfd: fs/ext4/dev.o: in function `ext4fs_set_blk_dev': >>> /home/uboot/u-boot-riscv/fs/ext4/dev.c:46: undefined reference to `__lshrdi3' >>> riscv64-linux-ld.bfd: lib/display_options.o: in function `print_size': >>> /home/uboot/u-boot-riscv/lib/display_options.c:103: undefined reference to `__lshrdi3' >>> riscv64-linux-ld.bfd: /home/uboot/u-boot-riscv/lib/display_options.c:102: undefined reference to `__ashldi3' >>> riscv64-linux-ld.bfd: /home/uboot/u-boot-riscv/lib/display_options.c:95: undefined reference to `__ashldi3' >>> riscv64-linux-ld.bfd: /home/uboot/u-boot-riscv/lib/display_options.c:124: undefined reference to `__lshrdi3' >>> riscv64-linux-ld.bfd: lib/vsprintf.o: in function `number': >>> /home/uboot/u-boot-riscv/lib/vsprintf.c:213: undefined reference to `__lshrdi3' >>> make: *** [Makefile:1800: u-boot] Error 1 >>> >>> Furthermore, Using the same container and perform the identical build commands listed above on master branch produce no error. >>> >>> What seems a bit odd is that when testing qemu-riscv32_spl, >>> buildman still uses 64 bit toolchain for the build. >>> I am not sure what the effect of "riscv64-linux-ld.bfd -m elf32lriscv" is. >>> >>> Do you have any thoughts ? >>> >>> [1] https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-riscv64-linux.tar.xz >>> [2] https://hub.docker.com/layers/trini/u-boot-gitlab-ci-runner/focal-20220113-03Feb2022/images/sha256-b1cddec69526015c550ac8e528114c976ccd1a0e75676328c4f81b834e06b2b3?context=explore >>> >>> Best regards, >>> Leo >>> >> >> I re-ran again with buildman's verbose option (./tools/buildman/buildman -o /tmp/qemu-riscv32_spl/ -w -E -W -e -V -v --board qemu-riscv32_spl) >> and found that the failed command is as below: >> >> riscv64-linux-ld.bfd -m elf32lriscv --gc-sections -static -pie -Bstatic --no-dynamic-linker -z notext --build-id=none -Ttext 0x81200000 -o u-boot -T u-boot.lds arch/riscv/cpu/start.o --whole-archive arch/riscv/cpu/built-in.o arch/riscv/cpu/generic/built-in.o arch/riscv/lib/built-in.o board/emulation/common/built-in.o board/emulation/qemu-riscv/built-in.o boot/built-in.o cmd/built-in.o common/built-in.o disk/built-in.o drivers/built-in.o drivers/usb/cdns3/built-in.o drivers/usb/common/built-in.o drivers/usb/dwc3/built-in.o drivers/usb/emul/built-in.o drivers/usb/eth/built-in.o drivers/usb/host/built-in.o drivers/usb/mtu3/built-in.o drivers/usb/musb-new/built-in.o drivers/usb/musb/built-in.o drivers/usb/phy/built-in.o drivers/usb/ulpi/built-in.o env/built-in.o fs/built-in.o lib/built-in.o net/built-in.o --no-whole-archive -L /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0 -lgcc -Map u-boot.map; true >> ^^^^^^^^^^^^^^^^^^^^^^ >> >> The reason for this to fail is due to the wrong search path for libgcc. >> The above command uses "-L /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0", >> where it should be "-L /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0/lib32/ilp32[d]". >> >> This search path is generated from Makefile:865 `PLATFORM_LIBGCC := -L $(shell dirname `$(CC) $(c_flags) -print-libgcc-file-name`) -lgcc`. >> >> I'll try to find out how buildman executes the make process. >> >> ``` >> $ ./tools/buildman/buildman -o /tmp/qemu-riscv32_spl/ -w -E -W -e -V -v --board qemu-riscv32_spl >> $ cat /tmp/qemu-riscv32_spl/log | grep u-boot.map >> riscv64-linux-ld.bfd -m elf32lriscv --gc-sections -static -pie -Bstatic --no-dynamic-linker -z notext --build-id=none -Ttext 0x81200000 -o u-boot -T u-boot.lds arch/riscv/cpu/start.o --whole-archive arch/riscv/cpu/built-in.o arch/riscv/cpu/generic/built-in.o arch/riscv/lib/built-in.o board/emulation/common/built-in.o board/emulation/qemu-riscv/built-in.o boot/built-in.o cmd/built-in.o common/built-in.o disk/built-in.o drivers/built-in.o drivers/usb/cdns3/built-in.o drivers/usb/common/built-in.o drivers/usb/dwc3/built-in.o drivers/usb/emul/built-in.o drivers/usb/eth/built-in.o drivers/usb/host/built-in.o drivers/usb/mtu3/built-in.o drivers/usb/musb-new/built-in.o drivers/usb/musb/built-in.o drivers/usb/phy/built-in.o drivers/usb/ulpi/built-in.o env/built-in.o fs/built-in.o lib/built-in.o net/built-in.o --no-whole-archive -L /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0 -lgcc -Map u-boot.map; true >> ``` >> > > This patch looks valid, it's the CI process which causes the error to occur. > > The command "dirname $(riscv64-linux-gcc -march=rv32imac -mabi=ilp32 -print-libgcc-filename)" gives > different result from "dirname $(riscv64-linux-gcc -march=rv32imac_zicsr_zifencei -mabi=ilp32 -print-libgcc-filename)". > > The former gives "/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0/lib32/ilp32" > while the latter gives "/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0". > > The symbol __ashldi3 and __lshrdi3 only exist in libgcc.a under /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0/lib32/ilp32, > so linker search path have to be set to it to link successfully. > > The multilib setting does not cover zifencei and zicsr, so we get a default multilib path from -print-libgcc-file-name, and thus run into error like this. > > We should be able to fix this by using 32 bit toolchain in the CI process for 32 bit platform/configuration. > > Any thoughts or comments ? In our Docker container command tools/buildman/buildman -o build -w -E -W -e --board qemu-riscv32_spl leads to build/toolchain with content gcc /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/riscv64-linux-gcc path /opt/gcc-11.1.0-nolibc/riscv64-linux/bin cross riscv64-linux- arch riscv64 When compiling qemu-riscv32_defconfig with Alexandre's patch and export CROSS_COMPILE=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/riscv64-linux- I see undefined reference to `__ashldi3'. With export CROSS_COMPILE=/opt/gcc-11.1.0-nolibc/riscv32-linux/bin/riscv32-linux- it works correctly. So first thing to do is to fix the Docker container to install the riscv32 toolchain. Next we have to tell binman to use (installing the tool chain is not enough). Best regards Heinrich