From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, LOTS_OF_MONEY,MAILING_LIST_MULTI,MONEY_NOHTML,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4EEDC433DB for ; Mon, 11 Jan 2021 20:04:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 684B922C7B for ; Mon, 11 Jan 2021 20:04:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391465AbhAKUEX (ORCPT ); Mon, 11 Jan 2021 15:04:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391137AbhAKUEV (ORCPT ); Mon, 11 Jan 2021 15:04:21 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86A3EC061795 for ; Mon, 11 Jan 2021 12:03:41 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id c79so615633pfc.2 for ; Mon, 11 Jan 2021 12:03:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=ZrJB+VMtGpCDCw4tM56I2rie2eVd0wx7yFHoaKknnYg=; b=UAs6pdpvplb3evozh776RK+Y7oY1aYI/sHPY0xeva0O56+dVywtAJM/5eecmlvKLta tPbZQDW5ISleDtiey5TcQZ5Y3f4maAZrQDtNDpkslUXrcPyimVGjvXpJfDJ3973aeMhc mxbOHCIhelpxcXsND8NVkjbAFc4b6kiWm3u2mjKaB8aQxs8C0+XIk+42KfE2QoHi8D5t +FNO3rjcYNoA45249wLHza+GO79FF+UhYeBADcsUG2okqIym5LCDQB7tpM0YpSPv4Rsk 7ZsRNCWaE5gcxdS+xGL7THXqaZCywa8lprgs0Fq/LSWEGTmrbBE++eWm08ti9fXEH2/q a5lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=ZrJB+VMtGpCDCw4tM56I2rie2eVd0wx7yFHoaKknnYg=; b=suA6fY4LT31WWNflZ5fnkBtJmFgIgVMtUX3ArR7n+AKGFAcDF0+xKDe07mr1ROIfXH yMEjjo76xJwQ47CqZ2ym9hhOp9VdrpwpETqai6IdVx42TqEHFzYVqJteEPVYpks6Xfzn 6oNLrwEyXfy8OEq69Qmh8ux3CaJBen5A9qtNY//mmE72twZN/a4nxjUEsSUOXeHMCuF2 N8TTQedt+mNRMcESsATtl5TAXRAqPSmKkaBs3EyBD5iIKPBJjVUG9exkr0MJ9FuPNquw qqPG4020B+8lasgkEnIr6/Nu8gSMT4lwHCacNYiFgv9Ln5dDiOHI6j8esyyY6pfboGYg R8xQ== X-Gm-Message-State: AOAM532uYxwwig/uTnCiFu42qMiV+6YSKgvLrnM9+Q4rgloaHs2Hc2jc 62QICndrD0jMMEIxnb2J4SFew5rfTScvj1cZ45ciVQ== X-Google-Smtp-Source: ABdhPJxuoye7X2mfkcy+WoKggyaSsMD8piRi3S8uQqJDMwkZ5+yLuLV+oH/Qv5jPm+Cx/V9ac7+aWmlZJXrLL1hmiL0= X-Received: by 2002:a63:5701:: with SMTP id l1mr1118729pgb.381.1610395420744; Mon, 11 Jan 2021 12:03:40 -0800 (PST) MIME-Version: 1.0 References: <20210109171058.497636-1-alobakin@pm.me> <20210109191457.786517-1-alobakin@pm.me> In-Reply-To: <20210109191457.786517-1-alobakin@pm.me> From: Nick Desaulniers Date: Mon, 11 Jan 2021 12:03:29 -0800 Message-ID: Subject: Re: [BUG mips llvm] MIPS: malformed R_MIPS_{HI16,LO16} with LLVM To: Alexander Lobakin , Fangrui Song Cc: clang-built-linux , linux-mips@vger.kernel.org, Thomas Bogendoerfer , Kees Cook , Nathan Chancellor , Sami Tolvanen , Ralf Baechle , LKML , linux-arch Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Alexander, I'm genuinely trying to reproduce/understand this report, questions below: On Sat, Jan 9, 2021 at 11:15 AM Alexander Lobakin wrote: > > From: Nick Desaulniers > Date: Sat, 9 Jan 2021 09:50:44 -0800 > > > On Sat, Jan 9, 2021 at 9:11 AM Alexander Lobakin wrote= : > >> > >> Machine: MIPS32 R2 Big Endian (interAptiv (multi)) > >> > >> While testing MIPS with LLVM, I found a weird and very rare bug with > >> MIPS relocs that LLVM emits into kernel modules. It happens on both > >> 11.0.0 and latest git snapshot and applies, as I can see, only to > >> references to static symbols. > >> > >> When the kernel loads the module, it allocates a space for every > >> section and then manually apply the relocations relative to the > >> new address. > >> > >> Let's say we have a function phy_probe() in drivers/net/phy/libphy.ko. > >> It's static and referenced only in phy_register_driver(), where it's > >> used to fill callback pointer in a structure. > >> > >> The real function address after module loading is 0xc06c1444, that > >> is observed in its ELF st_value field. > >> There are two relocs related to this usage in phy_register_driver(): > >> > >> R_MIPS_HI16 refers to 0x3c010000 > >> R_MIPS_LO16 refers to 0x24339444 Sorry, how are these calculated? (Explicit shell commands invoked would be appreciated) I'm doing: $ ARCH=3Dmips CROSS_COMPILE=3Dmips-linux-gnu- make CC=3Dclang -j71 32r2_def= config $ ARCH=3Dmips CROSS_COMPILE=3Dmips-linux-gnu- make CC=3Dclang -j71 modules $ llvm-nm --format=3Dsysv drivers/net/phy/phy_device.o | grep phy_probe $ llvm-objdump -Dr --disassemble-symbols=3Dphy_driver_register drivers/net/phy/phy_device.o $ llvm-readelf -r drivers/net/phy/phy_device.o | grep -e R_MIPS_HI16 -e R_MIPS_LO16 for some of the commands trying to verify. > >> > >> The address of .text is 0xc06b8000. So the destination is calculated > >> as follows: > >> > >> 0x00000000 from hi16; > >> 0xffff9444 from lo16 (sign extend as it's always treated as signed); > >> 0xc06b8000 from base. > >> > >> =3D 0xc06b1444. The value is lower than the real phy_probe() address > >> (0xc06c1444) by 0x10000 and is lower than the base address of > >> module's .text, so it's 100% incorrect. The disassembly for me produces: 399c: 3c 03 00 00 lui $3, 0 0000399c: R_MIPS_HI16 .text ... 39a8: 24 63 3a 5c addiu $3, $3, 14940 000039a8: R_MIPS_LO16 .text I'm not really sure how to manually resolve the relocations; Fangrui do you have any tips? (I'm coincidentally reading through Linkers & Loaders currently, but only just started chpt. 4). > >> > >> This results in: > >> > >> [ 2.204022] CPU 3 Unable to handle kernel paging request at virtual > >> address c06b1444, epc =3D=3D c06b1444, ra =3D=3D 803f1090 > >> > >> The correct instructions should be: > >> > >> R_MIPS_HI16 0x3c010001 > >> R_MIPS_LO16 0x24339444 > >> > >> so there'll be 0x00010000 from hi16. > >> > >> I tried to catch those bugs in arch/mips/kernel/module.c (by checking > >> if the destination is lower than the base address, which should never > >> happen), and seems like I have only 3 such places in libphy.ko (and > >> one in nf_tables.ko). > >> I don't think it should be handled somehow in mentioned source code > >> as it would look rather ugly and may break kernels build with GNU > >> stack, which seems to not produce such bad codes. > >> > >> If I should report this to any other resources, please let me know. > >> I chose clang-built-linux and LKML as it may not happen with userland > >> (didn't tried to catch). > > > > Thanks for the report. Sounds like we may indeed be producing an > > incorrect relocation. This is only seen for big endian triples? > > Unfortunately I don't have a LE board to play with, so can confirm > only Big Endian. > > (BTW, if someone can say if it's possible for MIPS (and how if it is) > to launch a LE kernel from BE-booted preloader and U-Boot, that would > be super cool) > > > Getting a way for us to deterministically reproduce would be a good > > first step. Which config or configs beyond defconfig, and which > > relocations specifically are you observing this with? > > I use `make 32r2_defconfig` which combines several configs from > arch/mips/configs: > - generic_defconfig; > - generic/32r2.config; > - generic/eb.config. > > Aside from that, I enable a bunch of my WIP drivers and the > Netfilter. On my setup, this bug is always present in libphy.ko, > so CONFIG_PHYLIB=3Dm (with all deps) should be enough. > > The three failed relocs belongs to this part of code: [0] > > llvm-readelf on them: > > Relocation section '.rel.text' at offset 0xbf60 contains 2281 entries:=C2= =AC > [...] > 00005740 00029305 R_MIPS_HI16 00000000 .text > 00005744 00029306 R_MIPS_LO16 00000000 .text > 00005720 00029305 R_MIPS_HI16 00000000 .text > 00005748 00029306 R_MIPS_LO16 00000000 .text > 0000573c 00029305 R_MIPS_HI16 00000000 .text > 0000574c 00029306 R_MIPS_LO16 00000000 .text > > The first pair is the one from my first mail: > 0x3c010000 <-- should be 0x3c010001 to work properly > 0x24339444 > > I'm planning to hunt for more now, will let you know. > > [0] https://elixir.bootlin.com/linux/v5.11-rc2/source/drivers/net/phy/phy= _device.c#L2989 > > > Thanks, > > ~Nick Desaulniers > > Thanks, > Al > --=20 Thanks, ~Nick Desaulniers