From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE067C433B4 for ; Wed, 21 Apr 2021 15:12:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8BEA261426 for ; Wed, 21 Apr 2021 15:12:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239096AbhDUPMn (ORCPT ); Wed, 21 Apr 2021 11:12:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235784AbhDUPMm (ORCPT ); Wed, 21 Apr 2021 11:12:42 -0400 Received: from mail-oo1-xc29.google.com (mail-oo1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D9E6C06174A for ; Wed, 21 Apr 2021 08:12:09 -0700 (PDT) Received: by mail-oo1-xc29.google.com with SMTP id c12-20020a4ae24c0000b02901bad05f40e4so9270360oot.4 for ; Wed, 21 Apr 2021 08:12:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=FEXEAgUao8/U1XoW751IUwiCrSmDVaqLIc6z62zqqMg=; b=Vo5glLC6k7D+/dE1nb/qouPscezMFfLgYz7H9PDhiS+225DnfgsTkqbIQ3YKQxcbKH N9WNrIISNRSCIXZ0Xa2geQUHPwPnz1H0/r2shySWTDVl9W9/1wWxlwISt92QkyHI1/D5 is4kUk+NU/HEU91rjPdqiJG5y3zN7p9qXt+Hmpoxbd5ZzCUXzUI+sZiJlvFysiAqPOzk 2JWPjDS8PDVfeR7oLmiyTUp1U5Sj+Recg00TFzL2m2g9M6d82OTR14ZDv16kvT/l4M2D BpDuIcq7ihOTMmVUiwcrQGjhGXbhQPyPPMY7gQ1x2zK9oJETG0hcADBQKWBqljmgmIpW xnPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FEXEAgUao8/U1XoW751IUwiCrSmDVaqLIc6z62zqqMg=; b=fdRjoaVo6U44INOn8HQsMzdBOn4c547/7Mea9+bVCFTFAs5eKHqT0tMSEMeAMiuVai mkuwCqu7ReNDstXOwZXnsdnPBizEICV5qn16ar5tnnjjVbtTmXIn5I+2as5U/fMwkiEC HHgELUmV6khs0oJD2webOC7Eko+zrvISJ0+iB+zX/dB3tp1pkPPCxFIOdknU64qS1Lvg wzXB/CDjgXOqyzXW3FVGGW3fPztrDYc/H14gcOZ9YE1DAHbEJFtQ0oG9UeL97QjxEwUN RwhmsLImy4yuAB1Dt2Tgxu9Zr47K4knLAMSujP/PeniAJHeqzIvaxAP/DQYAj6dVIBkf uwgA== X-Gm-Message-State: AOAM530fsXLMvh+6D0yBxkzWymH0+29Un/9U5J2zlA4QTux72wvzUHPL chdhYLl62Nzxh4Uq8ZI1SymKig9iBLM77FUnV/3Z0g== X-Google-Smtp-Source: ABdhPJyng+pvseRT0sVGDwFEJZ8TI5HsU+LGWY1XrOff8CcFEH9YHvKH2/GavzRvVwXXrQ+JymLm5bTa/K1m4eP+ih4= X-Received: by 2002:a4a:d80e:: with SMTP id f14mr18296328oov.54.1619017928732; Wed, 21 Apr 2021 08:12:08 -0700 (PDT) MIME-Version: 1.0 References: <20210408103605.1676875-1-elver@google.com> <20210408103605.1676875-6-elver@google.com> <1fbf3429-42e5-0959-9a5c-91de80f02b6a@samsung.com> <43f8a3bf-34c5-0fc9-c335-7f92eaf23022@samsung.com> <740077ce-efe1-b171-f807-bc5fd95a32ba@samsung.com> In-Reply-To: From: Marco Elver Date: Wed, 21 Apr 2021 17:11:57 +0200 Message-ID: Subject: Re: [PATCH v4 05/10] signal: Introduce TRAP_PERF si_code and si_perf to siginfo To: Marek Szyprowski Cc: Peter Zijlstra , Alexander Shishkin , Arnaldo Carvalho de Melo , Ingo Molnar , Jiri Olsa , Mark Rutland , Namhyung Kim , Thomas Gleixner , Alexander Potapenko , Al Viro , Arnd Bergmann , Christian Brauner , Dmitry Vyukov , Jann Horn , Jens Axboe , Matt Morehouse , Peter Collingbourne , Ian Rogers , Oleg Nesterov , kasan-dev , linux-arch , linux-fsdevel , LKML , "the arch/x86 maintainers" , "open list:KERNEL SELFTEST FRAMEWORK" , Geert Uytterhoeven , Bartlomiej Zolnierkiewicz , Linux ARM , linux-tegra@vger.kernel.org, jonathanh@nvidia.com Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-tegra@vger.kernel.org +Cc linux-arm-kernel On Wed, 21 Apr 2021 at 15:19, Marek Szyprowski wrote: > > Hi Marco, > > On 21.04.2021 13:03, Marco Elver wrote: > > On Wed, 21 Apr 2021 at 12:57, Marek Szyprowski wrote: > >> On 21.04.2021 11:35, Marek Szyprowski wrote: > >>> On 21.04.2021 10:11, Marco Elver wrote: > >>>> On Wed, 21 Apr 2021 at 09:35, Marek Szyprowski > >>>> wrote: > >>>>> On 21.04.2021 08:21, Marek Szyprowski wrote: > >>>>>> On 21.04.2021 00:42, Marco Elver wrote: > >>>>>>> On Tue, 20 Apr 2021 at 23:26, Marek Szyprowski > >>>>>>> wrote: > >>>>>>>> On 08.04.2021 12:36, Marco Elver wrote: > >>>>>>>>> Introduces the TRAP_PERF si_code, and associated siginfo_t field > >>>>>>>>> si_perf. These will be used by the perf event subsystem to send > >>>>>>>>> signals > >>>>>>>>> (if requested) to the task where an event occurred. > >>>>>>>>> > >>>>>>>>> Acked-by: Geert Uytterhoeven # m68k > >>>>>>>>> Acked-by: Arnd Bergmann # asm-generic > >>>>>>>>> Signed-off-by: Marco Elver > >>>>>>>> This patch landed in linux-next as commit fb6cc127e0b6 ("signal: > >>>>>>>> Introduce TRAP_PERF si_code and si_perf to siginfo"). It causes > >>>>>>>> regression on my test systems (arm 32bit and 64bit). Most systems > >>>>>>>> fails > >>>>>>>> to boot in the given time frame. I've observed that there is a > >>>>>>>> timeout > >>>>>>>> waiting for udev to populate /dev and then also during the network > >>>>>>>> interfaces configuration. Reverting this commit, together with > >>>>>>>> 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") to > >>>>>>>> let it > >>>>>>>> compile, on top of next-20210420 fixes the issue. > >>>>>>> Thanks, this is weird for sure and nothing in particular stands out. > >>>>>>> > >>>>>>> I have questions: > >>>>>>> -- Can you please share your config? > >>>>>> This happens with standard multi_v7_defconfig (arm) or just defconfig > >>>>>> for arm64. > >>>>>> > >>>>>>> -- Also, can you share how you run this? Can it be reproduced in > >>>>>>> qemu? > >>>>>> Nothing special. I just boot my test systems and see that they are > >>>>>> waiting lots of time during the udev populating /dev and network > >>>>>> interfaces configuration. I didn't try with qemu yet. > >>>>>>> -- How did you derive this patch to be at fault? Why not just > >>>>>>> 97ba62b27867, given you also need to revert it? > >>>>>> Well, I've just run my boot tests with automated 'git bisect' and that > >>>>>> was its result. It was a bit late in the evening, so I didn't analyze > >>>>>> it further, I've just posted a report about the issue I've found. It > >>>>>> looks that bisecting pointed to a wrong commit somehow. > >>>>>>> If you are unsure which patch exactly it is, can you try just > >>>>>>> reverting 97ba62b27867 and see what happens? > >>>>>> Indeed, this is a real faulty commit. Initially I've decided to revert > >>>>>> it to let kernel compile (it uses some symbols introduced by this > >>>>>> commit). Reverting only it on top of linux-next 20210420 also fixes > >>>>>> the issue. I'm sorry for the noise in this thread. I hope we will find > >>>>>> what really causes the issue. > >>>>> This was a premature conclusion. It looks that during the test I've did > >>>>> while writing that reply, the modules were not deployed properly and a > >>>>> test board (RPi4) booted without modules. In that case the board booted > >>>>> fine and there was no udev timeout. After deploying kernel modules, the > >>>>> udev timeout is back. > >>>> I'm confused now. Can you confirm that the problem is due to your > >>>> kernel modules, or do you think it's still due to 97ba62b27867? Or > >>>> fb6cc127e0b6 (this patch)? > >>> I don't use any custom kernel modules. I just deploy all modules that > >>> are being built from the given kernel defconfig (arm > >>> multi_v7_defconfig or arm64 default) and they are automatically loaded > >>> during the boot by udev. I've checked again and bisect was right. The > >>> kernel built from fb6cc127e0b6 suffers from the described issue, while > >>> the one build from the previous commit (2e498d0a74e5) works fine. > >> I've managed to reproduce this issue with qemu. I've compiled the kernel > >> for arm 32bit with multi_v7_defconfig and used some older Debian rootfs > >> image. The log and qemu parameters are here: > >> https://protect2.fireeye.com/v1/url?k=7cfc23a2-23671aa9-7cfda8ed-002590f5b904-dab7e2ec39dae1f9&q=1&e=36a5ed13-6ad5-430c-8f44-e95c4f0af5c3&u=https%3A%2F%2Fpaste.debian.net%2F1194526%2F > >> > >> Check the timestamp for the 'EXT4-fs (vda): re-mounted' message and > >> 'done (timeout)' status for the 'Waiting for /dev to be fully populated' > >> message. This happens only when kernel modules build from the > >> multi_v7_defconfig are deployed on the rootfs. > > Still hard to say what is going on and what is at fault. But being > > able to repro this in qemu helps debug quicker -- would you also be > > able to share the precise rootfs.img, i.e. upload it somewhere I can > > fetch it? And just to be sure, please also share your .config, as it > > might have compiler-version dependent configuration that might help > > repro (unlikely, but you never know). > > I've managed to reproduce this issue with a public Raspberry Pi OS Lite > rootfs image, even without deploying kernel modules: > > https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-03-25/2021-03-04-raspios-buster-armhf-lite.zip > > # qemu-system-arm -M virt -smp 2 -m 512 -kernel zImage -append "earlycon > console=ttyAMA0 root=/dev/vda2 rw rootwait" -serial stdio -display none > -monitor null -device virtio-blk-device,drive=virtio-blk -drive > file=/tmp/2021-03-04-raspios-buster-armhf-lite.img,id=virtio-blk,if=none,format=raw > -netdev user,id=user -device virtio-net-device,netdev=user > > The above one doesn't boot if zImage z compiled from commit fb6cc127e0b6 > and boots if compiled from 2e498d0a74e5. In both cases I've used default > arm/multi_v7_defconfig and > gcc-linaro-6.4.1-2017.11-x86_64_arm-linux-gnueabi toolchain. Yup, I've narrowed it down to the addition of "__u64 _perf" to siginfo_t. My guess is the __u64 causes a different alignment for a bunch of adjacent fields. It seems that x86 and m68k are the only ones that have compile-time tests for the offsets. Arm should probably add those -- I have added a bucket of static_assert() in arch/arm/kernel/signal.c and see that something's off. I'll hopefully have a fix in a day or so. Thanks, -- Marco From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53D79C433B4 for ; Wed, 21 Apr 2021 15:14:29 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A8DD761449 for ; Wed, 21 Apr 2021 15:14:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8DD761449 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JUSwXQsdpFD7C96kzvGsH46yEP7WuU4e3a42JlaAOdw=; b=aM09cqEEHRQX2xIKSV49Rcqc5 A+PkPAbvIyM8N5NQb6HZOa0t1X6tf5C6deONXtFROz+hvftbXpt0nc/WJSZEUnTNTLakky7ZACEJf GWRtKvGet9l1bPPLVzRvFigxYb1zEXOLbPNnG24ibgUKnTxyILhpdBkDR3vkeY7DpAnVbiY9bdBvh +LvhbSgFjJt/wLI/VIRxHI0S4YvjzeQ8NSjy78YnS/pv4Ael8ZR+911ADcTQjX7jNyDkJEdw54gtZ l9b7XEFNpvA1uhLgsy2ZiPch5QKR1cAMPe38dOfWf4udJdGrsEUmJ7X4TW1hufNTEAJddLv32/E+u GHxvPAxZA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lZEWY-00EeGY-EC; Wed, 21 Apr 2021 15:12:18 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lZEWU-00EeGJ-Ip for linux-arm-kernel@desiato.infradead.org; Wed, 21 Apr 2021 15:12:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:Subject:Message-ID :Date:From:In-Reply-To:References:MIME-Version:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=FEXEAgUao8/U1XoW751IUwiCrSmDVaqLIc6z62zqqMg=; b=mhstBNOtZ4pbcuPKlXmSao36/U wYPkdK/aJNa72T5V0VZKg0ElHKIHmGX57+4Bgg7UBunDmmsr7flHQIsy+R7Mtxo+r3B7MjmSiI1HM V20T0/fVO9mYfGu+lDCUUgq9/EtkRnysep3mkPW9XmcOPz37991mD71TItdrdpfvb5m3Xl2naALAR GlBTsGqeUF41KuMIVEpVFheGz/LTwsJwNrPsdzakWIM72QThxH3k6G7XITK5tyOiYeNYBCVyW0GTz MqwGEb+fZ014urIzq38vBOV/va4QIhS/5rtEix1sxwJRBjec6kISrzdLsSrpRhvKGHkZrJ91Y1rHg oFOBAAGg==; Received: from mail-oo1-xc2e.google.com ([2607:f8b0:4864:20::c2e]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lZEWR-00CzHh-4d for linux-arm-kernel@lists.infradead.org; Wed, 21 Apr 2021 15:12:13 +0000 Received: by mail-oo1-xc2e.google.com with SMTP id e12-20020a056820060cb02901e94efc049dso4341417oow.9 for ; Wed, 21 Apr 2021 08:12:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=FEXEAgUao8/U1XoW751IUwiCrSmDVaqLIc6z62zqqMg=; b=Vo5glLC6k7D+/dE1nb/qouPscezMFfLgYz7H9PDhiS+225DnfgsTkqbIQ3YKQxcbKH N9WNrIISNRSCIXZ0Xa2geQUHPwPnz1H0/r2shySWTDVl9W9/1wWxlwISt92QkyHI1/D5 is4kUk+NU/HEU91rjPdqiJG5y3zN7p9qXt+Hmpoxbd5ZzCUXzUI+sZiJlvFysiAqPOzk 2JWPjDS8PDVfeR7oLmiyTUp1U5Sj+Recg00TFzL2m2g9M6d82OTR14ZDv16kvT/l4M2D BpDuIcq7ihOTMmVUiwcrQGjhGXbhQPyPPMY7gQ1x2zK9oJETG0hcADBQKWBqljmgmIpW xnPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FEXEAgUao8/U1XoW751IUwiCrSmDVaqLIc6z62zqqMg=; b=iW/GLTLfHxo/YWCCNDdsRfHUL1944/rDIh2XhVF1uHHNCEFv+3GJOEeFkehoL5jHu3 VJXtbCtXycXZl/zEpoPiQwI58t7MdMyTHaLR526DpfMOp16T4Q3W2pYgGHIWpVx/rKwG 1H6pbYcyf8PGtEl8n/v/JhhLfap0MJhcPkiEBOEvS/U6qr2+mJndG9pGUNMVryd92MB2 w9zHbGIyntGvwalrrW6HsJwLoJ3N6voZgxousJAgYdQQRCz9mxRTKgcRf3Uj3cZBiF82 eO1sxMs3NYu3LSHEaVD+hKuVVL8SjokkyLaVo9q0485zTKGHXAZo9tmxdPxX/yWskE/G 2FMw== X-Gm-Message-State: AOAM533DzZ/oDrKtWsTlKxv88GeUucQGvoKSMwCDRfO28vUr/iWLUQiz OxQbpzq7s3aImsxcWNQt9WPKVH7mfcxPDWenvDFyhA== X-Google-Smtp-Source: ABdhPJyng+pvseRT0sVGDwFEJZ8TI5HsU+LGWY1XrOff8CcFEH9YHvKH2/GavzRvVwXXrQ+JymLm5bTa/K1m4eP+ih4= X-Received: by 2002:a4a:d80e:: with SMTP id f14mr18296328oov.54.1619017928732; Wed, 21 Apr 2021 08:12:08 -0700 (PDT) MIME-Version: 1.0 References: <20210408103605.1676875-1-elver@google.com> <20210408103605.1676875-6-elver@google.com> <1fbf3429-42e5-0959-9a5c-91de80f02b6a@samsung.com> <43f8a3bf-34c5-0fc9-c335-7f92eaf23022@samsung.com> <740077ce-efe1-b171-f807-bc5fd95a32ba@samsung.com> In-Reply-To: From: Marco Elver Date: Wed, 21 Apr 2021 17:11:57 +0200 Message-ID: Subject: Re: [PATCH v4 05/10] signal: Introduce TRAP_PERF si_code and si_perf to siginfo To: Marek Szyprowski Cc: Peter Zijlstra , Alexander Shishkin , Arnaldo Carvalho de Melo , Ingo Molnar , Jiri Olsa , Mark Rutland , Namhyung Kim , Thomas Gleixner , Alexander Potapenko , Al Viro , Arnd Bergmann , Christian Brauner , Dmitry Vyukov , Jann Horn , Jens Axboe , Matt Morehouse , Peter Collingbourne , Ian Rogers , Oleg Nesterov , kasan-dev , linux-arch , linux-fsdevel , LKML , "the arch/x86 maintainers" , "open list:KERNEL SELFTEST FRAMEWORK" , Geert Uytterhoeven , Bartlomiej Zolnierkiewicz , Linux ARM , linux-tegra@vger.kernel.org, jonathanh@nvidia.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210421_081211_192171_AB4C35A4 X-CRM114-Status: GOOD ( 49.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org +Cc linux-arm-kernel On Wed, 21 Apr 2021 at 15:19, Marek Szyprowski wrote: > > Hi Marco, > > On 21.04.2021 13:03, Marco Elver wrote: > > On Wed, 21 Apr 2021 at 12:57, Marek Szyprowski wrote: > >> On 21.04.2021 11:35, Marek Szyprowski wrote: > >>> On 21.04.2021 10:11, Marco Elver wrote: > >>>> On Wed, 21 Apr 2021 at 09:35, Marek Szyprowski > >>>> wrote: > >>>>> On 21.04.2021 08:21, Marek Szyprowski wrote: > >>>>>> On 21.04.2021 00:42, Marco Elver wrote: > >>>>>>> On Tue, 20 Apr 2021 at 23:26, Marek Szyprowski > >>>>>>> wrote: > >>>>>>>> On 08.04.2021 12:36, Marco Elver wrote: > >>>>>>>>> Introduces the TRAP_PERF si_code, and associated siginfo_t field > >>>>>>>>> si_perf. These will be used by the perf event subsystem to send > >>>>>>>>> signals > >>>>>>>>> (if requested) to the task where an event occurred. > >>>>>>>>> > >>>>>>>>> Acked-by: Geert Uytterhoeven # m68k > >>>>>>>>> Acked-by: Arnd Bergmann # asm-generic > >>>>>>>>> Signed-off-by: Marco Elver > >>>>>>>> This patch landed in linux-next as commit fb6cc127e0b6 ("signal: > >>>>>>>> Introduce TRAP_PERF si_code and si_perf to siginfo"). It causes > >>>>>>>> regression on my test systems (arm 32bit and 64bit). Most systems > >>>>>>>> fails > >>>>>>>> to boot in the given time frame. I've observed that there is a > >>>>>>>> timeout > >>>>>>>> waiting for udev to populate /dev and then also during the network > >>>>>>>> interfaces configuration. Reverting this commit, together with > >>>>>>>> 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") to > >>>>>>>> let it > >>>>>>>> compile, on top of next-20210420 fixes the issue. > >>>>>>> Thanks, this is weird for sure and nothing in particular stands out. > >>>>>>> > >>>>>>> I have questions: > >>>>>>> -- Can you please share your config? > >>>>>> This happens with standard multi_v7_defconfig (arm) or just defconfig > >>>>>> for arm64. > >>>>>> > >>>>>>> -- Also, can you share how you run this? Can it be reproduced in > >>>>>>> qemu? > >>>>>> Nothing special. I just boot my test systems and see that they are > >>>>>> waiting lots of time during the udev populating /dev and network > >>>>>> interfaces configuration. I didn't try with qemu yet. > >>>>>>> -- How did you derive this patch to be at fault? Why not just > >>>>>>> 97ba62b27867, given you also need to revert it? > >>>>>> Well, I've just run my boot tests with automated 'git bisect' and that > >>>>>> was its result. It was a bit late in the evening, so I didn't analyze > >>>>>> it further, I've just posted a report about the issue I've found. It > >>>>>> looks that bisecting pointed to a wrong commit somehow. > >>>>>>> If you are unsure which patch exactly it is, can you try just > >>>>>>> reverting 97ba62b27867 and see what happens? > >>>>>> Indeed, this is a real faulty commit. Initially I've decided to revert > >>>>>> it to let kernel compile (it uses some symbols introduced by this > >>>>>> commit). Reverting only it on top of linux-next 20210420 also fixes > >>>>>> the issue. I'm sorry for the noise in this thread. I hope we will find > >>>>>> what really causes the issue. > >>>>> This was a premature conclusion. It looks that during the test I've did > >>>>> while writing that reply, the modules were not deployed properly and a > >>>>> test board (RPi4) booted without modules. In that case the board booted > >>>>> fine and there was no udev timeout. After deploying kernel modules, the > >>>>> udev timeout is back. > >>>> I'm confused now. Can you confirm that the problem is due to your > >>>> kernel modules, or do you think it's still due to 97ba62b27867? Or > >>>> fb6cc127e0b6 (this patch)? > >>> I don't use any custom kernel modules. I just deploy all modules that > >>> are being built from the given kernel defconfig (arm > >>> multi_v7_defconfig or arm64 default) and they are automatically loaded > >>> during the boot by udev. I've checked again and bisect was right. The > >>> kernel built from fb6cc127e0b6 suffers from the described issue, while > >>> the one build from the previous commit (2e498d0a74e5) works fine. > >> I've managed to reproduce this issue with qemu. I've compiled the kernel > >> for arm 32bit with multi_v7_defconfig and used some older Debian rootfs > >> image. The log and qemu parameters are here: > >> https://protect2.fireeye.com/v1/url?k=7cfc23a2-23671aa9-7cfda8ed-002590f5b904-dab7e2ec39dae1f9&q=1&e=36a5ed13-6ad5-430c-8f44-e95c4f0af5c3&u=https%3A%2F%2Fpaste.debian.net%2F1194526%2F > >> > >> Check the timestamp for the 'EXT4-fs (vda): re-mounted' message and > >> 'done (timeout)' status for the 'Waiting for /dev to be fully populated' > >> message. This happens only when kernel modules build from the > >> multi_v7_defconfig are deployed on the rootfs. > > Still hard to say what is going on and what is at fault. But being > > able to repro this in qemu helps debug quicker -- would you also be > > able to share the precise rootfs.img, i.e. upload it somewhere I can > > fetch it? And just to be sure, please also share your .config, as it > > might have compiler-version dependent configuration that might help > > repro (unlikely, but you never know). > > I've managed to reproduce this issue with a public Raspberry Pi OS Lite > rootfs image, even without deploying kernel modules: > > https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-03-25/2021-03-04-raspios-buster-armhf-lite.zip > > # qemu-system-arm -M virt -smp 2 -m 512 -kernel zImage -append "earlycon > console=ttyAMA0 root=/dev/vda2 rw rootwait" -serial stdio -display none > -monitor null -device virtio-blk-device,drive=virtio-blk -drive > file=/tmp/2021-03-04-raspios-buster-armhf-lite.img,id=virtio-blk,if=none,format=raw > -netdev user,id=user -device virtio-net-device,netdev=user > > The above one doesn't boot if zImage z compiled from commit fb6cc127e0b6 > and boots if compiled from 2e498d0a74e5. In both cases I've used default > arm/multi_v7_defconfig and > gcc-linaro-6.4.1-2017.11-x86_64_arm-linux-gnueabi toolchain. Yup, I've narrowed it down to the addition of "__u64 _perf" to siginfo_t. My guess is the __u64 causes a different alignment for a bunch of adjacent fields. It seems that x86 and m68k are the only ones that have compile-time tests for the offsets. Arm should probably add those -- I have added a bucket of static_assert() in arch/arm/kernel/signal.c and see that something's off. I'll hopefully have a fix in a day or so. Thanks, -- Marco _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel