From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C072CCA479 for ; Thu, 2 Jun 2022 10:17:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233520AbiFBKRr (ORCPT ); Thu, 2 Jun 2022 06:17:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233519AbiFBKRq (ORCPT ); Thu, 2 Jun 2022 06:17:46 -0400 Received: from mail-yb1-xb2b.google.com (mail-yb1-xb2b.google.com [IPv6:2607:f8b0:4864:20::b2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59A9B1B046F; Thu, 2 Jun 2022 03:17:45 -0700 (PDT) Received: by mail-yb1-xb2b.google.com with SMTP id r82so7444160ybc.13; Thu, 02 Jun 2022 03:17:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ADk5iZiJhVeG/cWf0te+HCOZrWubhAAcNQujcLE/58U=; b=Yjyhtkt6uOpnbh4Xx6LvKbhfJRLgY9ze6V1oDpkgzObuREsOxaLK2a3UAaJ/61kJEi mcYQHMS7y2sI7atXg13x00FCZhVEOXmQh+KbHmFRht3zsicz9J9AgPiM/0xJmMehGW32 GkES6xLk9LiSQDLBlhWbNYMtmZRV3XzeAWVZ4K+kCPDBt4AA8Dd/iG6laEIJs06qovN6 ZvV68dkLVvYqW9JXvdroZn0onjcUjoYDM5CYi7OmnJUkpGPQ2JpFcTa1CkKdLOlkBf7z sgN2HeUEkCuJYdBtutAjFtQcXEBJQVVyw+blSlqWVVq7A92CUkymUsRyaG11g2Hr7Mw+ D90w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ADk5iZiJhVeG/cWf0te+HCOZrWubhAAcNQujcLE/58U=; b=gqvhEB8sIs9dET2mn4nz8zMtnF7WjV4w2h6LYzyY2mMhsBlXCNXlAWGrUGxxm2B6u7 V5vLOA2BfokD9MOZB8wS7KKKV7WoSclW9jzv4qypvld3hWsrnWZW3aYaEW37XYFQqVLP rw7utNiQbYZWDY3ftZBai4m+msdT5FoZ4x7BpiAziEx95DpyPe1Y5mMrcBnIxp+KxCq/ E+mFC4LIvB2gG6F8KFEF/M/sJM9JOb+1mMOxhAl3E39oRX2nWHQya/6byQdmEuRHVBxH ALWvMqOp/AB2rB1v6lJ0TnlS/tM6o9UGnwWC9rY8wl8D6qOMBraiGjH2md+nzmyQW/3u +yQg== X-Gm-Message-State: AOAM530UhRVhNGbOXp1StbYQebhkqDqMhR+h3pZKdVxZJZ+uBwiXSIfd yqCej4s8SG8806kmPyVK8DIRBTKJNAezS6io4f0= X-Google-Smtp-Source: ABdhPJwQuufe38w25lspgUqLBTNVJRZdVkrxK6a1Jl0GQFLaCIka66EJILJHy+YRs/+zHx7XKyF3LxWBMfvUK8JVSuQ= X-Received: by 2002:a05:6902:4ad:b0:65c:c0b3:d8e with SMTP id r13-20020a05690204ad00b0065cc0b30d8emr4475878ybs.582.1654165064541; Thu, 02 Jun 2022 03:17:44 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yegor Yefremov Date: Thu, 2 Jun 2022 12:17:33 +0200 Message-ID: Subject: Re: am335x: 5.18.x: system stalling To: Ard Biesheuvel Cc: Arnd Bergmann , Tony Lindgren , Linux-OMAP , linux-clk , Stephen Boyd , Linux ARM Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org On Wed, Jun 1, 2022 at 12:50 PM Ard Biesheuvel wrote: > > On Wed, 1 Jun 2022 at 12:46, Yegor Yefremov wrote: > > > > On Wed, Jun 1, 2022 at 12:06 PM Ard Biesheuvel wrote: > > > > > > On Wed, 1 Jun 2022 at 12:04, Yegor Yefremov wrote: > > > > > > > > On Wed, Jun 1, 2022 at 11:28 AM Ard Biesheuvel wrote: > > > > > > > > > > On Wed, 1 Jun 2022 at 10:08, Ard Biesheuvel wrote: > > > > > > > > > > > > On Wed, 1 Jun 2022 at 09:59, Arnd Bergmann wrote: > > > > > > > > > > > > > > On Wed, Jun 1, 2022 at 9:36 AM Yegor Yefremov > > > > > > > wrote: > > > > > > > > On Tue, May 31, 2022 at 5:23 PM Arnd Bergmann wrote: > > > > > > > > > I've pushed a modified branch now, with that fix on the broken commit, > > > > > > > > > and another change to make CONFIG_IRQSTACKS user-selectable rather > > > > > > > > > than always enabled. That should tell us if the problem is in the SMP > > > > > > > > > patching or in the irqstacks. > > > > > > > > > > > > > > > > > > Can you test the top of this branch with CONFIG_IRQSTACKS disabled, > > > > > > > > > and (if that still stalls) retest the fixed commit f0191ea5c2e5 ("[PART 1] > > > > > > > > > ARM: implement THREAD_INFO_IN_TASK for uniprocessor systems")? > > > > > > > > > > > > > > > > 1. the top of this branch with CONFIG_IRQSTACKS disabled stalls > > > > > > > > 2. f0191ea5c2e5 with the same config - not > > > > > > > > > > > > > > Ok, perfect, that does narrow down the problem quite a bit: The final > > > > > > > patch has seven changes, all of which can be done individually because > > > > > > > in each case the simplified version in f0191ea5c2e5 is meant to run > > > > > > > the exact same instructions as the version after the change, when running > > > > > > > on a uniprocessor machine such as your am335x. > > > > > > > > > > > > > > You have already shown earlier that the get_current() and > > > > > > > __my_cpu_offset() functions are not to blame here, as reverting > > > > > > > only those does not change the behavior. > > > > > > > > > > > > > > This leaves the is_smp() check in set_current(), and the > > > > > > > four macros in . I don't see anything obviously > > > > > > > wrong with any of those five, but I would bet on the macros > > > > > > > here. Can you try bisecting into this commit, maybe reverting > > > > > > > the changes to set_current and get_current first, and then > > > > > > > narrowing it down to (hopefully) a single macro that causes the > > > > > > > problem? > > > > > > > > > > > > > > > > > > > set_current() is never called by the primary CPU, which is why the > > > > > > is_smp() check was removed from there in 57a420435edcb0b94 ("ARM: drop > > > > > > pointless SMP check on secondary startup path"). > > > > > > > > > > > > So that leaves only the four macros in asm/assembler.h, but I don't > > > > > > see anything obviously wrong with those either. > > > > > > > > > > I pushed a patch on top of Arnd's branch at the link below that gets > > > > > rid of the subsections, and uses normal branches (and code patching) > > > > > to switch between the thread ID register and the LDR to retrieve the > > > > > CPU offset and the current pointer. I have no explanation whether or > > > > > why it could make a difference, but I think it's worth a try. > > > > > > > > The link to your repo is missing. > > > > > > > > > > Oops, sorry :-) > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=am335x-stall-test > > > > I have tested your branch and it stalls: > > > > OK, thanks for verifying. My bisection results for f0191ea5c2e5aab29484ede0493ca385eec5472f as a base: percpu.h: sporadic stalls current.h: always stalls assembler.h: no stalls smp.c: no stalls Yegor From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83DF4C433EF for ; Thu, 2 Jun 2022 10:18:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/xA/FnX+qNDYdAdelVLCFAwi4XUbszNshJbuLqkjAKA=; b=naBr/5+sax9ETX xn14mSxQAuRXDdQy0MX4ds1CasCb5nNmugIDZJKv9FaHF2eIh7GU9GNM9Mahzio3PvnaU3Qu7R+RZ tV/d28G0eo8lgZumv9DSiPtR8pbYgwOg8+PcjRvYwuOwn1T2GxOKumI+oM0GdmwYxae6tDVKf0Hfl 6AjFLKsbrVpVC4Bml+lNm3cXn27z1RR5NsugiCmfZfz/Cy15gMGyXqa4HF4UO0PUYj9tUStBuc1yv YxQC77R/XxQpRVoeMBKir7pwMmi5xFg0GG3Q0ELJFUTP7XN6kBvZr+t34ZZvwtk0ZWGFnBmPeYtGw DCtF4CckJ4rXlE0BsSug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nwhto-002lHU-GV; Thu, 02 Jun 2022 10:17:52 +0000 Received: from mail-yb1-xb34.google.com ([2607:f8b0:4864:20::b34]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nwhtl-002lFd-0Q for linux-arm-kernel@lists.infradead.org; Thu, 02 Jun 2022 10:17:50 +0000 Received: by mail-yb1-xb34.google.com with SMTP id z186so7506395ybz.3 for ; Thu, 02 Jun 2022 03:17:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ADk5iZiJhVeG/cWf0te+HCOZrWubhAAcNQujcLE/58U=; b=Yjyhtkt6uOpnbh4Xx6LvKbhfJRLgY9ze6V1oDpkgzObuREsOxaLK2a3UAaJ/61kJEi mcYQHMS7y2sI7atXg13x00FCZhVEOXmQh+KbHmFRht3zsicz9J9AgPiM/0xJmMehGW32 GkES6xLk9LiSQDLBlhWbNYMtmZRV3XzeAWVZ4K+kCPDBt4AA8Dd/iG6laEIJs06qovN6 ZvV68dkLVvYqW9JXvdroZn0onjcUjoYDM5CYi7OmnJUkpGPQ2JpFcTa1CkKdLOlkBf7z sgN2HeUEkCuJYdBtutAjFtQcXEBJQVVyw+blSlqWVVq7A92CUkymUsRyaG11g2Hr7Mw+ D90w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ADk5iZiJhVeG/cWf0te+HCOZrWubhAAcNQujcLE/58U=; b=2Gvm60bMBBnRL/dpu+onqnICRK3dwDKMnkvkMQRzD2uhOlQjUSuZQzO8H2C5gb09AW ic4JF9UczRlj3dISWgwXQcBMglcq+mIcTpYqUfip/2V4675ODyLdWDGIROHNqstEg5RD Kf9FlZUxFjXUv8oNth/tkVkjwV6spxe/EUWrJ3WqrwDH0UZRiyH7Ab/dsfyfh5eoy0li a1vQT/dGra8ksbHZNuxUV7GVDZ1R355cRQlOcKODb2vA2jT+hz03gKiMto2SO3YowlO5 q6E2+SughcKBwzYdPzRT/2Krm+ts2wI9ylxWyEBKbiV6jHf4h5i9+zRWekRxot+upxOD CmVw== X-Gm-Message-State: AOAM5305JpekCT1auNEeXtyjQ9g4PBMCYWSgm7/JkLstnpAfPVdZnQhO CJFIDqK/8PBe+gB3SwhY+oWuVa9ODab6WdXBhQhqCi9uo4k= X-Google-Smtp-Source: ABdhPJwQuufe38w25lspgUqLBTNVJRZdVkrxK6a1Jl0GQFLaCIka66EJILJHy+YRs/+zHx7XKyF3LxWBMfvUK8JVSuQ= X-Received: by 2002:a05:6902:4ad:b0:65c:c0b3:d8e with SMTP id r13-20020a05690204ad00b0065cc0b30d8emr4475878ybs.582.1654165064541; Thu, 02 Jun 2022 03:17:44 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yegor Yefremov Date: Thu, 2 Jun 2022 12:17:33 +0200 Message-ID: Subject: Re: am335x: 5.18.x: system stalling To: Ard Biesheuvel Cc: Arnd Bergmann , Tony Lindgren , Linux-OMAP , linux-clk , Stephen Boyd , Linux ARM X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220602_031749_148194_38CBCD19 X-CRM114-Status: GOOD ( 39.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Jun 1, 2022 at 12:50 PM Ard Biesheuvel wrote: > > On Wed, 1 Jun 2022 at 12:46, Yegor Yefremov wrote: > > > > On Wed, Jun 1, 2022 at 12:06 PM Ard Biesheuvel wrote: > > > > > > On Wed, 1 Jun 2022 at 12:04, Yegor Yefremov wrote: > > > > > > > > On Wed, Jun 1, 2022 at 11:28 AM Ard Biesheuvel wrote: > > > > > > > > > > On Wed, 1 Jun 2022 at 10:08, Ard Biesheuvel wrote: > > > > > > > > > > > > On Wed, 1 Jun 2022 at 09:59, Arnd Bergmann wrote: > > > > > > > > > > > > > > On Wed, Jun 1, 2022 at 9:36 AM Yegor Yefremov > > > > > > > wrote: > > > > > > > > On Tue, May 31, 2022 at 5:23 PM Arnd Bergmann wrote: > > > > > > > > > I've pushed a modified branch now, with that fix on the broken commit, > > > > > > > > > and another change to make CONFIG_IRQSTACKS user-selectable rather > > > > > > > > > than always enabled. That should tell us if the problem is in the SMP > > > > > > > > > patching or in the irqstacks. > > > > > > > > > > > > > > > > > > Can you test the top of this branch with CONFIG_IRQSTACKS disabled, > > > > > > > > > and (if that still stalls) retest the fixed commit f0191ea5c2e5 ("[PART 1] > > > > > > > > > ARM: implement THREAD_INFO_IN_TASK for uniprocessor systems")? > > > > > > > > > > > > > > > > 1. the top of this branch with CONFIG_IRQSTACKS disabled stalls > > > > > > > > 2. f0191ea5c2e5 with the same config - not > > > > > > > > > > > > > > Ok, perfect, that does narrow down the problem quite a bit: The final > > > > > > > patch has seven changes, all of which can be done individually because > > > > > > > in each case the simplified version in f0191ea5c2e5 is meant to run > > > > > > > the exact same instructions as the version after the change, when running > > > > > > > on a uniprocessor machine such as your am335x. > > > > > > > > > > > > > > You have already shown earlier that the get_current() and > > > > > > > __my_cpu_offset() functions are not to blame here, as reverting > > > > > > > only those does not change the behavior. > > > > > > > > > > > > > > This leaves the is_smp() check in set_current(), and the > > > > > > > four macros in . I don't see anything obviously > > > > > > > wrong with any of those five, but I would bet on the macros > > > > > > > here. Can you try bisecting into this commit, maybe reverting > > > > > > > the changes to set_current and get_current first, and then > > > > > > > narrowing it down to (hopefully) a single macro that causes the > > > > > > > problem? > > > > > > > > > > > > > > > > > > > set_current() is never called by the primary CPU, which is why the > > > > > > is_smp() check was removed from there in 57a420435edcb0b94 ("ARM: drop > > > > > > pointless SMP check on secondary startup path"). > > > > > > > > > > > > So that leaves only the four macros in asm/assembler.h, but I don't > > > > > > see anything obviously wrong with those either. > > > > > > > > > > I pushed a patch on top of Arnd's branch at the link below that gets > > > > > rid of the subsections, and uses normal branches (and code patching) > > > > > to switch between the thread ID register and the LDR to retrieve the > > > > > CPU offset and the current pointer. I have no explanation whether or > > > > > why it could make a difference, but I think it's worth a try. > > > > > > > > The link to your repo is missing. > > > > > > > > > > Oops, sorry :-) > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=am335x-stall-test > > > > I have tested your branch and it stalls: > > > > OK, thanks for verifying. My bisection results for f0191ea5c2e5aab29484ede0493ca385eec5472f as a base: percpu.h: sporadic stalls current.h: always stalls assembler.h: no stalls smp.c: no stalls Yegor _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel