From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A01AFC433F5 for ; Wed, 22 Sep 2021 15:26:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7D1CA60F21 for ; Wed, 22 Sep 2021 15:26:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236405AbhIVP2V (ORCPT ); Wed, 22 Sep 2021 11:28:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232318AbhIVP2U (ORCPT ); Wed, 22 Sep 2021 11:28:20 -0400 Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F020C061574 for ; Wed, 22 Sep 2021 08:26:50 -0700 (PDT) Received: by mail-lf1-x12f.google.com with SMTP id m3so13465719lfu.2 for ; Wed, 22 Sep 2021 08:26:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sVblMwcY20Jt1iJwJZzOQQzmXaPjWYdpKIFg44drlAI=; b=Iv8Fu1IRNg70EMHGzFeMTd8j/ThtaiAem9UrK07EGRyvDmdYcny/SyCehISsXmkqpx l92lTNBSyBbDXYg1Yhc6RFGTAV9NS5ml7ZmwcQJaPFjWpMPUte3T+pogP3++9F+/Z1A/ kzeL2ruJbqjYn3yHBIgPlbAR8WOp/hvzXBCGw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sVblMwcY20Jt1iJwJZzOQQzmXaPjWYdpKIFg44drlAI=; b=W/PSyDFDWz4k07JlGz4bKAzH5ILQ6PI7ELESMovvxFiwJDr8nEKymu+5paTFv7dCVQ FDiVMOpyrt/4ujs0hJlSUDVMJlU/h/0aH1Ti4EhStFKymeLms0p2iubjl3mctjJtPBED Qtx+/4i+o7seWhxYemVMPu5B1kT17vQCoGW2Rbq2T3ZdQFLD+jMqZ1VI4DQJlurRPm3z /l8si0Hu+wnpJVTXRH5bpGSPRF2MZGRwM7l0PLT5ATe7lMrvbs+1axw1iNZ+ttOh9Qaj 2U7NMCKlAlXq8sHaa2JoCNV/i8uhs6JOlFPeZdBIS725LJ6wzD4RKMHqIIt1ZhD0Ii5V JFiA== X-Gm-Message-State: AOAM530K2QWVh9/Sc6870BGmPTL/cdcNRAseoFhL8sM63H24yhMKt0Fc kahtFByvR+8QpFVQLoCl2T7sLVojoPCLhWqdMAY= X-Google-Smtp-Source: ABdhPJy6t72nDcA3hhFnGO09RsRc0NcYnba863lm4Cwv5JTXpJhWxh8rGh618/H2q0DdleFAUk/pSw== X-Received: by 2002:a05:651c:a12:: with SMTP id k18mr118578ljq.207.1632324303566; Wed, 22 Sep 2021 08:25:03 -0700 (PDT) Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com. [209.85.167.44]) by smtp.gmail.com with ESMTPSA id 5sm282995ljj.13.2021.09.22.08.25.03 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Sep 2021 08:25:03 -0700 (PDT) Received: by mail-lf1-f44.google.com with SMTP id e15so13164614lfr.10 for ; Wed, 22 Sep 2021 08:25:03 -0700 (PDT) X-Received: by 2002:a2e:5815:: with SMTP id m21mr126446ljb.95.1632324177022; Wed, 22 Sep 2021 08:22:57 -0700 (PDT) MIME-Version: 1.0 References: <20210903160302.yh42vpkuob45dbpb@gilmour> <20210904091050.g5axxctgelciihjn@gilmour> <20210920144730.d7oabqfbx7pmyyfb@gilmour> <20210920154333.vunyxeshdb7jt5ka@gilmour> <20210920155350.h6624mt65vwg72p2@gilmour> <20210920171042.oq3ndp3ox4xv5odh@gilmour> <20210922095725.dk4vk42zb3kh7y6s@gilmour> In-Reply-To: From: Linus Torvalds Date: Wed, 22 Sep 2021 08:22:40 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Regression with mainline kernel on rpi4 To: Sudip Mukherjee Cc: Maxime Ripard , Emma Anholt , David Airlie , Daniel Vetter , Philipp Zabel , dri-devel , linux-kernel Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Sep 22, 2021 at 3:11 AM Sudip Mukherjee wrote: > > That test script is triggering the openqa job, but its running only > after lava is able to login. The trace is appearing before the login > prompt even, so test_mainline.sh should not matter here. Side note: the traces might be more legible if you have debug info in the kernel, and run the dmesg through the script in scripts/decode_stacktrace.sh which should give line numbers and inlining information. That often makes it much easier to see which access it is that hits a NULL pointer dereference. On x86-64, generally just decode the instruction stream, and look at the instruction patterns and try to figure out where an oops is coming from, but that's much less useful on arm64 (partly because I'm not as used to it, but because the arm64 oopses don't print out much of the instructions so there's often little to go by). Linus From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3F89C433F5 for ; Wed, 22 Sep 2021 15:26:34 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A150560F21 for ; Wed, 22 Sep 2021 15:26:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A150560F21 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E08316EC1B; Wed, 22 Sep 2021 15:26:33 +0000 (UTC) Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com [IPv6:2a00:1450:4864:20::130]) by gabe.freedesktop.org (Postfix) with ESMTPS id D79C46EC1B for ; Wed, 22 Sep 2021 15:26:32 +0000 (UTC) Received: by mail-lf1-x130.google.com with SMTP id i4so13375823lfv.4 for ; Wed, 22 Sep 2021 08:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sVblMwcY20Jt1iJwJZzOQQzmXaPjWYdpKIFg44drlAI=; b=Iv8Fu1IRNg70EMHGzFeMTd8j/ThtaiAem9UrK07EGRyvDmdYcny/SyCehISsXmkqpx l92lTNBSyBbDXYg1Yhc6RFGTAV9NS5ml7ZmwcQJaPFjWpMPUte3T+pogP3++9F+/Z1A/ kzeL2ruJbqjYn3yHBIgPlbAR8WOp/hvzXBCGw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sVblMwcY20Jt1iJwJZzOQQzmXaPjWYdpKIFg44drlAI=; b=SF34WPH7sSOJCqFY2FEMthPJN2mFZK6WOPkMNToTfthEHJdkLsZ+JvFc92Yc9ZVucr j/5UDdjEa3AAXRia6KHnKPafMzB/FXa+PVlskLxNr4UGg2PnO0hZyLxtIBN+xZkO5VpG WVRzFPP3dck0XH26l5WBaX7aQXoUztWsQSQmTWa5UbG3Bmn9XuYoG0aidAEumEwqO67d kMexkzd1sD/hEWU81Bdt9GjkmK3azi/mwSKYM4rm0eNU17HBUx2UmQxJHesI1p67jVc8 qRtpX9YfgmwJ4Ddxt/+07VVoujhzO9tZV8GLOfUOrXOgUu1h1Ht/W/l+mC9Q/22CphrM FV3Q== X-Gm-Message-State: AOAM530kh49Jr73gB6A3ONd5oqvQtAnPES+C599pbSt6M7jqwUJ/3mZS GJ02B+3t/6X/ptEXKmA3E70txv62ZqLypF5UP0A= X-Google-Smtp-Source: ABdhPJwlUzvBRY4x82HlIDfuuh9lefwdvrj1cz33vB3KduleXtGkQMHV+KpRp6rjgLE9WQw5+MzWLQ== X-Received: by 2002:ac2:4c47:: with SMTP id o7mr214495lfk.202.1632324279030; Wed, 22 Sep 2021 08:24:39 -0700 (PDT) Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com. [209.85.167.44]) by smtp.gmail.com with ESMTPSA id a11sm199823lfj.166.2021.09.22.08.24.38 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Sep 2021 08:24:38 -0700 (PDT) Received: by mail-lf1-f44.google.com with SMTP id i25so13458637lfg.6 for ; Wed, 22 Sep 2021 08:24:38 -0700 (PDT) X-Received: by 2002:a2e:5815:: with SMTP id m21mr126446ljb.95.1632324177022; Wed, 22 Sep 2021 08:22:57 -0700 (PDT) MIME-Version: 1.0 References: <20210903160302.yh42vpkuob45dbpb@gilmour> <20210904091050.g5axxctgelciihjn@gilmour> <20210920144730.d7oabqfbx7pmyyfb@gilmour> <20210920154333.vunyxeshdb7jt5ka@gilmour> <20210920155350.h6624mt65vwg72p2@gilmour> <20210920171042.oq3ndp3ox4xv5odh@gilmour> <20210922095725.dk4vk42zb3kh7y6s@gilmour> In-Reply-To: From: Linus Torvalds Date: Wed, 22 Sep 2021 08:22:40 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Regression with mainline kernel on rpi4 To: Sudip Mukherjee Cc: Maxime Ripard , Emma Anholt , David Airlie , Daniel Vetter , Philipp Zabel , dri-devel , linux-kernel Content-Type: text/plain; charset="UTF-8" X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Wed, Sep 22, 2021 at 3:11 AM Sudip Mukherjee wrote: > > That test script is triggering the openqa job, but its running only > after lava is able to login. The trace is appearing before the login > prompt even, so test_mainline.sh should not matter here. Side note: the traces might be more legible if you have debug info in the kernel, and run the dmesg through the script in scripts/decode_stacktrace.sh which should give line numbers and inlining information. That often makes it much easier to see which access it is that hits a NULL pointer dereference. On x86-64, generally just decode the instruction stream, and look at the instruction patterns and try to figure out where an oops is coming from, but that's much less useful on arm64 (partly because I'm not as used to it, but because the arm64 oopses don't print out much of the instructions so there's often little to go by). Linus