From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8758FC4332F for ; Mon, 6 Sep 2021 08:05:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5FA8760F43 for ; Mon, 6 Sep 2021 08:05:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240500AbhIFIGG (ORCPT ); Mon, 6 Sep 2021 04:06:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57586 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240513AbhIFIEX (ORCPT ); Mon, 6 Sep 2021 04:04:23 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD161C0617A8 for ; Mon, 6 Sep 2021 01:02:33 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id b7so7629402iob.4 for ; Mon, 06 Sep 2021 01:02:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WZ2JrZUxjomDnhpkXV3UkUZoXyQGVexwecJgXeES0JE=; b=RkEBMsUcqXWVba6jg6GNyX1QSg9Z2w6C+7dlrHeO/RDBP0nWKvHew1z1Sbhi1QwMeC 4SxnoCW+llDiw/b0TnXGbaw1wXwgwmiSVC+inI6f1Frkaln1CFkeK2/bMtpAGA40kC/T TedqOSDulzE/wi0uLU5gVLrO6XNn7XFwm4cA8iO+klEcs58lR4MXXTLwxIcP5QwWTjmb XvRFXqtzcirpnP8WzMxsBgCBPDB1LQ97mUcFHiGJnV2pWnUvbXCBMPSH21UcZgErHv/q le/eKFXaASV2+709G+8GA7ECFAdIJumjFUUYek5/Fl1I09oftcDCmuoxJs7kNiwx+1Gi 18nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WZ2JrZUxjomDnhpkXV3UkUZoXyQGVexwecJgXeES0JE=; b=TbcEHJ2NYsLGVSElBwujgl5qBMqlfs8n4/lEmum42lgaXZniGyKm4di2BJj/1Sy8pr /mrpS8kW2IKJm5fXeVqvuWRSBPjgzJb+yGeYnml4ItMBFgQ8KXcy46Ml3B7HVJRvWPhg YqZuBnbHyMLUUCxgOTuHF9QaGKM4N4iOzptwvLw9hF+kJAFAwMKZfoKngrkNCnb3cll7 YFZd0S9OnQa5/ohxUTl+tBqhxFOb98rb0xXpa4yEGcxAGOzWlvYslaUN+qAWoLmkWier L+78G4iPhgktNiPQGF7vasarCChR28abimcErYk+hqcW0/I0uTd2L6uXXndNVQaG+5yD 8pOg== X-Gm-Message-State: AOAM533nFPOIhRqjUSj8Re5bJmJPtYsHL1NOwn3KGP9OPrNfdIlgS+Pp flgvIJKtPKaT02GJVPVvsDfOZt5dgyzFiejtijQWCA== X-Google-Smtp-Source: ABdhPJyTvUQQpstrXnLI1vn4QAa5MprOCKyEKtqMUvuMXUR0YaUtyc1jLZkxnD+RRhbjYl/pF4jE3czU1Qh0ZEaYGtw= X-Received: by 2002:a6b:8bcf:: with SMTP id n198mr8922767iod.178.1630915353263; Mon, 06 Sep 2021 01:02:33 -0700 (PDT) MIME-Version: 1.0 References: <20210729183942.2839925-1-robdclark@gmail.com> <1a38a590-a64e-58ef-1bbf-0ae49c004d05@linaro.org> In-Reply-To: From: Amit Pundir Date: Mon, 6 Sep 2021 13:31:57 +0530 Message-ID: Subject: Re: [PATCH] drm/msm: Disable frequency clamping on a630 To: Rob Clark Cc: John Stultz , Caleb Connolly , Rob Clark , freedreno , Sai Prakash Ranjan , Jonathan Marek , David Airlie , linux-arm-msm , Sharat Masetty , Akhil P Oommen , dri-devel , Jordan Crouse , Stephen Boyd , Bjorn Andersson , Sean Paul , open list Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Sat, 4 Sept 2021 at 01:55, Rob Clark wrote: > > On Fri, Sep 3, 2021 at 12:39 PM John Stultz wrote: > > > > On Thu, Jul 29, 2021 at 1:49 PM Rob Clark wrote: > > > On Thu, Jul 29, 2021 at 1:28 PM Caleb Connolly > > > wrote: > > > > On 29/07/2021 21:24, Rob Clark wrote: > > > > > On Thu, Jul 29, 2021 at 1:06 PM Caleb Connolly > > > > > wrote: > > > > >> > > > > >> Hi Rob, > > > > >> > > > > >> I've done some more testing! It looks like before that patch ("drm/msm: Devfreq tuning") the GPU would never get above > > > > >> the second frequency in the OPP table (342MHz) (at least, not in glxgears). With the patch applied it would more > > > > >> aggressively jump up to the max frequency which seems to be unstable at the default regulator voltages. > > > > > > > > > > *ohh*, yeah, ok, that would explain it > > > > > > > > > >> Hacking the pm8005 s1 regulator (which provides VDD_GFX) up to 0.988v (instead of the stock 0.516v) makes the GPU stable > > > > >> at the higher frequencies. > > > > >> > > > > >> Applying this patch reverts the behaviour, and the GPU never goes above 342MHz in glxgears, losing ~30% performance in > > > > >> glxgear. > > > > >> > > > > >> I think (?) that enabling CPR support would be the proper solution to this - that would ensure that the regulators run > > > > >> at the voltage the hardware needs to be stable. > > > > >> > > > > >> Is hacking the voltage higher (although ideally not quite that high) an acceptable short term solution until we have > > > > >> CPR? Or would it be safer to just not make use of the higher frequencies on a630 for now? > > > > >> > > > > > > > > > > tbh, I'm not sure about the regulator stuff and CPR.. Bjorn is already > > > > > on CC and I added sboyd, maybe one of them knows better. > > > > > > > > > > In the short term, removing the higher problematic OPPs from dts might > > > > > be a better option than this patch (which I'm dropping), since there > > > > > is nothing stopping other workloads from hitting higher OPPs. > > > > Oh yeah that sounds like a more sensible workaround than mine . > > > > > > > > > > I'm slightly curious why I didn't have problems at higher OPPs on my > > > > > c630 laptop (sdm850) > > > > Perhaps you won the sillicon lottery - iirc sdm850 is binned for higher clocks as is out of the factory. > > > > > > > > Would it be best to drop the OPPs for all devices? Or just those affected? I guess it's possible another c630 might > > > > crash where yours doesn't? > > > > > > I've not heard any reports of similar issues from the handful of other > > > folks with c630's on #aarch64-laptops.. but I can't really say if that > > > is luck or not. > > > > > > Maybe just remove it for affected devices? But I'll defer to Bjorn. > > > > Just as another datapoint, I was just marveling at how suddenly smooth > > the UI was performing on db845c and Caleb pointed me at the "drm/msm: > > Devfreq tuning" patch as the likely cause of the improvement, and > > mid-discussion my board crashed into USB crash mode: > > [ 146.157696][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.163303][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.168837][ C0] adreno 5000000.gpu: RBBM | ATB bus overflow > > [ 146.174960][ C0] adreno 5000000.gpu: CP | HW fault | status=0x00000000 > > [ 146.181917][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.187547][ C0] adreno 5000000.gpu: CP illegal instruction error > > [ 146.194009][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.308909][ T9] Internal error: synchronous external abort: > > 96000010 [#1] PREEMPT SMP > > [ 146.317150][ T9] Modules linked in: > > [ 146.320941][ T9] CPU: 3 PID: 9 Comm: kworker/u16:1 Tainted: G > > W 5.14.0-mainline-06795-g42b258c2275c #24 > > [ 146.331974][ T9] Hardware name: Thundercomm Dragonboar > > Format: Log Type - Time(microsec) - Message - Optional Info > > Log Type: B - Since Boot(Power On Reset), D - Delta, S - Statistic > > S - QC_IMAGE_VERSION_STRING=BOOT.XF.2.0-00371-SDM845LZB-1 > > S - IMAGE_VARIANT_STRING=SDM845LA > > S - OEM_IMAGE_VERSION_STRING=TSBJ-FA-PC-02170 > > > > So Caleb sent me to this thread. :) > > > > I'm still trying to trip it again, but it does seem like db845c is > > also seeing some stability issues with Linus' HEAD. > > > > Caleb's original pastebin seems to have expired (or at least require > some sort of ubuntu login to access).. were the crashes he was seeing > also 'AHB bus error'? I can reproduce this hard crash https://www.irccloud.com/pastebin/Cu6UJntE/ and a gpu lockup https://www.irccloud.com/pastebin/6Ryd2Pug/ at times reliably, by running antutu benchmark on pocof1. Reverting 9bc95570175a ("drm/msm: Devfreq tuning") helps and I no longer see these errors. Complete dmesg for hardcrash https://pastebin.com/raw/GLZVQFQN Regards, Amit Pundir > > If you have a reliable reproducer, I guess it would be worth seeing if > increasing the min_freq (ie. to limit how far we jump the freq in one > shot) "fixes" it? > > I guess I could check downstream kgsl to see if they were doing > something to increase freq in smaller increments.. I don't recall that > they were but it has been a while since I dug thru that code. And I > suppose downstream it could also be done in their custom tz governor. > > BR, > -R From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60221C433F5 for ; Mon, 6 Sep 2021 10:43:41 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2DCD060BD3 for ; Mon, 6 Sep 2021 10:43:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2DCD060BD3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6AF60898A4; Mon, 6 Sep 2021 10:43:40 +0000 (UTC) Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by gabe.freedesktop.org (Postfix) with ESMTPS id 051B989B20 for ; Mon, 6 Sep 2021 08:02:34 +0000 (UTC) Received: by mail-io1-xd2d.google.com with SMTP id b10so7606293ioq.9 for ; Mon, 06 Sep 2021 01:02:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WZ2JrZUxjomDnhpkXV3UkUZoXyQGVexwecJgXeES0JE=; b=RkEBMsUcqXWVba6jg6GNyX1QSg9Z2w6C+7dlrHeO/RDBP0nWKvHew1z1Sbhi1QwMeC 4SxnoCW+llDiw/b0TnXGbaw1wXwgwmiSVC+inI6f1Frkaln1CFkeK2/bMtpAGA40kC/T TedqOSDulzE/wi0uLU5gVLrO6XNn7XFwm4cA8iO+klEcs58lR4MXXTLwxIcP5QwWTjmb XvRFXqtzcirpnP8WzMxsBgCBPDB1LQ97mUcFHiGJnV2pWnUvbXCBMPSH21UcZgErHv/q le/eKFXaASV2+709G+8GA7ECFAdIJumjFUUYek5/Fl1I09oftcDCmuoxJs7kNiwx+1Gi 18nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WZ2JrZUxjomDnhpkXV3UkUZoXyQGVexwecJgXeES0JE=; b=hRdZ62TfxBDngspTJJx+Dp+dNR86QNGassx38hndng6pvnGADckbzkrXyfgcgxaWgc ucZBTTTUPOwX1WylL7kpOfBYIUZ0ikl1/OG6sODukjp6uAzpmlGLISu+XCgfgCTPfuDq mTTXvgH1/BuEIKU+o2VfVb3nueTmVtGs9yCwg9Q4TRuwmLPzSsXwovWWbf9cNlhEQtl3 tM3hVqSc8aM1kIlwc55FgM9v61ANcw1crJQnv9cYDDjfHbjj7cvcBHlaZ2vPntH0w19N lxYXOFjkjDtY3/4aXkc5vmnMr4d0B5OZ1gFvGD8AeE7EGtQhNSyZePbdAFFVTri54iIy NI0A== X-Gm-Message-State: AOAM53158Z35r4MW4JvJ/wK0Voq2vWkwDQ8GiJwAzm4rP/iMZfiPL4w+ s6f39sagnvnvMhZAv/xzHoz++2CWyWKAawnywC1/Ww== X-Google-Smtp-Source: ABdhPJyTvUQQpstrXnLI1vn4QAa5MprOCKyEKtqMUvuMXUR0YaUtyc1jLZkxnD+RRhbjYl/pF4jE3czU1Qh0ZEaYGtw= X-Received: by 2002:a6b:8bcf:: with SMTP id n198mr8922767iod.178.1630915353263; Mon, 06 Sep 2021 01:02:33 -0700 (PDT) MIME-Version: 1.0 References: <20210729183942.2839925-1-robdclark@gmail.com> <1a38a590-a64e-58ef-1bbf-0ae49c004d05@linaro.org> In-Reply-To: From: Amit Pundir Date: Mon, 6 Sep 2021 13:31:57 +0530 Message-ID: Subject: Re: [PATCH] drm/msm: Disable frequency clamping on a630 To: Rob Clark Cc: John Stultz , Caleb Connolly , Rob Clark , freedreno , Sai Prakash Ranjan , Jonathan Marek , David Airlie , linux-arm-msm , Sharat Masetty , Akhil P Oommen , dri-devel , Jordan Crouse , Stephen Boyd , Bjorn Andersson , Sean Paul , open list Content-Type: text/plain; charset="UTF-8" X-Mailman-Approved-At: Mon, 06 Sep 2021 10:43:38 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Sat, 4 Sept 2021 at 01:55, Rob Clark wrote: > > On Fri, Sep 3, 2021 at 12:39 PM John Stultz wrote: > > > > On Thu, Jul 29, 2021 at 1:49 PM Rob Clark wrote: > > > On Thu, Jul 29, 2021 at 1:28 PM Caleb Connolly > > > wrote: > > > > On 29/07/2021 21:24, Rob Clark wrote: > > > > > On Thu, Jul 29, 2021 at 1:06 PM Caleb Connolly > > > > > wrote: > > > > >> > > > > >> Hi Rob, > > > > >> > > > > >> I've done some more testing! It looks like before that patch ("drm/msm: Devfreq tuning") the GPU would never get above > > > > >> the second frequency in the OPP table (342MHz) (at least, not in glxgears). With the patch applied it would more > > > > >> aggressively jump up to the max frequency which seems to be unstable at the default regulator voltages. > > > > > > > > > > *ohh*, yeah, ok, that would explain it > > > > > > > > > >> Hacking the pm8005 s1 regulator (which provides VDD_GFX) up to 0.988v (instead of the stock 0.516v) makes the GPU stable > > > > >> at the higher frequencies. > > > > >> > > > > >> Applying this patch reverts the behaviour, and the GPU never goes above 342MHz in glxgears, losing ~30% performance in > > > > >> glxgear. > > > > >> > > > > >> I think (?) that enabling CPR support would be the proper solution to this - that would ensure that the regulators run > > > > >> at the voltage the hardware needs to be stable. > > > > >> > > > > >> Is hacking the voltage higher (although ideally not quite that high) an acceptable short term solution until we have > > > > >> CPR? Or would it be safer to just not make use of the higher frequencies on a630 for now? > > > > >> > > > > > > > > > > tbh, I'm not sure about the regulator stuff and CPR.. Bjorn is already > > > > > on CC and I added sboyd, maybe one of them knows better. > > > > > > > > > > In the short term, removing the higher problematic OPPs from dts might > > > > > be a better option than this patch (which I'm dropping), since there > > > > > is nothing stopping other workloads from hitting higher OPPs. > > > > Oh yeah that sounds like a more sensible workaround than mine . > > > > > > > > > > I'm slightly curious why I didn't have problems at higher OPPs on my > > > > > c630 laptop (sdm850) > > > > Perhaps you won the sillicon lottery - iirc sdm850 is binned for higher clocks as is out of the factory. > > > > > > > > Would it be best to drop the OPPs for all devices? Or just those affected? I guess it's possible another c630 might > > > > crash where yours doesn't? > > > > > > I've not heard any reports of similar issues from the handful of other > > > folks with c630's on #aarch64-laptops.. but I can't really say if that > > > is luck or not. > > > > > > Maybe just remove it for affected devices? But I'll defer to Bjorn. > > > > Just as another datapoint, I was just marveling at how suddenly smooth > > the UI was performing on db845c and Caleb pointed me at the "drm/msm: > > Devfreq tuning" patch as the likely cause of the improvement, and > > mid-discussion my board crashed into USB crash mode: > > [ 146.157696][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.163303][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.168837][ C0] adreno 5000000.gpu: RBBM | ATB bus overflow > > [ 146.174960][ C0] adreno 5000000.gpu: CP | HW fault | status=0x00000000 > > [ 146.181917][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.187547][ C0] adreno 5000000.gpu: CP illegal instruction error > > [ 146.194009][ C0] adreno 5000000.gpu: CP | AHB bus error > > [ 146.308909][ T9] Internal error: synchronous external abort: > > 96000010 [#1] PREEMPT SMP > > [ 146.317150][ T9] Modules linked in: > > [ 146.320941][ T9] CPU: 3 PID: 9 Comm: kworker/u16:1 Tainted: G > > W 5.14.0-mainline-06795-g42b258c2275c #24 > > [ 146.331974][ T9] Hardware name: Thundercomm Dragonboar > > Format: Log Type - Time(microsec) - Message - Optional Info > > Log Type: B - Since Boot(Power On Reset), D - Delta, S - Statistic > > S - QC_IMAGE_VERSION_STRING=BOOT.XF.2.0-00371-SDM845LZB-1 > > S - IMAGE_VARIANT_STRING=SDM845LA > > S - OEM_IMAGE_VERSION_STRING=TSBJ-FA-PC-02170 > > > > So Caleb sent me to this thread. :) > > > > I'm still trying to trip it again, but it does seem like db845c is > > also seeing some stability issues with Linus' HEAD. > > > > Caleb's original pastebin seems to have expired (or at least require > some sort of ubuntu login to access).. were the crashes he was seeing > also 'AHB bus error'? I can reproduce this hard crash https://www.irccloud.com/pastebin/Cu6UJntE/ and a gpu lockup https://www.irccloud.com/pastebin/6Ryd2Pug/ at times reliably, by running antutu benchmark on pocof1. Reverting 9bc95570175a ("drm/msm: Devfreq tuning") helps and I no longer see these errors. Complete dmesg for hardcrash https://pastebin.com/raw/GLZVQFQN Regards, Amit Pundir > > If you have a reliable reproducer, I guess it would be worth seeing if > increasing the min_freq (ie. to limit how far we jump the freq in one > shot) "fixes" it? > > I guess I could check downstream kgsl to see if they were doing > something to increase freq in smaller increments.. I don't recall that > they were but it has been a while since I dug thru that code. And I > suppose downstream it could also be done in their custom tz governor. > > BR, > -R