From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75505C433FE for ; Thu, 9 Sep 2021 19:50:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 588FB61041 for ; Thu, 9 Sep 2021 19:50:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344661AbhIITvW (ORCPT ); Thu, 9 Sep 2021 15:51:22 -0400 Received: from so254-9.mailgun.net ([198.61.254.9]:13092 "EHLO so254-9.mailgun.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237348AbhIITvV (ORCPT ); Thu, 9 Sep 2021 15:51:21 -0400 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.codeaurora.org; q=dns/txt; s=smtp; t=1631217011; h=Content-Transfer-Encoding: Content-Type: In-Reply-To: MIME-Version: Date: Message-ID: From: References: Cc: To: Subject: Sender; bh=3CekIbdYG/DWaxayOnRmOo9RT1gL6hIM3S/cqAXoD8M=; b=RzK/OTq4kF0CEXU1+yhYT8qNyd6lG9PH9L2EfygrX2K5k3W13YJc9e094WUzJl63XQQn4kIA XCX81XDHk8BOT1rUpiFFB7aGI4u4ucjPvukWvE69q7spT+8DtRzEvuDTsSZ7xhThwdfXcFMm E+jF8t79Vx7chDUgYkCd97eGlXA= X-Mailgun-Sending-Ip: 198.61.254.9 X-Mailgun-Sid: WyI1MzIzYiIsICJsaW51eC1hcm0tbXNtQHZnZXIua2VybmVsLm9yZyIsICJiZTllNGEiXQ== Received: from smtp.codeaurora.org (ec2-35-166-182-171.us-west-2.compute.amazonaws.com [35.166.182.171]) by smtp-out-n06.prod.us-east-1.postgun.com with SMTP id 613a6569266d62f83239425c (version=TLS1.2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256); Thu, 09 Sep 2021 19:50:00 GMT Sender: akhilpo=codeaurora.org@mg.codeaurora.org Received: by smtp.codeaurora.org (Postfix, from userid 1001) id F0DFAC4338F; Thu, 9 Sep 2021 19:49:59 +0000 (UTC) Received: from [192.168.1.12] (unknown [59.88.226.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: akhilpo) by smtp.codeaurora.org (Postfix) with ESMTPSA id 223BFC4338F; Thu, 9 Sep 2021 19:49:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 smtp.codeaurora.org 223BFC4338F Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; spf=fail smtp.mailfrom=codeaurora.org Subject: Re: [PATCH] drm/msm: Disable frequency clamping on a630 To: Amit Pundir , Bjorn Andersson Cc: Caleb Connolly , Rob Clark , dri-devel , freedreno , linux-arm-msm , Rob Clark , Sean Paul , David Airlie , Daniel Vetter , Jordan Crouse , Jonathan Marek , Sai Prakash Ranjan , open list , Stephen Boyd References: <8aa590be-6a9f-9343-e897-18e86ea48202@linaro.org> <6eefedb2-9e59-56d2-7703-2faf6cb0ca3a@codeaurora.org> <83ecbe74-caf0-6c42-e6f5-4887b3b534c6@linaro.org> <53d3e5b7-9dc0-a806-70e9-b9b5ff877462@codeaurora.org> From: Akhil P Oommen Message-ID: Date: Fri, 10 Sep 2021 01:19:50 +0530 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On 9/9/2021 9:42 PM, Amit Pundir wrote: > On Thu, 9 Sept 2021 at 17:47, Amit Pundir wrote: >> >> On Wed, 8 Sept 2021 at 07:50, Bjorn Andersson >> wrote: >>> >>> On Mon 09 Aug 10:26 PDT 2021, Akhil P Oommen wrote: >>> >>>> On 8/9/2021 9:48 PM, Caleb Connolly wrote: >>>>> >>>>> >>>>> On 09/08/2021 17:12, Rob Clark wrote: >>>>>> On Mon, Aug 9, 2021 at 7:52 AM Akhil P Oommen >>>>>> wrote: >>> [..] >>>>>>> I am a bit confused. We don't define a power domain for gpu in dt, >>>>>>> correct? Then what exactly set_opp do here? Do you think this usleep is >>>>>>> what is helping here somehow to mask the issue? >>>>> The power domains (for cx and gx) are defined in the GMU DT, the OPPs in >>>>> the GPU DT. For the sake of simplicity I'll refer to the lowest >>>>> frequency (257000000) and OPP level (RPMH_REGULATOR_LEVEL_LOW_SVS) as >>>>> the "min" state, and the highest frequency (710000000) and OPP level >>>>> (RPMH_REGULATOR_LEVEL_TURBO_L1) as the "max" state. These are defined in >>>>> sdm845.dtsi under the gpu node. >>>>> >>>>> The new devfreq behaviour unmasks what I think is a driver bug, it >>>>> inadvertently puts much more strain on the GPU regulators than they >>>>> usually get. With the new behaviour the GPU jumps from it's min state to >>>>> the max state and back again extremely rapidly under workloads as small >>>>> as refreshing UI. Where previously the GPU would rarely if ever go above >>>>> 342MHz when interacting with the device, it now jumps between min and >>>>> max many times per second. >>>>> >>>>> If my understanding is correct, the current implementation of the GMU >>>>> set freq is the following: >>>>> - Get OPP for frequency to set >>>>> - Push the frequency to the GMU - immediately updating the core clock >>>>> - Call dev_pm_opp_set_opp() which triggers a notify chain, this winds >>>>> up somewhere in power management code and causes the gx regulator level >>>>> to be updated >>>> >>>> Nope. dev_pm_opp_set_opp() sets the bandwidth for gpu and nothing else. We >>>> were using a different api earlier which got deprecated - >>>> dev_pm_opp_set_bw(). >>>> >>> >>> On the Lenovo Yoga C630 this is reproduced by starting alacritty and if >>> I'm lucky I managed to hit a few keys before it crashes, so I spent a >>> few hours looking into this as well... >>> >>> As you say, the dev_pm_opp_set_opp() will only cast a interconnect vote. >>> The opp-level is just there for show and isn't used by anything, at >>> least not on 845. >>> >>> Further more, I'm missing something in my tree, so the interconnect >>> doesn't hit sync_state, and as such we're not actually scaling the >>> buses. So the problem is not that Linux doesn't turn on the buses in >>> time. >>> >>> So I suspect that the "AHB bus error" isn't saying that we turned off >>> the bus, but rather that the GPU becomes unstable or something of that >>> sort. >>> >>> >>> Lastly, I reverted 9bc95570175a ("drm/msm: Devfreq tuning") and ran >>> Aquarium for 20 minutes without a problem. I then switched the gpu >>> devfreq governor to "userspace" and ran the following: >>> >>> while true; do >>> echo 257000000 > /sys/class/devfreq/5000000.gpu/userspace/set_freq >>> echo 710000000 > /sys/class/devfreq/5000000.gpu/userspace/set_freq >>> done >>> >>> It took 19 iterations of this loop to crash the GPU. >> >> Ack. With your above script, I can reproduce a crash too on db845c >> (A630) running v5.14. I didn't get any crash log though and device >> just rebooted to USB crash mode. >> >> And same crash on RB5 (A650) too https://hastebin.com/raw/ejutetuwun Are we sure this is the same issue? It could be, but I thought we were seeing a bunch of random gpu errors (which may eventually hit device crash). -Akhil > > fwiw I can't reproduce this crash on RB5 so far with v5.15-rc1 merge > window (HEAD: 477f70cd2a67) > >> >>> >>> So the problem doesn't seem to be Rob's change, it's just that prior to >>> it the chance to hitting it is way lower. Question is still what it is >>> that we're triggering. >>> >>> Regards, >>> Bjorn