From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.4 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEE7EC433E0 for ; Thu, 4 Feb 2021 13:41:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 873E864DBA for ; Thu, 4 Feb 2021 13:41:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236394AbhBDNlX (ORCPT ); Thu, 4 Feb 2021 08:41:23 -0500 Received: from foss.arm.com ([217.140.110.172]:58592 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236358AbhBDNju (ORCPT ); Thu, 4 Feb 2021 08:39:50 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E4035D6E; Thu, 4 Feb 2021 05:39:02 -0800 (PST) Received: from [10.57.49.26] (unknown [10.57.49.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B90333F694; Thu, 4 Feb 2021 05:39:01 -0800 (PST) Subject: Re: [PATCH] drm/lima: Use delayed timer as default in devfreq profile To: Qiang Yu , Lukasz Luba Cc: lima@lists.freedesktop.org, David Airlie , Christian Hewitt , Linux Kernel Mailing List , dri-devel References: <20210127105121.20345-1-lukasz.luba@arm.com> <3d1b4696-0172-f88a-f41f-c66ac3baa429@arm.com> From: Robin Murphy Message-ID: <0afa6299-1c35-ab98-702e-8dcd168bcaac@arm.com> Date: Thu, 4 Feb 2021 13:39:00 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2021-02-03 02:01, Qiang Yu wrote: > On Tue, Feb 2, 2021 at 10:02 PM Lukasz Luba wrote: >> >> >> >> On 2/2/21 1:01 AM, Qiang Yu wrote: >>> Hi Lukasz, >>> >>> Thanks for the explanation. So the deferred timer option makes a mistake that >>> when GPU goes from idle to busy for only one poll periodic, in this >>> case 50ms, right? >> >> Not exactly. Driver sets the polling interval to 50ms (in this case) >> because it needs ~3-frame average load (in 60fps). I have discovered the >> issue quite recently that on systems with 2 CPUs or more, the devfreq >> core is not monitoring the devices even for seconds. Therefore, we might >> end up with quite big amount of work that GPU is doing, but we don't >> know about it. Devfreq core didn't check <- timer didn't fired. Then >> suddenly that CPU, which had the deferred timer registered last time, >> is waking up and timer triggers to check our device. We get the stats, >> but they might be showing load from 1sec not 50ms. We feed them into >> governor. Governor sees the new load, but was tested and configured for >> 50ms, so it might try to rise the frequency to max. The GPU work might >> be already lower and there is no need for such freq. Then the CPU goes >> idle again, so no devfreq core check for next e.g. 1sec, but the >> frequency stays at max OPP and we burn power. >> >> So, it's completely unreliable. We might stuck at min frequency and >> suffer the frame drops, or sometimes stuck to max freq and burn more >> power when there is no such need. >> >> Similar for thermal governor, which is confused by this old stats and >> long period stats, longer than 50ms. >> >> Stats from last e.g. ~1sec tells you nothing about real recent GPU >> workload. > Oh, right, I missed this case. > >> >>> But delayed timer will wakeup CPU every 50ms even when system is idle, will this >>> cause more power consumption for the case like phone suspend? >> >> No, in case of phone suspend it won't increase the power consumption. >> The device won't be woken up, it will stay in suspend. > I mean the CPU is waked up frequently by timer when phone suspend, > not the whole device (like the display). > > Seems it's better to have deferred timer when device is suspended for > power saving, > and delayed timer when device in working state. User knows this and > can use sysfs > to change it. Doesn't devfreq_suspend_device() already cancel any timer work either way in that case? Robin. > Set the delayed timer as default is reasonable, so patch is: > Reviewed-by: Qiang Yu > > Regards, > Qiang > >> >> Regards, >> Lukasz >> >> >>> >>> Regards, >>> Qiang >>> >>> >>> On Mon, Feb 1, 2021 at 5:53 PM Lukasz Luba wrote: >>>> >>>> Hi Qiang, >>>> >>>> On 1/30/21 1:51 PM, Qiang Yu wrote: >>>>> Thanks for the patch. But I can't observe any difference on glmark2 >>>>> with or without this patch. >>>>> Maybe you can provide other test which can benefit from it. >>>> >>>> This is a design problem and has impact on the whole system. >>>> There is a few issues. When the device is not checked and there are >>>> long delays between last check and current, the history is broken. >>>> It confuses the devfreq governor and thermal governor (Intelligent Power >>>> Allocation (IPA)). Thermal governor works on stale stats data and makes >>>> stupid decisions, because there is no new stats (device not checked). >>>> Similar applies to devfreq simple_ondemand governor, where it 'tires' to >>>> work on a loooong period even 3sec and make prediction for the next >>>> frequency based on it (which is broken). >>>> >>>> How it should be done: constant reliable check is needed, then: >>>> - period is guaranteed and has fixed size, e.g 50ms or 100ms. >>>> - device status is quite recent so thermal devfreq cooling provides >>>> 'fresh' data into thermal governor >>>> >>>> This would prevent odd behavior and solve the broken cases. >>>> >>>>> >>>>> Considering it will wake up CPU more frequently, and user may choose >>>>> to change this by sysfs, >>>>> I'd like to not apply it. >>>> >>>> The deferred timer for GPU is wrong option, for UFS or eMMC makes more >>>> sense. It's also not recommended for NoC busses. I've discovered that >>>> some time ago and proposed to have option to switch into delayed timer. >>>> Trust me, it wasn't obvious to find out that this missing check has >>>> those impacts. So the other engineers or users might not know that some >>>> problems they faces (especially when the device load is changing) is due >>>> to this delayed vs deffered timer and they will change it in the sysfs. >>>> >>>> Regards, >>>> Lukasz >>>> >>>>> >>>>> Regards, >>>>> Qiang >>>>> > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.4 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B2D9C433E0 for ; Thu, 4 Feb 2021 13:39:05 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2820D64F10 for ; Thu, 4 Feb 2021 13:39:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2820D64F10 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 67AFD6E0AB; Thu, 4 Feb 2021 13:39:04 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by gabe.freedesktop.org (Postfix) with ESMTP id 735CF6E07F; Thu, 4 Feb 2021 13:39:03 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E4035D6E; Thu, 4 Feb 2021 05:39:02 -0800 (PST) Received: from [10.57.49.26] (unknown [10.57.49.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B90333F694; Thu, 4 Feb 2021 05:39:01 -0800 (PST) Subject: Re: [PATCH] drm/lima: Use delayed timer as default in devfreq profile To: Qiang Yu , Lukasz Luba References: <20210127105121.20345-1-lukasz.luba@arm.com> <3d1b4696-0172-f88a-f41f-c66ac3baa429@arm.com> From: Robin Murphy Message-ID: <0afa6299-1c35-ab98-702e-8dcd168bcaac@arm.com> Date: Thu, 4 Feb 2021 13:39:00 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Airlie , dri-devel , Christian Hewitt , Linux Kernel Mailing List , lima@lists.freedesktop.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On 2021-02-03 02:01, Qiang Yu wrote: > On Tue, Feb 2, 2021 at 10:02 PM Lukasz Luba wrote: >> >> >> >> On 2/2/21 1:01 AM, Qiang Yu wrote: >>> Hi Lukasz, >>> >>> Thanks for the explanation. So the deferred timer option makes a mistake that >>> when GPU goes from idle to busy for only one poll periodic, in this >>> case 50ms, right? >> >> Not exactly. Driver sets the polling interval to 50ms (in this case) >> because it needs ~3-frame average load (in 60fps). I have discovered the >> issue quite recently that on systems with 2 CPUs or more, the devfreq >> core is not monitoring the devices even for seconds. Therefore, we might >> end up with quite big amount of work that GPU is doing, but we don't >> know about it. Devfreq core didn't check <- timer didn't fired. Then >> suddenly that CPU, which had the deferred timer registered last time, >> is waking up and timer triggers to check our device. We get the stats, >> but they might be showing load from 1sec not 50ms. We feed them into >> governor. Governor sees the new load, but was tested and configured for >> 50ms, so it might try to rise the frequency to max. The GPU work might >> be already lower and there is no need for such freq. Then the CPU goes >> idle again, so no devfreq core check for next e.g. 1sec, but the >> frequency stays at max OPP and we burn power. >> >> So, it's completely unreliable. We might stuck at min frequency and >> suffer the frame drops, or sometimes stuck to max freq and burn more >> power when there is no such need. >> >> Similar for thermal governor, which is confused by this old stats and >> long period stats, longer than 50ms. >> >> Stats from last e.g. ~1sec tells you nothing about real recent GPU >> workload. > Oh, right, I missed this case. > >> >>> But delayed timer will wakeup CPU every 50ms even when system is idle, will this >>> cause more power consumption for the case like phone suspend? >> >> No, in case of phone suspend it won't increase the power consumption. >> The device won't be woken up, it will stay in suspend. > I mean the CPU is waked up frequently by timer when phone suspend, > not the whole device (like the display). > > Seems it's better to have deferred timer when device is suspended for > power saving, > and delayed timer when device in working state. User knows this and > can use sysfs > to change it. Doesn't devfreq_suspend_device() already cancel any timer work either way in that case? Robin. > Set the delayed timer as default is reasonable, so patch is: > Reviewed-by: Qiang Yu > > Regards, > Qiang > >> >> Regards, >> Lukasz >> >> >>> >>> Regards, >>> Qiang >>> >>> >>> On Mon, Feb 1, 2021 at 5:53 PM Lukasz Luba wrote: >>>> >>>> Hi Qiang, >>>> >>>> On 1/30/21 1:51 PM, Qiang Yu wrote: >>>>> Thanks for the patch. But I can't observe any difference on glmark2 >>>>> with or without this patch. >>>>> Maybe you can provide other test which can benefit from it. >>>> >>>> This is a design problem and has impact on the whole system. >>>> There is a few issues. When the device is not checked and there are >>>> long delays between last check and current, the history is broken. >>>> It confuses the devfreq governor and thermal governor (Intelligent Power >>>> Allocation (IPA)). Thermal governor works on stale stats data and makes >>>> stupid decisions, because there is no new stats (device not checked). >>>> Similar applies to devfreq simple_ondemand governor, where it 'tires' to >>>> work on a loooong period even 3sec and make prediction for the next >>>> frequency based on it (which is broken). >>>> >>>> How it should be done: constant reliable check is needed, then: >>>> - period is guaranteed and has fixed size, e.g 50ms or 100ms. >>>> - device status is quite recent so thermal devfreq cooling provides >>>> 'fresh' data into thermal governor >>>> >>>> This would prevent odd behavior and solve the broken cases. >>>> >>>>> >>>>> Considering it will wake up CPU more frequently, and user may choose >>>>> to change this by sysfs, >>>>> I'd like to not apply it. >>>> >>>> The deferred timer for GPU is wrong option, for UFS or eMMC makes more >>>> sense. It's also not recommended for NoC busses. I've discovered that >>>> some time ago and proposed to have option to switch into delayed timer. >>>> Trust me, it wasn't obvious to find out that this missing check has >>>> those impacts. So the other engineers or users might not know that some >>>> problems they faces (especially when the device load is changing) is due >>>> to this delayed vs deffered timer and they will change it in the sysfs. >>>> >>>> Regards, >>>> Lukasz >>>> >>>>> >>>>> Regards, >>>>> Qiang >>>>> > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel > _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel