Fwd: [Bug 1854502] New: Noticeably Increased CPU temperature

Fwd: [Bug 1854502] New: Noticeably Increased CPU temperature

[Hans de Goede]

Hi Rafael,

I got a copy of the below bugreport in my mailbox and I
was wondering if this rings any bells.

Are there any relevant power-management / scheduler
changes in 5.7 (vs 5.6) which might be related? And
is there any quick way for the reporter to test this
(by e.g. picking a different scheduler / cpufreq-governor).

Regards,

Hans


-------- Forwarded Message --------
Subject: [Bug 1854502] New: Noticeably Increased CPU temperature
Date: Tue, 07 Jul 2020 15:23:38 +0000
From: bugzilla@redhat.com
To: hdegoede@redhat.com

https://bugzilla.redhat.com/show_bug.cgi?id=1854502

             Bug ID: 1854502
            Summary: Noticeably Increased CPU temperature
            Product: Fedora
            Version: 32
           Hardware: x86_64
                 OS: Linux
             Status: NEW
          Component: kernel
           Severity: medium
           Assignee: kernel-maint@redhat.com
           Reporter: aria.aghazade@gmail.com
         QA Contact: extras-qa@fedoraproject.org
                 CC: acaringi@redhat.com, airlied@redhat.com,
                     bskeggs@redhat.com, hdegoede@redhat.com,
                     ichavero@redhat.com, itamar@ispbrasil.com.br,
                     jarodwilson@gmail.com, jeremy@jcline.org,
                     jglisse@redhat.com, john.j5live@gmail.com,
                     jonathan@jonmasters.org, josef@toxicpanda.com,
                     kernel-maint@redhat.com, lgoncalv@redhat.com,
                     linville@redhat.com, masami256@gmail.com,
                     mchehab@infradead.org, mjg59@srcf.ucam.org,
                     steved@redhat.com
   Target Milestone: ---
     Classification: Fedora



First things first, I'm running Fedora 32 with latest packages on intel 6700hq.
With latest kernel-5.7, cpu temperatures are noticeably higher.
Using kernel-5.6 my idle cpu temperature is around 35-40°C, but with kernel-5.7
its around 50-55. Htop and KDE System Monitor don't show anything using cpu and
idle cpu usage on both kernels is 0-2%.

I first noticed this issue on kernel-5.7.6-201.fc32.x86_64, and it is also
there with kernel-5.7.7-200.fc32.x86_64.

Booting with kernel-5.6.19-300.fc32.x86_64 fixes the issue.


Are you running any modules that not shipped with directly Fedora's kernel? No.


-- 
You are receiving this mail because:
You are on the CC list for the bug.

Re: [Bug 1854502] New: Noticeably Increased CPU temperature

[Rafael J. Wysocki]

Hi Hans,

On Tue, Jul 7, 2020 at 5:32 PM Hans de Goede <hdegoede@redhat.com> wrote:
>
> Hi Rafael,
>
> I got a copy of the below bugreport in my mailbox and I
> was wondering if this rings any bells.
>
> Are there any relevant power-management / scheduler
> changes in 5.7 (vs 5.6) which might be related?

There were a few, but I'm not sure how they can cause thermal issues to happen.

> And is there any quick way for the reporter to test this
> (by e.g. picking a different scheduler / cpufreq-governor).

Well, as usual, I would recommend looking for possible configuration
differences between the good and bad cases, ie. different default
governors, different cpufreq/cpuidle drivers etc.

Also, if the system is Intel-based, run turbostat to collect CPU PM
information for both the good and bad cases.

Cheers,
Rafael

> -------- Forwarded Message --------
> Subject: [Bug 1854502] New: Noticeably Increased CPU temperature
> Date: Tue, 07 Jul 2020 15:23:38 +0000
> From: bugzilla@redhat.com
> To: hdegoede@redhat.com
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1854502
>
>              Bug ID: 1854502
>             Summary: Noticeably Increased CPU temperature
>             Product: Fedora
>             Version: 32
>            Hardware: x86_64
>                  OS: Linux
>              Status: NEW
>           Component: kernel
>            Severity: medium
>            Assignee: kernel-maint@redhat.com
>            Reporter: aria.aghazade@gmail.com
>          QA Contact: extras-qa@fedoraproject.org
>                  CC: acaringi@redhat.com, airlied@redhat.com,
>                      bskeggs@redhat.com, hdegoede@redhat.com,
>                      ichavero@redhat.com, itamar@ispbrasil.com.br,
>                      jarodwilson@gmail.com, jeremy@jcline.org,
>                      jglisse@redhat.com, john.j5live@gmail.com,
>                      jonathan@jonmasters.org, josef@toxicpanda.com,
>                      kernel-maint@redhat.com, lgoncalv@redhat.com,
>                      linville@redhat.com, masami256@gmail.com,
>                      mchehab@infradead.org, mjg59@srcf.ucam.org,
>                      steved@redhat.com
>    Target Milestone: ---
>      Classification: Fedora
>
>
>
> First things first, I'm running Fedora 32 with latest packages on intel 6700hq.
> With latest kernel-5.7, cpu temperatures are noticeably higher.
> Using kernel-5.6 my idle cpu temperature is around 35-40°C, but with kernel-5.7
> its around 50-55. Htop and KDE System Monitor don't show anything using cpu and
> idle cpu usage on both kernels is 0-2%.
>
> I first noticed this issue on kernel-5.7.6-201.fc32.x86_64, and it is also
> there with kernel-5.7.7-200.fc32.x86_64.
>
> Booting with kernel-5.6.19-300.fc32.x86_64 fixes the issue.
>
>
> Are you running any modules that not shipped with directly Fedora's kernel? No.

Re: [Bug 1854502] New: Noticeably Increased CPU temperature

[Hans de Goede]

Hi,

On 7/7/20 5:43 PM, Rafael J. Wysocki wrote:
> Hi Hans,
> 
> On Tue, Jul 7, 2020 at 5:32 PM Hans de Goede <hdegoede@redhat.com> wrote:
>>
>> Hi Rafael,
>>
>> I got a copy of the below bugreport in my mailbox and I
>> was wondering if this rings any bells.
>>
>> Are there any relevant power-management / scheduler
>> changes in 5.7 (vs 5.6) which might be related?
> 
> There were a few, but I'm not sure how they can cause thermal issues to happen.
> 
>> And is there any quick way for the reporter to test this
>> (by e.g. picking a different scheduler / cpufreq-governor).
> 
> Well, as usual, I would recommend looking for possible configuration
> differences between the good and bad cases, ie. different default
> governors, different cpufreq/cpuidle drivers etc.
> 
> Also, if the system is Intel-based, run turbostat to collect CPU PM
> information for both the good and bad cases.

Ok I've asked the user to run some tests and provide powertop
and turbostat outputs for both the cold and hot cases.

I will follow-up on this when I know more.

Regards,

Hans


>> -------- Forwarded Message --------
>> Subject: [Bug 1854502] New: Noticeably Increased CPU temperature
>> Date: Tue, 07 Jul 2020 15:23:38 +0000
>> From: bugzilla@redhat.com
>> To: hdegoede@redhat.com
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=1854502
>>
>>               Bug ID: 1854502
>>              Summary: Noticeably Increased CPU temperature
>>              Product: Fedora
>>              Version: 32
>>             Hardware: x86_64
>>                   OS: Linux
>>               Status: NEW
>>            Component: kernel
>>             Severity: medium
>>             Assignee: kernel-maint@redhat.com
>>             Reporter: aria.aghazade@gmail.com
>>           QA Contact: extras-qa@fedoraproject.org
>>                   CC: acaringi@redhat.com, airlied@redhat.com,
>>                       bskeggs@redhat.com, hdegoede@redhat.com,
>>                       ichavero@redhat.com, itamar@ispbrasil.com.br,
>>                       jarodwilson@gmail.com, jeremy@jcline.org,
>>                       jglisse@redhat.com, john.j5live@gmail.com,
>>                       jonathan@jonmasters.org, josef@toxicpanda.com,
>>                       kernel-maint@redhat.com, lgoncalv@redhat.com,
>>                       linville@redhat.com, masami256@gmail.com,
>>                       mchehab@infradead.org, mjg59@srcf.ucam.org,
>>                       steved@redhat.com
>>     Target Milestone: ---
>>       Classification: Fedora
>>
>>
>>
>> First things first, I'm running Fedora 32 with latest packages on intel 6700hq.
>> With latest kernel-5.7, cpu temperatures are noticeably higher.
>> Using kernel-5.6 my idle cpu temperature is around 35-40°C, but with kernel-5.7
>> its around 50-55. Htop and KDE System Monitor don't show anything using cpu and
>> idle cpu usage on both kernels is 0-2%.
>>
>> I first noticed this issue on kernel-5.7.6-201.fc32.x86_64, and it is also
>> there with kernel-5.7.7-200.fc32.x86_64.
>>
>> Booting with kernel-5.6.19-300.fc32.x86_64 fixes the issue.
>>
>>
>> Are you running any modules that not shipped with directly Fedora's kernel? No.
> 

Re: [Bug 1854502] New: Noticeably Increased CPU temperature

[Hans de Goede]

Hi,

On 7/9/20 2:31 PM, Hans de Goede wrote:
> Hi,
> 
> On 7/7/20 5:43 PM, Rafael J. Wysocki wrote:
>> Hi Hans,
>>
>> On Tue, Jul 7, 2020 at 5:32 PM Hans de Goede <hdegoede@redhat.com> wrote:
>>>
>>> Hi Rafael,
>>>
>>> I got a copy of the below bugreport in my mailbox and I
>>> was wondering if this rings any bells.
>>>
>>> Are there any relevant power-management / scheduler
>>> changes in 5.7 (vs 5.6) which might be related?
>>
>> There were a few, but I'm not sure how they can cause thermal issues to happen.
>>
>>> And is there any quick way for the reporter to test this
>>> (by e.g. picking a different scheduler / cpufreq-governor).
>>
>> Well, as usual, I would recommend looking for possible configuration
>> differences between the good and bad cases, ie. different default
>> governors, different cpufreq/cpuidle drivers etc.
>>
>> Also, if the system is Intel-based, run turbostat to collect CPU PM
>> information for both the good and bad cases.
> 
> Ok I've asked the user to run some tests and provide powertop
> and turbostat outputs for both the cold and hot cases.
> 
> I will follow-up on this when I know more.

Ok, I've asked the user to gather the necessary info. He has collected
screenshots from both powertop and turbostat under both 5.6 (with the
CPU staying cool) and with 5.7 (where the CPU runs a bit hot) both with
the system fully idle.

The screenshots are attached to the bug as a tarbal:

https://bugzilla.redhat.com/show_bug.cgi?id=1854502

I've never looked at turbostat output before so I do not know what
I'm looking for there. But the powertop output does show a significant
difference.

On the powertop overview page we can see that 5.7 has slightly less
wakeup then 5.6 (probably just normal variation) but despite this,
the "idle stats" page shows that under 5.6 we get 90/91% C7 residency,
where as 5.7 only reaches 85%, that seems like a significant difference
to me. IOW the CPU is in a higher powerstae (consuming more power) for
15% of the time with 5.6, vs 10% with 5.7.

Note that this users system is somewhat customized, he is using
bumblebee with the nvidia binary driver to run games, while
using the i915 gfx for his normal desktop stuff. Doing his
tests no apps are running using the nvidia GPU and in both
cases the nvidia GPU is powered-down (according to the /proc/...
info).

He is also using TLP and some utility to undervolt the GPU. None
of this seems to explain the C7 residency difference though, so
despite his non-standard setup it would be good (IMHO) to still
see if we can try to root-cause that.

Regards,

Hans






>>> -------- Forwarded Message --------
>>> Subject: [Bug 1854502] New: Noticeably Increased CPU temperature
>>> Date: Tue, 07 Jul 2020 15:23:38 +0000
>>> From: bugzilla@redhat.com
>>> To: hdegoede@redhat.com
>>>
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1854502
>>>
>>>               Bug ID: 1854502
>>>              Summary: Noticeably Increased CPU temperature
>>>              Product: Fedora
>>>              Version: 32
>>>             Hardware: x86_64
>>>                   OS: Linux
>>>               Status: NEW
>>>            Component: kernel
>>>             Severity: medium
>>>             Assignee: kernel-maint@redhat.com
>>>             Reporter: aria.aghazade@gmail.com
>>>           QA Contact: extras-qa@fedoraproject.org
>>>                   CC: acaringi@redhat.com, airlied@redhat.com,
>>>                       bskeggs@redhat.com, hdegoede@redhat.com,
>>>                       ichavero@redhat.com, itamar@ispbrasil.com.br,
>>>                       jarodwilson@gmail.com, jeremy@jcline.org,
>>>                       jglisse@redhat.com, john.j5live@gmail.com,
>>>                       jonathan@jonmasters.org, josef@toxicpanda.com,
>>>                       kernel-maint@redhat.com, lgoncalv@redhat.com,
>>>                       linville@redhat.com, masami256@gmail.com,
>>>                       mchehab@infradead.org, mjg59@srcf.ucam.org,
>>>                       steved@redhat.com
>>>     Target Milestone: ---
>>>       Classification: Fedora
>>>
>>>
>>>
>>> First things first, I'm running Fedora 32 with latest packages on intel 6700hq.
>>> With latest kernel-5.7, cpu temperatures are noticeably higher.
>>> Using kernel-5.6 my idle cpu temperature is around 35-40°C, but with kernel-5.7
>>> its around 50-55. Htop and KDE System Monitor don't show anything using cpu and
>>> idle cpu usage on both kernels is 0-2%.
>>>
>>> I first noticed this issue on kernel-5.7.6-201.fc32.x86_64, and it is also
>>> there with kernel-5.7.7-200.fc32.x86_64.
>>>
>>> Booting with kernel-5.6.19-300.fc32.x86_64 fixes the issue.
>>>
>>>
>>> Are you running any modules that not shipped with directly Fedora's kernel? No.
>>

Re: [Bug 1854502] New: Noticeably Increased CPU temperature

[Hans de Goede]

Hi,

On 7/10/20 2:35 PM, Hans de Goede wrote:
> Hi,
> 
> On 7/9/20 2:31 PM, Hans de Goede wrote:
>> Hi,
>>
>> On 7/7/20 5:43 PM, Rafael J. Wysocki wrote:
>>> Hi Hans,
>>>
>>> On Tue, Jul 7, 2020 at 5:32 PM Hans de Goede <hdegoede@redhat.com> wrote:
>>>>
>>>> Hi Rafael,
>>>>
>>>> I got a copy of the below bugreport in my mailbox and I
>>>> was wondering if this rings any bells.
>>>>
>>>> Are there any relevant power-management / scheduler
>>>> changes in 5.7 (vs 5.6) which might be related?
>>>
>>> There were a few, but I'm not sure how they can cause thermal issues to happen.
>>>
>>>> And is there any quick way for the reporter to test this
>>>> (by e.g. picking a different scheduler / cpufreq-governor).
>>>
>>> Well, as usual, I would recommend looking for possible configuration
>>> differences between the good and bad cases, ie. different default
>>> governors, different cpufreq/cpuidle drivers etc.
>>>
>>> Also, if the system is Intel-based, run turbostat to collect CPU PM
>>> information for both the good and bad cases.
>>
>> Ok I've asked the user to run some tests and provide powertop
>> and turbostat outputs for both the cold and hot cases.
>>
>> I will follow-up on this when I know more.
> 
> Ok, I've asked the user to gather the necessary info. He has collected
> screenshots from both powertop and turbostat under both 5.6 (with the
> CPU staying cool) and with 5.7 (where the CPU runs a bit hot) both with
> the system fully idle.
> 
> The screenshots are attached to the bug as a tarbal:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1854502
> 
> I've never looked at turbostat output before so I do not know what
> I'm looking for there. But the powertop output does show a significant
> difference.
> 
> On the powertop overview page we can see that 5.7 has slightly less
> wakeup then 5.6 (probably just normal variation) but despite this,
> the "idle stats" page shows that under 5.6 we get 90/91% C7 residency,
> where as 5.7 only reaches 85%, that seems like a significant difference
> to me. IOW the CPU is in a higher powerstae (consuming more power) for
> 15% of the time with 5.6, vs 10% with 5.7.
> 
> Note that this users system is somewhat customized, he is using
> bumblebee with the nvidia binary driver to run games, while
> using the i915 gfx for his normal desktop stuff. Doing his
> tests no apps are running using the nvidia GPU and in both
> cases the nvidia GPU is powered-down (according to the /proc/...
> info).
> 
> He is also using TLP and some utility to undervolt the GPU. None
> of this seems to explain the C7 residency difference though, so
> despite his non-standard setup it would be good (IMHO) to still
> see if we can try to root-cause that.

The user just reported in bugzilla that the issue is gone with
5.7.8, not sure what fixed it. But now there is no need to look into
this anymore.

Regards,

Hans