* Request for assistance - excessive kworker CPU wakeups
@ 2011-12-09 21:27 David H. Durgee
2012-01-25 17:44 ` Mandeep Singh Baines
0 siblings, 1 reply; 5+ messages in thread
From: David H. Durgee @ 2011-12-09 21:27 UTC (permalink / raw)
To: linux-kernel
I recently purchased a Lenovo IdeaPad Z560, model 09143YU, and as I am
not a Windows fan I installed Linux Mint 11 Katya x64 to use instead of
the supplied W7. I was encountering a known hang and had to upgrade to
a later kernel, so I am now using the 2.6.38-11-generic #50-Ubuntu SMP
Mon Sep 12 21:17:25 UTC 2011 x86_64 kernel.
I had my first occasion to use the laptop for an extended period for the
first time over the Thanksgiving holiday and I found it needed to be
tuned. I downloaded powertop and used it to discover where my problems
were. After addressing excessive i915 interrupts due to DRI the next
most frequent cause of CPU wakeups is a kworker on the system. A search
lead to a post by Tejun, indicating the need to trace such issues.
Running the trace showed that 1933 of 2748 events were of the form:
<idle>-0 [000] 22005.355346: workqueue_queue_work: work
struct=ffff8800bb411188 function=do_dbs_timer workqueue=ffff88012b5d2c00
req_cpu=0 cpu=0
Tejun indicated that this is a workitem used by cpufreq and likely
caused by something else hitting the CPU frequently. So how do I
diagnose this further and isolate the cause for correction?
If you would like a summary of this, download this spreadsheet:
http://home.comcast.net/%7Eddurgee/Tracelog.ods
If you would like to inspect the trace log itself:
http://home.comcast.net/%7Eddurgee/tracelog.zip
Dave
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Request for assistance - excessive kworker CPU wakeups
2011-12-09 21:27 Request for assistance - excessive kworker CPU wakeups David H. Durgee
@ 2012-01-25 17:44 ` Mandeep Singh Baines
2012-01-25 18:56 ` David H. Durgee
0 siblings, 1 reply; 5+ messages in thread
From: Mandeep Singh Baines @ 2012-01-25 17:44 UTC (permalink / raw)
To: David H. Durgee; +Cc: linux-kernel
On Fri, Dec 9, 2011 at 1:27 PM, David H. Durgee <dhdurgee@verizon.net> wrote:
> I recently purchased a Lenovo IdeaPad Z560, model 09143YU, and as I am not a
> Windows fan I installed Linux Mint 11 Katya x64 to use instead of the
> supplied W7. I was encountering a known hang and had to upgrade to a later
> kernel, so I am now using the 2.6.38-11-generic #50-Ubuntu SMP Mon Sep 12
> 21:17:25 UTC 2011 x86_64 kernel.
>
> I had my first occasion to use the laptop for an extended period for the
> first time over the Thanksgiving holiday and I found it needed to be tuned.
> I downloaded powertop and used it to discover where my problems were.
> After addressing excessive i915 interrupts due to DRI the next most
> frequent cause of CPU wakeups is a kworker on the system. A search lead to
> a post by Tejun, indicating the need to trace such issues. Running the trace
> showed that 1933 of 2748 events were of the form:
>
> <idle>-0 [000] 22005.355346: workqueue_queue_work: work
> struct=ffff8800bb411188 function=do_dbs_timer workqueue=ffff88012b5d2c00
> req_cpu=0 cpu=0
>
> Tejun indicated that this is a workitem used by cpufreq and likely caused by
> something else hitting the CPU frequently. So how do I diagnose this
> further and isolate the cause for correction?
>
> If you would like a summary of this, download this spreadsheet:
>
> http://home.comcast.net/%7Eddurgee/Tracelog.ods
>
> If you would like to inspect the trace log itself:
>
> http://home.comcast.net/%7Eddurgee/tracelog.zip
>
Hi Dave,
I don't know the cpufreq code that well, but it seem that this
workqueue is schedule periodically. You
can examine the sampling rate via /sys:
$ grep "" /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate*
Just curious, are you running nohz:
$ dmesg | grep -i nohz
Regards,
Mandeep
> Dave
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Request for assistance - excessive kworker CPU wakeups
2012-01-25 17:44 ` Mandeep Singh Baines
@ 2012-01-25 18:56 ` David H. Durgee
2012-01-25 23:44 ` Mandeep Singh Baines
0 siblings, 1 reply; 5+ messages in thread
From: David H. Durgee @ 2012-01-25 18:56 UTC (permalink / raw)
To: Mandeep Singh Baines; +Cc: linux-kernel
Mandeep Singh Baines wrote:
> On Fri, Dec 9, 2011 at 1:27 PM, David H. Durgee<dhdurgee@verizon.net> wrote:
>> I recently purchased a Lenovo IdeaPad Z560, model 09143YU, and as I am not a
>> Windows fan I installed Linux Mint 11 Katya x64 to use instead of the
>> supplied W7. I was encountering a known hang and had to upgrade to a later
>> kernel, so I am now using the 2.6.38-11-generic #50-Ubuntu SMP Mon Sep 12
>> 21:17:25 UTC 2011 x86_64 kernel.
>>
>> I had my first occasion to use the laptop for an extended period for the
>> first time over the Thanksgiving holiday and I found it needed to be tuned.
>> I downloaded powertop and used it to discover where my problems were.
>> After addressing excessive i915 interrupts due to DRI the next most
>> frequent cause of CPU wakeups is a kworker on the system. A search lead to
>> a post by Tejun, indicating the need to trace such issues. Running the trace
>> showed that 1933 of 2748 events were of the form:
>>
>> <idle>-0 [000] 22005.355346: workqueue_queue_work: work
>> struct=ffff8800bb411188 function=do_dbs_timer workqueue=ffff88012b5d2c00
>> req_cpu=0 cpu=0
>>
>> Tejun indicated that this is a workitem used by cpufreq and likely caused by
>> something else hitting the CPU frequently. So how do I diagnose this
>> further and isolate the cause for correction?
>>
>> If you would like a summary of this, download this spreadsheet:
>>
>> http://home.comcast.net/%7Eddurgee/Tracelog.ods
>>
>> If you would like to inspect the trace log itself:
>>
>> http://home.comcast.net/%7Eddurgee/tracelog.zip
>>
> Hi Dave,
>
> I don't know the cpufreq code that well, but it seem that this
> workqueue is schedule periodically. You
> can examine the sampling rate via /sys:
>
> $ grep "" /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate*
>
> Just curious, are you running nohz:
>
> $ dmesg | grep -i nohz
>
> Regards,
> Mandeep
dhdurgee@DHD-Z560 ~/Downloads $ grep ""
/sys/devices/system/cpu/cpufreq/ondemand/sampling_rate*
/sys/devices/system/cpu/cpufreq/ondemand/sampling_rate:10000
/sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_max:4294967295
/sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_min:10000
dhdurgee@DHD-Z560 ~/Downloads $ dmesg | grep -i nohz
dhdurgee@DHD-Z560 ~/Downloads $
Does this explain what I am seeing? Does this square with the
observations in my spreadsheet? Does this suggest a means of reducing
these excessive wakeups? Is there more information I can provide to
suggest a course of action?
Dave
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Request for assistance - excessive kworker CPU wakeups
2012-01-25 18:56 ` David H. Durgee
@ 2012-01-25 23:44 ` Mandeep Singh Baines
2012-01-26 0:31 ` David H. Durgee
0 siblings, 1 reply; 5+ messages in thread
From: Mandeep Singh Baines @ 2012-01-25 23:44 UTC (permalink / raw)
To: David H. Durgee; +Cc: Mandeep Singh Baines, linux-kernel
David H. Durgee (dhdurgee@verizon.net) wrote:
> Mandeep Singh Baines wrote:
> >On Fri, Dec 9, 2011 at 1:27 PM, David H. Durgee<dhdurgee@verizon.net> wrote:
> >>I recently purchased a Lenovo IdeaPad Z560, model 09143YU, and as I am not a
> >>Windows fan I installed Linux Mint 11 Katya x64 to use instead of the
> >>supplied W7. I was encountering a known hang and had to upgrade to a later
> >>kernel, so I am now using the 2.6.38-11-generic #50-Ubuntu SMP Mon Sep 12
> >>21:17:25 UTC 2011 x86_64 kernel.
> >>
> >>I had my first occasion to use the laptop for an extended period for the
> >>first time over the Thanksgiving holiday and I found it needed to be tuned.
> >> I downloaded powertop and used it to discover where my problems were.
> >> After addressing excessive i915 interrupts due to DRI the next most
> >>frequent cause of CPU wakeups is a kworker on the system. A search lead to
> >>a post by Tejun, indicating the need to trace such issues. Running the trace
> >>showed that 1933 of 2748 events were of the form:
> >>
> >><idle>-0 [000] 22005.355346: workqueue_queue_work: work
> >>struct=ffff8800bb411188 function=do_dbs_timer workqueue=ffff88012b5d2c00
> >>req_cpu=0 cpu=0
> >>
> >>Tejun indicated that this is a workitem used by cpufreq and likely caused by
> >>something else hitting the CPU frequently. So how do I diagnose this
> >>further and isolate the cause for correction?
> >>
> >>If you would like a summary of this, download this spreadsheet:
> >>
> >>http://home.comcast.net/%7Eddurgee/Tracelog.ods
> >>
> >>If you would like to inspect the trace log itself:
> >>
> >>http://home.comcast.net/%7Eddurgee/tracelog.zip
> >>
> >Hi Dave,
> >
> >I don't know the cpufreq code that well, but it seem that this
> >workqueue is schedule periodically. You
> >can examine the sampling rate via /sys:
> >
> >$ grep "" /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate*
> >
> >Just curious, are you running nohz:
> >
> >$ dmesg | grep -i nohz
> >
> >Regards,
> >Mandeep
> dhdurgee@DHD-Z560 ~/Downloads $ grep ""
> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate*
> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate:10000
> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_max:4294967295
> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_min:10000
> dhdurgee@DHD-Z560 ~/Downloads $ dmesg | grep -i nohz
> dhdurgee@DHD-Z560 ~/Downloads $
>
> Does this explain what I am seeing? Does this square with the
> observations in my spreadsheet? Does this suggest a means of
> reducing these excessive wakeups? Is there more information I can
> provide to suggest a course of action?
>
IIUC, you should be seeing 100 such events per second * number of CPUs.
Is that what you are seeing? You could reduce this by changing HZ. Maybe
change to CONFIG_HZ_100 in your .config from CONFIG_HZ_1000. That
should reduce the number of events by a factor of 10.
Regards,
Mandeep
> Dave
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Request for assistance - excessive kworker CPU wakeups
2012-01-25 23:44 ` Mandeep Singh Baines
@ 2012-01-26 0:31 ` David H. Durgee
0 siblings, 0 replies; 5+ messages in thread
From: David H. Durgee @ 2012-01-26 0:31 UTC (permalink / raw)
To: Mandeep Singh Baines; +Cc: linux-kernel
Mandeep Singh Baines wrote:
> David H. Durgee (dhdurgee@verizon.net) wrote:
>> Mandeep Singh Baines wrote:
>>> On Fri, Dec 9, 2011 at 1:27 PM, David H. Durgee<dhdurgee@verizon.net> wrote:
>>>> I recently purchased a Lenovo IdeaPad Z560, model 09143YU, and as I am not a
>>>> Windows fan I installed Linux Mint 11 Katya x64 to use instead of the
>>>> supplied W7. I was encountering a known hang and had to upgrade to a later
>>>> kernel, so I am now using the 2.6.38-11-generic #50-Ubuntu SMP Mon Sep 12
>>>> 21:17:25 UTC 2011 x86_64 kernel.
>>>>
>>>> I had my first occasion to use the laptop for an extended period for the
>>>> first time over the Thanksgiving holiday and I found it needed to be tuned.
>>>> I downloaded powertop and used it to discover where my problems were.
>>>> After addressing excessive i915 interrupts due to DRI the next most
>>>> frequent cause of CPU wakeups is a kworker on the system. A search lead to
>>>> a post by Tejun, indicating the need to trace such issues. Running the trace
>>>> showed that 1933 of 2748 events were of the form:
>>>>
>>>> <idle>-0 [000] 22005.355346: workqueue_queue_work: work
>>>> struct=ffff8800bb411188 function=do_dbs_timer workqueue=ffff88012b5d2c00
>>>> req_cpu=0 cpu=0
>>>>
>>>> Tejun indicated that this is a workitem used by cpufreq and likely caused by
>>>> something else hitting the CPU frequently. So how do I diagnose this
>>>> further and isolate the cause for correction?
>>>>
>>>> If you would like a summary of this, download this spreadsheet:
>>>>
>>>> http://home.comcast.net/%7Eddurgee/Tracelog.ods
>>>>
>>>> If you would like to inspect the trace log itself:
>>>>
>>>> http://home.comcast.net/%7Eddurgee/tracelog.zip
>>>>
>>> Hi Dave,
>>>
>>> I don't know the cpufreq code that well, but it seem that this
>>> workqueue is schedule periodically. You
>>> can examine the sampling rate via /sys:
>>>
>>> $ grep "" /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate*
>>>
>>> Just curious, are you running nohz:
>>>
>>> $ dmesg | grep -i nohz
>>>
>>> Regards,
>>> Mandeep
>> dhdurgee@DHD-Z560 ~/Downloads $ grep ""
>> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate*
>> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate:10000
>> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_max:4294967295
>> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_min:10000
>> dhdurgee@DHD-Z560 ~/Downloads $ dmesg | grep -i nohz
>> dhdurgee@DHD-Z560 ~/Downloads $
>>
>> Does this explain what I am seeing? Does this square with the
>> observations in my spreadsheet? Does this suggest a means of
>> reducing these excessive wakeups? Is there more information I can
>> provide to suggest a course of action?
>>
> IIUC, you should be seeing 100 such events per second * number of CPUs.
> Is that what you are seeing? You could reduce this by changing HZ. Maybe
> change to CONFIG_HZ_100 in your .config from CONFIG_HZ_1000. That
> should reduce the number of events by a factor of 10.
>
> Regards,
> Mandeep
Looking at /boot/config-2.6.38-11-generic I see the following:
CONFIG_HZ_100=y
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=100
So it appears that this is already set. Looking at the raw data I
collected and summarized I see 22005.355346 in the first record and
22031.110816 in the last record. Am I correct in interpreting this to
mean that 25.76 seconds elapsed while data was collected? During the
collection I saw 1,993 total idle do_dbs_timer calls. I saw a total of
2,754 wakeups for all events. You can get the details from the files I
referenced.
Dave
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-01-26 1:32 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-12-09 21:27 Request for assistance - excessive kworker CPU wakeups David H. Durgee
2012-01-25 17:44 ` Mandeep Singh Baines
2012-01-25 18:56 ` David H. Durgee
2012-01-25 23:44 ` Mandeep Singh Baines
2012-01-26 0:31 ` David H. Durgee
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).