KS/Plumbers: c-state governor BOF

KS/Plumbers: c-state governor BOF

[Rik van Riel]

After talking about my RFC patches to the c-state governor with
Matthew and Arjan, it is clear that the whole concept of how
things are done could use some more discussion.

Since a good number of us will be in San Diego next week, at
Kernel Summit / Plumbers / etc, I will organize a c-state
governor BOF for those who are interested.

Things to think about:
- what should the c-state governor do?
- how to best predict the future?
- what kinds of odd workloads do we need to accomodate?

I will talk to the LF people about getting space for such a
BOF (probably after a mini-summit is done with a room, or a
spare room), and will announce when and where such a BOF will
take place.

Watch whatever board is used to announce BOFs next week :)

-- 
All rights reversed

Re: KS/Plumbers: c-state governor BOF

[Rik van Riel]

On 08/24/2012 01:01 PM, Rik van Riel wrote:

> Since a good number of us will be in San Diego next week, at
> Kernel Summit / Plumbers / etc, I will organize a c-state
> governor BOF for those who are interested.

Here are the note from today's c-state governor BOF session:

- idle time estimator can be improved to return more
   conservative values
- have a trace point to verify how well idle time estimation works,
   returning expected idle time for the current algorithm, expected
   idle time for a test algorithm, and actually measured idle time
- we can objectively measure whether one idle time estimation
   algorithm is better than another
- example: last 8 idle times are 180 200 30 210 1000 150 10000 190
   - average idle time around 1500, but not typical
   - typical idle time just under 200
   - almost all idle time spent in one interval
- scheduler and cpufreq people also want idle time
   statistics, idea is to move the idle time statistics
   to the scheduler
- having idle time statistics in the scheduler allows
   the stats to be migrated when the scheduler migrates
   a task
- interface: expected idle time, demotion time
   - if we sleep much more than the expected idle time, put
     CPU into a deeper c-state
   - on x86 timed mwait could be used
   - generic timer code would be usable too
   - take into account both the idle statistics and power break-even

Re: KS/Plumbers: c-state governor BOF

[Rik van Riel]

On 09/11/2012 10:43 PM, Youquan Song wrote:
>> After talking about my RFC patches to the c-state governor with
>> Matthew and Arjan, it is clear that the whole concept of how
>> things are done could use some more discussion.
>>
>> Since a good number of us will be in San Diego next week, at
>> Kernel Summit / Plumbers / etc, I will organize a c-state
>> governor BOF for those who are interested.
>>
>> Things to think about:
>> - what should the c-state governor do?
>> - how to best predict the future?
>> - what kinds of odd workloads do we need to accomodate?
>
> Hi Rik,
>
> Just notice there is a topic to discuss menu governor at Kernel Summit.
> Acutally, I have posted a patchset to at May 11 2012 to bring up the
> topic, at that time, I only have a convinced and proved application
> turbostat v1 to prove that my patch are useful. I try to find other
> workloads to prove that the patchset are also solidated useful. But I
> stucked in other high priority tasks, so I move slow on it.
>  From you bring up the issue I guess that you already has real workload
> to show this issue.
> My patchset is not only improve repeat mode failure but also improve
> general prediction failure. Let's have a discuss and talk about it.
>
> Here is the patchset posted at May 11 2012.
>
> http://lwn.net/Articles/496919/ "x86,idle: Enhance cpuidle prediction to
> handle its failure"
> http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02267.html
> "[PATCH 1/3] x86,idle: Quickly notice prediction failure for repeat mode"
> http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02268.html
> "[PATCH 2/3] x86,idle: Quickly notice prediction failure in general case"
> http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02269.html
> "[PATCH 3/3] x86,idle: Set residency to 0 if target Cstate not really
> enter"

Your patches could make a lot of sense when integrated with my
patches:

http://people.redhat.com/riel/cstate/

However, we should probably get the tracepoint upstream first,
so we can know for sure :)

-- 
All rights reversed

Re: KS/Plumbers: c-state governor BOF

[Rik van Riel]

On 09/12/2012 12:15 AM, Youquan Song wrote:
>> Your patches could make a lot of sense when integrated with my
>> patches:
>>
>> http://people.redhat.com/riel/cstate/
>> However, we should probably get the tracepoint upstream first,
>> so we can know for sure :)
>
> I can not access the patches at this directory. Can you send it to me?
> I will look at your patches and then integrated with my patches to look
> what will happen tomorrow.

Argh, fixed!

> Do you have test case share? or ideas how to show the benefit.
>
> I have done many test for my pathes. It show some benefit big or small
> in various cases, but there is no negative effect showed at least.
>
> I have two onviced test cases to show the great benefit
> 1.  turbostat v1 (before 3.5)
> 2. I write the simple test application which also show greate benefit.
> running it by #./idle_predict -l 8
>
>
> I write a simple application using usleep which it is clear to the
> repeat mode prediction failure will greatly effect the application with
> such repeat pattern.

Your test looks exactly like the kind of thing that could benefit
from my changes to the menu governor :)

-- 
All rights reversed

Re: KS/Plumbers: c-state governor BOF

[Youquan Song]

> After talking about my RFC patches to the c-state governor with
> Matthew and Arjan, it is clear that the whole concept of how
> things are done could use some more discussion.
>
> Since a good number of us will be in San Diego next week, at
> Kernel Summit / Plumbers / etc, I will organize a c-state
> governor BOF for those who are interested.
>
> Things to think about:
> - what should the c-state governor do?
> - how to best predict the future?
> - what kinds of odd workloads do we need to accomodate?

Hi Rik,

Just notice there is a topic to discuss menu governor at Kernel Summit.
Acutally, I have posted a patchset to at May 11 2012 to bring up the
topic, at that time, I only have a convinced and proved application 
turbostat v1 to prove that my patch are useful. I try to find other
workloads to prove that the patchset are also solidated useful. But I
stucked in other high priority tasks, so I move slow on it.
>From you bring up the issue I guess that you already has real workload
to show this issue. 
My patchset is not only improve repeat mode failure but also improve 
general prediction failure. Let's have a discuss and talk about it.

Here is the patchset posted at May 11 2012.

http://lwn.net/Articles/496919/ "x86,idle: Enhance cpuidle prediction to
handle its failure"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02267.html
"[PATCH 1/3] x86,idle: Quickly notice prediction failure for repeat mode"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02268.html
"[PATCH 2/3] x86,idle: Quickly notice prediction failure in general case"
http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02269.html
"[PATCH 3/3] x86,idle: Set residency to 0 if target Cstate not really
enter"

Thanks
-Youquan

Re: KS/Plumbers: c-state governor BOF

[Youquan Song]

> Your patches could make a lot of sense when integrated with my
> patches:
>
> http://people.redhat.com/riel/cstate/
> However, we should probably get the tracepoint upstream first,
> so we can know for sure :)

I can not access the patches at this directory. Can you send it to me?
I will look at your patches and then integrated with my patches to look
what will happen tomorrow.

Do you have test case share? or ideas how to show the benefit.

I have done many test for my pathes. It show some benefit big or small
in various cases, but there is no negative effect showed at least. 
 
I have two onviced test cases to show the great benefit
1.  turbostat v1 (before 3.5)
2. I write the simple test application which also show greate benefit.
running it by #./idle_predict -l 8


I write a simple application using usleep which it is clear to the
repeat mode prediction failure will greatly effect the application with
such repeat pattern.

-----------------------
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <signal.h>
#include <sys/time.h>
#include <time.h>
#include <pthread.h>

volatile int * shutdown;
volatile long * count;
int delay = 20;
int loop = 8;

void usage(void)
{
	fprintf(stderr,
		"Usage: idle_predict [options]\n"
		"  --help	-h  Print this help\n"
		"  --thread	-n  Thread number\n"
		"  --loop     	-l  Loop times in shallow Cstate\n"
		"  --delay	-t  Sleep time (uS)in shallow
Cstate\n");
}

void *simple_loop() {
	int idle_num = 1;
	while (*shutdown) {
		*count = *count + 1;
	
		if (idle_num % loop)
			usleep(delay);
		else {
			/* sleep 1 second */
			usleep(1000000);
			idle_num = 0;
		}
		idle_num++;
	}

}

static void sighand(int sig)
{
	*shutdown = 0;
}

int main(int argc, char *argv[])
{
	sigset_t sigset;
	int signum = SIGALRM;
	int i, c, er = 0, thread_num = 8;
	pthread_t pt[1024];

	static char optstr[] = "n:l:t:h:";

	while ((c = getopt(argc, argv, optstr)) != EOF)
		switch (c) {
			case 'n':
				thread_num = atoi(optarg);
				break;
			case 'l':
				loop = atoi(optarg);
				break;
			case 't':
				delay = atoi(optarg);
				break;
			case 'h':
			default:
				usage();
				exit(1);
		}

	printf("thread=%d,loop=%d,delay=%d\n",thread_num,loop,delay);
	count = malloc(sizeof(long));
	shutdown = malloc(sizeof(int));
	*count = 0;
	*shutdown = 1;

	sigemptyset(&sigset);
	sigaddset(&sigset, signum);
	sigprocmask (SIG_BLOCK, &sigset, NULL);
	signal(SIGINT, sighand);
	signal(SIGTERM, sighand);

	for(i = 0; i < thread_num ; i++)
		pthread_create(&pt[i], NULL, simple_loop, NULL);

	for (i = 0; i < thread_num; i++)
		pthread_join(pt[i], NULL);

	exit(0);
}