Processor sets (pset) for linux kernel 2.5/2.6?

Processor sets (pset) for linux kernel 2.5/2.6?

[Shaheed R. Haque]

Hi,

Looking through the archive etc., I cannot see any references to pset 
functionality more recently than

http://www.ussg.iu.edu/hypermail/linux/kernel/9904.1/0521.html

announcing an updated patch in April 1999. Are there any plans out there to 
include this, or similar functionality in 2.5/2.6? I'm particularly interested 
in getting exclusive access to a CPU (plus or minus HT support, for now anyway).

Thanks, Shaheed

Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Dave Jones]

On Sat, Apr 12, 2003 at 12:20:35PM +0100, Shaheed R. Haque wrote:

 > announcing an updated patch in April 1999. Are there any plans out there to 
 > include this, or similar functionality in 2.5/2.6? I'm particularly interested 
 > in getting exclusive access to a CPU (plus or minus HT support, for now anyway).

sched_getaffinity() and sched_setaffinity()
You'll also need schedutils from http://tech9.net/rml/schedutils

all this (and more) documented at http://www.codemonkey.org.uk/post-halloween-2.5.txt

		Dave

Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Antonio Vargas]

On Sat, Apr 12, 2003 at 12:20:35PM +0100, Shaheed R. Haque wrote:
> 
> Hi,
> 
> Looking through the archive etc., I cannot see any references to pset 
> functionality more recently than
> 
> http://www.ussg.iu.edu/hypermail/linux/kernel/9904.1/0521.html
> 
> announcing an updated patch in April 1999. Are there any plans out there to 
> include this, or similar functionality in 2.5/2.6? I'm particularly interested 
> in getting exclusive access to a CPU (plus or minus HT support, for now anyway).

Shaheed, I think that task->cpu_allowed covers your need, since it
describes a mask of cpu�'s allowed to run the task. CPU migration, for
example, is implemented in current scheduler by saving the
current cpu_allowed, forcing it to contain only the destination CPU and
then restoring it.

I think it's carried over by fork(), but have been unable to find
it on the sources.

HT is accounted as a NUMA SMP system with strong memory affinity
for his 2 cores, so that when running 2 HT processors (2+2 cores),
the tasks are kept preferably on the same HT processor and just bounce
cores, since they share the same cache (don't know exactly L1, L2 or
both, tough).

Greets, Antonio.

Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Martin J. Bligh]

> HT is accounted as a NUMA SMP system with strong memory affinity
> for his 2 cores, so that when running 2 HT processors (2+2 cores),
> the tasks are kept preferably on the same HT processor and just bounce
> cores, since they share the same cache (don't know exactly L1, L2 or
> both, tough).

We don't actually do that right now (though it was discussed) - we treat it 
as flat SMP. Ingo had some patches to share 1 runqueue between each HT pair, 
which I think is a better plan for this, but we're not seeing much 
performance improvment from it.

M.

Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Robert Love]

On Sat, 2003-04-12 at 08:24, Antonio Vargas wrote:

> I think it's carried over by fork(), but have been unable to find
> it on the sources.

Yep, its inherited over fork().

	Robert Love

Re: Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Shaheed R. Haque]

Hmmm, AFAICS, sched_getaffinity() and sched_setaffinity() 
allow the calling process to be bound to the nominated CPU(s), but that is not 
the same as giving them exclusive access, is it? In other words, other 
processes which have no particualr affinity needs can presumably still be 
scheduled to run on the same processor. 

I am looking for something more akin to the patch I referred to...or did I miss 
something in the effect of set_cpus_allowed()?

Thanks for the replies, Shaheed

Re: Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Robert Love]

On Sat, 2003-04-12 at 15:56, Shaheed R. Haque wrote:

> Hmmm, AFAICS, sched_getaffinity() and sched_setaffinity() 
> allow the calling process to be bound to the nominated CPU(s), but that is not 
> the same as giving them exclusive access, is it? In other words, other 
> processes which have no particualr affinity needs can presumably still be 
> scheduled to run on the same processor. 
> 
> I am looking for something more akin to the patch I referred to...or did I miss 
> something in the effect of set_cpus_allowed()?

We strive for simple interfaces here in Linux :)

If you want to give them exclusive access, you need to bind all the
other processes to the other processors.  One easy way to do this is to
have init bind itself elsewhere on boot.

	Robert Love

Re: Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Martin J. Bligh]

> Hmmm, AFAICS, sched_getaffinity() and sched_setaffinity() 
> allow the calling process to be bound to the nominated CPU(s), but that is not 
> the same as giving them exclusive access, is it? In other words, other 
> processes which have no particualr affinity needs can presumably still be 
> scheduled to run on the same processor. 
> 
> I am looking for something more akin to the patch I referred to...or did I miss 
> something in the effect of set_cpus_allowed()?

The NUMA scheduler work can trivially be converted into arbitrary
scheduler pools - they are not designed for dynamic modification
at the moment, but could be without too much effort I think.

Around 2.5.59 or so I posted a patch to rename them to pools, though
we had a few other things to sort out at the time. I might revive it
at some point - it's pretty simple.

M.

Re: Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Shaheed R. Haque]

Quoting Robert Love <rml@tech9.net>:

> We strive for simple interfaces here in Linux :)
> 
> If you want to give them exclusive access, you need to bind all the
> other processes to the other processors.  One easy way to do this is to
> have init bind itself elsewhere on boot.

Interesting idea. AFAICS, this involves either changing the code of /sbin/init 
to set its affinity to a default cpu mask (provided by a kernel boot flag, I 
presume), or using taskset-like functionality and then hoping that all current 
shells etc. die before the programs I care about start.

Would it not be better to simply have the kernel use the boot flag directly as 
the default CPU mask setting?

[ I can see the argument that "if you need such control, you should set up your 
distro to deal with it", but that is a level of tweaking I would like to avoid 
for my customers ]

Thanks, Shaheed

Re: Re: Processor sets (pset) for linux kernel 2.5/2.6?

[Robert Love]

On Sun, 2003-04-13 at 04:30, Shaheed R. Haque wrote:

> Interesting idea. AFAICS, this involves either changing the code of /sbin/init 
> to set its affinity to a default cpu mask (provided by a kernel boot flag, I 
> presume), or using taskset-like functionality and then hoping that all current 
> shells etc. die before the programs I care about start.

Well, you can get away with it easier by having a program (like taskset
in schedutils) run in rc.sysinit.  Have it bind init to the appropriate
processor.

> Would it not be better to simply have the kernel use the boot flag directly as 
> the default CPU mask setting?

This is not a bad idea either, as it involves minimal code.

	Robert Love

[RFC] patch to allow CPUs to be reserved to callers of sys_setaffinity [was Processor sets (pset) for linux kernel 2.5/2.6?]

[shaheed]

Hi,

Following up from http://www.ussg.iu.edu/hypermail/linux/kernel/0304.1/1113.html, here is a prototype patch against 2.5.67 to allow CPUs to be reserved to callers of sys_setaffinity(). It seems to work safely on a uniprocessor, and I'll test it on a multi-processor in a few days time. I'm not requesting inclusion of this patch until then.

Any feedback on form as well as content appreciated...I'm particularly wondering if it is worth stealing sys_setaffinity()/sys_getaffinity() for PID 0 to set/return this value since that would allow user-mode code to dynamically choose a non-restricted processor if required.

Thanks, Shaheed

1. The patch is against a clean 2.5.67

2. There is a new boot flag "restricted_cpus". This takes a bit mask of CPUs.

3. The initial task has it's cpus_allowed mask set to ~restricted_cpus in the fork initialisation code (it cannot be done in TASK_INIT since the value is not static). This should mean that any task that does not override cpus_allowed with sys_setaffinity does not get to run on the restricted processors.

4. The parsing of the boot flag does a sanity check to prevent all available CPUs being restricted.

5. Patch follows...

diff -urN -X dontdiff linux-2.5.67/kernel/fork.c newlinux-2.5.67/kernel/fork.c
--- linux-2.5.67/kernel/fork.c	2003-04-07 18:30:42.000000000 +0100
+++ newlinux-2.5.67/kernel/fork.c	2003-04-14 21:36:59.000000000 +0100
@@ -206,6 +206,7 @@
 
 	init_task.rlim[RLIMIT_NPROC].rlim_cur = max_threads/2;
 	init_task.rlim[RLIMIT_NPROC].rlim_max = max_threads/2;
+	init_task.cpus_allowed = ~restricted_cpus;
 }
 
 static struct task_struct *dup_task_struct(struct task_struct *orig)
diff -urN -X dontdiff linux-2.5.67/kernel/sched.c newlinux-2.5.67/kernel/sched.c
--- linux-2.5.67/kernel/sched.c	2003-04-07 18:32:23.000000000 +0100
+++ newlinux-2.5.67/kernel/sched.c	2003-04-14 22:28:31.000000000 +0100
@@ -40,6 +40,13 @@
 #endif
 
 /*
+ * The boot flag "restricted_cpus" is negated to form the default per-task
+ * cpus_allowed mask. This has the effect of only allowing processes which
+ * call sys_setaffinity() to run on restricted_cpus.
+ */ 
+unsigned long restricted_cpus = 0;
+
+/*
  * Convert user-nice values [ -20 ... 0 ... 19 ]
  * to static priority [ MAX_RT_PRIO..MAX_PRIO-1 ],
  * and back.
@@ -2338,6 +2345,34 @@
 }
 
 /*
+ * set_restricted_cpus - process boot flag to set restricted_cpus.
+ *
+ * NOTE: This function is a __setup and __init function.
+ */
+int __init set_restricted_cpus(char *options)
+{
+	unsigned long temp;
+	char *value;
+
+	if (!options || !*options)
+		return 0;
+	temp = simple_strtoul(value, &value, 0);
+
+	/*
+	 * Make sure that the user has not restricted all available CPUs,
+	 * otherwise a functioning system won't result!
+	 */
+	if (~temp & cpu_online_map) {
+		restricted_cpus = temp;
+	} else {
+		printk(KERN_ERR "cannot restrict all 0x%lx online CPUs!\n", cpu_online_map);
+	}
+	return 0;
+}
+
+__setup("restricted_cpus=", set_restricted_cpus);
+
+/*
  * migration_thread - this is a highprio system thread that performs
  * thread migration by 'pulling' threads into the target runqueue.
  */
diff -urN -X dontdiff linux-2.5.67/Documentation/kernel-parameters.txt newlinux-2.5.67/Documentation/kernel-parameters.txt
--- linux-2.5.67/Documentation/kernel-parameters.txt	2003-04-07 18:33:03.000000000 +0100
+++ newlinux-2.5.67/Documentation/kernel-parameters.txt	2003-04-14 22:45:25.000000000 +0100
@@ -809,6 +809,10 @@
 
 	reserve=	[KNL,BUGS] Force the kernel to ignore some iomem area
 
+	restricted_cpus=
+	                [KNL] Don't use these CPUs for processes by default.
+			Format: <cpu_mask>
+
 	resume=		[SWSUSP] Specify the partition device for software suspension
 
 	riscom8=	[HW,SERIAL]
diff -urN -X dontdiff linux-2.5.67/include/linux/sched.h newlinux-2.5.67/include/linux/sched.h
--- linux-2.5.67/include/linux/sched.h	2003-04-07 18:30:42.000000000 +0100
+++ newlinux-2.5.67/include/linux/sched.h	2003-04-14 21:37:42.000000000 +0100
@@ -88,6 +88,7 @@
 
 extern int nr_threads;
 extern int last_pid;
+extern unsigned long restricted_cpus;
 DECLARE_PER_CPU(unsigned long, process_counts);
 extern int nr_processes(void);
 extern unsigned long nr_running(void);

Working .config for a Dell 2650 for 2.5.6x? (was Re: Processor sets (pset) for linux kernel 2.5/2.6?)

[shaheed]

Hi,

Can any kind soul with a working .config (no modules, boot from AIC7xxx JBOD 
disk is fine) for a Dell 2650 for 2.5.6anything kindly send me a copy? 
Off-list is probably best -  I'm trying to get this patch going, but am 
having an unbeliveable amount of difficulty getting this system booted.

Thanks, Shaheed

Re: 2.6 must-fix list, v2

[Shaheed R. Haque]

Can I propose an addition?

Not-ready features and speedups
===============================

kernel/fork.c
-------------

- Add ability to restrict the the default CPU affinity mask so that 
sys_setaffinity() can be used to implement exclusive access to a CPU. Patch 
exists on LKML.

Re: 2.6 must-fix list, v2

[Andrew Morton]

"Shaheed R. Haque" <srhaque@iee.org> wrote:
>
> - Add ability to restrict the the default CPU affinity mask so that 
>  sys_setaffinity() can be used to implement exclusive access to a CPU.

Why is this useful?

Re: 2.6 must-fix list, v2

[Shaheed R. Haque]

Quoting Andrew Morton <akpm@digeo.com>:

> "Shaheed R. Haque" <srhaque@iee.org> wrote:
> >
> > - Add ability to restrict the the default CPU affinity mask so that 
> >  sys_setaffinity() can be used to implement exclusive access to a CPU.
> 
> Why is this useful?

Because it allows one to dedicate a CPU to a process. For example, lets say you
have a quad processor,and want to run joe-random stuff on CPU 0, but a
specialised program on CPUs 1, 2, 3 that does not want to compete with
joe-random stuff.

With sys_setaffinity(), one can set the affinity of the special program to
0xe...but the default affinity for all the joe-random stuff is still 0xf (from
cpu_online_map)! Since its impractical to to modify every single joe-random
executable to set its affinity to 0x1, a way is needed to set the default. The
logical place is in init(), a.k.a. kernel/fork.c.

I hope that make sense.

Thanks, Shaheed

Re: 2.6 must-fix list, v2

[Shaheed R. Haque]

Quoting Andrew Morton <akpm@digeo.com>:

> > - Add ability to restrict the the default CPU affinity mask so that 
> >  sys_setaffinity() can be used to implement exclusive access to a CPU.
> 
> Why is this useful?

I forgot to add that the result is the rough equivalent of Digital UNIX's psets
and Irix's sysmp for my prurposes at least.

Re: 2.6 must-fix list, v2

[Steven Cole]

On Tue, 2003-05-13 at 16:46, Shaheed R. Haque wrote:
> Quoting Andrew Morton <akpm@digeo.com>:
> 
> > "Shaheed R. Haque" <srhaque@iee.org> wrote:
> > >
> > > - Add ability to restrict the the default CPU affinity mask so that 
> > >  sys_setaffinity() can be used to implement exclusive access to a CPU.
> > 
> > Why is this useful?
> 
> Because it allows one to dedicate a CPU to a process. For example, lets say you
> have a quad processor,and want to run joe-random stuff on CPU 0, but a
> specialised program on CPUs 1, 2, 3 that does not want to compete with
> joe-random stuff.
> 
> With sys_setaffinity(), one can set the affinity of the special program to
> 0xe...but the default affinity for all the joe-random stuff is still 0xf (from
> cpu_online_map)! Since its impractical to to modify every single joe-random
> executable to set its affinity to 0x1, a way is needed to set the default. The
> logical place is in init(), a.k.a. kernel/fork.c.
> 
> I hope that make sense.
> 
> Thanks, Shaheed
> 

Is this related or not to processor shielding used by RedHawk Linux?
Here is a link to their page:

http://www.ccur.com/realtime/sys_rdhwklnx.html

I saw a presentation by these guys over a year ago.  I'm not sure what
they're up to now.

Steven

Re: 2.6 must-fix list, v2

[Felipe Alfaro Solana]

On Wed, 2003-05-14 at 00:49, Shaheed R. Haque wrote:
> Quoting Andrew Morton <akpm@digeo.com>:
> 
> > > - Add ability to restrict the the default CPU affinity mask so that 
> > >  sys_setaffinity() can be used to implement exclusive access to a CPU.
> > 
> > Why is this useful?
> 
> I forgot to add that the result is the rough equivalent of Digital UNIX's psets
> and Irix's sysmp for my prurposes at least.

And psets and fencing in Solaris too...

Re: 2.6 must-fix list, v2

[Shaheed R. Haque]

Quoting Steven Cole <elenstev@mesatop.com>:

> Is this related or not to processor shielding used by RedHawk Linux?
> Here is a link to their page:
> 
> http://www.ccur.com/realtime/sys_rdhwklnx.html
> 
> I saw a presentation by these guys over a year ago.  I'm not sure what
> they're up to now.

Yes, if I correctly read the description of this feature, it seems to be the 
same thing.

Re: 2.6 must-fix list, v2

[Steven Cole]

On Wed, 2003-05-14 at 05:49, Shaheed R. Haque wrote:
> Quoting Steven Cole <elenstev@mesatop.com>:
> 
> > Is this related or not to processor shielding used by RedHawk Linux?
> > Here is a link to their page:
> > 
> > http://www.ccur.com/realtime/sys_rdhwklnx.html
> > 
> > I saw a presentation by these guys over a year ago.  I'm not sure what
> > they're up to now.
> 
> Yes, if I correctly read the description of this feature, it seems to be the 
> same thing.
> 
Thanks, that is what I suspected.

There seemed to be quite a bit of interest in this from the other
customers, although our facility doesn't presently need this
functionality.  In the spirit of the "squeaky wheel", I'll squeak softly
for them.  

>From the above web page, thus quoth the RedHawk:

"In tightly-coupled symmetric multiprocessing systems such as
Concurrent’s iHawk real-time systems, RedHawk Linux allows individual
CPUs to be shielded from interrupt processing, daemons, bottom halves,
and other Linux tasks. Processor shielding provides a highly
deterministic execution environment where interrupt response is
guaranteed. RedHawk implements shielding via the industry-accepted
shield(1) command."

Steven

Re: 2.6 must-fix list, v2

[Robert Love]

On Wed, 2003-05-14 at 07:02, Felipe Alfaro Solana wrote:

> > > > - Add ability to restrict the the default CPU affinity mask so that 
> > > >  sys_setaffinity() can be used to implement exclusive access to a CPU.
> > > 
> > > Why is this useful?
> > 
> > I forgot to add that the result is the rough equivalent of Digital UNIX's psets
> > and Irix's sysmp for my prurposes at least.
> 
> And psets and fencing in Solaris too...

You can get exclusive access with mangling the system call, simply by
having init  bind itself to the non-exclusive processors on boot.

Try it. Every task will then end up on only the non-exclusive
processors.  Seems a very simple change to me, and one that can be done
in user-space.

You do not even have to modify init, if you do not want.  Grab
http://tech9.net/rml/schedutils and put a taskset call in your rc.d

	Robert Love

Re: 2.6 must-fix list, v2

[Robert Love]

On Wed, 2003-05-14 at 11:59, Robert Love wrote:

> You can get exclusive access with mangling the system call, simply by
> having init bind itself to the non-exclusive processors on boot.

Whoops. s/with/without/

Sorry,

	Robert Love

Re: 2.6 must-fix list, v2

[shaheed]

On Wednesday 14 May 2003 4:59 pm, Robert Love wrote:

> You can get exclusive access with mangling the system call, simply by
> having init  bind itself to the non-exclusive processors on boot.
>
> Try it. Every task will then end up on only the non-exclusive
> processors.  Seems a very simple change to me, and one that can be done
> in user-space.
>
> You do not even have to modify init, if you do not want.  Grab
> http://tech9.net/rml/schedutils and put a taskset call in your rc.d

Ah. I think I misread your previous note to me on this...that's why my patch 
modifies init itself (it does not muck with the syscall in any way). I'll try 
this as soon as I have my 2.5 multiprocessor back. BTW: what are the plans 
for getting schedutils (and specifically taskset) into a normal 2.6-based 
distribution? Can I be reasonably sure that this will happen?

Re: 2.6 must-fix list, v2

[Robert Love]

On Wed, 2003-05-14 at 17:01, shaheed wrote:

> Ah. I think I misread your previous note to me on this...that's why my patch 
> modifies init itself (it does not muck with the syscall in any way). I'll try 
> this as soon as I have my 2.5 multiprocessor back. BTW: what are the plans 
> for getting schedutils (and specifically taskset) into a normal 2.6-based 
> distribution? Can I be reasonably sure that this will happen?

I think it is in Debian (unstable at least).

More important to me is getting it into Red Hat and SuSE. I have heard
encouring words from Matt Wilson at Red Hat about schedutils possibly
going into Rawhide soon. It would not hurt to let Red Hat/SuSE/whoever
know that schedutils is something their customers want.

Both Red Hat and SuSE's kernels have the CPU affinity system calls
merged, so you do not need to wait until 2.6 is out to use them.

	Robert Love

Re: 2.6 must-fix list, v2

[Shaheed R. Haque]

Quoting Robert Love <rml@tech9.net>:

> More important to me is getting it into Red Hat and SuSE. I have heard
> encouring words from Matt Wilson at Red Hat about schedutils possibly
> going into Rawhide soon. It would not hurt to let Red Hat/SuSE/whoever
> know that schedutils is something their customers want.
>
> Both Red Hat and SuSE's kernels have the CPU affinity system calls
> merged, so you do not need to wait until 2.6 is out to use them.

These are the distros I am interested in too. I knew it was in RH AS/ES, but 
are you saying it is in RH9.0? That would be good news.

On the technical point, I tried out taskset in rc.sysinit, and as you said, it 
works just fine. On reflection, I feel that editing rc.sysinit is not the right 
answer given the confidence/competence level of our customers' typical 
sysadmins: but I can see that a carefully crafted rc5.d/S00aaaaa script could 
set the affinity of the executing shell, and its parent(s) upto init to fix all 
subsequent rcN.d children in the desired manner.

I do suspect that other commercial users will also baulk at editing rc.sysint, 
and so have to brew the same rcN.d solution. Now, the rcN.d script hackery 
would be greatly simplified if taskset had a mode of "set the affinity of the 
identified process, and all its parent processes upto init". Would you accept a 
patch to taskset along those lines?

I think that would be a very acceptable, easy to deploy, solution.

Thoughts? Shaheed

Re: 2.6 must-fix list, v2

[Robert Love]

On Thu, 2003-05-15 at 05:19, Shaheed R. Haque wrote:

> These are the distros I am interested in too. I knew it was in RH AS/ES, but 
> are you saying it is in RH9.0? That would be good news.

No, I am saying with luck it will be in the next RH release.

> On the technical point, I tried out taskset in rc.sysinit, and as you said, it 
> works just fine.

Good :)

> On reflection, I feel that editing rc.sysinit is not the right 
> answer given the confidence/competence level of our customers' typical 
> sysadmins: but I can see that a carefully crafted rc5.d/S00aaaaa script could 
> set the affinity of the executing shell, and its parent(s) upto init to fix all 
> subsequent rcN.d children in the desired manner.
> 
> I do suspect that other commercial users will also baulk at editing rc.sysint, 
> and so have to brew the same rcN.d solution. Now, the rcN.d script hackery 
> would be greatly simplified if taskset had a mode of "set the affinity of the 
> identified process, and all its parent processes upto init". Would you accept a 
> patch to taskset along those lines?

It is racey to do this, so its something that should remain a hack and
not part of taskset, I think.

If you do it in rc.d, you don't need to set all the parents. rc.d is the
first thing run, so if you do it at the top of the script, nothing else
is running. Just put:

	taskset <mask> 1
	taskset <mask> $$

at the top of rc.d.

Another consideration is modifying init (and hopefully having said
changes merged back). Init could call sched_setaffinity() when it is
first created, based on a setting in /etc/inittab or a command line
parameter passed during boot.

My reservation is against doing it in the kernel. I do not particularly
care _how_ its done in user-space.

	Robert Love

Re: 2.6 must-fix list, v2

[shaheed]

On Thursday 15 May 2003 4:32 pm, Robert Love wrote:

> It is racey to do this, so its something that should remain a hack and
> not part of taskset, I think.

Hmm. I guess you are thinking of daemons?

> If you do it in rc.d, you don't need to set all the parents. rc.d is the
> first thing run, so if you do it at the top of the script, nothing else
> is running. Just put:
>
> 	taskset <mask> 1
> 	taskset <mask> $$
>
> at the top of rc.d.

Perhaps we are talking at cross purposes. As I understand it the calling chain 
is:

1. kernel bootstrap
2. /sbin/init
3. bash to run /etc/rc.sysinit
4. bash to run individual /etc/rcN.d/whatever

I feel wary of doing it in 3 as you seem to suggest because I am pretty sure 
this will intimidate my customers. I am happy to do it in 4 - I can avoid the 
races by only doing it for the distros I care about.

That leaves options 1 and 2 for a community-wide solution. I guess I haven't 
quite understood the reluctance to do it in 1 given that:

- we know who owns 1 (!!)

- AFAICS, it isn't conceptual bloat because the utility of the implementation 
of sys_setaffinity() in 1 is greatly limited by not including this feature.

- its hardly physical bloat because the number of bytes required to implement 
this is absolutely in the noise, and virtually all __init()ed away.

> Another consideration is modifying init (and hopefully having said
> changes merged back). Init could call sched_setaffinity() when it is
> first created, based on a setting in /etc/inittab or a command line
> parameter passed during boot.

I have no idea with whom to persue this path, and as I say, I feel that 
solving this once for each distro is crazy IMHO.

> My reservation is against doing it in the kernel. I do not particularly
> care _how_ its done in user-space.

I'm sorry to appear foolish, but as explained above, I genuinely don't 
understand why this does not belong in the kernel. I would be grateful for 
elaboration. If I really am being thick, then just ignore me and I'll just 
solve this for myself using route 4.

In any case, thanks for all the patience and kind suggestions so far - it is 
appreciated.

Regards, Shaheed

Re: 2.6 must-fix list, v2

[Robert Love]

On Thu, 2003-05-15 at 16:07, shaheed wrote:

> I have no idea with whom to persue this path, and as I say, I feel that 
> solving this once for each distro is crazy IMHO.

It does not have to be done for each distribution. Modify the SysVinit
package directly to support this feature. All init needs to do is bind
itself to the allowed processors prior to its first fork(). This can be
done as part of the core init package, and thus all distributions
automatically reap the benefits.

If init is not modified, then it can be done in rc.d or wherever by
hand.

> I'm sorry to appear foolish, but as explained above, I genuinely don't 
> understand why this does not belong in the kernel. I would be grateful for 
> elaboration. If I really am being thick, then just ignore me and I'll just 
> solve this for myself using route 4.

Things which can be done in user-space should be done in user-space.
There is absolutely zero reason to do this in the kernel.  init can do
it.

Submit a patch to the init maintainer to have it bind itself on boot to
a given command line value. Maybe I will do this if I find the time...

	Robert Love

Re: 2.6 must-fix list, v2

[Robert Love]

On Thu, 2003-05-15 at 16:07, shaheed wrote:

> I'm sorry to appear foolish, but as explained above, I genuinely don't 
> understand why this does not belong in the kernel. I would be grateful for 
> elaboration. If I really am being thick, then just ignore me and I'll just 
> solve this for myself using route 4.

Oh, one other problem with doing it in the kernel via INIT_TASK:

You end up affining any kernel threads, which you absolutely do not want
to do _implicitly_. Maybe explicitly, but certainly not implicitly as a
blind consequence.

Doing it via init is really the way to go.

Regards,

	Robert Love

Re: 2.6 must-fix list, v2

[shaheed]

On Thursday 15 May 2003 9:24 pm, Robert Love wrote:

> Oh, one other problem with doing it in the kernel via INIT_TASK:
>
> You end up affining any kernel threads, which you absolutely do not want
> to do _implicitly_. Maybe explicitly, but certainly not implicitly as a
> blind consequence.
>
> Doing it via init is really the way to go.

OK, that does make sense.

>Submit a patch to the init maintainer to have it bind itself on boot to
>a given command line value. Maybe I will do this if I find the time...

I will have a go myself too...

[srhaque@chiswick srhaque]$ rpm -q --whatprovides /sbin/init
SysVinit-2.84-2mdk

and rpmfind.net points to ftp://ftp.cistron.nl/pub/people/miquels/software. 
I'll drop you a line if I make progress.

Thanks, Shaheed

[HOWTO] Emulate processor sets (pset) for linux kernel 2.5/2.6

[\"shaheed r. haque\"]

A while ago, I was banging on about kernel patches and whatnot to implement the 
equivalent of "pset" functionality. After some shoves/pointers from Robert and 
Miquel, I found that init(8) has the concept of an initscript(5), so that the 
trivial solution to my problem is to say:

#
# initscript   Executed by init(8) for every program it
#              wants to spawn like this:
#
#              /bin/sh /etc/initscript <id> <level> <action> <process>
#
taskset -p 1 $$
eval exec "$4"

and then modify the relevant programs to set their affinity to the other 
processors as required. Bingo!

Hope this helps, Shaheed