[PATCH 0/3] scheduler include file reorganization

[PATCH 0/3] scheduler include file reorganization

[Clark Williams]

[-- Attachment #1: Type: text/plain, Size: 2045 bytes --]


Ingo,

Is this more in line with what you wanted? 

I moved the sysctl bits out into include/linux/sched/sysctl.h, then
updated any source dependent on sysctl bits, then added my SCHED_RR
tuning knob, then finally created include/linux/sched/rt.h to hold
most of the rt scheduler specific bits. 

Clark Williams (3):
  sched: move sched.h sysctl bits into separate header
  sched/rt: add a tuning knob to allow changing SCHED_RR timeslice
  sched/rt: move rt specific bits into new header file

 block/blk-exec.c                  |   1 +
 drivers/spi/spi.c                 |   2 +-
 drivers/staging/csr/bh.c          |   2 +-
 drivers/staging/csr/unifi_sme.c   |   2 +-
 drivers/tty/sysrq.c               |   1 +
 fs/select.c                       |   1 +
 include/linux/sched.h             | 144 +-------------------------------------
 include/linux/sched/rt.h          |  58 +++++++++++++++
 include/linux/sched/sysctl.h      | 110 +++++++++++++++++++++++++++++
 init/init_task.c                  |   2 +
 kernel/futex.c                    |   1 +
 kernel/hrtimer.c                  |   2 +
 kernel/irq/manage.c               |   1 +
 kernel/rtmutex-debug.c            |   1 +
 kernel/rtmutex-tester.c           |   1 +
 kernel/rtmutex.c                  |   1 +
 kernel/sched/core.c               |  19 +++++
 kernel/sched/cpupri.c             |   2 +
 kernel/sched/rt.c                 |   6 +-
 kernel/sched/sched.h              |   2 +
 kernel/sysctl.c                   |   8 +++
 kernel/timer.c                    |   1 +
 kernel/trace/trace.c              |   1 +
 kernel/trace/trace_sched_wakeup.c |   2 +-
 kernel/watchdog.c                 |   1 +
 mm/mmap.c                         |   1 +
 mm/mremap.c                       |   1 +
 mm/page-writeback.c               |   1 +
 mm/page_alloc.c                   |   1 +
 29 files changed, 227 insertions(+), 149 deletions(-)
 create mode 100644 include/linux/sched/rt.h
 create mode 100644 include/linux/sched/sysctl.h

-- 
1.8.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH 0/3] scheduler include file reorganization

[Ingo Molnar]

* Clark Williams <williams@redhat.com> wrote:

> 
> Ingo,
> 
> Is this more in line with what you wanted? 
> 
> I moved the sysctl bits out into include/linux/sched/sysctl.h, then
> updated any source dependent on sysctl bits, then added my SCHED_RR
> tuning knob, then finally created include/linux/sched/rt.h to hold
> most of the rt scheduler specific bits. 
> 
> Clark Williams (3):
>   sched: move sched.h sysctl bits into separate header
>   sched/rt: add a tuning knob to allow changing SCHED_RR timeslice
>   sched/rt: move rt specific bits into new header file
> 
>  block/blk-exec.c                  |   1 +
>  drivers/spi/spi.c                 |   2 +-
>  drivers/staging/csr/bh.c          |   2 +-
>  drivers/staging/csr/unifi_sme.c   |   2 +-
>  drivers/tty/sysrq.c               |   1 +
>  fs/select.c                       |   1 +
>  include/linux/sched.h             | 144 +-------------------------------------
>  include/linux/sched/rt.h          |  58 +++++++++++++++
>  include/linux/sched/sysctl.h      | 110 +++++++++++++++++++++++++++++
>  init/init_task.c                  |   2 +
>  kernel/futex.c                    |   1 +
>  kernel/hrtimer.c                  |   2 +
>  kernel/irq/manage.c               |   1 +
>  kernel/rtmutex-debug.c            |   1 +
>  kernel/rtmutex-tester.c           |   1 +
>  kernel/rtmutex.c                  |   1 +
>  kernel/sched/core.c               |  19 +++++
>  kernel/sched/cpupri.c             |   2 +
>  kernel/sched/rt.c                 |   6 +-
>  kernel/sched/sched.h              |   2 +
>  kernel/sysctl.c                   |   8 +++
>  kernel/timer.c                    |   1 +
>  kernel/trace/trace.c              |   1 +
>  kernel/trace/trace_sched_wakeup.c |   2 +-
>  kernel/watchdog.c                 |   1 +
>  mm/mmap.c                         |   1 +
>  mm/mremap.c                       |   1 +
>  mm/page-writeback.c               |   1 +
>  mm/page_alloc.c                   |   1 +
>  29 files changed, 227 insertions(+), 149 deletions(-)
>  create mode 100644 include/linux/sched/rt.h
>  create mode 100644 include/linux/sched/sysctl.h

Yeah, that looks pretty good.

I'll give it some testing.

Thanks,

	Ingo

Re: [PATCH 0/3] scheduler include file reorganization

[Ingo Molnar]

* Ingo Molnar <mingo@kernel.org> wrote:

> Yeah, that looks pretty good.
> 
> I'll give it some testing.

Found one build error so far, applied the fix below.

Thanks,

	Ingo

---
 kernel/mutex.c |    1 +
 1 file changed, 1 insertion(+)

Index: linux/kernel/mutex.c
===================================================================
--- linux.orig/kernel/mutex.c
+++ linux/kernel/mutex.c
@@ -19,6 +19,7 @@
  */
 #include <linux/mutex.h>
 #include <linux/sched.h>
+#include <linux/sched/rt.h>
 #include <linux/export.h>
 #include <linux/spinlock.h>
 #include <linux/interrupt.h>

Re: [PATCH 0/3] scheduler include file reorganization

[Ingo Molnar]

* Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Ingo Molnar <mingo@kernel.org> wrote:
> 
> > Yeah, that looks pretty good.
> > 
> > I'll give it some testing.
> 
> Found one build error so far, applied the fix below.

Another one was in NOMMU mode (fixed below) - a hidden sched.h 
include file dependency in mm/nommu.c.

It's looking good otherwise!

Thanks,

	Ingo

--------->
diff --git a/mm/nommu.c b/mm/nommu.c
index 79c3cac..b20db4e 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -29,6 +29,7 @@
 #include <linux/security.h>
 #include <linux/syscalls.h>
 #include <linux/audit.h>
+#include <linux/sched/sysctl.h>
 
 #include <asm/uaccess.h>
 #include <asm/tlb.h>

Re: [PATCH 0/3] scheduler include file reorganization

[Clark Williams]

[-- Attachment #1: Type: text/plain, Size: 702 bytes --]

On Thu, 7 Feb 2013 20:52:57 +0100
Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Ingo Molnar <mingo@kernel.org> wrote:
> 
> > 
> > * Ingo Molnar <mingo@kernel.org> wrote:
> > 
> > > Yeah, that looks pretty good.
> > > 
> > > I'll give it some testing.
> > 
> > Found one build error so far, applied the fix below.
> 
> Another one was in NOMMU mode (fixed below) - a hidden sched.h 
> include file dependency in mm/nommu.c.
> 
> It's looking good otherwise!
> 
> Thanks,
> 
> 	Ingo
>

Huh, I did a build with an allmodconfig config file. You must be doing
some odd permutations. 

I have both of your fixes applied, send me any more you find and I'll
regenerate. 

Clark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH 0/3] scheduler include file reorganization

[Ingo Molnar]

* Clark Williams <williams@redhat.com> wrote:

> On Thu, 7 Feb 2013 20:52:57 +0100
> Ingo Molnar <mingo@kernel.org> wrote:
> 
> > 
> > * Ingo Molnar <mingo@kernel.org> wrote:
> > 
> > > 
> > > * Ingo Molnar <mingo@kernel.org> wrote:
> > > 
> > > > Yeah, that looks pretty good.
> > > > 
> > > > I'll give it some testing.
> > > 
> > > Found one build error so far, applied the fix below.
> > 
> > Another one was in NOMMU mode (fixed below) - a hidden sched.h 
> > include file dependency in mm/nommu.c.
> > 
> > It's looking good otherwise!
> > 
> > Thanks,
> > 
> > 	Ingo
> >
> 
> Huh, I did a build with an allmodconfig config file. You must 
> be doing some odd permutations.

Yeah, randconfig coverage and cross-arch build coverage. One of 
the build errors triggered on allnoconfig, the other on the 
blackfin architecture.

> I have both of your fixes applied, send me any more you find 
> and I'll regenerate.

No need, I fixed your series locally and pushed out the result.

Wanna split out some other parts of sched.h? ;-) I can do the 
build coverage testing part for those as well.

Thanks,

	Ingo

Re: [PATCH 0/3] scheduler include file reorganization

[Clark Williams]

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]

On Fri, 8 Feb 2013 15:18:41 +0100
Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Clark Williams <williams@redhat.com> wrote:
> 
> > On Thu, 7 Feb 2013 20:52:57 +0100
> > Ingo Molnar <mingo@kernel.org> wrote:
> > 
> > > 
> > > * Ingo Molnar <mingo@kernel.org> wrote:
> > > 
> > > > 
> > > > * Ingo Molnar <mingo@kernel.org> wrote:
> > > > 
> > > > > Yeah, that looks pretty good.
> > > > > 
> > > > > I'll give it some testing.
> > > > 
> > > > Found one build error so far, applied the fix below.
> > > 
> > > Another one was in NOMMU mode (fixed below) - a hidden sched.h 
> > > include file dependency in mm/nommu.c.
> > > 
> > > It's looking good otherwise!
> > > 
> > > Thanks,
> > > 
> > > 	Ingo
> > >
> > 
> > Huh, I did a build with an allmodconfig config file. You must 
> > be doing some odd permutations.
> 
> Yeah, randconfig coverage and cross-arch build coverage. One of 
> the build errors triggered on allnoconfig, the other on the 
> blackfin architecture.
> 
> > I have both of your fixes applied, send me any more you find 
> > and I'll regenerate.
> 
> No need, I fixed your series locally and pushed out the result.
> 
> Wanna split out some other parts of sched.h? ;-) I can do the 
> build coverage testing part for those as well.
> 
> Thanks,
> 
> 	Ingo


I figured that was coming. :)

I'll look at it again and see about pulling the autogroup/cgroup stuff
into it's own header. After that it's probably going to require some
serious changes. 

Any suggestions?

Clark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH 0/3] scheduler include file reorganization

[Ingo Molnar]

* Clark Williams <williams@redhat.com> wrote:

> I figured that was coming. :)

;-)

> I'll look at it again and see about pulling the 
> autogroup/cgroup stuff into it's own header. After that it's 
> probably going to require some serious changes.
> 
> Any suggestions?

I'd suggest doing it as finegrained as possible - potentially 
one concept at a time. I wouldn't mind a dozen small files in 
include/linux/sched/ - possibly more.

In the end sched.h would include core wakeup/sleep methods that 
tons of drivers rely on, and it would include the 'struct 
task_struct' data type definition (and all its prereqs), which 
we rely on in tons of drivers as well.

Not much else should remain in sched.h - in theory :-)

In terms of build coverage: just build an x86 defconfig with 
perhaps the specific sub-feature (such as autogroups/cgroups) 
turned off/on - I'd suggest for you to not even do allmodconfig 
testing (which is really slow unless you have a cluster of build 
machines), I can test all that and more and fix the fallout 
before applying it.

Thanks,

	Ingo

Re: [PATCH 0/3] scheduler include file reorganization

[Namhyung Kim]

Hi,

On Mon, 11 Feb 2013 10:54:58 +0100, Ingo Molnar wrote:
> * Clark Williams <williams@redhat.com> wrote:
>
>> I figured that was coming. :)
>
> ;-)
>
>> I'll look at it again and see about pulling the 
>> autogroup/cgroup stuff into it's own header. After that it's 
>> probably going to require some serious changes.
>> 
>> Any suggestions?
>
> I'd suggest doing it as finegrained as possible - potentially 
> one concept at a time. I wouldn't mind a dozen small files in 
> include/linux/sched/ - possibly more.

What about the .c files?  AFAICS the sched/core.c and sched/fair.c are
rather huge and contain various concepts which might be separated to
their own files.  It'd be better reorganizing them too IMHO.

Thanks,
Namhyung

Re: [PATCH 0/3] scheduler include file reorganization

[Ingo Molnar]

* Namhyung Kim <namhyung@kernel.org> wrote:

> Hi,
> 
> On Mon, 11 Feb 2013 10:54:58 +0100, Ingo Molnar wrote:
> > * Clark Williams <williams@redhat.com> wrote:
> >
> >> I figured that was coming. :)
> >
> > ;-)
> >
> >> I'll look at it again and see about pulling the 
> >> autogroup/cgroup stuff into it's own header. After that it's 
> >> probably going to require some serious changes.
> >> 
> >> Any suggestions?
> >
> > I'd suggest doing it as finegrained as possible - potentially 
> > one concept at a time. I wouldn't mind a dozen small files in 
> > include/linux/sched/ - possibly more.
> 
> What about the .c files?  AFAICS the sched/core.c and 
> sched/fair.c are rather huge and contain various concepts 
> which might be separated to their own files.  It'd be better 
> reorganizing them too IMHO.

I'd be more careful about those, because there's various 
scheduler patch-sets floating modifying them.

sched.h is much more static and it is the one that actually gets 
included in like 60% of all *other* .c files, adding a few 
thousand lines to every .o compilation and causing measurable 
compile time overhead ...

So sched.h splitting is something we should really do, if 
there's people interested in and capable of pulling it off.

Thanks,

	Ingo

Re: [PATCH 0/3] scheduler include file reorganization

[Clark Williams]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

On Wed, 13 Feb 2013 10:15:12 +0100
Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Namhyung Kim <namhyung@kernel.org> wrote:
> 
> > Hi,
> > 
> > On Mon, 11 Feb 2013 10:54:58 +0100, Ingo Molnar wrote:
> > > * Clark Williams <williams@redhat.com> wrote:
> > >
> > >> I figured that was coming. :)
> > >
> > > ;-)
> > >
> > >> I'll look at it again and see about pulling the 
> > >> autogroup/cgroup stuff into it's own header. After that it's 
> > >> probably going to require some serious changes.
> > >> 
> > >> Any suggestions?
> > >
> > > I'd suggest doing it as finegrained as possible - potentially 
> > > one concept at a time. I wouldn't mind a dozen small files in 
> > > include/linux/sched/ - possibly more.
> > 
> > What about the .c files?  AFAICS the sched/core.c and 
> > sched/fair.c are rather huge and contain various concepts 
> > which might be separated to their own files.  It'd be better 
> > reorganizing them too IMHO.
> 
> I'd be more careful about those, because there's various 
> scheduler patch-sets floating modifying them.
> 
> sched.h is much more static and it is the one that actually gets 
> included in like 60% of all *other* .c files, adding a few 
> thousand lines to every .o compilation and causing measurable 
> compile time overhead ...
> 
> So sched.h splitting is something we should really do, if 
> there's people interested in and capable of pulling it off.
> 
> Thanks,
> 
> 	Ingo


And since I'm one of the people that care about the RT patch (which
modifies the scheduler files) I'll just start with baby steps and reorg
the headers. 

Clark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH 0/3] scheduler include file reorganization

[Namhyung Kim]

On Wed, 13 Feb 2013 09:19:37 -0600, Clark Williams wrote:
> On Wed, 13 Feb 2013 10:15:12 +0100
> Ingo Molnar <mingo@kernel.org> wrote:
>> * Namhyung Kim <namhyung@kernel.org> wrote:
>> > On Mon, 11 Feb 2013 10:54:58 +0100, Ingo Molnar wrote:
>> > > * Clark Williams <williams@redhat.com> wrote:
>> > >
>> > >> I figured that was coming. :)
>> > >
>> > > ;-)
>> > >
>> > >> I'll look at it again and see about pulling the 
>> > >> autogroup/cgroup stuff into it's own header. After that it's 
>> > >> probably going to require some serious changes.
>> > >> 
>> > >> Any suggestions?
>> > >
>> > > I'd suggest doing it as finegrained as possible - potentially 
>> > > one concept at a time. I wouldn't mind a dozen small files in 
>> > > include/linux/sched/ - possibly more.
>> > 
>> > What about the .c files?  AFAICS the sched/core.c and 
>> > sched/fair.c are rather huge and contain various concepts 
>> > which might be separated to their own files.  It'd be better 
>> > reorganizing them too IMHO.
>> 
>> I'd be more careful about those, because there's various 
>> scheduler patch-sets floating modifying them.
>> 
>> sched.h is much more static and it is the one that actually gets 
>> included in like 60% of all *other* .c files, adding a few 
>> thousand lines to every .o compilation and causing measurable 
>> compile time overhead ...
>> 
>> So sched.h splitting is something we should really do, if 
>> there's people interested in and capable of pulling it off.
>
> And since I'm one of the people that care about the RT patch (which
> modifies the scheduler files) I'll just start with baby steps and reorg
> the headers. 

Understood.  Thanks for the explanation!

Thanks,
Namhyung

Re: [PATCH 0/3] scheduler include file reorganization

[Li Zefan]

On 2013/2/13 17:15, Ingo Molnar wrote:
> 
> * Namhyung Kim <namhyung@kernel.org> wrote:
> 
>> Hi,
>>
>> On Mon, 11 Feb 2013 10:54:58 +0100, Ingo Molnar wrote:
>>> * Clark Williams <williams@redhat.com> wrote:
>>>
>>>> I figured that was coming. :)
>>>
>>> ;-)
>>>
>>>> I'll look at it again and see about pulling the 
>>>> autogroup/cgroup stuff into it's own header. After that it's 
>>>> probably going to require some serious changes.
>>>>
>>>> Any suggestions?
>>>
>>> I'd suggest doing it as finegrained as possible - potentially 
>>> one concept at a time. I wouldn't mind a dozen small files in 
>>> include/linux/sched/ - possibly more.
>>
>> What about the .c files?  AFAICS the sched/core.c and 
>> sched/fair.c are rather huge and contain various concepts 
>> which might be separated to their own files.  It'd be better 
>> reorganizing them too IMHO.
> 
> I'd be more careful about those, because there's various 
> scheduler patch-sets floating modifying them.
> 
> sched.h is much more static and it is the one that actually gets 
> included in like 60% of all *other* .c files, adding a few 
> thousand lines to every .o compilation and causing measurable 
> compile time overhead ...
> 
> So sched.h splitting is something we should really do, if 
> there's people interested in and capable of pulling it off.
> 

While previously working on a cgroup patchset that also touched sched.h,
I noticed some lines can be moved to kernel/sched/sched.h. I've cooked
up a patchset to do that, and it results in reduction of 200+ lines in
sched.h. I'll do some compile testing before sending it out.