[PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Leonardo Brás]

Special thanks for the feedback from:
- Finn Thain (I fixed the build problem)
- Geert Uytterhoeven (The cross compilers were very useful)
- Rolf Eike Beer (Was unintentional, thanks for the help!)

This Patchset changes some driver's Makefile to allow them building
using the command 'make drivers/DRIVERNAME', if compatible.

The changed drivers would return error if the above command was run
on them, after an x86 allyesconfig.

The main reason of this patchset is to allow building lists of
drivers looking for warnings and errors to be fixed.

I see this change as a new feature, not a bugfix. I understand
the default bahavior may be building with a simple 'make', but I
believe adding this new possibility will not be harmful.

My main objective is to allow developers with low processing power
to do changes in the kernel and look bugs using free services like
GiltabCI, before submitting to community.

If there is any interest helping/using this, I have a prototype in:
https://gitlab.com/LeoBras/linux-next


Leonardo Brás (7):
  drivers: dio: Avoids building driver if CONFIG_DIO is disabled
  drivers: nubus: Avoids building driver if CONFIG_NUBUS is disabled
  drivers: parisc: Avoids building driver if CONFIG_PARISC is disabled
  drivers: zorro: Avoids building proc.o if CONFIG_ZORRO is disabled
  drivers: s390: Avoids building drivers if ARCH is not s390.
  drivers: oprofile: Avoids building driver from direct make command
  drivers: hwtracing: Adds Makefile to enable building from directory.

 drivers/Makefile           | 4 +---
 drivers/dio/Makefile       | 2 +-
 drivers/hwtracing/Makefile | 3 +++
 drivers/nubus/Makefile     | 5 +++--
 drivers/oprofile/Makefile  | 1 +
 drivers/parisc/Makefile    | 2 +-
 drivers/s390/Makefile      | 8 ++++----
 drivers/zorro/Makefile     | 7 ++++---
 8 files changed, 18 insertions(+), 14 deletions(-)
 create mode 100644 drivers/hwtracing/Makefile
 create mode 100644 drivers/oprofile/Makefile

-- 
2.19.0

[PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Leonardo Brás]

Special thanks for the feedback from:
- Finn Thain (I fixed the build problem)
- Geert Uytterhoeven (The cross compilers were very useful)
- Rolf Eike Beer (Was unintentional, thanks for the help!)

This Patchset changes some driver's Makefile to allow them building
using the command 'make drivers/DRIVERNAME', if compatible.

The changed drivers would return error if the above command was run
on them, after an x86 allyesconfig.

The main reason of this patchset is to allow building lists of
drivers looking for warnings and errors to be fixed.

I see this change as a new feature, not a bugfix. I understand
the default bahavior may be building with a simple 'make', but I
believe adding this new possibility will not be harmful.

My main objective is to allow developers with low processing power
to do changes in the kernel and look bugs using free services like
GiltabCI, before submitting to community.

If there is any interest helping/using this, I have a prototype in:
https://gitlab.com/LeoBras/linux-next


Leonardo Brás (7):
  drivers: dio: Avoids building driver if CONFIG_DIO is disabled
  drivers: nubus: Avoids building driver if CONFIG_NUBUS is disabled
  drivers: parisc: Avoids building driver if CONFIG_PARISC is disabled
  drivers: zorro: Avoids building proc.o if CONFIG_ZORRO is disabled
  drivers: s390: Avoids building drivers if ARCH is not s390.
  drivers: oprofile: Avoids building driver from direct make command
  drivers: hwtracing: Adds Makefile to enable building from directory.

 drivers/Makefile           | 4 +---
 drivers/dio/Makefile       | 2 +-
 drivers/hwtracing/Makefile | 3 +++
 drivers/nubus/Makefile     | 5 +++--
 drivers/oprofile/Makefile  | 1 +
 drivers/parisc/Makefile    | 2 +-
 drivers/s390/Makefile      | 8 ++++----
 drivers/zorro/Makefile     | 7 ++++---
 8 files changed, 18 insertions(+), 14 deletions(-)
 create mode 100644 drivers/hwtracing/Makefile
 create mode 100644 drivers/oprofile/Makefile

-- 
2.19.0

Re: [PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Robert Richter]

On 27.09.18 23:08:09, Leonardo Brás wrote:
> This Patchset changes some driver's Makefile to allow them building
> using the command 'make drivers/DRIVERNAME', if compatible.
> 
> The changed drivers would return error if the above command was run
> on them, after an x86 allyesconfig.

I don't see what you are trying to achieve here. Why shouldn't the
command fail if it is not the intended way to call it? There are a
couple of use cases where drivers/ is used to share common code over
different archs and it is not always the intention to build them in
drivers/ directly.

> 
> The main reason of this patchset is to allow building lists of
> drivers looking for warnings and errors to be fixed.

If a list is the intention here, aren't there other ways to create it
other than using drivers/*?

-Robert

> 
> I see this change as a new feature, not a bugfix. I understand
> the default bahavior may be building with a simple 'make', but I
> believe adding this new possibility will not be harmful.
> 
> My main objective is to allow developers with low processing power
> to do changes in the kernel and look bugs using free services like
> GiltabCI, before submitting to community.
> 
> If there is any interest helping/using this, I have a prototype in:
> https://gitlab.com/LeoBras/linux-next

Re: [PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Leonardo Bras]

On Mon, Oct 1, 2018 at 4:56 AM Robert Richter <rric@kernel.org> wrote:
>
> On 27.09.18 23:08:09, Leonardo Br=C3=A1s wrote:
> > This Patchset changes some driver's Makefile to allow them building
> > using the command 'make drivers/DRIVERNAME', if compatible.
> >
> > The changed drivers would return error if the above command was run
> > on them, after an x86 allyesconfig.
>
> I don't see what you are trying to achieve here. Why shouldn't the
> command fail if it is not the intended way to call it? There are a
> couple of use cases where drivers/ is used to share common code over
> different archs and it is not always the intention to build them in
> drivers/ directly.


Sorry, I was not very clear at my reasons why this change is important,
I will try to briefly explain the whole story.

Some weeks ago I was trying to solve a task that needed to change some
compiling options, build the whole kernel (allyesconfig) and look for error=
s.
The problem was: It would take a long time to build everything in my comput=
er.
And many friends with slimmer laptops would take much longer.

So, I was looking for a service that could do that for me, in the cloud.
I found out Gitlab.com offers free 50k minutes of CI for open source projec=
ts,
and allows anyone get this CI time by only forking my project, adding their
changes and pushing to Gitlab.

But Gitlab don't allow 'jobs' to take longer than 3 hours, after that the '=
job'
is killed.

The kernel could not be built in 3h, not with allyesconfig. So, I created a
'job' for each directory in Linux kernel, and tried to build them separatel=
y.
All went fine, except for drivers/, which took over 3 hours.

Most logical thing was to continue the division and create five 'jobs' that
could divide the building time of drivers. To do that, each job took care o=
f a
range of starting letters, as you can see in this link:
https://gitlab.com/LeoBras/linux-next/blob/build-ci/.gitlab-ci.yml

But then I found out some drivers were failing to build. Even if they were =
not
enabled in my .config. After some work I found out some drivers selection i=
s
done in drivers/Makefile, and incompatible drivers would break my build if
tried to call them directly on drivers/DRIVERNAME.

This patchset intents to let the .config selection happen also in
drivers/DRIVERNAME/Makefile, avoiding accidentally building drivers
that are not in .config.

This would allow the kernel to be build on Gitlab CI, and would benefit man=
y
people who wants to help in the kernel development, but have not much
processing power in their machines.

I understand my changes may have mistakes, and I am trying to fix them all.
I thank you any suggestion to make the code better.

Also, I would be happy to know of any other solution to remote build my cha=
nges
and look for warnkings / errors.


Thank you for reading,
Leonardo Bras

Re: [PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Leonardo Bras]

On Mon, Oct 1, 2018 at 4:56 AM Robert Richter <rric@kernel.org> wrote:
>
> On 27.09.18 23:08:09, Leonardo Brás wrote:
> > This Patchset changes some driver's Makefile to allow them building
> > using the command 'make drivers/DRIVERNAME', if compatible.
> >
> > The changed drivers would return error if the above command was run
> > on them, after an x86 allyesconfig.
>
> I don't see what you are trying to achieve here. Why shouldn't the
> command fail if it is not the intended way to call it? There are a
> couple of use cases where drivers/ is used to share common code over
> different archs and it is not always the intention to build them in
> drivers/ directly.


Sorry, I was not very clear at my reasons why this change is important,
I will try to briefly explain the whole story.

Some weeks ago I was trying to solve a task that needed to change some
compiling options, build the whole kernel (allyesconfig) and look for errors.
The problem was: It would take a long time to build everything in my computer.
And many friends with slimmer laptops would take much longer.

So, I was looking for a service that could do that for me, in the cloud.
I found out Gitlab.com offers free 50k minutes of CI for open source projects,
and allows anyone get this CI time by only forking my project, adding their
changes and pushing to Gitlab.

But Gitlab don't allow 'jobs' to take longer than 3 hours, after that the 'job'
is killed.

The kernel could not be built in 3h, not with allyesconfig. So, I created a
'job' for each directory in Linux kernel, and tried to build them separately.
All went fine, except for drivers/, which took over 3 hours.

Most logical thing was to continue the division and create five 'jobs' that
could divide the building time of drivers. To do that, each job took care of a
range of starting letters, as you can see in this link:
https://gitlab.com/LeoBras/linux-next/blob/build-ci/.gitlab-ci.yml

But then I found out some drivers were failing to build. Even if they were not
enabled in my .config. After some work I found out some drivers selection is
done in drivers/Makefile, and incompatible drivers would break my build if
tried to call them directly on drivers/DRIVERNAME.

This patchset intents to let the .config selection happen also in
drivers/DRIVERNAME/Makefile, avoiding accidentally building drivers
that are not in .config.

This would allow the kernel to be build on Gitlab CI, and would benefit many
people who wants to help in the kernel development, but have not much
processing power in their machines.

I understand my changes may have mistakes, and I am trying to fix them all.
I thank you any suggestion to make the code better.

Also, I would be happy to know of any other solution to remote build my changes
and look for warnkings / errors.


Thank you for reading,
Leonardo Bras

Re: [PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Finn Thain]

On Wed, 3 Oct 2018, Leonardo Bras wrote:

> 
> Sorry, I was not very clear at my reasons why this change is important, 
> I will try to briefly explain the whole story.
> 
> Some weeks ago I was trying to solve a task that needed to change some 
> compiling options, build the whole kernel (allyesconfig) and look for 
> errors. The problem was: It would take a long time to build everything 
> in my computer. And many friends with slimmer laptops would take much 
> longer.
> 

It seems to me that you shouldn't need expensive optimization for 
continuous integration. In theory that could make a big difference though 
I admit to no experience of build farms.

Have you looked at ccache? It will save you from having to recompile any 
files not changed.

You could also leverage all of your laptops together using distcc.

HTH.

-- 

Re: [PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Leonardo Bras]

On Wed, Oct 3, 2018 at 8:27 PM Finn Thain <fthain@telegraphics.com.au> wrote:
>
> On Wed, 3 Oct 2018, Leonardo Bras wrote:
>
> >
> > Sorry, I was not very clear at my reasons why this change is important,
> > I will try to briefly explain the whole story.
> >
> > Some weeks ago I was trying to solve a task that needed to change some
> > compiling options, build the whole kernel (allyesconfig) and look for
> > errors. The problem was: It would take a long time to build everything
> > in my computer. And many friends with slimmer laptops would take much
> > longer.
> >
>
> It seems to me that you shouldn't need expensive optimization for
> continuous integration. In theory that could make a big difference though
> I admit to no experience of build farms.
>
> Have you looked at ccache? It will save you from having to recompile any
> files not changed.
>
> You could also leverage all of your laptops together using distcc.
>
> HTH.
>
> --

Both ccache and distcc seem very interesting,
I will take my time to study them better as they can solve some situations
I face. Thanks for sharing!

Still, I believe my patchset is relevant, as it would enable people to rebuild
the whole kernel (with build flags changes, for example) at the cost of only
git-pushing the changes to a fork of my Gitlab repo, and waiting 3 hours.

Thanks for helping!

Re: [PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Finn Thain]

On Wed, 3 Oct 2018, Leonardo Bras wrote:

> Both ccache and distcc seem very interesting, I will take my time to 
> study them better as they can solve some situations I face. Thanks for 
> sharing!
> 

You might also want to check out 'gcc -O0', 'gcc -fopt-info' and 'gcc 
--help=optimizers' etc to see if you can reduce the compute cost.

To reduce IO cost, my build tests always use 'make O=/some/path' where 
/some/path is on a tmpfs mountpoint.

HTH.

-- 

Re: [PATCH v3 0/7] Remove errors building drivers/DRIVERNAME

[Leonardo Bras]

Thanks Finn, I will take a good look at that and try to use it in my build.

Thank you,
Leonardo Bras

On Wed, Oct 3, 2018 at 11:00 PM Finn Thain <fthain@telegraphics.com.au> wrote:
>
> On Wed, 3 Oct 2018, Leonardo Bras wrote:
>
> > Both ccache and distcc seem very interesting, I will take my time to
> > study them better as they can solve some situations I face. Thanks for
> > sharing!
> >
>
> You might also want to check out 'gcc -O0', 'gcc -fopt-info' and 'gcc
> --help=optimizers' etc to see if you can reduce the compute cost.
>
> To reduce IO cost, my build tests always use 'make O=/some/path' where
> /some/path is on a tmpfs mountpoint.
>
> HTH.
>
> --