Kbuild and Kconfig

Kbuild and Kconfig

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 1350 bytes --]

I just wanted to bring this to a top level post since Jonathan Creekmore
and myself have talked with a few maintainers in different threads and
on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
I would like to get a rough idea on what the Xen community wants the
system to look like before starting work on it to both save myself time
and save maintainers review cycles. So that being said rough proposal as
follows:

* target only the xen/ directory tree (i.e. not the toolstack, stubdoms
or docs)
* split top level config bits to not affect xen/ tree (currently only
XSM_ENABLE / FLASK_ENABLE do)
* convert xen/ to Kbuild first and merge this in (since Kconfig relies
on Kbuild-y bits which can be undone but if we're going to go to Kbuild
in the end why undo it and then redo it)
* convert existing xen/ config bits into Kconfig and merge that in

Jonathan and I, in a former life, converted a project to Kbuild and
Kconfig successfully. I have looked at starting with
https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
old it does separate out the build bits from the Linux specific bits
pretty nicely while removing module support which arguably is the most
complicated part. Alternatively we could start with Linux 4.2 if that's
more desirable.

Thanks.
-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Kbuild and Kconfig

[Andrew Cooper]

On 02/09/15 18:50, Doug Goldstein wrote:
> I just wanted to bring this to a top level post since Jonathan Creekmore
> and myself have talked with a few maintainers in different threads and
> on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
> I would like to get a rough idea on what the Xen community wants the
> system to look like before starting work on it to both save myself time
> and save maintainers review cycles. So that being said rough proposal as
> follows:
>
> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
> or docs)
> * split top level config bits to not affect xen/ tree (currently only
> XSM_ENABLE / FLASK_ENABLE do)
> * convert xen/ to Kbuild first and merge this in (since Kconfig relies
> on Kbuild-y bits which can be undone but if we're going to go to Kbuild
> in the end why undo it and then redo it)
> * convert existing xen/ config bits into Kconfig and merge that in
>
> Jonathan and I, in a former life, converted a project to Kbuild and
> Kconfig successfully. I have looked at starting with
> https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
> old it does separate out the build bits from the Linux specific bits
> pretty nicely while removing module support which arguably is the most
> complicated part. Alternatively we could start with Linux 4.2 if that's
> more desirable.

Thinking longterm, it would be nice to have xen, tools and stubdoms
covered by a system like this (I cant immediately see how any of this
would be usefully applied to docs/).

Therefore, so long as the eventual plan doesn't preclude this (and it
doesn't appear to), I have no specific concerns with this proposal.

I will defer to your judgement as to which is the correct path to take
to end up with a Kconfig system, as you clearly have more experience
than I have in this area.  If that means doing both Kconfig and Kbuild
at the same time as it is less net work, then fine.  (I will certainly
be happier with a more Kbuild-like system where I can sensibly do
out-of-tree builds).

~Andrew

Re: Kbuild and Kconfig

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 2696 bytes --]

On 9/2/15 1:29 PM, Andrew Cooper wrote:
> On 02/09/15 18:50, Doug Goldstein wrote:
>> I just wanted to bring this to a top level post since Jonathan Creekmore
>> and myself have talked with a few maintainers in different threads and
>> on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
>> I would like to get a rough idea on what the Xen community wants the
>> system to look like before starting work on it to both save myself time
>> and save maintainers review cycles. So that being said rough proposal as
>> follows:
>>
>> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
>> or docs)
>> * split top level config bits to not affect xen/ tree (currently only
>> XSM_ENABLE / FLASK_ENABLE do)
>> * convert xen/ to Kbuild first and merge this in (since Kconfig relies
>> on Kbuild-y bits which can be undone but if we're going to go to Kbuild
>> in the end why undo it and then redo it)
>> * convert existing xen/ config bits into Kconfig and merge that in
>>
>> Jonathan and I, in a former life, converted a project to Kbuild and
>> Kconfig successfully. I have looked at starting with
>> https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
>> old it does separate out the build bits from the Linux specific bits
>> pretty nicely while removing module support which arguably is the most
>> complicated part. Alternatively we could start with Linux 4.2 if that's
>> more desirable.
> 
> Thinking longterm, it would be nice to have xen, tools and stubdoms
> covered by a system like this (I cant immediately see how any of this
> would be usefully applied to docs/).
> 
> Therefore, so long as the eventual plan doesn't preclude this (and it
> doesn't appear to), I have no specific concerns with this proposal.
> 
> I will defer to your judgement as to which is the correct path to take
> to end up with a Kconfig system, as you clearly have more experience
> than I have in this area.  If that means doing both Kconfig and Kbuild
> at the same time as it is less net work, then fine.  (I will certainly
> be happier with a more Kbuild-like system where I can sensibly do
> out-of-tree builds).
> 
> ~Andrew
> 

Long term, yes we can tackle tools and stubdoms as well but initially I
was trying to eat the elephant one bite at a time. My idea was that we
would do xen/ first and you would have to go into that directory and run
"make defconfig/menuconfig/config/etc && make" inside that directory.
Once the other directories got converted we could move the setup to the
top level and include xen/, tools/, stubdom/. It should be mostly a
mechanical move.


-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Kbuild and Kconfig

[Andrew Cooper]

On 02/09/2015 21:28, Doug Goldstein wrote:
> On 9/2/15 1:29 PM, Andrew Cooper wrote:
>> On 02/09/15 18:50, Doug Goldstein wrote:
>>> I just wanted to bring this to a top level post since Jonathan Creekmore
>>> and myself have talked with a few maintainers in different threads and
>>> on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
>>> I would like to get a rough idea on what the Xen community wants the
>>> system to look like before starting work on it to both save myself time
>>> and save maintainers review cycles. So that being said rough proposal as
>>> follows:
>>>
>>> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
>>> or docs)
>>> * split top level config bits to not affect xen/ tree (currently only
>>> XSM_ENABLE / FLASK_ENABLE do)
>>> * convert xen/ to Kbuild first and merge this in (since Kconfig relies
>>> on Kbuild-y bits which can be undone but if we're going to go to Kbuild
>>> in the end why undo it and then redo it)
>>> * convert existing xen/ config bits into Kconfig and merge that in
>>>
>>> Jonathan and I, in a former life, converted a project to Kbuild and
>>> Kconfig successfully. I have looked at starting with
>>> https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
>>> old it does separate out the build bits from the Linux specific bits
>>> pretty nicely while removing module support which arguably is the most
>>> complicated part. Alternatively we could start with Linux 4.2 if that's
>>> more desirable.
>> Thinking longterm, it would be nice to have xen, tools and stubdoms
>> covered by a system like this (I cant immediately see how any of this
>> would be usefully applied to docs/).
>>
>> Therefore, so long as the eventual plan doesn't preclude this (and it
>> doesn't appear to), I have no specific concerns with this proposal.
>>
>> I will defer to your judgement as to which is the correct path to take
>> to end up with a Kconfig system, as you clearly have more experience
>> than I have in this area.  If that means doing both Kconfig and Kbuild
>> at the same time as it is less net work, then fine.  (I will certainly
>> be happier with a more Kbuild-like system where I can sensibly do
>> out-of-tree builds).
>>
>> ~Andrew
>>
> Long term, yes we can tackle tools and stubdoms as well but initially I
> was trying to eat the elephant one bite at a time. My idea was that we
> would do xen/ first and you would have to go into that directory and run
> "make defconfig/menuconfig/config/etc && make" inside that directory.
> Once the other directories got converted we could move the setup to the
> top level and include xen/, tools/, stubdom/. It should be mostly a
> mechanical move.
>
>

Sounds reasonable.

~Andrew

Re: Kbuild and Kconfig

[Ian Campbell]

On Wed, 2015-09-02 at 19:29 +0100, Andrew Cooper wrote:
> On 02/09/15 18:50, Doug Goldstein wrote:
> > I just wanted to bring this to a top level post since Jonathan 
> > Creekmore
> > and myself have talked with a few maintainers in different threads and
> > on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
> > I would like to get a rough idea on what the Xen community wants the
> > system to look like before starting work on it to both save myself time
> > and save maintainers review cycles. So that being said rough proposal 
> > as
> > follows:
> > 
> > * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
> > or docs)
> > * split top level config bits to not affect xen/ tree (currently only
> > XSM_ENABLE / FLASK_ENABLE do)
> > * convert xen/ to Kbuild first and merge this in (since Kconfig relies
> > on Kbuild-y bits which can be undone but if we're going to go to Kbuild
> > in the end why undo it and then redo it)
> > * convert existing xen/ config bits into Kconfig and merge that in
> > 
> > Jonathan and I, in a former life, converted a project to Kbuild and
> > Kconfig successfully. I have looked at starting with
> > https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
> > old it does separate out the build bits from the Linux specific bits
> > pretty nicely while removing module support which arguably is the most
> > complicated part. Alternatively we could start with Linux 4.2 if that's
> > more desirable.
> 
> Thinking longterm, it would be nice to have xen, tools and stubdoms
> covered by a system like this

Is the proposal here then to abandon autoconf for the tools subtree in
favour of Kconfig? Or maybe to somehow hybridize autoconf (for e.g. library
and feature detection) with Kconfig (for user selection of options)? I'm
not sure how I feel about either of those approaches, they certainly both
need careful consideration, and the second in particular regarding the
interactions...

FWIW it seems to me that the link between things which are optional in Xen
and which are optional in the tools is (or should be) pretty loose. i.e.
the tools today _always_ support XSM and correctly handle the errors from
Xen if it is not enabled there. Personally I think this is the right way to
do things. Likewise Xen doesn't care if the tools have particular opinions
on the qemu to use or whatever.

IOW I'm not sure have xen and tools use a common .config would make sense.

Ian.

Re: Kbuild and Kconfig

[Tim Deegan]

Hi,

At 12:50 -0500 on 02 Sep (1441198200), Doug Goldstein wrote:
> I just wanted to bring this to a top level post since Jonathan Creekmore
> and myself have talked with a few maintainers in different threads and
> on IRC about potentially using Kconfig and/or Kbuild for Xen.

If we're going to need a configure step before building the
hypervisor, maybe we should consider using the autoconf runes instead.

I'm one of the people who objected to requiring ./configure before
building the hypervisor, but I think it would be better than having
_two_ config systems which are not synced with each other, esp. if
we'll want to enable/disable features with matching tools-side code.

If we do go this way, I think we'd need a ./configure --hypervisor-only,
or similar, that _only_ makes the various CONFIG_ flags, and doesn't
do the tedious library & compiler checks that are needed for the
user-space code.

Cheers,

Tim.

Re: Kbuild and Kconfig

[Tim Deegan]

At 10:56 +0100 on 03 Sep (1441277769), Ian Campbell wrote:
> On Wed, 2015-09-02 at 19:29 +0100, Andrew Cooper wrote:
> > On 02/09/15 18:50, Doug Goldstein wrote:
> > > I just wanted to bring this to a top level post since Jonathan 
> > > Creekmore
> > > and myself have talked with a few maintainers in different threads and
> > > on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
> > > I would like to get a rough idea on what the Xen community wants the
> > > system to look like before starting work on it to both save myself time
> > > and save maintainers review cycles. So that being said rough proposal 
> > > as
> > > follows:
> > > 
> > > * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
> > > or docs)
> > > * split top level config bits to not affect xen/ tree (currently only
> > > XSM_ENABLE / FLASK_ENABLE do)
> > > * convert xen/ to Kbuild first and merge this in (since Kconfig relies
> > > on Kbuild-y bits which can be undone but if we're going to go to Kbuild
> > > in the end why undo it and then redo it)
> > > * convert existing xen/ config bits into Kconfig and merge that in
> > > 
> > > Jonathan and I, in a former life, converted a project to Kbuild and
> > > Kconfig successfully. I have looked at starting with
> > > https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
> > > old it does separate out the build bits from the Linux specific bits
> > > pretty nicely while removing module support which arguably is the most
> > > complicated part. Alternatively we could start with Linux 4.2 if that's
> > > more desirable.
> > 
> > Thinking longterm, it would be nice to have xen, tools and stubdoms
> > covered by a system like this
> 
> Is the proposal here then to abandon autoconf for the tools subtree in
> favour of Kconfig? Or maybe to somehow hybridize autoconf (for e.g. library
> and feature detection) with Kconfig (for user selection of options)? I'm
> not sure how I feel about either of those approaches, they certainly both
> need careful consideration, and the second in particular regarding the
> interactions...
> 
> FWIW it seems to me that the link between things which are optional in Xen
> and which are optional in the tools is (or should be) pretty loose. i.e.
> the tools today _always_ support XSM and correctly handle the errors from
> Xen if it is not enabled there. Personally I think this is the right way to
> do things. Likewise Xen doesn't care if the tools have particular opinions
> on the qemu to use or whatever.

This is as it should be, but I can see the argument for cutting out
whole features at build time, from both sides.  If I were embedding
Xen in an appliance, or building my own cloud, I'd be very happy to
./configure --disable all sorts of things from the entire build,
without having to figure out how to disable each feature twice.

Cheers,

Tim.

Re: Kbuild and Kconfig

[Jan Beulich]

>>> On 03.09.15 at 12:09, <tim@xen.org> wrote:
> At 10:56 +0100 on 03 Sep (1441277769), Ian Campbell wrote:
>> Is the proposal here then to abandon autoconf for the tools subtree in
>> favour of Kconfig? Or maybe to somehow hybridize autoconf (for e.g. library
>> and feature detection) with Kconfig (for user selection of options)? I'm
>> not sure how I feel about either of those approaches, they certainly both
>> need careful consideration, and the second in particular regarding the
>> interactions...
>> 
>> FWIW it seems to me that the link between things which are optional in Xen
>> and which are optional in the tools is (or should be) pretty loose. i.e.
>> the tools today _always_ support XSM and correctly handle the errors from
>> Xen if it is not enabled there. Personally I think this is the right way to
>> do things. Likewise Xen doesn't care if the tools have particular opinions
>> on the qemu to use or whatever.
> 
> This is as it should be, but I can see the argument for cutting out
> whole features at build time, from both sides.  If I were embedding
> Xen in an appliance, or building my own cloud, I'd be very happy to
> ./configure --disable all sorts of things from the entire build,
> without having to figure out how to disable each feature twice.

I don't see why ./configure shouldn't be able to invoke the
configure mechanism in xen/, passing it appropriate overrides
based on the --disable settings it was handed.

I definitely agree with Ian that using a single mechanism for
both tools and hypervisor is rather not what we want (due to
the different nature of the two).

Jan

Re: Kbuild and Kconfig

[Jan Beulich]

>>> On 02.09.15 at 19:50, <cardoe@cardoe.com> wrote:
> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
> or docs)

As just said in another reply, allowing for ./configure to pass down
options to the configure mechanism in xen/ would seem desirable (as
long as configuring in xen/ alone would still work).

> * split top level config bits to not affect xen/ tree (currently only
> XSM_ENABLE / FLASK_ENABLE do)

As already said by someone else, this shouldn't be necessary. In fact
I would hope there's (other than debug-build-or-not) no top level
setting affecting both tools and hypervisor.

> * convert xen/ to Kbuild first and merge this in (since Kconfig relies
> on Kbuild-y bits which can be undone but if we're going to go to Kbuild
> in the end why undo it and then redo it)

To be honest I'm not convinced we want to pull in Kbuild as a whole.
Achieving the ability to do out-of-tree builds (viewed as desirable
by Andrew, and I've also long been meaning to see whether this
could be made work, as that's also my preferred build setup in other
projects) should be possible without importing everything (and
perhaps even without importing anything).

Jan

Re: Kbuild and Kconfig

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 2951 bytes --]

On 9/3/15 4:56 AM, Ian Campbell wrote:
> On Wed, 2015-09-02 at 19:29 +0100, Andrew Cooper wrote:
>> On 02/09/15 18:50, Doug Goldstein wrote:
>>> I just wanted to bring this to a top level post since Jonathan 
>>> Creekmore
>>> and myself have talked with a few maintainers in different threads and
>>> on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
>>> I would like to get a rough idea on what the Xen community wants the
>>> system to look like before starting work on it to both save myself time
>>> and save maintainers review cycles. So that being said rough proposal 
>>> as
>>> follows:
>>>
>>> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
>>> or docs)
>>> * split top level config bits to not affect xen/ tree (currently only
>>> XSM_ENABLE / FLASK_ENABLE do)
>>> * convert xen/ to Kbuild first and merge this in (since Kconfig relies
>>> on Kbuild-y bits which can be undone but if we're going to go to Kbuild
>>> in the end why undo it and then redo it)
>>> * convert existing xen/ config bits into Kconfig and merge that in
>>>
>>> Jonathan and I, in a former life, converted a project to Kbuild and
>>> Kconfig successfully. I have looked at starting with
>>> https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
>>> old it does separate out the build bits from the Linux specific bits
>>> pretty nicely while removing module support which arguably is the most
>>> complicated part. Alternatively we could start with Linux 4.2 if that's
>>> more desirable.
>>
>> Thinking longterm, it would be nice to have xen, tools and stubdoms
>> covered by a system like this
> 
> Is the proposal here then to abandon autoconf for the tools subtree in
> favour of Kconfig? Or maybe to somehow hybridize autoconf (for e.g. library
> and feature detection) with Kconfig (for user selection of options)? I'm
> not sure how I feel about either of those approaches, they certainly both
> need careful consideration, and the second in particular regarding the
> interactions...
> 
> FWIW it seems to me that the link between things which are optional in Xen
> and which are optional in the tools is (or should be) pretty loose. i.e.
> the tools today _always_ support XSM and correctly handle the errors from
> Xen if it is not enabled there. Personally I think this is the right way to
> do things. Likewise Xen doesn't care if the tools have particular opinions
> on the qemu to use or whatever.
> 
> IOW I'm not sure have xen and tools use a common .config would make sense.
> 
> Ian.
> 
> 

So with my initial approach of targeting the xen/ directory how you
described it is how it would work. The optional items would be separate
and I really think in a lot of cases they are separate like you
describe. (e.g. turn off XSM in the hypervisor but not in the tools and
the tools gracefully handle that).

-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Kbuild and Kconfig

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 2569 bytes --]

On 9/3/15 5:31 AM, Jan Beulich wrote:
>>>> On 02.09.15 at 19:50, <cardoe@cardoe.com> wrote:
>> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
>> or docs)
> 
> As just said in another reply, allowing for ./configure to pass down
> options to the configure mechanism in xen/ would seem desirable (as
> long as configuring in xen/ alone would still work).

My concern would be that the ./configure options would be pretty
unwieldy. e.g.

./configure --without-kexec --without-xenlinux --with-schedule=credit2

You would effectively have to write a script to contain what your
"distro" (e.g. XenServer, Ubuntu with Xen, Amazon's build) wants.
However the Linux kernel has a nice way to pass around a defconfig
and/or .config files to ensure your build always behaves the same. Users
also have an easy way to see what new options have been added since
their .config was generated but autoconf does not have that.

> 
>> * split top level config bits to not affect xen/ tree (currently only
>> XSM_ENABLE / FLASK_ENABLE do)
> 
> As already said by someone else, this shouldn't be necessary. In fact
> I would hope there's (other than debug-build-or-not) no top level
> setting affecting both tools and hypervisor.

In the case of XSM_ENABLE and FLASK_ENABLE which are at the top level
they do affect both the tools/ directory and the xen/ directory but in
two totally different ways. For the tools side XSM is always enabled no
matter the setting but setting either of those to 'n' results in the
Flask policy not being built. For the xen/ directory it appears to
disable XSM support. Hence why I argue that not having top level
settings would be clearer from a usage standpoint.

> 
>> * convert xen/ to Kbuild first and merge this in (since Kconfig relies
>> on Kbuild-y bits which can be undone but if we're going to go to Kbuild
>> in the end why undo it and then redo it)
> 
> To be honest I'm not convinced we want to pull in Kbuild as a whole.
> Achieving the ability to do out-of-tree builds (viewed as desirable
> by Andrew, and I've also long been meaning to see whether this
> could be made work, as that's also my preferred build setup in other
> projects) should be possible without importing everything (and
> perhaps even without importing anything).
> 
> Jan
> 

I can certainly go that route. The xen/ directory already uses Kbuild's
grandfather with a lot of customization so it probably wouldn't be that
world changing but like I said I can avoid it.

-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Kbuild and Kconfig

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 2008 bytes --]

On 9/3/15 4:56 AM, Tim Deegan wrote:
> Hi,
> 
> At 12:50 -0500 on 02 Sep (1441198200), Doug Goldstein wrote:
>> I just wanted to bring this to a top level post since Jonathan Creekmore
>> and myself have talked with a few maintainers in different threads and
>> on IRC about potentially using Kconfig and/or Kbuild for Xen.
> 
> If we're going to need a configure step before building the
> hypervisor, maybe we should consider using the autoconf runes instead.
> 
> I'm one of the people who objected to requiring ./configure before
> building the hypervisor, but I think it would be better than having
> _two_ config systems which are not synced with each other, esp. if
> we'll want to enable/disable features with matching tools-side code.
> 
> If we do go this way, I think we'd need a ./configure --hypervisor-only,
> or similar, that _only_ makes the various CONFIG_ flags, and doesn't
> do the tedious library & compiler checks that are needed for the
> user-space code.
> 
> Cheers,
> 
> Tim.
> 

I'd only be concerned about the fact that different config switches
would do different things and you would end up with a lot of configure
switches.

e.g. ./configure --with-kexec --without-xenlinux --with-xsm

Currently there is a top level XSM_ENABLE and FLASK_ENABLE and they do
different things in the tools/ and the xen/ directories. For the tools
directory all it does is disable the Flask policy being built (e.g.
XSM_ENABLE=n does that) while XSM bits in the tools appear to always be
built. Then for the xen/ directory XSM_ENABLE=n will actually turn off
XSM support and Flask support while FLASK_ENABLE=n will keep XSM on and
just build the dummy XSM module.

You may also have cases where you want to build functionality into the
tools but not into the hypervisor to ensure that the hypervisor returns
an error and the tools gracefully handle that case which having a top
level configure script would make more difficult.

-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Kbuild and Kconfig

[Jan Beulich]

>>> On 03.09.15 at 16:04, <cardoe@cardoe.com> wrote:
> On 9/3/15 5:31 AM, Jan Beulich wrote:
>>>>> On 02.09.15 at 19:50, <cardoe@cardoe.com> wrote:
>>> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
>>> or docs)
>> 
>> As just said in another reply, allowing for ./configure to pass down
>> options to the configure mechanism in xen/ would seem desirable (as
>> long as configuring in xen/ alone would still work).
> 
> My concern would be that the ./configure options would be pretty
> unwieldy. e.g.
> 
> ./configure --without-kexec --without-xenlinux --with-schedule=credit2
> 
> You would effectively have to write a script to contain what your
> "distro" (e.g. XenServer, Ubuntu with Xen, Amazon's build) wants.
> However the Linux kernel has a nice way to pass around a defconfig
> and/or .config files to ensure your build always behaves the same. Users
> also have an easy way to see what new options have been added since
> their .config was generated but autoconf does not have that.

And note that I didn't say this "ordinary" way shouldn't be possible;
in fact I meant it to be the default, with ./configure only allowed to
control things that aren't already set in xen/.config (or whatever
it's going to be named). The idea being that when you say
"--without-<xyz>" and it affects both parts of the tree, you won't
have to select the option another time in the hypervisor configure
process.

>>> * split top level config bits to not affect xen/ tree (currently only
>>> XSM_ENABLE / FLASK_ENABLE do)
>> 
>> As already said by someone else, this shouldn't be necessary. In fact
>> I would hope there's (other than debug-build-or-not) no top level
>> setting affecting both tools and hypervisor.
> 
> In the case of XSM_ENABLE and FLASK_ENABLE which are at the top level
> they do affect both the tools/ directory and the xen/ directory but in
> two totally different ways. For the tools side XSM is always enabled no
> matter the setting but setting either of those to 'n' results in the
> Flask policy not being built. For the xen/ directory it appears to
> disable XSM support. Hence why I argue that not having top level
> settings would be clearer from a usage standpoint.

I think where top level setting make sense (like enabling debug builds)
they should be made work. For existing things at the top level not
really belonging there I agree.

Jan

Re: Kbuild and Kconfig

[Andrew Cooper]

On 03/09/15 10:56, Ian Campbell wrote:
> On Wed, 2015-09-02 at 19:29 +0100, Andrew Cooper wrote:
>> On 02/09/15 18:50, Doug Goldstein wrote:
>>> I just wanted to bring this to a top level post since Jonathan
>>> Creekmore
>>> and myself have talked with a few maintainers in different threads and
>>> on IRC about potentially using Kconfig and/or Kbuild for Xen. Basically
>>> I would like to get a rough idea on what the Xen community wants the
>>> system to look like before starting work on it to both save myself time
>>> and save maintainers review cycles. So that being said rough proposal
>>> as
>>> follows:
>>>
>>> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
>>> or docs)
>>> * split top level config bits to not affect xen/ tree (currently only
>>> XSM_ENABLE / FLASK_ENABLE do)
>>> * convert xen/ to Kbuild first and merge this in (since Kconfig relies
>>> on Kbuild-y bits which can be undone but if we're going to go to Kbuild
>>> in the end why undo it and then redo it)
>>> * convert existing xen/ config bits into Kconfig and merge that in
>>>
>>> Jonathan and I, in a former life, converted a project to Kbuild and
>>> Kconfig successfully. I have looked at starting with
>>> https://github.com/masahir0y/kbuild_skeleton while the tree is fairly
>>> old it does separate out the build bits from the Linux specific bits
>>> pretty nicely while removing module support which arguably is the most
>>> complicated part. Alternatively we could start with Linux 4.2 if that's
>>> more desirable.
>> Thinking longterm, it would be nice to have xen, tools and stubdoms
>> covered by a system like this
> Is the proposal here then to abandon autoconf for the tools subtree in
> favour of Kconfig? Or maybe to somehow hybridize autoconf (for e.g. library
> and feature detection) with Kconfig (for user selection of options)? I'm
> not sure how I feel about either of those approaches, they certainly both
> need careful consideration, and the second in particular regarding the
> interactions...
>
> FWIW it seems to me that the link between things which are optional in Xen
> and which are optional in the tools is (or should be) pretty loose. i.e.
> the tools today _always_ support XSM and correctly handle the errors from
> Xen if it is not enabled there. Personally I think this is the right way to
> do things. Likewise Xen doesn't care if the tools have particular opinions
> on the qemu to use or whatever.
>
> IOW I'm not sure have xen and tools use a common .config would make sense.

autoconf and Kconfig do two different things.

Kconfig is of no use working out where $(libdir) should go, and 
./configure --with-X --without-Y --with-Z has nothing on menuconfig for 
selecting what to build.

if `make` does an implicit `make silentdefaultconfig` given no other 
information (to avoid a situation where using two configuration 
mechanisms is mandatory to get anything to build), then I don't see a 
problem with the two systems living together.

~Andrew

Re: Kbuild and Kconfig

[Ian Campbell]

On Fri, 2015-09-04 at 11:59 +0100, Andrew Cooper wrote:
> On 03/09/15 10:56, Ian Campbell wrote:
> > On Wed, 2015-09-02 at 19:29 +0100, Andrew Cooper wrote:
> > > On 02/09/15 18:50, Doug Goldstein wrote:
> > > > I just wanted to bring this to a top level post since Jonathan
> > > > Creekmore
> > > > and myself have talked with a few maintainers in different threads 
> > > > and
> > > > on IRC about potentially using Kconfig and/or Kbuild for Xen. 
> > > > Basically
> > > > I would like to get a rough idea on what the Xen community wants 
> > > > the
> > > > system to look like before starting work on it to both save myself 
> > > > time
> > > > and save maintainers review cycles. So that being said rough 
> > > > proposal
> > > > as
> > > > follows:
> > > > 
> > > > * target only the xen/ directory tree (i.e. not the toolstack, 
> > > > stubdoms
> > > > or docs)
> > > > * split top level config bits to not affect xen/ tree (currently 
> > > > only
> > > > XSM_ENABLE / FLASK_ENABLE do)
> > > > * convert xen/ to Kbuild first and merge this in (since Kconfig 
> > > > relies
> > > > on Kbuild-y bits which can be undone but if we're going to go to 
> > > > Kbuild
> > > > in the end why undo it and then redo it)
> > > > * convert existing xen/ config bits into Kconfig and merge that in
> > > > 
> > > > Jonathan and I, in a former life, converted a project to Kbuild and
> > > > Kconfig successfully. I have looked at starting with
> > > > https://github.com/masahir0y/kbuild_skeleton while the tree is 
> > > > fairly
> > > > old it does separate out the build bits from the Linux specific 
> > > > bits
> > > > pretty nicely while removing module support which arguably is the 
> > > > most
> > > > complicated part. Alternatively we could start with Linux 4.2 if 
> > > > that's
> > > > more desirable.
> > > Thinking longterm, it would be nice to have xen, tools and stubdoms
> > > covered by a system like this
> > Is the proposal here then to abandon autoconf for the tools subtree in
> > favour of Kconfig? Or maybe to somehow hybridize autoconf (for e.g. 
> > library
> > and feature detection) with Kconfig (for user selection of options)? 
> > I'm
> > not sure how I feel about either of those approaches, they certainly 
> > both
> > need careful consideration, and the second in particular regarding the
> > interactions...
> > 
> > FWIW it seems to me that the link between things which are optional in 
> > Xen
> > and which are optional in the tools is (or should be) pretty loose. 
> > i.e.
> > the tools today _always_ support XSM and correctly handle the errors 
> > from
> > Xen if it is not enabled there. Personally I think this is the right 
> > way to
> > do things. Likewise Xen doesn't care if the tools have particular 
> > opinions
> > on the qemu to use or whatever.
> > 
> > IOW I'm not sure have xen and tools use a common .config would make 
> > sense.
> 
> autoconf and Kconfig do two different things.

I, obviously, am aware of this.

> Kconfig is of no use working out where $(libdir) should go,

Indeed.

>  and 
> ./configure --with-X --without-Y --with-Z has nothing on menuconfig for 
> selecting what to build.

It can and is used to do so though, so we have two tools which can do some
of the same things, regardless of which one is better at any individual
thing.

> if `make` does an implicit `make silentdefaultconfig` given no other 
> information (to avoid a situation where using two configuration 
> mechanisms is mandatory to get anything to build), then I don't see a 
> problem with the two systems living together.

Nonetheless, I would like to see an explicit proposal for how they are
going to sanely and safely coexist, since AFAIK this is not something
others have done and I think the interactions need to be considered
carefully.

(also, none of this has any bearing on "I'm not sure have xen and tools use
a common .config would make sense" which what you appear to have replied
to).

Re: Kbuild and Kconfig

[Stefano Stabellini]

On Wed, 2 Sep 2015, Doug Goldstein wrote:
> Long term, yes we can tackle tools and stubdoms as well but initially I
> was trying to eat the elephant one bite at a time. My idea was that we
> would do xen/ first and you would have to go into that directory and run
> "make defconfig/menuconfig/config/etc && make" inside that directory.
> Once the other directories got converted we could move the setup to the
> top level and include xen/, tools/, stubdom/. It should be mostly a
> mechanical move.

FYI I would like to remove the QEMU, SeaBIOS, OVMF and stubdoms builds
from xen-unstable completely and rely on raisin to build them:

http://www.slideshare.net/StefanoStabellini/xds15-project-raisin
git://xenbits.xen.org/raisin.git

Re: Kbuild and Kconfig

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 3320 bytes --]

On 9/3/15 10:00 AM, Jan Beulich wrote:
>>>> On 03.09.15 at 16:04, <cardoe@cardoe.com> wrote:
>> On 9/3/15 5:31 AM, Jan Beulich wrote:
>>>>>> On 02.09.15 at 19:50, <cardoe@cardoe.com> wrote:
>>>> * target only the xen/ directory tree (i.e. not the toolstack, stubdoms
>>>> or docs)
>>>
>>> As just said in another reply, allowing for ./configure to pass down
>>> options to the configure mechanism in xen/ would seem desirable (as
>>> long as configuring in xen/ alone would still work).
>>
>> My concern would be that the ./configure options would be pretty
>> unwieldy. e.g.
>>
>> ./configure --without-kexec --without-xenlinux --with-schedule=credit2
>>
>> You would effectively have to write a script to contain what your
>> "distro" (e.g. XenServer, Ubuntu with Xen, Amazon's build) wants.
>> However the Linux kernel has a nice way to pass around a defconfig
>> and/or .config files to ensure your build always behaves the same. Users
>> also have an easy way to see what new options have been added since
>> their .config was generated but autoconf does not have that.
> 
> And note that I didn't say this "ordinary" way shouldn't be possible;
> in fact I meant it to be the default, with ./configure only allowed to
> control things that aren't already set in xen/.config (or whatever
> it's going to be named). The idea being that when you say
> "--without-<xyz>" and it affects both parts of the tree, you won't
> have to select the option another time in the hypervisor configure
> process.
> 
>>>> * split top level config bits to not affect xen/ tree (currently only
>>>> XSM_ENABLE / FLASK_ENABLE do)
>>>
>>> As already said by someone else, this shouldn't be necessary. In fact
>>> I would hope there's (other than debug-build-or-not) no top level
>>> setting affecting both tools and hypervisor.
>>
>> In the case of XSM_ENABLE and FLASK_ENABLE which are at the top level
>> they do affect both the tools/ directory and the xen/ directory but in
>> two totally different ways. For the tools side XSM is always enabled no
>> matter the setting but setting either of those to 'n' results in the
>> Flask policy not being built. For the xen/ directory it appears to
>> disable XSM support. Hence why I argue that not having top level
>> settings would be clearer from a usage standpoint.
> 
> I think where top level setting make sense (like enabling debug builds)
> they should be made work. For existing things at the top level not
> really belonging there I agree.
> 
> Jan
> 

Jan,

So locally I have in the xen/ (the hypervisor) "make menuconfig" working
but I'd likely address any concerns you might have upfront in the
patches. Effectively the xen/ directory now when built runs "make
defconfig" before it builds to generate the configuration. The default
configs I've used match what the tree currently contains. However you
expressed interest in having from the top level certain variables
controlled from that level like debug. I can pretty easily make it that
the effective command line is "make debug=y/1/n/0 defconfig" and that
would set CONFIG_DEBUG appropriately. If that's desired I can also
include that. Are there other flags that it would be desirable to have
them controllable from the top level?

-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Kbuild and Kconfig

[Jan Beulich]

>>> On 18.09.15 at 21:31, <cardoe@cardoe.com> wrote:
> So locally I have in the xen/ (the hypervisor) "make menuconfig" working
> but I'd likely address any concerns you might have upfront in the
> patches. Effectively the xen/ directory now when built runs "make
> defconfig" before it builds to generate the configuration. The default
> configs I've used match what the tree currently contains. However you
> expressed interest in having from the top level certain variables
> controlled from that level like debug. I can pretty easily make it that
> the effective command line is "make debug=y/1/n/0 defconfig" and that
> would set CONFIG_DEBUG appropriately. If that's desired I can also
> include that. Are there other flags that it would be desirable to have
> them controllable from the top level?

First of all I think top level settings should only be used to set values
that currently have no setting at all in .config (or however you intend
to name that file). And then I think all top level setting currently
effectable via make command line option and affecting the hypervisor
build) should be treated the same.

Jan