From: Andrew Cooper <andrew.cooper3@citrix.com>
To: "Jürgen Groß" <jgross@suse.com>, "Jan Beulich" <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
Wei Liu <wl@xen.org>, Ian Jackson <ian.jackson@eu.citrix.com>,
George Dunlap <George.Dunlap@citrix.com>
Subject: Re: Getting rid of (many) dynamic link creations in the xen build
Date: Thu, 15 Oct 2020 21:52:56 +0100 [thread overview]
Message-ID: <ad909278-8ab0-dc7a-2004-5efd08e5acbd@citrix.com> (raw)
In-Reply-To: <abd6d752-9a7f-fcf6-3273-82512c590151@suse.com>
On 15/10/2020 11:41, Jürgen Groß wrote:
> On 15.10.20 12:09, Jan Beulich wrote:
>> On 15.10.2020 09:58, Jürgen Groß wrote:
>>> After a short discussion on IRC yesterday I promised to send a mail
>>> how I think we could get rid of creating dynamic links especially
>>> for header files in the Xen build process.
>>>
>>> This will require some restructuring, the amount will depend on the
>>> selected way to proceed:
>>>
>>> - avoid links completely, requires more restructuring
>>> - avoid only dynamically created links, i.e. allowing some static
>>> links which are committed to git
>>
>> While I like the latter better, I'd like to point out that not all
>> file systems support symlinks, and hence the repo then couldn't be
>> stored on (or the tarball expanded onto) such a file system. Note
>> that this may be just for viewing purposes - I do this typically at
>> home -, i.e. there's no resulting limitation from the build process
>> needing symlinks. Similarly, once we fully support out of tree
>> builds, there wouldn't be any restriction from this as long as just
>> the build tree is placed on a capable file system.
>>
>> As a result I'd like to propose variant 2´: Reduce the number of
>> dynamically created symlinks to a minimum. This said, I have to
>> admit that I haven't really understood yet why symlinks are bad.
>> They exist for exactly such purposes, I would think.
>
> Not the symlinks as such, but the dynamically created ones seem to be
> a problem, as we stumble upon those again and again.
We have multiple build system bugs every release to do with dynamically
generated symlinks. Given that symlinks aren't a hard requirement, this
is a massive price to pay, and time which could be better spent doing
other other things.
Also, they prohibit the ability to build from a read-only source dir.
The asm symlink in particular prevents any attempt to do concurrent
builds of xen. In some future, I'd love to be able to do concurrent
out-of-tree builds of Xen on different architectures, because elapsed
time to do this is one limiting factor of mine on pre-push sanity checks.
Personally, I'd prefer option 1 in the long run, but I've got no
problems with achieving option 2 as an intermediate goal.
>
>>
>>> The difference between both variants is affecting the public headers
>>> in xen/include/public/: avoiding even static links would require to
>>> add another directory or to move those headers to another place in the
>>> tree (either use xen/include/public/xen/, or some other path */xen),
>>> leading to the need to change all #include statements in the hypervisor
>>> using <public/...> today.
>>>
>>> The need for the path to have "xen/" is due to the Xen library headers
>>> (which are installed on user's machines) are including the public
>>> hypervisor headers via "#include <xen/...>" and we can't change that
>>> scheme. A static link can avoid this problem via a different path, but
>>> without any link we can't do that.
>>>
>>> Apart from that decision, lets look which links are created today for
>>> accessing the header files (I'll assume my series putting the library
>>> headers to tools/include will be taken, so those links being created
>>> in staging today are not mentioned) and what can be done to avoid them:
>>>
>>> - xen/include/asm -> xen/include/asm-<arch>:
>>> Move all headers from xen/include/asm-<arch> to
>>> xen/arch/<arch>/include/asm and add that path via "-I" flag to
>>> CFLAGS.
>>> This has the other nice advantages that most architecture specific
>>> files are now in xen/arch (apart from the public headers) and
>>> that we
>>> can even add generic fallback headers in xen/include/asm in case an
>>> arch doesn't need a specific header file.
>>
>> Iirc Andrew suggested years ago that we follow Linux in this regard
>> (and XTF already does). My only concern here is the churn this will
>> cause for backports.
>
> Changing a directory name in a patch isn't that hard, IMO.
Also git (if you throw it the correct runes) can cope with this
automatically.
>>> - xen/arch/<arch>/efi/*.[ch] -> xen/common/efi/*.[ch]:
>>> Use vpath for the *.c files and the "-I" flag for adding
>>> common/efi to
>>> the include path in the xen/arch/<arch>/efi/Makefile.
>>
>> Fine with me.
Something which has been irritating me for years is that cscope doesn't
tollerate the efi symlinks. This would be a great solution.
~Andrew
next prev parent reply other threads:[~2020-10-15 20:53 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-15 7:58 Getting rid of (many) dynamic link creations in the xen build Jürgen Groß
2020-10-15 10:09 ` Jan Beulich
2020-10-15 10:41 ` Jürgen Groß
2020-10-15 20:52 ` Andrew Cooper [this message]
2020-10-16 6:52 ` Jan Beulich
2020-10-16 6:58 ` Jan Beulich
2020-10-16 7:25 ` Jürgen Groß
2020-10-16 8:11 ` Jan Beulich
2020-10-15 10:49 ` Jürgen Groß
2020-10-16 6:59 ` Jan Beulich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ad909278-8ab0-dc7a-2004-5efd08e5acbd@citrix.com \
--to=andrew.cooper3@citrix.com \
--cc=George.Dunlap@citrix.com \
--cc=ian.jackson@eu.citrix.com \
--cc=jbeulich@suse.com \
--cc=jgross@suse.com \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).