From: Masahiro Yamada <masahiroy@kernel.org>
To: Sedat Dilek <sedat.dilek@gmail.com>
Cc: Jiri Slaby <jslaby@suse.cz>,
Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Michal Marek <michal.lkml@markovi.net>,
Nick Desaulniers <ndesaulniers@google.com>,
Nathan Chancellor <nathan@kernel.org>,
Sami Tolvanen <samitolvanen@google.com>,
clang-built-linux <llvm@lists.linux.dev>,
Fangrui Song <maskray@google.com>
Subject: Re: [PATCH] kbuild: pass jobserver to cmd_ld_vmlinux.o
Date: Sat, 18 Jun 2022 00:21:28 +0900 [thread overview]
Message-ID: <CAK7LNATHY88PbJ_=A6g7v8NMQnBcQ9g06k1+SCe+NM+xd5dLwA@mail.gmail.com> (raw)
In-Reply-To: <CA+icZUXDGdPrPKUnevt99LUpTRPe=ogqF33uHQRYrQ6Kh-iTAw@mail.gmail.com>
(+LLVM list, Fangrui Song)
On Fri, Jun 17, 2022 at 7:41 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Fri, Jun 17, 2022 at 12:35 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Fri, Jun 17, 2022 at 12:53 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Thu, Jun 16, 2022 at 4:09 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > >
> > > > On Thu, Jun 16, 2022 at 12:45 PM Jiri Slaby <jslaby@suse.cz> wrote:
> > > > >
> > > > > Until the link-vmlinux.sh split (cf. the commit below), the linker was
> > > > > run with jobserver set in MAKEFLAGS. After the split, the command in
> > > > > Makefile.vmlinux_o is not prefixed by "+" anymore, so this information
> > > > > is lost.
> > > > >
> > > > > Restore it as linkers working in parallel (esp. the LTO ones) make a use
> > > > > of i
Hi Jiri,
Please let me clarify first.
Here, is it OK to assume you are talking about Clang LTO
instead of GCC LTO because the latter is not upstreamed ?
I tested this patch but I did not see any performance change for Clang LTO.
[1] CONFIG_CLANG_LTO_FULL
lld always runs sequential.
It never runs in parallel even if you pass -j option to Make
[2] CONFIG_CLANG_LTO_THIN
lld always runs in parallel even if you do not pass -j option
In my machine, lld always allocated 12 threads.
This is irrespective of the Make parallelisms.
One more thing, if a program wants to participate in
Make's jobserver, it must parse MAKEFLAGS, and extract
file descriptors to be used to communicate to the jobserver.
As a code example in the kernel tree,
scripts/jobserver-exec parses "MAKEFLAGS" and "--jobserver".
I grepped the lld source code, but it does not contain
"MAKEFLAGS" or "jobserver".
masahiro@oscar:~/ref/lld$ git remote show origin
* remote origin
Fetch URL: https://github.com/llvm-mirror/lld.git
Push URL: https://github.com/llvm-mirror/lld.git
HEAD branch: master
Remote branches:
master tracked
release_36 tracked
release_37 tracked
release_38 tracked
release_39 tracked
release_40 tracked
release_50 tracked
release_60 tracked
release_70 tracked
release_80 tracked
release_90 tracked
Local branch configured for 'git pull':
master merges with remote master
Local ref configured for 'git push':
master pushes to master (up to date)
masahiro@oscar:~/ref/lld$ git grep MAKEFLAGS
masahiro@oscar:~/ref/lld$ git grep jobserver
So, in my research, LLD does not seem to support the jobserver.
If you are talking about GCC LTO, yes, the code
tries to parse "--jobserver-auth=" from the MAKEFLAGS
environment variable. [1]
[1]: https://github.com/gcc-mirror/gcc/blob/releases/gcc-12.1.0/gcc/lto-wrapper.cc#L1341
But, as you may know, GCC LTO works in a different way,
at least, we cannot do it before modpost.
--
Best Regards
Masahiro Yamada
next prev parent reply other threads:[~2022-06-17 15:22 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-16 10:45 [PATCH] kbuild: pass jobserver to cmd_ld_vmlinux.o Jiri Slaby
2022-06-16 14:09 ` Sedat Dilek
2022-06-16 22:53 ` Sedat Dilek
2022-06-17 10:35 ` Sedat Dilek
2022-06-17 10:41 ` Sedat Dilek
2022-06-17 15:21 ` Masahiro Yamada [this message]
2022-06-17 20:05 ` Fangrui Song
2022-06-18 6:13 ` Sedat Dilek
2022-06-17 16:32 ` Nick Desaulniers
2022-06-17 17:50 ` Sedat Dilek
2022-06-17 19:34 ` Nathan Chancellor
2022-06-18 5:57 ` Sedat Dilek
2022-06-21 7:30 ` Jiri Slaby
2022-06-21 8:41 ` Masahiro Yamada
2022-09-20 8:43 ` Jiri Slaby
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAK7LNATHY88PbJ_=A6g7v8NMQnBcQ9g06k1+SCe+NM+xd5dLwA@mail.gmail.com' \
--to=masahiroy@kernel.org \
--cc=jslaby@suse.cz \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=maskray@google.com \
--cc=michal.lkml@markovi.net \
--cc=nathan@kernel.org \
--cc=ndesaulniers@google.com \
--cc=samitolvanen@google.com \
--cc=sedat.dilek@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.