Linux-kselftest Archive on lore.kernel.org
 help / color / Atom feed
From: Knut Omang <knut.omang@oracle.com>
To: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: "open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
	Shuah Khan <shuah@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
	Michal Marek <michal.lkml@markovi.net>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Shreyans Devendra Doshi <0xinfosect0r@gmail.com>,
	Alan Maguire <alan.maguire@oracle.com>,
	Brendan Higgins <brendanhiggins@google.com>,
	Kevin Hilman <khilman@baylibre.com>,
	Hidenori Yamaji <hidenori.yamaji@sony.com>,
	Frank Rowand <frowand.list@gmail.com>,
	Timothy Bird <Tim.Bird@sony.com>,
	Luis Chamberlain <mcgrof@kernel.org>,
	"Theodore Ts'o" <tytso@mit.edu>, Daniel Vetter <daniel@ffwll.ch>,
	Stephen Boyd <sboyd@kernel.org>
Subject: Re: [RFC 01/19] kbuild: Fixes to rules for host-cshlib and host-cxxshlib
Date: Wed, 21 Aug 2019 06:03:39 +0200
Message-ID: <bedcf00ab76b416fca6b2bd97d5b0970a76e31bc.camel@oracle.com> (raw)
In-Reply-To: <CAK7LNATp2tNSUakQ1j81AC40QVJUoDaA6hJ7duLwqCOgkXObJg@mail.gmail.com>

On Wed, 2019-08-21 at 10:47 +0900, Masahiro Yamada wrote:
> On Wed, Aug 14, 2019 at 9:53 PM Knut Omang <knut.omang@oracle.com> wrote:
> >
> > On Wed, 2019-08-14 at 07:52 +0200, Knut Omang wrote:
> > > On Wed, 2019-08-14 at 11:02 +0900, Masahiro Yamada wrote:
> > > > Hi Knut,
> > > >
> > > > On Wed, Aug 14, 2019 at 1:19 AM Knut Omang <knut.omang@oracle.com> wrote:
> > > > > On Tue, 2019-08-13 at 23:01 +0900, Masahiro Yamada wrote:
> > > > > > On Tue, Aug 13, 2019 at 3:13 PM Knut Omang <knut.omang@oracle.com> wrote:
> > > > > > > C++ libraries interfacing to C APIs might sometimes need some glue
> > > > > > > logic more easily written in C.
> > > > > > > Allow a C++ library to also contain 0 or more C objects.
> > > > > > >
> > > > > > > Also fix rules for both C and C++ shared libraries:
> > > > > > > - C++ shared libraries depended on .c instead of .cc files
> > > > > > > - Rules were referenced as -objs instead of the intended
> > > > > > >   -cobjs and -cxxobjs following the pattern from hostprogs*.
> > > > > > >
> > > > > > > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> > > > > >
> > > > > > How is this patch related to the rest of this series?
> > > > >
> > > > > This is just my (likely naive) way I to get what I had working
> > > > > using autotools in the Github version of KTF) translated into something
> > > > > comparable using kbuild only. We need to build a shared library consisting
> > > > > of a few C++ files and a very simple C file, and a couple of simple binaries,
> > > > > and the rule in there does seem to take .c files and subject them to the
> > > > > C++ compiler, which makes this difficult to achieve?
> > > >
> > > > Looking at the diff stat of the cover-letter,
> > > > the rest of this patch series is touching only
> > > > Documentation/ and tools/testing/kselftests/.
> > > >
> > > > So, this one is unused by the rest of the changes, isn't it?
> > > > Am I missing something?
> > > >
> > > >
> > > >
> > > > > > This patch breaks GCC-plugins.
> > > > > > Did you really compile-test this patch before the submission?
> > > > >
> > > > > Sorry for my ignorance here:
> > > > > I ran through the kernel build and installed the resulting kernel
> > > > > on a VM that I used to test this, if that's what you are asking
> > > > > about?
> > > > >
> > > > > Do I need some unusual .config options or run a special make target
> > > > > to trigger the problem you see?
> > > > >
> > > > > I used a recent Fedora config with default values for new options,
> > > > > and ran the normal default make target (also with O=) and make selftests
> > > > > to test the patch itself.
> > > >
> > > > I just built allmodconfig for arm.
> > > >
> > > > (The 0-day bot tests allmodconfig for most of architectures,
> > > > so you may receive error reports anyway.)
> > > >
> > > >
> > > > With your patch, I got the following:
> > > >
> > > >
> > > > masahiro@grover:~/ref/linux$ make  ARCH=arm
> > > > CROSS_COMPILE=-  allmodconfig all
> > > >   HOSTCC  scripts/basic/fixdep
> > > >   HOSTCC  scripts/kconfig/conf.o
> > > >   HOSTCC  scripts/kconfig/confdata.o
> > > >   HOSTCC  scripts/kconfig/expr.o
> > > >   LEX     scripts/kconfig/lexer.lex.c
> > > >   YACC    scripts/kconfig/parser.tab.h
> > > >   HOSTCC  scripts/kconfig/lexer.lex.o
> > > >   YACC    scripts/kconfig/parser.tab.c
> > > >   HOSTCC  scripts/kconfig/parser.tab.o
> > > >   HOSTCC  scripts/kconfig/preprocess.o
> > > >   HOSTCC  scripts/kconfig/symbol.o
> > > >   HOSTLD  scripts/kconfig/conf
> > > > scripts/kconfig/conf  --allmodconfig Kconfig
> > > > #
> > > > # configuration written to .config
> > > > #
> > > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-common.h
> > > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
> > > >   SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
> > > >   HOSTCC  scripts/dtc/dtc.o
> > > >   HOSTCC  scripts/dtc/flattree.o
> > > >   HOSTCC  scripts/dtc/fstree.o
> > > >   HOSTCC  scripts/dtc/data.o
> > > >   HOSTCC  scripts/dtc/livetree.o
> > > >   HOSTCC  scripts/dtc/treesource.o
> > > >   HOSTCC  scripts/dtc/srcpos.o
> > > >   HOSTCC  scripts/dtc/checks.o
> > > >   HOSTCC  scripts/dtc/util.o
> > > >   LEX     scripts/dtc/dtc-lexer.lex.c
> > > >   YACC    scripts/dtc/dtc-parser.tab.h
> > > >   HOSTCC  scripts/dtc/dtc-lexer.lex.o
> > > >   YACC    scripts/dtc/dtc-parser.tab.c
> > > >   HOSTCC  scripts/dtc/dtc-parser.tab.o
> > > >   HOSTCC  scripts/dtc/yamltree.o
> > > >   HOSTLD  scripts/dtc/dtc
> > > >   CC      scripts/gcc-plugins/latent_entropy_plugin.o
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so
> > > >    ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: cannot open
> > > > shared object file: No such file or directory
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
> > > >    ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared
> > > > object file: No such file or directory
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
> > > >    ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared
> > > > object file: No such file or directory
> > > > cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
> > > >    ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open
> > > > shared object file: No such file or directory
> > > > make[3]: *** [scripts/Makefile.build;281:
> > > > scripts/gcc-plugins/latent_entropy_plugin.o] Error 1
> > > > make[2]: *** [scripts/Makefile.build;497: scripts/gcc-plugins] Error 2
> > > > make[1]: *** [Makefile;1097: scripts] Error 2
> > > > make: *** [Makefile;330: __build_one_by_one] Error 2
> > >
> > > Ok, I see!
> > >
> > > I'll recall this target and look into it!
> >
> > Ok, so I have tried installing the arm-linux-gnueabihf cross compiler and compiled the
> kernel for arm,
> > but allmodconfig does not seem to enable any GCC plugins per default even on ARM and I
> haven't been able
> > to figure out how to enable any.
> 
> 
> Linaro toolchain supports gcc plugins.
> 
> https://releases.linaro.org/components/toolchain/binaries/latest-7/arm-linux-gnueabihf/
> 
> 
> kernel.org one supports it as well.
> 
> https://mirrors.edge.kernel.org/pub/tools/crosstool/
> 
> 
> 
> 
> 
> > A plain allmodconfig generated config compiles perfectly for me both native x86 and
> w/arm cross compile,
> > but it doesn't seem to enable any gcc plugins.
> >
> > Anyway, maybe I am getting this wrong anyway:
> > Having played with cross compile, it starts to become clear to me that HOSTCC rules
> > might not be the right rules to use, as it will generate host user land binaries as
> opposed to
> > target user land binaries (in a cross compile world obviously these differ)
> >
> > Now, I started off with using the rules in the selftests makefiles for this, but they
> do not play that well with
> > kernel module building. My goal is to be able to do both user land and kernel module
> **target** compiles
> > from the same subtree. Any hints on how to accomplish this appreciated :-)
> 
> 
> tools/ is out of scope of kbuild because it adopted a different (more
> adhoc) build system.

That resonates with my understanding, thanks!
I think that's what I am trying to understand how to improve:

To sum up, my conclusion so far is that there's a need for a set of more integrated Kbuild
rules for target platform binaries and libraries similar to the host* rules. The rules in
tools/ could then be modified to use that generic support instead of the current adhoc
approach. That would allow the needs of tests that have both user space and kernel space
code to be more nicely co-located, with several benefits IMHO.

Any further insights appreciated!

> I have no idea. Please talk to the kselftest maintainer.

She's on the Cc:-list, I was hoping to get a some of the challenges/choices highlighted
here ;-)

Thanks!
Knut



  reply index

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-13  6:09 [RFC 00/19] Integration of Kernel Test Framework (KTF) into the kernel tree Knut Omang
2019-08-13  6:09 ` [RFC 01/19] kbuild: Fixes to rules for host-cshlib and host-cxxshlib Knut Omang
2019-08-13 14:01   ` Masahiro Yamada
2019-08-13 16:19     ` Knut Omang
2019-08-14  2:02       ` Masahiro Yamada
2019-08-14  5:50         ` Knut Omang
2019-08-14  5:52         ` Knut Omang
2019-08-14 12:52           ` Knut Omang
2019-08-21  1:47             ` Masahiro Yamada
2019-08-21  4:03               ` Knut Omang [this message]
2019-08-13  6:09 ` [RFC 02/19] ktf: Introduce the main part of the kernel side of ktf Knut Omang
2019-09-09  1:23   ` Brendan Higgins
2019-09-10  6:15     ` Knut Omang
2019-08-13  6:09 ` [RFC 03/19] ktf: Introduce a generic netlink protocol for test result communication Knut Omang
2019-09-09  1:28   ` Brendan Higgins
2019-09-10  6:30     ` Knut Omang
2019-08-13  6:09 ` [RFC 04/19] ktf: An implementation of a generic associative array container Knut Omang
2019-08-13  6:09 ` [RFC 05/19] ktf: Implementation of ktf support for overriding function entry and return Knut Omang
2019-08-13  6:09 ` [RFC 06/19] ktf: A simple debugfs interface to test results Knut Omang
2019-08-13  8:21   ` Greg Kroah-Hartman
2019-08-14 17:17     ` Knut Omang
2019-08-15  8:49       ` Greg Kroah-Hartman
2019-08-15 10:35         ` Knut Omang
2019-08-15 10:52           ` Greg Kroah-Hartman
2019-08-13  6:09 ` [RFC 07/19] ktf: Simple coverage support Knut Omang
2019-08-13  6:09 ` [RFC 08/19] ktf: Configurable context support for network info setup Knut Omang
2019-08-13  6:09 ` [RFC 09/19] ktf: resolve: A helper utility to aid in exposing private kernel symbols to KTF tests Knut Omang
2019-08-13  6:09 ` [RFC 10/19] ktf: Add documentation for Kernel Test Framework (KTF) Knut Omang
2019-08-13  6:09 ` [RFC 11/19] ktf: Add a small test suite with a few tests to test KTF itself Knut Omang
2019-08-13  6:09 ` [RFC 12/19] ktf: Main part of user land library for executing tests Knut Omang
2019-08-13  6:09 ` [RFC 13/19] ktf: Integration logic for running ktf tests from googletest Knut Omang
2019-08-13  6:09 ` [RFC 14/19] ktf: Internal debugging facilities Knut Omang
2019-08-13  6:09 ` [RFC 15/19] ktf: Some simple examples Knut Omang
2019-08-13  6:09 ` [RFC 16/19] ktf: Some user applications to run tests Knut Omang
2019-08-13  6:09 ` [RFC 17/19] ktf: Toplevel ktf Makefile/makefile includes and scripts to run from kselftest Knut Omang
2019-08-13  6:09 ` [RFC 18/19] kselftests: Enable building ktf Knut Omang
2019-08-13  6:09 ` [RFC 19/19] Documentation/dev-tools: Add index entry for KTF documentation Knut Omang
2019-08-13  8:10 ` [RFC 00/19] Integration of Kernel Test Framework (KTF) into the kernel tree Brendan Higgins
2019-08-13  8:17 ` Brendan Higgins
2019-08-13 11:29   ` Knut Omang
2019-08-13 17:50     ` Brendan Higgins
2019-08-13  8:23 ` Greg Kroah-Hartman
2019-08-13  9:51   ` Knut Omang
2019-08-13 17:02     ` Brendan Higgins

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=bedcf00ab76b416fca6b2bd97d5b0970a76e31bc.camel@oracle.com \
    --to=knut.omang@oracle.com \
    --cc=0xinfosect0r@gmail.com \
    --cc=Tim.Bird@sony.com \
    --cc=alan.maguire@oracle.com \
    --cc=brendanhiggins@google.com \
    --cc=corbet@lwn.net \
    --cc=daniel@ffwll.ch \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hidenori.yamaji@sony.com \
    --cc=khilman@baylibre.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=mcgrof@kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=sboyd@kernel.org \
    --cc=shuah@kernel.org \
    --cc=tytso@mit.edu \
    --cc=yamada.masahiro@socionext.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-kselftest Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kselftest/0 linux-kselftest/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kselftest linux-kselftest/ https://lore.kernel.org/linux-kselftest \
		linux-kselftest@vger.kernel.org
	public-inbox-index linux-kselftest

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kselftest


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git