Netdev Archive on lore.kernel.org
 help / color / Atom feed
From: Masahiro Yamada <masahiroy@kernel.org>
To: Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>
Cc: bpf <bpf@vger.kernel.org>, Sam Ravnborg <sam@ravnborg.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Andrii Nakryiko <andriin@fb.com>,
	Daniel Borkmann <daniel@iogearbox.net>,
	John Fastabend <john.fastabend@gmail.com>,
	KP Singh <kpsingh@chromium.org>, Martin KaFai Lau <kafai@fb.com>,
	Michal Marek <michal.lkml@markovi.net>,
	Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Networking <netdev@vger.kernel.org>
Subject: Re: [PATCH 03/16] kbuild: add infrastructure to build userspace programs
Date: Fri, 24 Apr 2020 00:15:12 +0900
Message-ID: <CAK7LNASpLsw5SUasmpwQ++yyXJi714HDOfEo4=O5SywwYQxtQA@mail.gmail.com> (raw)
In-Reply-To: <20200423073929.127521-4-masahiroy@kernel.org>

On Thu, Apr 23, 2020 at 4:40 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Kbuild supports the infrastructure to build host programs, but there
> was no support to build userspace programs for the target architecture
> (i.e. the same architecture as the kernel).
>
> Sam Ravnborg worked on this a long time ago.
>
>   https://lkml.org/lkml/2014/7/13/154
>
> But, it was not merged. One problem at that time was, there was no
> good way to know whether $(CC) can link standalone programs. In fact,
> pre-built kernel.org toolchains [1] do not provide libc.
>
> Now, we can handle this cleanly because the compiler capability is
> evaluated at the Kconfig time. If $(CC) cannot link standalone programs,
> the relevant options are hidden by 'depends on CC_CAN_LINK'.
>
> The implementation just mimics scripts/Makefile.host
>
> The userspace programs are compiled with the same flags as the host
> programs. In addition, it uses -m32 or -m64 if it is found in
> $(KBUILD_CFLAGS).
>
> This new syntax has at least two usecases.
>
> - Sample programs
>
>   Several userspace programs under samples/ include UAPI headers
>   installed in usr/include. Most of them were previously built for
>   the host architecture just to use 'hostprogs' syntax.
>
>   However, 'make headers' always works for the target architecture.
>   This caused the arch mismatch in cross-compiling. To fix this
>   distortion, sample code should be built for the target architecture.
>
> - Bpfilter
>
>   net/bpfilter/Makefile compiles bpfilter_umh as the user mode helper,
>   and embeds it into the kernel code. Currently, it overrides HOSTCC
>   with CC to use the 'hostprogs' syntax. This hack should go away.
>
> [1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
>  Makefile                   | 11 +++++++---
>  scripts/Makefile.build     |  5 +++++
>  scripts/Makefile.clean     |  2 +-
>  scripts/Makefile.userprogs | 44 ++++++++++++++++++++++++++++++++++++++
>  4 files changed, 58 insertions(+), 4 deletions(-)
>  create mode 100644 scripts/Makefile.userprogs
>
> diff --git a/Makefile b/Makefile
> index 49b2709ff44e..f20597820131 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -406,9 +406,11 @@ else
>  HOSTCC = gcc
>  HOSTCXX        = g++
>  endif
> -KBUILD_HOSTCFLAGS   := -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 \
> -               -fomit-frame-pointer -std=gnu89 $(HOST_LFS_CFLAGS) \
> -               $(HOSTCFLAGS)
> +
> +export KBUILD_USERCFLAGS := -Wall -Wmissing-prototypes -Wstrict-prototypes \
> +                             -O2 -fomit-frame-pointer -std=gnu89
> +
> +KBUILD_HOSTCFLAGS   := $(KBUILD_USERCFLAGS) $(HOST_LFS_CFLAGS) $(HOSTCFLAGS)
>  KBUILD_HOSTCXXFLAGS := -Wall -O2 $(HOST_LFS_CFLAGS) $(HOSTCXXFLAGS)
>  KBUILD_HOSTLDFLAGS  := $(HOST_LFS_LDFLAGS) $(HOSTLDFLAGS)
>  KBUILD_HOSTLDLIBS   := $(HOST_LFS_LIBS) $(HOSTLDLIBS)
> @@ -937,6 +939,9 @@ ifeq ($(CONFIG_RELR),y)
>  LDFLAGS_vmlinux        += --pack-dyn-relocs=relr
>  endif
>
> +# Align the bit size of userspace programs with the kernel
> +KBUILD_USERCFLAGS += $(filter -m32 -m64, $(KBUILD_CFLAGS))
> +
>  # make the checker run with the right architecture
>  CHECKFLAGS += --arch=$(ARCH)
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 9fcbfac15d1d..94f2f7016172 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -50,6 +50,11 @@ ifneq ($(hostprogs)$(hostcxxlibs-y)$(hostcxxlibs-m),)
>  include scripts/Makefile.host
>  endif
>
> +# Do not include userprogs rules unless needed
> +ifneq ($(userprogs),)
> +include scripts/Makefile.userprogs
> +endif
> +
>  ifndef obj
>  $(warning kbuild: Makefile.build is included improperly)
>  endif
> diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
> index 075f0cc2d8d7..e2c76122319d 100644
> --- a/scripts/Makefile.clean
> +++ b/scripts/Makefile.clean
> @@ -29,7 +29,7 @@ subdir-ymn    := $(addprefix $(obj)/,$(subdir-ymn))
>
>  __clean-files  := $(extra-y) $(extra-m) $(extra-)       \
>                    $(always) $(always-y) $(always-m) $(always-) $(targets) $(clean-files)   \
> -                  $(hostprogs) $(hostprogs-y) $(hostprogs-m) $(hostprogs-) \
> +                  $(hostprogs) $(hostprogs-y) $(hostprogs-m) $(hostprogs-) $(userprogs) \
>                    $(hostcxxlibs-y) $(hostcxxlibs-m)
>
>  __clean-files   := $(filter-out $(no-clean-files), $(__clean-files))
> diff --git a/scripts/Makefile.userprogs b/scripts/Makefile.userprogs
> new file mode 100644
> index 000000000000..0d987085819b
> --- /dev/null
> +++ b/scripts/Makefile.userprogs
> @@ -0,0 +1,44 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +# Build userspace programs for the target system
> +#
> +
> +userprogs      := $(sort $(userprogs))
> +
> +# Executables compiled from a single .c file
> +user-csingle   := $(foreach m, $(userprogs), $(if $($(m)-objs),,$(m)))
> +
> +# C executables linked based on several .o files
> +user-cmulti    := $(foreach m, $(userprogs), $(if $($(m)-objs),$(m)))
> +
> +# Object (.o) files compiled from .c files
> +user-cobjs     := $(foreach m, $(userprogs), $($(m)-objs))
> +
> +user-csingle   := $(addprefix $(obj)/, $(user-csingle))
> +user-cmulti    := $(addprefix $(obj)/, $(user-cmulti))
> +user-cobjs     := $(addprefix $(obj)/, $(user-cobjs))
> +
> +user_c_flags   = -Wp,-MMD,$(depfile) $(KBUILD_USERCFLAGS) $(user-ccflags) \
> +                       $($(target-stem)-ccflags)
> +
> +# Create an executable from a single .c file
> +quiet_cmd_user_cc_c = CC [U]  $@
> +      cmd_user_cc_c = $(CC) $(user_c_flags) -o $@ $<


                  $($(target-stem)-ldlibs)

is needed here too.



> +$(user-csingle): $(obj)/%: $(src)/%.c FORCE
> +       $(call if_changed_dep,user_cc_c)
> +
> +# Link an executable based on list of .o files
> +quiet_cmd_user_ld = LD [U]  $@
> +      cmd_user_ld = $(CC) -o $@ $(addprefix $(obj)/, $($(target-stem)-objs)) \
> +                      $($(target-stem)-ldlibs)
> +$(user-cmulti): FORCE
> +       $(call if_changed,user_ld)
> +$(call multi_depend, $(user-cmulti), , -objs)
> +
> +# Create .o file from a .c file
> +quiet_cmd_user_cc_o_c = CC [U]  $@
> +      cmd_user_cc_o_c = $(CC) $(user_c_flags) -c -o $@ $<
> +$(user-cobjs): $(obj)/%.o: $(src)/%.c FORCE
> +       $(call if_changed_dep,user_cc_o_c)
> +
> +targets += $(user-csingle) $(user-cmulti) $(user-cobjs)
> --
> 2.25.1
>


-- 
Best Regards
Masahiro Yamada

  reply index

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-23  7:39 [PATCH 00/16] kbuild: support 'userprogs' syntax Masahiro Yamada
2020-04-23  7:39 ` [PATCH 03/16] kbuild: add infrastructure to build userspace programs Masahiro Yamada
2020-04-23 15:15   ` Masahiro Yamada [this message]
2020-04-23  7:39 ` [PATCH 04/16] net: bpfilter: use 'userprogs' syntax to build bpfilter_umh Masahiro Yamada
2020-06-08 11:56   ` Michal Kubecek
2020-06-08 13:32     ` Masahiro Yamada
2020-06-30  6:30     ` Masahiro Yamada
2020-06-30  8:57       ` Michal Kubecek
2020-06-30 16:47       ` Alexei Starovoitov
2020-04-23  7:39 ` [PATCH 05/16] samples: seccomp: build sample programs for target architecture Masahiro Yamada
2020-04-23  7:39 ` [PATCH 06/16] kbuild: doc: document the new syntax 'userprogs' Masahiro Yamada
2020-04-25 11:53 ` [PATCH 00/16] kbuild: support 'userprogs' syntax Sam Ravnborg
2020-04-29  2:38   ` Masahiro Yamada

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAK7LNASpLsw5SUasmpwQ++yyXJi714HDOfEo4=O5SywwYQxtQA@mail.gmail.com' \
    --to=masahiroy@kernel.org \
    --cc=andriin@fb.com \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=john.fastabend@gmail.com \
    --cc=kafai@fb.com \
    --cc=kpsingh@chromium.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=netdev@vger.kernel.org \
    --cc=sam@ravnborg.org \
    --cc=songliubraving@fb.com \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Netdev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/netdev/0 netdev/git/0.git
	git clone --mirror https://lore.kernel.org/netdev/1 netdev/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 netdev netdev/ https://lore.kernel.org/netdev \
		netdev@vger.kernel.org
	public-inbox-index netdev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.netdev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git