[PATCH v3] kbuild: Add support to generate LLVM bitcode files

[PATCH v3] kbuild: Add support to generate LLVM bitcode files

[Matthias Kaehlcke]

From: Vinícius Tinti <viniciustinti@gmail.com>

Add rules to kbuild in order to generate LLVM bitcode files with the .ll
extension when using clang.

  # from c code
  CC=clang make kernel/pid.ll

  # from asm code
  CC=clang make arch/x86/kernel/preempt.ll

From: Vinícius Tinti <viniciustinti@gmail.com>
Signed-off-by: Vinícius Tinti <viniciustinti@gmail.com>
Signed-off-by: Behan Webster <behanw@converseincode.com>
Signed-off-by: Matthias Kaehlcke <mka@chromium.org>
---
Resending, original v3 patch: https://patchwork.kernel.org/patch/4891071/

 .gitignore             |  1 +
 Makefile               |  6 ++++++
 scripts/Makefile.build | 14 ++++++++++++++
 3 files changed, 21 insertions(+)

diff --git a/.gitignore b/.gitignore
index c2ed4ecb0acd..0c39aa20b6ba 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,6 +33,7 @@
 *.lzo
 *.patch
 *.gcno
+*.ll
 modules.builtin
 Module.symvers
 *.dwo
diff --git a/Makefile b/Makefile
index e11989d36c87..d998ce363335 100644
--- a/Makefile
+++ b/Makefile
@@ -1361,6 +1361,8 @@ help:
 	@echo  '                    (default: $$(INSTALL_MOD_PATH)/lib/firmware)'
 	@echo  '  dir/            - Build all files in dir and below'
 	@echo  '  dir/file.[ois]  - Build specified target only'
+	@echo  '  dir/file.ll     - Build the LLVM bitcode file'
+	@echo  '                    (requires compiler support for LLVM bitcode generation)'
 	@echo  '  dir/file.lst    - Build specified mixed source/assembly target only'
 	@echo  '                    (requires a recent binutils and recent build (System.map))'
 	@echo  '  dir/file.ko     - Build module including final link'
@@ -1648,6 +1650,10 @@ endif
 	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
 %.symtypes: %.c prepare scripts FORCE
 	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
+%.ll: %.c prepare scripts FORCE
+	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
+%.ll: %.S prepare scripts FORCE
+	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
 
 # Modules
 /: prepare scripts FORCE
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index d883116ebaa4..e5a28da2e6fa 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -177,6 +177,20 @@ cmd_cc_symtypes_c =                                                         \
 $(obj)/%.symtypes : $(src)/%.c FORCE
 	$(call cmd,cc_symtypes_c)
 
+# LLVM bitcode
+# Generate .ll files from .s and .c
+quiet_cmd_cc_ll_c = CC $(quiet_modtag)  $@
+      cmd_cc_ll_c = $(CC) $(c_flags) -emit-llvm -S -o $@ $<
+
+$(obj)/%.ll: $(src)/%.c FORCE
+	$(call if_changed_dep,cc_ll_c)
+
+quiet_cmd_as_ll_S = CPP $(quiet_modtag) $@
+      cmd_as_ll_S = $(CPP) $(a_flags)   -o $@ $<
+
+$(obj)/%.ll: $(src)/%.S FORCE
+	$(call if_changed_dep,as_ll_S)
+
 # C (.c) files
 # The C file is compiled and updated dependency information is generated.
 # (See cmd_cc_o_c + relevant part of rule_cc_o_c)
-- 
2.12.2.715.g7642488e1d-goog

Re: [PATCH v3] kbuild: Add support to generate LLVM bitcode files

[Matthias Kaehlcke]

El Tue, Apr 04, 2017 at 10:27:06AM -0700 Matthias Kaehlcke ha dit:

> From: Vinícius Tinti <viniciustinti@gmail.com>
> 
> Add rules to kbuild in order to generate LLVM bitcode files with the .ll
> extension when using clang.
> 
>   # from c code
>   CC=clang make kernel/pid.ll
> 
>   # from asm code
>   CC=clang make arch/x86/kernel/preempt.ll
> 
> From: Vinícius Tinti <viniciustinti@gmail.com>
> Signed-off-by: Vinícius Tinti <viniciustinti@gmail.com>
> Signed-off-by: Behan Webster <behanw@converseincode.com>
> Signed-off-by: Matthias Kaehlcke <mka@chromium.org>
> ---

Ping, any comments on this patch?

Thanks

Matthias

Re: [PATCH v3] kbuild: Add support to generate LLVM bitcode files

[Masahiro Yamada]

Hi Matthias,


2017-04-05 2:27 GMT+09:00 Matthias Kaehlcke <mka@chromium.org>:
> From: Vinícius Tinti <viniciustinti@gmail.com>
>
> Add rules to kbuild in order to generate LLVM bitcode files with the .ll
> extension when using clang.


First, I'd like to be sure about the terminology "LLVM bitcode"
because "bitcode" sounds like human-unreadable binary.


For example, 'man llvm-as' says:
    llvm-as  is  the  LLVM  assembler.  It reads a file containing
    human-readable LLVM assembly language, translates it to LLVM
    bitcode, and writes the result into a file or to standard output.


As far as I understood:

*.ll   -  LLVM assembly  (human readable file)
*.bc   -  LLVM bitcode   (binary file)

Is this correct?






>   # from c code
>   CC=clang make kernel/pid.ll

This does not work because CC is overridden in the top-level Makefile.
It should be
    make CC=clang kernel/pid.ll




>   # from asm code
>   CC=clang make arch/x86/kernel/preempt.ll

arch/x86/kernel/preempt.* does not exist
(at least in the latest tree).




> +
> +quiet_cmd_as_ll_S = CPP $(quiet_modtag) $@
> +      cmd_as_ll_S = $(CPP) $(a_flags)   -o $@ $<
> +
> +$(obj)/%.ll: $(src)/%.S FORCE
> +       $(call if_changed_dep,as_ll_S)
> +

I could not understand how this rule can convert
architecture-specific assembly to LLVM intermediate expression.

This is just pre-processing *.S file.


Actually, this is completely the same as the rule *.S -> *.s

quiet_cmd_cpp_s_S = CPP $(quiet_modtag) $@
cmd_cpp_s_S       = $(CPP) $(a_flags) -o $@ $<

$(obj)/%.s: $(src)/%.S FORCE
      $(call if_changed_dep,cpp_s_S)





-- 
Best Regards
Masahiro Yamada

Re: [PATCH v3] kbuild: Add support to generate LLVM bitcode files

[Matthias Kaehlcke]

Hi Masahiro,

El Fri, Apr 21, 2017 at 02:02:46PM +0900 Masahiro Yamada ha dit:

> 2017-04-05 2:27 GMT+09:00 Matthias Kaehlcke <mka@chromium.org>:
> > From: Vinícius Tinti <viniciustinti@gmail.com>
> >
> > Add rules to kbuild in order to generate LLVM bitcode files with the .ll
> > extension when using clang.
> 
> 
> First, I'd like to be sure about the terminology "LLVM bitcode"
> because "bitcode" sounds like human-unreadable binary.
> 
> 
> For example, 'man llvm-as' says:
>     llvm-as  is  the  LLVM  assembler.  It reads a file containing
>     human-readable LLVM assembly language, translates it to LLVM
>     bitcode, and writes the result into a file or to standard output.
> 
> 
> As far as I understood:
> 
> *.ll   -  LLVM assembly  (human readable file)
> *.bc   -  LLVM bitcode   (binary file)
> 
> Is this correct?

Yes, the terminology should be changed to talk about 'LLVM assembly'.

> >   # from c code
> >   CC=clang make kernel/pid.ll
> 
> This does not work because CC is overridden in the top-level Makefile.
> It should be
>     make CC=clang kernel/pid.ll

Will change

> >   # from asm code
> >   CC=clang make arch/x86/kernel/preempt.ll
> 
> arch/x86/kernel/preempt.* does not exist
> (at least in the latest tree).
> 
> 
> 
> 
> > +
> > +quiet_cmd_as_ll_S = CPP $(quiet_modtag) $@
> > +      cmd_as_ll_S = $(CPP) $(a_flags)   -o $@ $<
> > +
> > +$(obj)/%.ll: $(src)/%.S FORCE
> > +       $(call if_changed_dep,as_ll_S)
> > +
> 
> I could not understand how this rule can convert
> architecture-specific assembly to LLVM intermediate expression.
> 
> This is just pre-processing *.S file.
> 
> 
> Actually, this is completely the same as the rule *.S -> *.s
> 
> quiet_cmd_cpp_s_S = CPP $(quiet_modtag) $@
> cmd_cpp_s_S       = $(CPP) $(a_flags) -o $@ $<
> 
> $(obj)/%.s: $(src)/%.S FORCE
>       $(call if_changed_dep,cpp_s_S)

Indeed, unsurprisingly the content of a .ll file generated from a .S
is the same as the corresponding .s.

Besides the Makefile rules it isn't clear to me how assembly would be
converted to LLVM IR. I suggest to remove the rules for assembly.

Cheers

Matthias

Re: [PATCH v3] kbuild: Add support to generate LLVM bitcode files

[Masahiro Yamada]

Hi Matthias,




2017-04-22 4:55 GMT+09:00 Matthias Kaehlcke <mka@chromium.org>:
> Hi Masahiro,
>
> El Fri, Apr 21, 2017 at 02:02:46PM +0900 Masahiro Yamada ha dit:
>
>> 2017-04-05 2:27 GMT+09:00 Matthias Kaehlcke <mka@chromium.org>:
>> > From: Vinícius Tinti <viniciustinti@gmail.com>
>> >
>> > Add rules to kbuild in order to generate LLVM bitcode files with the .ll
>> > extension when using clang.
>>
>>
>> First, I'd like to be sure about the terminology "LLVM bitcode"
>> because "bitcode" sounds like human-unreadable binary.
>>
>>
>> For example, 'man llvm-as' says:
>>     llvm-as  is  the  LLVM  assembler.  It reads a file containing
>>     human-readable LLVM assembly language, translates it to LLVM
>>     bitcode, and writes the result into a file or to standard output.
>>
>>
>> As far as I understood:
>>
>> *.ll   -  LLVM assembly  (human readable file)
>> *.bc   -  LLVM bitcode   (binary file)
>>
>> Is this correct?
>
> Yes, the terminology should be changed to talk about 'LLVM assembly'.
>
>> >   # from c code
>> >   CC=clang make kernel/pid.ll
>>
>> This does not work because CC is overridden in the top-level Makefile.
>> It should be
>>     make CC=clang kernel/pid.ll
>
> Will change
>
>> >   # from asm code
>> >   CC=clang make arch/x86/kernel/preempt.ll
>>
>> arch/x86/kernel/preempt.* does not exist
>> (at least in the latest tree).
>>
>>
>>
>>
>> > +
>> > +quiet_cmd_as_ll_S = CPP $(quiet_modtag) $@
>> > +      cmd_as_ll_S = $(CPP) $(a_flags)   -o $@ $<
>> > +
>> > +$(obj)/%.ll: $(src)/%.S FORCE
>> > +       $(call if_changed_dep,as_ll_S)
>> > +
>>
>> I could not understand how this rule can convert
>> architecture-specific assembly to LLVM intermediate expression.
>>
>> This is just pre-processing *.S file.
>>
>>
>> Actually, this is completely the same as the rule *.S -> *.s
>>
>> quiet_cmd_cpp_s_S = CPP $(quiet_modtag) $@
>> cmd_cpp_s_S       = $(CPP) $(a_flags) -o $@ $<
>>
>> $(obj)/%.s: $(src)/%.S FORCE
>>       $(call if_changed_dep,cpp_s_S)
>
> Indeed, unsurprisingly the content of a .ll file generated from a .S
> is the same as the corresponding .s.
>
> Besides the Makefile rules it isn't clear to me how assembly would be
> converted to LLVM IR. I suggest to remove the rules for assembly.
>

I agree.  This rule should be removed.



-- 
Best Regards
Masahiro Yamada

Re: [PATCH v3] kbuild: Add support to generate LLVM bitcode files

[Masahiro Yamada]

Hi Matthias,



2017-04-22 4:55 GMT+09:00 Matthias Kaehlcke <mka@chromium.org>:
> Hi Masahiro,
>
> El Fri, Apr 21, 2017 at 02:02:46PM +0900 Masahiro Yamada ha dit:
>
>> 2017-04-05 2:27 GMT+09:00 Matthias Kaehlcke <mka@chromium.org>:
>> > From: Vinícius Tinti <viniciustinti@gmail.com>
>> >
>> > Add rules to kbuild in order to generate LLVM bitcode files with the .ll
>> > extension when using clang.
>>
>>
>> First, I'd like to be sure about the terminology "LLVM bitcode"
>> because "bitcode" sounds like human-unreadable binary.
>>
>>
>> For example, 'man llvm-as' says:
>>     llvm-as  is  the  LLVM  assembler.  It reads a file containing
>>     human-readable LLVM assembly language, translates it to LLVM
>>     bitcode, and writes the result into a file or to standard output.
>>


One more thing:

Please add '*.ll' pattern to the following clean target.


clean: $(clean-dirs)
        $(call cmd,rmdirs)
        $(call cmd,rmfiles)
        @find $(if $(KBUILD_EXTMOD), $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
                \( -name '*.[oas]' -o -name '*.ko' -o -name '.*.cmd' \
                -o -name '*.ko.*' \
                -o -name '*.dwo'  \
                -o -name '*.su'  \
                -o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
                -o -name '*.symtypes' -o -name 'modules.order' \
                -o -name modules.builtin -o -name '.tmp_*.o.*' \
                -o -name '*.c.[012]*.*' \
                -o -name '*.gcno' \) -type f -print | xargs rm -f





-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2] kbuild: add support to generate LLVM bitcode files

[Tinti]

On Wed, Jul 23, 2014 at 8:56 AM, Vinícius Tinti <viniciustinti@gmail.com> wrote:
> Allows kbuild to generate LLVM bitcode files with the .ll extension.
>
>   # from c code
>   CC=clang make kernel/pid.ll
>
>   # from asm code
>   CC=clang make arch/x86/kernel/preempt.ll
>
> Signed-off-by: Vinícius Tinti <viniciustinti@gmail.com>
> Signed-off-by: Behan Webster <behanw@converseincode.com>
> ---
>  .gitignore             |  1 +
>  Makefile               |  6 ++++++
>  scripts/Makefile.build | 14 ++++++++++++++
>  3 files changed, 21 insertions(+)
>
> diff --git a/.gitignore b/.gitignore
> index f4c0b09..c5a7656 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -32,6 +32,7 @@
>  *.lzo
>  *.patch
>  *.gcno
> +*.ll
>  modules.builtin
>  Module.symvers
>
> diff --git a/Makefile b/Makefile
> index 6b27741..7826c33 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1213,6 +1213,8 @@ help:
>         @echo  '                    (default: $$(INSTALL_MOD_PATH)/lib/firmware)'
>         @echo  '  dir/            - Build all files in dir and below'
>         @echo  '  dir/file.[oisS] - Build specified target only'
> +       @echo  '  dir/file.ll     - Build the LLVM bitcode file'
> +       @echo  '                    (requires a compiler support for LLVM bitcode generation)'
>         @echo  '  dir/file.lst    - Build specified mixed source/assembly target only'
>         @echo  '                    (requires a recent binutils and recent build (System.map))'
>         @echo  '  dir/file.ko     - Build module including final link'
> @@ -1483,6 +1485,10 @@ endif
>         $(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
>  %.symtypes: %.c prepare scripts FORCE
>         $(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
> +%.ll: %.c prepare scripts FORCE
> +       $(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
> +%.ll: %.S prepare scripts FORCE
> +       $(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
>
>  # Modules
>  /: prepare scripts FORCE
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index bf3e677..4d97e4f 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -174,6 +174,20 @@ cmd_cc_symtypes_c =                                                         \
>  $(obj)/%.symtypes : $(src)/%.c FORCE
>         $(call cmd,cc_symtypes_c)
>
> +# LLVM bitcode
> +# Generate .ll files from .s and .c
> +quiet_cmd_cc_ll_c = CC $(quiet_modtag)  $@
> +      cmd_cc_ll_c = $(CC) $(c_flags) -emit-llvm -S -o $@ $<
> +
> +$(obj)/%.ll: $(src)/%.c FORCE
> +       $(call if_changed_dep,cc_ll_c)
> +
> +quiet_cmd_as_ll_S = CPP $(quiet_modtag) $@
> +      cmd_as_ll_S = $(CPP) $(a_flags)   -o $@ $<
> +
> +$(obj)/%.ll: $(src)/%.S FORCE
> +       $(call if_changed_dep,as_ll_S)
> +
>  # C (.c) files
>  # The C file is compiled and updated dependency information is generated.
>  # (See cmd_cc_o_c + relevant part of rule_cc_o_c)
> --
> 2.0.1
>

The preempt.s works for me as well the preempt.ll.

The make help now displays about the .ll files but I have not found
other section in Documentation about it. If there is please let me know.
I have also added the *.ll in .gitignore.

Regards

-- 
Simplicity is the ultimate sophistication

[PATCH v3] kbuild: add support to generate LLVM bitcode files

[Vinícius Tinti]

Allows kbuild to generate LLVM bitcode files using '.ll' suffix.

  # from c code
  CC=clang make kernel/pid.ll

  # from asm code
  CC=clang make arch/x86/kernel/preempt.ll

Signed-off-by: Vinícius Tinti <viniciustinti@gmail.com>
Signed-off-by: Behan Webster <behanw@converseincode.com>
---
 .gitignore             |  1 +
 Makefile               |  6 ++++++
 scripts/Makefile.build | 14 ++++++++++++++
 3 files changed, 21 insertions(+)

diff --git a/.gitignore b/.gitignore
index e213b27..823b9d6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -32,6 +32,7 @@
 *.lzo
 *.patch
 *.gcno
+*.ll
 modules.builtin
 Module.symvers
 *.dwo
diff --git a/Makefile b/Makefile
index 1a60bdd..cd4a120 100644
--- a/Makefile
+++ b/Makefile
@@ -1250,6 +1250,8 @@ help:
 	@echo  '                    (default: $$(INSTALL_MOD_PATH)/lib/firmware)'
 	@echo  '  dir/            - Build all files in dir and below'
 	@echo  '  dir/file.[oisS] - Build specified target only'
+	@echo  '  dir/file.ll     - Build the LLVM bitcode file'
+	@echo  '                    (requires compiler support for LLVM bitcode generation)'
 	@echo  '  dir/file.lst    - Build specified mixed source/assembly target only'
 	@echo  '                    (requires a recent binutils and recent build (System.map))'
 	@echo  '  dir/file.ko     - Build module including final link'
@@ -1526,6 +1528,10 @@ endif
 	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
 %.symtypes: %.c prepare scripts FORCE
 	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
+%.ll: %.c prepare scripts FORCE
+	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
+%.ll: %.S prepare scripts FORCE
+	$(Q)$(MAKE) $(build)=$(build-dir) $(target-dir)$(notdir $@)
 
 # Modules
 /: prepare scripts FORCE
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index bf3e677..4d97e4f 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -174,6 +174,20 @@ cmd_cc_symtypes_c =                                                         \
 $(obj)/%.symtypes : $(src)/%.c FORCE
 	$(call cmd,cc_symtypes_c)
 
+# LLVM bitcode
+# Generate .ll files from .s and .c
+quiet_cmd_cc_ll_c = CC $(quiet_modtag)  $@
+      cmd_cc_ll_c = $(CC) $(c_flags) -emit-llvm -S -o $@ $<
+
+$(obj)/%.ll: $(src)/%.c FORCE
+	$(call if_changed_dep,cc_ll_c)
+
+quiet_cmd_as_ll_S = CPP $(quiet_modtag) $@
+      cmd_as_ll_S = $(CPP) $(a_flags)   -o $@ $<
+
+$(obj)/%.ll: $(src)/%.S FORCE
+	$(call if_changed_dep,as_ll_S)
+
 # C (.c) files
 # The C file is compiled and updated dependency information is generated.
 # (See cmd_cc_o_c + relevant part of rule_cc_o_c)
-- 
2.1.0