[Makefile] asm: handle comments when creating header file

[Makefile] asm: handle comments when creating header file

[Norbert Manthey]

In the early steps of compilation, the asm header files are created, such
as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
before. Depending on the used assembler, there might be comments in the
assembly files.

This commit adds handling comments in the assembler during the creation of
the asm header files.

Signed-off-by: Norbert Manthey <nmanthey@amazon.de>
Signed-off-by: Michael Tautschnig <tautschn@amazon.co.uk>
Reviewed-by: Pawel Wieczorkiewicz <wipawel@amazon.de>
Reviewed-by: Bjoern Doebel <doebel@amazon.de>
---
 xen/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/Makefile b/xen/Makefile
index 044e7c8..841a9e5 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -189,7 +189,7 @@ include/asm-$(TARGET_ARCH)/asm-offsets.h: arch/$(TARGET_ARCH)/asm-offsets.s
 	  echo "#ifndef __ASM_OFFSETS_H__"; \
 	  echo "#define __ASM_OFFSETS_H__"; \
 	  echo ""; \
-	  sed -rne "/==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
+	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
 	  echo ""; \
 	  echo "#endif") <$< >$@
 
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Makefile] asm: handle comments when creating header file

[Jan Beulich]

>>> On 10.01.18 at 16:14, <nmanthey@amazon.com> wrote:
> In the early steps of compilation, the asm header files are created, such
> as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
> assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
> before. Depending on the used assembler, there might be comments in the
> assembly files.
> 
> This commit adds handling comments in the assembler during the creation of
> the asm header files.

I have a hard time seeing how ...

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -189,7 +189,7 @@ include/asm-$(TARGET_ARCH)/asm-offsets.h: arch/$(TARGET_ARCH)/asm-offsets.s
>  	  echo "#ifndef __ASM_OFFSETS_H__"; \
>  	  echo "#define __ASM_OFFSETS_H__"; \
>  	  echo ""; \
> -	  sed -rne "/==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \

... this pattern could match any comment that we currently have.
Would you mind clarifying what it is that is actually broken (and
hence wants/needs fixing)?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Makefile] asm: handle comments when creating header file

[Tautschnig, Michael]

Jan, All,

> On 10 Jan 2018, at 16:02, Jan Beulich <JBeulich@suse.com> wrote:
> 
>>>> On 10.01.18 at 16:14, <nmanthey@amazon.com> wrote:
>> In the early steps of compilation, the asm header files are created, such
>> as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
>> assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
>> before. Depending on the used assembler, there might be comments in the
>> assembly files.
>> 
>> This commit adds handling comments in the assembler during the creation of
>> the asm header files.
> 
> I have a hard time seeing how ...
> 
>> --- a/xen/Makefile
>> +++ b/xen/Makefile
>> @@ -189,7 +189,7 @@ include/asm-$(TARGET_ARCH)/asm-offsets.h: arch/$(TARGET_ARCH)/asm-offsets.s
>> 	  echo "#ifndef __ASM_OFFSETS_H__"; \
>> 	  echo "#define __ASM_OFFSETS_H__"; \
>> 	  echo ""; \
>> -	  sed -rne "/==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
> 
> ... this pattern could match any comment that we currently have.
> Would you mind clarifying what it is that is actually broken (and
> hence wants/needs fixing)?
> 

Re-adding the new line:

+	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \

The change "handles comments" by not printing those to the generated header
file, where they would confuse the preprocessor (as "#" starts comments at
the assembler level). It seems perfectly ok if this matches any existing
comments as none such should get printed to the header file.

The problems showed up when using goto-gcc (a compiler for subsequent use of
static analysis tooling), which embeds additional information in the comments.

Hope this helps,
Michael





Amazon Web Services UK Limited. Registered in England and Wales with registration number 08650665 and which has its registered office at 60 Holborn Viaduct, London EC1A 2FD, United Kingdom.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Makefile] asm: handle comments when creating header file

[Jan Beulich]

>>> On 22.01.18 at 22:19, <tautschn@amazon.co.uk> wrote:
> Jan, All,
> 
>> On 10 Jan 2018, at 16:02, Jan Beulich <JBeulich@suse.com> wrote:
>> 
>>>>> On 10.01.18 at 16:14, <nmanthey@amazon.com> wrote:
>>> In the early steps of compilation, the asm header files are created, such
>>> as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
>>> assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
>>> before. Depending on the used assembler, there might be comments in the
>>> assembly files.
>>> 
>>> This commit adds handling comments in the assembler during the creation of
>>> the asm header files.
>> 
>> I have a hard time seeing how ...
>> 
>>> --- a/xen/Makefile
>>> +++ b/xen/Makefile
>>> @@ -189,7 +189,7 @@ include/asm-$(TARGET_ARCH)/asm-offsets.h: arch/$(TARGET_ARCH)/asm-offsets.s
>>> 	  echo "#ifndef __ASM_OFFSETS_H__"; \
>>> 	  echo "#define __ASM_OFFSETS_H__"; \
>>> 	  echo ""; \
>>> -	  sed -rne "/==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
>> 
>> ... this pattern could match any comment that we currently have.
>> Would you mind clarifying what it is that is actually broken (and
>> hence wants/needs fixing)?
>> 
> 
> Re-adding the new line:
> 
> +	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
> 
> The change "handles comments" by not printing those to the generated header
> file, where they would confuse the preprocessor (as "#" starts comments at
> the assembler level). It seems perfectly ok if this matches any existing
> comments as none such should get printed to the header file.
> 
> The problems showed up when using goto-gcc (a compiler for subsequent use of
> static analysis tooling), which embeds additional information in the 
> comments.

Please add some clarification along those lines to the description.
I then also assume that you mean "used compiler" instead of "used
assembler" in the description? Additionally, if you want to strip
comments, please strip all of them, i.e. including such starting
with # after some leading white space.

Furthermore, the character (or sequence of characters) starting
a comment is target specific - this wants specifying by the per-arch
Rules.mk, I think, so that it'll also work on ARM (where # has a
different meaning). Along those lines, x86 allows // to start a
(full line) comment as well.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH Makefile v2] asm: handle comments when creating header file

[Norbert Manthey]

From: Norbert Manthey <nmanthey@amazon.com>

In the early steps of compilation, the asm header files are created, such
as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
before. Depending on the used toolchain, there might be comments in the
assembly files. Especially the goto-gcc compiler of the bounded model
checker CBMC adds comments that start with a '#' symbol at the beginning
of the line.

This commit adds handling comments in assembler during the creation of the
asm header files, especially ignoring lines that start with '#', which
indicate comments for both ARM and x86 assembler. The used tool goto-as
produces exactly comments of this kind.

Signed-off-by: Norbert Manthey <nmanthey@amazon.de>
Signed-off-by: Michael Tautschnig <tautschn@amazon.co.uk>
---
 xen/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/Makefile b/xen/Makefile
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -191,7 +191,7 @@ include/asm-$(TARGET_ARCH)/asm-offsets.h: arch/$(TARGET_ARCH)/asm-offsets.s
 	  echo "#ifndef __ASM_OFFSETS_H__"; \
 	  echo "#define __ASM_OFFSETS_H__"; \
 	  echo ""; \
-	  sed -rne "/==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
+	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
 	  echo ""; \
 	  echo "#endif") <$< >$@
 
-- 
2.7.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
Ust-ID: DE 289 237 879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH Makefile v2] asm: handle comments when creating header file

[Jan Beulich]

>>> On 06.02.19 at 15:09, <nmanthey@amazon.de> wrote:
> From: Norbert Manthey <nmanthey@amazon.com>
> 
> In the early steps of compilation, the asm header files are created, such
> as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
> assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
> before. Depending on the used toolchain, there might be comments in the
> assembly files. Especially the goto-gcc compiler of the bounded model
> checker CBMC adds comments that start with a '#' symbol at the beginning
> of the line.
> 
> This commit adds handling comments in assembler during the creation of the
> asm header files, especially ignoring lines that start with '#', which
> indicate comments for both ARM and x86 assembler. The used tool goto-as
> produces exactly comments of this kind.
> 
> Signed-off-by: Norbert Manthey <nmanthey@amazon.de>
> Signed-off-by: Michael Tautschnig <tautschn@amazon.co.uk>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH Makefile v2] asm: handle comments when creating header file

[Norbert Manthey]

On 2/6/19 16:10, Jan Beulich wrote:
>>>> On 06.02.19 at 15:09, <nmanthey@amazon.de> wrote:
>> From: Norbert Manthey <nmanthey@amazon.com>
>>
>> In the early steps of compilation, the asm header files are created, such
>> as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
>> assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
>> before. Depending on the used toolchain, there might be comments in the
>> assembly files. Especially the goto-gcc compiler of the bounded model
>> checker CBMC adds comments that start with a '#' symbol at the beginning
>> of the line.
>>
>> This commit adds handling comments in assembler during the creation of the
>> asm header files, especially ignoring lines that start with '#', which
>> indicate comments for both ARM and x86 assembler. The used tool goto-as
>> produces exactly comments of this kind.
>>
>> Signed-off-by: Norbert Manthey <nmanthey@amazon.de>
>> Signed-off-by: Michael Tautschnig <tautschn@amazon.co.uk>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
Jürgen, is there a chance to get this patch into the 4.12 release? It
would be nice to be able to compile upstream Xen with the tool chain for
the CBMC model checker (i.e. the goto-gcc compiler), as that tool chain
allows to apply further reasoning. Thanks!

Best,
Norbert





Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
Ust-ID: DE 289 237 879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH Makefile v2] asm: handle comments when creating header file

[Juergen Gross]

On 11/02/2019 14:07, Norbert Manthey wrote:
> On 2/6/19 16:10, Jan Beulich wrote:
>>>>> On 06.02.19 at 15:09, <nmanthey@amazon.de> wrote:
>>> From: Norbert Manthey <nmanthey@amazon.com>
>>>
>>> In the early steps of compilation, the asm header files are created, such
>>> as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
>>> assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
>>> before. Depending on the used toolchain, there might be comments in the
>>> assembly files. Especially the goto-gcc compiler of the bounded model
>>> checker CBMC adds comments that start with a '#' symbol at the beginning
>>> of the line.
>>>
>>> This commit adds handling comments in assembler during the creation of the
>>> asm header files, especially ignoring lines that start with '#', which
>>> indicate comments for both ARM and x86 assembler. The used tool goto-as
>>> produces exactly comments of this kind.
>>>
>>> Signed-off-by: Norbert Manthey <nmanthey@amazon.de>
>>> Signed-off-by: Michael Tautschnig <tautschn@amazon.co.uk>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>
> Jürgen, is there a chance to get this patch into the 4.12 release? It
> would be nice to be able to compile upstream Xen with the tool chain for
> the CBMC model checker (i.e. the goto-gcc compiler), as that tool chain
> allows to apply further reasoning. Thanks!

Release-acked-by: Juergen Gross <jgross@suse.com>


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH Makefile v2] asm: handle comments when creating header file

[Wei Liu]

On Mon, Feb 11, 2019 at 03:14:55PM +0100, Juergen Gross wrote:
> On 11/02/2019 14:07, Norbert Manthey wrote:
> > On 2/6/19 16:10, Jan Beulich wrote:
> >>>>> On 06.02.19 at 15:09, <nmanthey@amazon.de> wrote:
> >>> From: Norbert Manthey <nmanthey@amazon.com>
> >>>
> >>> In the early steps of compilation, the asm header files are created, such
> >>> as include/asm-$(TARGET_ARCH)/asm-offsets.h. These files depend on the
> >>> assembly file arch/$(TARGET_ARCH)/asm-offsets.s, which is generated
> >>> before. Depending on the used toolchain, there might be comments in the
> >>> assembly files. Especially the goto-gcc compiler of the bounded model
> >>> checker CBMC adds comments that start with a '#' symbol at the beginning
> >>> of the line.
> >>>
> >>> This commit adds handling comments in assembler during the creation of the
> >>> asm header files, especially ignoring lines that start with '#', which
> >>> indicate comments for both ARM and x86 assembler. The used tool goto-as
> >>> produces exactly comments of this kind.
> >>>
> >>> Signed-off-by: Norbert Manthey <nmanthey@amazon.de>
> >>> Signed-off-by: Michael Tautschnig <tautschn@amazon.co.uk>
> >> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> >>
> > Jürgen, is there a chance to get this patch into the 4.12 release? It
> > would be nice to be able to compile upstream Xen with the tool chain for
> > the CBMC model checker (i.e. the goto-gcc compiler), as that tool chain
> > allows to apply further reasoning. Thanks!
> 
> Release-acked-by: Juergen Gross <jgross@suse.com>

Applied.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel