* [PATCH] allow show_token() on TOKEN_ZERO_IDENT
@ 2022-06-07 12:54 Luc Van Oostenryck
2022-06-07 18:26 ` Linus Torvalds
0 siblings, 1 reply; 3+ messages in thread
From: Luc Van Oostenryck @ 2022-06-07 12:54 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
From: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
TOKEN_ZERO_IDENTs are created during the evaluation of pre-processor
expressions but which otherwise are normal idents and were first tokenized
as TOKEN_IDENTs.
As such, they could perfectly be displayed by show_token() but are not.
So, in error messages they are displayed as "unhandled token type '4'",
which is not at all informative.
Fix this by letting show_token() process them like usual TOKEN_IDENTs.
Idem for quote_token().
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
tokenize.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tokenize.c b/tokenize.c
index ea7105438270..fdaea370cc48 100644
--- a/tokenize.c
+++ b/tokenize.c
@@ -201,6 +201,7 @@ const char *show_token(const struct token *token)
return "end-of-input";
case TOKEN_IDENT:
+ case TOKEN_ZERO_IDENT:
return show_ident(token->ident);
case TOKEN_NUMBER:
@@ -259,6 +260,7 @@ const char *quote_token(const struct token *token)
return "syntax error";
case TOKEN_IDENT:
+ case TOKEN_ZERO_IDENT:
return show_ident(token->ident);
case TOKEN_NUMBER:
--
2.36.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] allow show_token() on TOKEN_ZERO_IDENT
2022-06-07 12:54 [PATCH] allow show_token() on TOKEN_ZERO_IDENT Luc Van Oostenryck
@ 2022-06-07 18:26 ` Linus Torvalds
2022-06-07 20:48 ` Luc Van Oostenryck
0 siblings, 1 reply; 3+ messages in thread
From: Linus Torvalds @ 2022-06-07 18:26 UTC (permalink / raw)
To: Luc Van Oostenryck; +Cc: Sparse Mailing-list, Luc Van Oostenryck
On Tue, Jun 7, 2022 at 5:55 AM Luc Van Oostenryck <lucvoo@kernel.org> wrote:
>
> TOKEN_ZERO_IDENTs are created during the evaluation of pre-processor
> expressions but which otherwise are normal idents and were first tokenized
> as TOKEN_IDENTs.
>
> As such, they could perfectly be displayed by show_token() but are not.
> So, in error messages they are displayed as "unhandled token type '4'",
> which is not at all informative.
>
> Fix this by letting show_token() process them like usual TOKEN_IDENTs.
> Idem for quote_token().
Ack.
I do wonder if it should be marked somehow as being that special case.
The main reason for 'show_token()' is debugging, after all, and
TOKEN_ZERO_IDENT does have magical properties in how it either
silently expands to the constant '0', or it generates a warning about
undefined preprocessor symbol.
But considering that we've apparently reported it as "unhandled token
type '4'" since 2005, I guess it's not exactly a big deal.
Linus
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] allow show_token() on TOKEN_ZERO_IDENT
2022-06-07 18:26 ` Linus Torvalds
@ 2022-06-07 20:48 ` Luc Van Oostenryck
0 siblings, 0 replies; 3+ messages in thread
From: Luc Van Oostenryck @ 2022-06-07 20:48 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Luc Van Oostenryck, Sparse Mailing-list
On Tue, Jun 07, 2022 at 11:26:36AM -0700, Linus Torvalds wrote:
> On Tue, Jun 7, 2022 at 5:55 AM Luc Van Oostenryck <lucvoo@kernel.org> wrote:
> >
> > TOKEN_ZERO_IDENTs are created during the evaluation of pre-processor
> > expressions but which otherwise are normal idents and were first tokenized
> > as TOKEN_IDENTs.
> >
> > As such, they could perfectly be displayed by show_token() but are not.
> > So, in error messages they are displayed as "unhandled token type '4'",
> > which is not at all informative.
> >
> > Fix this by letting show_token() process them like usual TOKEN_IDENTs.
> > Idem for quote_token().
>
> Ack.
>
> I do wonder if it should be marked somehow as being that special case.
> The main reason for 'show_token()' is debugging, after all, and
> TOKEN_ZERO_IDENT does have magical properties in how it either
> silently expands to the constant '0', or it generates a warning about
> undefined preprocessor symbol.
>
> But considering that we've apparently reported it as "unhandled token
> type '4'" since 2005, I guess it's not exactly a big deal.
Yes, I first thought to do so but then choose not because I could not
convince myself that its special property was irrelevant in warning/error
messages. It looks to me more as an internal thing, more semantical than
lexical, and a non-faithful representation would be confusing in messages.
For context, the input text I had (from GCC's testsuite) was:
#define empty
#if empty#cpu(m68k)
#endif
and the error message sparse issued was:
error: garbage at end: #unhandled token type '4' (unhandled token type '4' )
with this patch it's:
error: garbage at end: #cpu(m68k)
-- Luc
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-06-08 0:58 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-07 12:54 [PATCH] allow show_token() on TOKEN_ZERO_IDENT Luc Van Oostenryck
2022-06-07 18:26 ` Linus Torvalds
2022-06-07 20:48 ` Luc Van Oostenryck
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.