[PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ed Swierk]

- Use %lex-param instead of obsolete YYLEX_PARAM to override lex scanner
  parameter
- Change deprecated %name-prefix= to %name-prefix

Tested against bison 2.4.1 and 3.0.2.

Signed-off-by: Ed Swierk <eswierk@skyportsystems.com>
---
 tools/libxl/libxlu_cfg_y.y | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/libxl/libxlu_cfg_y.y b/tools/libxl/libxlu_cfg_y.y
index aa9f787..5acd438 100644
--- a/tools/libxl/libxlu_cfg_y.y
+++ b/tools/libxl/libxlu_cfg_y.y
@@ -17,7 +17,7 @@
  */
 
 %{
-#define YYLEX_PARAM ctx->scanner
+#define ctx_scanner ctx->scanner
 #include "libxlu_cfg_i.h"
 #include "libxlu_cfg_l.h"
 %}
@@ -31,9 +31,9 @@
 %pure-parser
 %defines
 %error-verbose
-%name-prefix="xlu__cfg_yy"
+%name-prefix "xlu__cfg_yy"
 %parse-param { CfgParseContext *ctx }
-%lex-param { void *scanner }
+%lex-param { ctx_scanner }
 
 %token <string>                IDENT STRING NUMBER NEWLINE
 %type <string>            atom
-- 
1.9.1

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ian Campbell]

On Sat, 2014-11-29 at 21:23 -0800, Ed Swierk wrote:
> - Use %lex-param instead of obsolete YYLEX_PARAM to override lex scanner
>   parameter
> - Change deprecated %name-prefix= to %name-prefix
> 
> Tested against bison 2.4.1 and 3.0.2.
> 
> Signed-off-by: Ed Swierk <eswierk@skyportsystems.com>

Copying Ian J who is the bison guy among the toolstack maintainers.

> ---
>  tools/libxl/libxlu_cfg_y.y | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/libxl/libxlu_cfg_y.y b/tools/libxl/libxlu_cfg_y.y
> index aa9f787..5acd438 100644
> --- a/tools/libxl/libxlu_cfg_y.y
> +++ b/tools/libxl/libxlu_cfg_y.y
> @@ -17,7 +17,7 @@
>   */
>  
>  %{
> -#define YYLEX_PARAM ctx->scanner
> +#define ctx_scanner ctx->scanner
>  #include "libxlu_cfg_i.h"
>  #include "libxlu_cfg_l.h"
>  %}
> @@ -31,9 +31,9 @@
>  %pure-parser
>  %defines
>  %error-verbose
> -%name-prefix="xlu__cfg_yy"
> +%name-prefix "xlu__cfg_yy"
>  %parse-param { CfgParseContext *ctx }
> -%lex-param { void *scanner }
> +%lex-param { ctx_scanner }
>  
>  %token <string>                IDENT STRING NUMBER NEWLINE
>  %type <string>            atom

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Wei Liu]

On Mon, Dec 01, 2014 at 09:42:13AM +0000, Ian Campbell wrote:
> On Sat, 2014-11-29 at 21:23 -0800, Ed Swierk wrote:
> > - Use %lex-param instead of obsolete YYLEX_PARAM to override lex scanner
> >   parameter
> > - Change deprecated %name-prefix= to %name-prefix
> > 
> > Tested against bison 2.4.1 and 3.0.2.
> > 
> > Signed-off-by: Ed Swierk <eswierk@skyportsystems.com>
> 
> Copying Ian J who is the bison guy among the toolstack maintainers.
> 

FWIW I can confirm that libxlu_cfg_y.y won't build in Debian Jessie
(bison 3.0.2) as is. And this patch fixes the problem for me.

Wei.

> > ---
> >  tools/libxl/libxlu_cfg_y.y | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/tools/libxl/libxlu_cfg_y.y b/tools/libxl/libxlu_cfg_y.y
> > index aa9f787..5acd438 100644
> > --- a/tools/libxl/libxlu_cfg_y.y
> > +++ b/tools/libxl/libxlu_cfg_y.y
> > @@ -17,7 +17,7 @@
> >   */
> >  
> >  %{
> > -#define YYLEX_PARAM ctx->scanner
> > +#define ctx_scanner ctx->scanner
> >  #include "libxlu_cfg_i.h"
> >  #include "libxlu_cfg_l.h"
> >  %}
> > @@ -31,9 +31,9 @@
> >  %pure-parser
> >  %defines
> >  %error-verbose
> > -%name-prefix="xlu__cfg_yy"
> > +%name-prefix "xlu__cfg_yy"
> >  %parse-param { CfgParseContext *ctx }
> > -%lex-param { void *scanner }
> > +%lex-param { ctx_scanner }
> >  
> >  %token <string>                IDENT STRING NUMBER NEWLINE
> >  %type <string>            atom
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ian Campbell]

On Mon, 2014-12-01 at 12:19 +0000, Wei Liu wrote:
> On Mon, Dec 01, 2014 at 09:42:13AM +0000, Ian Campbell wrote:
> > On Sat, 2014-11-29 at 21:23 -0800, Ed Swierk wrote:
> > > - Use %lex-param instead of obsolete YYLEX_PARAM to override lex scanner
> > >   parameter
> > > - Change deprecated %name-prefix= to %name-prefix
> > > 
> > > Tested against bison 2.4.1 and 3.0.2.
> > > 
> > > Signed-off-by: Ed Swierk <eswierk@skyportsystems.com>
> > 
> > Copying Ian J who is the bison guy among the toolstack maintainers.
> > 
> 
> FWIW I can confirm that libxlu_cfg_y.y won't build in Debian Jessie
> (bison 3.0.2) as is. And this patch fixes the problem for me.

That would seem like a pretty strong case for 4.5, *except* we ship the
generated files so it should be possible to build anywhere without
requiring any version of bison at all. If Bison is installed then
"./configure BISON=/bin/true" or some such might be needed to stop it
trying to regenerate.

Konrad, any thoughts.

> 
> Wei.
> 
> > > ---
> > >  tools/libxl/libxlu_cfg_y.y | 6 +++---
> > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/tools/libxl/libxlu_cfg_y.y b/tools/libxl/libxlu_cfg_y.y
> > > index aa9f787..5acd438 100644
> > > --- a/tools/libxl/libxlu_cfg_y.y
> > > +++ b/tools/libxl/libxlu_cfg_y.y
> > > @@ -17,7 +17,7 @@
> > >   */
> > >  
> > >  %{
> > > -#define YYLEX_PARAM ctx->scanner
> > > +#define ctx_scanner ctx->scanner
> > >  #include "libxlu_cfg_i.h"
> > >  #include "libxlu_cfg_l.h"
> > >  %}
> > > @@ -31,9 +31,9 @@
> > >  %pure-parser
> > >  %defines
> > >  %error-verbose
> > > -%name-prefix="xlu__cfg_yy"
> > > +%name-prefix "xlu__cfg_yy"
> > >  %parse-param { CfgParseContext *ctx }
> > > -%lex-param { void *scanner }
> > > +%lex-param { ctx_scanner }
> > >  
> > >  %token <string>                IDENT STRING NUMBER NEWLINE
> > >  %type <string>            atom
> > 
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Andrew Cooper]

On 02/12/14 13:47, Ian Campbell wrote:
> On Mon, 2014-12-01 at 12:19 +0000, Wei Liu wrote:
>> On Mon, Dec 01, 2014 at 09:42:13AM +0000, Ian Campbell wrote:
>>> On Sat, 2014-11-29 at 21:23 -0800, Ed Swierk wrote:
>>>> - Use %lex-param instead of obsolete YYLEX_PARAM to override lex scanner
>>>>   parameter
>>>> - Change deprecated %name-prefix= to %name-prefix
>>>>
>>>> Tested against bison 2.4.1 and 3.0.2.
>>>>
>>>> Signed-off-by: Ed Swierk <eswierk@skyportsystems.com>
>>> Copying Ian J who is the bison guy among the toolstack maintainers.
>>>
>> FWIW I can confirm that libxlu_cfg_y.y won't build in Debian Jessie
>> (bison 3.0.2) as is. And this patch fixes the problem for me.
> That would seem like a pretty strong case for 4.5, *except* we ship the
> generated files so it should be possible to build anywhere without
> requiring any version of bison at all. If Bison is installed then
> "./configure BISON=/bin/true" or some such might be needed to stop it
> trying to regenerate.
>
> Konrad, any thoughts.

The automatically generating doesn't actually work.  Depending on the
relative timestamps caused by a SCM checkout, or a tarball extraction,
the files will be attempted to be regenerated.

These files are regenerated in the XenServer build, simply because of
their order in the archived tarball.

~Andrew

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ed Swierk]

On Tue, Dec 2, 2014 at 6:00 AM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> The automatically generating doesn't actually work.  Depending on the
> relative timestamps caused by a SCM checkout, or a tarball extraction,
> the files will be attempted to be regenerated.
>
> These files are regenerated in the XenServer build, simply because of
> their order in the archived tarball.

When I clone the xen tree from git, the timestamps match about 95% of
the time, but the 5% failure rate was annoying enough that I finally
dug in to fix the parser build.

IMHO the generated files should be omitted from the source tree; as
long as the source files are actually buildable, there's no reason not
to treat them like any other source file.

--Ed

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Konrad Rzeszutek Wilk]

On Tue, Dec 02, 2014 at 01:47:16PM +0000, Ian Campbell wrote:
> On Mon, 2014-12-01 at 12:19 +0000, Wei Liu wrote:
> > On Mon, Dec 01, 2014 at 09:42:13AM +0000, Ian Campbell wrote:
> > > On Sat, 2014-11-29 at 21:23 -0800, Ed Swierk wrote:
> > > > - Use %lex-param instead of obsolete YYLEX_PARAM to override lex scanner
> > > >   parameter
> > > > - Change deprecated %name-prefix= to %name-prefix
> > > > 
> > > > Tested against bison 2.4.1 and 3.0.2.
> > > > 
> > > > Signed-off-by: Ed Swierk <eswierk@skyportsystems.com>
> > > 
> > > Copying Ian J who is the bison guy among the toolstack maintainers.
> > > 
> > 
> > FWIW I can confirm that libxlu_cfg_y.y won't build in Debian Jessie
> > (bison 3.0.2) as is. And this patch fixes the problem for me.
> 
> That would seem like a pretty strong case for 4.5, *except* we ship the
> generated files so it should be possible to build anywhere without
> requiring any version of bison at all. If Bison is installed then
> "./configure BISON=/bin/true" or some such might be needed to stop it
> trying to regenerate.
> 
> Konrad, any thoughts.

Using the autogenerated one means that it might introduce a bug.

However we do allow regeneration of it - and it would be a shame if
the auto-generated well, could not be autogenerated!

That is a bug, not a major one, but a bug nonethless.

It is not a regression thought.

The build systems (debian, fedora) all have buildrequires which do
not list 'bison' and as such don't encounter this I think - which means
it is only encountered by folks who build from the sources - which
is a small list.

I would lean towards documenting this in the release notes (the work
around mentioned above) and defer this to Xen 4.6.



> 
> > 
> > Wei.
> > 
> > > > ---
> > > >  tools/libxl/libxlu_cfg_y.y | 6 +++---
> > > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > > > 
> > > > diff --git a/tools/libxl/libxlu_cfg_y.y b/tools/libxl/libxlu_cfg_y.y
> > > > index aa9f787..5acd438 100644
> > > > --- a/tools/libxl/libxlu_cfg_y.y
> > > > +++ b/tools/libxl/libxlu_cfg_y.y
> > > > @@ -17,7 +17,7 @@
> > > >   */
> > > >  
> > > >  %{
> > > > -#define YYLEX_PARAM ctx->scanner
> > > > +#define ctx_scanner ctx->scanner
> > > >  #include "libxlu_cfg_i.h"
> > > >  #include "libxlu_cfg_l.h"
> > > >  %}
> > > > @@ -31,9 +31,9 @@
> > > >  %pure-parser
> > > >  %defines
> > > >  %error-verbose
> > > > -%name-prefix="xlu__cfg_yy"
> > > > +%name-prefix "xlu__cfg_yy"
> > > >  %parse-param { CfgParseContext *ctx }
> > > > -%lex-param { void *scanner }
> > > > +%lex-param { ctx_scanner }
> > > >  
> > > >  %token <string>                IDENT STRING NUMBER NEWLINE
> > > >  %type <string>            atom
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xen.org
> > > http://lists.xen.org/xen-devel
> 
> 

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ian Campbell]

On Tue, 2014-12-02 at 09:49 -0800, Ed Swierk wrote:
> On Tue, Dec 2, 2014 at 6:00 AM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> > The automatically generating doesn't actually work.  Depending on the
> > relative timestamps caused by a SCM checkout, or a tarball extraction,
> > the files will be attempted to be regenerated.
> >
> > These files are regenerated in the XenServer build, simply because of
> > their order in the archived tarball.
> 
> When I clone the xen tree from git, the timestamps match about 95% of
> the time, but the 5% failure rate was annoying enough that I finally
> dug in to fix the parser build.
> 
> IMHO the generated files should be omitted from the source tree; as
> long as the source files are actually buildable, there's no reason not
> to treat them like any other source file.

There was a point in time where the prevailing version of bison (or
maybe flex) in stable distro releases had a bug which meant these files
could not be regenerated easily on common distros. I don't recall the
details well enough to know if that time has now passed. Perhaps Ian J
does.

Ian.

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Olaf Hering]

On Wed, Dec 03, Ian Campbell wrote:

> There was a point in time where the prevailing version of bison (or
> maybe flex) in stable distro releases had a bug which meant these files
> could not be regenerated easily on common distros. I don't recall the
> details well enough to know if that time has now passed. Perhaps Ian J
> does.

Its very easy to build a private bison/flex/whatever and use this
instead of one from the system. Our configure can easily detect a broken
version and refuse to compile.  After all we dont ship .i or .s files
because some compilers happen to optimize better.

Olaf

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0"):
> There was a point in time where the prevailing version of bison (or
> maybe flex) in stable distro releases had a bug which meant these files
> could not be regenerated easily on common distros. I don't recall the
> details well enough to know if that time has now passed. Perhaps Ian J
> does.

We use (essential) features found only in non-ancient versions of
bison and flex.  The last time it was proposed to remove these
pregenerated files, there were complaints from people who were using
six-year-old bison and flex.

I think we should prioritise compatibility with new versions of bison
and flex, and retain the pregenerated versions of people with
incompatibly old version.

I think for 4.5 we should:

 * Regenerate the flex files with current wheezy's flex.  (I have
   reviewed the diff in the generated code.  It produces trivial
   changes which add declarations of xlu__cfg_yyget_column and
   xlu__cfg_yyset_column, but no code body changes - see below.)

 * Take the patch from Ed and regenerate the bison files too.


Konrad: can we have two freeze exceptions please ?

Risk analysis for Ed's patch:

Not taking the patch hurts systems with bison 2.7.1 or later:

  - Affected systems include Debian jessie, which will be
    released during the lifetime of 4.5.

  - Bison 2.7.1 was released in April 2013, a year and
    a half ago.  So any system which is less than 18 months out of
    date suffers pain from not updating.

Taking the patch hurts systems with old bison:

  - Bison 2.4.1 is known to work and was released in December 2008.
    So systems which suffer pain from updating are using at least
    4-year-old bison.

  - I have not investigated whether there are even older bison
    versions which are still OK with updating.

On the affected systems:

  - Attempts to build actually-from-source, or with modified
    bison input, will definitely fail.

  - Some proportion of other builds will fail anyway due to
    timestamp skew.


Risk analysis for regenerating with current wheezy's flex:

There doesn't seem to be any actual change in the generated code apart
from a few new function declarations and changes to #line directives.
So the risk of breakage is small.  Furthermore:

Not taking the patch now means that our flex file will be out of date
and may be regenerated and updated accidentally (or need to be
regenerated) at some point in the future.  That means that (a)
whatever risk of breakage we are taking might be discovered at an
inconvenient time (b) additional build system trouble might result
from an out of date file.


There isn't AFAICT a necessary connection between these two but we
normally regenerate both the bison and flex files together, if
necessary, before making any changes to the input files.

Thanks,
Ian.



diff --git a/tools/libxl/libxlu_cfg_l.c b/tools/libxl/libxlu_cfg_l.c
index df352aa..450863a 100644
--- a/tools/libxl/libxlu_cfg_l.c
+++ b/tools/libxl/libxlu_cfg_l.c
@@ -610,6 +610,10 @@ int xlu__cfg_yyget_lineno (yyscan_t yyscanner );
 
 void xlu__cfg_yyset_lineno (int line_number ,yyscan_t yyscanner );
 
+int xlu__cfg_yyget_column  (yyscan_t yyscanner );
+
+void xlu__cfg_yyset_column (int column_no ,yyscan_t yyscanner );
+
 YYSTYPE * xlu__cfg_yyget_lval (yyscan_t yyscanner );
 
 void xlu__cfg_yyset_lval (YYSTYPE * yylval_param ,yyscan_t yyscanner );
@@ -762,7 +766,7 @@ YY_DECL
 #line 53 "libxlu_cfg_l.l"
 
 
-#line 766 "libxlu_cfg_l.c"
+#line 770 "libxlu_cfg_l.c"
 
     yylval = yylval_param;
 
@@ -971,7 +975,7 @@ YY_RULE_SETUP
 #line 104 "libxlu_cfg_l.l"
 YY_FATAL_ERROR( "flex scanner jammed" );
 	YY_BREAK
-#line 975 "libxlu_cfg_l.c"
+#line 979 "libxlu_cfg_l.c"
 case YY_STATE_EOF(INITIAL):
 case YY_STATE_EOF(lexerr):
 	yyterminate();
diff --git a/tools/libxl/libxlu_cfg_l.h b/tools/libxl/libxlu_cfg_l.h
index 4078302..151064e 100644
--- a/tools/libxl/libxlu_cfg_l.h
+++ b/tools/libxl/libxlu_cfg_l.h
@@ -276,6 +276,10 @@ int xlu__cfg_yyget_lineno (yyscan_t yyscanner );
 
 void xlu__cfg_yyset_lineno (int line_number ,yyscan_t yyscanner );
 
+int xlu__cfg_yyget_column  (yyscan_t yyscanner );
+
+void xlu__cfg_yyset_column (int column_no ,yyscan_t yyscanner );
+
 YYSTYPE * xlu__cfg_yyget_lval (yyscan_t yyscanner );
 
 void xlu__cfg_yyset_lval (YYSTYPE * yylval_param ,yyscan_t yyscanner );
@@ -352,6 +356,6 @@ extern int xlu__cfg_yylex \
 
 #line 104 "libxlu_cfg_l.l"
 
-#line 356 "libxlu_cfg_l.h"
+#line 360 "libxlu_cfg_l.h"
 #undef xlu__cfg_yyIN_HEADER
 #endif /* xlu__cfg_yyHEADER_H */
diff --git a/tools/libxl/libxlu_disk_l.c b/tools/libxl/libxlu_disk_l.c
index 2c6e8e3..beea7f9 100644
--- a/tools/libxl/libxlu_disk_l.c
+++ b/tools/libxl/libxlu_disk_l.c
@@ -1011,6 +1011,10 @@ int xlu__disk_yyget_lineno (yyscan_t yyscanner );
 
 void xlu__disk_yyset_lineno (int line_number ,yyscan_t yyscanner );
 
+int xlu__disk_yyget_column  (yyscan_t yyscanner );
+
+void xlu__disk_yyset_column (int column_no ,yyscan_t yyscanner );
+
 /* Macros after this point can all be overridden by user definitions in
  * section 1.
  */
@@ -1155,7 +1159,7 @@ YY_DECL
 
  /*----- the scanner rules which do the parsing -----*/
 
-#line 1159 "libxlu_disk_l.c"
+#line 1163 "libxlu_disk_l.c"
 
 	if ( !yyg->yy_init )
 		{
@@ -1498,7 +1502,7 @@ YY_RULE_SETUP
 #line 259 "libxlu_disk_l.l"
 YY_FATAL_ERROR( "flex scanner jammed" );
 	YY_BREAK
-#line 1502 "libxlu_disk_l.c"
+#line 1506 "libxlu_disk_l.c"
 			case YY_STATE_EOF(INITIAL):
 			case YY_STATE_EOF(LEXERR):
 				yyterminate();
diff --git a/tools/libxl/libxlu_disk_l.h b/tools/libxl/libxlu_disk_l.h
index 08f60e5..f615582 100644
--- a/tools/libxl/libxlu_disk_l.h
+++ b/tools/libxl/libxlu_disk_l.h
@@ -280,6 +280,10 @@ int xlu__disk_yyget_lineno (yyscan_t yyscanner );
 
 void xlu__disk_yyset_lineno (int line_number ,yyscan_t yyscanner );
 
+int xlu__disk_yyget_column  (yyscan_t yyscanner );
+
+void xlu__disk_yyset_column (int column_no ,yyscan_t yyscanner );
+
 /* Macros after this point can all be overridden by user definitions in
  * section 1.
  */
@@ -346,6 +350,6 @@ extern int xlu__disk_yylex (yyscan_t yyscanner);
 
 #line 259 "libxlu_disk_l.l"
 
-#line 350 "libxlu_disk_l.h"
+#line 354 "libxlu_disk_l.h"
 #undef xlu__disk_yyIN_HEADER
 #endif /* xlu__disk_yyHEADER_H */

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Konrad Rzeszutek Wilk]

On Tue, Dec 09, 2014 at 03:25:29PM +0000, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0"):
> > There was a point in time where the prevailing version of bison (or
> > maybe flex) in stable distro releases had a bug which meant these files
> > could not be regenerated easily on common distros. I don't recall the
> > details well enough to know if that time has now passed. Perhaps Ian J
> > does.
> 
> We use (essential) features found only in non-ancient versions of
> bison and flex.  The last time it was proposed to remove these
> pregenerated files, there were complaints from people who were using
> six-year-old bison and flex.
> 
> I think we should prioritise compatibility with new versions of bison
> and flex, and retain the pregenerated versions of people with
> incompatibly old version.
> 
> I think for 4.5 we should:
> 
>  * Regenerate the flex files with current wheezy's flex.  (I have
>    reviewed the diff in the generated code.  It produces trivial
>    changes which add declarations of xlu__cfg_yyget_column and
>    xlu__cfg_yyset_column, but no code body changes - see below.)
> 
>  * Take the patch from Ed and regenerate the bison files too.
> 
> 
> Konrad: can we have two freeze exceptions please ?
> 
> Risk analysis for Ed's patch:
> 
> Not taking the patch hurts systems with bison 2.7.1 or later:
> 
>   - Affected systems include Debian jessie, which will be
>     released during the lifetime of 4.5.

My understanding is that Jessie won't be using Xen 4.5 but Xen 4.4 And the
next version of Debian that will be released will be most likely using Xen 4.6.

However, this issue (building) would even show up in Xen 4.4 - but
I hadn't seen any Debian bugs about this?

> 
>   - Bison 2.7.1 was released in April 2013, a year and
>     a half ago.  So any system which is less than 18 months out of
>     date suffers pain from not updating.

Aye. Any other distro that will use Xen 4.5 that is new (Fedora 21
ships with 3.0.2) can hit this when building the RPMs or out of
tree.

> 
> Taking the patch hurts systems with old bison:
> 
>   - Bison 2.4.1 is known to work and was released in December 2008.
>     So systems which suffer pain from updating are using at least
>     4-year-old bison.

I am not following that. Are you saying that taking this patch will
break building of the sources when using bison 2.4.1 (like Fedora
Core 13 - which is what I use for my small regression testing vehicles).

That would imply an regression?
> 
>   - I have not investigated whether there are even older bison
>     versions which are still OK with updating.

Anything older than Fedora Core 13 is so ancient I would be scared
to run on contemporary hardware.
> 
> On the affected systems:
> 
>   - Attempts to build actually-from-source, or with modified
>     bison input, will definitely fail.
> 
>   - Some proportion of other builds will fail anyway due to
>     timestamp skew.
> 
> 
> Risk analysis for regenerating with current wheezy's flex:
> 
> There doesn't seem to be any actual change in the generated code apart
> from a few new function declarations and changes to #line directives.
> So the risk of breakage is small.  Furthermore:
> 
> Not taking the patch now means that our flex file will be out of date
> and may be regenerated and updated accidentally (or need to be
> regenerated) at some point in the future.  That means that (a)
> whatever risk of breakage we are taking might be discovered at an
> inconvenient time (b) additional build system trouble might result
> from an out of date file.
> 
> 
> There isn't AFAICT a necessary connection between these two but we
> normally regenerate both the bison and flex files together, if
> necessary, before making any changes to the input files.
> 
> Thanks,
> Ian.
> 
> 
> 
> diff --git a/tools/libxl/libxlu_cfg_l.c b/tools/libxl/libxlu_cfg_l.c
> index df352aa..450863a 100644
> --- a/tools/libxl/libxlu_cfg_l.c
> +++ b/tools/libxl/libxlu_cfg_l.c
> @@ -610,6 +610,10 @@ int xlu__cfg_yyget_lineno (yyscan_t yyscanner );
>  
>  void xlu__cfg_yyset_lineno (int line_number ,yyscan_t yyscanner );
>  
> +int xlu__cfg_yyget_column  (yyscan_t yyscanner );
> +
> +void xlu__cfg_yyset_column (int column_no ,yyscan_t yyscanner );
> +
>  YYSTYPE * xlu__cfg_yyget_lval (yyscan_t yyscanner );
>  
>  void xlu__cfg_yyset_lval (YYSTYPE * yylval_param ,yyscan_t yyscanner );
> @@ -762,7 +766,7 @@ YY_DECL
>  #line 53 "libxlu_cfg_l.l"
>  
>  
> -#line 766 "libxlu_cfg_l.c"
> +#line 770 "libxlu_cfg_l.c"
>  
>      yylval = yylval_param;
>  
> @@ -971,7 +975,7 @@ YY_RULE_SETUP
>  #line 104 "libxlu_cfg_l.l"
>  YY_FATAL_ERROR( "flex scanner jammed" );
>  	YY_BREAK
> -#line 975 "libxlu_cfg_l.c"
> +#line 979 "libxlu_cfg_l.c"
>  case YY_STATE_EOF(INITIAL):
>  case YY_STATE_EOF(lexerr):
>  	yyterminate();
> diff --git a/tools/libxl/libxlu_cfg_l.h b/tools/libxl/libxlu_cfg_l.h
> index 4078302..151064e 100644
> --- a/tools/libxl/libxlu_cfg_l.h
> +++ b/tools/libxl/libxlu_cfg_l.h
> @@ -276,6 +276,10 @@ int xlu__cfg_yyget_lineno (yyscan_t yyscanner );
>  
>  void xlu__cfg_yyset_lineno (int line_number ,yyscan_t yyscanner );
>  
> +int xlu__cfg_yyget_column  (yyscan_t yyscanner );
> +
> +void xlu__cfg_yyset_column (int column_no ,yyscan_t yyscanner );
> +
>  YYSTYPE * xlu__cfg_yyget_lval (yyscan_t yyscanner );
>  
>  void xlu__cfg_yyset_lval (YYSTYPE * yylval_param ,yyscan_t yyscanner );
> @@ -352,6 +356,6 @@ extern int xlu__cfg_yylex \
>  
>  #line 104 "libxlu_cfg_l.l"
>  
> -#line 356 "libxlu_cfg_l.h"
> +#line 360 "libxlu_cfg_l.h"
>  #undef xlu__cfg_yyIN_HEADER
>  #endif /* xlu__cfg_yyHEADER_H */
> diff --git a/tools/libxl/libxlu_disk_l.c b/tools/libxl/libxlu_disk_l.c
> index 2c6e8e3..beea7f9 100644
> --- a/tools/libxl/libxlu_disk_l.c
> +++ b/tools/libxl/libxlu_disk_l.c
> @@ -1011,6 +1011,10 @@ int xlu__disk_yyget_lineno (yyscan_t yyscanner );
>  
>  void xlu__disk_yyset_lineno (int line_number ,yyscan_t yyscanner );
>  
> +int xlu__disk_yyget_column  (yyscan_t yyscanner );
> +
> +void xlu__disk_yyset_column (int column_no ,yyscan_t yyscanner );
> +
>  /* Macros after this point can all be overridden by user definitions in
>   * section 1.
>   */
> @@ -1155,7 +1159,7 @@ YY_DECL
>  
>   /*----- the scanner rules which do the parsing -----*/
>  
> -#line 1159 "libxlu_disk_l.c"
> +#line 1163 "libxlu_disk_l.c"
>  
>  	if ( !yyg->yy_init )
>  		{
> @@ -1498,7 +1502,7 @@ YY_RULE_SETUP
>  #line 259 "libxlu_disk_l.l"
>  YY_FATAL_ERROR( "flex scanner jammed" );
>  	YY_BREAK
> -#line 1502 "libxlu_disk_l.c"
> +#line 1506 "libxlu_disk_l.c"
>  			case YY_STATE_EOF(INITIAL):
>  			case YY_STATE_EOF(LEXERR):
>  				yyterminate();
> diff --git a/tools/libxl/libxlu_disk_l.h b/tools/libxl/libxlu_disk_l.h
> index 08f60e5..f615582 100644
> --- a/tools/libxl/libxlu_disk_l.h
> +++ b/tools/libxl/libxlu_disk_l.h
> @@ -280,6 +280,10 @@ int xlu__disk_yyget_lineno (yyscan_t yyscanner );
>  
>  void xlu__disk_yyset_lineno (int line_number ,yyscan_t yyscanner );
>  
> +int xlu__disk_yyget_column  (yyscan_t yyscanner );
> +
> +void xlu__disk_yyset_column (int column_no ,yyscan_t yyscanner );
> +
>  /* Macros after this point can all be overridden by user definitions in
>   * section 1.
>   */
> @@ -346,6 +350,6 @@ extern int xlu__disk_yylex (yyscan_t yyscanner);
>  
>  #line 259 "libxlu_disk_l.l"
>  
> -#line 350 "libxlu_disk_l.h"
> +#line 354 "libxlu_disk_l.h"
>  #undef xlu__disk_yyIN_HEADER
>  #endif /* xlu__disk_yyHEADER_H */

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ian Campbell]

On Wed, 2014-12-10 at 11:41 -0500, Konrad Rzeszutek Wilk wrote:
> On Tue, Dec 09, 2014 at 03:25:29PM +0000, Ian Jackson wrote:
> > Ian Campbell writes ("Re: [Xen-devel] [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0"):
> > > There was a point in time where the prevailing version of bison (or
> > > maybe flex) in stable distro releases had a bug which meant these files
> > > could not be regenerated easily on common distros. I don't recall the
> > > details well enough to know if that time has now passed. Perhaps Ian J
> > > does.
> > 
> > We use (essential) features found only in non-ancient versions of
> > bison and flex.  The last time it was proposed to remove these
> > pregenerated files, there were complaints from people who were using
> > six-year-old bison and flex.
> > 
> > I think we should prioritise compatibility with new versions of bison
> > and flex, and retain the pregenerated versions of people with
> > incompatibly old version.
> > 
> > I think for 4.5 we should:
> > 
> >  * Regenerate the flex files with current wheezy's flex.  (I have
> >    reviewed the diff in the generated code.  It produces trivial
> >    changes which add declarations of xlu__cfg_yyget_column and
> >    xlu__cfg_yyset_column, but no code body changes - see below.)
> > 
> >  * Take the patch from Ed and regenerate the bison files too.
> > 
> > 
> > Konrad: can we have two freeze exceptions please ?
> > 
> > Risk analysis for Ed's patch:
> > 
> > Not taking the patch hurts systems with bison 2.7.1 or later:
> > 
> >   - Affected systems include Debian jessie, which will be
> >     released during the lifetime of 4.5.
> 
> My understanding is that Jessie won't be using Xen 4.5 but Xen 4.4 And the
> next version of Debian that will be released will be most likely using Xen 4.6.

What Jessie (or Fedora etc) has available in packages has no real
bearing on whether people might run Xen 4.5 on them, plenty of people
will still build from source, including developers.

Ian.

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Konrad Rzeszutek Wilk]

On Wed, Dec 10, 2014 at 04:44:00PM +0000, Ian Campbell wrote:
> On Wed, 2014-12-10 at 11:41 -0500, Konrad Rzeszutek Wilk wrote:
> > On Tue, Dec 09, 2014 at 03:25:29PM +0000, Ian Jackson wrote:
> > > Ian Campbell writes ("Re: [Xen-devel] [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0"):
> > > > There was a point in time where the prevailing version of bison (or
> > > > maybe flex) in stable distro releases had a bug which meant these files
> > > > could not be regenerated easily on common distros. I don't recall the
> > > > details well enough to know if that time has now passed. Perhaps Ian J
> > > > does.
> > > 
> > > We use (essential) features found only in non-ancient versions of
> > > bison and flex.  The last time it was proposed to remove these
> > > pregenerated files, there were complaints from people who were using
> > > six-year-old bison and flex.
> > > 
> > > I think we should prioritise compatibility with new versions of bison
> > > and flex, and retain the pregenerated versions of people with
> > > incompatibly old version.
> > > 
> > > I think for 4.5 we should:
> > > 
> > >  * Regenerate the flex files with current wheezy's flex.  (I have
> > >    reviewed the diff in the generated code.  It produces trivial
> > >    changes which add declarations of xlu__cfg_yyget_column and
> > >    xlu__cfg_yyset_column, but no code body changes - see below.)
> > > 
> > >  * Take the patch from Ed and regenerate the bison files too.
> > > 
> > > 
> > > Konrad: can we have two freeze exceptions please ?
> > > 
> > > Risk analysis for Ed's patch:
> > > 
> > > Not taking the patch hurts systems with bison 2.7.1 or later:
> > > 
> > >   - Affected systems include Debian jessie, which will be
> > >     released during the lifetime of 4.5.
> > 
> > My understanding is that Jessie won't be using Xen 4.5 but Xen 4.4 And the
> > next version of Debian that will be released will be most likely using Xen 4.6.
> 
> What Jessie (or Fedora etc) has available in packages has no real
> bearing on whether people might run Xen 4.5 on them, plenty of people
> will still build from source, including developers.

My initial reasoning for not giving the exception was that this bug did
not affect users.

But it does affect developers and distros - which in number comparison to
users make a miniscule amount.

However, from an perspective of bug-fixing, they (developers) are the most
important - not users. Hence making the environement for them to fix issues
and develop new features without much fuss is paramount. As in, I do not want
them to start looking at fixing an issue/developing, and this crops up and
they say "WTF? I just want to add this tiny libxl feature and I get this!
This is a quagmire".

Based on that reasoning (And thank you for pointing out the developers
factors), I retract my earlier reason and:

Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

P.S.
And I just verified that this does not cause regressions with bison 2.4.1, so
that is OK.

Re: [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0

[Ian Jackson]

Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [PATCH] libxl: Fix building libxlu_cfg_y.y with bison 3.0"):
> On Wed, Dec 10, 2014 at 04:44:00PM +0000, Ian Campbell wrote:
> > On Wed, 2014-12-10 at 11:41 -0500, Konrad Rzeszutek Wilk wrote:
> > > On Tue, Dec 09, 2014 at 03:25:29PM +0000, Ian Jackson wrote:
> > > > I think for 4.5 we should:
> > > > 
> > > >  * Regenerate the flex files with current wheezy's flex.  (I have
> > > >    reviewed the diff in the generated code.  It produces trivial
> > > >    changes which add declarations of xlu__cfg_yyget_column and
> > > >    xlu__cfg_yyset_column, but no code body changes - see below.)
> > > > 
> > > >  * Take the patch from Ed and regenerate the bison files too.
> > > > 
> > > > Konrad: can we have two freeze exceptions please ?
...
> Based on that reasoning (And thank you for pointing out the developers
> factors), I retract my earlier reason and:
> 
> Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Thanks.  I have now pushed these two.

Ian.