[PATCH] qapi: Fix code generation with Python 3.5

[PATCH] qapi: Fix code generation with Python 3.5

[Markus Armbruster]

Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
modules" switched QAPISchema.visit() from

    for entity in self._entity_list:

effectively to

    for mod in self._module_dict.values():
        for entity in mod._entity_list:

Visits in the same order as long as .values() is in insertion order.
That's the case only for Python 3.6 and later.  Before, it's in some
arbitrary order, which results in broken generated code.

Fix by making self._module_dict an OrderedDict rather than a dict.

Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 scripts/qapi/schema.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 0bfc5256fb..5100110fa2 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -795,7 +795,7 @@ class QAPISchema(object):
         self.docs = parser.docs
         self._entity_list = []
         self._entity_dict = {}
-        self._module_dict = {}
+        self._module_dict = OrderedDict()
         self._schema_dir = os.path.dirname(fname)
         self._make_module(None) # built-ins
         self._make_module(fname)
-- 
2.21.1

Re: [PATCH] qapi: Fix code generation with Python 3.5

[John Snow]

On 1/16/20 3:25 PM, Markus Armbruster wrote:
> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
> 
>     for entity in self._entity_list:
> 
> effectively to
> 
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
> 
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
> 
> Fix by making self._module_dict an OrderedDict rather than a dict.
> 
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  scripts/qapi/schema.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 0bfc5256fb..5100110fa2 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -795,7 +795,7 @@ class QAPISchema(object):
>          self.docs = parser.docs
>          self._entity_list = []
>          self._entity_dict = {}
> -        self._module_dict = {}
> +        self._module_dict = OrderedDict()
>          self._schema_dir = os.path.dirname(fname)
>          self._make_module(None) # built-ins
>          self._make_module(fname)
> 

This problem has bitten me *many* times. I'm wondering if there's a
prescription that isn't just "Wait until we can stipulate 3.6+".

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>> modules" switched QAPISchema.visit() from
>> 
>>     for entity in self._entity_list:
>> 
>> effectively to
>> 
>>     for mod in self._module_dict.values():
>>         for entity in mod._entity_list:
>> 
>> Visits in the same order as long as .values() is in insertion order.
>> That's the case only for Python 3.6 and later.  Before, it's in some
>> arbitrary order, which results in broken generated code.
>> 
>> Fix by making self._module_dict an OrderedDict rather than a dict.
>> 
>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  scripts/qapi/schema.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>> index 0bfc5256fb..5100110fa2 100644
>> --- a/scripts/qapi/schema.py
>> +++ b/scripts/qapi/schema.py
>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>          self.docs = parser.docs
>>          self._entity_list = []
>>          self._entity_dict = {}
>> -        self._module_dict = {}
>> +        self._module_dict = OrderedDict()
>>          self._schema_dir = os.path.dirname(fname)
>>          self._make_module(None) # built-ins
>>          self._make_module(fname)
>> 
>
> This problem has bitten me *many* times. I'm wondering if there's a
> prescription that isn't just "Wait until we can stipulate 3.6+".

No clue.

3.5 EOL is scheduled for 2020-09-13.
https://devguide.python.org/#status-of-python-branches

We support 3.5 because we support Debian 9.

We'd normally drop support for Debian 9 two years after Debian 10,
i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
truly support Python 3.5 after uptstream EOL seems doubtful.

Re: [PATCH] qapi: Fix code generation with Python 3.5

[John Snow]

On 1/17/20 2:07 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
> 
>> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>> modules" switched QAPISchema.visit() from
>>>
>>>     for entity in self._entity_list:
>>>
>>> effectively to
>>>
>>>     for mod in self._module_dict.values():
>>>         for entity in mod._entity_list:
>>>
>>> Visits in the same order as long as .values() is in insertion order.
>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>> arbitrary order, which results in broken generated code.
>>>
>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>
>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> ---
>>>  scripts/qapi/schema.py | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>> index 0bfc5256fb..5100110fa2 100644
>>> --- a/scripts/qapi/schema.py
>>> +++ b/scripts/qapi/schema.py
>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>          self.docs = parser.docs
>>>          self._entity_list = []
>>>          self._entity_dict = {}
>>> -        self._module_dict = {}
>>> +        self._module_dict = OrderedDict()
>>>          self._schema_dir = os.path.dirname(fname)
>>>          self._make_module(None) # built-ins
>>>          self._make_module(fname)
>>>
>>
>> This problem has bitten me *many* times. I'm wondering if there's a
>> prescription that isn't just "Wait until we can stipulate 3.6+".
> 
> No clue.
> 
> 3.5 EOL is scheduled for 2020-09-13.
> https://devguide.python.org/#status-of-python-branches
> 
> We support 3.5 because we support Debian 9.
> 
> We'd normally drop support for Debian 9 two years after Debian 10,
> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
> truly support Python 3.5 after uptstream EOL seems doubtful.
> 

We should decide whether we consider Debian LTS to be adequately
supported, yes-or-no.

We should use a rule of "two years after successor, or End-of-Support,
whichever comes first."

For Debian, is end of support three years after it comes out, or is it
when the LTS is EOL?

In this specific case, do we trust that Debian 9 LTS will continue to
patch Python3.5 all the way up until July 2021?

--js

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> On 1/17/20 2:07 AM, Markus Armbruster wrote:
>> John Snow <jsnow@redhat.com> writes:
>> 
>>> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>>> modules" switched QAPISchema.visit() from
>>>>
>>>>     for entity in self._entity_list:
>>>>
>>>> effectively to
>>>>
>>>>     for mod in self._module_dict.values():
>>>>         for entity in mod._entity_list:
>>>>
>>>> Visits in the same order as long as .values() is in insertion order.
>>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>>> arbitrary order, which results in broken generated code.
>>>>
>>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>>
>>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>>> ---
>>>>  scripts/qapi/schema.py | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>>> index 0bfc5256fb..5100110fa2 100644
>>>> --- a/scripts/qapi/schema.py
>>>> +++ b/scripts/qapi/schema.py
>>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>>          self.docs = parser.docs
>>>>          self._entity_list = []
>>>>          self._entity_dict = {}
>>>> -        self._module_dict = {}
>>>> +        self._module_dict = OrderedDict()
>>>>          self._schema_dir = os.path.dirname(fname)
>>>>          self._make_module(None) # built-ins
>>>>          self._make_module(fname)
>>>>
>>>
>>> This problem has bitten me *many* times. I'm wondering if there's a
>>> prescription that isn't just "Wait until we can stipulate 3.6+".
>> 
>> No clue.
>> 
>> 3.5 EOL is scheduled for 2020-09-13.
>> https://devguide.python.org/#status-of-python-branches
>> 
>> We support 3.5 because we support Debian 9.
>> 
>> We'd normally drop support for Debian 9 two years after Debian 10,
>> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
>> truly support Python 3.5 after uptstream EOL seems doubtful.
>> 
>
> We should decide whether we consider Debian LTS to be adequately
> supported, yes-or-no.
>
> We should use a rule of "two years after successor, or End-of-Support,
> whichever comes first."

Yes.

> For Debian, is end of support three years after it comes out, or is it
> when the LTS is EOL?

We need to define end-of-support for Debian: is it Debian proper or is
it Debian LTS?

<https://wiki.debian.org/DebianOldStable>:

    Q) How long will security updates be provided?

    The security team tries to support a stable distribution for about
    one year after the next stable distribution has been released,
    except when another stable distribution is released within this
    year.  It is not possible to support three distributions; supporting
    two simultaneously is already difficult enough.

<https://wiki.debian.org/LTS>:

    Debian Long Term Support (LTS) is a project to extend the lifetime
    of all Debian stable releases to (at least) 5 years.  Debian LTS is
    not handled by the Debian security team, but by a separate group of
    volunteers and companies interested in making it a success.

    Thus the Debian LTS team takes over security maintenance of the
    various releases once the Debian Security team stops its work.

Debian 10 "Buster" was released in July 2019.  Debian 9 "Stretch" will
receive security updates from Debian until mid 2020, i.e. just about
when Python 3.5 reaches EOL.  LTS will attempt to support it until June
2022.

I think we should give ourselves a bit more flexibility than the
categorical "Support for the previous major version will be dropped 2
years after the new major version is released."  At some point, the cost
of supporting old hosts exceeds the utility.  We should face this
tradeoff.

> In this specific case, do we trust that Debian 9 LTS will continue to
> patch Python3.5 all the way up until July 2021?

June 2022 even.

We use Python at compile time with trusted input.  The need for security
maintenance is relatively low there.  I'm not ready to vouch for "and we
don't use Python for anything else".

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Eduardo Habkost]

On Sat, Jan 18, 2020 at 07:54:18AM +0100, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
> > On 1/17/20 2:07 AM, Markus Armbruster wrote:
> >> John Snow <jsnow@redhat.com> writes:
[...]
> >>> This problem has bitten me *many* times. I'm wondering if there's a
> >>> prescription that isn't just "Wait until we can stipulate 3.6+".
> >> 
> >> No clue.
> >> 
> >> 3.5 EOL is scheduled for 2020-09-13.
> >> https://devguide.python.org/#status-of-python-branches
> >> 
> >> We support 3.5 because we support Debian 9.
> >> 
> >> We'd normally drop support for Debian 9 two years after Debian 10,
> >> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
> >> truly support Python 3.5 after uptstream EOL seems doubtful.
> >> 
> >
> > We should decide whether we consider Debian LTS to be adequately
> > supported, yes-or-no.
> >
> > We should use a rule of "two years after successor, or End-of-Support,
> > whichever comes first."
> 
> Yes.
> 
> > For Debian, is end of support three years after it comes out, or is it
> > when the LTS is EOL?
> 
> We need to define end-of-support for Debian: is it Debian proper or is
> it Debian LTS?
> 
> <https://wiki.debian.org/DebianOldStable>:
> 
>     Q) How long will security updates be provided?
> 
>     The security team tries to support a stable distribution for about
>     one year after the next stable distribution has been released,
>     except when another stable distribution is released within this
>     year.  It is not possible to support three distributions; supporting
>     two simultaneously is already difficult enough.
> 
> <https://wiki.debian.org/LTS>:
> 
>     Debian Long Term Support (LTS) is a project to extend the lifetime
>     of all Debian stable releases to (at least) 5 years.  Debian LTS is
>     not handled by the Debian security team, but by a separate group of
>     volunteers and companies interested in making it a success.
> 
>     Thus the Debian LTS team takes over security maintenance of the
>     various releases once the Debian Security team stops its work.

As Debian LTS is maintained by a separate group, I interpret
"Debian EOL" as not including LTS.

Supporting Debian 9 in 2020 is already being a burden.
Supporting it until mid-2021 seems pointless.


> 
> Debian 10 "Buster" was released in July 2019.  Debian 9 "Stretch" will
> receive security updates from Debian until mid 2020, i.e. just about
> when Python 3.5 reaches EOL.  LTS will attempt to support it until June
> 2022.
> 
> I think we should give ourselves a bit more flexibility than the
> categorical "Support for the previous major version will be dropped 2
> years after the new major version is released."  At some point, the cost
> of supporting old hosts exceeds the utility.  We should face this
> tradeoff.

Agreed.

-- 
Eduardo

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Philippe Mathieu-Daudé]

On 1/17/20 8:41 PM, John Snow wrote:
> On 1/17/20 2:07 AM, Markus Armbruster wrote:
>> John Snow <jsnow@redhat.com> writes:
>>
>>> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>>> modules" switched QAPISchema.visit() from
>>>>
>>>>      for entity in self._entity_list:
>>>>
>>>> effectively to
>>>>
>>>>      for mod in self._module_dict.values():
>>>>          for entity in mod._entity_list:
>>>>
>>>> Visits in the same order as long as .values() is in insertion order.
>>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>>> arbitrary order, which results in broken generated code.
>>>>
>>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>>
>>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>>> ---
>>>>   scripts/qapi/schema.py | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>>> index 0bfc5256fb..5100110fa2 100644
>>>> --- a/scripts/qapi/schema.py
>>>> +++ b/scripts/qapi/schema.py
>>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>>           self.docs = parser.docs
>>>>           self._entity_list = []
>>>>           self._entity_dict = {}
>>>> -        self._module_dict = {}
>>>> +        self._module_dict = OrderedDict()
>>>>           self._schema_dir = os.path.dirname(fname)
>>>>           self._make_module(None) # built-ins
>>>>           self._make_module(fname)
>>>>
>>>
>>> This problem has bitten me *many* times. I'm wondering if there's a
>>> prescription that isn't just "Wait until we can stipulate 3.6+".
>>
>> No clue.
>>
>> 3.5 EOL is scheduled for 2020-09-13.
>> https://devguide.python.org/#status-of-python-branches
>>
>> We support 3.5 because we support Debian 9.
>>
>> We'd normally drop support for Debian 9 two years after Debian 10,
>> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
>> truly support Python 3.5 after uptstream EOL seems doubtful.
>>
> 
> We should decide whether we consider Debian LTS to be adequately
> supported, yes-or-no.
> 
> We should use a rule of "two years after successor, or End-of-Support,
> whichever comes first."
> 
> For Debian, is end of support three years after it comes out, or is it
> when the LTS is EOL?
> 
> In this specific case, do we trust that Debian 9 LTS will continue to
> patch Python3.5 all the way up until July 2021?

This broke 29 of the 32 Travis jobs we have:

https://travis-ci.org/qemu/qemu/builds/637999366

Since we started to use Travis CI, it catched quite some bugs...

I think it is important to add in the equation we also depend of our CIs.

[Qemu-devel] Xenial in Travis (was: qapi: Fix code generation with Python 3.5)

[Thomas Huth]

On 18/01/2020 09.29, Philippe Mathieu-Daudé wrote:
[...]
> This broke 29 of the 32 Travis jobs we have:
> 
> https://travis-ci.org/qemu/qemu/builds/637999366
> 
> Since we started to use Travis CI, it catched quite some bugs...
> 
> I think it is important to add in the equation we also depend of our CIs.

Thinking about the state of our Travis CI ... The Travis CI jobs that
broke are still running Ubuntu Xenial (v16.04). According to our support
policy, we will stop supporting Xenial two years after v18.04 LTS has
been released, i.e. in April this year. So we have to update our Travis
CI jobs to a newer version with a newer Python very soon anyway - the
question is here only whether we update to v18.04 (with the broken
libssh version that we don't support in QEMU), or rather directly to
v20.04 (which might take a while 'till it shows up as an option in
Travis)...

 Thomas

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Thomas Huth]

On 16/01/2020 21.25, Markus Armbruster wrote:
> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
> 
>     for entity in self._entity_list:
> 
> effectively to
> 
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
> 
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
> 
> Fix by making self._module_dict an OrderedDict rather than a dict.
> 
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  scripts/qapi/schema.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 0bfc5256fb..5100110fa2 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -795,7 +795,7 @@ class QAPISchema(object):
>          self.docs = parser.docs
>          self._entity_list = []
>          self._entity_dict = {}
> -        self._module_dict = {}
> +        self._module_dict = OrderedDict()
>          self._schema_dir = os.path.dirname(fname)
>          self._make_module(None) # built-ins
>          self._make_module(fname)
> 

Thanks, this fixes the problems on Travis for me!

Tested-by: Thomas Huth <thuth@redhat.com>

Peter, could you maybe apply this directly to the master branch as a
build fix?

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Markus Armbruster]

Thomas Huth <thuth@redhat.com> writes:

> On 16/01/2020 21.25, Markus Armbruster wrote:
>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>> modules" switched QAPISchema.visit() from
>> 
>>     for entity in self._entity_list:
>> 
>> effectively to
>> 
>>     for mod in self._module_dict.values():
>>         for entity in mod._entity_list:
>> 
>> Visits in the same order as long as .values() is in insertion order.
>> That's the case only for Python 3.6 and later.  Before, it's in some
>> arbitrary order, which results in broken generated code.
>> 
>> Fix by making self._module_dict an OrderedDict rather than a dict.
>> 
>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  scripts/qapi/schema.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>> index 0bfc5256fb..5100110fa2 100644
>> --- a/scripts/qapi/schema.py
>> +++ b/scripts/qapi/schema.py
>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>          self.docs = parser.docs
>>          self._entity_list = []
>>          self._entity_dict = {}
>> -        self._module_dict = {}
>> +        self._module_dict = OrderedDict()
>>          self._schema_dir = os.path.dirname(fname)
>>          self._make_module(None) # built-ins
>>          self._make_module(fname)
>> 
>
> Thanks, this fixes the problems on Travis for me!
>
> Tested-by: Thomas Huth <thuth@redhat.com>
>
> Peter, could you maybe apply this directly to the master branch as a
> build fix?

The commit message isn't quite right: s/Visits in the same order/Visits
modules in the same order/.  Peter, want me to respin for that?

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Philippe Mathieu-Daudé]

On 1/17/20 11:49 AM, Markus Armbruster wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> On 16/01/2020 21.25, Markus Armbruster wrote:
>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>> modules" switched QAPISchema.visit() from
>>>
>>>      for entity in self._entity_list:
>>>
>>> effectively to
>>>
>>>      for mod in self._module_dict.values():
>>>          for entity in mod._entity_list:
>>>
>>> Visits in the same order as long as .values() is in insertion order.
>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>> arbitrary order, which results in broken generated code.
>>>
>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>
>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> ---
>>>   scripts/qapi/schema.py | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>> index 0bfc5256fb..5100110fa2 100644
>>> --- a/scripts/qapi/schema.py
>>> +++ b/scripts/qapi/schema.py
>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>           self.docs = parser.docs
>>>           self._entity_list = []
>>>           self._entity_dict = {}
>>> -        self._module_dict = {}
>>> +        self._module_dict = OrderedDict()
>>>           self._schema_dir = os.path.dirname(fname)
>>>           self._make_module(None) # built-ins
>>>           self._make_module(fname)
>>>
>>
>> Thanks, this fixes the problems on Travis for me!
>>
>> Tested-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

>> Peter, could you maybe apply this directly to the master branch as a
>> build fix?
> 
> The commit message isn't quite right: s/Visits in the same order/Visits
> modules in the same order/.  Peter, want me to respin for that?

Since it is a single patch, it shouldn't be too much work to respin :)

I agree this patch is candidate for direct fix on /master.

Thanks,

Phil.

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Philippe Mathieu-Daudé]

On 1/18/20 9:33 AM, Philippe Mathieu-Daudé wrote:
> On 1/17/20 11:49 AM, Markus Armbruster wrote:
>> Thomas Huth <thuth@redhat.com> writes:
>>
>>> On 16/01/2020 21.25, Markus Armbruster wrote:
>>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>>> modules" switched QAPISchema.visit() from
>>>>
>>>>      for entity in self._entity_list:
>>>>
>>>> effectively to
>>>>
>>>>      for mod in self._module_dict.values():
>>>>          for entity in mod._entity_list:
>>>>
>>>> Visits in the same order as long as .values() is in insertion order.
>>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>>> arbitrary order, which results in broken generated code.
>>>>
>>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>>
>>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>>> ---
>>>>   scripts/qapi/schema.py | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>>> index 0bfc5256fb..5100110fa2 100644
>>>> --- a/scripts/qapi/schema.py
>>>> +++ b/scripts/qapi/schema.py
>>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>>           self.docs = parser.docs
>>>>           self._entity_list = []
>>>>           self._entity_dict = {}
>>>> -        self._module_dict = {}
>>>> +        self._module_dict = OrderedDict()
>>>>           self._schema_dir = os.path.dirname(fname)
>>>>           self._make_module(None) # built-ins
>>>>           self._make_module(fname)
>>>>
>>>
>>> Thanks, this fixes the problems on Travis for me!
>>>
>>> Tested-by: Thomas Huth <thuth@redhat.com>
> 
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

And per https://www.mail-archive.com/qemu-devel@nongnu.org/msg671745.html:

Tested-by: BALATON Zoltan <balaton@eik.bme.hu>

>>> Peter, could you maybe apply this directly to the master branch as a
>>> build fix?
>>
>> The commit message isn't quite right: s/Visits in the same order/Visits
>> modules in the same order/.  Peter, want me to respin for that?
> 
> Since it is a single patch, it shouldn't be too much work to respin :)
> 
> I agree this patch is candidate for direct fix on /master.
> 
> Thanks,
> 
> Phil.
> 
> 

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Alex Bennée]

Markus Armbruster <armbru@redhat.com> writes:

> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
>
>     for entity in self._entity_list:
>
> effectively to
>
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
>
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
>
> Fix by making self._module_dict an OrderedDict rather than a dict.
>
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>

Well that certainly clears up a bunch of red. Can we apply it directly
as a build fix?

> ---
>  scripts/qapi/schema.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 0bfc5256fb..5100110fa2 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -795,7 +795,7 @@ class QAPISchema(object):
>          self.docs = parser.docs
>          self._entity_list = []
>          self._entity_dict = {}
> -        self._module_dict = {}
> +        self._module_dict = OrderedDict()
>          self._schema_dir = os.path.dirname(fname)
>          self._make_module(None) # built-ins
>          self._make_module(fname)


-- 
Alex Bennée

Re: [PATCH] qapi: Fix code generation with Python 3.5

[Peter Maydell]

On Thu, 16 Jan 2020 at 20:27, Markus Armbruster <armbru@redhat.com> wrote:
>
> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
>
>     for entity in self._entity_list:
>
> effectively to
>
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
>
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
>
> Fix by making self._module_dict an OrderedDict rather than a dict.
>
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> --


Applied to master as a buildfix, thanks.

-- PMM