any remaining for-6.0 issues?

any remaining for-6.0 issues?

[Peter Maydell]

Last call to note anything we need to fix for 6.0 on
https://wiki.qemu.org/Planning/6.0#Known_issues please.
The schedule is to tag rc3 tomorrow, which I would ideally like
to be the last rc before release. After rc3 I will only be taking
fixes for bugs which are absolutely critical...

thanks
-- PMM

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 12/04/2021 16:32, Peter Maydell wrote:

> Last call to note anything we need to fix for 6.0 on
> https://wiki.qemu.org/Planning/6.0#Known_issues please.
> The schedule is to tag rc3 tomorrow, which I would ideally like
> to be the last rc before release. After rc3 I will only be taking
> fixes for bugs which are absolutely critical...

Thanks for reviewing one of the patches from the ESP security patchset earlier.

Should I send a PR for the ESP security patchset and the acceptance test fix? 
Unfortunately the timing of the holidays has meant that these are missing a review 
from a SCSI maintainer but IMO both of these are worthy for 6.0.


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Peter Maydell]

On Mon, 12 Apr 2021 at 19:44, Mark Cave-Ayland
<mark.cave-ayland@ilande.co.uk> wrote:
>
> On 12/04/2021 16:32, Peter Maydell wrote:
>
> > Last call to note anything we need to fix for 6.0 on
> > https://wiki.qemu.org/Planning/6.0#Known_issues please.
> > The schedule is to tag rc3 tomorrow, which I would ideally like
> > to be the last rc before release. After rc3 I will only be taking
> > fixes for bugs which are absolutely critical...
>
> Thanks for reviewing one of the patches from the ESP security patchset earlier.
>
> Should I send a PR for the ESP security patchset and the acceptance test fix?
> Unfortunately the timing of the holidays has meant that these are missing a review
> from a SCSI maintainer but IMO both of these are worthy for 6.0.

Yes, please do. If you could add your missing copyright line to
the comment in the new file in patch 12 as well that would be great.

thanks
-- PMM

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 12/04/2021 20:40, Peter Maydell wrote:

>> On 12/04/2021 16:32, Peter Maydell wrote:
>>
>>> Last call to note anything we need to fix for 6.0 on
>>> https://wiki.qemu.org/Planning/6.0#Known_issues please.
>>> The schedule is to tag rc3 tomorrow, which I would ideally like
>>> to be the last rc before release. After rc3 I will only be taking
>>> fixes for bugs which are absolutely critical...
>>
>> Thanks for reviewing one of the patches from the ESP security patchset earlier.
>>
>> Should I send a PR for the ESP security patchset and the acceptance test fix?
>> Unfortunately the timing of the holidays has meant that these are missing a review
>> from a SCSI maintainer but IMO both of these are worthy for 6.0.
> 
> Yes, please do. If you could add your missing copyright line to
> the comment in the new file in patch 12 as well that would be great.

Okay, great - just reading over today's thread so should be able to send a PR shortly.


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 12/04/2021 20:40, Peter Maydell wrote:

> On Mon, 12 Apr 2021 at 19:44, Mark Cave-Ayland
> <mark.cave-ayland@ilande.co.uk> wrote:
>>
>> On 12/04/2021 16:32, Peter Maydell wrote:
>>
>>> Last call to note anything we need to fix for 6.0 on
>>> https://wiki.qemu.org/Planning/6.0#Known_issues please.
>>> The schedule is to tag rc3 tomorrow, which I would ideally like
>>> to be the last rc before release. After rc3 I will only be taking
>>> fixes for bugs which are absolutely critical...
>>
>> Thanks for reviewing one of the patches from the ESP security patchset earlier.
>>
>> Should I send a PR for the ESP security patchset and the acceptance test fix?
>> Unfortunately the timing of the holidays has meant that these are missing a review
>> from a SCSI maintainer but IMO both of these are worthy for 6.0.
> 
> Yes, please do. If you could add your missing copyright line to
> the comment in the new file in patch 12 as well that would be great.

All done. I've added the copyright to patch 12 (along with yours and Phil's R-B tags) 
so it will appear in the upcoming PR.


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Markus Armbruster]

Peter Maydell <peter.maydell@linaro.org> writes:

> Last call to note anything we need to fix for 6.0 on
> https://wiki.qemu.org/Planning/6.0#Known_issues please.

Done for

    [PATCH 0/1] sphinx: qapidoc: Wrap "If" section body in a paragraph node

I asked how to get it merged last week:

    Message-ID: <87r1jmzb1n.fsf@dusky.pond.sub.org>
    https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg00818.html

"In 6.1" is a valid, if sad answer.

> The schedule is to tag rc3 tomorrow, which I would ideally like
> to be the last rc before release. After rc3 I will only be taking
> fixes for bugs which are absolutely critical...

Understood.

Re: any remaining for-6.0 issues?

[Bin Meng]

On Mon, Apr 12, 2021 at 11:33 PM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> Last call to note anything we need to fix for 6.0 on
> https://wiki.qemu.org/Planning/6.0#Known_issues please.
> The schedule is to tag rc3 tomorrow, which I would ideally like
> to be the last rc before release. After rc3 I will only be taking
> fixes for bugs which are absolutely critical...

This patch (affects Windows install)
http://patchwork.ozlabs.org/project/qemu-devel/patch/20210326062140.367861-1-bmeng.cn@gmail.com/

is still not applied.

Regards,
Bin

Re: any remaining for-6.0 issues?

[Stefan Weil]

Am 13.04.21 um 07:56 schrieb Bin Meng:

> On Mon, Apr 12, 2021 at 11:33 PM Peter Maydell <peter.maydell@linaro.org> wrote:
>> Last call to note anything we need to fix for 6.0 on
>> https://wiki.qemu.org/Planning/6.0#Known_issues please.
>> The schedule is to tag rc3 tomorrow, which I would ideally like
>> to be the last rc before release. After rc3 I will only be taking
>> fixes for bugs which are absolutely critical...
> This patch (affects Windows install)
> http://patchwork.ozlabs.org/project/qemu-devel/patch/20210326062140.367861-1-bmeng.cn@gmail.com/
>
> is still not applied.
>
> Regards,
> Bin


That patch is based on an older version of my personal QEMU sources and 
not required for 6.0.

The official QEMU sources install the whole share directory, so all 
required firmware files are included. My latest installer 
(https://qemu.weilnetz.de/w64/qemu-w64-setup-20210409.exe) installs 
those files, too.

Regards,

Stefan

Re: any remaining for-6.0 issues?

[Bin Meng]

Hi Stefan,

On Tue, Apr 13, 2021 at 2:19 PM Stefan Weil <sw@weilnetz.de> wrote:
>
> Am 13.04.21 um 07:56 schrieb Bin Meng:
>
> > On Mon, Apr 12, 2021 at 11:33 PM Peter Maydell <peter.maydell@linaro.org> wrote:
> >> Last call to note anything we need to fix for 6.0 on
> >> https://wiki.qemu.org/Planning/6.0#Known_issues please.
> >> The schedule is to tag rc3 tomorrow, which I would ideally like
> >> to be the last rc before release. After rc3 I will only be taking
> >> fixes for bugs which are absolutely critical...
> > This patch (affects Windows install)
> > http://patchwork.ozlabs.org/project/qemu-devel/patch/20210326062140.367861-1-bmeng.cn@gmail.com/
> >
> > is still not applied.
> >
> > Regards,
> > Bin
>
>
> That patch is based on an older version of my personal QEMU sources and
> not required for 6.0.

I am confused.

I see https://repo.or.cz/qemu/ar7.git/blob/HEAD:/qemu.nsi still does
not contain the fix.

Or is this qemu.nsi file not used in the latest 6.0 installer?

>
> The official QEMU sources install the whole share directory, so all
> required firmware files are included. My latest installer
> (https://qemu.weilnetz.de/w64/qemu-w64-setup-20210409.exe) installs
> those files, too.

Regards,
Bin

Re: any remaining for-6.0 issues?

[Stefan Weil]

Am 14.04.21 um 10:30 schrieb Bin Meng:

> Hi Stefan,
>
> On Tue, Apr 13, 2021 at 2:19 PM Stefan Weil <sw@weilnetz.de> wrote:
>> Am 13.04.21 um 07:56 schrieb Bin Meng:
>>
>>> This patch (affects Windows install)
>>> http://patchwork.ozlabs.org/project/qemu-devel/patch/20210326062140.367861-1-bmeng.cn@gmail.com/
>>>
>>> is still not applied.
>>>
>>> Regards,
>>> Bin
>>
>> That patch is based on an older version of my personal QEMU sources and
>> not required for 6.0.
> I am confused.
>
> I see https://repo.or.cz/qemu/ar7.git/blob/HEAD:/qemu.nsi still does
> not contain the fix.
>
> Or is this qemu.nsi file not used in the latest 6.0 installer?


https://repo.or.cz/qemu/ar7.git/blob/HEAD:/qemu.nsi#l139 installs all 
shared files recursively (same as in the official QEMU git master), so 
no fix is required.

Stefan

Re: any remaining for-6.0 issues?

[Peter Maydell]

On Wed, 14 Apr 2021 at 09:31, Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Stefan,
>
> On Tue, Apr 13, 2021 at 2:19 PM Stefan Weil <sw@weilnetz.de> wrote:
> > That patch is based on an older version of my personal QEMU sources and
> > not required for 6.0.
>
> I am confused.
>
> I see https://repo.or.cz/qemu/ar7.git/blob/HEAD:/qemu.nsi still does
> not contain the fix.
>
> Or is this qemu.nsi file not used in the latest 6.0 installer?

In any case, that is a branch, and so any issues in it are not relevant
to 6.0 unless they are also a problem for QEMU master itself.

thanks
-- PMM

Re: any remaining for-6.0 issues?

[Thomas Huth]

On 12/04/2021 17.32, Peter Maydell wrote:
> Last call to note anything we need to fix for 6.0 on
> https://wiki.qemu.org/Planning/6.0#Known_issues please.
> The schedule is to tag rc3 tomorrow, which I would ideally like
> to be the last rc before release. After rc3 I will only be taking
> fixes for bugs which are absolutely critical...

  Hi Peter,

I've seen some intermittend, non-reproducible crashes with usermode QEMU in 
some of my CI runs, e.g.:

  https://gitlab.com/thuth/qemu/-/jobs/1178256498#L3876

  https://gitlab.com/thuth/qemu/-/jobs/1146276208#L3241

This was only with ccache enabled, so I thought that it might be related to 
my work-in-progress ccache patches...

... but now Cornelia reported that she has seen such a crash in one of her 
branches, too (which is completely unrelated to my ccache patches):

  https://gitlab.com/cohuck/qemu/-/jobs/1178860927#L3867

That makes me wonder whether we currently have a real problem with user-mode 
in the master branch? Did anybody else see such problems?

  Thomas

Re: any remaining for-6.0 issues?

[Bin Meng]

On Wed, Apr 14, 2021 at 5:35 PM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Wed, 14 Apr 2021 at 09:31, Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Stefan,
> >
> > On Tue, Apr 13, 2021 at 2:19 PM Stefan Weil <sw@weilnetz.de> wrote:
> > > That patch is based on an older version of my personal QEMU sources and
> > > not required for 6.0.
> >
> > I am confused.
> >
> > I see https://repo.or.cz/qemu/ar7.git/blob/HEAD:/qemu.nsi still does
> > not contain the fix.
> >
> > Or is this qemu.nsi file not used in the latest 6.0 installer?
>
> In any case, that is a branch, and so any issues in it are not relevant
> to 6.0 unless they are also a problem for QEMU master itself.

That's why I mentioned in the patch notes, that I was not aware of the
Windows installer patch process.

The QEMU Windows binary download page [1] says:

> The installers are generated from my latest QEMU sources and updated frequently. See the build instructions for details.

which refers to Stefan's repo.

If this is no longer true, which means QEMU Windows binaries are built
from the mainline, please update the page to avoid further confusion.

[1] https://qemu.weilnetz.de/w64/

Regards,
Bin

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 14/04/2021 13:48, Thomas Huth wrote:

> On 12/04/2021 17.32, Peter Maydell wrote:
>> Last call to note anything we need to fix for 6.0 on
>> https://wiki.qemu.org/Planning/6.0#Known_issues please.
>> The schedule is to tag rc3 tomorrow, which I would ideally like
>> to be the last rc before release. After rc3 I will only be taking
>> fixes for bugs which are absolutely critical...
> 
> Hi Peter,
> 
> I've seen some intermittend, non-reproducible crashes with usermode QEMU in some of 
> my CI runs, e.g.:
> 
> https://gitlab.com/thuth/qemu/-/jobs/1178256498#L3876
> 
> https://gitlab.com/thuth/qemu/-/jobs/1146276208#L3241
> 
> This was only with ccache enabled, so I thought that it might be related to my 
> work-in-progress ccache patches...
> 
> ... but now Cornelia reported that she has seen such a crash in one of her branches, 
> too (which is completely unrelated to my ccache patches):
> 
> https://gitlab.com/cohuck/qemu/-/jobs/1178860927#L3867
> 
> That makes me wonder whether we currently have a real problem with user-mode in the 
> master branch? Did anybody else see such problems?

I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP 
security fixes (first version of which appeared just before rc0). The user builds 
always fail on "run-tcg-tests-s390x-linux-user" for me.

Looking back through my gitlab CI history for the ESP patches I first see the issue 
in clang-user here:

   https://gitlab.com/mcayland/qemu/-/pipelines/278781899 (master: ec2e6e01)

and for the PR I just sent the failure is now present in build-user:

   https://gitlab.com/mcayland/qemu/-/pipelines/285320238

The last build I have that succeeded before seeing the clang-user failure is this one:

   https://gitlab.com/mcayland/qemu/-/pipelines/271328298 (master: 757acb9a)

That seems to suggest that the issue was first introduced in git master somewhere 
between 757acb9a and ec2e6e01.


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 14/04/2021 14:57, Mark Cave-Ayland wrote:

> I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP 
> security fixes (first version of which appeared just before rc0). The user builds 
> always fail on "run-tcg-tests-s390x-linux-user" for me.

Commit 23fff7a17f lies within that range and certainly feels like it could be 
relevant to the above failure:


commit 23fff7a17f47420797ac6480147941612152a9ad
Author: Andreas Krebbel <krebbel@linux.ibm.com>
Date:   Wed Mar 24 19:51:28 2021 +0100

     linux-user/s390x: Use the guest pointer for the sigreturn stub

     When setting up the pointer for the sigreturn stub in the return
     address register (r14) we currently use the host frame address instead
     of the guest frame address.

     Note: This only caused problems if Qemu has been built with
     --disable-pie (as it is in distros nowadays). Otherwise guest_base
     defaults to 0 hiding the actual problem.

     Signed-off-by: Andreas Krebbel <krebbel@linux.ibm.com>
     Reviewed-by: Laurent Vivier <laurent@vivier.eu>
     Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
     Message-Id: <20210324185128.63971-1-krebbel@linux.ibm.com>
     Signed-off-by: Laurent Vivier <laurent@vivier.eu>


I'll try pushing a test build to Gitlab CI with a revert for that patch on top and 
see if it succeeds for me...


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Cornelia Huck]

On Wed, 14 Apr 2021 15:15:04 +0100
Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> wrote:

> On 14/04/2021 14:57, Mark Cave-Ayland wrote:
> 
> > I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP 
> > security fixes (first version of which appeared just before rc0). The user builds 
> > always fail on "run-tcg-tests-s390x-linux-user" for me.  
> 
> Commit 23fff7a17f lies within that range and certainly feels like it could be 
> relevant to the above failure:
> 
> 
> commit 23fff7a17f47420797ac6480147941612152a9ad
> Author: Andreas Krebbel <krebbel@linux.ibm.com>
> Date:   Wed Mar 24 19:51:28 2021 +0100
> 
>      linux-user/s390x: Use the guest pointer for the sigreturn stub
> 
>      When setting up the pointer for the sigreturn stub in the return
>      address register (r14) we currently use the host frame address instead
>      of the guest frame address.
> 
>      Note: This only caused problems if Qemu has been built with
>      --disable-pie (as it is in distros nowadays). Otherwise guest_base
>      defaults to 0 hiding the actual problem.
> 
>      Signed-off-by: Andreas Krebbel <krebbel@linux.ibm.com>
>      Reviewed-by: Laurent Vivier <laurent@vivier.eu>
>      Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>      Message-Id: <20210324185128.63971-1-krebbel@linux.ibm.com>
>      Signed-off-by: Laurent Vivier <laurent@vivier.eu>
> 
> 
> I'll try pushing a test build to Gitlab CI with a revert for that patch on top and 
> see if it succeeds for me...

Hm. I'm actually seeing it sometimes succeeding, and sometimes failing.
I'm wondering if there's something else at play here...

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 14/04/2021 15:36, Cornelia Huck wrote:

> On Wed, 14 Apr 2021 15:15:04 +0100
> Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> wrote:
> 
>> On 14/04/2021 14:57, Mark Cave-Ayland wrote:
>>
>>> I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP
>>> security fixes (first version of which appeared just before rc0). The user builds
>>> always fail on "run-tcg-tests-s390x-linux-user" for me.
>>
>> Commit 23fff7a17f lies within that range and certainly feels like it could be
>> relevant to the above failure:
>>
>>
>> commit 23fff7a17f47420797ac6480147941612152a9ad
>> Author: Andreas Krebbel <krebbel@linux.ibm.com>
>> Date:   Wed Mar 24 19:51:28 2021 +0100
>>
>>       linux-user/s390x: Use the guest pointer for the sigreturn stub
>>
>>       When setting up the pointer for the sigreturn stub in the return
>>       address register (r14) we currently use the host frame address instead
>>       of the guest frame address.
>>
>>       Note: This only caused problems if Qemu has been built with
>>       --disable-pie (as it is in distros nowadays). Otherwise guest_base
>>       defaults to 0 hiding the actual problem.
>>
>>       Signed-off-by: Andreas Krebbel <krebbel@linux.ibm.com>
>>       Reviewed-by: Laurent Vivier <laurent@vivier.eu>
>>       Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>>       Message-Id: <20210324185128.63971-1-krebbel@linux.ibm.com>
>>       Signed-off-by: Laurent Vivier <laurent@vivier.eu>
>>
>>
>> I'll try pushing a test build to Gitlab CI with a revert for that patch on top and
>> see if it succeeds for me...
> 
> Hm. I'm actually seeing it sometimes succeeding, and sometimes failing.
> I'm wondering if there's something else at play here...

That could also be possible, and it may be that I have just been unlucky with my 
builds. Even if it isn't particularly conclusive I can report back that my test 
Gitlab CI build with a revert of the above patch did pass clang-user and build-user: 
https://gitlab.com/mcayland/qemu/-/pipelines/286332527.


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Peter Maydell]

On Wed, 14 Apr 2021 at 14:58, Mark Cave-Ayland
<mark.cave-ayland@ilande.co.uk> wrote:
>
> On 14/04/2021 13:48, Thomas Huth wrote:
> > I've seen some intermittend, non-reproducible crashes with usermode QEMU in some of
> > my CI runs, e.g.:
> >
> > https://gitlab.com/thuth/qemu/-/jobs/1178256498#L3876
> >
> > https://gitlab.com/thuth/qemu/-/jobs/1146276208#L3241
> >
> > This was only with ccache enabled, so I thought that it might be related to my
> > work-in-progress ccache patches...
> >
> > ... but now Cornelia reported that she has seen such a crash in one of her branches,
> > too (which is completely unrelated to my ccache patches):
> >
> > https://gitlab.com/cohuck/qemu/-/jobs/1178860927#L3867
> >
> > That makes me wonder whether we currently have a real problem with user-mode in the
> > master branch? Did anybody else see such problems?
>
> I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP
> security fixes (first version of which appeared just before rc0). The user builds
> always fail on "run-tcg-tests-s390x-linux-user" for me.

Do we have any better understanding yet of the cause here?
(I ask because I think we're going to need an rc4 for other reasons,
so if there's a ready-to-go fix then we could consider it.)

thanks
-- PMM

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 17/04/2021 20:46, Peter Maydell wrote:

>> On 14/04/2021 13:48, Thomas Huth wrote:
>>> I've seen some intermittend, non-reproducible crashes with usermode QEMU in some of
>>> my CI runs, e.g.:
>>>
>>> https://gitlab.com/thuth/qemu/-/jobs/1178256498#L3876
>>>
>>> https://gitlab.com/thuth/qemu/-/jobs/1146276208#L3241
>>>
>>> This was only with ccache enabled, so I thought that it might be related to my
>>> work-in-progress ccache patches...
>>>
>>> ... but now Cornelia reported that she has seen such a crash in one of her branches,
>>> too (which is completely unrelated to my ccache patches):
>>>
>>> https://gitlab.com/cohuck/qemu/-/jobs/1178860927#L3867
>>>
>>> That makes me wonder whether we currently have a real problem with user-mode in the
>>> master branch? Did anybody else see such problems?
>>
>> I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP
>> security fixes (first version of which appeared just before rc0). The user builds
>> always fail on "run-tcg-tests-s390x-linux-user" for me.
> 
> Do we have any better understanding yet of the cause here?
> (I ask because I think we're going to need an rc4 for other reasons,
> so if there's a ready-to-go fix then we could consider it.)

I don't think so. I tried a run with a possible candidate patch reverted (see 
https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg02345.html) but Cornelia's 
response indicates that the result is still inconclusive :(


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Cornelia Huck]

On Sun, 18 Apr 2021 11:38:09 +0100
Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> wrote:

> On 17/04/2021 20:46, Peter Maydell wrote:
> 
> >> On 14/04/2021 13:48, Thomas Huth wrote:  
> >>> I've seen some intermittend, non-reproducible crashes with usermode QEMU in some of
> >>> my CI runs, e.g.:
> >>>
> >>> https://gitlab.com/thuth/qemu/-/jobs/1178256498#L3876
> >>>
> >>> https://gitlab.com/thuth/qemu/-/jobs/1146276208#L3241
> >>>
> >>> This was only with ccache enabled, so I thought that it might be related to my
> >>> work-in-progress ccache patches...
> >>>
> >>> ... but now Cornelia reported that she has seen such a crash in one of her branches,
> >>> too (which is completely unrelated to my ccache patches):
> >>>
> >>> https://gitlab.com/cohuck/qemu/-/jobs/1178860927#L3867
> >>>
> >>> That makes me wonder whether we currently have a real problem with user-mode in the
> >>> master branch? Did anybody else see such problems?  
> >>
> >> I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP
> >> security fixes (first version of which appeared just before rc0). The user builds
> >> always fail on "run-tcg-tests-s390x-linux-user" for me.  
> > 
> > Do we have any better understanding yet of the cause here?
> > (I ask because I think we're going to need an rc4 for other reasons,
> > so if there's a ready-to-go fix then we could consider it.)  
> 
> I don't think so. I tried a run with a possible candidate patch reverted (see 
> https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg02345.html) but Cornelia's 
> response indicates that the result is still inconclusive :(

That patch seems to be our best candidate so far, but the intermittent
nature of the failures make it hard to pin down... I don't see anything
obviously wrong with the patch, maybe some linux-user experts have a
better idea?

Re: any remaining for-6.0 issues?

[Cornelia Huck]

On Mon, 19 Apr 2021 09:05:51 +0200
Cornelia Huck <cohuck@redhat.com> wrote:

> On Sun, 18 Apr 2021 11:38:09 +0100
> Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> wrote:
> 
> > On 17/04/2021 20:46, Peter Maydell wrote:
> >   
> > >> On 14/04/2021 13:48, Thomas Huth wrote:    
> > >>> I've seen some intermittend, non-reproducible crashes with usermode QEMU in some of
> > >>> my CI runs, e.g.:
> > >>>
> > >>> https://gitlab.com/thuth/qemu/-/jobs/1178256498#L3876
> > >>>
> > >>> https://gitlab.com/thuth/qemu/-/jobs/1146276208#L3241
> > >>>
> > >>> This was only with ccache enabled, so I thought that it might be related to my
> > >>> work-in-progress ccache patches...
> > >>>
> > >>> ... but now Cornelia reported that she has seen such a crash in one of her branches,
> > >>> too (which is completely unrelated to my ccache patches):
> > >>>
> > >>> https://gitlab.com/cohuck/qemu/-/jobs/1178860927#L3867
> > >>>
> > >>> That makes me wonder whether we currently have a real problem with user-mode in the
> > >>> master branch? Did anybody else see such problems?    
> > >>
> > >> I've definitely seen the same issue as Cornelia in my Gitlab CI builds for the ESP
> > >> security fixes (first version of which appeared just before rc0). The user builds
> > >> always fail on "run-tcg-tests-s390x-linux-user" for me.    
> > > 
> > > Do we have any better understanding yet of the cause here?
> > > (I ask because I think we're going to need an rc4 for other reasons,
> > > so if there's a ready-to-go fix then we could consider it.)    
> > 
> > I don't think so. I tried a run with a possible candidate patch reverted (see 
> > https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg02345.html) but Cornelia's 
> > response indicates that the result is still inconclusive :(  
> 
> That patch seems to be our best candidate so far, but the intermittent
> nature of the failures make it hard to pin down... I don't see anything
> obviously wrong with the patch, maybe some linux-user experts have a
> better idea?

FWIW, I tried reproducing the issue on some local systems (no luck),
and on code pushed out to gitlab (where it works most of the time, and
the user builds where it fails are unpredictable.)

I fear the best we can do right now is stare at the code and try to
figure out what might be wrong :(

Re: any remaining for-6.0 issues?

[Mark Cave-Ayland]

On 19/04/2021 18:02, Cornelia Huck wrote:

>> That patch seems to be our best candidate so far, but the intermittent
>> nature of the failures make it hard to pin down... I don't see anything
>> obviously wrong with the patch, maybe some linux-user experts have a
>> better idea?
> 
> FWIW, I tried reproducing the issue on some local systems (no luck),
> and on code pushed out to gitlab (where it works most of the time, and
> the user builds where it fails are unpredictable.)
> 
> I fear the best we can do right now is stare at the code and try to
> figure out what might be wrong :(

Is there any particular reason why the unsigned long cast was removed from the front? 
Could that have an effect?


ATB,

Mark.

Re: any remaining for-6.0 issues?

[Cornelia Huck]

On Mon, 19 Apr 2021 20:38:05 +0100
Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> wrote:

> On 19/04/2021 18:02, Cornelia Huck wrote:
> 
> >> That patch seems to be our best candidate so far, but the intermittent
> >> nature of the failures make it hard to pin down... I don't see anything
> >> obviously wrong with the patch, maybe some linux-user experts have a
> >> better idea?  
> > 
> > FWIW, I tried reproducing the issue on some local systems (no luck),
> > and on code pushed out to gitlab (where it works most of the time, and
> > the user builds where it fails are unpredictable.)
> > 
> > I fear the best we can do right now is stare at the code and try to
> > figure out what might be wrong :(  
> 
> Is there any particular reason why the unsigned long cast was removed from the front? 
> Could that have an effect?

Indeed, that looks strange.

Will give it a try with the cast readded, but I'm still unable to
reproduce the error reliably...