* Occasional hung with UM after enable VMAP_STACK
@ 2022-01-04 18:10 ` Walter Lozano
0 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-04 18:10 UTC (permalink / raw)
To: johannes, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
Hi all,
I noticed that after "um: enable VMAP_STACK" [1] I experienced some
occasional hung in my Gitlab CI jobs that use user-mode-linux to build
distro images. The configuration use for the building is the one
available from Debian, see [2] and [3]. Unfortunately I don't have much
information, when the issue arises the job seems to hang and it is
cancelled by Gitlab due to a job timeout.
So far I did some bisect to get the commit which introduces the issue
and a quick check with latest Linux stable branch master to confirm the
issue is still present. I know this is not a proper bug report but I
thought it will be better to at least raise a comment.
Thanks in advance,
Walter
[1] http://lists.infradead.org/pipermail/linux-um/2021-August/001657.html
[2] https://packages.debian.org/bookworm/user-mode-linux
[3] https://salsa.debian.org/uml-team/user-mode-linux
--
Walter Lozano
Collabora Ltd.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Occasional hung with UM after enable VMAP_STACK
@ 2022-01-04 18:10 ` Walter Lozano
0 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-04 18:10 UTC (permalink / raw)
To: johannes, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
Hi all,
I noticed that after "um: enable VMAP_STACK" [1] I experienced some
occasional hung in my Gitlab CI jobs that use user-mode-linux to build
distro images. The configuration use for the building is the one
available from Debian, see [2] and [3]. Unfortunately I don't have much
information, when the issue arises the job seems to hang and it is
cancelled by Gitlab due to a job timeout.
So far I did some bisect to get the commit which introduces the issue
and a quick check with latest Linux stable branch master to confirm the
issue is still present. I know this is not a proper bug report but I
thought it will be better to at least raise a comment.
Thanks in advance,
Walter
[1] http://lists.infradead.org/pipermail/linux-um/2021-August/001657.html
[2] https://packages.debian.org/bookworm/user-mode-linux
[3] https://salsa.debian.org/uml-team/user-mode-linux
--
Walter Lozano
Collabora Ltd.
_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
2022-01-04 18:10 ` Walter Lozano
@ 2022-01-04 19:04 ` Johannes Berg
-1 siblings, 0 replies; 14+ messages in thread
From: Johannes Berg @ 2022-01-04 19:04 UTC (permalink / raw)
To: Walter Lozano, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
> Hi all,
>
> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
> distro images.
>
Did you actually *enable* VMAP_STACK in the config as well? The commit
just makes it *possible* to enable it, you still have to set it
yourself. So you should be able to easily check with/without that
setting.
johannes
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
@ 2022-01-04 19:04 ` Johannes Berg
0 siblings, 0 replies; 14+ messages in thread
From: Johannes Berg @ 2022-01-04 19:04 UTC (permalink / raw)
To: Walter Lozano, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
> Hi all,
>
> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
> distro images.
>
Did you actually *enable* VMAP_STACK in the config as well? The commit
just makes it *possible* to enable it, you still have to set it
yourself. So you should be able to easily check with/without that
setting.
johannes
_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
2022-01-04 19:04 ` Johannes Berg
@ 2022-01-04 19:26 ` Walter Lozano
-1 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-04 19:26 UTC (permalink / raw)
To: Johannes Berg, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
Hi Johannes,
On 1/4/22 16:04, Johannes Berg wrote:
> On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
>> Hi all,
>>
>> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
>> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
>> distro images.
>>
> Did you actually *enable* VMAP_STACK in the config as well? The commit
> just makes it *possible* to enable it, you still have to set it
> yourself. So you should be able to easily check with/without that
> setting.
Thank you for your quick response. The Debian configuration on package
user-mode-linux have these settings
CONFIG_HAVE_ARCH_VMAP_STACK=y
CONFIG_VMAP_STACK=y
as you can see in [1]. I did run some tests disabling those settings,
which passed without any hung.
Unfortunately the "occasional" behavior makes this issue a bit tricky to
debug.
Regards,
Walter
[1]
https://salsa.debian.org/uml-team/user-mode-linux/-/blob/master/config.amd64#L321
--
Walter Lozano
Collabora Ltd.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
@ 2022-01-04 19:26 ` Walter Lozano
0 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-04 19:26 UTC (permalink / raw)
To: Johannes Berg, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
Hi Johannes,
On 1/4/22 16:04, Johannes Berg wrote:
> On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
>> Hi all,
>>
>> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
>> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
>> distro images.
>>
> Did you actually *enable* VMAP_STACK in the config as well? The commit
> just makes it *possible* to enable it, you still have to set it
> yourself. So you should be able to easily check with/without that
> setting.
Thank you for your quick response. The Debian configuration on package
user-mode-linux have these settings
CONFIG_HAVE_ARCH_VMAP_STACK=y
CONFIG_VMAP_STACK=y
as you can see in [1]. I did run some tests disabling those settings,
which passed without any hung.
Unfortunately the "occasional" behavior makes this issue a bit tricky to
debug.
Regards,
Walter
[1]
https://salsa.debian.org/uml-team/user-mode-linux/-/blob/master/config.amd64#L321
--
Walter Lozano
Collabora Ltd.
_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
2022-01-04 19:26 ` Walter Lozano
@ 2022-01-04 19:49 ` Johannes Berg
-1 siblings, 0 replies; 14+ messages in thread
From: Johannes Berg @ 2022-01-04 19:49 UTC (permalink / raw)
To: Walter Lozano, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
On Tue, 2022-01-04 at 16:26 -0300, Walter Lozano wrote:
>
> Thank you for your quick response. The Debian configuration on package
> user-mode-linux have these settings
>
> CONFIG_HAVE_ARCH_VMAP_STACK=y
> CONFIG_VMAP_STACK=y
OK, so it actually _is_ enabled.
> as you can see in [1]. I did run some tests disabling those settings,
> which passed without any hung.
>
> Unfortunately the "occasional" behavior makes this issue a bit tricky to
> debug.
>
Right.
Hm. I've been running our tests with it for about three months and
haven't observed any hangs, but I guess that doesn't mean much.
To be honest, I have no particular reason to even want it, other than
that it catches accidental DMA from stack more easily ... so I guess if
we can't find anything, we might as well revert it.
Feels like it _should_ work though, since it's just a different location
for the stack.
johannes
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
@ 2022-01-04 19:49 ` Johannes Berg
0 siblings, 0 replies; 14+ messages in thread
From: Johannes Berg @ 2022-01-04 19:49 UTC (permalink / raw)
To: Walter Lozano, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
On Tue, 2022-01-04 at 16:26 -0300, Walter Lozano wrote:
>
> Thank you for your quick response. The Debian configuration on package
> user-mode-linux have these settings
>
> CONFIG_HAVE_ARCH_VMAP_STACK=y
> CONFIG_VMAP_STACK=y
OK, so it actually _is_ enabled.
> as you can see in [1]. I did run some tests disabling those settings,
> which passed without any hung.
>
> Unfortunately the "occasional" behavior makes this issue a bit tricky to
> debug.
>
Right.
Hm. I've been running our tests with it for about three months and
haven't observed any hangs, but I guess that doesn't mean much.
To be honest, I have no particular reason to even want it, other than
that it catches accidental DMA from stack more easily ... so I guess if
we can't find anything, we might as well revert it.
Feels like it _should_ work though, since it's just a different location
for the stack.
johannes
_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
2022-01-04 19:26 ` Walter Lozano
@ 2022-01-04 21:39 ` Anton Ivanov
-1 siblings, 0 replies; 14+ messages in thread
From: Anton Ivanov @ 2022-01-04 21:39 UTC (permalink / raw)
To: Walter Lozano, Johannes Berg, linux-um, linux-kernel
Cc: Sjoerd Simons, ritesh sarraf
On 04/01/2022 19:26, Walter Lozano wrote:
> Hi Johannes,
>
> On 1/4/22 16:04, Johannes Berg wrote:
>> On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
>>> Hi all,
>>>
>>> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
>>> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
>>> distro images.
>>>
>> Did you actually *enable* VMAP_STACK in the config as well? The commit
>> just makes it *possible* to enable it, you still have to set it
>> yourself. So you should be able to easily check with/without that
>> setting.
>
> Thank you for your quick response. The Debian configuration on package
> user-mode-linux have these settings
>
> CONFIG_HAVE_ARCH_VMAP_STACK=y
> CONFIG_VMAP_STACK=y
>
>
> as you can see in [1]. I did run some tests disabling those settings,
> which passed without any hung.
>
> Unfortunately the "occasional" behavior makes this issue a bit tricky to
> debug.
>
> Regards,
>
> Walter
>
> [1]
> https://salsa.debian.org/uml-team/user-mode-linux/-/blob/master/config.amd64#L321
>
>
Just to narrow things down - 64 bit or 32 bit?
--
Anton R. Ivanov
https://www.kot-begemot.co.uk/
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
@ 2022-01-04 21:39 ` Anton Ivanov
0 siblings, 0 replies; 14+ messages in thread
From: Anton Ivanov @ 2022-01-04 21:39 UTC (permalink / raw)
To: Walter Lozano, Johannes Berg, linux-um, linux-kernel
Cc: Sjoerd Simons, ritesh sarraf
On 04/01/2022 19:26, Walter Lozano wrote:
> Hi Johannes,
>
> On 1/4/22 16:04, Johannes Berg wrote:
>> On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
>>> Hi all,
>>>
>>> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
>>> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
>>> distro images.
>>>
>> Did you actually *enable* VMAP_STACK in the config as well? The commit
>> just makes it *possible* to enable it, you still have to set it
>> yourself. So you should be able to easily check with/without that
>> setting.
>
> Thank you for your quick response. The Debian configuration on package
> user-mode-linux have these settings
>
> CONFIG_HAVE_ARCH_VMAP_STACK=y
> CONFIG_VMAP_STACK=y
>
>
> as you can see in [1]. I did run some tests disabling those settings,
> which passed without any hung.
>
> Unfortunately the "occasional" behavior makes this issue a bit tricky to
> debug.
>
> Regards,
>
> Walter
>
> [1]
> https://salsa.debian.org/uml-team/user-mode-linux/-/blob/master/config.amd64#L321
>
>
Just to narrow things down - 64 bit or 32 bit?
--
Anton R. Ivanov
https://www.kot-begemot.co.uk/
_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
2022-01-04 19:49 ` Johannes Berg
@ 2022-01-05 12:58 ` Walter Lozano
-1 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-05 12:58 UTC (permalink / raw)
To: Johannes Berg, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
Hi Johannes,
On 1/4/22 16:49, Johannes Berg wrote:
> On Tue, 2022-01-04 at 16:26 -0300, Walter Lozano wrote:
>> Thank you for your quick response. The Debian configuration on package
>> user-mode-linux have these settings
>>
>> CONFIG_HAVE_ARCH_VMAP_STACK=y
>> CONFIG_VMAP_STACK=y
> OK, so it actually _is_ enabled.
>
>> as you can see in [1]. I did run some tests disabling those settings,
>> which passed without any hung.
>>
>> Unfortunately the "occasional" behavior makes this issue a bit tricky to
>> debug.
>>
> Right.
>
> Hm. I've been running our tests with it for about three months and
> haven't observed any hangs, but I guess that doesn't mean much.
The issue is very rare at least in my setup, I had to prepare a test
environment to stress the system in order to be able to reproduce it
more or less consistently. I will continue investigating and share any
useful information.
> To be honest, I have no particular reason to even want it, other than
> that it catches accidental DMA from stack more easily ... so I guess if
> we can't find anything, we might as well revert it.
I hope we will be able to find the root cause of the issue.
> Feels like it _should_ work though, since it's just a different location
> for the stack.
I see. Thank you again for your quick reply, I will keep you updated.
Regards,
Walter
--
Walter Lozano
Collabora Ltd.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
@ 2022-01-05 12:58 ` Walter Lozano
0 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-05 12:58 UTC (permalink / raw)
To: Johannes Berg, linux-um, linux-kernel; +Cc: Sjoerd Simons, ritesh sarraf
Hi Johannes,
On 1/4/22 16:49, Johannes Berg wrote:
> On Tue, 2022-01-04 at 16:26 -0300, Walter Lozano wrote:
>> Thank you for your quick response. The Debian configuration on package
>> user-mode-linux have these settings
>>
>> CONFIG_HAVE_ARCH_VMAP_STACK=y
>> CONFIG_VMAP_STACK=y
> OK, so it actually _is_ enabled.
>
>> as you can see in [1]. I did run some tests disabling those settings,
>> which passed without any hung.
>>
>> Unfortunately the "occasional" behavior makes this issue a bit tricky to
>> debug.
>>
> Right.
>
> Hm. I've been running our tests with it for about three months and
> haven't observed any hangs, but I guess that doesn't mean much.
The issue is very rare at least in my setup, I had to prepare a test
environment to stress the system in order to be able to reproduce it
more or less consistently. I will continue investigating and share any
useful information.
> To be honest, I have no particular reason to even want it, other than
> that it catches accidental DMA from stack more easily ... so I guess if
> we can't find anything, we might as well revert it.
I hope we will be able to find the root cause of the issue.
> Feels like it _should_ work though, since it's just a different location
> for the stack.
I see. Thank you again for your quick reply, I will keep you updated.
Regards,
Walter
--
Walter Lozano
Collabora Ltd.
_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
2022-01-04 21:39 ` Anton Ivanov
@ 2022-01-05 12:59 ` Walter Lozano
-1 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-05 12:59 UTC (permalink / raw)
To: Anton Ivanov, Johannes Berg, linux-um, linux-kernel
Cc: Sjoerd Simons, ritesh sarraf
Hi Anton,
On 1/4/22 18:39, Anton Ivanov wrote:
> On 04/01/2022 19:26, Walter Lozano wrote:
>> Hi Johannes,
>>
>> On 1/4/22 16:04, Johannes Berg wrote:
>>> On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
>>>> Hi all,
>>>>
>>>> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
>>>> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
>>>> distro images.
>>>>
>>> Did you actually *enable* VMAP_STACK in the config as well? The commit
>>> just makes it *possible* to enable it, you still have to set it
>>> yourself. So you should be able to easily check with/without that
>>> setting.
>>
>> Thank you for your quick response. The Debian configuration on
>> package user-mode-linux have these settings
>>
>> CONFIG_HAVE_ARCH_VMAP_STACK=y
>> CONFIG_VMAP_STACK=y
>>
>>
>> as you can see in [1]. I did run some tests disabling those settings,
>> which passed without any hung.
>>
>> Unfortunately the "occasional" behavior makes this issue a bit tricky
>> to debug.
>>
>> Regards,
>>
>> Walter
>>
>> [1]
>> https://salsa.debian.org/uml-team/user-mode-linux/-/blob/master/config.amd64#L321
>>
>>
>
> Just to narrow things down - 64 bit or 32 bit?
>
Thank you for commenting on this thread. All my tests were done with 64
bits.
Regards,
--
Walter Lozano
Collabora Ltd.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Occasional hung with UM after enable VMAP_STACK
@ 2022-01-05 12:59 ` Walter Lozano
0 siblings, 0 replies; 14+ messages in thread
From: Walter Lozano @ 2022-01-05 12:59 UTC (permalink / raw)
To: Anton Ivanov, Johannes Berg, linux-um, linux-kernel
Cc: Sjoerd Simons, ritesh sarraf
Hi Anton,
On 1/4/22 18:39, Anton Ivanov wrote:
> On 04/01/2022 19:26, Walter Lozano wrote:
>> Hi Johannes,
>>
>> On 1/4/22 16:04, Johannes Berg wrote:
>>> On Tue, 2022-01-04 at 15:10 -0300, Walter Lozano wrote:
>>>> Hi all,
>>>>
>>>> I noticed that after "um: enable VMAP_STACK" [1] I experienced some
>>>> occasional hung in my Gitlab CI jobs that use user-mode-linux to build
>>>> distro images.
>>>>
>>> Did you actually *enable* VMAP_STACK in the config as well? The commit
>>> just makes it *possible* to enable it, you still have to set it
>>> yourself. So you should be able to easily check with/without that
>>> setting.
>>
>> Thank you for your quick response. The Debian configuration on
>> package user-mode-linux have these settings
>>
>> CONFIG_HAVE_ARCH_VMAP_STACK=y
>> CONFIG_VMAP_STACK=y
>>
>>
>> as you can see in [1]. I did run some tests disabling those settings,
>> which passed without any hung.
>>
>> Unfortunately the "occasional" behavior makes this issue a bit tricky
>> to debug.
>>
>> Regards,
>>
>> Walter
>>
>> [1]
>> https://salsa.debian.org/uml-team/user-mode-linux/-/blob/master/config.amd64#L321
>>
>>
>
> Just to narrow things down - 64 bit or 32 bit?
>
Thank you for commenting on this thread. All my tests were done with 64
bits.
Regards,
--
Walter Lozano
Collabora Ltd.
_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2022-01-05 12:59 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-04 18:10 Occasional hung with UM after enable VMAP_STACK Walter Lozano
2022-01-04 18:10 ` Walter Lozano
2022-01-04 19:04 ` Johannes Berg
2022-01-04 19:04 ` Johannes Berg
2022-01-04 19:26 ` Walter Lozano
2022-01-04 19:26 ` Walter Lozano
2022-01-04 19:49 ` Johannes Berg
2022-01-04 19:49 ` Johannes Berg
2022-01-05 12:58 ` Walter Lozano
2022-01-05 12:58 ` Walter Lozano
2022-01-04 21:39 ` Anton Ivanov
2022-01-04 21:39 ` Anton Ivanov
2022-01-05 12:59 ` Walter Lozano
2022-01-05 12:59 ` Walter Lozano
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.