[PATCH] kunit: tool: Force the use of the 'tty' console for UML

[PATCH] kunit: tool: Force the use of the 'tty' console for UML

[David Gow]

kunit_tool relies on the UML console outputting printk() output to the
tty in order to get results. Since the default console driver could
change, pass 'console=tty' to the kernel.

This is triggered by a change[1] to use ttynull as a fallback console
driver which -- by chance or by design -- seems to have changed the
default console output on UML, breaking kunit_tool. While this may be
fixed, we should be less fragile to such changes in the default.

[1]:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e

Signed-off-by: David Gow <davidgow@google.com>
Fixes: 757055ae8ded ("init/console: Use ttynull as a fallback when there is no console")
---
 tools/testing/kunit/kunit_kernel.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 57c1724b7e5d..698358c9c0d6 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -198,7 +198,7 @@ class LinuxSourceTree(object):
 		return self.validate_config(build_dir)
 
 	def run_kernel(self, args=[], build_dir='', timeout=None):
-		args.extend(['mem=1G'])
+		args.extend(['mem=1G', 'console=tty'])
 		self._ops.linux_bin(args, timeout, build_dir)
 		outfile = get_outfile_path(build_dir)
 		subprocess.call(['stty', 'sane'])
-- 
2.29.2.729.g45daf8777d-goog

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Andy Shevchenko]

On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
> kunit_tool relies on the UML console outputting printk() output to the
> tty in order to get results. Since the default console driver could
> change, pass 'console=tty' to the kernel.
> 
> This is triggered by a change[1] to use ttynull as a fallback console
> driver which -- by chance or by design -- seems to have changed the
> default console output on UML, breaking kunit_tool. While this may be
> fixed, we should be less fragile to such changes in the default.
> 
> [1]:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e

Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

Thanks!

> Signed-off-by: David Gow <davidgow@google.com>
> Fixes: 757055ae8ded ("init/console: Use ttynull as a fallback when there is no console")


> ---
>  tools/testing/kunit/kunit_kernel.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
> index 57c1724b7e5d..698358c9c0d6 100644
> --- a/tools/testing/kunit/kunit_kernel.py
> +++ b/tools/testing/kunit/kunit_kernel.py
> @@ -198,7 +198,7 @@ class LinuxSourceTree(object):
>  		return self.validate_config(build_dir)
>  
>  	def run_kernel(self, args=[], build_dir='', timeout=None):
> -		args.extend(['mem=1G'])
> +		args.extend(['mem=1G', 'console=tty'])
>  		self._ops.linux_bin(args, timeout, build_dir)
>  		outfile = get_outfile_path(build_dir)
>  		subprocess.call(['stty', 'sane'])
> -- 
> 2.29.2.729.g45daf8777d-goog
> 

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Brendan Higgins]

On Mon, Dec 21, 2020 at 11:39 PM David Gow <davidgow@google.com> wrote:
>
> kunit_tool relies on the UML console outputting printk() output to the
> tty in order to get results. Since the default console driver could
> change, pass 'console=tty' to the kernel.
>
> This is triggered by a change[1] to use ttynull as a fallback console
> driver which -- by chance or by design -- seems to have changed the
> default console output on UML, breaking kunit_tool. While this may be
> fixed, we should be less fragile to such changes in the default.
>
> [1]:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
>
> Signed-off-by: David Gow <davidgow@google.com>
> Fixes: 757055ae8ded ("init/console: Use ttynull as a fallback when there is no console")

Acked-by: Brendan Higgins <brendanhiggins@google.com>

Thanks for taking care of this!

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Shuah Khan]

On 12/22/20 4:11 AM, Andy Shevchenko wrote:
> On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
>> kunit_tool relies on the UML console outputting printk() output to the
>> tty in order to get results. Since the default console driver could
>> change, pass 'console=tty' to the kernel.
>>
>> This is triggered by a change[1] to use ttynull as a fallback console
>> driver which -- by chance or by design -- seems to have changed the
>> default console output on UML, breaking kunit_tool. While this may be
>> fixed, we should be less fragile to such changes in the default.
>>
>> [1]:
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
> 
> Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> 

Thank you all. Now in linux-kselftest kunit-fixes branch.

Will send this up for rc3.

Sorry for the delay - have been away from the keyboard for a
bit.

thanks,
-- Shuah

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Petr Mladek]

On Mon 2021-01-04 09:23:57, Shuah Khan wrote:
> On 12/22/20 4:11 AM, Andy Shevchenko wrote:
> > On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
> > > kunit_tool relies on the UML console outputting printk() output to the
> > > tty in order to get results. Since the default console driver could
> > > change, pass 'console=tty' to the kernel.
> > > 
> > > This is triggered by a change[1] to use ttynull as a fallback console
> > > driver which -- by chance or by design -- seems to have changed the
> > > default console output on UML, breaking kunit_tool. While this may be
> > > fixed, we should be less fragile to such changes in the default.
> > > 
> > > [1]:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
> > 
> > Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > 
> 
> Thank you all. Now in linux-kselftest kunit-fixes branch.
> 
> Will send this up for rc3.
> 
> Sorry for the delay - have been away from the keyboard for a
> bit.

JFYI, I am not sure that this is the right solution. I am
looking into it, see
https://lore.kernel.org/linux-kselftest/X%2FSRA1P8t+ONZFKb@alley/
for more details.

Best Regards,
Petr

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Shuah Khan]

On 1/5/21 9:21 AM, Petr Mladek wrote:
> On Mon 2021-01-04 09:23:57, Shuah Khan wrote:
>> On 12/22/20 4:11 AM, Andy Shevchenko wrote:
>>> On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
>>>> kunit_tool relies on the UML console outputting printk() output to the
>>>> tty in order to get results. Since the default console driver could
>>>> change, pass 'console=tty' to the kernel.
>>>>
>>>> This is triggered by a change[1] to use ttynull as a fallback console
>>>> driver which -- by chance or by design -- seems to have changed the
>>>> default console output on UML, breaking kunit_tool. While this may be
>>>> fixed, we should be less fragile to such changes in the default.
>>>>
>>>> [1]:
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
>>>
>>> Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>>> Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>>>
>>
>> Thank you all. Now in linux-kselftest kunit-fixes branch.
>>
>> Will send this up for rc3.
>>
>> Sorry for the delay - have been away from the keyboard for a
>> bit.
> 
> JFYI, I am not sure that this is the right solution. I am
> looking into it, see
> https://lore.kernel.org/linux-kselftest/X%2FSRA1P8t+ONZFKb@alley/
> for more details.
> 

Thanks Petr. I will hold off on sending the patch up to Linus and
let you find a the right solution.

thanks,
-- Shuah

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Andy Shevchenko]

On Tue, Jan 05, 2021 at 09:34:33AM -0700, Shuah Khan wrote:
> On 1/5/21 9:21 AM, Petr Mladek wrote:
> > On Mon 2021-01-04 09:23:57, Shuah Khan wrote:
> > > On 12/22/20 4:11 AM, Andy Shevchenko wrote:
> > > > On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
> > > > > kunit_tool relies on the UML console outputting printk() output to the
> > > > > tty in order to get results. Since the default console driver could
> > > > > change, pass 'console=tty' to the kernel.
> > > > > 
> > > > > This is triggered by a change[1] to use ttynull as a fallback console
> > > > > driver which -- by chance or by design -- seems to have changed the
> > > > > default console output on UML, breaking kunit_tool. While this may be
> > > > > fixed, we should be less fragile to such changes in the default.
> > > > > 
> > > > > [1]:
> > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
> > > > 
> > > > Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > > > Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > > > 
> > > 
> > > Thank you all. Now in linux-kselftest kunit-fixes branch.
> > > 
> > > Will send this up for rc3.
> > > 
> > > Sorry for the delay - have been away from the keyboard for a
> > > bit.
> > 
> > JFYI, I am not sure that this is the right solution. I am
> > looking into it, see
> > https://lore.kernel.org/linux-kselftest/X%2FSRA1P8t+ONZFKb@alley/
> > for more details.
> > 
> 
> Thanks Petr. I will hold off on sending the patch up to Linus and
> let you find a the right solution.

Please. leave it in Linux Next at least. Otherwise kunit will be broken for a
long time which is not good.


-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Shuah Khan]

On 1/5/21 11:57 AM, Andy Shevchenko wrote:
> On Tue, Jan 05, 2021 at 09:34:33AM -0700, Shuah Khan wrote:
>> On 1/5/21 9:21 AM, Petr Mladek wrote:
>>> On Mon 2021-01-04 09:23:57, Shuah Khan wrote:
>>>> On 12/22/20 4:11 AM, Andy Shevchenko wrote:
>>>>> On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
>>>>>> kunit_tool relies on the UML console outputting printk() output to the
>>>>>> tty in order to get results. Since the default console driver could
>>>>>> change, pass 'console=tty' to the kernel.
>>>>>>
>>>>>> This is triggered by a change[1] to use ttynull as a fallback console
>>>>>> driver which -- by chance or by design -- seems to have changed the
>>>>>> default console output on UML, breaking kunit_tool. While this may be
>>>>>> fixed, we should be less fragile to such changes in the default.
>>>>>>
>>>>>> [1]:
>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
>>>>>
>>>>> Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>>>>> Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>>>>>
>>>>
>>>> Thank you all. Now in linux-kselftest kunit-fixes branch.
>>>>
>>>> Will send this up for rc3.
>>>>
>>>> Sorry for the delay - have been away from the keyboard for a
>>>> bit.
>>>
>>> JFYI, I am not sure that this is the right solution. I am
>>> looking into it, see
>>> https://lore.kernel.org/linux-kselftest/X%2FSRA1P8t+ONZFKb@alley/
>>> for more details.
>>>
>>
>> Thanks Petr. I will hold off on sending the patch up to Linus and
>> let you find a the right solution.
> 
> Please. leave it in Linux Next at least. Otherwise kunit will be broken for a
> long time which is not good.
> 
> 

Yes. That is the plan. It will be in there until real fix comes in.

thanks,
-- Shuah

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[David Gow]

On Wed, Jan 6, 2021 at 3:52 AM Shuah Khan <skhan@linuxfoundation.org> wrote:
>
> On 1/5/21 11:57 AM, Andy Shevchenko wrote:
> > On Tue, Jan 05, 2021 at 09:34:33AM -0700, Shuah Khan wrote:
> >> On 1/5/21 9:21 AM, Petr Mladek wrote:
> >>> On Mon 2021-01-04 09:23:57, Shuah Khan wrote:
> >>>> On 12/22/20 4:11 AM, Andy Shevchenko wrote:
> >>>>> On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
> >>>>>> kunit_tool relies on the UML console outputting printk() output to the
> >>>>>> tty in order to get results. Since the default console driver could
> >>>>>> change, pass 'console=tty' to the kernel.
> >>>>>>
> >>>>>> This is triggered by a change[1] to use ttynull as a fallback console
> >>>>>> driver which -- by chance or by design -- seems to have changed the
> >>>>>> default console output on UML, breaking kunit_tool. While this may be
> >>>>>> fixed, we should be less fragile to such changes in the default.
> >>>>>>
> >>>>>> [1]:
> >>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
> >>>>>
> >>>>> Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> >>>>> Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> >>>>>
> >>>>
> >>>> Thank you all. Now in linux-kselftest kunit-fixes branch.
> >>>>
> >>>> Will send this up for rc3.
> >>>>
> >>>> Sorry for the delay - have been away from the keyboard for a
> >>>> bit.
> >>>
> >>> JFYI, I am not sure that this is the right solution. I am
> >>> looking into it, see
> >>> https://lore.kernel.org/linux-kselftest/X%2FSRA1P8t+ONZFKb@alley/
> >>> for more details.
> >>>
> >>
> >> Thanks Petr. I will hold off on sending the patch up to Linus and
> >> let you find a the right solution.
> >
> > Please. leave it in Linux Next at least. Otherwise kunit will be broken for a
> > long time which is not good.
> >
> >
>
> Yes. That is the plan. It will be in there until real fix comes in.
>

Thanks, Shuah.

Personally, I think that this patch makes some sense to keep even if
the underlying issue with ttynull is resolved. Given that kunit.py
requires the console output, explicitly stating we want console=tty
set is probably worth doing rather than relying on it being the
default. That being said, I definitely agree that this patch doesn't
fix the underlying issue with UML/ttynull: it just makes the kunit.py
script less sensitive to such changes (which, while unlikely, could
potentially occur legitimately down the track).

Cheers,
-- David

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Petr Mladek]

On Wed 2021-01-06 12:29:12, David Gow wrote:
> On Wed, Jan 6, 2021 at 3:52 AM Shuah Khan <skhan@linuxfoundation.org> wrote:
> >
> > On 1/5/21 11:57 AM, Andy Shevchenko wrote:
> > > On Tue, Jan 05, 2021 at 09:34:33AM -0700, Shuah Khan wrote:
> > >> On 1/5/21 9:21 AM, Petr Mladek wrote:
> > >>> On Mon 2021-01-04 09:23:57, Shuah Khan wrote:
> > >>>> On 12/22/20 4:11 AM, Andy Shevchenko wrote:
> > >>>>> On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
> > >>>>>> kunit_tool relies on the UML console outputting printk() output to the
> > >>>>>> tty in order to get results. Since the default console driver could
> > >>>>>> change, pass 'console=tty' to the kernel.
> > >>>>>>
> > >>>>>> This is triggered by a change[1] to use ttynull as a fallback console
> > >>>>>> driver which -- by chance or by design -- seems to have changed the
> > >>>>>> default console output on UML, breaking kunit_tool. While this may be
> > >>>>>> fixed, we should be less fragile to such changes in the default.
> > >>>>>>
> > >>>>>> [1]:
> > >>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
> > >>>>>
> > >>>>> Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > >>>>> Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > >>>>>
> > >>>>
> > >>>> Thank you all. Now in linux-kselftest kunit-fixes branch.
> > >>>>
> > >>>> Will send this up for rc3.
> > >>>>
> > >>>> Sorry for the delay - have been away from the keyboard for a
> > >>>> bit.
> > >>>
> > >>> JFYI, I am not sure that this is the right solution. I am
> > >>> looking into it, see
> > >>> https://lore.kernel.org/linux-kselftest/X%2FSRA1P8t+ONZFKb@alley/
> > >>> for more details.
> > >>>
> > >>
> > >> Thanks Petr. I will hold off on sending the patch up to Linus and
> > >> let you find a the right solution.
> > >
> > > Please. leave it in Linux Next at least. Otherwise kunit will be broken for a
> > > long time which is not good.
> > >
> > >
> >
> > Yes. That is the plan. It will be in there until real fix comes in.

The real fix would be too complicated for 5.11-rc3. Instead, I
proposed to revert the problematic commit, see
https://lore.kernel.org/lkml/20210107164400.17904-2-pmladek@suse.com/
I would like to push it for 5.11-rc3.

> Personally, I think that this patch makes some sense to keep even if
> the underlying issue with ttynull is resolved. Given that kunit.py
> requires the console output, explicitly stating we want console=tty
> set is probably worth doing rather than relying on it being the
> default.

I agree that the patch makes sense on its own. kunit depends on the
particular console. Note that "tty" is actually the UML-specific
stdio console implemented in arch/um/drivers/stdio_console.c.

Best Regards,
Petr

Re: [PATCH] kunit: tool: Force the use of the 'tty' console for UML

[Shuah Khan]

On 1/7/21 9:53 AM, Petr Mladek wrote:
> On Wed 2021-01-06 12:29:12, David Gow wrote:
>> On Wed, Jan 6, 2021 at 3:52 AM Shuah Khan <skhan@linuxfoundation.org> wrote:
>>>
>>> On 1/5/21 11:57 AM, Andy Shevchenko wrote:
>>>> On Tue, Jan 05, 2021 at 09:34:33AM -0700, Shuah Khan wrote:
>>>>> On 1/5/21 9:21 AM, Petr Mladek wrote:
>>>>>> On Mon 2021-01-04 09:23:57, Shuah Khan wrote:
>>>>>>> On 12/22/20 4:11 AM, Andy Shevchenko wrote:
>>>>>>>> On Mon, Dec 21, 2020 at 11:39:00PM -0800, David Gow wrote:
>>>>>>>>> kunit_tool relies on the UML console outputting printk() output to the
>>>>>>>>> tty in order to get results. Since the default console driver could
>>>>>>>>> change, pass 'console=tty' to the kernel.
>>>>>>>>>
>>>>>>>>> This is triggered by a change[1] to use ttynull as a fallback console
>>>>>>>>> driver which -- by chance or by design -- seems to have changed the
>>>>>>>>> default console output on UML, breaking kunit_tool. While this may be
>>>>>>>>> fixed, we should be less fragile to such changes in the default.
>>>>>>>>>
>>>>>>>>> [1]:
>>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e
>>>>>>>>
>>>>>>>> Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>>>>>>>> Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>>>>>>>>
>>>>>>>
>>>>>>> Thank you all. Now in linux-kselftest kunit-fixes branch.
>>>>>>>
>>>>>>> Will send this up for rc3.
>>>>>>>
>>>>>>> Sorry for the delay - have been away from the keyboard for a
>>>>>>> bit.
>>>>>>
>>>>>> JFYI, I am not sure that this is the right solution. I am
>>>>>> looking into it, see
>>>>>> https://lore.kernel.org/linux-kselftest/X%2FSRA1P8t+ONZFKb@alley/
>>>>>> for more details.
>>>>>>
>>>>>
>>>>> Thanks Petr. I will hold off on sending the patch up to Linus and
>>>>> let you find a the right solution.
>>>>
>>>> Please. leave it in Linux Next at least. Otherwise kunit will be broken for a
>>>> long time which is not good.
>>>>
>>>>
>>>
>>> Yes. That is the plan. It will be in there until real fix comes in.
> 
> The real fix would be too complicated for 5.11-rc3. Instead, I
> proposed to revert the problematic commit, see
> https://lore.kernel.org/lkml/20210107164400.17904-2-pmladek@suse.com/
> I would like to push it for 5.11-rc3.
> 
>> Personally, I think that this patch makes some sense to keep even if
>> the underlying issue with ttynull is resolved. Given that kunit.py
>> requires the console output, explicitly stating we want console=tty
>> set is probably worth doing rather than relying on it being the
>> default.
> 
> I agree that the patch makes sense on its own. kunit depends on the
> particular console. Note that "tty" is actually the UML-specific
> stdio console implemented in arch/um/drivers/stdio_console.c.
> 

The proposal sounds like revert the problem commit 
https://lore.kernel.org/lkml/20210107164400.17904-2-pmladek@suse.com/

and also send kunit fix up. Sounds reasonable to me. I will send
it for 5.11-rc3

thanks,
-- Shuah