[PATCH] init: Fix crash observed if there is no initial console

[PATCH] init: Fix crash observed if there is no initial console

[Guenter Roeck]

Systems with no initial console crash in f_dupfd(). This happens because
console_on_rootfs() was changed to call filp_open(). Its return value
(a file pointer) is checked against NULL, but returns an ERR_PTR after
errors. This ERR_PTR is then passed as file parameter to f_dupfd(),
which experiences a severe case of indigestion.

Cc: Dominik Brodowski <linux@dominikbrodowski.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Fixes: 8243186f0cc7 ("fs: remove ksys_dup()"),
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
---
 init/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/init/main.c b/init/main.c
index ec3a1463ac69..1ecfd43ed464 100644
--- a/init/main.c
+++ b/init/main.c
@@ -1163,7 +1163,7 @@ void console_on_rootfs(void)
 
 	/* Open /dev/console in kernelspace, this should never fail */
 	file = filp_open("/dev/console", O_RDWR, 0);
-	if (!file)
+	if (IS_ERR(file))
 		goto err_out;
 
 	/* create stdin/stdout/stderr, this should never fail */
-- 
2.17.1

Re: [PATCH] init: Fix crash observed if there is no initial console

[Guenter Roeck]

On Wed, Dec 18, 2019 at 03:01:49PM -0800, Guenter Roeck wrote:
> Systems with no initial console crash in f_dupfd(). This happens because
> console_on_rootfs() was changed to call filp_open(). Its return value
> (a file pointer) is checked against NULL, but returns an ERR_PTR after
> errors. This ERR_PTR is then passed as file parameter to f_dupfd(),
> which experiences a severe case of indigestion.
> 

I should have pulled upstream before sending this. Sorry for the noise.

Guenter

> Cc: Dominik Brodowski <linux@dominikbrodowski.net>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Fixes: 8243186f0cc7 ("fs: remove ksys_dup()"),
> Signed-off-by: Guenter Roeck <linux@roeck-us.net>
> ---
>  init/main.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/init/main.c b/init/main.c
> index ec3a1463ac69..1ecfd43ed464 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -1163,7 +1163,7 @@ void console_on_rootfs(void)
>  
>  	/* Open /dev/console in kernelspace, this should never fail */
>  	file = filp_open("/dev/console", O_RDWR, 0);
> -	if (!file)
> +	if (IS_ERR(file))
>  		goto err_out;
>  
>  	/* create stdin/stdout/stderr, this should never fail */
> -- 
> 2.17.1
>