All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] Fix Python crash on getprop deallocation
@ 2021-12-24 10:28 Luca Weiss
       [not found] ` <20211224102811.70695-1-luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
  0 siblings, 1 reply; 6+ messages in thread
From: Luca Weiss @ 2021-12-24 10:28 UTC (permalink / raw)
  To: devicetree-compiler-u79uwXL29TY76Z2rM5mHXA; +Cc: Luca Weiss

Fatal Python error: none_dealloc: deallocating None
Python runtime state: finalizing (tstate=0x000055c9bac70920)

Current thread 0x00007fbe34e47740 (most recent call first):
  <no Python frame>
Aborted (core dumped)

This is caused by a missing Py_INCREF on the returned Py_None, as
demonstrated e.g. in https://github.com/mythosil/swig-python-incref or
described at https://edcjones.tripod.com/refcount.html ("Remember to
INCREF Py_None!")

A PoC for triggering this crash is uploaded to
https://github.com/z3ntu/pylibfdt-crash .
With this patch applied to pylibfdt the crash does not happen.

Signed-off-by: Luca Weiss <luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
---
Unrelated but I've noticed that in this file the indentation is quite
mixed between spaces and tabs. This patch tries to keep to the style in
the lines around.

 pylibfdt/libfdt.i | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/pylibfdt/libfdt.i b/pylibfdt/libfdt.i
index 075ef70..9ccc57b 100644
--- a/pylibfdt/libfdt.i
+++ b/pylibfdt/libfdt.i
@@ -1040,14 +1040,16 @@ typedef uint32_t fdt32_t;
 
 /* typemap used for fdt_getprop() */
 %typemap(out) (const void *) {
-	if (!$1)
+	if (!$1) {
 		$result = Py_None;
-	else
+		Py_INCREF($result);
+	} else {
         %#if PY_VERSION_HEX >= 0x03000000
             $result = Py_BuildValue("y#", $1, (Py_ssize_t)*arg4);
         %#else
             $result = Py_BuildValue("s#", $1, (Py_ssize_t)*arg4);
         %#endif
+    }
 }
 
 /* typemap used for fdt_setprop() */
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] Fix Python crash on getprop deallocation
       [not found] ` <20211224102811.70695-1-luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
@ 2021-12-24 13:17   ` Simon Glass
       [not found]     ` <CAPnjgZ227FLO_UYqsq44dQwOYqa+vXuu5BGgmjoCThHnTGS5FA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  2021-12-25  6:29   ` David Gibson
  1 sibling, 1 reply; 6+ messages in thread
From: Simon Glass @ 2021-12-24 13:17 UTC (permalink / raw)
  To: Luca Weiss; +Cc: Devicetree Compiler

Hi Luca,

On Fri, 24 Dec 2021 at 03:38, Luca Weiss <luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org> wrote:
>
> Fatal Python error: none_dealloc: deallocating None
> Python runtime state: finalizing (tstate=0x000055c9bac70920)
>
> Current thread 0x00007fbe34e47740 (most recent call first):
>   <no Python frame>
> Aborted (core dumped)
>
> This is caused by a missing Py_INCREF on the returned Py_None, as
> demonstrated e.g. in https://github.com/mythosil/swig-python-incref or
> described at https://edcjones.tripod.com/refcount.html ("Remember to
> INCREF Py_None!")
>
> A PoC for triggering this crash is uploaded to
> https://github.com/z3ntu/pylibfdt-crash .
> With this patch applied to pylibfdt the crash does not happen.
>
> Signed-off-by: Luca Weiss <luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
> ---
> Unrelated but I've noticed that in this file the indentation is quite
> mixed between spaces and tabs. This patch tries to keep to the style in
> the lines around.
>
>  pylibfdt/libfdt.i | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)

Reviewed-by: Simon Glass <sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org>

My original idea was to use tabs for the C code (to match the libfdt
style) and spaces for the Python code (for PEP8). Looking at it now,
that idea has not continued and I'm not even sure it was a good idea.

Regards,
Simon

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] Fix Python crash on getprop deallocation
       [not found] ` <20211224102811.70695-1-luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
  2021-12-24 13:17   ` Simon Glass
@ 2021-12-25  6:29   ` David Gibson
  2021-12-25 10:25     ` Luca Weiss
  1 sibling, 1 reply; 6+ messages in thread
From: David Gibson @ 2021-12-25  6:29 UTC (permalink / raw)
  To: Luca Weiss; +Cc: devicetree-compiler-u79uwXL29TY76Z2rM5mHXA

[-- Attachment #1: Type: text/plain, Size: 1996 bytes --]

On Fri, Dec 24, 2021 at 11:28:12AM +0100, Luca Weiss wrote:
> Fatal Python error: none_dealloc: deallocating None
> Python runtime state: finalizing (tstate=0x000055c9bac70920)
> 
> Current thread 0x00007fbe34e47740 (most recent call first):
>   <no Python frame>
> Aborted (core dumped)
> 
> This is caused by a missing Py_INCREF on the returned Py_None, as
> demonstrated e.g. in https://github.com/mythosil/swig-python-incref or
> described at https://edcjones.tripod.com/refcount.html ("Remember to
> INCREF Py_None!")
> 
> A PoC for triggering this crash is uploaded to
> https://github.com/z3ntu/pylibfdt-crash .
> With this patch applied to pylibfdt the crash does not happen.

Any chance you could rework your testcase into the libfdt testsuite
(make check)?

> Signed-off-by: Luca Weiss <luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
> ---
> Unrelated but I've noticed that in this file the indentation is quite
> mixed between spaces and tabs. This patch tries to keep to the style in
> the lines around.
> 
>  pylibfdt/libfdt.i | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/pylibfdt/libfdt.i b/pylibfdt/libfdt.i
> index 075ef70..9ccc57b 100644
> --- a/pylibfdt/libfdt.i
> +++ b/pylibfdt/libfdt.i
> @@ -1040,14 +1040,16 @@ typedef uint32_t fdt32_t;
>  
>  /* typemap used for fdt_getprop() */
>  %typemap(out) (const void *) {
> -	if (!$1)
> +	if (!$1) {
>  		$result = Py_None;
> -	else
> +		Py_INCREF($result);
> +	} else {
>          %#if PY_VERSION_HEX >= 0x03000000
>              $result = Py_BuildValue("y#", $1, (Py_ssize_t)*arg4);
>          %#else
>              $result = Py_BuildValue("s#", $1, (Py_ssize_t)*arg4);
>          %#endif
> +    }
>  }
>  
>  /* typemap used for fdt_setprop() */

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] Fix Python crash on getprop deallocation
       [not found]     ` <CAPnjgZ227FLO_UYqsq44dQwOYqa+vXuu5BGgmjoCThHnTGS5FA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2021-12-25  6:30       ` David Gibson
  0 siblings, 0 replies; 6+ messages in thread
From: David Gibson @ 2021-12-25  6:30 UTC (permalink / raw)
  To: Simon Glass; +Cc: Luca Weiss, Devicetree Compiler

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

On Fri, Dec 24, 2021 at 06:17:25AM -0700, Simon Glass wrote:
> Hi Luca,
> 
> On Fri, 24 Dec 2021 at 03:38, Luca Weiss <luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org> wrote:
> >
> > Fatal Python error: none_dealloc: deallocating None
> > Python runtime state: finalizing (tstate=0x000055c9bac70920)
> >
> > Current thread 0x00007fbe34e47740 (most recent call first):
> >   <no Python frame>
> > Aborted (core dumped)
> >
> > This is caused by a missing Py_INCREF on the returned Py_None, as
> > demonstrated e.g. in https://github.com/mythosil/swig-python-incref or
> > described at https://edcjones.tripod.com/refcount.html ("Remember to
> > INCREF Py_None!")
> >
> > A PoC for triggering this crash is uploaded to
> > https://github.com/z3ntu/pylibfdt-crash .
> > With this patch applied to pylibfdt the crash does not happen.
> >
> > Signed-off-by: Luca Weiss <luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
> > ---
> > Unrelated but I've noticed that in this file the indentation is quite
> > mixed between spaces and tabs. This patch tries to keep to the style in
> > the lines around.
> >
> >  pylibfdt/libfdt.i | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> Reviewed-by: Simon Glass <sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org>
> 
> My original idea was to use tabs for the C code (to match the libfdt
> style) and spaces for the Python code (for PEP8). Looking at it now,
> that idea has not continued and I'm not even sure it was a good idea.

Sounds like a good rationale, but probably not practical when combined
into a single file.  I'd suggest making it all spaces.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] Fix Python crash on getprop deallocation
  2021-12-25  6:29   ` David Gibson
@ 2021-12-25 10:25     ` Luca Weiss
  2021-12-26  4:46       ` David Gibson
  0 siblings, 1 reply; 6+ messages in thread
From: Luca Weiss @ 2021-12-25 10:25 UTC (permalink / raw)
  To: David Gibson; +Cc: devicetree-compiler-u79uwXL29TY76Z2rM5mHXA

Hi David,

On Samstag, 25. Dezember 2021 07:29:58 CET David Gibson wrote:
> On Fri, Dec 24, 2021 at 11:28:12AM +0100, Luca Weiss wrote:
> > Fatal Python error: none_dealloc: deallocating None
> > Python runtime state: finalizing (tstate=0x000055c9bac70920)
> > 
> > Current thread 0x00007fbe34e47740 (most recent call first):
> >   <no Python frame>
> > 
> > Aborted (core dumped)
> > 
> > This is caused by a missing Py_INCREF on the returned Py_None, as
> > demonstrated e.g. in https://github.com/mythosil/swig-python-incref or
> > described at https://edcjones.tripod.com/refcount.html ("Remember to
> > INCREF Py_None!")
> > 
> > A PoC for triggering this crash is uploaded to
> > https://github.com/z3ntu/pylibfdt-crash .
> > With this patch applied to pylibfdt the crash does not happen.
> 
> Any chance you could rework your testcase into the libfdt testsuite
> (make check)?
> 

To be completely honest I don't exactly understand why this crash is 
happening. If you reduce the iteration count in my PoC from the "10" I used to 
just 1 or 2, then the crash doesn't happen. But I don't have any insights into 
how Python actually allocates and deallocates things internally, as this crash 
happens during dellocation when Python exits and after the supplied code is 
already run.

Regards
Luca

> > Signed-off-by: Luca Weiss <luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
> > ---
> > Unrelated but I've noticed that in this file the indentation is quite
> > mixed between spaces and tabs. This patch tries to keep to the style in
> > the lines around.
> > 
> >  pylibfdt/libfdt.i | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> > 
> > diff --git a/pylibfdt/libfdt.i b/pylibfdt/libfdt.i
> > index 075ef70..9ccc57b 100644
> > --- a/pylibfdt/libfdt.i
> > +++ b/pylibfdt/libfdt.i
> > @@ -1040,14 +1040,16 @@ typedef uint32_t fdt32_t;
> > 
> >  /* typemap used for fdt_getprop() */
> >  %typemap(out) (const void *) {
> > 
> > -	if (!$1)
> > +	if (!$1) {
> > 
> >  		$result = Py_None;
> > 
> > -	else
> > +		Py_INCREF($result);
> > +	} else {
> > 
> >          %#if PY_VERSION_HEX >= 0x03000000
> >          
> >              $result = Py_BuildValue("y#", $1, (Py_ssize_t)*arg4);
> >          
> >          %#else
> >          
> >              $result = Py_BuildValue("s#", $1, (Py_ssize_t)*arg4);
> >          
> >          %#endif
> > 
> > +    }
> > 
> >  }
> >  
> >  /* typemap used for fdt_setprop() */





^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] Fix Python crash on getprop deallocation
  2021-12-25 10:25     ` Luca Weiss
@ 2021-12-26  4:46       ` David Gibson
  0 siblings, 0 replies; 6+ messages in thread
From: David Gibson @ 2021-12-26  4:46 UTC (permalink / raw)
  To: Luca Weiss; +Cc: devicetree-compiler-u79uwXL29TY76Z2rM5mHXA

[-- Attachment #1: Type: text/plain, Size: 1683 bytes --]

On Sat, Dec 25, 2021 at 11:25:51AM +0100, Luca Weiss wrote:
> Hi David,
> 
> On Samstag, 25. Dezember 2021 07:29:58 CET David Gibson wrote:
> > On Fri, Dec 24, 2021 at 11:28:12AM +0100, Luca Weiss wrote:
> > > Fatal Python error: none_dealloc: deallocating None
> > > Python runtime state: finalizing (tstate=0x000055c9bac70920)
> > > 
> > > Current thread 0x00007fbe34e47740 (most recent call first):
> > >   <no Python frame>
> > > 
> > > Aborted (core dumped)
> > > 
> > > This is caused by a missing Py_INCREF on the returned Py_None, as
> > > demonstrated e.g. in https://github.com/mythosil/swig-python-incref or
> > > described at https://edcjones.tripod.com/refcount.html ("Remember to
> > > INCREF Py_None!")
> > > 
> > > A PoC for triggering this crash is uploaded to
> > > https://github.com/z3ntu/pylibfdt-crash .
> > > With this patch applied to pylibfdt the crash does not happen.
> > 
> > Any chance you could rework your testcase into the libfdt testsuite
> > (make check)?
> > 
> 
> To be completely honest I don't exactly understand why this crash is 
> happening. If you reduce the iteration count in my PoC from the "10" I used to 
> just 1 or 2, then the crash doesn't happen. But I don't have any insights into 
> how Python actually allocates and deallocates things internally, as this crash 
> happens during dellocation when Python exits and after the supplied code is 
> already run.

Ok, fair enough.  Applied to main branch.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-12-26  4:46 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-24 10:28 [PATCH] Fix Python crash on getprop deallocation Luca Weiss
     [not found] ` <20211224102811.70695-1-luca-IfPCFPJWly+lVyrhU4qvOw@public.gmane.org>
2021-12-24 13:17   ` Simon Glass
     [not found]     ` <CAPnjgZ227FLO_UYqsq44dQwOYqa+vXuu5BGgmjoCThHnTGS5FA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2021-12-25  6:30       ` David Gibson
2021-12-25  6:29   ` David Gibson
2021-12-25 10:25     ` Luca Weiss
2021-12-26  4:46       ` David Gibson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.