[PATCH 1/3] drivers/media/dvb-core/en50221: use kref to manage struct dvb_ca_private

[PATCH 1/3] drivers/media/dvb-core/en50221: use kref to manage struct dvb_ca_private

[Max Kellermann]

Don't free the object until the file handle has been closed.  Fixes
use-after-free bug which occurs when I disconnect my DVB-S received
while VDR is running.

Signed-off-by: Max Kellermann <max@duempel.org>
---
 drivers/media/dvb-core/dvb_ca_en50221.c |   24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/drivers/media/dvb-core/dvb_ca_en50221.c b/drivers/media/dvb-core/dvb_ca_en50221.c
index b1e3a26..b5b5b19 100644
--- a/drivers/media/dvb-core/dvb_ca_en50221.c
+++ b/drivers/media/dvb-core/dvb_ca_en50221.c
@@ -123,6 +123,7 @@ struct dvb_ca_slot {
 
 /* Private CA-interface information */
 struct dvb_ca_private {
+	struct kref refcount;
 
 	/* pointer back to the public data structure */
 	struct dvb_ca_en50221 *pub;
@@ -173,6 +174,22 @@ static void dvb_ca_private_free(struct dvb_ca_private *ca)
 	kfree(ca);
 }
 
+static void dvb_ca_private_release(struct kref *ref)
+{
+	struct dvb_ca_private *ca = container_of(ref, struct dvb_ca_private, refcount);
+	dvb_ca_private_free(ca);
+}
+
+static void dvb_ca_private_get(struct dvb_ca_private *ca)
+{
+	kref_get(&ca->refcount);
+}
+
+static void dvb_ca_private_put(struct dvb_ca_private *ca)
+{
+	kref_put(&ca->refcount, dvb_ca_private_release);
+}
+
 static void dvb_ca_en50221_thread_wakeup(struct dvb_ca_private *ca);
 static int dvb_ca_en50221_read_data(struct dvb_ca_private *ca, int slot, u8 * ebuf, int ecount);
 static int dvb_ca_en50221_write_data(struct dvb_ca_private *ca, int slot, u8 * ebuf, int ecount);
@@ -1570,6 +1587,8 @@ static int dvb_ca_en50221_io_open(struct inode *inode, struct file *file)
 	dvb_ca_en50221_thread_update_delay(ca);
 	dvb_ca_en50221_thread_wakeup(ca);
 
+	dvb_ca_private_get(ca);
+
 	return 0;
 }
 
@@ -1598,6 +1617,8 @@ static int dvb_ca_en50221_io_release(struct inode *inode, struct file *file)
 
 	module_put(ca->pub->owner);
 
+	dvb_ca_private_put(ca);
+
 	return err;
 }
 
@@ -1693,6 +1714,7 @@ int dvb_ca_en50221_init(struct dvb_adapter *dvb_adapter,
 		ret = -ENOMEM;
 		goto exit;
 	}
+	kref_init(&ca->refcount);
 	ca->pub = pubca;
 	ca->flags = flags;
 	ca->slot_count = slot_count;
@@ -1772,6 +1794,6 @@ void dvb_ca_en50221_release(struct dvb_ca_en50221 *pubca)
 	for (i = 0; i < ca->slot_count; i++) {
 		dvb_ca_en50221_slot_shutdown(ca, i);
 	}
-	dvb_ca_private_free(ca);
+	dvb_ca_private_put(ca);
 	pubca->private = NULL;
 }

[PATCH 2/3] drivers/media/media-entity: clear media_gobj.mdev in _destroy()

[Max Kellermann]

media_gobj_destroy() may be called twice on one instance - once by
media_device_unregister() and again by dvb_media_device_free().  The
function media_remove_intf_links() establishes and documents the
convention that mdev==NULL means that the object is not registered,
but nobody ever NULLs this variable.  So this patch really implements
this behavior, and adds another mdev==NULL check to
media_gobj_destroy() to protect against double removal.

Signed-off-by: Max Kellermann <max@duempel.org>
---
 drivers/media/media-entity.c |    6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index d8a2299..9526338 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -203,10 +203,16 @@ void media_gobj_destroy(struct media_gobj *gobj)
 {
 	dev_dbg_obj(__func__, gobj);
 
+	/* Do nothing if the object is not linked. */
+	if (gobj->mdev == NULL)
+		return;
+
 	gobj->mdev->topology_version++;
 
 	/* Remove the object from mdev list */
 	list_del(&gobj->list);
+
+	gobj->mdev = NULL;
 }
 
 int media_entity_pads_init(struct media_entity *entity, u16 num_pads,

Re: [PATCH 2/3] drivers/media/media-entity: clear media_gobj.mdev in _destroy()

[Shuah Khan]

On 06/15/2016 02:15 PM, Max Kellermann wrote:
> media_gobj_destroy() may be called twice on one instance - once by
> media_device_unregister() and again by dvb_media_device_free().  The
> function media_remove_intf_links() establishes and documents the
> convention that mdev==NULL means that the object is not registered,
> but nobody ever NULLs this variable.  So this patch really implements
> this behavior, and adds another mdev==NULL check to
> media_gobj_destroy() to protect against double removal.

Are you seeing null pointer dereference on gobj->mdev? In any case,
we have to look at if there is a missing mutex hold that creates a
race between media_device_unregister() and dvb_media_device_free()

I don't this patch will solve the race condition.

thanks,
-- Shuah


> 
> Signed-off-by: Max Kellermann <max@duempel.org>
> ---
>  drivers/media/media-entity.c |    6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index d8a2299..9526338 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -203,10 +203,16 @@ void media_gobj_destroy(struct media_gobj *gobj)
>  {
>  	dev_dbg_obj(__func__, gobj);
>  
> +	/* Do nothing if the object is not linked. */
> +	if (gobj->mdev == NULL)
> +		return;
> +
>  	gobj->mdev->topology_version++;
>  
>  	/* Remove the object from mdev list */
>  	list_del(&gobj->list);
> +
> +	gobj->mdev = NULL;
>  }
>  
>  int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 2/3] drivers/media/media-entity: clear media_gobj.mdev in _destroy()

[Max Kellermann]

On 2016/06/16 18:24, Shuah Khan <shuahkh@osg.samsung.com> wrote:
> On 06/15/2016 02:15 PM, Max Kellermann wrote:
> > media_gobj_destroy() may be called twice on one instance - once by
> > media_device_unregister() and again by dvb_media_device_free().  The
> > function media_remove_intf_links() establishes and documents the
> > convention that mdev==NULL means that the object is not registered,
> > but nobody ever NULLs this variable.  So this patch really implements
> > this behavior, and adds another mdev==NULL check to
> > media_gobj_destroy() to protect against double removal.
> 
> Are you seeing null pointer dereference on gobj->mdev? In any case,
> we have to look at if there is a missing mutex hold that creates a
> race between media_device_unregister() and dvb_media_device_free()
> 
> I don't this patch will solve the race condition.

I think we misunderstand.  This is not about a race condition.  And
the problem cannot be a NULL pointer dereference.

That's because nobody NULLs the pointer!

Pointer NULLing is what my patch adds, and AFTER my patch, there may
be NULL pointer dereferences (if there are more previously existing
bugs, which we should fix as well).  I added this NULL assignment
because there are NULL checks - and if nobody NULLs the pointer, that
check doesn't make any sense!

Max

Re: [PATCH 2/3] drivers/media/media-entity: clear media_gobj.mdev in _destroy()

[Shuah Khan]

On 06/16/2016 12:43 PM, Max Kellermann wrote:
> On 2016/06/16 18:24, Shuah Khan <shuahkh@osg.samsung.com> wrote:
>> On 06/15/2016 02:15 PM, Max Kellermann wrote:
>>> media_gobj_destroy() may be called twice on one instance - once by
>>> media_device_unregister() and again by dvb_media_device_free().  The
>>> function media_remove_intf_links() establishes and documents the
>>> convention that mdev==NULL means that the object is not registered,
>>> but nobody ever NULLs this variable.  So this patch really implements
>>> this behavior, and adds another mdev==NULL check to
>>> media_gobj_destroy() to protect against double removal.
>>
>> Are you seeing null pointer dereference on gobj->mdev? In any case,
>> we have to look at if there is a missing mutex hold that creates a
>> race between media_device_unregister() and dvb_media_device_free()
>>
>> I don't this patch will solve the race condition.
> 
> I think we misunderstand.  This is not about a race condition.  And
> the problem cannot be a NULL pointer dereference.
> 
> That's because nobody NULLs the pointer!

I see 7 calls to media_gobj_destroy(). In 6 cases, calling routines
fee the pointer that contains the graph_obj.

__media_device_unregister_entity() sets mdev ot null.

entity->graph_obj.mdev = NULL;

That is why I am confused when you say it never set to null.

thanks,
-- Shuah

Re: [PATCH 2/3] drivers/media/media-entity: clear media_gobj.mdev in _destroy()

[Sakari Ailus]

Hi Max,

On Wed, Jun 15, 2016 at 10:15:07PM +0200, Max Kellermann wrote:
> media_gobj_destroy() may be called twice on one instance - once by
> media_device_unregister() and again by dvb_media_device_free().  The

Is that something that should really happen, and why? The same object should
not be unregistered more than once --- in many call paths gobj
unregistration is followed by kfree() on the gobj.

> function media_remove_intf_links() establishes and documents the
> convention that mdev==NULL means that the object is not registered,
> but nobody ever NULLs this variable.  So this patch really implements
> this behavior, and adds another mdev==NULL check to
> media_gobj_destroy() to protect against double removal.
> 
> Signed-off-by: Max Kellermann <max@duempel.org>
> ---
>  drivers/media/media-entity.c |    6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index d8a2299..9526338 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -203,10 +203,16 @@ void media_gobj_destroy(struct media_gobj *gobj)
>  {
>  	dev_dbg_obj(__func__, gobj);
>  
> +	/* Do nothing if the object is not linked. */
> +	if (gobj->mdev == NULL)
> +		return;
> +
>  	gobj->mdev->topology_version++;
>  
>  	/* Remove the object from mdev list */
>  	list_del(&gobj->list);
> +
> +	gobj->mdev = NULL;
>  }
>  
>  int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
> 

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH 2/3] drivers/media/media-entity: clear media_gobj.mdev in _destroy()

[Max Kellermann]

On 2016/06/17 14:53, Sakari Ailus <sakari.ailus@iki.fi> wrote:
> On Wed, Jun 15, 2016 at 10:15:07PM +0200, Max Kellermann wrote:
> > media_gobj_destroy() may be called twice on one instance - once by
> > media_device_unregister() and again by dvb_media_device_free().  The
> 
> Is that something that should really happen, and why? The same object should
> not be unregistered more than once --- in many call paths gobj
> unregistration is followed by kfree() on the gobj.

True, it should not happen, and I think the code is currently
misdesigned (or I just don't grasp it correctly; I may be wrong).

The "gobj" is inserted into a linked list, the list's owner
(media_device) feels responsible to free items in that list.  Plus,
the dvb_device instances holds a pointer and also tries to free it.

Usually, dvbdev.c destruction gets called first, which removes the
"gobj" from the linked list, and media_device never sees it during its
own destruction.  But that ordering is all but guaranteed.  It just
happens to be that way under "normal" circumstances.

None of this makes any sense to me.  There appears to be lots of bogus
and unsafe code.  I'm still waiting for somebody with more clue to
enlighten me.

Max

[PATCH 3/3] drivers/media/media-device: fix double free bug in _unregister()

[Max Kellermann]

While removing all interfaces in media_device_unregister(), all
media_interface pointers are freed.  This is illegal and results in
double kfree() if any media_interface is still linked at this point;
maybe because a userspace process still has a file handle.  Once the
process closes the file handle, dvb_media_device_free() gets called,
which frees the dvb_device.intf_devnode again.

This patch removes the unnecessary kfree() call, and documents who's
responsible for really freeing it.

Signed-off-by: Max Kellermann <max@duempel.org>
---
 drivers/media/media-device.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index 33a9952..1db4707 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -799,9 +799,11 @@ void media_device_unregister(struct media_device *mdev)
 	/* Remove all interfaces from the media device */
 	list_for_each_entry_safe(intf, tmp_intf, &mdev->interfaces,
 				 graph_obj.list) {
+		/* unlink the interface, but don't free it here; the
+		   module which created it is responsible for freeing
+		   it */
 		__media_remove_intf_links(intf);
 		media_gobj_destroy(&intf->graph_obj);
-		kfree(intf);
 	}
 
 	mutex_unlock(&mdev->graph_mutex);

Re: [PATCH 3/3] drivers/media/media-device: fix double free bug in _unregister()

[Shuah Khan]

On 06/15/2016 02:15 PM, Max Kellermann wrote:
> While removing all interfaces in media_device_unregister(), all
> media_interface pointers are freed.  This is illegal and results in
> double kfree() if any media_interface is still linked at this point;
> maybe because a userspace process still has a file handle.  Once the
> process closes the file handle, dvb_media_device_free() gets called,
> which frees the dvb_device.intf_devnode again.
> 
> This patch removes the unnecessary kfree() call, and documents who's
> responsible for really freeing it.
> 
> Signed-off-by: Max Kellermann <max@duempel.org>
> ---
>  drivers/media/media-device.c |    4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index 33a9952..1db4707 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -799,9 +799,11 @@ void media_device_unregister(struct media_device *mdev)
>  	/* Remove all interfaces from the media device */
>  	list_for_each_entry_safe(intf, tmp_intf, &mdev->interfaces,
>  				 graph_obj.list) {
> +		/* unlink the interface, but don't free it here; the
> +		   module which created it is responsible for freeing
> +		   it */
>  		__media_remove_intf_links(intf);
>  		media_gobj_destroy(&intf->graph_obj);
> -		kfree(intf);

This change introduces memory leaks, since drivers are relying on
media_device_unregister() to free interfaces.

thanks,
-- Shuah

Re: [PATCH 3/3] drivers/media/media-device: fix double free bug in _unregister()

[Max Kellermann]

On 2016/06/15 22:32, Shuah Khan <shuahkh@osg.samsung.com> wrote:
> This change introduces memory leaks, since drivers are relying on
> media_device_unregister() to free interfaces.

This is what I thought, too, until I checked the code paths.  Who adds
entries to that list?  Only media_gobj_create() does, and only when
type==MEDIA_GRAPH_INTF_DEVNODE.  That is called via
media_interface_init(), via media_devnode_create().

In the whole kernel, there are two calls to media_devnode_create():
one in dvbdev.c and another one in v4l2-dev.c.  Both callers take care
for freeing their interface.  Both would crash if somebody else would
free it for them before they get a chance to do it.  Which is the very
thing my patch addresses.

Did I miss something?

Max

Re: [PATCH 3/3] drivers/media/media-device: fix double free bug in _unregister()

[Shuah Khan]

On 06/15/2016 02:37 PM, Max Kellermann wrote:
> On 2016/06/15 22:32, Shuah Khan <shuahkh@osg.samsung.com> wrote:
>> This change introduces memory leaks, since drivers are relying on
>> media_device_unregister() to free interfaces.
> 
> This is what I thought, too, until I checked the code paths.  Who adds
> entries to that list?  Only media_gobj_create() does, and only when
> type==MEDIA_GRAPH_INTF_DEVNODE.  That is called via
> media_interface_init(), via media_devnode_create().
> 
> In the whole kernel, there are two calls to media_devnode_create():
> one in dvbdev.c and another one in v4l2-dev.c.  Both callers take care
> for freeing their interface.  Both would crash if somebody else would
> free it for them before they get a chance to do it.  Which is the very
> thing my patch addresses.
> 
> Did I miss something?
> 

Yes media_devnode_create() creates the interfaces links and these links
are deleted by media_devnode_remove(). media_device_unregister() still
needs to delete the interfaces links. The reason for that is the API
dynalic use-case.

Drivers (other than dvb-core and v4l2-core) can create and delete media
devnode interfaces during run-time, hence media_devnode_remove() has to
call media_remove_intf_links(). However, driver isn't required to call
media_devnode_remove() and media-core can't enforce that. So it is safe
for media_device_unregister() to remove interface links if the list isn't
empty. If driver does delete them,  media_device_unregister() has nothing
to do since the list is going to be empty.

So removing kfree() from media_device_unregister() isn't the correct
fix.

I don't see the stack trace for the double free error you are seeing? Could
it be that there is a driver problem in the order in which it is calling
media_device_unregister()?

thanks,
-- Shuah

Re: [PATCH 3/3] drivers/media/media-device: fix double free bug in _unregister()

[Max Kellermann]

(Shuah, I did not receive your second reply; I only found it in an
email archive.)

> Yes media_devnode_create() creates the interfaces links and these
> links are deleted by media_devnode_remove().
> media_device_unregister() still needs to delete the interfaces
> links. The reason for that is the API dynalic use-case.
> 
> Drivers (other than dvb-core and v4l2-core) can create and delete
> media devnode interfaces during run-time

My point was that they do not.  There are no other
media_devnode_create() callers.

> So removing kfree() from media_device_unregister() isn't the correct
> fix.

Then what is?  I don't know anything other than the (mostly
undocumented) code I read, and my patch implements the design that I
interpreted from the code.  Apparently my interpretation of the design
is wrong after all.

> I don't see the stack trace for the double free error you are
> seeing?

Actually, it didn't crash at the double free; it hung forever because
it tried to lock a mutex which was already stale.  I don't have a
stack trace of that; would it help to produce one?

> Could it be that there is a driver problem in the order in which it
> is calling media_device_unregister()?

Maybe it's due to my patch 1/3 which adds a kref, and it only occurs
if one process still has a file handle.

In any case, the kernel must decide who's responsible for freeing the
object, and how the dvbdev.c library gets to know that its pointer has
been invalidated.

Please explain how it should be done, and I'll try to adapt my patches
to the "grand design".

Re: [PATCH 3/3] drivers/media/media-device: fix double free bug in _unregister()

[Shuah Khan]

On 06/16/2016 03:29 AM, Max Kellermann wrote:
> (Shuah, I did not receive your second reply; I only found it in an
> email archive.)
> 
>> Yes media_devnode_create() creates the interfaces links and these
>> links are deleted by media_devnode_remove().
>> media_device_unregister() still needs to delete the interfaces
>> links. The reason for that is the API dynalic use-case.
>>
>> Drivers (other than dvb-core and v4l2-core) can create and delete
>> media devnode interfaces during run-time
> 
> My point was that they do not.  There are no other
> media_devnode_create() callers.

Correct. There are none in the base now. However as I explained the
dynamic use-case. There is work in progress that uses this feature
in the API.

> 
>> So removing kfree() from media_device_unregister() isn't the correct
>> fix.
> 
> Then what is?  I don't know anything other than the (mostly
> undocumented) code I read, and my patch implements the design that I
> interpreted from the code.  Apparently my interpretation of the design
> is wrong after all.
> 
>> I don't see the stack trace for the double free error you are
>> seeing?
> 
> Actually, it didn't crash at the double free; it hung forever because
> it tried to lock a mutex which was already stale.  I don't have a
> stack trace of that; would it help to produce one?

I think you are running into another set of problems related to media
devnode, cdev, and race between media ioctl/syscall and media unregister
sequence. These patches are in

git://linuxtv.org/media_tree.git master branch

> 
>> Could it be that there is a driver problem in the order in which it
>> is calling media_device_unregister()?
> 
> Maybe it's due to my patch 1/3 which adds a kref, and it only occurs
> if one process still has a file handle.

So you are adding another refcounted object to the mix, in addition to
media_device, media_devnode, and cdev. Now you have three or more objects
with varying lifetimes. Not a good situation to be in.

> 
> In any case, the kernel must decide who's responsible for freeing the
> object, and how the dvbdev.c library gets to know that its pointer has
> been invalidated.

Yes it does that. intf links need to be free'd in both cases, one when
driver does a devnode_remove() and then when unregister is done. There
could be two drivers that are bound to the media hardware and both might
own their own sections of the media graph. Media Controller core has to
allow the possibility of one driver unbind/rmmod and be able to delete
the devnode it created.

I don't think the problem you are running into is due to this code path.
Without seeing the stack trace, it is hard to debug as you really don't
know what the problem is, leave alone being able to fix it.

thanks,
-- Shuah

Re: [PATCH 1/3] drivers/media/dvb-core/en50221: use kref to manage struct dvb_ca_private

[Shuah Khan]

On 06/15/2016 02:15 PM, Max Kellermann wrote:
> Don't free the object until the file handle has been closed.  Fixes
> use-after-free bug which occurs when I disconnect my DVB-S received
> while VDR is running.

Which file handle? /dev/dvb---

There seems to be a problem in the driver release routine:
dvb_ca_en50221_release() routine:

	kfree(ca->slot_info);
	dvb_unregister_device(ca->dvbdev);
	kfree(ca);

I think this should be since ioctl references slot info

	dvb_unregister_device(ca->dvbdev);
	kfree(ca->slot_info);
	kfree(ca);

I think dvb_ca_en50221_release() and dvb_ca_en50221_io_do_ioctl()
should serialize access to ca. dvb_ca_en50221_io_do_ioctl() holds
the ioctl_mutex, however, dvb_ca_en50221_release() could happen while
ioctl is in progress. Maybe you can try fixing those first.

As I mentioned in my review on your 3/3 patch, adding a kref here
adds more refcounted objects to the mix. You want to avoid that.

thanks,
-- Shuah

Re: [PATCH 1/3] drivers/media/dvb-core/en50221: use kref to manage struct dvb_ca_private

[Max Kellermann]

On 2016/06/16 18:06, Shuah Khan <shuahkh@osg.samsung.com> wrote:
> On 06/15/2016 02:15 PM, Max Kellermann wrote:
> > Don't free the object until the file handle has been closed.  Fixes
> > use-after-free bug which occurs when I disconnect my DVB-S received
> > while VDR is running.
> 
> Which file handle? /dev/dvb---

I don't know which one triggers it.  I get crashes with VDR, and VDR
opens all of them (ca0, demux0, frontend0), but won't release the file
handles even if they become defunct.  Only restarting the VDR process
leads to recovery (or crash).

> I think dvb_ca_en50221_release() and dvb_ca_en50221_io_do_ioctl()
> should serialize access to ca. dvb_ca_en50221_io_do_ioctl() holds
> the ioctl_mutex, however, dvb_ca_en50221_release() could happen while
> ioctl is in progress. Maybe you can try fixing those first.

True, there are LOTS of race conditions in the DVB code.  I see them
everywhere.  But that's orthogonal to my patch, isn't it?

> As I mentioned in my review on your 3/3 patch, adding a kref here
> adds more refcounted objects to the mix. You want to avoid that.

Mauro asked me to add the kref.  What is your suggestion to fix the
use-after-free bug?

I have a problem here, as mentioned in my last email: I don't know how
all of this is supposed to be, how it was designed; all I see is bugs
inside strange code, and I have to guess the previous author's
intentions and try to do the best to fix the code.

Max