From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout2.w1.samsung.com ([210.118.77.12]:46711 "EHLO mailout2.w1.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751910AbdLMM1m (ORCPT ); Wed, 13 Dec 2017 07:27:42 -0500 MIME-version: 1.0 Content-type: text/plain; charset="utf-8"; format="flowed" Subject: Re: [PATCH] drm: rework delayed connector cleanup in connector_iter To: Daniel Vetter , Intel Graphics Development Cc: Ben Widawsky , Tomeu Vizoso , David Airlie , Mark Brown , Shuah Khan , DRI Development , Thierry Escande , Guillaume Tucker , stable@vger.kernel.org, Enric Balletbo i Serra , Daniel Vetter , Javier Martinez Canillas , Kevin Hilman , Matt Hart From: Marek Szyprowski Message-id: <2afb1fd9-fc9f-df1e-2a97-1071764ed9b0@samsung.com> Date: Wed, 13 Dec 2017 13:27:37 +0100 In-reply-to: <20171213104553.30821-1-daniel.vetter@ffwll.ch> Content-transfer-encoding: 8bit Content-language: en-US References: <20171213104553.30821-1-daniel.vetter@ffwll.ch> Sender: stable-owner@vger.kernel.org List-ID: Hi Daniel, On 2017-12-13 11:45, Daniel Vetter wrote: > PROBE_DEFER also uses system_wq to reprobe drivers, which means when > that again fails, and we try to flush the overall system_wq (to get > all the delayed connectore cleanup work_struct completed), we > deadlock. > > Fix this by using just a single cleanup work, so that we can only > flush that one and don't block on anything else. That means a free > list plus locking, a standard pattern. > > Fixes: a703c55004e1 ("drm: safely free connectors from connector_iter") > Fixes: 613051dac40d ("drm: locking&new iterators for connector_list") > Cc: Ben Widawsky > Cc: Dave Airlie > Cc: Chris Wilson > Cc: Sean Paul > Cc: # v4.11+: 613051dac40d ("drm: locking&new iterators for connector_list" > Cc: # v4.11+ > Cc: Daniel Vetter > Cc: Jani Nikula > Cc: Gustavo Padovan > Cc: David Airlie > Cc: Javier Martinez Canillas > Cc: Shuah Khan > Cc: Guillaume Tucker > Cc: Mark Brown > Cc: Kevin Hilman > Cc: Matt Hart > Cc: Thierry Escande > Cc: Tomeu Vizoso > Cc: Enric Balletbo i Serra > Signed-off-by: Daniel Vetter Sadly, this patch doesn't work in my test environment (Samsung Snow Chromebook with modified DTS), which reproduces the problem discussed inhttps://www.spinics.net/lists/arm-kernel/msg622332.html [    4.925178] exynos-drm exynos-drm: failed to bind 14530000.hdmi (ops hdmi_component_ops): -19 [    4.936167] Unable to handle kernel paging request at virtual address fffffcac [    4.942004] pgd = ad8213ad [    4.944654] [fffffcac] *pgd=6fffd861, *pte=00000000, *ppte=00000000 [    4.950944] Internal error: Oops: 37 [#1] PREEMPT SMP ARM [    4.956284] Modules linked in: [    4.959324] CPU: 1 PID: 80 Comm: kworker/1:3 Not tainted 4.15.0-rc3-next-20171212-00004-g879e6206b6fb-dirty #56 [    4.969391] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree) [    4.975472] Workqueue: events drm_connector_free_work_fn [    4.980765] PC is at drm_mode_object_unregister+0x1c/0x48 [    4.986146] LR is at __mutex_lock+0x3ec/0xa0c [    4.990483] pc : []    lr : [] psr: 60000113 [    4.996731] sp : ed27dec0  ip : ed27c000  fp : ed25042c [    5.001940] r10: 00000000  r9 : c1008c6c  r8 : c10ae6d4 [    5.007148] r7 : ed2503ec  r6 : ed250000  r5 : fffffcac  r4 : ed250330 [    5.013659] r3 : 00000000  r2 : 00000001  r1 : ed27de50  r0 : 00000000 [    5.020170] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM Segment none [    5.027286] Control: 10c5387d  Table: 4000406a  DAC: 00000051 [    5.033016] Process kworker/1:3 (pid: 80, stack limit = 0x37cd389f) [    5.039265] Stack: (0xed27dec0 to 0xed27e000) [    5.043607] dec0: fffffc98 ed27ded0 ed250000 c050d374 ed242f78 ed242f78 00000001 ed22e200 [    5.051766] dee0: eefc91c0 eefcc500 c10ad21d c0143438 00000001 00000000 c0143384 c0143534 [    5.059925] df00: 00000001 00000000 c0143384 00000000 edd0ce00 c181658c c1214a78 00000000 [    5.068084] df20: c0d7c514 00000000 c120f048 ed22e200 ed22e218 eefc91f8 c0143ae8 c10ae166 [    5.076243] df40: 00000008 ed22e218 c1005d00 c0143a2c eefcc5f5 ed22e200 eefc91c0 c0143c4c [    5.084402] df60: c0a09894 ed709100 00000000 ed709100 00000000 ed25e000 ed709180 ed22e200 [    5.092561] df80: edfdfe68 c0143a3c 00000000 c014a1c4 ed25e000 c014a074 00000000 00000000 [    5.100720] dfa0: 00000000 00000000 00000000 c01010b4 00000000 00000000 00000000 00000000 [    5.108879] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [    5.117039] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000 [    5.125202] [] (drm_mode_object_unregister) from [] (drm_connector_free_work_fn+0x80/0xa8) [    5.135185] [] (drm_connector_free_work_fn) from [] (process_one_work+0x1dc/0x7a8) [    5.144472] [] (process_one_work) from [] (process_scheduled_works+0x28/0x38) [    5.153324] [] (process_scheduled_works) from [] (worker_thread+0x210/0x4dc) [    5.162092] [] (worker_thread) from [] (kthread+0x150/0x19c) [    5.169469] [] (kthread) from [] (ret_from_fork+0x14/0x20) [    5.176669] Exception stack(0xed27dfb0 to 0xed27dff8) [    5.181704] dfa0:                                     00000000 00000000 00000000 00000000 [    5.189864] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [    5.198023] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [    5.204622] Code: e1a06000 e3a01000 e1a00004 eb13d968 (e5951000) [    5.210731] ---[ end trace 55b8b9b30581db4f ]--- I will try to provide a bit more details soon. > --- > drivers/gpu/drm/drm_connector.c | 46 ++++++++++++++++++++++++++----------- > drivers/gpu/drm/drm_crtc_internal.h | 1 + > drivers/gpu/drm/drm_mode_config.c | 5 +++- > include/drm/drm_connector.h | 4 ++-- > include/drm/drm_mode_config.h | 17 +++++++++++++- > 5 files changed, 55 insertions(+), 18 deletions(-) > > diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c > index 0b7e0974e6da..2f43dc217936 100644 > --- a/drivers/gpu/drm/drm_connector.c > +++ b/drivers/gpu/drm/drm_connector.c > @@ -153,14 +153,23 @@ static void drm_connector_free(struct kref *kref) > connector->funcs->destroy(connector); > } > > -static void drm_connector_free_work_fn(struct work_struct *work) > +void drm_connector_free_work_fn(struct work_struct *work) > { > - struct drm_connector *connector = > - container_of(work, struct drm_connector, free_work); > - struct drm_device *dev = connector->dev; > + struct drm_connector *connector; > + struct drm_device *dev = > + container_of(work, struct drm_device, mode_config.connector_free_work); > + struct drm_mode_config *config = &dev->mode_config; > + unsigned long flags; > + LIST_HEAD(list_head); > > - drm_mode_object_unregister(dev, &connector->base); > - connector->funcs->destroy(connector); > + spin_lock_irqsave(&config->connector_list_lock, flags); > + list_splice(&config->connector_free_list, &list_head); > + spin_unlock_irqrestore(&config->connector_list_lock, flags); > + > + list_for_each_entry(connector, &list_head, free_head) { > + drm_mode_object_unregister(dev, &connector->base); > + connector->funcs->destroy(connector); > + } > } > > /** > @@ -192,8 +201,6 @@ int drm_connector_init(struct drm_device *dev, > if (ret) > return ret; > > - INIT_WORK(&connector->free_work, drm_connector_free_work_fn); > - > connector->base.properties = &connector->properties; > connector->dev = dev; > connector->funcs = funcs; > @@ -550,10 +557,15 @@ EXPORT_SYMBOL(drm_connector_list_iter_begin); > * actually release the connector when dropping our final reference. > */ > static void > -drm_connector_put_safe(struct drm_connector *conn) > +__drm_connector_put_safe(struct drm_connector *conn) > { > + struct drm_mode_config *config = &conn->dev->mode_config; > + > + lockdep_assert_held(&config->connector_list_lock); > + list_add(&conn->free_head, &config->connector_free_list); > + > if (refcount_dec_and_test(&conn->base.refcount.refcount)) > - schedule_work(&conn->free_work); > + schedule_work(&config->connector_free_work); > } > > /** > @@ -585,10 +597,10 @@ drm_connector_list_iter_next(struct drm_connector_list_iter *iter) > > /* loop until it's not a zombie connector */ > } while (!kref_get_unless_zero(&iter->conn->base.refcount)); > - spin_unlock_irqrestore(&config->connector_list_lock, flags); > > if (old_conn) > - drm_connector_put_safe(old_conn); > + __drm_connector_put_safe(old_conn); > + spin_unlock_irqrestore(&config->connector_list_lock, flags); > > return iter->conn; > } > @@ -605,9 +617,15 @@ EXPORT_SYMBOL(drm_connector_list_iter_next); > */ > void drm_connector_list_iter_end(struct drm_connector_list_iter *iter) > { > + struct drm_mode_config *config = &iter->dev->mode_config; > + unsigned long flags; > + > iter->dev = NULL; > - if (iter->conn) > - drm_connector_put_safe(iter->conn); > + if (iter->conn) { > + spin_lock_irqsave(&config->connector_list_lock, flags); > + __drm_connector_put_safe(iter->conn); > + spin_unlock_irqrestore(&config->connector_list_lock, flags); > + } > lock_release(&connector_list_iter_dep_map, 0, _RET_IP_); > } > EXPORT_SYMBOL(drm_connector_list_iter_end); > diff --git a/drivers/gpu/drm/drm_crtc_internal.h b/drivers/gpu/drm/drm_crtc_internal.h > index 9ebb8841778c..af00f42ba269 100644 > --- a/drivers/gpu/drm/drm_crtc_internal.h > +++ b/drivers/gpu/drm/drm_crtc_internal.h > @@ -142,6 +142,7 @@ int drm_mode_connector_set_obj_prop(struct drm_mode_object *obj, > uint64_t value); > int drm_connector_create_standard_properties(struct drm_device *dev); > const char *drm_get_connector_force_name(enum drm_connector_force force); > +void drm_connector_free_work_fn(struct work_struct *work); > > /* IOCTL */ > int drm_mode_connector_property_set_ioctl(struct drm_device *dev, > diff --git a/drivers/gpu/drm/drm_mode_config.c b/drivers/gpu/drm/drm_mode_config.c > index 6ffe952142e6..025fc23cf7c6 100644 > --- a/drivers/gpu/drm/drm_mode_config.c > +++ b/drivers/gpu/drm/drm_mode_config.c > @@ -373,6 +373,7 @@ void drm_mode_config_init(struct drm_device *dev) > INIT_LIST_HEAD(&dev->mode_config.fb_list); > INIT_LIST_HEAD(&dev->mode_config.crtc_list); > INIT_LIST_HEAD(&dev->mode_config.connector_list); > + INIT_LIST_HEAD(&dev->mode_config.connector_free_list); > INIT_LIST_HEAD(&dev->mode_config.encoder_list); > INIT_LIST_HEAD(&dev->mode_config.property_list); > INIT_LIST_HEAD(&dev->mode_config.property_blob_list); > @@ -382,6 +383,8 @@ void drm_mode_config_init(struct drm_device *dev) > ida_init(&dev->mode_config.connector_ida); > spin_lock_init(&dev->mode_config.connector_list_lock); > > + INIT_WORK(&dev->mode_config.connector_free_work, drm_connector_free_work_fn); > + > drm_mode_create_standard_properties(dev); > > /* Just to be sure */ > @@ -432,7 +435,7 @@ void drm_mode_config_cleanup(struct drm_device *dev) > } > drm_connector_list_iter_end(&conn_iter); > /* connector_iter drops references in a work item. */ > - flush_scheduled_work(); > + flush_work(&dev->mode_config.connector_free_work); > if (WARN_ON(!list_empty(&dev->mode_config.connector_list))) { > drm_connector_list_iter_begin(dev, &conn_iter); > drm_for_each_connector_iter(connector, &conn_iter) > diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h > index ee3155391243..50bc21050fbf 100644 > --- a/include/drm/drm_connector.h > +++ b/include/drm/drm_connector.h > @@ -966,12 +966,12 @@ struct drm_connector { > uint16_t tile_h_size, tile_v_size; > > /** > - * @free_work: > + * @free_head: > * > * Work used only by &drm_connector_iter to be able to clean up a > * connector from any context. > */ > - struct work_struct free_work; > + struct list_head free_head; > }; > > #define obj_to_connector(x) container_of(x, struct drm_connector, base) > diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h > index e5f3b43014e1..5e62e52a2c58 100644 > --- a/include/drm/drm_mode_config.h > +++ b/include/drm/drm_mode_config.h > @@ -402,7 +402,7 @@ struct drm_mode_config { > > /** > * @connector_list_lock: Protects @num_connector and > - * @connector_list. > + * @connector_list and @connector_free_list. > */ > spinlock_t connector_list_lock; > /** > @@ -422,6 +422,21 @@ struct drm_mode_config { > * &struct drm_connector_list_iter to walk this list. > */ > struct list_head connector_list; > + /** > + * @connector_free_list: > + * > + * List of connector objects linked with &drm_connector.free_head. > + * Protected by @connector_list_lock. Used by > + * drm_for_each_connector_iter() and > + * &struct drm_connector_list_iter to savely free connectors using > + * @connector_free_work. > + */ > + struct list_head connector_free_list; > + /** > + * @connector_free_work: Work to clean up @connector_free_list. > + */ > + struct work_struct connector_free_work; > + > /** > * @num_encoder: > * Best regards -- Marek Szyprowski, PhD Samsung R&D Institute Poland