From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F871C433E7 for ; Mon, 12 Oct 2020 14:36:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C1B4720870 for ; Mon, 12 Oct 2020 14:36:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="fkxMwduP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388963AbgJLOgA (ORCPT ); Mon, 12 Oct 2020 10:36:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726631AbgJLOf7 (ORCPT ); Mon, 12 Oct 2020 10:35:59 -0400 Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22D28C0613D1 for ; Mon, 12 Oct 2020 07:35:59 -0700 (PDT) Received: by mail-wm1-x342.google.com with SMTP id e2so17770631wme.1 for ; Mon, 12 Oct 2020 07:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to; bh=CS4OyMq4fS0Bhwsobzu7guLhX4olV/G/ClLkhuhMjgI=; b=fkxMwduPy7Mi/n+bV9EyV6Z1vJ1ek7B2dpi5Z+Cc1zoGQaPeAwtnD3hIUYpNwz8NNy grfTiephWpviO0XUTqemeoX5Fy0if/33WSQfHcs3Bnf5eC6FYLBpVWNoHVBL75gAP37l OLHc52gBx4Y8FnkvvzekCPn+IErOtdoHNRGKI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=CS4OyMq4fS0Bhwsobzu7guLhX4olV/G/ClLkhuhMjgI=; b=DC7BICbyD06684Oc4SqG/l0ZX2R8h13AJpDC3pboxrVjV8m8gsMy+61w3XkrLo9hYR DfowQ3QllPwQJtz0Jaol6XtbgYLftHsW3t74f97Zp6w3ZmUNc9IHrjfT94Pc0PPfp+rb 2yZx201TXjMfYhA1c8wfpn1099GCwYvzKdpW6IVAW+dv/48Sc22y+K7EA4HTFH3PBbCn YBjzrZ8SoB/iofioCoqchpVXhL6Km2AMfAEFUVDd/jdpROEn5KJecwa54hdGhomfyK4O JGM5tu3kYQomo+7MikaEXAChi56pyYO/qR10X94LfxIRpTXVLJW/fj+kLLc5K1CKXfCZ 3UJg== X-Gm-Message-State: AOAM5320qws9hb5tabe4Vdo1j4jD9vCk6OQMTeSdAbIOKTYE/oENiSXH QbKNOcPhtnK0Pk6Tb+OddnmDlg== X-Google-Smtp-Source: ABdhPJxGVuL+rWFfcFuPZBFAL6pDzTxq2Ad70ao8xyHZW6uniXqiGrdMDz9Voec37zpOONepDpbVBQ== X-Received: by 2002:a7b:c8c9:: with SMTP id f9mr2895067wml.162.1602513357830; Mon, 12 Oct 2020 07:35:57 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id 30sm15869709wrs.84.2020.10.12.07.35.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Oct 2020 07:35:56 -0700 (PDT) Date: Mon, 12 Oct 2020 16:35:55 +0200 From: Daniel Vetter To: Rob Clark Cc: dri-devel@lists.freedesktop.org, Daniel Vetter , Rob Clark , Sean Paul , David Airlie , "open list:DRM DRIVER FOR MSM ADRENO GPU" , "open list:DRM DRIVER FOR MSM ADRENO GPU" , open list Subject: Re: [PATCH v2 07/22] drm/msm: Do rpm get sooner in the submit path Message-ID: <20201012143555.GA438822@phenom.ffwll.local> Mail-Followup-To: Rob Clark , dri-devel@lists.freedesktop.org, Rob Clark , Sean Paul , David Airlie , "open list:DRM DRIVER FOR MSM ADRENO GPU" , "open list:DRM DRIVER FOR MSM ADRENO GPU" , open list References: <20201012020958.229288-1-robdclark@gmail.com> <20201012020958.229288-8-robdclark@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201012020958.229288-8-robdclark@gmail.com> X-Operating-System: Linux phenom 5.7.0-1-amd64 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Sun, Oct 11, 2020 at 07:09:34PM -0700, Rob Clark wrote: > From: Rob Clark > > Unfortunately, due to an dev_pm_opp locking interaction with > mm->mmap_sem, we need to do pm get before aquiring obj locks, > otherwise we can have anger lockdep with the chain: tbh this sounds like a bug in that subsystem, since it means we cannot use said subsystem in mmap handlers either. So if you have some remapping unit or need to wake up your gpu to blt the buffer into system memory first, we're toast. That doesn't sound right. So maybe Cc: pm folks and figure out how to fix this long term properly? Imo not a good reason to hold up this patch set, since unwrangling mmap_sem tends to be work ... -Daniel > > opp_table_lock --> &mm->mmap_sem --> reservation_ww_class_mutex > > For an explicit fencing userspace, the impact should be minimal > as we do all the fence waits before this point. It could result > in some needless resumes in error cases, etc. > > Signed-off-by: Rob Clark > --- > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++++++++-- > 1 file changed, 13 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c > index 002130d826aa..a9422d043bfe 100644 > --- a/drivers/gpu/drm/msm/msm_gem_submit.c > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c > @@ -744,11 +744,20 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, > > ret = submit_lookup_objects(submit, args, file); > if (ret) > - goto out; > + goto out_pre_pm; > > ret = submit_lookup_cmds(submit, args, file); > if (ret) > - goto out; > + goto out_pre_pm; > + > + /* > + * Thanks to dev_pm_opp opp_table_lock interactions with mm->mmap_sem > + * in the resume path, we need to to rpm get before we lock objs. > + * Which unfortunately might involve powering up the GPU sooner than > + * is necessary. But at least in the explicit fencing case, we will > + * have already done all the fence waiting. > + */ > + pm_runtime_get_sync(&gpu->pdev->dev); > > /* copy_*_user while holding a ww ticket upsets lockdep */ > ww_acquire_init(&submit->ticket, &reservation_ww_class); > @@ -825,6 +834,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, > > > out: > + pm_runtime_put(&gpu->pdev->dev); > +out_pre_pm: > submit_cleanup(submit); > if (has_ww_ticket) > ww_acquire_fini(&submit->ticket); > -- > 2.26.2 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E3A4C433DF for ; Mon, 12 Oct 2020 14:36:01 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 91D5220735 for ; Mon, 12 Oct 2020 14:36:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="fkxMwduP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91D5220735 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0ED406E49F; Mon, 12 Oct 2020 14:36:00 +0000 (UTC) Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by gabe.freedesktop.org (Postfix) with ESMTPS id 227D26E49C for ; Mon, 12 Oct 2020 14:35:59 +0000 (UTC) Received: by mail-wm1-x342.google.com with SMTP id f21so17344634wml.3 for ; Mon, 12 Oct 2020 07:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to; bh=CS4OyMq4fS0Bhwsobzu7guLhX4olV/G/ClLkhuhMjgI=; b=fkxMwduPy7Mi/n+bV9EyV6Z1vJ1ek7B2dpi5Z+Cc1zoGQaPeAwtnD3hIUYpNwz8NNy grfTiephWpviO0XUTqemeoX5Fy0if/33WSQfHcs3Bnf5eC6FYLBpVWNoHVBL75gAP37l OLHc52gBx4Y8FnkvvzekCPn+IErOtdoHNRGKI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=CS4OyMq4fS0Bhwsobzu7guLhX4olV/G/ClLkhuhMjgI=; b=Q0Vh1mutKghjEiepJ1YQg3B1wtOzup5rcHsbQlclcnOOz5uOJ3G+gloxi3EDW4K8FH lL5QMGuH7VZoYIVa3gV4+HMm1RZZLm+OIHxcgxPn+32n2SJ9ppaGGx6ikkD38au3mZCD HGP4BlC6jnXljbMx1velIFbjFYd5c+mYQUG91Y+RVHxSZmpYvNd6L7PuZg751OBzaREU 6O0aOThaNXxdBKEmR6mehNsnoVYhPxKa9tfjG1A4PLDpenrozrTjvqaCGC3YMt5c/q+E FcIVD1QwM+lYk49e73CG0WWUfcSEo/dLsfPGzA+K75P3+G0H6YbBA4tCz1auMdYas90S q64w== X-Gm-Message-State: AOAM532632UnXHkRAKRhYqkYYeQFoUvpuUpsrqV35V0PV8c/6wE037J8 XRBwe1HAu+vCGcheOGz/lyswVQ== X-Google-Smtp-Source: ABdhPJxGVuL+rWFfcFuPZBFAL6pDzTxq2Ad70ao8xyHZW6uniXqiGrdMDz9Voec37zpOONepDpbVBQ== X-Received: by 2002:a7b:c8c9:: with SMTP id f9mr2895067wml.162.1602513357830; Mon, 12 Oct 2020 07:35:57 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id 30sm15869709wrs.84.2020.10.12.07.35.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Oct 2020 07:35:56 -0700 (PDT) Date: Mon, 12 Oct 2020 16:35:55 +0200 From: Daniel Vetter To: Rob Clark Subject: Re: [PATCH v2 07/22] drm/msm: Do rpm get sooner in the submit path Message-ID: <20201012143555.GA438822@phenom.ffwll.local> Mail-Followup-To: Rob Clark , dri-devel@lists.freedesktop.org, Rob Clark , Sean Paul , David Airlie , "open list:DRM DRIVER FOR MSM ADRENO GPU" , "open list:DRM DRIVER FOR MSM ADRENO GPU" , open list References: <20201012020958.229288-1-robdclark@gmail.com> <20201012020958.229288-8-robdclark@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20201012020958.229288-8-robdclark@gmail.com> X-Operating-System: Linux phenom 5.7.0-1-amd64 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Clark , "open list:DRM DRIVER FOR MSM ADRENO GPU" , David Airlie , "open list:DRM DRIVER FOR MSM ADRENO GPU" , open list , dri-devel@lists.freedesktop.org, Sean Paul Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Sun, Oct 11, 2020 at 07:09:34PM -0700, Rob Clark wrote: > From: Rob Clark > > Unfortunately, due to an dev_pm_opp locking interaction with > mm->mmap_sem, we need to do pm get before aquiring obj locks, > otherwise we can have anger lockdep with the chain: tbh this sounds like a bug in that subsystem, since it means we cannot use said subsystem in mmap handlers either. So if you have some remapping unit or need to wake up your gpu to blt the buffer into system memory first, we're toast. That doesn't sound right. So maybe Cc: pm folks and figure out how to fix this long term properly? Imo not a good reason to hold up this patch set, since unwrangling mmap_sem tends to be work ... -Daniel > > opp_table_lock --> &mm->mmap_sem --> reservation_ww_class_mutex > > For an explicit fencing userspace, the impact should be minimal > as we do all the fence waits before this point. It could result > in some needless resumes in error cases, etc. > > Signed-off-by: Rob Clark > --- > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++++++++-- > 1 file changed, 13 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c > index 002130d826aa..a9422d043bfe 100644 > --- a/drivers/gpu/drm/msm/msm_gem_submit.c > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c > @@ -744,11 +744,20 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, > > ret = submit_lookup_objects(submit, args, file); > if (ret) > - goto out; > + goto out_pre_pm; > > ret = submit_lookup_cmds(submit, args, file); > if (ret) > - goto out; > + goto out_pre_pm; > + > + /* > + * Thanks to dev_pm_opp opp_table_lock interactions with mm->mmap_sem > + * in the resume path, we need to to rpm get before we lock objs. > + * Which unfortunately might involve powering up the GPU sooner than > + * is necessary. But at least in the explicit fencing case, we will > + * have already done all the fence waiting. > + */ > + pm_runtime_get_sync(&gpu->pdev->dev); > > /* copy_*_user while holding a ww ticket upsets lockdep */ > ww_acquire_init(&submit->ticket, &reservation_ww_class); > @@ -825,6 +834,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, > > > out: > + pm_runtime_put(&gpu->pdev->dev); > +out_pre_pm: > submit_cleanup(submit); > if (has_ww_ticket) > ww_acquire_fini(&submit->ticket); > -- > 2.26.2 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel