From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D87AC433E1 for ; Fri, 19 Jun 2020 17:23:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1C531208D5 for ; Fri, 19 Jun 2020 17:23:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="jQ8P3pnI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392240AbgFSRXM (ORCPT ); Fri, 19 Jun 2020 13:23:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731175AbgFSRXL (ORCPT ); Fri, 19 Jun 2020 13:23:11 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6345EC06174E for ; Fri, 19 Jun 2020 10:23:10 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id q8so9618382qkm.12 for ; Fri, 19 Jun 2020 10:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=rta7XgSy0ckbOhf+G5R08zHN2raoo0kZf2xw2Sh0l/k=; b=jQ8P3pnIGa3vZkblMbXH41QZ61fe/Sb9vA5uPOR3SMNdZKsu0blkPtrxpS3aWVoNKr DxLMI5S5juOldxRe7+RlT3GajySSEpNFCHak6ENUR11aQ3nJENjJ+AjGwVIg3GcU1gtL AFg4tJnhN/LzdKUCOPE+zScs2pSCySWCm6gLf1r8dvVDgTF8uLAu4b0cSB8SVQcBr9IO hvo365OipT4dztwxdexoBi2dZXF91j/xcy5DKKz/ErbusQM/iZVWPTDKzHQRUnZ5G+PQ 1Vw552gg3QCopo/fc6PBC9ArUokmC/IBfhMuddzwZFS3/FrWylp92kucX6mn/XCcEkQF 2/SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=rta7XgSy0ckbOhf+G5R08zHN2raoo0kZf2xw2Sh0l/k=; b=H8H8IeVdwlRgMdI70ZjYLbuaW0eSIFd4E0Qg5gRfDO20G7XLRNykcWhZ9rzMXU6pVg x5q+ZlMzKx093+jMuDosj3UISCVNT5o6woBKh6Vv7Ova0WCu50hHb9NjT1yN6oYdSyvl vn0uXosJlrIMsXXoMRuqS3fISuVw1ah0UOKEIh3LfY+yuCpTkMUfcebloTQnjClrBlb4 Lke0D1BzKBmiZ54RqqrhYadCw37/BU/f6sF0/GEusdxtm6hVL+bP8algqN3lfBR6d7F9 /JHHVyBNUG+OmDcyJUm0xiN8D+v+qHfJZgtrkyVrUd2oZVhVD8HAR2sf6YDhxeWtcjoY KHbg== X-Gm-Message-State: AOAM531PuA56kVxk+boNqAMR6R539Tho6ltY1+2tGTwt5j1gMbDyrqCG OYdWmUQlZkC6WrL9H71Ohx1l5JG0M2jl5g== X-Google-Smtp-Source: ABdhPJzDMW9WT5FfWlenUCjhHpgpjiA2TJ2PiIGS5w9RkI8RANL7HnmArRn+HLKwhDBh+PaKr6DrLw== X-Received: by 2002:ae9:ed0d:: with SMTP id c13mr4076677qkg.181.1592587389664; Fri, 19 Jun 2020 10:23:09 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-48-30.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.48.30]) by smtp.gmail.com with ESMTPSA id z77sm7263705qka.59.2020.06.19.10.23.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Jun 2020 10:23:08 -0700 (PDT) Received: from jgg by mlx with local (Exim 4.93) (envelope-from ) id 1jmKjM-00ArZ4-2Z; Fri, 19 Jun 2020 14:23:08 -0300 Date: Fri, 19 Jun 2020 14:23:08 -0300 From: Jason Gunthorpe To: Daniel Vetter Cc: Thomas =?utf-8?B?SGVsbHN0csO2bSAoSW50ZWwp?= , DRI Development , linux-rdma , Intel Graphics Development , Maarten Lankhorst , LKML , amd-gfx list , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Thomas Hellstrom , Daniel Vetter , "open list:DMA BUFFER SHARING FRAMEWORK" , Christian =?utf-8?B?S8O2bmln?= , Mika Kuoppala Subject: Re: [Linaro-mm-sig] [PATCH 04/18] dma-fence: prime lockdep annotations Message-ID: <20200619172308.GQ6578@ziepe.ca> References: <20200616120719.GL20149@phenom.ffwll.local> <20200617152835.GF6578@ziepe.ca> <20200618150051.GS20149@phenom.ffwll.local> <20200618172338.GM6578@ziepe.ca> <20200619113934.GN6578@ziepe.ca> <20200619151551.GP6578@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jun 19, 2020 at 06:19:41PM +0200, Daniel Vetter wrote: > The madness is only that device B's mmu notifier might need to wait > for fence_B so that the dma operation finishes. Which in turn has to > wait for device A to finish first. So, it sound, fundamentally you've got this graph of operations across an unknown set of drivers and the kernel cannot insert itself in dma_fence hand offs to re-validate any of the buffers involved? Buffers which by definition cannot be touched by the hardware yet. That really is a pretty horrible place to end up.. Pinning really is right answer for this kind of work flow. I think converting pinning to notifers should not be done unless notifier invalidation is relatively bounded. I know people like notifiers because they give a bit nicer performance in some happy cases, but this cripples all the bad cases.. If pinning doesn't work for some reason maybe we should address that? > Full disclosure: We are aware that we've designed ourselves into an > impressive corner here, and there's lots of talks going on about > untangling the dma synchronization from the memory management > completely. But I think the documenting is really important: only GPU should be using this stuff and driving notifiers this way. Complete NO for any totally-not-a-GPU things in drivers/accel for sure. Jason