From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB551C4332D for ; Wed, 18 Mar 2020 23:43:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BD8DB2076F for ; Wed, 18 Mar 2020 23:43:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i3FPub1D" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727095AbgCRXnU (ORCPT ); Wed, 18 Mar 2020 19:43:20 -0400 Received: from mail-ed1-f65.google.com ([209.85.208.65]:38478 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726777AbgCRXnU (ORCPT ); Wed, 18 Mar 2020 19:43:20 -0400 Received: by mail-ed1-f65.google.com with SMTP id h5so356902edn.5; Wed, 18 Mar 2020 16:43:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=T6rxV8z4faDRzhqK467Xv2a7dOy/TE520C3YV3WjV8c=; b=i3FPub1DIHkvCD+yi/2Y0pX69MrTIvO1RmbzVf6wk00DV0ApmY13YfnnFpks/NVTLs G1QplMy/+0w+GsFrwD1fV1STvoKneEr8NQlSW0vIhii6MAK+xDn8ddbmfxyIfh+bWkPE NEeRdRZ2QE6MO1YpPwSPzXvQKexEK5sr8Dox9C9jyfSyvqP6X6trr48Ms86RoFLn9dAG 92zK1vRSIdLYDibQabEa5foOn7Ajn4QGZfbFS6XCk1yr6R/d+WaM4jqJ31uE4ljyQmMb YiomgNSyyErYriEW6Abwe6sruyYqTy62RHKzpwUnN7Vinym7R/FBnsL8LzowJAa3l8q2 3Z7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=T6rxV8z4faDRzhqK467Xv2a7dOy/TE520C3YV3WjV8c=; b=O1nEDNd9FSAUNGQHVc6BjpCTY0iaCckSwTESS23JEBb0jXNfg5aYUNZTspKC6t0Hwz oqGpUYH3lGjOcp6jNX6IjK6LJadyknTgTLB1qLKyu1PBz308HsUwASFx6zVO+rSznWDo lZgm4bDogt7Th5zeWaRMbKvFZ3MYOvCfpaChIR3T/Yp6FJdQkxMcoz2AmisY5ktxpUwr jNFKV6+Q7nRA+y+1gG8diMlNd63E8NlMdZnmHU+D39wOubdDwH02CV2y9d9jGkFje0jF G/Pvvutb53oYWTEmlvqvo/Qvmi/a6XWcA8PyTGInBH0ctZJMmbsS08S4kXJBbmh5zPoH abyA== X-Gm-Message-State: ANhLgQ1kLEpvZznWhcEJx/I0rWwTN15B9sE9k1mHG4S6K3M+6ZslQHZP Fp2x09LPQmAO5wIswFRP/+iWh9d3q3KhaciAjFDStbvC X-Google-Smtp-Source: ADFU+vunbg/mabJAoT4ECckSo9VlWXEI7Tk7MCM2NEIzGX//dTyumqWFksHG62C18XwPsVHwF/wqELk+aHHpZR6UYqI= X-Received: by 2002:a17:906:b888:: with SMTP id hb8mr686525ejb.166.1584574997823; Wed, 18 Mar 2020 16:43:17 -0700 (PDT) MIME-Version: 1.0 References: <1580249770-1088-1-git-send-email-jcrouse@codeaurora.org> <1580249770-1088-3-git-send-email-jcrouse@codeaurora.org> <20200318224840.GA10796@willie-the-truck> In-Reply-To: <20200318224840.GA10796@willie-the-truck> From: Rob Clark Date: Wed, 18 Mar 2020 16:43:07 -0700 Message-ID: Subject: Re: [PATCH v1 2/6] arm/smmu: Add auxiliary domain support for arm-smmuv2 To: Will Deacon Cc: Jordan Crouse , linux-arm-msm , Linux Kernel Mailing List , "list@263.net:IOMMU DRIVERS , Joerg Roedel ," , Robin Murphy , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" Content-Type: text/plain; charset="UTF-8" Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Wed, Mar 18, 2020 at 3:48 PM Will Deacon wrote: > > On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > > Support auxiliary domains for arm-smmu-v2 to initialize and support > > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > > hardware doesn't have any built in support for switching the pagetable > > base it is left as an exercise to the caller to actually use the pagetable. > > > > Aux domains are supported if split pagetable (TTBR1) support has been > > enabled on the master domain. Each auxiliary domain will reuse the > > configuration of the master domain. By default the a domain with TTBR1 > > support will have the TTBR0 region disabled so the first attached aux > > domain will enable the TTBR0 region in the hardware and conversely the > > last domain to be detached will disable TTBR0 translations. All subsequent > > auxiliary domains create a pagetable but not touch the hardware. > > > > The leaf driver will be able to query the physical address of the > > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > > address with whatever means it has to switch the pagetable base. > > > > Following is a pseudo code example of how a domain can be created > > > > /* Check to see if aux domains are supported */ > > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > > iommu = iommu_domain_alloc(...); > > > > if (iommu_aux_attach_device(domain, dev)) > > return FAIL; > > > > /* Save the base address of the pagetable for use by the driver > > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > > } > > I'm not really understanding what the pagetable base gets used for here and, > to be honest with you, the whole thing feels like a huge layering violation > with the way things are structured today. Why doesn't the caller just > interface with io-pgtable directly? > > Finally, if we need to support context-switching TTBR0 for a live domain > then that code really needs to live inside the SMMU driver because the > ASID and TLB management necessary to do that safely doesn't belong anywhere > else. Hi Will, We do in fact need live domain switching, that is really the whole point. The GPU CP (command processor/parser) is directly updating TTBR0 and triggering TLB flush, asynchronously from the CPU. And I think the answer about ASID is easy (on current hw).. it must be zero[*]. BR, -R [*] my rough theory/plan there, and to solve the issue with drm/msm getting dma-iommu ops when it really would rather not (since blacklisting idea wasn't popular and I couldn't figure out a way to deal with case where device gets attached before driver shows up) is to invent some API that drm/msm can call to unhook the dma-iommu ops and detatch the DMA domain. Hopefully that at least gets us closer to the point where, when drm/msm attaches it's UNMANAGED domain, we get cbidx/asid zero. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23665C4332D for ; Wed, 18 Mar 2020 23:43:25 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D889520768 for ; Wed, 18 Mar 2020 23:43:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i3FPub1D" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D889520768 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id A958187767; Wed, 18 Mar 2020 23:43:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y8JxqGm+Qvf7; Wed, 18 Mar 2020 23:43:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 8276187750; Wed, 18 Mar 2020 23:43:24 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7D193C1799; Wed, 18 Mar 2020 23:43:22 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id E1F01C087F for ; Wed, 18 Mar 2020 23:43:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id CF3C7882F6 for ; Wed, 18 Mar 2020 23:43:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tunGIjiSE6fD for ; Wed, 18 Mar 2020 23:43:20 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) by hemlock.osuosl.org (Postfix) with ESMTPS id AB44087CC9 for ; Wed, 18 Mar 2020 23:43:19 +0000 (UTC) Received: by mail-ed1-f67.google.com with SMTP id b21so326982edy.9 for ; Wed, 18 Mar 2020 16:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=T6rxV8z4faDRzhqK467Xv2a7dOy/TE520C3YV3WjV8c=; b=i3FPub1DIHkvCD+yi/2Y0pX69MrTIvO1RmbzVf6wk00DV0ApmY13YfnnFpks/NVTLs G1QplMy/+0w+GsFrwD1fV1STvoKneEr8NQlSW0vIhii6MAK+xDn8ddbmfxyIfh+bWkPE NEeRdRZ2QE6MO1YpPwSPzXvQKexEK5sr8Dox9C9jyfSyvqP6X6trr48Ms86RoFLn9dAG 92zK1vRSIdLYDibQabEa5foOn7Ajn4QGZfbFS6XCk1yr6R/d+WaM4jqJ31uE4ljyQmMb YiomgNSyyErYriEW6Abwe6sruyYqTy62RHKzpwUnN7Vinym7R/FBnsL8LzowJAa3l8q2 3Z7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=T6rxV8z4faDRzhqK467Xv2a7dOy/TE520C3YV3WjV8c=; b=t7JOCVxir1YDvWpuzePISXRJKtrpuPs0iN3FO2pmdgGMTPk2e/Xvq95elyL9SJLLOc rRzQ0fYETw7HeHNtjsfdj2tAK3PmzzVewsysbB/kesyK68wKRp5mzk/XMYEpB0rzel+o Bini/Jirc2N97SmE3fsXkmES09OxrcjaGTwSNJslcWrSGOKg/KEEfLYMTU5uzb4WLI3g CGZ66JzSP1a46wgE8IstgfVl9db1IyF2EfEtJtDUu1nbAejjiW4U4PJ6LBf/dhVFXXmX Y0n6ivqBysmxoTCTlUr7g2tyG6dp5S7Xvy30BLDBhmU2yoMLtUFAHQI1HA51i611PcKC pUOA== X-Gm-Message-State: ANhLgQ1oa02/zcO+1ZhJUA33Cl3Hah6yUQZfY3AWrlk9m/FhmK2okj0+ n32ByNJXwNB8UxcHLnJ/x+8oaDGHncV7dp/B+no= X-Google-Smtp-Source: ADFU+vunbg/mabJAoT4ECckSo9VlWXEI7Tk7MCM2NEIzGX//dTyumqWFksHG62C18XwPsVHwF/wqELk+aHHpZR6UYqI= X-Received: by 2002:a17:906:b888:: with SMTP id hb8mr686525ejb.166.1584574997823; Wed, 18 Mar 2020 16:43:17 -0700 (PDT) MIME-Version: 1.0 References: <1580249770-1088-1-git-send-email-jcrouse@codeaurora.org> <1580249770-1088-3-git-send-email-jcrouse@codeaurora.org> <20200318224840.GA10796@willie-the-truck> In-Reply-To: <20200318224840.GA10796@willie-the-truck> From: Rob Clark Date: Wed, 18 Mar 2020 16:43:07 -0700 Message-ID: Subject: Re: [PATCH v1 2/6] arm/smmu: Add auxiliary domain support for arm-smmuv2 To: Will Deacon Cc: linux-arm-msm , Linux Kernel Mailing List , "list@263.net:IOMMU DRIVERS , Joerg Roedel , " , Robin Murphy , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Wed, Mar 18, 2020 at 3:48 PM Will Deacon wrote: > > On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > > Support auxiliary domains for arm-smmu-v2 to initialize and support > > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > > hardware doesn't have any built in support for switching the pagetable > > base it is left as an exercise to the caller to actually use the pagetable. > > > > Aux domains are supported if split pagetable (TTBR1) support has been > > enabled on the master domain. Each auxiliary domain will reuse the > > configuration of the master domain. By default the a domain with TTBR1 > > support will have the TTBR0 region disabled so the first attached aux > > domain will enable the TTBR0 region in the hardware and conversely the > > last domain to be detached will disable TTBR0 translations. All subsequent > > auxiliary domains create a pagetable but not touch the hardware. > > > > The leaf driver will be able to query the physical address of the > > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > > address with whatever means it has to switch the pagetable base. > > > > Following is a pseudo code example of how a domain can be created > > > > /* Check to see if aux domains are supported */ > > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > > iommu = iommu_domain_alloc(...); > > > > if (iommu_aux_attach_device(domain, dev)) > > return FAIL; > > > > /* Save the base address of the pagetable for use by the driver > > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > > } > > I'm not really understanding what the pagetable base gets used for here and, > to be honest with you, the whole thing feels like a huge layering violation > with the way things are structured today. Why doesn't the caller just > interface with io-pgtable directly? > > Finally, if we need to support context-switching TTBR0 for a live domain > then that code really needs to live inside the SMMU driver because the > ASID and TLB management necessary to do that safely doesn't belong anywhere > else. Hi Will, We do in fact need live domain switching, that is really the whole point. The GPU CP (command processor/parser) is directly updating TTBR0 and triggering TLB flush, asynchronously from the CPU. And I think the answer about ASID is easy (on current hw).. it must be zero[*]. BR, -R [*] my rough theory/plan there, and to solve the issue with drm/msm getting dma-iommu ops when it really would rather not (since blacklisting idea wasn't popular and I couldn't figure out a way to deal with case where device gets attached before driver shows up) is to invent some API that drm/msm can call to unhook the dma-iommu ops and detatch the DMA domain. Hopefully that at least gets us closer to the point where, when drm/msm attaches it's UNMANAGED domain, we get cbidx/asid zero. _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.7 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C748C4332E for ; Wed, 18 Mar 2020 23:43:24 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3B5B920768 for ; Wed, 18 Mar 2020 23:43:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="lkQaj4pZ"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i3FPub1D" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B5B920768 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w69cZwQOHU/g8IHt43YSYSFzlKCr0a1i6yBUJa83fgw=; b=lkQaj4pZOhsePa +d8jOGPvmARqa3/vmo4WHMLZ7+8+BJLzAXMlgDg7txT2EsqWY5B65iGPM/5uxi4Fsy3d3fx+v9Sml GKx/VVLb7zvvowCRvmDk4UwvoQP5cqfXwSrM2rzOmAqwvEVm6Kw+0I6/8+JrCcTs483y0Bd9EAfD+ PHn8pYFEK/HCqpeoQRrSKelV+y0oE5+rxuhDuAlh180CN5F3SFbmdMa5ifvktMW27T4UoGeCGKsQn 60eKWawl+JvLaHRpFEjlmBVRVdlUVm79fHh9+7pBieTiVXrZrjJuAVz+Sd9wezUTMXDA+qnxYvkW8 /WW/mCgpfypfEZBkcogg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jEiLK-0006NF-TP; Wed, 18 Mar 2020 23:43:22 +0000 Received: from mail-ed1-x542.google.com ([2a00:1450:4864:20::542]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jEiLI-0006Mv-Er for linux-arm-kernel@lists.infradead.org; Wed, 18 Mar 2020 23:43:21 +0000 Received: by mail-ed1-x542.google.com with SMTP id a20so376634edj.2 for ; Wed, 18 Mar 2020 16:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=T6rxV8z4faDRzhqK467Xv2a7dOy/TE520C3YV3WjV8c=; b=i3FPub1DIHkvCD+yi/2Y0pX69MrTIvO1RmbzVf6wk00DV0ApmY13YfnnFpks/NVTLs G1QplMy/+0w+GsFrwD1fV1STvoKneEr8NQlSW0vIhii6MAK+xDn8ddbmfxyIfh+bWkPE NEeRdRZ2QE6MO1YpPwSPzXvQKexEK5sr8Dox9C9jyfSyvqP6X6trr48Ms86RoFLn9dAG 92zK1vRSIdLYDibQabEa5foOn7Ajn4QGZfbFS6XCk1yr6R/d+WaM4jqJ31uE4ljyQmMb YiomgNSyyErYriEW6Abwe6sruyYqTy62RHKzpwUnN7Vinym7R/FBnsL8LzowJAa3l8q2 3Z7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=T6rxV8z4faDRzhqK467Xv2a7dOy/TE520C3YV3WjV8c=; b=aWrgGURHpo1LgQVz2YojbIsj3sx0UVXLuw8RxpGBbr2Aeepe1X0jKJmRiGqU6ZIA/d g5Zv4tbJ+J09unR5Atm5F/ksGQcqYdenq20uaL2vwmQiVMLwInVSAmYBKHYtDaVjUceI lqcY49XSN/E3T4Qh0iSfqRi3qmeJktAGjvtpS6jbJPwwyGhcauF2Hytzte4jzhA4tTpX j9d4KSJlLOVRCi8J2oOYWaBQNGmlB0Z+VQJaNQz4Fv5MG5vp5Gcjt3Xon/cH1TaN/Bfp sFE2OH4Na4gLLDh3MlTiu+kX/XI7EVapybZX+w75Wtn3+Zx3M0sGYxc8sq2/+BLlzBht vogw== X-Gm-Message-State: ANhLgQ0VJzynrVgbwv5+l5e/9YNKhB8QJlNQx+RabKpWdUEF8ZkhzPex CMjzrwlOk1zGGO1CBe67EQWdEN9L0rTr33cJwmw= X-Google-Smtp-Source: ADFU+vunbg/mabJAoT4ECckSo9VlWXEI7Tk7MCM2NEIzGX//dTyumqWFksHG62C18XwPsVHwF/wqELk+aHHpZR6UYqI= X-Received: by 2002:a17:906:b888:: with SMTP id hb8mr686525ejb.166.1584574997823; Wed, 18 Mar 2020 16:43:17 -0700 (PDT) MIME-Version: 1.0 References: <1580249770-1088-1-git-send-email-jcrouse@codeaurora.org> <1580249770-1088-3-git-send-email-jcrouse@codeaurora.org> <20200318224840.GA10796@willie-the-truck> In-Reply-To: <20200318224840.GA10796@willie-the-truck> From: Rob Clark Date: Wed, 18 Mar 2020 16:43:07 -0700 Message-ID: Subject: Re: [PATCH v1 2/6] arm/smmu: Add auxiliary domain support for arm-smmuv2 To: Will Deacon X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200318_164320_523476_78127688 X-CRM114-Status: GOOD ( 23.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arm-msm , Linux Kernel Mailing List , "list@263.net:IOMMU DRIVERS , Joerg Roedel , " , Jordan Crouse , Robin Murphy , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Mar 18, 2020 at 3:48 PM Will Deacon wrote: > > On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > > Support auxiliary domains for arm-smmu-v2 to initialize and support > > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > > hardware doesn't have any built in support for switching the pagetable > > base it is left as an exercise to the caller to actually use the pagetable. > > > > Aux domains are supported if split pagetable (TTBR1) support has been > > enabled on the master domain. Each auxiliary domain will reuse the > > configuration of the master domain. By default the a domain with TTBR1 > > support will have the TTBR0 region disabled so the first attached aux > > domain will enable the TTBR0 region in the hardware and conversely the > > last domain to be detached will disable TTBR0 translations. All subsequent > > auxiliary domains create a pagetable but not touch the hardware. > > > > The leaf driver will be able to query the physical address of the > > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > > address with whatever means it has to switch the pagetable base. > > > > Following is a pseudo code example of how a domain can be created > > > > /* Check to see if aux domains are supported */ > > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > > iommu = iommu_domain_alloc(...); > > > > if (iommu_aux_attach_device(domain, dev)) > > return FAIL; > > > > /* Save the base address of the pagetable for use by the driver > > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > > } > > I'm not really understanding what the pagetable base gets used for here and, > to be honest with you, the whole thing feels like a huge layering violation > with the way things are structured today. Why doesn't the caller just > interface with io-pgtable directly? > > Finally, if we need to support context-switching TTBR0 for a live domain > then that code really needs to live inside the SMMU driver because the > ASID and TLB management necessary to do that safely doesn't belong anywhere > else. Hi Will, We do in fact need live domain switching, that is really the whole point. The GPU CP (command processor/parser) is directly updating TTBR0 and triggering TLB flush, asynchronously from the CPU. And I think the answer about ASID is easy (on current hw).. it must be zero[*]. BR, -R [*] my rough theory/plan there, and to solve the issue with drm/msm getting dma-iommu ops when it really would rather not (since blacklisting idea wasn't popular and I couldn't figure out a way to deal with case where device gets attached before driver shows up) is to invent some API that drm/msm can call to unhook the dma-iommu ops and detatch the DMA domain. Hopefully that at least gets us closer to the point where, when drm/msm attaches it's UNMANAGED domain, we get cbidx/asid zero. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel