From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56330C433E0 for ; Mon, 1 Feb 2021 16:19:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FC2164EBD for ; Mon, 1 Feb 2021 16:19:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230298AbhBAQSy (ORCPT ); Mon, 1 Feb 2021 11:18:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229793AbhBAQSx (ORCPT ); Mon, 1 Feb 2021 11:18:53 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A807DC061573; Mon, 1 Feb 2021 08:18:11 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id v15so17250138wrx.4; Mon, 01 Feb 2021 08:18:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=gS7cvhd9tIsWsQWamd844erzMzEv+0azffG4y1wnD7p6yHLCPnwK1aG4LbChngEOSC 8/weiexIBXRCHk53nWXfv/Iyj8jDwsjtuplhe5CGty81EU/CxmmwQ9qj8Ad2xBUhkyoa BPLiuGV/g9R2Y7wGLkD2eEL7eY3SBOT0jaoYns5AV+pkP8mD83TDp1eXF5osBlCtAEfx Af01yVytJ4pAuQAfCOc/FCxZosdmRwhn4p6PIWo/9BtvnILStCOvKMcOVQbspyJmReZk zUXVtvGTZgeNd/BAfuhrvN9U1c7h8QzABedZOgYceTPH3Ot98yLe651CV1PaIFqQSYeN V5nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=if57/V1hfnwh0sR1VK7VXYvPYPidUoaVQ6tN5896Ssh/q7H0yjDFa+4SVlTfwGaWW5 QaqP7cNQQ42glaZB1X494wZWtorw75gahYuULmAHE6PVG7rmjQV1FOEjvsH2HiFoQNTG ysU5umbpwnQN8JNAifEPO0q3nQ/Dnnadogu9Gvr2QLIfgphBWc7WlIZ1L/VQ0bbXFrII QGgO9QVCiuNkDqW6g57pBy6NaUCLYxGdln9iqq4Lkyxwvsadqu2dHTGy1MAHDDzJ/+3E c33Use8eNIi3Z2uH3Zc216j7C7bbfid2zowmw4IJHymyJ7p+LUrKTvp8TTQElKVbowtD mvgw== X-Gm-Message-State: AOAM532TP+VL7pTI9LO43pEBIf7x8Br3PAicKbBUSwG2yGXF9oV+tK5J bywY4ZFGqi4mQQ3q1Im3YlWUtFQYZiOpCjSTd3s= X-Google-Smtp-Source: ABdhPJx8q9wwVZBia3ylOOk6tIuKE9b+ySiZiBv3V7ppswlXm6ZEtFt5l8YWZ+VVvUpdPPdbcnjj4tChykQ3ODiG83E= X-Received: by 2002:adf:dd07:: with SMTP id a7mr19293832wrm.83.1612196290263; Mon, 01 Feb 2021 08:18:10 -0800 (PST) MIME-Version: 1.0 References: <3f589e7de3f9fa93e84c83420c5270c546a0c368.1610372717.git.saiprakash.ranjan@codeaurora.org> <20210129090516.GB3998@willie-the-truck> <5d23fce629323bcda71594010824aad0@codeaurora.org> <20210201111556.GA7172@willie-the-truck> In-Reply-To: <20210201111556.GA7172@willie-the-truck> From: Rob Clark Date: Mon, 1 Feb 2021 08:20:44 -0800 Message-ID: Subject: Re: [PATCH 2/3] iommu/io-pgtable-arm: Add IOMMU_LLC page protection flag To: Will Deacon Cc: Sai Prakash Ranjan , Robin Murphy , Joerg Roedel , Jordan Crouse , Akhil P Oommen , "Isaac J. Manjarres" , "list@263.net:IOMMU DRIVERS , Joerg Roedel ," , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , Linux Kernel Mailing List , linux-arm-msm , freedreno , Kristian H Kristensen , Sean Paul , David Airlie , Daniel Vetter , dri-devel Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Mon, Feb 1, 2021 at 3:16 AM Will Deacon wrote: > > On Fri, Jan 29, 2021 at 03:12:59PM +0530, Sai Prakash Ranjan wrote: > > On 2021-01-29 14:35, Will Deacon wrote: > > > On Mon, Jan 11, 2021 at 07:45:04PM +0530, Sai Prakash Ranjan wrote: > > > > Add a new page protection flag IOMMU_LLC which can be used > > > > by non-coherent masters to set cacheable memory attributes > > > > for an outer level of cache called as last-level cache or > > > > system cache. Initial user of this page protection flag is > > > > the adreno gpu and then can later be used by other clients > > > > such as video where this can be used for per-buffer based > > > > mapping. > > > > > > > > Signed-off-by: Sai Prakash Ranjan > > > > --- > > > > drivers/iommu/io-pgtable-arm.c | 3 +++ > > > > include/linux/iommu.h | 6 ++++++ > > > > 2 files changed, 9 insertions(+) > > > > > > > > diff --git a/drivers/iommu/io-pgtable-arm.c > > > > b/drivers/iommu/io-pgtable-arm.c > > > > index 7439ee7fdcdb..ebe653ef601b 100644 > > > > --- a/drivers/iommu/io-pgtable-arm.c > > > > +++ b/drivers/iommu/io-pgtable-arm.c > > > > @@ -415,6 +415,9 @@ static arm_lpae_iopte > > > > arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, > > > > else if (prot & IOMMU_CACHE) > > > > pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE > > > > << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > + else if (prot & IOMMU_LLC) > > > > + pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE > > > > + << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > } > > > > > > > > if (prot & IOMMU_CACHE) > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > > > index ffaa389ea128..1f82057df531 100644 > > > > --- a/include/linux/iommu.h > > > > +++ b/include/linux/iommu.h > > > > @@ -31,6 +31,12 @@ > > > > * if the IOMMU page table format is equivalent. > > > > */ > > > > #define IOMMU_PRIV (1 << 5) > > > > +/* > > > > + * Non-coherent masters can use this page protection flag to set > > > > cacheable > > > > + * memory attributes for only a transparent outer level of cache, > > > > also known as > > > > + * the last-level or system cache. > > > > + */ > > > > +#define IOMMU_LLC (1 << 6) > > > > > > On reflection, I'm a bit worried about exposing this because I think it > > > will > > > introduce a mismatched virtual alias with the CPU (we don't even have a > > > MAIR > > > set up for this memory type). Now, we also have that issue for the PTW, > > > but > > > since we always use cache maintenance (i.e. the streaming API) for > > > publishing the page-tables to a non-coheren walker, it works out. > > > However, > > > if somebody expects IOMMU_LLC to be coherent with a DMA API coherent > > > allocation, then they're potentially in for a nasty surprise due to the > > > mismatched outer-cacheability attributes. > > > > > > > Can't we add the syscached memory type similar to what is done on android? > > Maybe. How does the GPU driver map these things on the CPU side? Currently we use writecombine mappings for everything, although there are some cases that we'd like to use cached (but have not merged patches that would give userspace a way to flush/invalidate) BR, -R From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36BE8C433E6 for ; Mon, 1 Feb 2021 16:18:16 +0000 (UTC) Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ADB0764E9B for ; Mon, 1 Feb 2021 16:18:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADB0764E9B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7972A8545F; Mon, 1 Feb 2021 16:18:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HdXBPm-L0cec; Mon, 1 Feb 2021 16:18:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id EC8458544C; Mon, 1 Feb 2021 16:18:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D94A0C0174; Mon, 1 Feb 2021 16:18:14 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 76038C013A for ; Mon, 1 Feb 2021 16:18:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 6311787028 for ; Mon, 1 Feb 2021 16:18:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aINksKJmVrWV for ; Mon, 1 Feb 2021 16:18:12 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by hemlock.osuosl.org (Postfix) with ESMTPS id DA17787029 for ; Mon, 1 Feb 2021 16:18:11 +0000 (UTC) Received: by mail-wr1-f43.google.com with SMTP id p15so17168524wrq.8 for ; Mon, 01 Feb 2021 08:18:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=gS7cvhd9tIsWsQWamd844erzMzEv+0azffG4y1wnD7p6yHLCPnwK1aG4LbChngEOSC 8/weiexIBXRCHk53nWXfv/Iyj8jDwsjtuplhe5CGty81EU/CxmmwQ9qj8Ad2xBUhkyoa BPLiuGV/g9R2Y7wGLkD2eEL7eY3SBOT0jaoYns5AV+pkP8mD83TDp1eXF5osBlCtAEfx Af01yVytJ4pAuQAfCOc/FCxZosdmRwhn4p6PIWo/9BtvnILStCOvKMcOVQbspyJmReZk zUXVtvGTZgeNd/BAfuhrvN9U1c7h8QzABedZOgYceTPH3Ot98yLe651CV1PaIFqQSYeN V5nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=mBlY9bBz09oj2G/Y5phJWR2PrP/ugP9a7fvbQN50njaiO54YeJYxAEURGXagTr1hsk gIObIXKOTMvjJwnwGGdAFkLDu82P7R9kXqGowDxmOIPEI7m1THKuHT09vUctgNrR0Fub yUNuut2BMvbGFrMDKRRbPmRZ2Ts9hwA5bi/dte8RgyfYLCU3qYjOssaCgE+flybHO0sZ VcC+GFc4oBRuUrbOo2MZDz8f5HK/d4RWYsiNo6Msk4e4K6hOPAPYRZEpuxX8FVMdw/SM bain54TQ4mWQIwDjrjbnJ10No1oF73/LduWvr8XEf/aIsd0WNz2XPBiCJMFYKm4LZ7rM p7fw== X-Gm-Message-State: AOAM533q2Uz3Vk3850JeZY4nk1ChhjjNNRzpcEjNHDGOP+haEh2GSNqj k6/Wfk+Xk7/nW+RSuBHtOkHx1RMHNgiD1dVdU6k= X-Google-Smtp-Source: ABdhPJx8q9wwVZBia3ylOOk6tIuKE9b+ySiZiBv3V7ppswlXm6ZEtFt5l8YWZ+VVvUpdPPdbcnjj4tChykQ3ODiG83E= X-Received: by 2002:adf:dd07:: with SMTP id a7mr19293832wrm.83.1612196290263; Mon, 01 Feb 2021 08:18:10 -0800 (PST) MIME-Version: 1.0 References: <3f589e7de3f9fa93e84c83420c5270c546a0c368.1610372717.git.saiprakash.ranjan@codeaurora.org> <20210129090516.GB3998@willie-the-truck> <5d23fce629323bcda71594010824aad0@codeaurora.org> <20210201111556.GA7172@willie-the-truck> In-Reply-To: <20210201111556.GA7172@willie-the-truck> From: Rob Clark Date: Mon, 1 Feb 2021 08:20:44 -0800 Message-ID: Subject: Re: [PATCH 2/3] iommu/io-pgtable-arm: Add IOMMU_LLC page protection flag To: Will Deacon Cc: "Isaac J. Manjarres" , freedreno , David Airlie , linux-arm-msm , Akhil P Oommen , "list@263.net:IOMMU DRIVERS , Joerg Roedel , " , Linux Kernel Mailing List , Sean Paul , Kristian H Kristensen , dri-devel , Daniel Vetter , Robin Murphy , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Mon, Feb 1, 2021 at 3:16 AM Will Deacon wrote: > > On Fri, Jan 29, 2021 at 03:12:59PM +0530, Sai Prakash Ranjan wrote: > > On 2021-01-29 14:35, Will Deacon wrote: > > > On Mon, Jan 11, 2021 at 07:45:04PM +0530, Sai Prakash Ranjan wrote: > > > > Add a new page protection flag IOMMU_LLC which can be used > > > > by non-coherent masters to set cacheable memory attributes > > > > for an outer level of cache called as last-level cache or > > > > system cache. Initial user of this page protection flag is > > > > the adreno gpu and then can later be used by other clients > > > > such as video where this can be used for per-buffer based > > > > mapping. > > > > > > > > Signed-off-by: Sai Prakash Ranjan > > > > --- > > > > drivers/iommu/io-pgtable-arm.c | 3 +++ > > > > include/linux/iommu.h | 6 ++++++ > > > > 2 files changed, 9 insertions(+) > > > > > > > > diff --git a/drivers/iommu/io-pgtable-arm.c > > > > b/drivers/iommu/io-pgtable-arm.c > > > > index 7439ee7fdcdb..ebe653ef601b 100644 > > > > --- a/drivers/iommu/io-pgtable-arm.c > > > > +++ b/drivers/iommu/io-pgtable-arm.c > > > > @@ -415,6 +415,9 @@ static arm_lpae_iopte > > > > arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, > > > > else if (prot & IOMMU_CACHE) > > > > pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE > > > > << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > + else if (prot & IOMMU_LLC) > > > > + pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE > > > > + << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > } > > > > > > > > if (prot & IOMMU_CACHE) > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > > > index ffaa389ea128..1f82057df531 100644 > > > > --- a/include/linux/iommu.h > > > > +++ b/include/linux/iommu.h > > > > @@ -31,6 +31,12 @@ > > > > * if the IOMMU page table format is equivalent. > > > > */ > > > > #define IOMMU_PRIV (1 << 5) > > > > +/* > > > > + * Non-coherent masters can use this page protection flag to set > > > > cacheable > > > > + * memory attributes for only a transparent outer level of cache, > > > > also known as > > > > + * the last-level or system cache. > > > > + */ > > > > +#define IOMMU_LLC (1 << 6) > > > > > > On reflection, I'm a bit worried about exposing this because I think it > > > will > > > introduce a mismatched virtual alias with the CPU (we don't even have a > > > MAIR > > > set up for this memory type). Now, we also have that issue for the PTW, > > > but > > > since we always use cache maintenance (i.e. the streaming API) for > > > publishing the page-tables to a non-coheren walker, it works out. > > > However, > > > if somebody expects IOMMU_LLC to be coherent with a DMA API coherent > > > allocation, then they're potentially in for a nasty surprise due to the > > > mismatched outer-cacheability attributes. > > > > > > > Can't we add the syscached memory type similar to what is done on android? > > Maybe. How does the GPU driver map these things on the CPU side? Currently we use writecombine mappings for everything, although there are some cases that we'd like to use cached (but have not merged patches that would give userspace a way to flush/invalidate) BR, -R _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE1C8C433DB for ; Mon, 1 Feb 2021 16:19:48 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 945ED64E9B for ; Mon, 1 Feb 2021 16:19:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 945ED64E9B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jBpuxKJI6NaPoe7Ggtnmfi+x1dxfR2T2UsYrS73GPP0=; b=EVfdLvTFSQmer7roRzuiNfI9t 4h4EdrPT+CT4YLc9F4VzsVUdxScgh3Quz2l0fU90yDFRiN6mapf4WxkQRc0VBjzH28roKQwf+6Glw lWx9npK/9tbbTk/QNtyBYzW27fuFacm4Tku+0XpRTPU4EF4yJaxWyNLfxOj6kJg6HK7GUns2Pcz4T ieptdxLbfsd9e6BMMXb5Ry7pA0M1LjZTl8/4H1ld9mszgrlsnTxW0Z2auG7hgbsNswXeyvL3rCPqT un3T9zcIgHhvqR4F8q+eX4EKFpEuT6lckMM3L+pYfbuAXoyLnfeYqE39JwP0sgwB35C6IQHzFASIJ T5wYJrzIw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6bu6-0008Uq-ET; Mon, 01 Feb 2021 16:18:18 +0000 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6bu1-0008T6-59 for linux-arm-kernel@lists.infradead.org; Mon, 01 Feb 2021 16:18:14 +0000 Received: by mail-wr1-x430.google.com with SMTP id m13so17221629wro.12 for ; Mon, 01 Feb 2021 08:18:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=gS7cvhd9tIsWsQWamd844erzMzEv+0azffG4y1wnD7p6yHLCPnwK1aG4LbChngEOSC 8/weiexIBXRCHk53nWXfv/Iyj8jDwsjtuplhe5CGty81EU/CxmmwQ9qj8Ad2xBUhkyoa BPLiuGV/g9R2Y7wGLkD2eEL7eY3SBOT0jaoYns5AV+pkP8mD83TDp1eXF5osBlCtAEfx Af01yVytJ4pAuQAfCOc/FCxZosdmRwhn4p6PIWo/9BtvnILStCOvKMcOVQbspyJmReZk zUXVtvGTZgeNd/BAfuhrvN9U1c7h8QzABedZOgYceTPH3Ot98yLe651CV1PaIFqQSYeN V5nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=YkROAjsA1N3lY3JHhnKnIK2wCgQ8dqT+zrQRKOCTnvaO2cG8pCpDGGCX2kiWW92hux qgAOFeRwwW+G5YZjcDjGOp0jS7rrCq+cBuEYHJhxghe4LiHhiYWUTWOeHhbu8XO2pkTr 02PSDsE8fCZGnhVmJk3BaSWXXte3UGxgMAjzEn7btAzYR2snnUrHa40qbkfh863fuDKy nOhhFuZcxie1LbsEqd5+Ayx/pv2IcUtbOs6QIBld2hP5ryN9InFheE6idvFhTUbDXDcO 091nzel8L7vJ3GTk8mcboiu5eVxlKQzSU81aRuL7tW1ogzsinc9jPR7kA2scoyhJlP7V iTtA== X-Gm-Message-State: AOAM530yyPdrM2Ql23q/Wzi5yj2dr0zr+gvCMWokWn9I00kdsGjPfcET rzPEZ42/FLyyRiF9QJxZlRK4EeQMe2mqiMeR8y8= X-Google-Smtp-Source: ABdhPJx8q9wwVZBia3ylOOk6tIuKE9b+ySiZiBv3V7ppswlXm6ZEtFt5l8YWZ+VVvUpdPPdbcnjj4tChykQ3ODiG83E= X-Received: by 2002:adf:dd07:: with SMTP id a7mr19293832wrm.83.1612196290263; Mon, 01 Feb 2021 08:18:10 -0800 (PST) MIME-Version: 1.0 References: <3f589e7de3f9fa93e84c83420c5270c546a0c368.1610372717.git.saiprakash.ranjan@codeaurora.org> <20210129090516.GB3998@willie-the-truck> <5d23fce629323bcda71594010824aad0@codeaurora.org> <20210201111556.GA7172@willie-the-truck> In-Reply-To: <20210201111556.GA7172@willie-the-truck> From: Rob Clark Date: Mon, 1 Feb 2021 08:20:44 -0800 Message-ID: Subject: Re: [PATCH 2/3] iommu/io-pgtable-arm: Add IOMMU_LLC page protection flag To: Will Deacon X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210201_111813_219177_19F17158 X-CRM114-Status: GOOD ( 34.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Isaac J. Manjarres" , freedreno , Sai Prakash Ranjan , David Airlie , linux-arm-msm , Joerg Roedel , Akhil P Oommen , "list@263.net:IOMMU DRIVERS , Joerg Roedel , " , Linux Kernel Mailing List , Sean Paul , Jordan Crouse , Kristian H Kristensen , dri-devel , Daniel Vetter , Robin Murphy , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Feb 1, 2021 at 3:16 AM Will Deacon wrote: > > On Fri, Jan 29, 2021 at 03:12:59PM +0530, Sai Prakash Ranjan wrote: > > On 2021-01-29 14:35, Will Deacon wrote: > > > On Mon, Jan 11, 2021 at 07:45:04PM +0530, Sai Prakash Ranjan wrote: > > > > Add a new page protection flag IOMMU_LLC which can be used > > > > by non-coherent masters to set cacheable memory attributes > > > > for an outer level of cache called as last-level cache or > > > > system cache. Initial user of this page protection flag is > > > > the adreno gpu and then can later be used by other clients > > > > such as video where this can be used for per-buffer based > > > > mapping. > > > > > > > > Signed-off-by: Sai Prakash Ranjan > > > > --- > > > > drivers/iommu/io-pgtable-arm.c | 3 +++ > > > > include/linux/iommu.h | 6 ++++++ > > > > 2 files changed, 9 insertions(+) > > > > > > > > diff --git a/drivers/iommu/io-pgtable-arm.c > > > > b/drivers/iommu/io-pgtable-arm.c > > > > index 7439ee7fdcdb..ebe653ef601b 100644 > > > > --- a/drivers/iommu/io-pgtable-arm.c > > > > +++ b/drivers/iommu/io-pgtable-arm.c > > > > @@ -415,6 +415,9 @@ static arm_lpae_iopte > > > > arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, > > > > else if (prot & IOMMU_CACHE) > > > > pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE > > > > << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > + else if (prot & IOMMU_LLC) > > > > + pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE > > > > + << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > } > > > > > > > > if (prot & IOMMU_CACHE) > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > > > index ffaa389ea128..1f82057df531 100644 > > > > --- a/include/linux/iommu.h > > > > +++ b/include/linux/iommu.h > > > > @@ -31,6 +31,12 @@ > > > > * if the IOMMU page table format is equivalent. > > > > */ > > > > #define IOMMU_PRIV (1 << 5) > > > > +/* > > > > + * Non-coherent masters can use this page protection flag to set > > > > cacheable > > > > + * memory attributes for only a transparent outer level of cache, > > > > also known as > > > > + * the last-level or system cache. > > > > + */ > > > > +#define IOMMU_LLC (1 << 6) > > > > > > On reflection, I'm a bit worried about exposing this because I think it > > > will > > > introduce a mismatched virtual alias with the CPU (we don't even have a > > > MAIR > > > set up for this memory type). Now, we also have that issue for the PTW, > > > but > > > since we always use cache maintenance (i.e. the streaming API) for > > > publishing the page-tables to a non-coheren walker, it works out. > > > However, > > > if somebody expects IOMMU_LLC to be coherent with a DMA API coherent > > > allocation, then they're potentially in for a nasty surprise due to the > > > mismatched outer-cacheability attributes. > > > > > > > Can't we add the syscached memory type similar to what is done on android? > > Maybe. How does the GPU driver map these things on the CPU side? Currently we use writecombine mappings for everything, although there are some cases that we'd like to use cached (but have not merged patches that would give userspace a way to flush/invalidate) BR, -R _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF4F0C433DB for ; Mon, 1 Feb 2021 16:18:13 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 97B7E64D9D for ; Mon, 1 Feb 2021 16:18:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 97B7E64D9D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0D7896E838; Mon, 1 Feb 2021 16:18:13 +0000 (UTC) Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by gabe.freedesktop.org (Postfix) with ESMTPS id B97F66E825; Mon, 1 Feb 2021 16:18:11 +0000 (UTC) Received: by mail-wr1-x436.google.com with SMTP id c12so17216475wrc.7; Mon, 01 Feb 2021 08:18:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=gS7cvhd9tIsWsQWamd844erzMzEv+0azffG4y1wnD7p6yHLCPnwK1aG4LbChngEOSC 8/weiexIBXRCHk53nWXfv/Iyj8jDwsjtuplhe5CGty81EU/CxmmwQ9qj8Ad2xBUhkyoa BPLiuGV/g9R2Y7wGLkD2eEL7eY3SBOT0jaoYns5AV+pkP8mD83TDp1eXF5osBlCtAEfx Af01yVytJ4pAuQAfCOc/FCxZosdmRwhn4p6PIWo/9BtvnILStCOvKMcOVQbspyJmReZk zUXVtvGTZgeNd/BAfuhrvN9U1c7h8QzABedZOgYceTPH3Ot98yLe651CV1PaIFqQSYeN V5nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tVx3P1GkJGLI1uKBNlxXY3RMDtB/IpyYjPT6P6zHw5s=; b=BuaIEGLLWzWFaVTA/oChTk1R6mjr49qXFRM3VvMPqFtel4KotW7PMzERs6wETfcq3q HVXpMi5jaF6FwfK6HX0Uzf0giSR9/yO0v3am9ERvTGE7d9ZuVtjNxtfMsU5uqjWIW8sP R9ptXT7r8OYlj6thgS5oqJjUukRW7c5oUwHTNZH/WZm/yCtg1IfZ6H8lxs2ERfX3caZc 8vRinZFEI/aj5Ja6FePuZpEGVWtcjnoRchJ3yFsNnlMoPUO236fmUrAKeEOcoM0x5578 xj82W0w1+B5chgHhUuagnCDsiB3h81fB21PtJT57fsZwmdgvCWmjFJU/ZN2AUb0Iy4XL +WJw== X-Gm-Message-State: AOAM5305ugXanqf+vCYBlU34niNSmqY9eSLsho45hvV1tOIjxRrVlvLI eOlHMgbnTYV/lMg3ul3aPVlZrub3uwkZyDPdDbQ= X-Google-Smtp-Source: ABdhPJx8q9wwVZBia3ylOOk6tIuKE9b+ySiZiBv3V7ppswlXm6ZEtFt5l8YWZ+VVvUpdPPdbcnjj4tChykQ3ODiG83E= X-Received: by 2002:adf:dd07:: with SMTP id a7mr19293832wrm.83.1612196290263; Mon, 01 Feb 2021 08:18:10 -0800 (PST) MIME-Version: 1.0 References: <3f589e7de3f9fa93e84c83420c5270c546a0c368.1610372717.git.saiprakash.ranjan@codeaurora.org> <20210129090516.GB3998@willie-the-truck> <5d23fce629323bcda71594010824aad0@codeaurora.org> <20210201111556.GA7172@willie-the-truck> In-Reply-To: <20210201111556.GA7172@willie-the-truck> From: Rob Clark Date: Mon, 1 Feb 2021 08:20:44 -0800 Message-ID: Subject: Re: [PATCH 2/3] iommu/io-pgtable-arm: Add IOMMU_LLC page protection flag To: Will Deacon X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Isaac J. Manjarres" , freedreno , Sai Prakash Ranjan , David Airlie , linux-arm-msm , Joerg Roedel , Akhil P Oommen , "list@263.net:IOMMU DRIVERS , Joerg Roedel , " , Linux Kernel Mailing List , Sean Paul , Kristian H Kristensen , dri-devel , Robin Murphy , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Mon, Feb 1, 2021 at 3:16 AM Will Deacon wrote: > > On Fri, Jan 29, 2021 at 03:12:59PM +0530, Sai Prakash Ranjan wrote: > > On 2021-01-29 14:35, Will Deacon wrote: > > > On Mon, Jan 11, 2021 at 07:45:04PM +0530, Sai Prakash Ranjan wrote: > > > > Add a new page protection flag IOMMU_LLC which can be used > > > > by non-coherent masters to set cacheable memory attributes > > > > for an outer level of cache called as last-level cache or > > > > system cache. Initial user of this page protection flag is > > > > the adreno gpu and then can later be used by other clients > > > > such as video where this can be used for per-buffer based > > > > mapping. > > > > > > > > Signed-off-by: Sai Prakash Ranjan > > > > --- > > > > drivers/iommu/io-pgtable-arm.c | 3 +++ > > > > include/linux/iommu.h | 6 ++++++ > > > > 2 files changed, 9 insertions(+) > > > > > > > > diff --git a/drivers/iommu/io-pgtable-arm.c > > > > b/drivers/iommu/io-pgtable-arm.c > > > > index 7439ee7fdcdb..ebe653ef601b 100644 > > > > --- a/drivers/iommu/io-pgtable-arm.c > > > > +++ b/drivers/iommu/io-pgtable-arm.c > > > > @@ -415,6 +415,9 @@ static arm_lpae_iopte > > > > arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, > > > > else if (prot & IOMMU_CACHE) > > > > pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE > > > > << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > + else if (prot & IOMMU_LLC) > > > > + pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE > > > > + << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > } > > > > > > > > if (prot & IOMMU_CACHE) > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > > > index ffaa389ea128..1f82057df531 100644 > > > > --- a/include/linux/iommu.h > > > > +++ b/include/linux/iommu.h > > > > @@ -31,6 +31,12 @@ > > > > * if the IOMMU page table format is equivalent. > > > > */ > > > > #define IOMMU_PRIV (1 << 5) > > > > +/* > > > > + * Non-coherent masters can use this page protection flag to set > > > > cacheable > > > > + * memory attributes for only a transparent outer level of cache, > > > > also known as > > > > + * the last-level or system cache. > > > > + */ > > > > +#define IOMMU_LLC (1 << 6) > > > > > > On reflection, I'm a bit worried about exposing this because I think it > > > will > > > introduce a mismatched virtual alias with the CPU (we don't even have a > > > MAIR > > > set up for this memory type). Now, we also have that issue for the PTW, > > > but > > > since we always use cache maintenance (i.e. the streaming API) for > > > publishing the page-tables to a non-coheren walker, it works out. > > > However, > > > if somebody expects IOMMU_LLC to be coherent with a DMA API coherent > > > allocation, then they're potentially in for a nasty surprise due to the > > > mismatched outer-cacheability attributes. > > > > > > > Can't we add the syscached memory type similar to what is done on android? > > Maybe. How does the GPU driver map these things on the CPU side? Currently we use writecombine mappings for everything, although there are some cases that we'd like to use cached (but have not merged patches that would give userspace a way to flush/invalidate) BR, -R _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel