From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 393A5C433E3 for ; Fri, 24 Jul 2020 11:07:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E27E2067D for ; Fri, 24 Jul 2020 11:07:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="BiTqCQ6x" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726754AbgGXLHa (ORCPT ); Fri, 24 Jul 2020 07:07:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726114AbgGXLHa (ORCPT ); Fri, 24 Jul 2020 07:07:30 -0400 Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F221C0619D3 for ; Fri, 24 Jul 2020 04:07:30 -0700 (PDT) Received: by mail-wr1-x441.google.com with SMTP id r12so7896792wrj.13 for ; Fri, 24 Jul 2020 04:07:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=EdQT63Ax+A3aODmg5U77uJh2icVxckFwj6Fw6w57Kic=; b=BiTqCQ6xZwjFwNXUjnfNOQmwwbGF1zO/bDH/fbt/YsdrblLH+TYtPD8oR2abGh2AGY c4KVUJWL0dLVAKBfwxEjla2NwHMKtsIeHSglXl2ti3i5MiAD5W3Agr2WayGqZWON7EZe WsS2ZgvcgFWo4s9AzQYU2z2iFbG1DPfpW1T3G5dxoN7rPmyCB1uIh6/z9u3rvo0ZAt6I LuUyoizw5wj/HRRE0juJfJgcOQYucim9Qdv0t3mHx/seCoWqKQSexSJkkri/uq469nj+ KYQ7vNsS3wWE14fFfWpRQDxtZNSuhGwgTUHVj3Ttdo5pBgyEK53y5p7JXEIz1mEBl97U WqMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=EdQT63Ax+A3aODmg5U77uJh2icVxckFwj6Fw6w57Kic=; b=XxD/lvsRz4EzYU6Gx5O2CeTzbZNHbF/gG4jVd22WlXV7unjHsI4oJkdhvU/aa8kNa/ yHN4IGjXw99MEUP2LTEd/ub18h3WcMnAnHvxoga80Fx0lLGIPCl/nAm5jFHNqIIIKVCY CS0DD7lKMEWQX8V0KYfsaPhmv/H8VPwXlJvS+ZhQ5vEqZ3G7n6WK4Frnaby4+iFbhULp ehsgQ0lodbRIKhlmGoCrV8+kfEipvMUCoU7/aqk/4CYomoslNIOt26qEXGN1btv46LHn ch7pP3cZKRgiC9qXOt6wzmIjkqh3UcZ2WMAITO8999XxL5Go8LyWSDljeTlcagKC2S56 1QGw== X-Gm-Message-State: AOAM530LRzCIIeZBb6SX5tNW06ROUXcB/sz9LUjg3qD7WKJzKezXCn0/ Exjn92XX0ZIvlrkoyMpLuHuX0nH1FPcl7aUR25/EiA== X-Google-Smtp-Source: ABdhPJxNaOA56Km6atDzUVWw9zPvlssdJZaf1ujzptAwAuQ3K/AVgnDaKbLddCq5PjILSV0SwQLvRTdywfHIhRtq6MI= X-Received: by 2002:adf:f44b:: with SMTP id f11mr8503038wrp.114.1595588848909; Fri, 24 Jul 2020 04:07:28 -0700 (PDT) MIME-Version: 1.0 References: <20200708164936.9340-1-nsaenzjulienne@suse.de> <550b30a86c0785049d24c945e2c6628d491cee3a.camel@suse.de> <011994f8a717a00dcd9ed7682a1ddeb421c2c43f.camel@suse.de> <01831596e4a2a6c9c066138b23bd30435f8e5569.camel@suse.de> <6db722947546221ed99d3f473f78e1a6de65d7d6.camel@suse.de> <0dc1e922bf87fa73790e7471b3974528dd261486.camel@suse.de> In-Reply-To: <0dc1e922bf87fa73790e7471b3974528dd261486.camel@suse.de> From: Amit Pundir Date: Fri, 24 Jul 2020 16:36:52 +0530 Message-ID: Subject: Re: [PATCH] dma-pool: Do not allocate pool memory from CMA To: Nicolas Saenz Julienne Cc: Christoph Hellwig , Marek Szyprowski , Robin Murphy , David Rientjes , linux-rpi-kernel@lists.infradead.org, jeremy.linton@arm.com, iommu@lists.linux-foundation.org, lkml , John Stultz , Sumit Semwal Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 24 Jul 2020 at 15:06, Nicolas Saenz Julienne wrote: > > Hi Amit, > > On Thu, 2020-07-23 at 10:44 +0530, Amit Pundir wrote: > > Hi Nicolas, > > > > Sorry I got stuck on other things yesterday. > > No worries :) > > > On Tue, 21 Jul 2020 at 21:57, Nicolas Saenz Julienne > > [...] > > > > > > > Let's get a bigger hammer, I'm just looking for clues here. Can you > > > apply this and provide the dmesg output. > > > > > > diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c > > > index 6bc74a2d5127..2160676bf488 100644 > > > --- a/kernel/dma/pool.c > > > +++ b/kernel/dma/pool.c > > > @@ -268,6 +268,8 @@ void *dma_alloc_from_pool(struct device *dev, size_t size, > > > schedule_work(&atomic_pool_work); > > > } > > > > > > + dev_info(dev, "%s: size %lx, phys addr %llx, flags 0x%x\n", __func__, size, phys, flags); > > > + > > > return ptr; > > > } > > > > I never made it to dma_alloc_from_pool() call from > > dma_direct_alloc_pages(), dma_should_alloc_from_pool() returns False > > from gfpflags_allow_blocking() block. > > I'm a little sceptical about this. The only way you can get memory from these > pools is through dma_alloc_from_pool(), and given how changes in the pool > initialization affected the phone, it's pretty clear some amount of pool > allocation is happening. Maybe from here iommu_dma_alloc_remap()? I see two paths to dma_alloc_from_pool(), one from dma_direct_alloc_pages() which I mentioned above and 2nd one is thru iommu_dma_alloc(), but the flow doesn't reach up to dma_alloc_from_pool(), and always returns from iommu_dma_alloc_remap(). Regards, Amit Pundir > > Regards, > Nicolas > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B852BC433E4 for ; Fri, 24 Jul 2020 11:07:34 +0000 (UTC) Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 793262073E for ; Fri, 24 Jul 2020 11:07:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="BiTqCQ6x" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 793262073E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 485DB864EE; Fri, 24 Jul 2020 11:07:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 80-DF1ms7g5i; Fri, 24 Jul 2020 11:07:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 5A03A86311; Fri, 24 Jul 2020 11:07:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 27E70C004D; Fri, 24 Jul 2020 11:07:33 +0000 (UTC) Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id A6259C004C for ; Fri, 24 Jul 2020 11:07:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 8F23486311 for ; Fri, 24 Jul 2020 11:07:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w4cZ4-BMN5Ir for ; Fri, 24 Jul 2020 11:07:30 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 9572686302 for ; Fri, 24 Jul 2020 11:07:30 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id 88so7936875wrh.3 for ; Fri, 24 Jul 2020 04:07:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=EdQT63Ax+A3aODmg5U77uJh2icVxckFwj6Fw6w57Kic=; b=BiTqCQ6xZwjFwNXUjnfNOQmwwbGF1zO/bDH/fbt/YsdrblLH+TYtPD8oR2abGh2AGY c4KVUJWL0dLVAKBfwxEjla2NwHMKtsIeHSglXl2ti3i5MiAD5W3Agr2WayGqZWON7EZe WsS2ZgvcgFWo4s9AzQYU2z2iFbG1DPfpW1T3G5dxoN7rPmyCB1uIh6/z9u3rvo0ZAt6I LuUyoizw5wj/HRRE0juJfJgcOQYucim9Qdv0t3mHx/seCoWqKQSexSJkkri/uq469nj+ KYQ7vNsS3wWE14fFfWpRQDxtZNSuhGwgTUHVj3Ttdo5pBgyEK53y5p7JXEIz1mEBl97U WqMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=EdQT63Ax+A3aODmg5U77uJh2icVxckFwj6Fw6w57Kic=; b=AQL9X4SFXVY1v/CFDFURnsI/vHJG0CwVyNagK/t+0urSYV0HhHLS25PkoTv2hT/MrM vtPMat6P3Sz7BQ8ZJsuwG7e1AxW7jp92GxlX8CNQgYaUbflU93pewf1RXfbSIBTZelpj tiRdzA8exUG99JyWYo+RLiU20y/tIvEO2PtfqNwW82fTzz+HksrLuM90dv4s/UIjFDJf 882OwckXcGQW99hSfQruaSyz05NdTxOFKJtQZjCoOPCZ6ilEy0flKsG7omSDSdC21MdX Hnd0Opy8udblK2sDPkopTPRFHASLuG+6gCnVFXzNgU6CmVLFz0py9LWJlFhn5yrkjMaW RhVA== X-Gm-Message-State: AOAM531JPeXdFhUZZL8RYXcHqBLFyru+RTdhJcNGp3At6q+2A0G21j9i semwCGEDaRB1Ojz8pEjqgy3HV8C7ELY1/VHX09e7Pw== X-Google-Smtp-Source: ABdhPJxNaOA56Km6atDzUVWw9zPvlssdJZaf1ujzptAwAuQ3K/AVgnDaKbLddCq5PjILSV0SwQLvRTdywfHIhRtq6MI= X-Received: by 2002:adf:f44b:: with SMTP id f11mr8503038wrp.114.1595588848909; Fri, 24 Jul 2020 04:07:28 -0700 (PDT) MIME-Version: 1.0 References: <20200708164936.9340-1-nsaenzjulienne@suse.de> <550b30a86c0785049d24c945e2c6628d491cee3a.camel@suse.de> <011994f8a717a00dcd9ed7682a1ddeb421c2c43f.camel@suse.de> <01831596e4a2a6c9c066138b23bd30435f8e5569.camel@suse.de> <6db722947546221ed99d3f473f78e1a6de65d7d6.camel@suse.de> <0dc1e922bf87fa73790e7471b3974528dd261486.camel@suse.de> In-Reply-To: <0dc1e922bf87fa73790e7471b3974528dd261486.camel@suse.de> From: Amit Pundir Date: Fri, 24 Jul 2020 16:36:52 +0530 Message-ID: Subject: Re: [PATCH] dma-pool: Do not allocate pool memory from CMA To: Nicolas Saenz Julienne Cc: lkml , jeremy.linton@arm.com, Sumit Semwal , iommu@lists.linux-foundation.org, John Stultz , linux-rpi-kernel@lists.infradead.org, David Rientjes , Robin Murphy , Christoph Hellwig X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Fri, 24 Jul 2020 at 15:06, Nicolas Saenz Julienne wrote: > > Hi Amit, > > On Thu, 2020-07-23 at 10:44 +0530, Amit Pundir wrote: > > Hi Nicolas, > > > > Sorry I got stuck on other things yesterday. > > No worries :) > > > On Tue, 21 Jul 2020 at 21:57, Nicolas Saenz Julienne > > [...] > > > > > > > Let's get a bigger hammer, I'm just looking for clues here. Can you > > > apply this and provide the dmesg output. > > > > > > diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c > > > index 6bc74a2d5127..2160676bf488 100644 > > > --- a/kernel/dma/pool.c > > > +++ b/kernel/dma/pool.c > > > @@ -268,6 +268,8 @@ void *dma_alloc_from_pool(struct device *dev, size_t size, > > > schedule_work(&atomic_pool_work); > > > } > > > > > > + dev_info(dev, "%s: size %lx, phys addr %llx, flags 0x%x\n", __func__, size, phys, flags); > > > + > > > return ptr; > > > } > > > > I never made it to dma_alloc_from_pool() call from > > dma_direct_alloc_pages(), dma_should_alloc_from_pool() returns False > > from gfpflags_allow_blocking() block. > > I'm a little sceptical about this. The only way you can get memory from these > pools is through dma_alloc_from_pool(), and given how changes in the pool > initialization affected the phone, it's pretty clear some amount of pool > allocation is happening. Maybe from here iommu_dma_alloc_remap()? I see two paths to dma_alloc_from_pool(), one from dma_direct_alloc_pages() which I mentioned above and 2nd one is thru iommu_dma_alloc(), but the flow doesn't reach up to dma_alloc_from_pool(), and always returns from iommu_dma_alloc_remap(). Regards, Amit Pundir > > Regards, > Nicolas > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu