From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50067C433B4 for ; Sun, 2 May 2021 03:58:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C68E611EE for ; Sun, 2 May 2021 03:58:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231685AbhEBD7N (ORCPT ); Sat, 1 May 2021 23:59:13 -0400 Received: from mail-bn7nam10on2074.outbound.protection.outlook.com ([40.107.92.74]:58176 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230409AbhEBD7N (ORCPT ); Sat, 1 May 2021 23:59:13 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=POi/Xecp3KdeLAfyWv7xtOu1MD6gYC6ifdTGACvkxhTwXn5bZIwpb5RIAJDjmCpLR5cpTOi3YN1e8POET+703tl1DG3Y9vWFg1H7HoNaUtNNf85UvRg/eNoSiMStf1CVZgVU7NW2Qr8qq0tLEtVRWqViLyI0zpTIRwsXEatlXzAGuAh7VnpcOCufP2q9E3ieQ1fhDw9MtYsR0BHHMZmSVbOWBvtBtzdb1H7zCgYwsX5oxJR1D9r3v3tTfqzshq6WJz9SN3XC8vFfOvwZoDf5p0mmaGu9HKmDpAbm0o8khTBq9L6KKiEoXrpEG3KfXkcg0nk5zD60M31rd1d/zjTohQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2967dfxJFy8E/ntaJND45cJzPttmocbX0JqH10oLB04=; b=Ld6ShBo/IOZrvbtdA4BXWjI0wL8X/B6nzUegrzUngRbu4LaHuSciCANH5AAUVr+aeAbn2GaTQfqP25goz0t+NPygaegJbNU11DqNbSAqtEL+4RATrL0x3KEt7IQ1z0PijQCTLbRkqRY/4+5lFDT8mLnydypPKhqLaufh4Q5bNDl8VcKJkivHGc3c+NG/xP853419q01D0I1QjzHt5UL1HyMIECX7I/tVSbgpwHAbiqFvDMIoD9wOKqEm9MWBWNtz+Teyfq/Utr88fapJrhP3c2hBFWl/Zbro73Tv+jj94jgCACruU7JZeYdDDSio15os8BWa2xKpMzoaOsJclwbHfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=lst.de smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2967dfxJFy8E/ntaJND45cJzPttmocbX0JqH10oLB04=; b=dUIIdTezdfX39mwoy/tKklpNsAGv+tjkktZ014z3l4Jb87nq5MzlR/lLcON03+7mLqrp0Smn+41Iju+plAiaklZ5vVCvHJRcHq2wBKCfd1cKs2WQettNAw4WFxub0xrVsVWGlMB6TeuVUcZqgPrw4CuWiyKUwrRjeM1KNdU7UxQtgompr+rPV7UfMWd91jAbNNyQW+CvYE6PCuFU9w/Xhy0PtLKlruK8HxPxUwnb20NP3UYAyivH/CVEvAQt+Au39+oYqpROUktaIS5QuoIP6KyTRReYYTwuyjfTs4FBA0wnGq1g60biH/fx7KJtYUeIGu/tOlNzTpPLyptVHJo83w== Received: from DM5PR20CA0004.namprd20.prod.outlook.com (2603:10b6:3:93::14) by MWHPR1201MB0224.namprd12.prod.outlook.com (2603:10b6:301:55::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.35; Sun, 2 May 2021 03:58:19 +0000 Received: from DM6NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:3:93:cafe::24) by DM5PR20CA0004.outlook.office365.com (2603:10b6:3:93::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.27 via Frontend Transport; Sun, 2 May 2021 03:58:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; lst.de; dkim=none (message not signed) header.d=none;lst.de; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT051.mail.protection.outlook.com (10.13.172.243) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4087.32 via Frontend Transport; Sun, 2 May 2021 03:58:18 +0000 Received: from [10.2.50.162] (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 2 May 2021 03:58:18 +0000 Subject: Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn() To: Logan Gunthorpe , , , , , , CC: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?UTF-8?Q?Christian_K=c3=b6nig?= , Don Dutile , Matthew Wilcox , Daniel Vetter , Jakowski Andrzej , Minturn Dave B , Jason Ekstrand , Dave Hansen , Xiong Jianxin , Bjorn Helgaas , Ira Weiny , Robin Murphy , Bjorn Helgaas References: <20210408170123.8788-1-logang@deltatee.com> <20210408170123.8788-2-logang@deltatee.com> From: John Hubbard Message-ID: Date: Sat, 1 May 2021 20:58:17 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <20210408170123.8788-2-logang@deltatee.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ea3d5f4f-029a-497c-1ef1-08d90d1e855f X-MS-TrafficTypeDiagnostic: MWHPR1201MB0224: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:345; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: V3gl9tz3AekPp69OLif8bbLHq69PtghIy29Y3Z9TfzJCgwzvHLJmc8Qald1boG5bTMcgdVLtPFFHE+Jw9DWKD+ejfij6TN+7ivqscApbLN7lK/2RPhsVxAcbl58xC7nITjLLySGUxsaPS8vygwmA+L2pgdWsbGsqv7ix+/0RputJ+dlhwyUCK6yENi7fCiZ6XQ+3OqHZrdEi3eLv9E1hjfgYzsTvuQ9pfDoykMHMIBGghryr07zEpyqTIpnLNmT3W6fzgxM29NHT5hSncbah0GVtpK+Br42ktHtahVdWgKVrDWFLQ179Xtu0NOYglBCFwYvOwY/E8tZIv0QruEobXFdp7uh/hQEGYgcewBNx1yGVdU9gf/nHNW8f7WPbNMXliPZ9yyG6nZqd8FudXnFtZSNbBSxF1nki1iI8nn1HrnfwF02sNvwAhSqvPHjH6otpMM0As3xzhsZQsGFcfntVTEn1MzrLSeVC5krlywgT4qcOx8NF6KUieUfMYyiAHONFwZkKoAloIzHGTm2aJhxCO4Q4bp/au4j8X9YQdgmr90cws5DiuqsD6rbawCk8ICtTdnc6dDr2YPl1pOWuhrFB+jFKGbV1zIDIF0q5qJe5Z7adm9pxjmQOYPIFvWkhjj3AAm9tSFIHSni1196rUTqqELHKEWha3qc8QzMjrP6k8uyVZspFIGidS2IKf/oX4Tus42cWeXShesxf7cVr7V9vZQ== X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(376002)(396003)(136003)(346002)(39860400002)(36840700001)(46966006)(7416002)(5660300002)(4326008)(2616005)(8676002)(8936002)(316002)(83380400001)(36906005)(336012)(478600001)(110136005)(36860700001)(426003)(356005)(86362001)(186003)(36756003)(26005)(16576012)(16526019)(70206006)(2906002)(53546011)(31696002)(7636003)(82310400003)(70586007)(31686004)(82740400003)(47076005)(54906003)(43740500002)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2021 03:58:18.6320 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ea3d5f4f-029a-497c-1ef1-08d90d1e855f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0224 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 4/8/21 10:01 AM, Logan Gunthorpe wrote: > In order to call upstream_bridge_distance_warn() from a dma_map function, > it must not sleep. The only reason it does sleep is to allocate the seqbuf > to print which devices are within the ACS path. > > Switch the kmalloc call to use a passed in gfp_mask and don't print that > message if the buffer fails to be allocated. > > Signed-off-by: Logan Gunthorpe > Acked-by: Bjorn Helgaas > --- > drivers/pci/p2pdma.c | 21 +++++++++++---------- > 1 file changed, 11 insertions(+), 10 deletions(-) > > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > index 196382630363..bd89437faf06 100644 > --- a/drivers/pci/p2pdma.c > +++ b/drivers/pci/p2pdma.c > @@ -267,7 +267,7 @@ static int pci_bridge_has_acs_redir(struct pci_dev *pdev) > > static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *pdev) > { > - if (!buf) > + if (!buf || !buf->buffer) This is not great, sort of from an overall design point of view, even though it makes the rest of the patch work. See below for other ideas, that will avoid the need for this sort of odd point fix. > return; > > seq_buf_printf(buf, "%s;", pci_name(pdev)); > @@ -495,25 +495,26 @@ upstream_bridge_distance(struct pci_dev *provider, struct pci_dev *client, > > static enum pci_p2pdma_map_type > upstream_bridge_distance_warn(struct pci_dev *provider, struct pci_dev *client, > - int *dist) > + int *dist, gfp_t gfp_mask) > { > struct seq_buf acs_list; > bool acs_redirects; > int ret; > > - seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); > - if (!acs_list.buffer) > - return -ENOMEM; Another odd thing: this used to check for memory failure and just give up, and now it doesn't. Yes, I realize that it all still works at the moment, but this is quirky and we shouldn't stop here. Instead, a cleaner approach would be to push the memory allocation slightly higher up the call stack, out to the pci_p2pdma_distance_many(). So pci_p2pdma_distance_many() should make the kmalloc() call, and fail out if it can't get a page for the seq_buf buffer. Then you don't have to do all this odd stuff. Furthermore, the call sites can then decide for themselves which GFP flags, GFP_ATOMIC or GFP_KERNEL or whatever they want for kmalloc(). A related thing: this whole exercise would go better if there were a preparatory patch or two that changed the return codes in this file to something less crazy. There are too many functions that can fail, but are treated as if they sort-of-mostly-would-never-fail, in the hopes of using the return value directly for counting and such. This is badly mistaken, and it leads developers to try to avoid returning -ENOMEM (which is what we need here). Really, these functions should all be doing "0 for success, -ERRNO for failure, and pass other values, including results, in the arg list". > + seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, gfp_mask), PAGE_SIZE); > > ret = upstream_bridge_distance(provider, client, dist, &acs_redirects, > &acs_list); > if (acs_redirects) { > pci_warn(client, "ACS redirect is set between the client and provider (%s)\n", > pci_name(provider)); > - /* Drop final semicolon */ > - acs_list.buffer[acs_list.len-1] = 0; > - pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > - acs_list.buffer); > + > + if (acs_list.buffer) { > + /* Drop final semicolon */ > + acs_list.buffer[acs_list.len - 1] = 0; > + pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > + acs_list.buffer); > + } > } > > if (ret == PCI_P2PDMA_MAP_NOT_SUPPORTED) { > @@ -566,7 +567,7 @@ int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, > > if (verbose) > ret = upstream_bridge_distance_warn(provider, > - pci_client, &distance); > + pci_client, &distance, GFP_KERNEL); > else > ret = upstream_bridge_distance(provider, pci_client, > &distance, NULL, NULL); > thanks, -- John Hubbard NVIDIA From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35E7BC433ED for ; Sun, 2 May 2021 03:59:05 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5AE58613F0 for ; Sun, 2 May 2021 03:59:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5AE58613F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:CC:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=37qMUo0x0VgugrDcCVgPnxkSBoZCJayUdBpIQDdcWPU=; b=P2THWJbEzgBka2KD4Sf3GgLu0 aeOnczDtsZ6MA282UWQ16DsP8p6xAFWp5vUC2LCSltIIGPpPKa4i5wt6qkXHc+oSPYwyvGmck5XrX Hwg1FbXg/MdLJ7HJN02y0AcxEvWSjgPDUQzC4uMWyjxxO0DE7SyNVM6U7G9JdFFzRG9ZYBRwW4MXe 9mbTPypZNAfIq9IrldKE/1z8fHarE+COaUjyl981mKqg1gDxcvcg/SF1g5SZFgSWcrgUfQrXGw64Y L/m76zN6Ri5aA3ZcmmO8HhPSwqriKkipgQUzgDvgbekr71dSdaBQxlps3iRe5Orua6VN8z3eoe10a SJJYV4JzQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ld3FV-00BAT9-QM; Sun, 02 May 2021 03:58:29 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ld3FS-00BASz-SW for linux-nvme@desiato.infradead.org; Sun, 02 May 2021 03:58:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:CC:To: Subject:Sender:Reply-To:Content-ID:Content-Description; bh=2967dfxJFy8E/ntaJND45cJzPttmocbX0JqH10oLB04=; b=kkMyeLCfSWwRp0GMQCW2dBUSmF 4SjgMkXcZ84XvPoKBf79cvrJY+Ss1uJcb+ZovX7Hcie/jxnbRAeL6UljemcHImg638GAxAT0rD4jL ECw21OEI3ooSRLqMJu2LHN0PNZnsvvJIHJkuj2QezmLU2bn5h4Tz1VoJgQcGemoInOOt+Y0E5MJJO vF/H6DQVvHNldC2sG1oME8SYNtfzyXmM99NGYxW1kV3u5Smp+8ba5jxz+tRwymcd27bgt5fnpYUYB d7BfEl707pvIOCWCCeplcPER57QV8l8IN55FO0GelLUappy4mFlEyNfb7P8O4HoqDrPma173W1XIo s3Ui98ww==; Received: from mail-bn7nam10on2063.outbound.protection.outlook.com ([40.107.92.63] helo=NAM10-BN7-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ld3FP-002F2Y-9J for linux-nvme@lists.infradead.org; Sun, 02 May 2021 03:58:25 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=POi/Xecp3KdeLAfyWv7xtOu1MD6gYC6ifdTGACvkxhTwXn5bZIwpb5RIAJDjmCpLR5cpTOi3YN1e8POET+703tl1DG3Y9vWFg1H7HoNaUtNNf85UvRg/eNoSiMStf1CVZgVU7NW2Qr8qq0tLEtVRWqViLyI0zpTIRwsXEatlXzAGuAh7VnpcOCufP2q9E3ieQ1fhDw9MtYsR0BHHMZmSVbOWBvtBtzdb1H7zCgYwsX5oxJR1D9r3v3tTfqzshq6WJz9SN3XC8vFfOvwZoDf5p0mmaGu9HKmDpAbm0o8khTBq9L6KKiEoXrpEG3KfXkcg0nk5zD60M31rd1d/zjTohQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2967dfxJFy8E/ntaJND45cJzPttmocbX0JqH10oLB04=; b=Ld6ShBo/IOZrvbtdA4BXWjI0wL8X/B6nzUegrzUngRbu4LaHuSciCANH5AAUVr+aeAbn2GaTQfqP25goz0t+NPygaegJbNU11DqNbSAqtEL+4RATrL0x3KEt7IQ1z0PijQCTLbRkqRY/4+5lFDT8mLnydypPKhqLaufh4Q5bNDl8VcKJkivHGc3c+NG/xP853419q01D0I1QjzHt5UL1HyMIECX7I/tVSbgpwHAbiqFvDMIoD9wOKqEm9MWBWNtz+Teyfq/Utr88fapJrhP3c2hBFWl/Zbro73Tv+jj94jgCACruU7JZeYdDDSio15os8BWa2xKpMzoaOsJclwbHfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=lst.de smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2967dfxJFy8E/ntaJND45cJzPttmocbX0JqH10oLB04=; b=dUIIdTezdfX39mwoy/tKklpNsAGv+tjkktZ014z3l4Jb87nq5MzlR/lLcON03+7mLqrp0Smn+41Iju+plAiaklZ5vVCvHJRcHq2wBKCfd1cKs2WQettNAw4WFxub0xrVsVWGlMB6TeuVUcZqgPrw4CuWiyKUwrRjeM1KNdU7UxQtgompr+rPV7UfMWd91jAbNNyQW+CvYE6PCuFU9w/Xhy0PtLKlruK8HxPxUwnb20NP3UYAyivH/CVEvAQt+Au39+oYqpROUktaIS5QuoIP6KyTRReYYTwuyjfTs4FBA0wnGq1g60biH/fx7KJtYUeIGu/tOlNzTpPLyptVHJo83w== Received: from DM5PR20CA0004.namprd20.prod.outlook.com (2603:10b6:3:93::14) by MWHPR1201MB0224.namprd12.prod.outlook.com (2603:10b6:301:55::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.35; Sun, 2 May 2021 03:58:19 +0000 Received: from DM6NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:3:93:cafe::24) by DM5PR20CA0004.outlook.office365.com (2603:10b6:3:93::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.27 via Frontend Transport; Sun, 2 May 2021 03:58:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; lst.de; dkim=none (message not signed) header.d=none;lst.de; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT051.mail.protection.outlook.com (10.13.172.243) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4087.32 via Frontend Transport; Sun, 2 May 2021 03:58:18 +0000 Received: from [10.2.50.162] (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 2 May 2021 03:58:18 +0000 Subject: Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn() To: Logan Gunthorpe , , , , , , CC: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?UTF-8?Q?Christian_K=c3=b6nig?= , Don Dutile , Matthew Wilcox , Daniel Vetter , Jakowski Andrzej , Minturn Dave B , Jason Ekstrand , Dave Hansen , Xiong Jianxin , Bjorn Helgaas , Ira Weiny , Robin Murphy , Bjorn Helgaas References: <20210408170123.8788-1-logang@deltatee.com> <20210408170123.8788-2-logang@deltatee.com> From: John Hubbard Message-ID: Date: Sat, 1 May 2021 20:58:17 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <20210408170123.8788-2-logang@deltatee.com> Content-Language: en-US X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ea3d5f4f-029a-497c-1ef1-08d90d1e855f X-MS-TrafficTypeDiagnostic: MWHPR1201MB0224: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:345; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: V3gl9tz3AekPp69OLif8bbLHq69PtghIy29Y3Z9TfzJCgwzvHLJmc8Qald1boG5bTMcgdVLtPFFHE+Jw9DWKD+ejfij6TN+7ivqscApbLN7lK/2RPhsVxAcbl58xC7nITjLLySGUxsaPS8vygwmA+L2pgdWsbGsqv7ix+/0RputJ+dlhwyUCK6yENi7fCiZ6XQ+3OqHZrdEi3eLv9E1hjfgYzsTvuQ9pfDoykMHMIBGghryr07zEpyqTIpnLNmT3W6fzgxM29NHT5hSncbah0GVtpK+Br42ktHtahVdWgKVrDWFLQ179Xtu0NOYglBCFwYvOwY/E8tZIv0QruEobXFdp7uh/hQEGYgcewBNx1yGVdU9gf/nHNW8f7WPbNMXliPZ9yyG6nZqd8FudXnFtZSNbBSxF1nki1iI8nn1HrnfwF02sNvwAhSqvPHjH6otpMM0As3xzhsZQsGFcfntVTEn1MzrLSeVC5krlywgT4qcOx8NF6KUieUfMYyiAHONFwZkKoAloIzHGTm2aJhxCO4Q4bp/au4j8X9YQdgmr90cws5DiuqsD6rbawCk8ICtTdnc6dDr2YPl1pOWuhrFB+jFKGbV1zIDIF0q5qJe5Z7adm9pxjmQOYPIFvWkhjj3AAm9tSFIHSni1196rUTqqELHKEWha3qc8QzMjrP6k8uyVZspFIGidS2IKf/oX4Tus42cWeXShesxf7cVr7V9vZQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(396003)(136003)(346002)(39860400002)(36840700001)(46966006)(7416002)(5660300002)(4326008)(2616005)(8676002)(8936002)(316002)(83380400001)(36906005)(336012)(478600001)(110136005)(36860700001)(426003)(356005)(86362001)(186003)(36756003)(26005)(16576012)(16526019)(70206006)(2906002)(53546011)(31696002)(7636003)(82310400003)(70586007)(31686004)(82740400003)(47076005)(54906003)(43740500002)(2101003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2021 03:58:18.6320 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ea3d5f4f-029a-497c-1ef1-08d90d1e855f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0224 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210501_205823_491737_3AA8717E X-CRM114-Status: GOOD ( 33.18 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org On 4/8/21 10:01 AM, Logan Gunthorpe wrote: > In order to call upstream_bridge_distance_warn() from a dma_map function, > it must not sleep. The only reason it does sleep is to allocate the seqbuf > to print which devices are within the ACS path. > > Switch the kmalloc call to use a passed in gfp_mask and don't print that > message if the buffer fails to be allocated. > > Signed-off-by: Logan Gunthorpe > Acked-by: Bjorn Helgaas > --- > drivers/pci/p2pdma.c | 21 +++++++++++---------- > 1 file changed, 11 insertions(+), 10 deletions(-) > > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > index 196382630363..bd89437faf06 100644 > --- a/drivers/pci/p2pdma.c > +++ b/drivers/pci/p2pdma.c > @@ -267,7 +267,7 @@ static int pci_bridge_has_acs_redir(struct pci_dev *pdev) > > static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *pdev) > { > - if (!buf) > + if (!buf || !buf->buffer) This is not great, sort of from an overall design point of view, even though it makes the rest of the patch work. See below for other ideas, that will avoid the need for this sort of odd point fix. > return; > > seq_buf_printf(buf, "%s;", pci_name(pdev)); > @@ -495,25 +495,26 @@ upstream_bridge_distance(struct pci_dev *provider, struct pci_dev *client, > > static enum pci_p2pdma_map_type > upstream_bridge_distance_warn(struct pci_dev *provider, struct pci_dev *client, > - int *dist) > + int *dist, gfp_t gfp_mask) > { > struct seq_buf acs_list; > bool acs_redirects; > int ret; > > - seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); > - if (!acs_list.buffer) > - return -ENOMEM; Another odd thing: this used to check for memory failure and just give up, and now it doesn't. Yes, I realize that it all still works at the moment, but this is quirky and we shouldn't stop here. Instead, a cleaner approach would be to push the memory allocation slightly higher up the call stack, out to the pci_p2pdma_distance_many(). So pci_p2pdma_distance_many() should make the kmalloc() call, and fail out if it can't get a page for the seq_buf buffer. Then you don't have to do all this odd stuff. Furthermore, the call sites can then decide for themselves which GFP flags, GFP_ATOMIC or GFP_KERNEL or whatever they want for kmalloc(). A related thing: this whole exercise would go better if there were a preparatory patch or two that changed the return codes in this file to something less crazy. There are too many functions that can fail, but are treated as if they sort-of-mostly-would-never-fail, in the hopes of using the return value directly for counting and such. This is badly mistaken, and it leads developers to try to avoid returning -ENOMEM (which is what we need here). Really, these functions should all be doing "0 for success, -ERRNO for failure, and pass other values, including results, in the arg list". > + seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, gfp_mask), PAGE_SIZE); > > ret = upstream_bridge_distance(provider, client, dist, &acs_redirects, > &acs_list); > if (acs_redirects) { > pci_warn(client, "ACS redirect is set between the client and provider (%s)\n", > pci_name(provider)); > - /* Drop final semicolon */ > - acs_list.buffer[acs_list.len-1] = 0; > - pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > - acs_list.buffer); > + > + if (acs_list.buffer) { > + /* Drop final semicolon */ > + acs_list.buffer[acs_list.len - 1] = 0; > + pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > + acs_list.buffer); > + } > } > > if (ret == PCI_P2PDMA_MAP_NOT_SUPPORTED) { > @@ -566,7 +567,7 @@ int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, > > if (verbose) > ret = upstream_bridge_distance_warn(provider, > - pci_client, &distance); > + pci_client, &distance, GFP_KERNEL); > else > ret = upstream_bridge_distance(provider, pci_client, > &distance, NULL, NULL); > thanks, -- John Hubbard NVIDIA _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19B4EC433ED for ; Sun, 2 May 2021 03:58:28 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A2FE361466 for ; Sun, 2 May 2021 03:58:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2FE361466 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 69E1560AE0; Sun, 2 May 2021 03:58:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SEAoJpA_Pdpv; Sun, 2 May 2021 03:58:26 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTP id 29E3460ADF; Sun, 2 May 2021 03:58:26 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0274DC000E; Sun, 2 May 2021 03:58:26 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0E63CC0001 for ; Sun, 2 May 2021 03:58:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id D4D76403A7 for ; Sun, 2 May 2021 03:58:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=nvidia.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gTu-Y9G2tsut for ; Sun, 2 May 2021 03:58:23 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2069.outbound.protection.outlook.com [40.107.92.69]) by smtp2.osuosl.org (Postfix) with ESMTPS id C976D40275 for ; Sun, 2 May 2021 03:58:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=POi/Xecp3KdeLAfyWv7xtOu1MD6gYC6ifdTGACvkxhTwXn5bZIwpb5RIAJDjmCpLR5cpTOi3YN1e8POET+703tl1DG3Y9vWFg1H7HoNaUtNNf85UvRg/eNoSiMStf1CVZgVU7NW2Qr8qq0tLEtVRWqViLyI0zpTIRwsXEatlXzAGuAh7VnpcOCufP2q9E3ieQ1fhDw9MtYsR0BHHMZmSVbOWBvtBtzdb1H7zCgYwsX5oxJR1D9r3v3tTfqzshq6WJz9SN3XC8vFfOvwZoDf5p0mmaGu9HKmDpAbm0o8khTBq9L6KKiEoXrpEG3KfXkcg0nk5zD60M31rd1d/zjTohQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2967dfxJFy8E/ntaJND45cJzPttmocbX0JqH10oLB04=; b=Ld6ShBo/IOZrvbtdA4BXWjI0wL8X/B6nzUegrzUngRbu4LaHuSciCANH5AAUVr+aeAbn2GaTQfqP25goz0t+NPygaegJbNU11DqNbSAqtEL+4RATrL0x3KEt7IQ1z0PijQCTLbRkqRY/4+5lFDT8mLnydypPKhqLaufh4Q5bNDl8VcKJkivHGc3c+NG/xP853419q01D0I1QjzHt5UL1HyMIECX7I/tVSbgpwHAbiqFvDMIoD9wOKqEm9MWBWNtz+Teyfq/Utr88fapJrhP3c2hBFWl/Zbro73Tv+jj94jgCACruU7JZeYdDDSio15os8BWa2xKpMzoaOsJclwbHfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=lst.de smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2967dfxJFy8E/ntaJND45cJzPttmocbX0JqH10oLB04=; b=dUIIdTezdfX39mwoy/tKklpNsAGv+tjkktZ014z3l4Jb87nq5MzlR/lLcON03+7mLqrp0Smn+41Iju+plAiaklZ5vVCvHJRcHq2wBKCfd1cKs2WQettNAw4WFxub0xrVsVWGlMB6TeuVUcZqgPrw4CuWiyKUwrRjeM1KNdU7UxQtgompr+rPV7UfMWd91jAbNNyQW+CvYE6PCuFU9w/Xhy0PtLKlruK8HxPxUwnb20NP3UYAyivH/CVEvAQt+Au39+oYqpROUktaIS5QuoIP6KyTRReYYTwuyjfTs4FBA0wnGq1g60biH/fx7KJtYUeIGu/tOlNzTpPLyptVHJo83w== Received: from DM5PR20CA0004.namprd20.prod.outlook.com (2603:10b6:3:93::14) by MWHPR1201MB0224.namprd12.prod.outlook.com (2603:10b6:301:55::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.35; Sun, 2 May 2021 03:58:19 +0000 Received: from DM6NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:3:93:cafe::24) by DM5PR20CA0004.outlook.office365.com (2603:10b6:3:93::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.27 via Frontend Transport; Sun, 2 May 2021 03:58:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; lst.de; dkim=none (message not signed) header.d=none;lst.de; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT051.mail.protection.outlook.com (10.13.172.243) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4087.32 via Frontend Transport; Sun, 2 May 2021 03:58:18 +0000 Received: from [10.2.50.162] (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 2 May 2021 03:58:18 +0000 Subject: Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn() To: Logan Gunthorpe , , , , , , References: <20210408170123.8788-1-logang@deltatee.com> <20210408170123.8788-2-logang@deltatee.com> From: John Hubbard Message-ID: Date: Sat, 1 May 2021 20:58:17 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <20210408170123.8788-2-logang@deltatee.com> Content-Language: en-US X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ea3d5f4f-029a-497c-1ef1-08d90d1e855f X-MS-TrafficTypeDiagnostic: MWHPR1201MB0224: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:345; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: V3gl9tz3AekPp69OLif8bbLHq69PtghIy29Y3Z9TfzJCgwzvHLJmc8Qald1boG5bTMcgdVLtPFFHE+Jw9DWKD+ejfij6TN+7ivqscApbLN7lK/2RPhsVxAcbl58xC7nITjLLySGUxsaPS8vygwmA+L2pgdWsbGsqv7ix+/0RputJ+dlhwyUCK6yENi7fCiZ6XQ+3OqHZrdEi3eLv9E1hjfgYzsTvuQ9pfDoykMHMIBGghryr07zEpyqTIpnLNmT3W6fzgxM29NHT5hSncbah0GVtpK+Br42ktHtahVdWgKVrDWFLQ179Xtu0NOYglBCFwYvOwY/E8tZIv0QruEobXFdp7uh/hQEGYgcewBNx1yGVdU9gf/nHNW8f7WPbNMXliPZ9yyG6nZqd8FudXnFtZSNbBSxF1nki1iI8nn1HrnfwF02sNvwAhSqvPHjH6otpMM0As3xzhsZQsGFcfntVTEn1MzrLSeVC5krlywgT4qcOx8NF6KUieUfMYyiAHONFwZkKoAloIzHGTm2aJhxCO4Q4bp/au4j8X9YQdgmr90cws5DiuqsD6rbawCk8ICtTdnc6dDr2YPl1pOWuhrFB+jFKGbV1zIDIF0q5qJe5Z7adm9pxjmQOYPIFvWkhjj3AAm9tSFIHSni1196rUTqqELHKEWha3qc8QzMjrP6k8uyVZspFIGidS2IKf/oX4Tus42cWeXShesxf7cVr7V9vZQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(396003)(136003)(346002)(39860400002)(36840700001)(46966006)(7416002)(5660300002)(4326008)(2616005)(8676002)(8936002)(316002)(83380400001)(36906005)(336012)(478600001)(110136005)(36860700001)(426003)(356005)(86362001)(186003)(36756003)(26005)(16576012)(16526019)(70206006)(2906002)(53546011)(31696002)(7636003)(82310400003)(70586007)(31686004)(82740400003)(47076005)(54906003)(43740500002)(2101003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2021 03:58:18.6320 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ea3d5f4f-029a-497c-1ef1-08d90d1e855f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0224 Cc: Minturn Dave B , Ira Weiny , Daniel Vetter , Dave Hansen , Robin Murphy , Matthew Wilcox , =?UTF-8?Q?Christian_K=c3=b6nig?= , Jason Gunthorpe , Jason Ekstrand , Bjorn Helgaas , Bjorn Helgaas , Dan Williams , Stephen Bates , Jakowski Andrzej , Christoph Hellwig , Xiong Jianxin X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On 4/8/21 10:01 AM, Logan Gunthorpe wrote: > In order to call upstream_bridge_distance_warn() from a dma_map function, > it must not sleep. The only reason it does sleep is to allocate the seqbuf > to print which devices are within the ACS path. > > Switch the kmalloc call to use a passed in gfp_mask and don't print that > message if the buffer fails to be allocated. > > Signed-off-by: Logan Gunthorpe > Acked-by: Bjorn Helgaas > --- > drivers/pci/p2pdma.c | 21 +++++++++++---------- > 1 file changed, 11 insertions(+), 10 deletions(-) > > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > index 196382630363..bd89437faf06 100644 > --- a/drivers/pci/p2pdma.c > +++ b/drivers/pci/p2pdma.c > @@ -267,7 +267,7 @@ static int pci_bridge_has_acs_redir(struct pci_dev *pdev) > > static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *pdev) > { > - if (!buf) > + if (!buf || !buf->buffer) This is not great, sort of from an overall design point of view, even though it makes the rest of the patch work. See below for other ideas, that will avoid the need for this sort of odd point fix. > return; > > seq_buf_printf(buf, "%s;", pci_name(pdev)); > @@ -495,25 +495,26 @@ upstream_bridge_distance(struct pci_dev *provider, struct pci_dev *client, > > static enum pci_p2pdma_map_type > upstream_bridge_distance_warn(struct pci_dev *provider, struct pci_dev *client, > - int *dist) > + int *dist, gfp_t gfp_mask) > { > struct seq_buf acs_list; > bool acs_redirects; > int ret; > > - seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); > - if (!acs_list.buffer) > - return -ENOMEM; Another odd thing: this used to check for memory failure and just give up, and now it doesn't. Yes, I realize that it all still works at the moment, but this is quirky and we shouldn't stop here. Instead, a cleaner approach would be to push the memory allocation slightly higher up the call stack, out to the pci_p2pdma_distance_many(). So pci_p2pdma_distance_many() should make the kmalloc() call, and fail out if it can't get a page for the seq_buf buffer. Then you don't have to do all this odd stuff. Furthermore, the call sites can then decide for themselves which GFP flags, GFP_ATOMIC or GFP_KERNEL or whatever they want for kmalloc(). A related thing: this whole exercise would go better if there were a preparatory patch or two that changed the return codes in this file to something less crazy. There are too many functions that can fail, but are treated as if they sort-of-mostly-would-never-fail, in the hopes of using the return value directly for counting and such. This is badly mistaken, and it leads developers to try to avoid returning -ENOMEM (which is what we need here). Really, these functions should all be doing "0 for success, -ERRNO for failure, and pass other values, including results, in the arg list". > + seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, gfp_mask), PAGE_SIZE); > > ret = upstream_bridge_distance(provider, client, dist, &acs_redirects, > &acs_list); > if (acs_redirects) { > pci_warn(client, "ACS redirect is set between the client and provider (%s)\n", > pci_name(provider)); > - /* Drop final semicolon */ > - acs_list.buffer[acs_list.len-1] = 0; > - pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > - acs_list.buffer); > + > + if (acs_list.buffer) { > + /* Drop final semicolon */ > + acs_list.buffer[acs_list.len - 1] = 0; > + pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > + acs_list.buffer); > + } > } > > if (ret == PCI_P2PDMA_MAP_NOT_SUPPORTED) { > @@ -566,7 +567,7 @@ int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, > > if (verbose) > ret = upstream_bridge_distance_warn(provider, > - pci_client, &distance); > + pci_client, &distance, GFP_KERNEL); > else > ret = upstream_bridge_distance(provider, pci_client, > &distance, NULL, NULL); > thanks, -- John Hubbard NVIDIA _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu