From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D41F6C43461 for ; Thu, 3 Sep 2020 15:54:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB09F2072A for ; Thu, 3 Sep 2020 15:54:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1599148483; bh=L6Pwk1G4hd01vHxA+aaPHpUX0cbjzQBKCHd1njAkuCg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=ZGbNEOKKUZg0b7j8GRPATvsMPkp74ZhiKZ/QxGLWPmVJtuYOf836XXaAD4sOxBezQ tngXnBe7GIZxTrUDHwP/B0enKaf5EN7e4ANzv9yG7KIZcC3CQLYJ2Qfr0gDiQwQ12B KEeuMORqSjTfERw2nSnNqOcoiUlKyA30OgoMFmXY= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728513AbgICPyk (ORCPT ); Thu, 3 Sep 2020 11:54:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:55444 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728483AbgICPyj (ORCPT ); Thu, 3 Sep 2020 11:54:39 -0400 Received: from localhost (unknown [213.57.247.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5597D2072A; Thu, 3 Sep 2020 15:54:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1599148479; bh=L6Pwk1G4hd01vHxA+aaPHpUX0cbjzQBKCHd1njAkuCg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=v2tYWLMZjyeyfnNxcY0rvw0WoOSRDME9rctiBY5yhaXo5PqN+4nvpnDGdFBFriOCs /LVuKJYNvGvj/ipPd7uLMMwfqEfDaRLU7K2ZNwwb5QPJYjSpUzPlaQNYcTCFrwmmK/ 891gHvmUlvGKxTnHqvK7gpjR0/Nrlj79fz2BPDRA= From: Leon Romanovsky To: Christoph Hellwig , Doug Ledford , Jason Gunthorpe Cc: Maor Gottlieb , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Subject: [PATCH rdma-next 1/4] lib/scatterlist: Refactor sg_alloc_table_from_pages Date: Thu, 3 Sep 2020 18:54:34 +0300 Message-Id: <20200903155434.1153934-1-leon@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903121853.1145976-1-leon@kernel.org> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Maor Gottlieb Currently, sg_alloc_table_from_pages doesn't support dynamic chaining of SG entries. Therefore it requires from user to allocate all the pages in advance and hold them in a large buffer. Such a buffer consumes a lot of temporary memory in HPC systems which do a very large memory registration. The next patches introduce API for dynamically allocation from pages and it requires us to do the following: * Extract the code to alloc_from_pages_common. * Change the build of the table to iterate on the chunks and not on the SGEs. It will allow dynamic allocation of more SGEs. Since sg_alloc_table_from_pages allocate exactly the number of chunks, therefore chunks are equal to the number of SG entries. Signed-off-by: Maor Gottlieb Signed-off-by: Leon Romanovsky --- lib/scatterlist.c | 75 ++++++++++++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 30 deletions(-) diff --git a/lib/scatterlist.c b/lib/scatterlist.c index 5d63a8857f36..292e785d21ee 100644 --- a/lib/scatterlist.c +++ b/lib/scatterlist.c @@ -365,38 +365,18 @@ int sg_alloc_table(struct sg_table *table, unsigned int nents, gfp_t gfp_mask) } EXPORT_SYMBOL(sg_alloc_table); -/** - * __sg_alloc_table_from_pages - Allocate and initialize an sg table from - * an array of pages - * @sgt: The sg table header to use - * @pages: Pointer to an array of page pointers - * @n_pages: Number of pages in the pages array - * @offset: Offset from start of the first page to the start of a buffer - * @size: Number of valid bytes in the buffer (after offset) - * @max_segment: Maximum size of a scatterlist node in bytes (page aligned) - * @gfp_mask: GFP allocation mask - * - * Description: - * Allocate and initialize an sg table from a list of pages. Contiguous - * ranges of the pages are squashed into a single scatterlist node up to the - * maximum size specified in @max_segment. An user may provide an offset at a - * start and a size of valid data in a buffer specified by the page array. - * The returned sg table is released by sg_free_table. - * - * Returns: - * 0 on success, negative error on failure - */ -int __sg_alloc_table_from_pages(struct sg_table *sgt, struct page **pages, - unsigned int n_pages, unsigned int offset, - unsigned long size, unsigned int max_segment, - gfp_t gfp_mask) +static struct scatterlist * +alloc_from_pages_common(struct sg_table *sgt, struct page **pages, + unsigned int n_pages, unsigned int offset, + unsigned long size, unsigned int max_segment, + gfp_t gfp_mask) { unsigned int chunks, cur_page, seg_len, i; + struct scatterlist *prv, *s = NULL; int ret; - struct scatterlist *s; if (WARN_ON(!max_segment || offset_in_page(max_segment))) - return -EINVAL; + return ERR_PTR(-EINVAL); /* compute number of contiguous chunks */ chunks = 1; @@ -412,11 +392,12 @@ int __sg_alloc_table_from_pages(struct sg_table *sgt, struct page **pages, ret = sg_alloc_table(sgt, chunks, gfp_mask); if (unlikely(ret)) - return ret; + return ERR_PTR(ret); /* merging chunks and putting them into the scatterlist */ cur_page = 0; - for_each_sg(sgt->sgl, s, sgt->orig_nents, i) { + s = sgt->sgl; + for (i = 0; i < chunks; i++) { unsigned int j, chunk_size; /* look for the end of the current chunk */ @@ -435,9 +416,43 @@ int __sg_alloc_table_from_pages(struct sg_table *sgt, struct page **pages, size -= chunk_size; offset = 0; cur_page = j; + prv = s; + s = sg_next(s); } + return prv; +} - return 0; +/** + * __sg_alloc_table_from_pages - Allocate and initialize an sg table from + * an array of pages + * @sgt: The sg table header to use + * @pages: Pointer to an array of page pointers + * @n_pages: Number of pages in the pages array + * @offset: Offset from start of the first page to the start of a buffer + * @size: Number of valid bytes in the buffer (after offset) + * @max_segment: Maximum size of a scatterlist node in bytes (page aligned) + * @gfp_mask: GFP allocation mask + * + * Description: + * Allocate and initialize an sg table from a list of pages. Contiguous + * ranges of the pages are squashed into a single scatterlist node up to the + * maximum size specified in @max_segment. A user may provide an offset at a + * start and a size of valid data in a buffer specified by the page array. + * The returned sg table is released by sg_free_table. + * + * Returns: + * 0 on success, negative error on failure + */ +int __sg_alloc_table_from_pages(struct sg_table *sgt, struct page **pages, + unsigned int n_pages, unsigned int offset, + unsigned long size, unsigned int max_segment, + gfp_t gfp_mask) +{ + struct scatterlist *sg; + + sg = alloc_from_pages_common(sgt, pages, n_pages, offset, size, + max_segment, gfp_mask); + return PTR_ERR_OR_ZERO(sg); } EXPORT_SYMBOL(__sg_alloc_table_from_pages); -- 2.26.2