From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25C94C47096 for ; Sun, 6 Jun 2021 13:57:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0062E6141E for ; Sun, 6 Jun 2021 13:56:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230025AbhFFN6s (ORCPT ); Sun, 6 Jun 2021 09:58:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230003AbhFFN6r (ORCPT ); Sun, 6 Jun 2021 09:58:47 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C015C061766; Sun, 6 Jun 2021 06:56:42 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id og14so16763926ejc.5; Sun, 06 Jun 2021 06:56:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=B0rXVH93m1esrgniY56F67/MKQHZRua4qaFSgZhso5A=; b=ZYJHPypwgqvPHEKh1lmmeAFviUz4jHRgvSd46LPFYJXXYtCax3Qq2jUvKxk6r/bDUb E1WP+81lV0p6dZ5ka+qUqhx/jbHC7Juk8TQzrh8POdjA5iq5dd8tES+luGdWE3WumfIP DLLYlOzaHOGPFU4OHA8KGwLth77X9IRSe6rf+LnHjas313S4YWSGj5cVwPk2UvgEPjWQ 1lcF9LMUxV7Sb0Y2dIyWi89JiEReP6GIp0rJqapAWqLMB/T9oAk3qVVafZYhO6CeWAC5 l4X0jwNeOupuMdCKjMaM7DeSnVSRClLivIszgZsr/bV2DedL+IPpg2UpkQRJKzXIKIbX CAXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=B0rXVH93m1esrgniY56F67/MKQHZRua4qaFSgZhso5A=; b=C8wdUmMRjSiFLpSDYQWRL0r7qE4Si1OvaZU6KI2y+rR+pxg2WAP6Vv9rkpqApOMh/V VWoqEWt78rkZ47j3cm6ByNFzjpoLgE5p9SBnWdrFHMCwYa/eWg5dFUaGYHok7oSRowM7 S2+z7HF4iPG8MgtI3UVL+fjIX/A8BLRNkcgwZn3KZHwzUKPxmfPSVF2EFu6eJ/aguJKo m6FeFR8vWEGFAw2v4squd0Kh0m6rsw+y9dp0JqgQ4oCrUJnGQoGpFqX4AabOlIa9wfYA H6i78EMroERZ2ACLYMm7OvYPytwjA88RT3uB32OKnHNS1MThkvI0q6GT3bEnb69kUbwb N1Dg== X-Gm-Message-State: AOAM531m/yGCe4gX4hgmPgB1sYWI465JU0aUSm6fd9mZnPveDBer9oJn yh/NDOmJzkmbW7EXIa5XJok= X-Google-Smtp-Source: ABdhPJzpzEG2M15A2d5WKbqVxjoSicjIl9ze9LY5kLBANp+FhBMFBy7rvOLF91TPoLFOTASzqqAJsA== X-Received: by 2002:a17:906:2bd3:: with SMTP id n19mr13710286ejg.210.1622987800696; Sun, 06 Jun 2021 06:56:40 -0700 (PDT) Received: from [192.168.0.108] ([77.124.85.114]) by smtp.gmail.com with ESMTPSA id n13sm2269048edx.30.2021.06.06.06.56.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 06 Jun 2021 06:56:40 -0700 (PDT) Subject: Re: [PATCH net-next v6 3/5] page_pool: Allow drivers to hint on SKB recycling To: Matteo Croce , David Ahern Cc: Ilias Apalodimas , Yunsheng Lin , netdev@vger.kernel.org, linux-mm@kvack.org, Ayush Sawal , Vinay Kumar Yadav , Rohit Maheshwari , "David S. Miller" , Jakub Kicinski , Thomas Petazzoni , Marcin Wojtas , Russell King , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Boris Pismenny , Arnd Bergmann , Andrew Morton , "Peter Zijlstra (Intel)" , Vlastimil Babka , Yu Zhao , Will Deacon , Fenghua Yu , Roman Gushchin , Hugh Dickins , Peter Xu , Jason Gunthorpe , Jonathan Lemon , Alexander Lobakin , Cong Wang , wenxu , Kevin Hao , Jakub Sitnicki , Marco Elver , Willem de Bruijn , Miaohe Lin , Guillaume Nault , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, Matthew Wilcox , Eric Dumazet , Lorenzo Bianconi , Saeed Mahameed , Andrew Lunn , Paolo Abeni , Sven Auhagen References: <20210521161527.34607-1-mcroce@linux.microsoft.com> <20210521161527.34607-4-mcroce@linux.microsoft.com> <722e5567-d8ee-228c-978e-9d5966257bb1@gmail.com> From: Tariq Toukan Message-ID: <63a4ea45-9938-3106-9eda-0f7e8fe079ce@gmail.com> Date: Sun, 6 Jun 2021 16:56:34 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On 6/5/2021 7:34 PM, Matteo Croce wrote: > On Sat, Jun 5, 2021 at 6:06 PM David Ahern wrote: >> >> On 6/4/21 2:42 AM, Ilias Apalodimas wrote: >>> [...] >>>>> + /* Driver set this to memory recycling info. Reset it on recycle. >>>>> + * This will *not* work for NIC using a split-page memory model. >>>>> + * The page will be returned to the pool here regardless of the >>>>> + * 'flipped' fragment being in use or not. >>>>> + */ >>>> >>>> I am not sure I understand how does the last part of comment related >>>> to the code below, as there is no driver using split-page memory model >>>> will reach here because those driver will not call skb_mark_for_recycle(), >>>> right? >>>> >>> >>> Yes the comment is there to prohibit people (mlx5 only actually) to add the >>> recycling bit on their driver. Because if they do it will *probably* work >>> but they might get random corrupted packets which will be hard to debug. >>> >> >> What's the complexity for getting it to work with split page model? >> Since 1500 is the default MTU, requiring a page per packet means a lot >> of wasted memory. > > We could create a new memory model, e.g. MEM_TYPE_PAGE_SPLIT, and > restore the behavior present in the previous versions of this serie, > which is, save xdp_mem_info in struct page. > As this could slightly impact the performances, this can be added in a > future change when the drivers which are doing it want to use this > recycling api. > page-split model doesn't only help reduce memory waste, but increase cache-locality, especially for aggregated GRO SKBs. I'm looking forward to integrating the page-pool SKB recycling API into mlx5e datapath. For this we need it to support the page-split model. Let's see what's missing and how we can help making this happen. Regards, Tariq