From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2080C433EF for ; Mon, 6 Sep 2021 00:45:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AF22D61027 for ; Mon, 6 Sep 2021 00:45:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234555AbhIFAJ3 (ORCPT ); Sun, 5 Sep 2021 20:09:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229735AbhIFAJ2 (ORCPT ); Sun, 5 Sep 2021 20:09:28 -0400 Received: from nautica.notk.org (ipv6.notk.org [IPv6:2001:41d0:1:7a93::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 126B8C061575 for ; Sun, 5 Sep 2021 17:08:24 -0700 (PDT) Received: by nautica.notk.org (Postfix, from userid 108) id D607DC01E; Mon, 6 Sep 2021 02:08:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=codewreck.org; s=2; t=1630886898; bh=OeCX4dJpwBgitXOSpmPOmvNzOoK+lAbEx+BXVeu2QM4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=17Fqq2Fo5jQOobm0vMf6lneLlkpDc5uxueDYcq1/GU3rsBSmycv9xTu8aZW3+hdPr fHaVmAu6ZbaPBGKN8BqbduccwhJ37y0xRln9YOUqM0kJFWZsXtR+Depfpte6tnclSP fy4NCB1YX3WYZT/jydnexd2eMi1fh1Nt5Q/13AlFcPDJHfouymV+Jr0OJZ+kD1+BWS fS7SAGBRPaZKoE8CcDL1zCaWEdeqoyTGQqu0IXFVQ3dSIjoJUyRyu2Y6Tit9QIIFYa D+ZhqkKNqcCDtxqr8eRRW1ip9405NF47N755hweocb1adesDKp4yqvlgDc9TNz0v1J Npdtr8yHM/Oxg== Received: from odin.codewreck.org (localhost [127.0.0.1]) by nautica.notk.org (Postfix) with ESMTPS id 7E163C009; Mon, 6 Sep 2021 02:08:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=codewreck.org; s=2; t=1630886897; bh=OeCX4dJpwBgitXOSpmPOmvNzOoK+lAbEx+BXVeu2QM4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=GynXHJhrXtydznYDXD1Zd+zU+gXHeODQ3ykenpx+yygjUyhg0FSE7pI2wmRoyRkjT VKwShy/ag4EWe87WDyF++0V0JFEqIQYxetyVBCwRCtjoXZTapoBQdd6A/LgmLmmdET nDv/i7yuaExhCV6h72kEy4LBI/JrseIKnR9Pcp/HEpCAf7ZETV6pAauYQk94Mmps+C BTkB63PnUNVZR9zuiwkACMD6HBNJ3RIsbdQaZtVK7TtZyyrk6KyuEyD1L7qtOKGoT4 Tz0h175G7O25e46iELfV2+dO384GSxVRO1KYoSNZacG4uCrBZEytOUHEgKzbhBNvJL uII+8LHqv2afQ== Received: from localhost (odin.codewreck.org [local]) by odin.codewreck.org (OpenSMTPD) with ESMTPA id e8c97575; Mon, 6 Sep 2021 00:08:11 +0000 (UTC) Date: Mon, 6 Sep 2021 09:07:56 +0900 From: Dominique Martinet To: Eric Van Hensbergen Cc: Christian Schoenebeck , Greg Kurz , Latchesar Ionkov , netdev@vger.kernel.org, v9fs-developer@lists.sourceforge.net Subject: Re: [PATCH 2/2] net/9p: increase default msize to 128k Message-ID: References: <61ea0f0faaaaf26dd3c762eabe4420306ced21b9.1630770829.git.linux_oss@crudebyte.com> <1915472.2DI3jHSlUk@silver> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Eric Van Hensbergen wrote on Sun, Sep 05, 2021 at 06:44:13PM -0500: > there will likely be a tradeoff with tcp in terms of latency to first > message so while > absolute bw may be higher processing time may suffer. 8k was default msize > to more closely match it to jumbo frames on an ethernet. of course all > that intuition is close to 30 years out of date…. It's not because the max size is 128k (or 1MB) that this much is sent over the wire everytime -- if a message used to fit in 8KB, then its on-the-wire size won't change and speed/latency won't be affected for these. For messages that do require more than 8KB (read/write/readdir) then you can fit more data per message, so for a given userspace request (feed me xyz amount of data) you'll have less client-server round-trips, and the final user-reflected latency will be better as well -- that's why e.g. NFS has been setting a max size of 1MB by default for a while now, and they allow even more (32MB iirc? not sure) I've only had done these tests years ago and no longer have access to the note that was written back then, but TCP also definitely benefits from > 64k msize as long as there's enough memory available. The downside (because it's not free) is there though, you need more memory for 9p with big buffers even if we didn't need so much in the first place. The code using a slab now means that the memory is not locked per mount as it used to, but that also means allocations can fail if there is a big pressure after not having been released. OTOH as long as it's consistently used the buffers will be recycled so it's not necessarily too bad performance-wise in hot periods. Ideally we'd need to rework transports to allow scatter-gather (iovec or similar API), and work with smaller allocations batched together on send, but I don't have time for something like that... If we do that we can probably get the best of both worlds -- and could consider >1MB, but that's unrealistic as of now, regardless of the transport. -- Dominique