From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 979C4C4332F for ; Mon, 6 Nov 2023 21:15:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233076AbjKFVPK (ORCPT ); Mon, 6 Nov 2023 16:15:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231739AbjKFVPG (ORCPT ); Mon, 6 Nov 2023 16:15:06 -0500 Received: from mail-vk1-xa33.google.com (mail-vk1-xa33.google.com [IPv6:2607:f8b0:4864:20::a33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3CC5AB3; Mon, 6 Nov 2023 13:15:03 -0800 (PST) Received: by mail-vk1-xa33.google.com with SMTP id 71dfb90a1353d-4ac459d7962so1005468e0c.1; Mon, 06 Nov 2023 13:15:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699305302; x=1699910102; darn=vger.kernel.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=pDNcwAtyE05sKduQb5rMgv/slyyDqYM9dQystHpAe00=; b=d/waIwl8ZhA+3fMGbwtKR/1HQ2FRL8sJJsPoARNUrIj8zRK8eO7BDXtgdodqgjiHLU unS+H7Eo+A7ubP2doo1X9VDwtqXBTb3ZE2Z826gxkVF4W0np3D84TjK0bwBUS4W6KEoL qBFnETI4bh8yk/HI3sm5F++bZk1MkQeh7O3+c3lcqALf1dcJTnQMH+BBcjMAcREoAYN1 AbMnGj1EzdZssMWjcM16p5fGRn00qXcOBLFkZxFhNtlcigbicTYrEPh7n57SXkrT2+Cj jhacDSfXLrzR+cF+pM2dASy3gyLaxagYqaM/x+jFxgRDfvXU/TyTl5RSnksMGtFehfvY wK8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699305302; x=1699910102; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pDNcwAtyE05sKduQb5rMgv/slyyDqYM9dQystHpAe00=; b=aX5cbGP1wCwveuRfm4tKEgwNVCIrlD5DrRm910xvVAVp5pmL5H/X5MiGc/fVpUbZl3 ykvtYO8cH5Ez3/ejMX7SWUcIn5YKC0k/ieHa54C+lsJQfmUiDnbg1OWaJtdgFtx616KE PV2OTpkPfHvPA9Fm4Ht3tgTK9Qq8h55Lw/WgOvsLWh+RqXxMaBUYjYRNFhDiKQKrf7mZ 4ZXR8CfbbQk6yNpRcFjtoLgPPPt0v23yPN9f3Az7xeObo4vsx2KwaM5fR/Oudog45rV5 73qi2paemTaAT40Q1m4AEtCvgjvf2goD4N6vaY4GiUItT2Du18y9A+ckgy5jN3nnmk/n 7YyA== X-Gm-Message-State: AOJu0YySOXuUrwY9uvsMK5z2lq1ujaCi0/VA0l+EtzQP6aA1saFuim5x ZqR5sZlDUW213kinphNnAxKUpM/4VKuN/IWN8wQ= X-Google-Smtp-Source: AGHT+IFSfXgdSlZ+tas3UVj7xs8CWVob5ysFpk92liyDA5WPCjjGPD1amiOzYCm06+9kKyONPbaGF06Qrr1pItVwWII= X-Received: by 2002:a05:6122:4694:b0:49d:20fb:c899 with SMTP id di20-20020a056122469400b0049d20fbc899mr441320vkb.4.1699305302147; Mon, 06 Nov 2023 13:15:02 -0800 (PST) MIME-Version: 1.0 References: <20231106024413.2801438-1-almasrymina@google.com> <20231106024413.2801438-11-almasrymina@google.com> In-Reply-To: From: Willem de Bruijn Date: Mon, 6 Nov 2023 13:14:25 -0800 Message-ID: Subject: Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP To: Mina Almasry Cc: Stanislav Fomichev , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jesper Dangaard Brouer , Ilias Apalodimas , Arnd Bergmann , David Ahern , Shuah Khan , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Shakeel Butt , Jeroen de Borst , Praveen Kaligineedi , Willem de Bruijn , Kaiyuan Zhang Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > IMHO, we need a better UAPI to receive the tokens and give them back to > > the kernel. CMSG + setsockopt(SO_DEVMEM_DONTNEED) get the job done, > > but look dated and hacky :-( > > > > We should either do some kind of user/kernel shared memory queue to > > receive/return the tokens (similar to what Jonathan was doing in his > > proposal?) > > I'll take a look at Jonathan's proposal, sorry, I'm not immediately > familiar but I wanted to respond :-) But is the suggestion here to > build a new kernel-user communication channel primitive for the > purpose of passing the information in the devmem cmsg? IMHO that seems > like an overkill. Why add 100-200 lines of code to the kernel to add > something that can already be done with existing primitives? I don't > see anything concretely wrong with cmsg & setsockopt approach, and if > we switch to something I'd prefer to switch to an existing primitive > for simplicity? > > The only other existing primitive to pass data outside of the linear > buffer is the MSG_ERRQUEUE that is used for zerocopy. Is that > preferred? Any other suggestions or existing primitives I'm not aware > of? > > > or bite the bullet and switch to io_uring. > > > > IMO io_uring & socket support are orthogonal, and one doesn't preclude > the other. As you know we like to use sockets and I believe there are > issues with io_uring adoption at Google that I'm not familiar with > (and could be wrong). I'm interested in exploring io_uring support as > a follow up but I think David Wei will be interested in io_uring > support as well anyway. I also disagree that we need to replace a standard socket interface with something "faster", in quotes. This interface is not the bottleneck to the target workload. Replacing the synchronous sockets interface with something more performant for workloads where it is, is an orthogonal challenge. However we do that, I think that traditional sockets should continue to be supported. The feature may already even work with io_uring, as both recvmsg with cmsg and setsockopt have io_uring support now.