From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76CF77C092 for ; Tue, 5 Mar 2024 19:38:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.52 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709667532; cv=none; b=XCJGu5KRQ5iMtzSeq0AtFgquYsBB9MKnOR1s7fTM4P+PIfbCQ2VvxG/THvr8RHhoopi8EM5a0NpctY7uLBG6nw7HQJgZPNIYez1IL6TCNl0V/8qlClCWcT/e+uECnhQx6RrSrRjZzfcosx4GZtekG/4UhZmc6ulCMOPsipN6Qt8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709667532; c=relaxed/simple; bh=dbu7jPJNusoAIuT8pNXSfufXvY1iYoRgBC5JcoUfIfk=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=NwQ0N66lm4QgssVdgh0xQKR5NlZB0MwRwg4F7EDawC4G1bNKayUwp+Ykv8ghYz77XpOge9FDjvwmFInBu/oeke6RohmvBve6Nt2B8j6QAC/hRr+kpVyQCOoiCuyv/heZae+7fKC5W9o2iHbLAjJdChXUDvCUjrIWbK5UxoC6PRU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FRcoz8Mb; arc=none smtp.client-ip=209.85.208.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FRcoz8Mb" Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-5658082d2c4so8322215a12.1 for ; Tue, 05 Mar 2024 11:38:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709667529; x=1710272329; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=dbu7jPJNusoAIuT8pNXSfufXvY1iYoRgBC5JcoUfIfk=; b=FRcoz8MbQeQDcV4GpXollhLmuyqntrRmwucMFDu5EmYMZ8nbOuOfvmIlvlgmVq1Qf4 c8lJN6BtQ/2TfFGDTo4HPiEpJzTWViGGaRilIyIcv/YXc0UkVYjdNrq+GnwgnSNZswFB Pwg6nffta5e0uvgM3W0mPp9zxsYyvJlniRkpeGoe5zpcYBAQopGXC4Z+BTDR8JLdoEOD V88jb3Ko2M9as9eI9Qr0iuz9SzXWQw5AjNC7hFnkfWYJw4EM1fEvnQS2lXdYIEDXZQnA nY7tNeKRoWpelHmLrOxuSUqt3/up1X9b9cwQdhDzb5wYSPQ3D/F4hmnXwL7TSHyC3s1t oqmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709667529; x=1710272329; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dbu7jPJNusoAIuT8pNXSfufXvY1iYoRgBC5JcoUfIfk=; b=JweoSfXcmn3WThqa+SJy/B67kV0RBrqpPxBBqorNO1qUpKqTb7DZqG/ixH8+t1F+wA eEJVrXaaUB7NqwIFWjRoTrz4iQv+tN+dWIJ1MktZGchEOtpMZ6z8MYOS+ebOZ/zYbwIf APx8CM0ZfxzTwjFOuO0ADSh5yCtj6qN7+86V2wXbxPXwpUqS//x4lyf5SSb0NNm2Gxmo r5ncZHT2VcpB4PCHVRr0oNms5EOfJ6G08DXJq4YFGjwvdxw5d/t89Txku5prO2rUNdBW n0C9OS+WW8axVKZNaYRC1E3LF8TApcMbCmc2FMdf0xBmZjfRitTfWNFszBmEJv/qOMCd sNbQ== X-Forwarded-Encrypted: i=1; AJvYcCWqYpRVzybYpKOP0Y09nzN1f050mTzVLBZZt4uIALAEnSJYlEX8rRM6HROcEHI3C15Q5T6QtnBFAaROpCmf/mOO0VMjtzETi+I7p+Q= X-Gm-Message-State: AOJu0Yx17TpvinrefH445wN+vVBn/nvB0ct5wpGlhvTYdcMq9/7xk4YS 86rM7YoYd99zn+hB2mQcZ+xN4Qilu6uYc7YF5okAtujUhl9BQRxV6HGpWCC8ZxTpi2FJK4ghWlG 4nZ1mWRRtl0Kq+4Bxy+HwHLkDgfm7dlZNEVo5 X-Google-Smtp-Source: AGHT+IE9+/I3bSukMGSV69NWucyK49DrCDskm5u3bRNu62YrXsb5GYolXDQb6+2mnF70pbKLSkXCgPSxIq5NsfN3srs= X-Received: by 2002:a17:906:b790:b0:a45:b36c:55c7 with SMTP id dt16-20020a170906b79000b00a45b36c55c7mr421516ejb.63.1709667528752; Tue, 05 Mar 2024 11:38:48 -0800 (PST) Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20240305020153.2787423-1-almasrymina@google.com> <6208950d-6453-e797-7fc3-1dcf15b49dbe@huawei.com> In-Reply-To: <6208950d-6453-e797-7fc3-1dcf15b49dbe@huawei.com> From: Mina Almasry Date: Tue, 5 Mar 2024 11:38:37 -0800 Message-ID: Subject: Re: [RFC PATCH net-next v6 00/15] Device Memory TCP To: Yunsheng Lin Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-alpha@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, sparclinux@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-arch@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jonathan Corbet , Richard Henderson , Ivan Kokshaysky , Matt Turner , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Andreas Larsson , Jesper Dangaard Brouer , Ilias Apalodimas , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Arnd Bergmann , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , David Ahern , Willem de Bruijn , Shuah Khan , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Pavel Begunkov , David Wei , Jason Gunthorpe , Shailend Chand , Harshitha Ramamurthy , Shakeel Butt , Jeroen de Borst , Praveen Kaligineedi Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Mar 5, 2024 at 4:54=E2=80=AFAM Yunsheng Lin wrote: > > On 2024/3/5 10:01, Mina Almasry wrote: > > ... > > > > > Perf - page-pool benchmark: > > --------------------------- > > > > bench_page_pool_simple.ko tests with and without these changes: > > https://pastebin.com/raw/ncHDwAbn > > > > AFAIK the number that really matters in the perf tests is the > > 'tasklet_page_pool01_fast_path Per elem'. This one measures at about 8 > > cycles without the changes but there is some 1 cycle noise in some > > results. > > > > With the patches this regresses to 9 cycles with the changes but there > > is 1 cycle noise occasionally running this test repeatedly. > > > > Lastly I tried disable the static_branch_unlikely() in > > netmem_is_net_iov() check. To my surprise disabling the > > static_branch_unlikely() check reduces the fast path back to 8 cycles, > > but the 1 cycle noise remains. > > > > The last sentence seems to be suggesting the above 1 ns regresses is caus= ed > by the static_branch_unlikely() checking? Note it's not a 1ns regression, it's looks like maybe a 1 cycle regression (slightly less than 1ns if I'm reading the output of the test correctly): # clean net-next time_bench: Type:tasklet_page_pool01_fast_path Per elem: 8 cycles(tsc) 2.993 ns (step:0) # with patches time_bench: Type:tasklet_page_pool01_fast_path Per elem: 9 cycles(tsc) 3.679 ns (step:0) # with patches and with diff that disables static branching: time_bench: Type:tasklet_page_pool01_fast_path Per elem: 8 cycles(tsc) 3.248 ns (step:0) I do see noise in the test results between run and run, and any regression (if any) is slightly obfuscated by the noise, so it's a bit hard to make confident statements. So far it looks like a ~0.25ns regression without static branch and about ~0.65ns with static branch. Honestly when I saw all 3 results were within some noise I did not investigate more, but if this looks concerning to you I can dig further. I likely need to gather a few test runs to filter out the noise and maybe investigate the assembly my compiler is generating to maybe narrow down what changes there. --=20 Thanks, Mina