From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8330C8563F for ; Tue, 5 Mar 2024 19:38:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.43 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709667532; cv=none; b=V+LFEEztmFB2CgP0V16FBxI/E5kll04dXWrLJ2Nj0GYoL0u1UJS8EXT+G32iWTGQ3t7LThqXyxTa1O44iOoGj83YPvqH9kDGpGnU6V/XxZUZafCvbGvuORMxCcG78HxLYiwAQr3YLQ7y1wOGiUweRmMbYETO6yip1oL3BIuhptQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709667532; c=relaxed/simple; bh=dbu7jPJNusoAIuT8pNXSfufXvY1iYoRgBC5JcoUfIfk=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=NwQ0N66lm4QgssVdgh0xQKR5NlZB0MwRwg4F7EDawC4G1bNKayUwp+Ykv8ghYz77XpOge9FDjvwmFInBu/oeke6RohmvBve6Nt2B8j6QAC/hRr+kpVyQCOoiCuyv/heZae+7fKC5W9o2iHbLAjJdChXUDvCUjrIWbK5UxoC6PRU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FRcoz8Mb; arc=none smtp.client-ip=209.85.218.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FRcoz8Mb" Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a45b25f6f84so38478266b.0 for ; Tue, 05 Mar 2024 11:38:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709667529; x=1710272329; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=dbu7jPJNusoAIuT8pNXSfufXvY1iYoRgBC5JcoUfIfk=; b=FRcoz8MbQeQDcV4GpXollhLmuyqntrRmwucMFDu5EmYMZ8nbOuOfvmIlvlgmVq1Qf4 c8lJN6BtQ/2TfFGDTo4HPiEpJzTWViGGaRilIyIcv/YXc0UkVYjdNrq+GnwgnSNZswFB Pwg6nffta5e0uvgM3W0mPp9zxsYyvJlniRkpeGoe5zpcYBAQopGXC4Z+BTDR8JLdoEOD V88jb3Ko2M9as9eI9Qr0iuz9SzXWQw5AjNC7hFnkfWYJw4EM1fEvnQS2lXdYIEDXZQnA nY7tNeKRoWpelHmLrOxuSUqt3/up1X9b9cwQdhDzb5wYSPQ3D/F4hmnXwL7TSHyC3s1t oqmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709667529; x=1710272329; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dbu7jPJNusoAIuT8pNXSfufXvY1iYoRgBC5JcoUfIfk=; b=aLnDAuexCs3cgVKkG7ZAcK17MLEv755vOukCM6mWFFIs4JvVvdsdJriGjaOfht4krg 0OSnUkLHU2H8eigPpM8Dmswb4JijcVl+j3pEPsodXeItTDHQtlok9hGIE/LwIaDcZwPH 4fZuPWT5VDrQUTpGoFtPDpXZbWG3Q4HcH2AO6JWdPpW6/2YzFbht8CKoxvAmuSzcP1Uc SDx3qwQ/Vv0okhJI/Pfzbt6PTHa/yVWCTCL0XjnLq5uXJmjqxg9lsT7SAGjnOMtfwuWd rRb5tV2ryboX7ZID+fI5QcrEbk4wMoM5aWPsQ1o22ZrWy+9j34vYTBuLKXElbnmyHWqf U5KQ== X-Forwarded-Encrypted: i=1; AJvYcCXGMxBGk7QOFjDsQ5/WJntPYtww2iR28wy5Ywlvi/jK8eqfkuLsQTPWRVH8WRdQp++YMrIL6cJqbuQX5JGV8x6o6e8S3vU0E0/SzVP+ X-Gm-Message-State: AOJu0Yzn8Z+zWcORyh8k/khXjgEf3R996LmjfwQHTZSNGSnPpAUEyFUM 5taqd1fapAKwpzc41RQy9keHWZ2Pzng6Vt++43QIiATQSH8POF9eqNpPokZQNZYUzNGMrUWE7Xz Z/W1zNuZeUYXC765rOkJrtz56qCDJMi5/+vRj X-Google-Smtp-Source: AGHT+IE9+/I3bSukMGSV69NWucyK49DrCDskm5u3bRNu62YrXsb5GYolXDQb6+2mnF70pbKLSkXCgPSxIq5NsfN3srs= X-Received: by 2002:a17:906:b790:b0:a45:b36c:55c7 with SMTP id dt16-20020a170906b79000b00a45b36c55c7mr421516ejb.63.1709667528752; Tue, 05 Mar 2024 11:38:48 -0800 (PST) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20240305020153.2787423-1-almasrymina@google.com> <6208950d-6453-e797-7fc3-1dcf15b49dbe@huawei.com> In-Reply-To: <6208950d-6453-e797-7fc3-1dcf15b49dbe@huawei.com> From: Mina Almasry Date: Tue, 5 Mar 2024 11:38:37 -0800 Message-ID: Subject: Re: [RFC PATCH net-next v6 00/15] Device Memory TCP To: Yunsheng Lin Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-alpha@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, sparclinux@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-arch@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jonathan Corbet , Richard Henderson , Ivan Kokshaysky , Matt Turner , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Andreas Larsson , Jesper Dangaard Brouer , Ilias Apalodimas , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Arnd Bergmann , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , David Ahern , Willem de Bruijn , Shuah Khan , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Pavel Begunkov , David Wei , Jason Gunthorpe , Shailend Chand , Harshitha Ramamurthy , Shakeel Butt , Jeroen de Borst , Praveen Kaligineedi Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Mar 5, 2024 at 4:54=E2=80=AFAM Yunsheng Lin wrote: > > On 2024/3/5 10:01, Mina Almasry wrote: > > ... > > > > > Perf - page-pool benchmark: > > --------------------------- > > > > bench_page_pool_simple.ko tests with and without these changes: > > https://pastebin.com/raw/ncHDwAbn > > > > AFAIK the number that really matters in the perf tests is the > > 'tasklet_page_pool01_fast_path Per elem'. This one measures at about 8 > > cycles without the changes but there is some 1 cycle noise in some > > results. > > > > With the patches this regresses to 9 cycles with the changes but there > > is 1 cycle noise occasionally running this test repeatedly. > > > > Lastly I tried disable the static_branch_unlikely() in > > netmem_is_net_iov() check. To my surprise disabling the > > static_branch_unlikely() check reduces the fast path back to 8 cycles, > > but the 1 cycle noise remains. > > > > The last sentence seems to be suggesting the above 1 ns regresses is caus= ed > by the static_branch_unlikely() checking? Note it's not a 1ns regression, it's looks like maybe a 1 cycle regression (slightly less than 1ns if I'm reading the output of the test correctly): # clean net-next time_bench: Type:tasklet_page_pool01_fast_path Per elem: 8 cycles(tsc) 2.993 ns (step:0) # with patches time_bench: Type:tasklet_page_pool01_fast_path Per elem: 9 cycles(tsc) 3.679 ns (step:0) # with patches and with diff that disables static branching: time_bench: Type:tasklet_page_pool01_fast_path Per elem: 8 cycles(tsc) 3.248 ns (step:0) I do see noise in the test results between run and run, and any regression (if any) is slightly obfuscated by the noise, so it's a bit hard to make confident statements. So far it looks like a ~0.25ns regression without static branch and about ~0.65ns with static branch. Honestly when I saw all 3 results were within some noise I did not investigate more, but if this looks concerning to you I can dig further. I likely need to gather a few test runs to filter out the noise and maybe investigate the assembly my compiler is generating to maybe narrow down what changes there. --=20 Thanks, Mina