From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_NEOMUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B814C282D7 for ; Wed, 30 Jan 2019 22:57:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C0DA22084C for ; Wed, 30 Jan 2019 22:57:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ilf03mmD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732147AbfA3W5r (ORCPT ); Wed, 30 Jan 2019 17:57:47 -0500 Received: from mail-pl1-f193.google.com ([209.85.214.193]:45787 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727408AbfA3W5r (ORCPT ); Wed, 30 Jan 2019 17:57:47 -0500 Received: by mail-pl1-f193.google.com with SMTP id a14so501285plm.12 for ; Wed, 30 Jan 2019 14:57:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=fq7Lsxd5AMmHBw99b9q5P/CrUSY3q+ENTP+eej4OFqQ=; b=Ilf03mmDLvKn46JTyJ8XiwRM639VVzzmRo+1r/PgICK/mvDKvUGhIVZHuYUnmUJ4ah GhjE/VZxXUDurEVaCY8vwXTSVR7cocoPA7ucFa7UIIj1YfFXSyUHPaOymHVAHmp02n/4 cHiDaHdp1T8SigfPP4V8oB/obwIKVcIyqCsgBVEU8g8t5oBFcKuV7uGQUB+WQE4ktUcq Vca3Ab5JDwDScWzAI612Llcc0aRwsJMa9HMDvpyTKcMMznAYV1wnM1yEU8v/McALoNy7 YGZrut6XPMDkKUzFbZT5Wi5z0tfC9aaaQoas/iRhYRmrPl3tN1XQ25/81wg6IsrhP2y9 pfJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=fq7Lsxd5AMmHBw99b9q5P/CrUSY3q+ENTP+eej4OFqQ=; b=AypAqNzK7ZJmdJZ6bwkmMaPvGnetRLqDX7MGHg5AlmxoHD5vHEcTO/jopcFaa790xY F0cAITOYsImTO+LYBULsvkcpFax5ZJ1G+bVBl/gPtmyACw2zbTa+U1VnxljBhI0X0AFJ JlB1Em8zDl49tfoPUJmC4eaqeCXZ7ASvnfo9QfW//HW9IEtUVn5RaTe5c3CwuUfFvGfV 0ihKDumEGyfoefaB7TVpHnIKM4A8uql4jQ0ENzGF2hH9PBz7Iit66demAYR9XSdQGXiu Id0GeJ+b/LBydUo4SevLyik12fQNho/lnSSBsj5LmhCXa38ycqWl5Fl4wtqtrsfhYZkh EOmw== X-Gm-Message-State: AJcUukeoylsp74OKw12LcqZ7MR2N20GHYSExjOPedgWGHaBB2o0mv2i5 PDecMTnMYqKLv9aRFfHV268= X-Google-Smtp-Source: ALg8bN7dlk5cTVLW4nEdhrl4Ma6RNwvLDq8oUVjsK0eq6vjU5/NjeTnktltjQuAoijQgwaeIyFQIvA== X-Received: by 2002:a17:902:a5ca:: with SMTP id t10mr32157505plq.139.1548889066125; Wed, 30 Jan 2019 14:57:46 -0800 (PST) Received: from ast-mbp.dhcp.thefacebook.com ([2620:10d:c090:180::1:9e50]) by smtp.gmail.com with ESMTPSA id 5sm7876520pfz.149.2019.01.30.14.57.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Jan 2019 14:57:45 -0800 (PST) Date: Wed, 30 Jan 2019 14:57:43 -0800 From: Alexei Starovoitov To: "Paul E. McKenney" Cc: Will Deacon , Peter Zijlstra , Alexei Starovoitov , davem@davemloft.net, daniel@iogearbox.net, jakub.kicinski@netronome.com, netdev@vger.kernel.org, kernel-team@fb.com, mingo@redhat.com, jannh@google.com Subject: Re: bpf memory model. Was: [PATCH v4 bpf-next 1/9] bpf: introduce bpf_spin_lock Message-ID: <20190130225741.3rgbw2c6nh5v2kdj@ast-mbp.dhcp.thefacebook.com> References: <20190124180109.GA27771@hirez.programming.kicks-ass.net> <20190124235857.xyb5xx2ufr6x5mbt@ast-mbp.dhcp.thefacebook.com> <20190125102312.GC4500@hirez.programming.kicks-ass.net> <20190126001725.roqqfrpysyljqiqx@ast-mbp.dhcp.thefacebook.com> <20190128092408.GD28467@hirez.programming.kicks-ass.net> <20190128215623.6eqskzhklydhympa@ast-mbp> <20190130181100.GA18558@fuggles.cambridge.arm.com> <20190130183618.GX4240@linux.ibm.com> <20190130195113.xyqre4sxasit6vpu@ast-mbp.dhcp.thefacebook.com> <20190130210536.GY4240@linux.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190130210536.GY4240@linux.ibm.com> User-Agent: NeoMutt/20180223 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, Jan 30, 2019 at 01:05:36PM -0800, Paul E. McKenney wrote: > On Wed, Jan 30, 2019 at 11:51:14AM -0800, Alexei Starovoitov wrote: > > On Wed, Jan 30, 2019 at 10:36:18AM -0800, Paul E. McKenney wrote: > > > On Wed, Jan 30, 2019 at 06:11:00PM +0000, Will Deacon wrote: > > > > Hi Alexei, > > > > > > > > On Mon, Jan 28, 2019 at 01:56:24PM -0800, Alexei Starovoitov wrote: > > > > > On Mon, Jan 28, 2019 at 10:24:08AM +0100, Peter Zijlstra wrote: > > > > > > On Fri, Jan 25, 2019 at 04:17:26PM -0800, Alexei Starovoitov wrote: > > > > > > > What I want to avoid is to define the whole execution ordering model upfront. > > > > > > > We cannot say that BPF ISA is weakly ordered like alpha. > > > > > > > Most of the bpf progs are written and running on x86. We shouldn't > > > > > > > twist bpf developer's arm by artificially relaxing memory model. > > > > > > > BPF memory model is equal to memory model of underlying architecture. > > > > > > > What we can do is to make it bpf progs a bit more portable with > > > > > > > smp_rmb instructions, but we must not force weak execution on the developer. > > > > > > > > > > > > Well, I agree with only introducing bits you actually need, and my > > > > > > smp_rmb() example might have been poorly chosen, smp_load_acquire() / > > > > > > smp_store_release() might have been a far more useful example. > > > > > > > > > > > > But I disagree with the last part; we have to pick a model now; > > > > > > otherwise you'll pain yourself into a corner. > > > > > > > > > > > > Also; Alpha isn't very relevant these days; however ARM64 does seem to > > > > > > be gaining a lot of attention and that is very much a weak architecture. > > > > > > Adding strongly ordered assumptions to BPF now, will penalize them in > > > > > > the long run. > > > > > > > > > > arm64 is gaining attention just like riscV is gaining it too. > > > > > BPF jit for arm64 is very solid, while BPF jit for riscV is being worked on. > > > > > BPF is not picking sides in CPU HW and ISA battles. > > > > > > > > It's not about picking a side, it's about providing an abstraction of the > > > > various CPU architectures out there so that the programmer doesn't need to > > > > worry about where their program may run. Hell, even if you just said "eBPF > > > > follows x86 semantics" that would be better than saying nothing (and then we > > > > could have a discussion about whether x86 semantics are really what you > > > > want). > > > > > > To reinforce this point, the Linux-kernel memory model (tools/memory-model) > > > is that abstraction for the Linux kernel. Why not just use that for BPF? > > > > I already answered this earlier in the thread. > > tldr: not going to sacrifice performance. > > Understood. > > But can we at least say that where there are no performance consequences, > BPF should follow LKMM? You already mentioned smp_load_acquire() > and smp_store_release(), but the void atomics (e.g., atomic_inc()) > should also work because they don't provide any ordering guarantees. > The _relaxed(), _release(), and _acquire() variants of the value-returning > atomics should be just fine as well. > > The other value-returning atomics have strong ordering, which is fine > on many systems, but potentially suboptimal for the weakly ordered ones. > Though you have to have pretty good locality of reference to be able to > see the difference, because otherwise cache-miss overhead dominates. > > Things like cmpxchg() don't seem to fit BPF because they are normally > used in spin loops, though there are some non-spinning use cases. > > You correctly pointed out that READ_ONCE() and WRITE_ONCE() are suboptimal > on systems that don't support all sizes of loads, but I bet that there > are some sizes for which they are just fine across systems, for example, > pointer size and int size. > > Does that help? Or am I missing additional cases where performance > could be degraded? bpf doesn't have smp_load_acquire, atomic_fetch_add, xchg, fence instructions. They can be added step by step. That's easy. I believe folks already started working on adding atomic_fetch_add. What I have problem with is making a statement today that bpf's end goal is LKMM. Even after adding all sorts of instructions it may not be practical. Only when real use case requires adding new instruction we do it. Do you have a bpf program that needs smp_load_acquire ?