From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC6D9C47404 for ; Fri, 4 Oct 2019 15:58:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 92D86222BE for ; Fri, 4 Oct 2019 15:58:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="J3H7HHcp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390184AbfJDP6t (ORCPT ); Fri, 4 Oct 2019 11:58:49 -0400 Received: from mail-ot1-f66.google.com ([209.85.210.66]:33309 "EHLO mail-ot1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389794AbfJDP6s (ORCPT ); Fri, 4 Oct 2019 11:58:48 -0400 Received: by mail-ot1-f66.google.com with SMTP id 60so5739935otu.0 for ; Fri, 04 Oct 2019 08:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Kii2t84FnpnUt+XgKeXePx9ynfaDpkgw73Li78q+P1g=; b=J3H7HHcpkA4nMMNjy8hzRpc9bYvv2Qp5Ao4uFjr6zVyKTGFd/1zIIzSsdUbnXKDejt PXOf+WJhfVQ1cEuHg4W+YdxfNnClR9ZEgbPbvO48b6CuWnyHM7jaVLqU9oDkGDmGbN7p Fg4sxQWdn5g606/E4II2GwxDufXL3ZnsGhL2Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Kii2t84FnpnUt+XgKeXePx9ynfaDpkgw73Li78q+P1g=; b=BoWSo+psd/FKLX1vZP1VvgV+XvjgpdYj3BpmAHp3PoA4OGtP1CBZCpVZ0ssKrg7soD CHEe1rTt011d2ivUKHWHuv4pUwgKEiFDYVJTdSlEq6SBq3borr5p9r6cffIcfWFdPOfR OhwU7P7iRsszu8PXN9MoiofaVokIZP19UT4X/6iq8HiPITm2uJIqK7xZ/4c6IKf9IT47 ZeDLNfN4LvMtI58KyyG29Tw186cVtc/5h49pDi3/nzxazGi6YQWxAfGajGGSW3tWdxbw gqKdeXvHj1PVJNjsbqB4lOU+feth9Kj02JrI2XMPs/1W83jOR4LeZSfZIXtzXq/vew53 9QPg== X-Gm-Message-State: APjAAAWeoioa72yaSUtqr6VHhv0QeYIntCnrpZfQyEqr/VSeExqZ91so nM8c5fuiti36LRP7qEvVNVWn+VpMSN9Sc6Z07loJPA== X-Google-Smtp-Source: APXvYqyKL1Kens97wBLPLsOvsJYc8PCr4lRbH4sMcBrXV994Y/EKtcQCA14vCjdq7k4WHKF1OgG7W4aVju1mSLnylQA= X-Received: by 2002:a9d:7398:: with SMTP id j24mr10951958otk.289.1570204727971; Fri, 04 Oct 2019 08:58:47 -0700 (PDT) MIME-Version: 1.0 References: <157002302448.1302756.5727756706334050763.stgit@alrua-x1> <87r23vq79z.fsf@toke.dk> <20191003105335.3cc65226@carbon> <87pnjdq4pi.fsf@toke.dk> <1c9b72f9-1b61-d89a-49a4-e0b8eead853d@solarflare.com> <5d964d8ccfd90_55732aec43fe05c47b@john-XPS-13-9370.notmuch> <87tv8pnd9c.fsf@toke.dk> <68466316-c796-7808-6932-01d9d8c0a40b@solarflare.com> In-Reply-To: <68466316-c796-7808-6932-01d9d8c0a40b@solarflare.com> From: Lorenz Bauer Date: Fri, 4 Oct 2019 16:58:36 +0100 Message-ID: Subject: Re: [PATCH bpf-next 0/9] xdp: Support multiple programs on a single interface through chain calls To: Edward Cree Cc: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= , John Fastabend , Alexei Starovoitov , Jesper Dangaard Brouer , Song Liu , Daniel Borkmann , Alexei Starovoitov , Martin Lau , Yonghong Song , Marek Majkowski , David Miller , "netdev@vger.kernel.org" , "bpf@vger.kernel.org" , kernel-team Content-Type: text/plain; charset="UTF-8" Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Fri, 4 Oct 2019 at 11:34, Edward Cree wrote: > > Enforcement is easily dealt with: you just don't give people the caps/ > perms to load XDP programs directly, so the only way they can do it is > via your loader (which you give them a socket or dbus or something to > talk to). Writing this daemon is actually harder than it sounds. Loading eBPF programs can become fairly complex, with eBPF maps being shared between different programs. If you want to support all use cases (which you kind of have to) then you'll end up writing an RPC wrapper for libbpf, which sounds very painful to me. So I think for this to work at all, loading has to happen in the user space components. Only construction of the control flow should be centralised. This has the knock on effect that these components need CAP_NET_ADMIN, since too much of eBPF relies on having that capability right now: various map types, safety features applied to non-root eBPF, etc. Given time this will be fixed, and maybe these programs can then just have CAP_BPF or whatever. I chatted with my colleague Arthur, and we think this might work if all programs are forced to comply with the xdpcap-style tail call map: a prog array with MAX_XDP_ACTION slots, which each program calls into via tail_call(map, action); return action; // to handle the empty slot case You could then send (program fd, tail call map fd) along with a priority of some sort via SCM_RIGHTS. The daemon can then update the tail call maps as needed. The problem is that this only allows for linear (not tree-like) control flow. We'll try and hack up a POC to see if it works at all. > In any case, it seems like XDP users in userspace still need to > communicate with each other in order to update the chain map (which > seems to rely on knowing where one's own program fits into it); you > suggest they might communicate through the chain map itself, and then > veer off into the weeds of finding race-free ways of doing that. This > seems (to me) needlessly complex. I agree. > Incidentally, there's also a performance advantage to an eBPF dispatcher, > because it means the calls to the individual programs can be JITted and > therefore be direct, whereas an in-kernel data-driven dispatcher has to > use indirect calls (*waves at spectre*). This is if we somehow got full blown calls between distinct eBPF programs? > Maybe Lorenz could describe what he sees as the difficulties with the > centralised daemon approach. In what ways is his current "xdpd" > solution unsatisfactory? xdpd contains the logic to load and install all the different XDP programs we have. If we want to change one of them we have to redeploy the whole thing. Same if we want to add one. It also makes life-cycle management harder than it should be. So our xdpd is not at all like the "loader" you envision. -- Lorenz Bauer | Systems Engineer 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK www.cloudflare.com