From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5F1FC432C0 for ; Tue, 26 Nov 2019 23:10:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B25932068E for ; Tue, 26 Nov 2019 23:10:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=fomichev-me.20150623.gappssmtp.com header.i=@fomichev-me.20150623.gappssmtp.com header.b="Qcqygajn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726437AbfKZXKe (ORCPT ); Tue, 26 Nov 2019 18:10:34 -0500 Received: from mail-pf1-f194.google.com ([209.85.210.194]:42049 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726231AbfKZXKd (ORCPT ); Tue, 26 Nov 2019 18:10:33 -0500 Received: by mail-pf1-f194.google.com with SMTP id s5so9940125pfh.9 for ; Tue, 26 Nov 2019 15:10:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fomichev-me.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=KBpvCHEa5sLELqT+roIOkYleYG/FrG+DhoT0SoH7wto=; b=QcqygajnmtxaddE8/+AVs9ixIiq/T+dAFkPCspbbuZ7uTZB05Nwtu5lcic9Sg8WP4R Xo+tgNd9z2hVN4Tp1KW/2rr7Kavy4k/VAzPhkgzC6bcYQepqYVBhj//ESPxcNPKBEFe+ kOHAoPJY+D49wBam22nG743QpapWP8hQmkY3OGJU6cMdmZYEWDvBUcH9Ni6Ymd5lUAeT BBwfxcLulCDgZda5ti3fY9z4uFQVbB4DPtgRNvoOD2nzBl5p1Iz4xR4OVya5qAy7k7nQ G2mf0521nk7tnXeuWYhRYMvWDjk0j3oC1+9AxnM6w55AsKBFqosPI3nGSp5yn2haHK1z WFjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=KBpvCHEa5sLELqT+roIOkYleYG/FrG+DhoT0SoH7wto=; b=ZenvL5M1xWN8DbPEQtR/m95krNciO1IKKx6qv2FbFaCqClHRFwViWUuDtqC4EQMB7V kU83iwhUIz62WfzvcRj/JgdZAI9HTApP9xF6rCaWQBM4fj1ccp3FBR8q4dq2+Tx0TSq4 KMUPmy43UThxAMS4rb/8EvRRnNu0PsqUbi4hOlB7TliXo6EtmW9JAHCfgeA0NQMiZiE+ he0MhoaUkK5BfO0tkGRTVU/NyUJ13vitwFPQBaZmu26I5UJu937XBqSXqeK2FRRVKO/r /HtamXoCKwkF1eUzJXVhSoBkcxtXl0ItH72HHbF3gcIxpLdmJtLf4USvwxp92ryiYOX6 rVNA== X-Gm-Message-State: APjAAAUglTzO1Q8UcfVE8fYSVFyaG2NAesTxeDgrI7z4aKQVRmB6fqri tSvxg4Cl/2lTjM2e6O1lfQkywg== X-Google-Smtp-Source: APXvYqzBYAFQIZHabnmN7XFlslj/eCfFqTCFvh934/aSie59lir3EYOcytq0kCJwg6n6vjGE1hVHeg== X-Received: by 2002:a63:190a:: with SMTP id z10mr1111375pgl.153.1574809832264; Tue, 26 Nov 2019 15:10:32 -0800 (PST) Received: from localhost ([2601:646:8f00:18d9:d0fa:7a4b:764f:de48]) by smtp.gmail.com with ESMTPSA id 206sm15686831pfu.45.2019.11.26.15.10.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 15:10:31 -0800 (PST) Date: Tue, 26 Nov 2019 15:10:30 -0800 From: Stanislav Fomichev To: Andrii Nakryiko Cc: Arnaldo Carvalho de Melo , Toke =?iso-8859-1?Q?H=F8iland-J=F8rgensen?= , Andrii Nakryiko , Adrian Hunter , Alexei Starovoitov , Daniel Borkmann , Jiri Olsa , Martin KaFai Lau , Namhyung Kim , bpf , Networking , linux-perf-users@vger.kernel.org, Linux Kernel Mailing List , Quentin Monnet Subject: Re: [PATCH] libbpf: Fix up generation of bpf_helper_defs.h Message-ID: <20191126231030.GE3145429@mini-arch.hsd1.ca.comcast.net> References: <20191126151045.GB19483@kernel.org> <20191126154836.GC19483@kernel.org> <87imn6y4n9.fsf@toke.dk> <20191126183451.GC29071@kernel.org> <87d0dexyij.fsf@toke.dk> <20191126190450.GD29071@kernel.org> <20191126221018.GA22719@kernel.org> <20191126221733.GB22719@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.12.2 (2019-09-21) Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On 11/26, Andrii Nakryiko wrote: > On Tue, Nov 26, 2019 at 2:17 PM Arnaldo Carvalho de Melo > wrote: > > > > Em Tue, Nov 26, 2019 at 07:10:18PM -0300, Arnaldo Carvalho de Melo escreveu: > > > Em Tue, Nov 26, 2019 at 02:05:41PM -0800, Andrii Nakryiko escreveu: > > > > On Tue, Nov 26, 2019 at 11:12 AM Arnaldo Carvalho de Melo > > > > wrote: > > > > > > > > > > Em Tue, Nov 26, 2019 at 07:50:44PM +0100, Toke Høiland-Jørgensen escreveu: > > > > > > Arnaldo Carvalho de Melo writes: > > > > > > > > > > > > > Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu: > > > > > > >> Arnaldo Carvalho de Melo writes: > > > > > > >> > > > > > > >> > Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu: > > > > > > >> >> Hi guys, > > > > > > >> >> > > > > > > >> >> While merging perf/core with mainline I found the problem below for > > > > > > >> >> which I'm adding this patch to my perf/core branch, that soon will go > > > > > > >> >> Ingo's way, etc. Please let me know if you think this should be handled > > > > > > >> >> some other way, > > > > > > >> > > > > > > > >> > This is still not enough, fails building in a container where all we > > > > > > >> > have is the tarball contents, will try to fix later. > > > > > > >> > > > > > > >> Wouldn't the right thing to do not be to just run the script, and then > > > > > > >> put the generated bpf_helper_defs.h into the tarball? > > > > > > > > > > > > I would rather continue just running tar and have the build process > > > > > > > in-tree or outside be the same. > > > > > > > > > > > > Hmm, right. Well that Python script basically just parses > > > > > > include/uapi/linux/bpf.h; and it can be given the path of that file with > > > > > > the --filename argument. So as long as that file is present, it should > > > > > > be possible to make it work, I guess? > > > > > > > > > > > However, isn't the point of the tarball to make a "stand-alone" source > > > > > > distribution? > > > > > > > > > > Yes, it is, and as far as possible without any prep, just include the > > > > > in-source tree files needed to build it. > > > > > > > > > > > I'd argue that it makes more sense to just include the > > > > > > generated header, then: The point of the Python script is specifically > > > > > > to extract the latest version of the helper definitions from the kernel > > > > > > source tree. And if you're "freezing" a version into a tarball, doesn't > > > > > > it make more sense to also freeze the list of BPF helpers? > > > > > > > > > > Your suggestion may well even be the only solution, as older systems > > > > > don't have python3, and that script requires it :-\ > > > > > > > > > > Some containers were showing this: > > > > > > > > > > /bin/sh: 1: /git/linux/scripts/bpf_helpers_doc.py: not found > > > > > Makefile:184: recipe for target 'bpf_helper_defs.h' failed > > > > > make[3]: *** [bpf_helper_defs.h] Error 127 > > > > > make[3]: *** Deleting file 'bpf_helper_defs.h' > > > > > Makefile.perf:778: recipe for target '/tmp/build/perf/libbpf.a' failed > > > > > > > > > > That "not found" doesn't mean what it looks from staring at the above, > > > > > its just that: > > > > > > > > > > nobody@1fb841e33ba3:/tmp/perf-5.4.0$ head -1 /tmp/perf-5.4.0/scripts/bpf_helpers_doc.py > > > > > #!/usr/bin/python3 > > > > > nobody@1fb841e33ba3:/tmp/perf-5.4.0$ ls -la /usr/bin/python3 > > > > > ls: cannot access /usr/bin/python3: No such file or directory > > > > > nobody@1fb841e33ba3:/tmp/perf-5.4.0$ > > > > > > > > > > So, for now, I'll keep my fix and start modifying the containers where > > > > > this fails and disable testing libbpf/perf integration with BPF on those > > > > > containers :-\ > > > > > > > > I don't think there is anything Python3-specific in that script. I > > > > changed first line to > > > > > > > > #!/usr/bin/env python > > > > > > > > and it worked just fine. Do you mind adding this fix and make those > > > > older containers happy(-ier?). > > > > > > I'll try it, was trying the other way around, i.e. adding python3 to > > > those containers and they got happier, but fatter, so I'll remove that > > > and try your way, thanks! > > > > > > I didn't try it that way due to what comes right after the interpreter > > > line: > > > > > > #!/usr/bin/python3 > > > # SPDX-License-Identifier: GPL-2.0-only > > > # > > > # Copyright (C) 2018-2019 Netronome Systems, Inc. > > > > > > # In case user attempts to run with Python 2. > > > from __future__ import print_function > > > > And that is why I think you got it working, that script uses things > > like: > > > > print('Parsed description of %d helper function(s)' % len(self.helpers), > > file=sys.stderr) > > > > That python2 thinks its science fiction, what tuple is that? Can't > > understand, print isn't a function back then. > > Not a Python expert (or even regular user), but quick googling showed > that this import is the way to go to use Python3 semantics of print > within Python2, so seems like that's fine. But maybe Quentin has > anything to say about this. We are using this script with python2.7, works just fine :-) So maybe doing s/python3/python/ is the way to go, whatever default python is installed, it should work with that. > > https://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html#the-print-function > > > > I've been adding python3 to where it is available and not yet in the > > container images, most are working after that, some don't need because > > they need other packages for BPF to work and those are not available, so > > nevermind, lets have just the fix I provided, I'll add python3 and life > > goes on. > > > > - Arnaldo