From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED,USER_AGENT_MUTT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1965C10F06 for ; Tue, 12 Mar 2019 00:39:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 870A120657 for ; Tue, 12 Mar 2019 00:39:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=joelfernandes.org header.i=@joelfernandes.org header.b="JI0daxOZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726630AbfCLAjQ (ORCPT ); Mon, 11 Mar 2019 20:39:16 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:46341 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726418AbfCLAjQ (ORCPT ); Mon, 11 Mar 2019 20:39:16 -0400 Received: by mail-qt1-f195.google.com with SMTP id z25so673143qti.13 for ; Mon, 11 Mar 2019 17:39:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ly3PyJhiRQi4oId3Jp/I4y9kAhs1iw828BmSysRKyZ8=; b=JI0daxOZRh0PkhQAbU8+ah7z11OagpX6fwhO38Lk6ozIcUs80I5Au1B/H+ckOkLRfN pv0Vr4WfmVRnGaAADux+nuB0lVD4e/ey885zZbAXm25lq5Ax7MRtanG/DDVzyzpxFcZ7 M7hooDTN5M3QxYhOreCsGfWXjuOpQFimpUSkU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ly3PyJhiRQi4oId3Jp/I4y9kAhs1iw828BmSysRKyZ8=; b=ZDiGvy+V0/08HUI1XjcpXZuO1vS7smv7cSbT4JjgPvC8fwaY/JFSUu8GvsLQXlUq3G x8bMXUveiP7RuAgTwds8EysjRuxlXVykRX+kDNbw0xZFhaRmYEyvip/gHuQKQWa3nKZ5 FdCT9dOrDarE3O+ZYAmfRcapArVZq35PlF28lNDdu3rPIlUJ5AaBDiAduskLy9vElbR3 5KUY853eydmXlfe7jqCTp0W02tiuPgNQvblWAOoFk9K9NHkst5SFYSBEsExJnezdjgJL k3JS+WcTQJcSk0BlkUeQyuxbacaEfFGssnututDmivVphVf3AYJ0MqtbWIABF5D1UREH TxRA== X-Gm-Message-State: APjAAAUsmIDAkJKN0Jzs9jBZNoabTSgOnMG7CXE0Nw460nPCq557nwRd ikL0dstm7w8EG2eCVXwvk8VFKA== X-Google-Smtp-Source: APXvYqz+iHMK77Oj/SvnFoapIG3HmPyK+b4/w/1OkGxjCjN0asdsP9LWAG5l+h/cqvR4h5vbjdEnBg== X-Received: by 2002:ac8:2527:: with SMTP id 36mr13155134qtm.308.1552351154545; Mon, 11 Mar 2019 17:39:14 -0700 (PDT) Received: from localhost ([2620:0:1004:1100:cca9:fccc:8667:9bdc]) by smtp.gmail.com with ESMTPSA id u48sm132343qtu.89.2019.03.11.17.39.13 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 11 Mar 2019 17:39:13 -0700 (PDT) Date: Mon, 11 Mar 2019 20:39:12 -0400 From: Joel Fernandes To: Daniel Colascione Cc: Steven Rostedt , Karim Yaghmour , Geert Uytterhoeven , Greg KH , LKML , Andrew Morton , Alexei Starovoitov , atish patra , Dan Williams , Dietmar Eggemann , Guenter Roeck , Jonathan Corbet , Kees Cook , Android Kernel Team , "open list:DOCUMENTATION" , "open list:KERNEL SELFTEST FRAMEWORK" , linux-trace-devel@vger.kernel.org, Manoj Rao , Masahiro Yamada , Masami Hiramatsu , Qais Yousef , Randy Dunlap , Shuah Khan , Yonghong Song Subject: Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel Message-ID: <20190312003912.GA170478@google.com> References: <20190308140251.GC25768@kroah.com> <20190309071648.GE3882@kroah.com> <20190309121141.GA30173@kroah.com> <3e84e1ef-e266-e983-5874-6c26ac7f38b8@opersys.com> <20190311193612.4f09bf11@oasis.local.home> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 11, 2019 at 04:58:28PM -0700, Daniel Colascione wrote: > On Mon, Mar 11, 2019 at 4:37 PM Steven Rostedt wrote: > > > > On Sat, 9 Mar 2019 16:44:31 -0500 > > Karim Yaghmour wrote: > > > > > > > Sorry, I should've been clearer. I'm including eBPF/BCC into the > > > "user-space tools" here. That was in fact my prime motivation in > > > encouraging Joel at the last LPC to look at this. I've been integrating > > > the teaching of eBPF into my AOSP debugging and performance analysis > > > class (see CC courseware here: > > > http://www.opersys.com/training/android-debug-and-performance), and it > > > was pretty messy trying to show people how to benefit from such tools > > > under Android. Joel's present set of patches would obviate this problem. > > > > I've been reading this thread and staying out for the most part. But I > > was thinking about how I could use kernel headers for things like > > kprobes, and I want to mention the pony that I would like to have :-) > > > > Are headers really needed, and more importantly, are they enough? What > > if userspace is 32bit and the kernel is 64bit. Can ebpf scripts handle > > that? > > Why would eBPF care about the bit width of userspace? I've thought a > lot about inspecting user state from the kernel and have some weird > ideas in that area (e.g., why not register eBPF programs to unwind > user stacks?) --- but I think that Joel's work is independent of all > that. I'm curious what you think we can do about userspace. I wish we > could just compile the world with frame pointers, but we can't. Yeah, most of the usecases for eBPF tracing are all instrumenting the kernel using kprobes. There are some usecases that instrument userspace with uprobes but those don't need headers. > > What I would love, is a table that has all structures and their fields > > with information about their types, size and signed types. Like the > > format fields in the events directory. This way ebpf (and kprobes, > > internally in the kernel) could access this table and be able to know > > what the data structures of the kernel is). > > Think of the headers as encoding this information and more and the C > compiler as a magical decoder ring. :-) I totally get the desire for a > metadata format a little less messy than C code, but as a practical > matter, a rich C-compilation pipeline already exists, and the > debuginfo you're proposing doesn't, except in the form of DWARF, which > I think would be even more controversial to embed in the kernel memory > image than headers are. A header blob provides a strict superset of > the information your scheme provides. I think even though the kernel-headers can't have information about all data structures, they do already contain a lot of data structure definitions we need already. And anything needed can/should arguably be moved to include/ if they are really needed for kernel extension by something "external" to the kernel such as kernel modules or eBPF, right? In any case, such a solution such as what Steve suggested, still cannot do what we can with headers - such as build kernel modules on the fly using the C-compiler without any auto-generation of C code from any debug artifiacts. Think systemtap working with the module-backend without any need for linux-headers package on the file system. So such a solution would still be a bit orthogonal in scope to what this proposed solution can solve IMO. thanks, - Joel