Linux-kselftest Archive on lore.kernel.org
 help / color / Atom feed
From: Knut Omang <knut.omang@oracle.com>
To: linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org, linux-kbuild@vger.kernel.org,
	Shuah Khan <shuah@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
	Masahiro Yamada <yamada.masahiro@socionext.com>,
	Michal Marek <michal.lkml@markovi.net>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Shreyans Devendra Doshi <0xinfosect0r@gmail.com>,
	Alan Maguire <alan.maguire@oracle.com>,
	Brendan Higgins <brendanhiggins@google.com>,
	Kevin Hilman <khilman@baylibre.com>,
	Hidenori Yamaji <hidenori.yamaji@sony.com>,
	Frank Rowand <frowand.list@gmail.com>,
	Timothy Bird <Tim.Bird@sony.com>,
	Luis Chamberlain <mcgrof@kernel.org>,
	"Theodore Ts'o" <tytso@mit.edu>, Daniel Vetter <daniel@ffwll.ch>,
	Stephen Boyd <sboyd@kernel.org>,
	Knut Omang <knut.omang@oracle.com>
Subject: [RFC 09/19] ktf: resolve: A helper utility to aid in exposing private kernel symbols to KTF tests.
Date: Tue, 13 Aug 2019 08:09:24 +0200
Message-ID: <09d79897772dd370a61967b03e83fa4b8663698b.1565676440.git-series.knut.omang@oracle.com> (raw)
In-Reply-To: <cover.92d76bb4f6dcedc971d0b72a49e8e459a98bca54.1565676440.git-series.knut.omang@oracle.com>

Takes an input file foo.txt with a list of symbols on the form:

#module foo
#header foo.h
private_foo_symbol_1
private_foo_symbol_2

and creates usable definitions to access these (requires CONFIG_KALLSYMS_ALL).
This is useful to be able to easily and cleanly test internal interfaces of a module.
Examples follows in the selftests later in the series.

Signed-off-by: Knut Omang <knut.omang@oracle.com>
---
 tools/testing/selftests/ktf/scripts/resolve | 188 +++++++++++++++++++++-
 1 file changed, 188 insertions(+)
 create mode 100755 tools/testing/selftests/ktf/scripts/resolve

diff --git a/tools/testing/selftests/ktf/scripts/resolve b/tools/testing/selftests/ktf/scripts/resolve
new file mode 100755
index 0000000..74005f7
--- /dev/null
+++ b/tools/testing/selftests/ktf/scripts/resolve
@@ -0,0 +1,188 @@
+#!/usr/bin/python
+
+# Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
+#    Author: Knut Omang <knut.omang@oracle.com>
+#
+# SPDX-License-Identifier: GPL-2.0
+#
+# A script to generate code and header definitions for
+# module and kernel symbols not directly exposed to KTF.
+# (See the documentation for KTF for details)
+#
+
+import os, sys, re, shutil, string, ConfigParser
+
+def usage():
+    print "Usage: resolve symbolfile outputfile"
+    exit(0)
+
+class FuncInfo:
+    def __init__(self, sym, re_sym, re_decl):
+        self.symbol = sym
+        self.re_sym = re_sym
+        self.re_decl = re_decl
+
+    def __repr__(self):
+        return "FuncInfo: [%s: %s]" % (self.symbol, self.re_decl)
+
+class Module:
+    def __init__(self, name, header):
+        self.cur_header = header
+        self.prefix = "Z"
+        self.name = name
+        self.symbols = {}  # Input:  headerfile -> symbol list
+        self.func_def = []  # FuncInfo list
+        self.debug = False
+        all_modules.append(self)
+
+    def log(self, str):
+        if self.debug:
+            print str
+
+    def SetHeader(self, header):
+        self.cur_header = header
+
+    def AddSymbol(self, sym):
+        try:
+            h = self.symbols[self.cur_header]
+            h.append(sym)
+        except:
+            self.symbols[self.cur_header] = [sym]
+
+    # Open along include path:
+    def Open(self, filename):
+        for p in includepath:
+            try:
+                f = os.path.join(p, filename)
+                self.log(" -- trying " + f)
+                header = open(f,'r')
+                return header
+            except:
+                continue
+        sys.stderr.write(" ** unable to open \"%s\"\n" % filename)
+        return None
+
+    # Parse the relevant header files for function defs:
+    def ParseDefs(self):
+        for hf in self.symbols.keys():
+            self.log(" ** Parsing %s:" % hf)
+            header = self.Open(hf)
+            if header == None:
+                return
+            content = header.read()
+            symbols = self.symbols[hf]
+            types = r"(extern|u8|u16|u32|u64|int|long|size_t|off_t|loff_t|void|struct|union\s)(.*[\*\s]"
+            funor = ")(" + "|".join(symbols) + r")(\([^\)]+\);)$"
+            tt = types + funor
+            s = re.compile(tt, re.MULTILINE)
+            miter = s.finditer(content)
+            s_count = 0
+            for m in miter:
+                sym = m.group(3)
+                re_name = "_".join([self.prefix, sym])
+                re_decl = "%s%s(*%s)%s" % (m.group(1), m.group(2), re_name, m.group(4))
+                self.func_def.append(FuncInfo(sym, re_name, re_decl))
+                s_count = s_count + 1
+
+            if s_count != len(symbols):
+                print " ** Warning: File %s: Found %d definitions from %d symbols!" % \
+                    (hf, s_count, len(symbols))
+                print " ** - please check/fix output manually!"
+
+    # Output functions:
+    def write_funcs(self, file):
+        for fi in self.func_def:
+            file.write("\t%s\n"% fi.re_decl)
+
+    def write_defines(self, file):
+        for fi in self.func_def:
+            file.write("#define %s ktf_syms.%s\n" % (fi.symbol, fi.re_sym))
+
+    def write_resolve_calls(self, file):
+        for fi in self.func_def:
+            file.write("\tktf_resolve_symbol(%s, %s);\n" % (self.name, fi.symbol))
+
+usage_h = False
+my_argv = []
+includepath = [""]
+
+for arg in sys.argv[1:]:
+    if arg == "-h":
+        usage_h = True
+        continue
+    incl = re.match(r"-I([^\s]+)", arg)
+    if incl != None:
+        includepath.append(incl.group(1))
+        continue
+    genopt = re.match(r"-([^\s]+)", arg)
+    if genopt != None:
+        # Silently ignore other cc options as we accept cc-flags-y:
+        #
+        continue
+    my_argv.append(arg)
+
+# Main program:
+
+if len(my_argv) != 2 or usage_h:
+    usage()
+
+symfile = my_argv[0]
+outputfile = my_argv[1]
+
+all_modules = []
+module = Module("kernel", None)  # Default, at the top of the file is the main kernel symbols
+header = None  # A header directive is required before any symbols
+
+try:
+    file = open(symfile, 'r')
+except:
+    print "Unable to open config file \"%s\"" % symfile
+    exit(1)
+for line in file:
+    match = re.match(r"^#(\w+) ([\w\.]+)\s*$", line)
+    if match != None:
+        cmd = match.group(1)
+        value = match.group(2)
+        if cmd == "module":
+            module = Module(value, header)
+        elif cmd == "header":
+            header = value
+            module.SetHeader(header)
+        else:
+            raise ResolveError("Unknown directive \"%s\"" % cmd)
+        #print "%s set to %s" % (cmd, value)
+        continue
+    match = re.match(r"\s*(\w+)\s*", line)
+    if match != None:
+        s = match.group(1)
+        module.AddSymbol(s)
+
+for m in all_modules:
+    m.ParseDefs()
+
+try:
+    output = open(outputfile, "w")
+except:
+    print "Failed to open output header file \"%s\"" % outputfile
+output.write('#ifndef _KTF_RESOLVE_H\n#define _KTF_RESOLVE_H\n')
+output.write('#include "ktf.h"\n\nstruct ktf_syms {\n')
+
+for m in all_modules:
+    m.write_funcs(output)
+
+output.write('};\n\n')
+
+for m in all_modules:
+    m.write_defines(output)
+
+output.write('\n\nextern struct ktf_syms ktf_syms;\n')
+output.write('\nint ktf_resolve_symbols(void);\n#ifndef KTF_CLIENT\n')
+output.write('struct ktf_syms ktf_syms;\n\n')
+
+output.write('\nint ktf_resolve_symbols(void)\n{\n')
+
+for m in all_modules:
+    m.write_resolve_calls(output)
+
+output.write('\treturn 0;\n}\n\n#endif /* !KTF_CLIENT */\n#endif /* _KTF_RESOLVE_H */ \n')
+output.close()
-- 
git-series 0.9.1

  parent reply index

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-13  6:09 [RFC 00/19] Integration of Kernel Test Framework (KTF) into the kernel tree Knut Omang
2019-08-13  6:09 ` [RFC 01/19] kbuild: Fixes to rules for host-cshlib and host-cxxshlib Knut Omang
2019-08-13 14:01   ` Masahiro Yamada
2019-08-13 16:19     ` Knut Omang
2019-08-14  2:02       ` Masahiro Yamada
2019-08-14  5:50         ` Knut Omang
2019-08-14  5:52         ` Knut Omang
2019-08-14 12:52           ` Knut Omang
2019-08-21  1:47             ` Masahiro Yamada
2019-08-21  4:03               ` Knut Omang
2019-08-13  6:09 ` [RFC 02/19] ktf: Introduce the main part of the kernel side of ktf Knut Omang
2019-09-09  1:23   ` Brendan Higgins
2019-09-10  6:15     ` Knut Omang
2019-08-13  6:09 ` [RFC 03/19] ktf: Introduce a generic netlink protocol for test result communication Knut Omang
2019-09-09  1:28   ` Brendan Higgins
2019-09-10  6:30     ` Knut Omang
2019-08-13  6:09 ` [RFC 04/19] ktf: An implementation of a generic associative array container Knut Omang
2019-08-13  6:09 ` [RFC 05/19] ktf: Implementation of ktf support for overriding function entry and return Knut Omang
2019-08-13  6:09 ` [RFC 06/19] ktf: A simple debugfs interface to test results Knut Omang
2019-08-13  8:21   ` Greg Kroah-Hartman
2019-08-14 17:17     ` Knut Omang
2019-08-15  8:49       ` Greg Kroah-Hartman
2019-08-15 10:35         ` Knut Omang
2019-08-15 10:52           ` Greg Kroah-Hartman
2019-08-13  6:09 ` [RFC 07/19] ktf: Simple coverage support Knut Omang
2019-08-13  6:09 ` [RFC 08/19] ktf: Configurable context support for network info setup Knut Omang
2019-08-13  6:09 ` Knut Omang [this message]
2019-08-13  6:09 ` [RFC 10/19] ktf: Add documentation for Kernel Test Framework (KTF) Knut Omang
2019-08-13  6:09 ` [RFC 11/19] ktf: Add a small test suite with a few tests to test KTF itself Knut Omang
2019-08-13  6:09 ` [RFC 12/19] ktf: Main part of user land library for executing tests Knut Omang
2019-08-13  6:09 ` [RFC 13/19] ktf: Integration logic for running ktf tests from googletest Knut Omang
2019-08-13  6:09 ` [RFC 14/19] ktf: Internal debugging facilities Knut Omang
2019-08-13  6:09 ` [RFC 15/19] ktf: Some simple examples Knut Omang
2019-08-13  6:09 ` [RFC 16/19] ktf: Some user applications to run tests Knut Omang
2019-08-13  6:09 ` [RFC 17/19] ktf: Toplevel ktf Makefile/makefile includes and scripts to run from kselftest Knut Omang
2019-08-13  6:09 ` [RFC 18/19] kselftests: Enable building ktf Knut Omang
2019-08-13  6:09 ` [RFC 19/19] Documentation/dev-tools: Add index entry for KTF documentation Knut Omang
2019-08-13  8:10 ` [RFC 00/19] Integration of Kernel Test Framework (KTF) into the kernel tree Brendan Higgins
2019-08-13  8:17 ` Brendan Higgins
2019-08-13 11:29   ` Knut Omang
2019-08-13 17:50     ` Brendan Higgins
2019-08-13  8:23 ` Greg Kroah-Hartman
2019-08-13  9:51   ` Knut Omang
2019-08-13 17:02     ` Brendan Higgins

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=09d79897772dd370a61967b03e83fa4b8663698b.1565676440.git-series.knut.omang@oracle.com \
    --to=knut.omang@oracle.com \
    --cc=0xinfosect0r@gmail.com \
    --cc=Tim.Bird@sony.com \
    --cc=alan.maguire@oracle.com \
    --cc=brendanhiggins@google.com \
    --cc=corbet@lwn.net \
    --cc=daniel@ffwll.ch \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hidenori.yamaji@sony.com \
    --cc=khilman@baylibre.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=mcgrof@kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=sboyd@kernel.org \
    --cc=shuah@kernel.org \
    --cc=tytso@mit.edu \
    --cc=yamada.masahiro@socionext.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-kselftest Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kselftest/0 linux-kselftest/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kselftest linux-kselftest/ https://lore.kernel.org/linux-kselftest \
		linux-kselftest@vger.kernel.org
	public-inbox-index linux-kselftest

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kselftest


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git