xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v4 02/26] xen/x86: Script to automatically process featureset information
Date: Wed, 23 Mar 2016 16:36:05 +0000	[thread overview]
Message-ID: <1458750989-28967-3-git-send-email-andrew.cooper3@citrix.com> (raw)
In-Reply-To: <1458750989-28967-1-git-send-email-andrew.cooper3@citrix.com>

This script consumes include/public/arch-x86/cpufeatureset.h and generates a
single include/asm-x86/cpuid-autogen.h containing all the processed
information.

It currently generates just FEATURESET_NR_ENTRIES.  Future changes will
generate more information.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <JBeulich@suse.com>
---
v2:
 * New
v3:
 * Rebased over the new namespacing in cpufeatureset.h
v4:
 * Speeling fixes
---
 .gitignore                       |   1 +
 xen/include/Makefile             |  10 ++
 xen/include/asm-x86/cpufeature.h |   3 +-
 xen/tools/gen-cpuid.py           | 197 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 210 insertions(+), 1 deletion(-)
 create mode 100755 xen/tools/gen-cpuid.py

diff --git a/.gitignore b/.gitignore
index 91f690c..b40453e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -252,6 +252,7 @@ xen/include/headers.chk
 xen/include/headers++.chk
 xen/include/asm
 xen/include/asm-*/asm-offsets.h
+xen/include/asm-x86/cpuid-autogen.h
 xen/include/compat/*
 xen/include/config/
 xen/include/generated/
diff --git a/xen/include/Makefile b/xen/include/Makefile
index 9c8188b..268bc9d 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -117,5 +117,15 @@ headers++.chk: $(PUBLIC_HEADERS) Makefile
 
 endif
 
+ifeq ($(XEN_TARGET_ARCH),x86_64)
+
+$(BASEDIR)/include/asm-x86/cpuid-autogen.h: $(BASEDIR)/include/public/arch-x86/cpufeatureset.h $(BASEDIR)/tools/gen-cpuid.py FORCE
+	$(PYTHON) $(BASEDIR)/tools/gen-cpuid.py -i $^ -o $@.new
+	$(call move-if-changed,$@.new,$@)
+
+all: $(BASEDIR)/include/asm-x86/cpuid-autogen.h
+endif
+
 clean::
 	rm -rf compat headers.chk headers++.chk
+	rm -f $(BASEDIR)/include/asm-x86/cpuid-autogen.h
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index a044616..bcda09b 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -11,8 +11,9 @@
 
 #include <xen/const.h>
 #include <asm/cpufeatureset.h>
+#include <asm/cpuid-autogen.h>
 
-#define FSCAPINTS 9
+#define FSCAPINTS FEATURESET_NR_ENTRIES
 #define NCAPINTS (FSCAPINTS + 1) /* N 32-bit words worth of info */
 
 /* Other features, Xen-defined mapping. */
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
new file mode 100755
index 0000000..c6bd98d
--- /dev/null
+++ b/xen/tools/gen-cpuid.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import sys, os, re
+
+class Fail(Exception):
+    pass
+
+class State(object):
+
+    def __init__(self, input, output):
+
+        self.source = input
+        self.input  = open_file_or_fd(input, "r", 2)
+        self.output = open_file_or_fd(output, "w", 2)
+
+        # State parsed from input
+        self.names = {} # Name => value mapping
+
+        # State calculated
+        self.nr_entries = 0 # Number of words in a featureset
+
+def parse_definitions(state):
+    """
+    Parse featureset information from @param f and mutate the global
+    namespace with symbols
+    """
+    feat_regex = re.compile(
+        r"^XEN_CPUFEATURE\(([A-Z0-9_]+),"
+        "\s+([\s\d]+\*[\s\d]+\+[\s\d]+)\).*$")
+
+    this = sys.modules[__name__]
+
+    for l in state.input.readlines():
+        # Short circuit the regex...
+        if not l.startswith("XEN_CPUFEATURE("):
+            continue
+
+        res = feat_regex.match(l)
+
+        if res is None:
+            raise Fail("Failed to interpret '%s'" % (l.strip(), ))
+
+        name = res.groups()[0]
+        val = eval(res.groups()[1]) # Regex confines this to a very simple expression
+
+        if hasattr(this, name):
+            raise Fail("Duplicate symbol %s" % (name,))
+
+        if val in state.names:
+            raise Fail("Aliased value between %s and %s" %
+                       (name, state.names[val]))
+
+        # Mutate the current namespace to insert a feature literal with its
+        # bit index.  Prepend an underscore if the name starts with a digit.
+        if name[0] in "0123456789":
+            this_name = "_" + name
+        else:
+            this_name = name
+        setattr(this, this_name, val)
+
+        # Construct a reverse mapping of value to name
+        state.names[val] = name
+
+    if len(state.names) == 0:
+        raise Fail("No features found")
+
+def featureset_to_uint32s(fs, nr):
+    """ Represent a featureset as a list of C-compatible uint32_t's """
+
+    bitmap = 0L
+    for f in fs:
+        bitmap |= 1L << f
+
+    words = []
+    while bitmap:
+        words.append(bitmap & ((1L << 32) - 1))
+        bitmap >>= 32
+
+    assert len(words) <= nr
+
+    if len(words) < nr:
+        words.extend([0] * (nr - len(words)))
+
+    return [ "0x%08xU" % x for x in words ]
+
+def format_uint32s(words, indent):
+    """ Format a list of uint32_t's suitable for a macro definition """
+    spaces = " " * indent
+    return spaces + (", \\\n" + spaces).join(words) + ", \\"
+
+
+def crunch_numbers(state):
+
+    # Size of bitmaps
+    state.nr_entries = nr_entries = (max(state.names.keys()) >> 5) + 1
+
+
+def write_results(state):
+    state.output.write(
+"""/*
+ * Automatically generated by %s - Do not edit!
+ * Source data: %s
+ */
+#ifndef __XEN_X86__FEATURESET_DATA__
+#define __XEN_X86__FEATURESET_DATA__
+""" % (sys.argv[0], state.source))
+
+    state.output.write(
+"""
+#define FEATURESET_NR_ENTRIES %sU
+""" % (state.nr_entries,
+       ))
+
+    state.output.write(
+"""
+#endif /* __XEN_X86__FEATURESET_DATA__ */
+""")
+
+
+def open_file_or_fd(val, mode, buffering):
+    """
+    If 'val' looks like a decimal integer, open it as an fd.  If not, try to
+    open it as a regular file.
+    """
+
+    fd = -1
+    try:
+        # Does it look like an integer?
+        try:
+            fd = int(val, 10)
+        except ValueError:
+            pass
+
+        if fd == 0:
+            return sys.stdin
+        elif fd == 1:
+            return sys.stdout
+        elif fd == 2:
+            return sys.stderr
+
+        # Try to open it...
+        if fd != -1:
+            return os.fdopen(fd, mode, buffering)
+        else:
+            return open(val, mode, buffering)
+
+    except StandardError, e:
+        if fd != -1:
+            raise Fail("Unable to open fd %d: %s: %s" %
+                       (fd, e.__class__.__name__, e))
+        else:
+            raise Fail("Unable to open file '%s': %s: %s" %
+                       (val, e.__class__.__name__, e))
+
+    raise SystemExit(2)
+
+def main():
+    from optparse import OptionParser
+
+    # Change stdout to be line-buffered.
+    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 1)
+
+    parser = OptionParser(usage = "%prog [options] -i INPUT -o OUTPUT",
+                          description =
+                          "Process featureset information")
+
+    parser.add_option("-i", "--in", dest = "fin", metavar = "<FD or FILE>",
+                      default = "0",
+                      help = "Featureset definitions")
+    parser.add_option("-o", "--out", dest = "fout", metavar = "<FD or FILE>",
+                      default = "1",
+                      help = "Featureset calculated information")
+
+    opts, _ = parser.parse_args()
+
+    if opts.fin is None or opts.fout is None:
+        parser.print_help(sys.stderr)
+        raise SystemExit(1)
+
+    state = State(opts.fin, opts.fout)
+
+    parse_definitions(state)
+    crunch_numbers(state)
+    write_results(state)
+
+
+if __name__ == "__main__":
+    try:
+        sys.exit(main())
+    except Fail, e:
+        print >>sys.stderr, "%s:" % (sys.argv[0],), e
+        sys.exit(1)
+    except SystemExit, e:
+        sys.exit(e.code)
+    except KeyboardInterrupt:
+        sys.exit(2)
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

  parent reply	other threads:[~2016-03-23 16:36 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-23 16:36 [PATCH v4 00/26] x86: Improvements to cpuid handling for guests Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 01/26] xen/public: Export cpu featureset information in the public API Andrew Cooper
2016-03-24 14:08   ` Jan Beulich
2016-03-24 14:12     ` Andrew Cooper
2016-03-24 14:16       ` Jan Beulich
2016-03-23 16:36 ` Andrew Cooper [this message]
2016-03-23 16:36 ` [PATCH v4 03/26] xen/x86: Collect more cpuid feature leaves Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 04/26] xen/x86: Mask out unknown features from Xen's capabilities Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 05/26] xen/x86: Annotate special features Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 06/26] xen/x86: Annotate VM applicability in featureset Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 07/26] xen/x86: Calculate maximum host and guest featuresets Andrew Cooper
2016-03-29  8:57   ` Jan Beulich
2016-03-23 16:36 ` [PATCH v4 08/26] xen/x86: Generate deep dependencies of features Andrew Cooper
2016-03-24 16:16   ` Jan Beulich
2016-03-23 16:36 ` [PATCH v4 09/26] xen/x86: Clear dependent features when clearing a cpu cap Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 10/26] xen/x86: Improve disabling of features which have dependencies Andrew Cooper
2016-03-28 15:18   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 11/26] xen/x86: Improvements to in-hypervisor cpuid sanity checks Andrew Cooper
2016-03-24 15:38   ` Andrew Cooper
2016-03-24 16:47   ` Jan Beulich
2016-03-24 17:01     ` Andrew Cooper
2016-03-24 17:11       ` Jan Beulich
2016-03-24 17:12         ` Andrew Cooper
2016-03-28 15:29   ` Konrad Rzeszutek Wilk
2016-04-05 15:25     ` Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 12/26] x86/cpu: Move set_cpumask() calls into c_early_init() Andrew Cooper
2016-03-28 15:55   ` Konrad Rzeszutek Wilk
2016-04-05 16:19     ` Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 13/26] x86/cpu: Sysctl and common infrastructure for levelling context switching Andrew Cooper
2016-03-24 16:58   ` Jan Beulich
2016-03-28 16:12   ` Konrad Rzeszutek Wilk
2016-04-05 16:33     ` Andrew Cooper
2016-03-28 17:37   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 14/26] x86/cpu: Rework AMD masking MSR setup Andrew Cooper
2016-03-28 18:55   ` Konrad Rzeszutek Wilk
2016-04-05 16:44     ` Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 15/26] x86/cpu: Rework Intel masking/faulting setup Andrew Cooper
2016-03-28 19:14   ` Konrad Rzeszutek Wilk
2016-04-05 16:45     ` Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 16/26] x86/cpu: Context switch cpuid masks and faulting state in context_switch() Andrew Cooper
2016-03-28 19:27   ` Konrad Rzeszutek Wilk
2016-04-05 18:34     ` Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 17/26] x86/pv: Provide custom cpumasks for PV domains Andrew Cooper
2016-03-28 19:40   ` Konrad Rzeszutek Wilk
2016-04-05 16:55     ` Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 18/26] x86/domctl: Update PV domain cpumasks when setting cpuid policy Andrew Cooper
2016-03-24 17:04   ` Jan Beulich
2016-03-24 17:05     ` Andrew Cooper
2016-03-28 19:51   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 19/26] xen+tools: Export maximum host and guest cpu featuresets via SYSCTL Andrew Cooper
2016-03-28 19:59   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 20/26] tools/libxc: Modify bitmap operations to take void pointers Andrew Cooper
2016-03-28 20:05   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 21/26] tools/libxc: Use public/featureset.h for cpuid policy generation Andrew Cooper
2016-03-28 20:07   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 22/26] tools/libxc: Expose the automatically generated cpu featuremask information Andrew Cooper
2016-03-28 20:08   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 23/26] tools: Utility for dealing with featuresets Andrew Cooper
2016-03-28 20:26   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 24/26] tools/libxc: Wire a featureset through to cpuid policy logic Andrew Cooper
2016-03-28 20:39   ` Konrad Rzeszutek Wilk
2016-03-23 16:36 ` [PATCH v4 25/26] tools/libxc: Use featuresets rather than guesswork Andrew Cooper
2016-03-23 16:36 ` [PATCH v4 26/26] tools/libxc: Calculate xstate cpuid leaf from guest information Andrew Cooper
2016-03-24 17:20   ` Wei Liu
2016-03-31  7:48   ` Jan Beulich
2016-04-05 17:48     ` Andrew Cooper
2016-04-07  0:16       ` Jan Beulich
2016-04-07  0:40         ` Andrew Cooper
2016-04-07  0:56           ` Jan Beulich
2016-04-07 11:34             ` Andrew Cooper
2016-03-24 10:27 ` [PATCH v4 00/26] x86: Improvements to cpuid handling for guests Jan Beulich
2016-03-24 10:28   ` Andrew Cooper

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1458750989-28967-3-git-send-email-andrew.cooper3@citrix.com \
    --to=andrew.cooper3@citrix.com \
    --cc=xen-devel@lists.xen.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).