From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v3 04/28] xen/x86: Script to automatically process featureset information
Date: Tue, 15 Mar 2016 15:35:00 +0000 [thread overview]
Message-ID: <1458056124-8024-5-git-send-email-andrew.cooper3@citrix.com> (raw)
In-Reply-To: <1458056124-8024-1-git-send-email-andrew.cooper3@citrix.com>
This script consumes include/public/arch-x86/cpufeatureset.h and generates a
single include/asm-x86/cpuid-autogen.h containing all the processed
information.
It currently generates just FEATURESET_NR_ENTRIES. Future changes will
generate more information.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <JBeulich@suse.com>
---
v3:
* Rebased over the new namespacing in cpufeatureset.h
For all intents and purposes, new in v2. All generated information is now
expressed by #defines (using C structure initialisers for most) and contained
in a single header file.
---
.gitignore | 1 +
xen/include/Makefile | 10 ++
xen/include/asm-x86/cpufeature.h | 3 +-
xen/tools/gen-cpuid.py | 197 +++++++++++++++++++++++++++++++++++++++
4 files changed, 210 insertions(+), 1 deletion(-)
create mode 100755 xen/tools/gen-cpuid.py
diff --git a/.gitignore b/.gitignore
index 91f690c..b40453e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -252,6 +252,7 @@ xen/include/headers.chk
xen/include/headers++.chk
xen/include/asm
xen/include/asm-*/asm-offsets.h
+xen/include/asm-x86/cpuid-autogen.h
xen/include/compat/*
xen/include/config/
xen/include/generated/
diff --git a/xen/include/Makefile b/xen/include/Makefile
index 9c8188b..268bc9d 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -117,5 +117,15 @@ headers++.chk: $(PUBLIC_HEADERS) Makefile
endif
+ifeq ($(XEN_TARGET_ARCH),x86_64)
+
+$(BASEDIR)/include/asm-x86/cpuid-autogen.h: $(BASEDIR)/include/public/arch-x86/cpufeatureset.h $(BASEDIR)/tools/gen-cpuid.py FORCE
+ $(PYTHON) $(BASEDIR)/tools/gen-cpuid.py -i $^ -o $@.new
+ $(call move-if-changed,$@.new,$@)
+
+all: $(BASEDIR)/include/asm-x86/cpuid-autogen.h
+endif
+
clean::
rm -rf compat headers.chk headers++.chk
+ rm -f $(BASEDIR)/include/asm-x86/cpuid-autogen.h
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index d9ff92a..aae15fc 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -11,8 +11,9 @@
#include <xen/const.h>
#include <asm/cpufeatureset.h>
+#include <asm/cpuid-autogen.h>
-#define FSCAPINTS 9
+#define FSCAPINTS FEATURESET_NR_ENTRIES
#define NCAPINTS (FSCAPINTS + 1) /* N 32-bit words worth of info */
/* Other features, Linux-defined mapping, FSMAX+1 */
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
new file mode 100755
index 0000000..fa32bf5
--- /dev/null
+++ b/xen/tools/gen-cpuid.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import sys, os, re
+
+class Fail(Exception):
+ pass
+
+class State(object):
+
+ def __init__(self, input, output):
+
+ self.source = input
+ self.input = open_file_or_fd(input, "r", 2)
+ self.output = open_file_or_fd(output, "w", 2)
+
+ # State parsed from input
+ self.names = {} # Name => value mapping
+
+ # State calculated
+ self.nr_entries = 0 # Number of words in a featureset
+
+def parse_definitions(state):
+ """
+ Parse featureset information from @param f and mutate the global
+ namespace with symbols
+ """
+ feat_regex = re.compile(
+ r"^XEN_CPUFEATURE\(([A-Z0-9_]+),"
+ "\s+([\s\d]+\*[\s\d]+\+[\s\d]+)\).*$")
+
+ this = sys.modules[__name__]
+
+ for l in state.input.readlines():
+ # Short circuit the regex...
+ if not l.startswith("XEN_CPUFEATURE("):
+ continue
+
+ res = feat_regex.match(l)
+
+ if res is None:
+ raise Fail("Failed to interpret '%s'" % (l.strip(), ))
+
+ name = res.groups()[0]
+ val = eval(res.groups()[1]) # Regex confines this to a very simple expression
+
+ if hasattr(this, name):
+ raise Fail("Duplicate symbol %s" % (name,))
+
+ if val in state.names:
+ raise Fail("Aliased value between %s and %s" %
+ (name, state.names[val]))
+
+ # Mutate the current namespace to insert a feature literal with its
+ # bit index. Prepend an underscore if the name starts with a digit.
+ if name[0] in "0123456789":
+ this_name = "_" + name
+ else:
+ this_name = name
+ setattr(this, this_name, val)
+
+ # Construct a reverse mapping of value to name
+ state.names[val] = name
+
+ if len(state.names) == 0:
+ raise Fail("No features found")
+
+def featureset_to_uint32s(fs, nr):
+ """ Represent a featureset as a list of C-compatible uint32_t's """
+
+ bitmap = 0L
+ for f in fs:
+ bitmap |= 1L << f
+
+ words = []
+ while bitmap:
+ words.append(bitmap & ((1L << 32) - 1))
+ bitmap >>= 32
+
+ assert len(words) <= nr
+
+ if len(words) < nr:
+ words.extend([0] * (nr - len(words)))
+
+ return [ "0x%08xU" % x for x in words ]
+
+def format_uint32s(words, indent):
+ """ Format a list of uint32_t's sutable for a macro definition """
+ spaces = " " * indent
+ return spaces + (", \\\n" + spaces).join(words) + ", \\"
+
+
+def crunch_numbers(state):
+
+ # Size of bitmaps
+ state.nr_entries = nr_entries = (max(state.names.keys()) >> 5) + 1
+
+
+def write_results(state):
+ state.output.write(
+"""/*
+ * Automatically generated by %s - Do not edit!
+ * Source data: %s
+ */
+#ifndef __XEN_X86__FEATURESET_DATA__
+#define __XEN_X86__FEATURESET_DATA__
+""" % (sys.argv[0], state.source))
+
+ state.output.write(
+"""
+#define FEATURESET_NR_ENTRIES %sU
+""" % (state.nr_entries,
+ ))
+
+ state.output.write(
+"""
+#endif /* __XEN_X86__FEATURESET_DATA__ */
+""")
+
+
+def open_file_or_fd(val, mode, buffering):
+ """
+ If 'val' looks like a decimal integer, open it as an fd. If not, try to
+ open it as a regular file.
+ """
+
+ fd = -1
+ try:
+ # Does it look like an integer?
+ try:
+ fd = int(val, 10)
+ except ValueError:
+ pass
+
+ if fd == 0:
+ return sys.stdin
+ elif fd == 1:
+ return sys.stdout
+ elif fd == 2:
+ return sys.stderr
+
+ # Try to open it...
+ if fd != -1:
+ return os.fdopen(fd, mode, buffering)
+ else:
+ return open(val, mode, buffering)
+
+ except StandardError, e:
+ if fd != -1:
+ raise Fail("Unable to open fd %d: %s: %s" %
+ (fd, e.__class__.__name__, e))
+ else:
+ raise Fail("Unable to open file '%s': %s: %s" %
+ (val, e.__class__.__name__, e))
+
+ raise SystemExit(2)
+
+def main():
+ from optparse import OptionParser
+
+ # Change stdout to be line-buffered.
+ sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 1)
+
+ parser = OptionParser(usage = "%prog [options] -i INPUT -o OUTPUT",
+ description =
+ "Process featureset information")
+
+ parser.add_option("-i", "--in", dest = "fin", metavar = "<FD or FILE>",
+ default = "0",
+ help = "Featureset definitions")
+ parser.add_option("-o", "--out", dest = "fout", metavar = "<FD or FILE>",
+ default = "1",
+ help = "Featureset calculated information")
+
+ opts, _ = parser.parse_args()
+
+ if opts.fin is None or opts.fout is None:
+ parser.print_help(sys.stderr)
+ raise SystemExit(1)
+
+ state = State(opts.fin, opts.fout)
+
+ parse_definitions(state)
+ crunch_numbers(state)
+ write_results(state)
+
+
+if __name__ == "__main__":
+ try:
+ sys.exit(main())
+ except Fail, e:
+ print >>sys.stderr, "%s:" % (sys.argv[0],), e
+ sys.exit(1)
+ except SystemExit, e:
+ sys.exit(e.code)
+ except KeyboardInterrupt:
+ sys.exit(2)
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
next prev parent reply other threads:[~2016-03-15 15:35 UTC|newest]
Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-15 15:34 [PATCH RFC v3 00/28] x86: Improvements to cpuid handling for guests Andrew Cooper
2016-03-15 15:34 ` [PATCH v3 01/28] xen/x86: Drop unused and non-useful feature definitions Andrew Cooper
2016-03-16 7:53 ` Konrad Rzeszutek Wilk
2016-03-16 9:49 ` Andrew Cooper
2016-03-22 14:06 ` Doug Goldstein
2016-03-22 14:38 ` Jan Beulich
2016-03-15 15:34 ` [PATCH v3 02/28] xen/x86: Rename features to be closer to the vendor definitions Andrew Cooper
2016-03-16 8:01 ` Konrad Rzeszutek Wilk
2016-03-17 19:46 ` Andrew Cooper
2016-03-15 15:34 ` [PATCH v3 03/28] xen/public: Export cpu featureset information in the public API Andrew Cooper
2016-03-16 8:32 ` Konrad Rzeszutek Wilk
2016-03-22 10:39 ` Andrew Cooper
2016-03-18 15:52 ` Jan Beulich
2016-03-15 15:35 ` Andrew Cooper [this message]
2016-03-16 8:41 ` [PATCH v3 04/28] xen/x86: Script to automatically process featureset information Konrad Rzeszutek Wilk
2016-03-15 15:35 ` [PATCH v3 05/28] xen/x86: Collect more cpuid feature leaves Andrew Cooper
2016-03-16 8:50 ` Konrad Rzeszutek Wilk
2016-03-15 15:35 ` [PATCH v3 06/28] xen/x86: Mask out unknown features from Xen's capabilities Andrew Cooper
2016-03-16 18:01 ` Konrad Rzeszutek Wilk
2016-03-15 15:35 ` [PATCH v3 07/28] xen/x86: Annotate special features Andrew Cooper
2016-03-16 18:04 ` Konrad Rzeszutek Wilk
2016-03-18 16:29 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 08/28] xen/x86: Annotate VM applicability in featureset Andrew Cooper
2016-03-16 18:15 ` Konrad Rzeszutek Wilk
2016-03-18 16:57 ` Jan Beulich
2016-03-18 18:56 ` Andrew Cooper
2016-03-21 11:53 ` Jan Beulich
2016-03-21 13:39 ` Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 09/28] xen/x86: Calculate maximum host and guest featuresets Andrew Cooper
2016-03-16 18:24 ` Konrad Rzeszutek Wilk
2016-03-18 17:09 ` Jan Beulich
2016-03-22 11:23 ` Andrew Cooper
2016-03-22 12:39 ` Jan Beulich
2016-03-22 14:37 ` Andrew Cooper
2016-03-22 14:52 ` Jan Beulich
2016-03-22 15:01 ` Andrew Cooper
2016-03-22 16:10 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 10/28] xen/x86: Generate deep dependencies of features Andrew Cooper
2016-03-17 19:45 ` Konrad Rzeszutek Wilk
2016-03-17 20:14 ` Andrew Cooper
2016-03-17 20:32 ` Konrad Rzeszutek Wilk
2016-03-21 15:41 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 11/28] xen/x86: Clear dependent features when clearing a cpu cap Andrew Cooper
2016-03-17 19:51 ` Konrad Rzeszutek Wilk
2016-03-17 19:56 ` Andrew Cooper
2016-03-28 15:02 ` Konrad Rzeszutek Wilk
2016-03-21 15:45 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 12/28] xen/x86: Improve disabling of features which have dependencies Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 13/28] xen/x86: Improvements to in-hypervisor cpuid sanity checks Andrew Cooper
2016-03-21 16:11 ` Jan Beulich
2016-03-22 15:30 ` Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 14/28] x86/cpu: Move set_cpumask() calls into c_early_init() Andrew Cooper
2016-03-21 16:16 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 15/28] x86/cpu: Sysctl and common infrastructure for levelling context switching Andrew Cooper
2016-03-15 17:35 ` Joao Martins
2016-03-15 19:29 ` Andrew Cooper
2016-03-15 19:34 ` Joao Martins
2016-03-21 16:23 ` Jan Beulich
2016-03-22 15:57 ` Andrew Cooper
2016-03-22 16:16 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 16/28] x86/cpu: Rework AMD masking MSR setup Andrew Cooper
2016-03-21 16:51 ` Jan Beulich
2016-03-21 16:55 ` Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 17/28] x86/cpu: Rework Intel masking/faulting setup Andrew Cooper
2016-03-21 16:44 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 18/28] x86/cpu: Context switch cpuid masks and faulting state in context_switch() Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 19/28] x86/pv: Provide custom cpumasks for PV domains Andrew Cooper
2016-03-21 16:53 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 20/28] x86/domctl: Update PV domain cpumasks when setting cpuid policy Andrew Cooper
2016-03-21 17:06 ` Jan Beulich
2016-03-22 16:37 ` Andrew Cooper
2016-03-22 16:51 ` Jan Beulich
2016-03-15 15:35 ` [PATCH v3 21/28] xen+tools: Export maximum host and guest cpu featuresets via SYSCTL Andrew Cooper
2016-03-16 18:23 ` Wei Liu
2016-03-16 20:38 ` David Scott
2016-03-22 8:43 ` Jan Beulich
2016-03-22 20:39 ` Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 22/28] tools/libxc: Modify bitmap operations to take void pointers Andrew Cooper
2016-03-16 15:24 ` Andrew Cooper
2016-03-17 12:17 ` Wei Liu
2016-03-15 15:35 ` [PATCH v3 23/28] tools/libxc: Use public/featureset.h for cpuid policy generation Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 24/28] tools/libxc: Expose the automatically generated cpu featuremask information Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 25/28] tools: Utility for dealing with featuresets Andrew Cooper
2016-03-16 18:23 ` Wei Liu
2016-03-15 15:35 ` [PATCH v3 26/28] tools/libxc: Wire a featureset through to cpuid policy logic Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 27/28] tools/libxc: Use featuresets rather than guesswork Andrew Cooper
2016-03-15 15:35 ` [PATCH v3 28/28] tools/libxc: Calculate xstate cpuid leaf from guest information Andrew Cooper
2016-03-16 18:23 ` Wei Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1458056124-8024-5-git-send-email-andrew.cooper3@citrix.com \
--to=andrew.cooper3@citrix.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).