[RFC PATCH 0/3] Add Documentation for /proc/cpuinfo flags

[RFC PATCH 0/3] Add Documentation for /proc/cpuinfo flags

[Kyung Min Park]

Currently, there isn't any documentation in the kernel about how
/proc/cpuinfo flags are generated and what it means when they are
missing. x86 maintainers have expressed objections on adding flags
for features without kernel use cases and that part of the reason is
the lack of documentation on the logic behind such flags.

If users want to know if a feature is available on a given system, they
try to find the flag in /proc/cpuinfo. If a given flag is present, it
means that Linux supports it. If such flag represents a hardware feature,
it also means that the hardware supports it. Therefore, it is not
sufficient to read CPUID to determine whether a hardware feature is
available. If the flag is missing in /proc/cpuinfo, users need to find
out the reason why the flag is missing and how to enable it, which is
not always easy.

/proc/cpuinfo should show all of the hardware features and bugs that any
application or end user might care about. Some of these flags are derived
from CPUID, and others are derived from other sources, including some that
are entirely software-based. Add documentation how features are created,
naming of the flags and when they are missing, what it means to users.

Include two instances of features for which there are not implemented
use cases in the kernel.

Patch 1 creates a new documentation for /proc/cpuinfo flags bits.
Patch 2 adds X86_FEATURE_SERIALIZE.
Patch 3 adds X86_FEATURE_TSXLDTRK.

This RFC series has been reviewed by Dave Hansen.

Kyung Min Park (2):
  Documentation/x86: Add documentation for /proc/cpuinfo feature flags
  x86/cpufeatures: Enumerate TSX suspend load address tracking
    instructions

Ricardo Neri (1):
  x86/cpufeatures: Add enumeration for SERIALIZE instruction

 Documentation/x86/cpuinfo.rst      | 152 +++++++++++++++++++++++++++++
 Documentation/x86/index.rst        |   1 +
 arch/x86/include/asm/cpufeatures.h |   2 +
 3 files changed, 155 insertions(+)
 create mode 100644 Documentation/x86/cpuinfo.rst

-- 
2.17.1

[RFC PATCH 1/3] Documentation/x86: Add documentation for /proc/cpuinfo feature flags

[Kyung Min Park]

Add documentation for /proc/cpuinfo feature flags enumeration.
Document how and when x86 feature flags are used. Also discuss what
their presence or absence mean for the kernel, users, and applications.

Suggested-by: Dave Hansen <dave.hansen@intel.com>
Co-developed-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Signed-off-by: Kyung Min Park <kyung.min.park@intel.com>
---
 Documentation/x86/cpuinfo.rst | 152 ++++++++++++++++++++++++++++++++++
 Documentation/x86/index.rst   |   1 +
 2 files changed, 153 insertions(+)
 create mode 100644 Documentation/x86/cpuinfo.rst

diff --git a/Documentation/x86/cpuinfo.rst b/Documentation/x86/cpuinfo.rst
new file mode 100644
index 000000000000..d01d2c03a4d7
--- /dev/null
+++ b/Documentation/x86/cpuinfo.rst
@@ -0,0 +1,152 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+x86 Feature Flags
+=================
+
+Introduction
+============
+
+On x86, flags appearing in /proc/cpuinfo have an X86_FEATURE definition
+in arch/x86/include/asm/cpufeatures.h. If the kernel, any application,
+or an end user might care about a feature, it can and should have
+X86_FEATURE_* defined. These flags represent hardware features as
+well as software features.
+
+If users want to know if a feature is available on a given system, they
+try to find the flag in /proc/cpuinfo. If a given flag is present, it
+means that the kernel supports it and is currently making it available.
+If such flag represents a hardware feature, it also means that the
+hardware supports it.
+
+If the expected flag does not appear in /proc/cpuinfo, things are murkier.
+Users need to find out the reason why the flag is missing and find the way
+how to enable it, which is not always easy. There are several factors that
+can explain missing flags: the expected feature failed to enable, the feature
+is missing in hardware, platform firmware did not enable it, the feature is
+disabled at build or run time, or an old kernel is in use. In such cases,
+the users need to rely on tools like http://www.etallen.com/cpuid.html
+(which is not updated with kernel releases) or other custom tools that
+read CPUID.
+
+How are feature flags created?
+==============================
+
+a: Feature flags can be derived from the contents of CPUID leaves.
+------------------------------------------------------------------
+These feature definitions are organized mirroring the layout of CPUID
+leaves and grouped in words with offsets as mapped in enum cpuid_leafs
+in cpufeatures.h (see arch/x86/include/asm/cpufeatures.h for details).
+If a feature is defined with a X86_FEATURE_<name> definition in
+cpufeatures.h, and if it is detected at run time, the flags will be
+displayed accordingly in /proc/cpuinfo. For example, the flag "avx2"
+comes from X86_FEATURE_AVX2 in cpufeatures.h.
+
+b: Flags can be from scattered CPUID-based features.
+----------------------------------------------------
+Hardware features enumerated in sparsely populated CPUID leaves get
+software-defined values. Still, CPUID needs to be queried to determine
+if a given feature is present. This is done in init_scattered_cpuid_features().
+For instance, X86_FEATURE_CQM_LLC is defined as 11*32 + 0 and its presence is
+checked at runtime in the respective CPUID leaf [EAX=f, ECX=0] bit EDX[1].
+
+The intent of scattering CPUID leaves is to not bloat struct
+cpuinfo_x86.x86_capability[] unnecessarily. For instance, the CPUID leaf
+[EAX=7, ECX=0] has 30 features and is dense, but the CPUID leaf [EAX=7, EAX=1]
+has only one feature and would waste 31 bits of space in the x86_capability[]
+array.
+
+c: Flags can be created synthetically under certain conditions for hardware features.
+-------------------------------------------------------------------------------------
+Examples of conditions include whether certain features are present in
+MSR_IA32_CORE_CAPS or specific CPU models are identified. If the needed
+conditions are met, the features are enabled by the macro set_cpu_cap or
+setup_force_cpu_cap macro. For example, if bit 5 is set in MSR_IA32_CORE_CAPS,
+the feature X86_FEATURE_SPLIT_LOCK_DETECT will be enabled and
+"split_lock_detect" will be displayed. The flag "ring3mwait" will be
+displayed only when running on INTEL_FAM6_XEON_PHI_[KNL|KNM] processors.
+
+d: Flags can represent purely software features.
+------------------------------------------------
+These flags do not represent hardware features. Instead, they represent a
+software feature implemented in the kernel. For example, Kernel Page Table
+Isolation is purely software feature and its feature flag X86_FEATURE_PTI is
+also defined in cpufeatures.h.
+
+Naming of Flags
+===============
+
+The script arch/x86/kernel/cpu/mkcapflags.sh processes the
+#define X86_FEATURE_<name> from cpufeatures.h and generates the
+x86_cap/bug_flags[] arrays in kernel/cpu/capflags.c. The names in the
+resulting x86_cap/bug_flags[] are used to populate /proc/cpuinfo. The naming
+of flags in the x86_cap/bug_flags[] are as follows:
+
+a: The name of the flag is from the string in X86_FEATURE_<name> by default.
+----------------------------------------------------------------------------
+By default, the flag <name> in /proc/cpuinfo is extracted from the respective
+X86_FEATURE_<name> in cpufeatures.h. For example, the flag "avx2" is from
+X86_FEATURE_AVX2.
+
+b: The naming can be overridden.
+--------------------------------
+If the comment on the line for the #define X86_FEATURE_* starts with a
+double-quote character (""), the string inside the double-quote characters
+will be the name of the flags. For example, the flag "sse4_1" comes from
+the comment "sse4_1" following the X86_FEATURE_XMM4_1 definition.
+
+There are situations in which overriding the displayed name of the flag is
+needed. For instance, /proc/cpuinfo is a userspace interface and must remain
+constant. If, for some reason, the naming of X86_FEATURE_<name> changes, one
+shall override the new naming with the name already used in /proc/cpuinfo.
+
+c: The naming override can be "", which means it will not appear in /proc/cpuinfo.
+----------------------------------------------------------------------------------
+The feature shall be omitted from /proc/cpuinfo if it does not make sense for
+the feature to be exposed to userspace. For example, X86_FEATURE_ALWAYS is
+defined in cpufeatures.h but that flag is an internal kernel feature used
+in the alternative runtime patching functionality. So, its name is overridden
+with "". Its flag will not appear in /proc/cpuinfo.
+
+Flags are missing when one or more of these happen
+==================================================
+
+a: The hardware does not enumerate support for it.
+--------------------------------------------------
+For example, when a new kernel is running on old hardware or the feature is
+not enabled by boot firmware. Even if the hardware is new, there might be a
+problem enabling the feature at run time, the flag will not be displayed.
+
+b: The kernel does not know about the flag.
+-------------------------------------------
+For example, when an old kernel is running on new hardware.
+
+c: The kernel disabled support for it at compile-time.
+------------------------------------------------------
+For example, if 5-level-paging is not enabled when building (i.e.,
+CONFIG_X86_5LEVEL is not selected) the flag "la57" will not show up [#f1]_.
+Even though the feature will still be detected via CPUID, the kernel disables
+it via cleared by setup_clear_cpu_cap(X86_FEATURE_LA57).
+
+d: The feature is disabled at boot-time.
+----------------------------------------
+A feature can be disabled either using a command-line parameter or because
+it failed to be enabled. The command-line parameter clearcpuid= can be used
+to disable features using the feature number as defined in
+/arch/x86/include/asm/cpufeatures.h. For instance, User Mode Instruction
+Protection can be disabled using clearcpuid=514. The number 514 is calculated
+from #define X86_FEATURE_UMIP (16*32 + 2).
+
+In addition, there exist a variety of custom command-line parameters that
+disable specific features. The list of parameters includes, but is not limited
+to, no5lvl, nosmap, and nosmep. 5-level paging can also be disabled using
+"no5lvl". SMAP and SMEP are disabled with the aforementioned parameters,
+respectively.
+
+e: The feature was known to be non-functional.
+----------------------------------------------
+The feature was known to be non-functional because a dependency was
+missing at runtime. For example, AVX flags will not show up if XSAVE feature
+is disabled since they depend on XSAVE feature.
+
+.. [#f1] 5-level paging uses linear address of 57 bits.
diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst
index 265d9e9a093b..d5adb0ab8668 100644
--- a/Documentation/x86/index.rst
+++ b/Documentation/x86/index.rst
@@ -9,6 +9,7 @@ x86-specific Documentation
    :numbered:
 
    boot
+   cpuinfo
    topology
    exception-tables
    kernel-stacks
-- 
2.17.1

[RFC PATCH 2/3] x86/cpufeatures: Add enumeration for SERIALIZE instruction

[Kyung Min Park]

From: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>

This instruction gives software a way to force the processor to complete
all modifications to flags, registers and memory from previous instructions
and drain all buffered writes to memory before the next instruction is
fetched and executed.

The same effect can be obtained using the cpuid instruction. However,
cpuid causes modification on the eax, ebx, ecx, and ecx regiters; it
also causes a VM exit.

A processor supports SERIALIZE instruction if CPUID.0x0x.0x0:EDX[14] is
present. The CPU feature flag is shown as "serialize" in /proc/cpuinfo.

Detailed information on the instructions and CPUID feature flag SERIALIZE
can be found in the latest Intel Architecture Instruction Set Extensions
and Future Features Programming Reference and Intel 64 and IA-32
Architectures Software Developer's Manual.

Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index db189945e9b0..cd9b1ec022ec 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -364,6 +364,7 @@
 #define X86_FEATURE_AVX512_VP2INTERSECT (18*32+ 8) /* AVX-512 Intersect for D/Q */
 #define X86_FEATURE_MD_CLEAR		(18*32+10) /* VERW clears CPU buffers */
 #define X86_FEATURE_TSX_FORCE_ABORT	(18*32+13) /* "" TSX_FORCE_ABORT */
+#define X86_FEATURE_SERIALIZE		(18*32+14) /* SERIALIZE instruction */
 #define X86_FEATURE_PCONFIG		(18*32+18) /* Intel PCONFIG */
 #define X86_FEATURE_SPEC_CTRL		(18*32+26) /* "" Speculation Control (IBRS + IBPB) */
 #define X86_FEATURE_INTEL_STIBP		(18*32+27) /* "" Single Thread Indirect Branch Predictors */
-- 
2.17.1

[RFC PATCH 3/3] x86/cpufeatures: Enumerate TSX suspend load address tracking instructions

[Kyung Min Park]

Intel TSX suspend load tracking instructions aim to give a way to
choose which memory accesses do not need to be tracked in the TSX
read set. Add TSX suspend load tracking CPUID feature flag TSXLDTRK
for enumeration.

A processor supports Intel TSX suspend load address tracking if
CPUID.0x07.0x0:EDX[16] is present. Two instructions XSUSLDTRK, XRESLDTRK
are available when this feature is present.

The CPU feature flag is shown as "tsxldtrk" in /proc/cpuinfo.

Detailed information on the instructions and CPUID feature flag TSXLDTRK
can be found in the latest Intel Architecture Instruction Set Extensions
and Future Features Programming Reference and Intel 64 and IA-32
Architectures Software Developer's Manual.

Signed-off-by: Kyung Min Park <kyung.min.park@intel.com>
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index cd9b1ec022ec..8d850d58ea3b 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -365,6 +365,7 @@
 #define X86_FEATURE_MD_CLEAR		(18*32+10) /* VERW clears CPU buffers */
 #define X86_FEATURE_TSX_FORCE_ABORT	(18*32+13) /* "" TSX_FORCE_ABORT */
 #define X86_FEATURE_SERIALIZE		(18*32+14) /* SERIALIZE instruction */
+#define X86_FEATURE_TSXLDTRK		(18*32+16) /* TSX Suspend Load Address Tracking */
 #define X86_FEATURE_PCONFIG		(18*32+18) /* Intel PCONFIG */
 #define X86_FEATURE_SPEC_CTRL		(18*32+26) /* "" Speculation Control (IBRS + IBPB) */
 #define X86_FEATURE_INTEL_STIBP		(18*32+27) /* "" Single Thread Indirect Branch Predictors */
-- 
2.17.1

Re: [RFC PATCH 0/3] Add Documentation for /proc/cpuinfo flags

[Borislav Petkov]

On Wed, Jun 10, 2020 at 01:06:58PM -0700, Kyung Min Park wrote:
> Include two instances of features for which there are not implemented
> use cases in the kernel.
> 
> Patch 1 creates a new documentation for /proc/cpuinfo flags bits.
> Patch 2 adds X86_FEATURE_SERIALIZE.
> Patch 3 adds X86_FEATURE_TSXLDTRK.
> 
> This RFC series has been reviewed by Dave Hansen.

Yeah, and he and I talked about this on IRC. If you really want to dump
CPUID to see what's there, we should do a userspace tool in tools/ or
even use the ones which are already out there: x86info, cpuid, ...

Just adding X86_FEATURE_* flags without usage in the kernel makes no
sense and will cause unnecessary bloat and doesn't help one bit because
userspace simply calls CPUID directly.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [RFC PATCH 0/3] Add Documentation for /proc/cpuinfo flags

[Greg KH]

On Wed, Jun 10, 2020 at 01:06:58PM -0700, Kyung Min Park wrote:
> This RFC series has been reviewed by Dave Hansen.

Then why isn't there a "Reviewed-by:" line with his name on it on the
patches?

Come on, you all know how to do this properly...

Re: [RFC PATCH 0/3] Add Documentation for /proc/cpuinfo flags

[Dave Hansen]

On 6/10/20 1:35 PM, Borislav Petkov wrote:
> On Wed, Jun 10, 2020 at 01:06:58PM -0700, Kyung Min Park wrote:
>> Include two instances of features for which there are not implemented
>> use cases in the kernel.
>>
>> Patch 1 creates a new documentation for /proc/cpuinfo flags bits.
>> Patch 2 adds X86_FEATURE_SERIALIZE.
>> Patch 3 adds X86_FEATURE_TSXLDTRK.
>>
>> This RFC series has been reviewed by Dave Hansen.
> Yeah, and he and I talked about this on IRC. If you really want to dump
> CPUID to see what's there, we should do a userspace tool in tools/ or
> even use the ones which are already out there: x86info, cpuid, ...
> 
> Just adding X86_FEATURE_* flags without usage in the kernel makes no
> sense and will cause unnecessary bloat and doesn't help one bit because
> userspace simply calls CPUID directly.

Could we ignore the new flag patches for the moment and make sure
everyone is on board with what the new Documentation/ says?

Re: [RFC PATCH 1/3] Documentation/x86: Add documentation for /proc/cpuinfo feature flags

[Borislav Petkov]

On Wed, Jun 10, 2020 at 01:06:59PM -0700, Kyung Min Park wrote:
> Add documentation for /proc/cpuinfo feature flags enumeration.
> Document how and when x86 feature flags are used. Also discuss what
> their presence or absence mean for the kernel, users, and applications.
> 
> Suggested-by: Dave Hansen <dave.hansen@intel.com>
> Co-developed-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
> Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
> Signed-off-by: Kyung Min Park <kyung.min.park@intel.com>
> ---
>  Documentation/x86/cpuinfo.rst | 152 ++++++++++++++++++++++++++++++++++
>  Documentation/x86/index.rst   |   1 +
>  2 files changed, 153 insertions(+)
>  create mode 100644 Documentation/x86/cpuinfo.rst

I guess, although if we ever change how all that works, we need to
update the documentation but this is the usual thing with documentation.
Maybe it should not be documented in such a detail. :)

> diff --git a/Documentation/x86/cpuinfo.rst b/Documentation/x86/cpuinfo.rst
> new file mode 100644
> index 000000000000..d01d2c03a4d7
> --- /dev/null
> +++ b/Documentation/x86/cpuinfo.rst
> @@ -0,0 +1,152 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=================
> +x86 Feature Flags
> +=================
> +
> +Introduction
> +============
> +
> +On x86, flags appearing in /proc/cpuinfo have an X86_FEATURE definition
> +in arch/x86/include/asm/cpufeatures.h. If the kernel, any application,
> +or an end user

The kernel yes - the other two, not really. Userspace simply does CPUID
directly.

> might care about a feature, it can and should have
> +X86_FEATURE_* defined. These flags represent hardware features as
> +well as software features.
> +
> +If users want to know if a feature is available on a given system, they
> +try to find the flag in /proc/cpuinfo. If a given flag is present, it
> +means that the kernel supports it and is currently making it available.
> +If such flag represents a hardware feature, it also means that the
> +hardware supports it.
> +
> +If the expected flag does not appear in /proc/cpuinfo, things are murkier.
> +Users need to find out the reason why the flag is missing and find the way
> +how to enable it, which is not always easy.

This needs to say:

It can be that the kernel doesn't support that feature and thus hasn't
enabled it.

> There are several factors that
> +can explain missing flags: the expected feature failed to enable, the feature
> +is missing in hardware, platform firmware did not enable it, the feature is
> +disabled at build or run time, or an old kernel is in use. In such cases,
> +the users need to rely on tools like http://www.etallen.com/cpuid.html
> +(which is not updated with kernel releases) or other custom tools that
> +read CPUID.

In general, this should say something along the lines that /proc/cpuinfo
shows features which the kernel supports.

"For a full list of CPUID flags which the CPU supports, use
tools/arch/x86/tools/cpuid/cpuid"

:-)

> +
> +How are feature flags created?
> +==============================
> +
> +a: Feature flags can be derived from the contents of CPUID leaves.
> +------------------------------------------------------------------
> +These feature definitions are organized mirroring the layout of CPUID
> +leaves and grouped in words with offsets as mapped in enum cpuid_leafs
> +in cpufeatures.h (see arch/x86/include/asm/cpufeatures.h for details).
> +If a feature is defined with a X86_FEATURE_<name> definition in
> +cpufeatures.h, and if it is detected at run time, the flags will be
> +displayed accordingly in /proc/cpuinfo. For example, the flag "avx2"
> +comes from X86_FEATURE_AVX2 in cpufeatures.h.
> +
> +b: Flags can be from scattered CPUID-based features.
> +----------------------------------------------------
> +Hardware features enumerated in sparsely populated CPUID leaves get
> +software-defined values. Still, CPUID needs to be queried to determine
> +if a given feature is present. This is done in init_scattered_cpuid_features().
> +For instance, X86_FEATURE_CQM_LLC is defined as 11*32 + 0 and its presence is
> +checked at runtime in the respective CPUID leaf [EAX=f, ECX=0] bit EDX[1].
> +
> +The intent of scattering CPUID leaves is to not bloat struct
> +cpuinfo_x86.x86_capability[] unnecessarily. For instance, the CPUID leaf
> +[EAX=7, ECX=0] has 30 features and is dense, but the CPUID leaf [EAX=7, EAX=1]
> +has only one feature and would waste 31 bits of space in the x86_capability[]
> +array.

... and that per-CPU.

> +
> +c: Flags can be created synthetically under certain conditions for hardware features.
> +-------------------------------------------------------------------------------------
> +Examples of conditions include whether certain features are present in
> +MSR_IA32_CORE_CAPS or specific CPU models are identified. If the needed
> +conditions are met, the features are enabled by the macro set_cpu_cap or
> +setup_force_cpu_cap macro.

... by the ... macros.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

RE: [RFC PATCH 1/3] Documentation/x86: Add documentation for /proc/cpuinfo feature flags

[Luck, Tony]

> In general, this should say something along the lines that /proc/cpuinfo
> shows features which the kernel supports.
>
> "For a full list of CPUID flags which the CPU supports, use
> tools/arch/x86/tools/cpuid/cpuid"
>
> :-)

Dave Hansen had suggested (offline) that we add a cpuid tool to the kernel sources.

I think that's a lot of (duplicated) work for someone to maintain.  The version of this
tool at http://www.etallen.com/cpuid.html is close to 10K lines of code.

-Tony

Re: [RFC PATCH 1/3] Documentation/x86: Add documentation for /proc/cpuinfo feature flags

[Dave Hansen]

On 6/15/20 11:31 AM, Luck, Tony wrote:
>> In general, this should say something along the lines that /proc/cpuinfo
>> shows features which the kernel supports.
>>
>> "For a full list of CPUID flags which the CPU supports, use
>> tools/arch/x86/tools/cpuid/cpuid"
>>
>> :-)
> Dave Hansen had suggested (offline) that we add a cpuid tool to the kernel sources.
> 
> I think that's a lot of (duplicated) work for someone to maintain.  The version of this
> tool at http://www.etallen.com/cpuid.html is close to 10K lines of code.

Borislav suggested starting with something as simple as this:

	http://sr71.net/~dave/intel/stupid-cpuid.c

That, plus code that consume the X86_FEATURE_* flag names and print out
what's supported would be pretty handy and way smaller than 10k lines.

Re: [RFC PATCH 1/3] Documentation/x86: Add documentation for /proc/cpuinfo feature flags

[gregkh]

On Mon, Jun 15, 2020 at 06:31:50PM +0000, Luck, Tony wrote:
> > In general, this should say something along the lines that /proc/cpuinfo
> > shows features which the kernel supports.
> >
> > "For a full list of CPUID flags which the CPU supports, use
> > tools/arch/x86/tools/cpuid/cpuid"
> >
> > :-)
> 
> Dave Hansen had suggested (offline) that we add a cpuid tool to the kernel sources.
> 
> I think that's a lot of (duplicated) work for someone to maintain.  The version of this
> tool at http://www.etallen.com/cpuid.html is close to 10K lines of code.

10k is nothing, have you looked at tools/perf/ ?  :)

greg k-h

Re: [RFC PATCH 1/3] Documentation/x86: Add documentation for /proc/cpuinfo feature flags

[Kyung Min Park]

Hi Borislav,

On Mon, 2020-06-15 at 20:15 +0200, Borislav Petkov wrote:
> On Wed, Jun 10, 2020 at 01:06:59PM -0700, Kyung Min Park wrote:
> > Add documentation for /proc/cpuinfo feature flags enumeration.
> > Document how and when x86 feature flags are used. Also discuss what
> > their presence or absence mean for the kernel, users, and
> > applications.
> > 
> > Suggested-by: Dave Hansen <dave.hansen@intel.com>
> > Co-developed-by: Ricardo Neri <
> > ricardo.neri-calderon@linux.intel.com>
> > Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
> > Signed-off-by: Kyung Min Park <kyung.min.park@intel.com>
> > ---
> >  Documentation/x86/cpuinfo.rst | 152
> > ++++++++++++++++++++++++++++++++++
> >  Documentation/x86/index.rst   |   1 +
> >  2 files changed, 153 insertions(+)
> >  create mode 100644 Documentation/x86/cpuinfo.rst
> 
> I guess, although if we ever change how all that works, we need to
> update the documentation but this is the usual thing with
> documentation.
> Maybe it should not be documented in such a detail. :)
> 

Sure :)

> > diff --git a/Documentation/x86/cpuinfo.rst
> > b/Documentation/x86/cpuinfo.rst
> > new file mode 100644
> > index 000000000000..d01d2c03a4d7
> > --- /dev/null
> > +++ b/Documentation/x86/cpuinfo.rst
> > @@ -0,0 +1,152 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +=================
> > +x86 Feature Flags
> > +=================
> > +
> > +Introduction
> > +============
> > +
> > +On x86, flags appearing in /proc/cpuinfo have an X86_FEATURE
> > definition
> > +in arch/x86/include/asm/cpufeatures.h. If the kernel, any
> > application,
> > +or an end user
> 
> The kernel yes - the other two, not really. Userspace simply does
> CPUID
> directly.
> 

Okay, let me remove those two (application, user) here.

> > might care about a feature, it can and should have
> > +X86_FEATURE_* defined. These flags represent hardware features as
> > +well as software features.
> > +
> > +If users want to know if a feature is available on a given system,
> > they
> > +try to find the flag in /proc/cpuinfo. If a given flag is present,
> > it
> > +means that the kernel supports it and is currently making it
> > available.
> > +If such flag represents a hardware feature, it also means that the
> > +hardware supports it.
> > +
> > +If the expected flag does not appear in /proc/cpuinfo, things are
> > murkier.
> > +Users need to find out the reason why the flag is missing and find
> > the way
> > +how to enable it, which is not always easy.
> 
> This needs to say:
> 
> It can be that the kernel doesn't support that feature and thus
> hasn't
> enabled it.
> 

Sure, Let me modify.

> > There are several factors that
> > +can explain missing flags: the expected feature failed to enable,
> > the feature
> > +is missing in hardware, platform firmware did not enable it, the
> > feature is
> > +disabled at build or run time, or an old kernel is in use. In such
> > cases,
> > +the users need to rely on tools like 
> > http://www.etallen.com/cpuid.html
> > +(which is not updated with kernel releases) or other custom tools
> > that
> > +read CPUID.
> 
> In general, this should say something along the lines that
> /proc/cpuinfo
> shows features which the kernel supports.
> 
> "For a full list of CPUID flags which the CPU supports, use
> tools/arch/x86/tools/cpuid/cpuid"
> 
> :-)

Sure.

> 
> > +
> > +How are feature flags created?
> > +==============================
> > +
> > +a: Feature flags can be derived from the contents of CPUID leaves.
> > +------------------------------------------------------------------
> > +These feature definitions are organized mirroring the layout of
> > CPUID
> > +leaves and grouped in words with offsets as mapped in enum
> > cpuid_leafs
> > +in cpufeatures.h (see arch/x86/include/asm/cpufeatures.h for
> > details).
> > +If a feature is defined with a X86_FEATURE_<name> definition in
> > +cpufeatures.h, and if it is detected at run time, the flags will
> > be
> > +displayed accordingly in /proc/cpuinfo. For example, the flag
> > "avx2"
> > +comes from X86_FEATURE_AVX2 in cpufeatures.h.
> > +
> > +b: Flags can be from scattered CPUID-based features.
> > +----------------------------------------------------
> > +Hardware features enumerated in sparsely populated CPUID leaves
> > get
> > +software-defined values. Still, CPUID needs to be queried to
> > determine
> > +if a given feature is present. This is done in
> > init_scattered_cpuid_features().
> > +For instance, X86_FEATURE_CQM_LLC is defined as 11*32 + 0 and its
> > presence is
> > +checked at runtime in the respective CPUID leaf [EAX=f, ECX=0] bit
> > EDX[1].
> > +
> > +The intent of scattering CPUID leaves is to not bloat struct
> > +cpuinfo_x86.x86_capability[] unnecessarily. For instance, the
> > CPUID leaf
> > +[EAX=7, ECX=0] has 30 features and is dense, but the CPUID leaf
> > [EAX=7, EAX=1]
> > +has only one feature and would waste 31 bits of space in the
> > x86_capability[]
> > +array.
> 
> ... and that per-CPU.

Let me add more explanation regarding with per-CPU.

> 
> > +
> > +c: Flags can be created synthetically under certain conditions for
> > hardware features.
> > +----------------------------------------------------------------
> > ---------------------
> > +Examples of conditions include whether certain features are
> > present in
> > +MSR_IA32_CORE_CAPS or specific CPU models are identified. If the
> > needed
> > +conditions are met, the features are enabled by the macro
> > set_cpu_cap or
> > +setup_force_cpu_cap macro.
> 
> ... by the ... macros.
> 
Sure, let me fix that.