[PATCH v2 0/4] arm64: Add AT_HWCAP3

[PATCH v2 0/4] arm64: Add AT_HWCAP3

[Mark Brown]

Currently for arm64 we expose hwcaps to userspace using the low 32 bits
of AT_HWCAP and AT_HWCAP2. Due to the ever expanding capabilties of the
architecture we have now allocated all the available bits in this scheme
so we need to expand, either using the higher bits or adding a new
AT_HWCAP3. Discussions with glibc developers suggested that AT_HWCAP3
would be cleaner for them so the series adopts that approach.

The final patch adds an initial value in AT_HWCAP3 reporting the
presence of FEAT_EBF16, this will conflict with my series converting
ID_AA64ISAR1_EL1 to be generated. I've got a version based on top of
that which I can send if that's convenient.

v2:
 - Rebase onto v5.19-rc3.

Mark Brown (4):
  arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned
    long
  elf: Allow architectures to provide AT_HWCAP3
  arm64/cpufeature: Support AT_HWCAP3
  arm64/hwcap: Support FEAT_EBF16

 Documentation/arm64/elf_hwcaps.rst  | 10 +++++++---
 arch/arm64/include/asm/cpufeature.h |  5 ++++-
 arch/arm64/include/asm/hwcap.h      | 17 ++++++++++++-----
 arch/arm64/include/uapi/asm/hwcap.h |  5 +++++
 arch/arm64/kernel/cpufeature.c      | 28 +++++++++++++++++++++-------
 arch/arm64/kernel/cpuinfo.c         |  1 +
 fs/binfmt_elf.c                     |  3 +++
 include/uapi/linux/auxvec.h         |  1 +
 8 files changed, 54 insertions(+), 16 deletions(-)


base-commit: a111daf0c53ae91e71fd2bfe7497862d14132e3e
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Mark Brown]

When we added support for AT_HWCAP2 we took advantage of the fact that we
have limited hwcaps to the low 32 bits and stored it along with AT_HWCAP
in a single unsigned integer. Thanks to the ever expanding capabilities of
the architecture we have now allocated all 64 of the bits in an unsigned
long so in preparation for adding more hwcaps convert elf_hwcap to be an
array instead. There should be no functional change from this patch.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/cpufeature.h |  2 ++
 arch/arm64/include/asm/hwcap.h      |  4 +++-
 arch/arm64/kernel/cpufeature.c      | 22 +++++++++++++++-------
 3 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 14a8f3d93add..84756c660b5e 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -11,6 +11,8 @@
 #include <asm/hwcap.h>
 #include <asm/sysreg.h>
 
+/* Note that bits 62 and 63 of each AT_HWCAP are reserved */
+#define CPU_FEATURES_PER_HWCAP	32
 #define MAX_CPU_FEATURES	64
 #define cpu_feature(x)		KERNEL_HWCAP_ ## x
 
diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h
index aa443d8f8cfb..e0054c7b3a98 100644
--- a/arch/arm64/include/asm/hwcap.h
+++ b/arch/arm64/include/asm/hwcap.h
@@ -41,6 +41,8 @@
 #ifndef __ASSEMBLY__
 #include <linux/log2.h>
 
+#define KHWCAP_OFFSET(n)		((n - 1) * CPU_FEATURES_PER_HWCAP)
+
 /*
  * For userspace we represent hwcaps as a collection of HWCAP{,2}_x bitfields
  * as described in uapi/asm/hwcap.h. For the kernel we represent hwcaps as
@@ -85,7 +87,7 @@
 #define KERNEL_HWCAP_PACA		__khwcap_feature(PACA)
 #define KERNEL_HWCAP_PACG		__khwcap_feature(PACG)
 
-#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + 32)
+#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + KHWCAP_OFFSET(2))
 #define KERNEL_HWCAP_DCPODP		__khwcap2_feature(DCPODP)
 #define KERNEL_HWCAP_SVE2		__khwcap2_feature(SVE2)
 #define KERNEL_HWCAP_SVEAES		__khwcap2_feature(SVEAES)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 8d88433de81d..1a8f60a6661e 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -91,7 +91,7 @@
 #include <asm/virt.h>
 
 /* Kernel representation of AT_HWCAP and AT_HWCAP2 */
-static unsigned long elf_hwcap __read_mostly;
+static unsigned long elf_hwcap[MAX_CPU_FEATURES / CPU_FEATURES_PER_HWCAP] __read_mostly;
 
 #ifdef CONFIG_COMPAT
 #define COMPAT_ELF_HWCAP_DEFAULT	\
@@ -3098,14 +3098,22 @@ static bool __maybe_unused __system_matches_cap(unsigned int n)
 
 void cpu_set_feature(unsigned int num)
 {
-	WARN_ON(num >= MAX_CPU_FEATURES);
-	elf_hwcap |= BIT(num);
+	int i = num / CPU_FEATURES_PER_HWCAP;
+	int bit = num % CPU_FEATURES_PER_HWCAP;
+
+	WARN_ON(i >= ARRAY_SIZE(elf_hwcap));
+
+	elf_hwcap[i] |= BIT(bit);
 }
 
 bool cpu_have_feature(unsigned int num)
 {
-	WARN_ON(num >= MAX_CPU_FEATURES);
-	return elf_hwcap & BIT(num);
+	int i = num / CPU_FEATURES_PER_HWCAP;
+	int bit = num % CPU_FEATURES_PER_HWCAP;
+
+	WARN_ON(i >= ARRAY_SIZE(elf_hwcap));
+
+	return elf_hwcap[i] & BIT(bit);
 }
 EXPORT_SYMBOL_GPL(cpu_have_feature);
 
@@ -3116,12 +3124,12 @@ unsigned long cpu_get_elf_hwcap(void)
 	 * note that for userspace compatibility we guarantee that bits 62
 	 * and 63 will always be returned as 0.
 	 */
-	return lower_32_bits(elf_hwcap);
+	return elf_hwcap[0];
 }
 
 unsigned long cpu_get_elf_hwcap2(void)
 {
-	return upper_32_bits(elf_hwcap);
+	return elf_hwcap[1];
 }
 
 static void __init setup_system_capabilities(void)
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/4] elf: Allow architectures to provide AT_HWCAP3

[Mark Brown]

In order to provide for architectures which have filled both AT_HWCAP and
AT_HWCAP2 add support for AT_HWCAP3.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 fs/binfmt_elf.c             | 3 +++
 include/uapi/linux/auxvec.h | 1 +
 2 files changed, 4 insertions(+)

diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
index 63c7ebb0da89..59d937dddc09 100644
--- a/fs/binfmt_elf.c
+++ b/fs/binfmt_elf.c
@@ -275,6 +275,9 @@ create_elf_tables(struct linux_binprm *bprm, const struct elfhdr *exec,
 	NEW_AUX_ENT(AT_RANDOM, (elf_addr_t)(unsigned long)u_rand_bytes);
 #ifdef ELF_HWCAP2
 	NEW_AUX_ENT(AT_HWCAP2, ELF_HWCAP2);
+#endif
+#ifdef ELF_HWCAP3
+	NEW_AUX_ENT(AT_HWCAP3, ELF_HWCAP3);
 #endif
 	NEW_AUX_ENT(AT_EXECFN, bprm->exec);
 	if (k_platform) {
diff --git a/include/uapi/linux/auxvec.h b/include/uapi/linux/auxvec.h
index c7e502bf5a6f..76631c7f9e8f 100644
--- a/include/uapi/linux/auxvec.h
+++ b/include/uapi/linux/auxvec.h
@@ -30,6 +30,7 @@
 				 * differ from AT_PLATFORM. */
 #define AT_RANDOM 25	/* address of 16 random bytes */
 #define AT_HWCAP2 26	/* extension of AT_HWCAP */
+#define AT_HWCAP3 27	/* further extension of AT_HWCAP */
 
 #define AT_EXECFN  31	/* filename of program */
 
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 3/4] arm64/cpufeature: Support AT_HWCAP3

[Mark Brown]

We have filled the low 32 bits of both AT_HWCAP and AT_HWCAP2 so in order
to support further architecture features we must either start allocating
from the upper 32 bits or add a new AT_HWCAP3. Feedback from the glibc
developers was that AT_HWCAP3 would be cleaner so go with that.

This patch merely adds the glue required for AT_HWCAP3, meaning that we
start providing AT_HWCAP3 for ELF executables but no bits will be set in
it.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 Documentation/arm64/elf_hwcaps.rst  |  6 +++---
 arch/arm64/include/asm/cpufeature.h |  3 ++-
 arch/arm64/include/asm/hwcap.h      | 12 ++++++++----
 arch/arm64/include/uapi/asm/hwcap.h |  4 ++++
 arch/arm64/kernel/cpufeature.c      |  5 +++++
 5 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/Documentation/arm64/elf_hwcaps.rst b/Documentation/arm64/elf_hwcaps.rst
index 3d116fb536c5..5f4d89c4afe2 100644
--- a/Documentation/arm64/elf_hwcaps.rst
+++ b/Documentation/arm64/elf_hwcaps.rst
@@ -16,9 +16,9 @@ architected discovery mechanism available to userspace code at EL0. The
 kernel exposes the presence of these features to userspace through a set
 of flags called hwcaps, exposed in the auxilliary vector.
 
-Userspace software can test for features by acquiring the AT_HWCAP or
-AT_HWCAP2 entry of the auxiliary vector, and testing whether the relevant
-flags are set, e.g.::
+Userspace software can test for features by acquiring the AT_HWCAP,
+AT_HWCAP2 or AT_HWCAP3 entry of the auxiliary vector, and testing
+whether the relevant flags are set, e.g.::
 
 	bool floating_point_is_present(void)
 	{
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 84756c660b5e..901cf318a556 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -13,7 +13,7 @@
 
 /* Note that bits 62 and 63 of each AT_HWCAP are reserved */
 #define CPU_FEATURES_PER_HWCAP	32
-#define MAX_CPU_FEATURES	64
+#define MAX_CPU_FEATURES	96
 #define cpu_feature(x)		KERNEL_HWCAP_ ## x
 
 #ifndef __ASSEMBLY__
@@ -436,6 +436,7 @@ void cpu_set_feature(unsigned int num);
 bool cpu_have_feature(unsigned int num);
 unsigned long cpu_get_elf_hwcap(void);
 unsigned long cpu_get_elf_hwcap2(void);
+unsigned long cpu_get_elf_hwcap3(void);
 
 #define cpu_set_named_feature(name) cpu_set_feature(cpu_feature(name))
 #define cpu_have_named_feature(name) cpu_have_feature(cpu_feature(name))
diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h
index e0054c7b3a98..1ef208ab6895 100644
--- a/arch/arm64/include/asm/hwcap.h
+++ b/arch/arm64/include/asm/hwcap.h
@@ -44,10 +44,11 @@
 #define KHWCAP_OFFSET(n)		((n - 1) * CPU_FEATURES_PER_HWCAP)
 
 /*
- * For userspace we represent hwcaps as a collection of HWCAP{,2}_x bitfields
- * as described in uapi/asm/hwcap.h. For the kernel we represent hwcaps as
- * natural numbers (in a single range of size MAX_CPU_FEATURES) defined here
- * with prefix KERNEL_HWCAP_ mapped to their HWCAP{,2}_x counterpart.
+ * For userspace we represent hwcaps as a collection of HWCAP{,[23]}_x
+ * bitfields as described in uapi/asm/hwcap.h. For the kernel we
+ * represent hwcaps as natural numbers (in a single range of size
+ * MAX_CPU_FEATURES) defined here with prefix KERNEL_HWCAP_ mapped to
+ * their HWCAP{,2}_x counterpart.
  *
  * Hwcaps should be set and tested within the kernel via the
  * cpu_{set,have}_named_feature(feature) where feature is the unique suffix
@@ -121,12 +122,15 @@
 #define KERNEL_HWCAP_SME_FA64		__khwcap2_feature(SME_FA64)
 #define KERNEL_HWCAP_WFXT		__khwcap2_feature(WFXT)
 
+#define __khwcap3_feature(x)		(const_ilog2(HWCAP3_ ## x) + KHWCAP_OFFSET(3))
+
 /*
  * This yields a mask that user programs can use to figure out what
  * instruction set this cpu supports.
  */
 #define ELF_HWCAP		cpu_get_elf_hwcap()
 #define ELF_HWCAP2		cpu_get_elf_hwcap2()
+#define ELF_HWCAP3		cpu_get_elf_hwcap3()
 
 #ifdef CONFIG_COMPAT
 #define COMPAT_ELF_HWCAP	(compat_elf_hwcap)
diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h
index 4bb2cc8ac446..7b818f8fc01b 100644
--- a/arch/arm64/include/uapi/asm/hwcap.h
+++ b/arch/arm64/include/uapi/asm/hwcap.h
@@ -89,4 +89,8 @@
 #define HWCAP2_SME_FA64		(1 << 30)
 #define HWCAP2_WFXT		(1UL << 31)
 
+/*
+ * HWCAP3 flags - for AT_HWCAP3
+ */
+
 #endif /* _UAPI__ASM_HWCAP_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 1a8f60a6661e..7605b213d9db 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -3132,6 +3132,11 @@ unsigned long cpu_get_elf_hwcap2(void)
 	return elf_hwcap[1];
 }
 
+unsigned long cpu_get_elf_hwcap3(void)
+{
+	return elf_hwcap[2];
+}
+
 static void __init setup_system_capabilities(void)
 {
 	/*
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 4/4] arm64/hwcap: Support FEAT_EBF16

[Mark Brown]

The v9.2 feature FEAT_EBF16 provides support for an extended BFloat16 mode.
Allow userspace to discover system support for this feature by adding a
hwcap for it.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 Documentation/arm64/elf_hwcaps.rst  | 4 ++++
 arch/arm64/include/asm/hwcap.h      | 1 +
 arch/arm64/include/uapi/asm/hwcap.h | 1 +
 arch/arm64/kernel/cpufeature.c      | 1 +
 arch/arm64/kernel/cpuinfo.c         | 1 +
 5 files changed, 8 insertions(+)

diff --git a/Documentation/arm64/elf_hwcaps.rst b/Documentation/arm64/elf_hwcaps.rst
index 5f4d89c4afe2..4dab74a70cc1 100644
--- a/Documentation/arm64/elf_hwcaps.rst
+++ b/Documentation/arm64/elf_hwcaps.rst
@@ -301,6 +301,10 @@ HWCAP2_WFXT
 
     Functionality implied by ID_AA64ISAR2_EL1.WFXT == 0b0010.
 
+HWCAP3_EBF16
+
+    Functionality implied by ID_AA64ISAR1_EL1.BF16 == 0b0010.
+
 4. Unused AT_HWCAP bits
 -----------------------
 
diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h
index 1ef208ab6895..cf670e4afc16 100644
--- a/arch/arm64/include/asm/hwcap.h
+++ b/arch/arm64/include/asm/hwcap.h
@@ -123,6 +123,7 @@
 #define KERNEL_HWCAP_WFXT		__khwcap2_feature(WFXT)
 
 #define __khwcap3_feature(x)		(const_ilog2(HWCAP3_ ## x) + KHWCAP_OFFSET(3))
+#define KERNEL_HWCAP_EBF16		__khwcap3_feature(EBF16)
 
 /*
  * This yields a mask that user programs can use to figure out what
diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h
index 7b818f8fc01b..3b6644f0c07f 100644
--- a/arch/arm64/include/uapi/asm/hwcap.h
+++ b/arch/arm64/include/uapi/asm/hwcap.h
@@ -92,5 +92,6 @@
 /*
  * HWCAP3 flags - for AT_HWCAP3
  */
+#define HWCAP3_EBF16		(1 << 0)
 
 #endif /* _UAPI__ASM_HWCAP_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 7605b213d9db..79850a0cd563 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -2623,6 +2623,7 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = {
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_FRINTTS_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_FRINT),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_SB_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_SB),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_BF16_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_BF16),
+	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_BF16_SHIFT, 4, FTR_UNSIGNED, 2, CAP_HWCAP, KERNEL_HWCAP_EBF16),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_DGH_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_DGH),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_I8MM_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_I8MM),
 	HWCAP_CAP(SYS_ID_AA64MMFR2_EL1, ID_AA64MMFR2_AT_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_USCAT),
diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
index 8eff0a34ffd4..2ca131243c30 100644
--- a/arch/arm64/kernel/cpuinfo.c
+++ b/arch/arm64/kernel/cpuinfo.c
@@ -107,6 +107,7 @@ static const char *const hwcap_str[] = {
 	[KERNEL_HWCAP_SME_F32F32]	= "smef32f32",
 	[KERNEL_HWCAP_SME_FA64]		= "smefa64",
 	[KERNEL_HWCAP_WFXT]		= "wfxt",
+	[KERNEL_HWCAP_EBF16]		= "ebf16",
 };
 
 #ifdef CONFIG_COMPAT
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Will Deacon]

On Mon, Jun 20, 2022 at 01:54:48PM +0100, Mark Brown wrote:
> When we added support for AT_HWCAP2 we took advantage of the fact that we
> have limited hwcaps to the low 32 bits and stored it along with AT_HWCAP
> in a single unsigned integer. Thanks to the ever expanding capabilities of
> the architecture we have now allocated all 64 of the bits in an unsigned
> long so in preparation for adding more hwcaps convert elf_hwcap to be an
> array instead. There should be no functional change from this patch.
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
>  arch/arm64/include/asm/cpufeature.h |  2 ++
>  arch/arm64/include/asm/hwcap.h      |  4 +++-
>  arch/arm64/kernel/cpufeature.c      | 22 +++++++++++++++-------
>  3 files changed, 20 insertions(+), 8 deletions(-)

I would like to see an Ack on this series from the libc side (i.e.
Szabolcs) before we extend the ABI. Have any other architectures used
HWCAP3?

> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 14a8f3d93add..84756c660b5e 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -11,6 +11,8 @@
>  #include <asm/hwcap.h>
>  #include <asm/sysreg.h>
>  
> +/* Note that bits 62 and 63 of each AT_HWCAP are reserved */

Can you expand a bit on this comment, please? It's not clear who has
reserved those bits (e.g. kernel or userspace) and why having two bit
reserved means we are limited to 32 instead of 62 bits.

> +#define CPU_FEATURES_PER_HWCAP	32
>  #define MAX_CPU_FEATURES	64
>  #define cpu_feature(x)		KERNEL_HWCAP_ ## x
>  
> diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h
> index aa443d8f8cfb..e0054c7b3a98 100644
> --- a/arch/arm64/include/asm/hwcap.h
> +++ b/arch/arm64/include/asm/hwcap.h
> @@ -41,6 +41,8 @@
>  #ifndef __ASSEMBLY__
>  #include <linux/log2.h>
>  
> +#define KHWCAP_OFFSET(n)		((n - 1) * CPU_FEATURES_PER_HWCAP)
> +
>  /*
>   * For userspace we represent hwcaps as a collection of HWCAP{,2}_x bitfields
>   * as described in uapi/asm/hwcap.h. For the kernel we represent hwcaps as
> @@ -85,7 +87,7 @@
>  #define KERNEL_HWCAP_PACA		__khwcap_feature(PACA)
>  #define KERNEL_HWCAP_PACG		__khwcap_feature(PACG)
>  
> -#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + 32)
> +#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + KHWCAP_OFFSET(2))
>  #define KERNEL_HWCAP_DCPODP		__khwcap2_feature(DCPODP)
>  #define KERNEL_HWCAP_SVE2		__khwcap2_feature(SVE2)
>  #define KERNEL_HWCAP_SVEAES		__khwcap2_feature(SVEAES)
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 8d88433de81d..1a8f60a6661e 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -91,7 +91,7 @@
>  #include <asm/virt.h>
>  
>  /* Kernel representation of AT_HWCAP and AT_HWCAP2 */
> -static unsigned long elf_hwcap __read_mostly;
> +static unsigned long elf_hwcap[MAX_CPU_FEATURES / CPU_FEATURES_PER_HWCAP] __read_mostly;

I can't help but think this would be neater as an explicit bitmap
(i.e. DECLARE_BITMAP).

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1392 bytes --]

On Tue, Jun 28, 2022 at 03:21:25PM +0100, Will Deacon wrote:
> On Mon, Jun 20, 2022 at 01:54:48PM +0100, Mark Brown wrote:

> I would like to see an Ack on this series from the libc side (i.e.
> Szabolcs) before we extend the ABI. Have any other architectures used

Me too.

> HWCAP3?

No, hence needing to add it.

> > +/* Note that bits 62 and 63 of each AT_HWCAP are reserved */

> Can you expand a bit on this comment, please? It's not clear who has
> reserved those bits (e.g. kernel or userspace) and why having two bit
> reserved means we are limited to 32 instead of 62 bits.

It's also not clear to me why they're reserved, I didn't manage to dig
far enough into the history to figure that out - I believe it's a glibc
thing, or at least something glibc now expects.  Indeed I can't
immediately dig up the specific reference now.  Szabolcs?  

> >  /* Kernel representation of AT_HWCAP and AT_HWCAP2 */
> > -static unsigned long elf_hwcap __read_mostly;
> > +static unsigned long elf_hwcap[MAX_CPU_FEATURES / CPU_FEATURES_PER_HWCAP] __read_mostly;

> I can't help but think this would be neater as an explicit bitmap
> (i.e. DECLARE_BITMAP).

IIRC my thinking there was that the need to chop the bitmap up into
chunks that aren't word sized that made me not use that - the code to
read and write is a lot simpler if hwcapN is a single array element.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Szabolcs Nagy]

The 06/28/2022 16:06, Mark Brown wrote:
> On Tue, Jun 28, 2022 at 03:21:25PM +0100, Will Deacon wrote:
> > On Mon, Jun 20, 2022 at 01:54:48PM +0100, Mark Brown wrote:
> > > +/* Note that bits 62 and 63 of each AT_HWCAP are reserved */
> 
> > Can you expand a bit on this comment, please? It's not clear who has
> > reserved those bits (e.g. kernel or userspace) and why having two bit
> > reserved means we are limited to 32 instead of 62 bits.
> 
> It's also not clear to me why they're reserved, I didn't manage to dig
> far enough into the history to figure that out - I believe it's a glibc
> thing, or at least something glibc now expects.  Indeed I can't
> immediately dig up the specific reference now.  Szabolcs?  

only the top two bits of AT_HWCAP are reserved for glibc.
(bits of AT_HWCAP2 are not reserved.)

we did the 32bit limit because of ilp32 and i requested to
keep following that.

there is no precedent for using more than 32bit AT_HWCAP nor
precedent for using AT_HWCAP3.

since you are interested in the details...

the reason i slightly preferred AT_HWCAP3 was that AT_HWCAP
is a bit special in ld.so: library lookups can depend on a
glibc internal 64bit "hwcap" value that is historically
AT_HWCAP but since that does not use all bits, some targets
reused the top bits for high level platform abis which we
may want to do on aarch64 too. it should be possible to make
"hwcap" separate from AT_HWCAP (i.e. the bottom 32bits of
"hwcap" just happens to come from AT_HWCAP but the top is
defined by glibc), but that would be awkward to do in the
ld.so code if AT_HWCAP started using the top bits too.
(i think we cannot change the "hwcap" bits once they are
allocated because they are in the ld.so.cache file.)

so the options are:

- AT_HWCAP3. (the only option that's ilp32 compatible.)

- start using top bits of AT_HWCAP2 but not of AT_HWCAP.
  (this is the least amount of work for now)

- i go and fix ld.so code so top bits of internal "hwcap"
  are independent of AT_HWCAP. (we might need to bump the
  ld.so.cache file format version for this, which happened
  22 years ago last time.. this needs more investigation)

- start using AT_HWCAP top bits and we don't care about
  the future of "hwcap" ld.so feature. this is rarely used,
  but i think it may become useful. e.g. it would allow us
  to define a new base line arch abi for distros (so they
  can ship some libs built twice: once for base armv8.0 and
  once for the new base line and ld.so would pick the right
  lib based on hwcaps/feature checks). i would avoid this.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1947 bytes --]

On Wed, Jun 29, 2022 at 11:01:43AM +0100, Szabolcs Nagy wrote:

> the reason i slightly preferred AT_HWCAP3 was that AT_HWCAP
> is a bit special in ld.so: library lookups can depend on a
> glibc internal 64bit "hwcap" value that is historically
> AT_HWCAP but since that does not use all bits, some targets
> reused the top bits for high level platform abis which we
> may want to do on aarch64 too. it should be possible to make
> "hwcap" separate from AT_HWCAP (i.e. the bottom 32bits of
> "hwcap" just happens to come from AT_HWCAP but the top is
> defined by glibc), but that would be awkward to do in the
> ld.so code if AT_HWCAP started using the top bits too.
> (i think we cannot change the "hwcap" bits once they are
> allocated because they are in the ld.so.cache file.)

So when you were saying that bits 62 and 63 were reserved it's actually
potentially all the top 32 bits but only in plain AT_HWCAP?

> so the options are:

> - AT_HWCAP3. (the only option that's ilp32 compatible.)

It's also a precedent we can follow when we run out of bits again which
does seem like a thing that will plausibly happen.

> - start using top bits of AT_HWCAP2 but not of AT_HWCAP.
>   (this is the least amount of work for now)

> - i go and fix ld.so code so top bits of internal "hwcap"
>   are independent of AT_HWCAP. (we might need to bump the
>   ld.so.cache file format version for this, which happened
>   22 years ago last time.. this needs more investigation)

This does sound like disproportionate effort TBH.

> - start using AT_HWCAP top bits and we don't care about
>   the future of "hwcap" ld.so feature. this is rarely used,
>   but i think it may become useful. e.g. it would allow us
>   to define a new base line arch abi for distros (so they
>   can ship some libs built twice: once for base armv8.0 and
>   once for the new base line and ld.so would pick the right
>   lib based on hwcaps/feature checks). i would avoid this.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Szabolcs Nagy]

The 06/29/2022 12:44, Mark Brown wrote:
> On Wed, Jun 29, 2022 at 11:01:43AM +0100, Szabolcs Nagy wrote:
> 
> > the reason i slightly preferred AT_HWCAP3 was that AT_HWCAP
> > is a bit special in ld.so: library lookups can depend on a
> > glibc internal 64bit "hwcap" value that is historically
> > AT_HWCAP but since that does not use all bits, some targets
> > reused the top bits for high level platform abis which we
> > may want to do on aarch64 too. it should be possible to make
> > "hwcap" separate from AT_HWCAP (i.e. the bottom 32bits of
> > "hwcap" just happens to come from AT_HWCAP but the top is
> > defined by glibc), but that would be awkward to do in the
> > ld.so code if AT_HWCAP started using the top bits too.
> > (i think we cannot change the "hwcap" bits once they are
> > allocated because they are in the ld.so.cache file.)
> 
> So when you were saying that bits 62 and 63 were reserved it's actually
> potentially all the top 32 bits but only in plain AT_HWCAP?
> 

currently only bits 62 and 63 are reserved and only in plain AT_HWCAP.

and yes it might be useful to reserve all top 32bits of AT_HWCAP.
(for glibc internal reasons those are easy to use for ld.so extensions).

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Catalin Marinas]

On Wed, Jun 29, 2022 at 11:01:43AM +0100, Szabolcs Nagy wrote:
> The 06/28/2022 16:06, Mark Brown wrote:
> > On Tue, Jun 28, 2022 at 03:21:25PM +0100, Will Deacon wrote:
> > > On Mon, Jun 20, 2022 at 01:54:48PM +0100, Mark Brown wrote:
> > > > +/* Note that bits 62 and 63 of each AT_HWCAP are reserved */
> > 
> > > Can you expand a bit on this comment, please? It's not clear who has
> > > reserved those bits (e.g. kernel or userspace) and why having two bit
> > > reserved means we are limited to 32 instead of 62 bits.
> > 
> > It's also not clear to me why they're reserved, I didn't manage to dig
> > far enough into the history to figure that out - I believe it's a glibc
> > thing, or at least something glibc now expects.  Indeed I can't
> > immediately dig up the specific reference now.  Szabolcs?  
> 
> only the top two bits of AT_HWCAP are reserved for glibc.
> (bits of AT_HWCAP2 are not reserved.)
> 
> we did the 32bit limit because of ilp32 and i requested to
> keep following that.
> 
> there is no precedent for using more than 32bit AT_HWCAP nor
> precedent for using AT_HWCAP3.
[...]
> - start using top bits of AT_HWCAP2 but not of AT_HWCAP.
>   (this is the least amount of work for now)

That's a bit more appealing to me in the short term, though we may still
end up with AT_HWCAP3 at some point. Is anyone still thinking seriously
about ILP32?

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/4] arm64/cpufeature: Store elf_hwcaps as an array rather than unsigned long

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 674 bytes --]

On Wed, Jun 29, 2022 at 02:55:55PM +0100, Catalin Marinas wrote:
> On Wed, Jun 29, 2022 at 11:01:43AM +0100, Szabolcs Nagy wrote:

> > - start using top bits of AT_HWCAP2 but not of AT_HWCAP.
> >   (this is the least amount of work for now)

> That's a bit more appealing to me in the short term, though we may still
> end up with AT_HWCAP3 at some point. Is anyone still thinking seriously
> about ILP32?

I am thinking it might be worth adding AT_HWCAP3 now even if we go that
route for the time being, simply so the addition has more time to work
it's way through the ecosystem before we actually need it.  It'll make
life a bit easier whenever we do actually get there.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/4] arm64: Add AT_HWCAP3

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 907 bytes --]

On Mon, Jun 20, 2022 at 01:54:47PM +0100, Mark Brown wrote:
> Currently for arm64 we expose hwcaps to userspace using the low 32 bits
> of AT_HWCAP and AT_HWCAP2. Due to the ever expanding capabilties of the
> architecture we have now allocated all the available bits in this scheme
> so we need to expand, either using the higher bits or adding a new
> AT_HWCAP3. Discussions with glibc developers suggested that AT_HWCAP3
> would be cleaner for them so the series adopts that approach.

This discussion appears to have ground to a halt.  Szabolcs prefers
AT_HWCAP3, Catalin using the high bits of AT_HWCAP2 for now and nobody
else said anything.  I don't have strong preferences either way but
wonder if it's worth defining AT_HWCAP3 even if we start allocating more
bits from AT_HWCAP2 so that it's got more time to percolate through
libcs and so on before it's actually needed.

What should we do here?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/4] arm64: Add AT_HWCAP3

[Szabolcs Nagy]

The 07/05/2022 18:53, Mark Brown wrote:
> On Mon, Jun 20, 2022 at 01:54:47PM +0100, Mark Brown wrote:
> > Currently for arm64 we expose hwcaps to userspace using the low 32 bits
> > of AT_HWCAP and AT_HWCAP2. Due to the ever expanding capabilties of the
> > architecture we have now allocated all the available bits in this scheme
> > so we need to expand, either using the higher bits or adding a new
> > AT_HWCAP3. Discussions with glibc developers suggested that AT_HWCAP3
> > would be cleaner for them so the series adopts that approach.
> 
> This discussion appears to have ground to a halt.  Szabolcs prefers
> AT_HWCAP3, Catalin using the high bits of AT_HWCAP2 for now and nobody
> else said anything.  I don't have strong preferences either way but
> wonder if it's worth defining AT_HWCAP3 even if we start allocating more
> bits from AT_HWCAP2 so that it's got more time to percolate through
> libcs and so on before it's actually needed.
> 
> What should we do here?

AT_HWCAP2 top bits works for me too.

then i'd do AT_HWCAP3 when we run out of hwcap2 bits.
i don't think defining AT_HWCAP3 without having any
hwcap3 bit is very useful.

this way we can postpone AT_HWCAP3 a bit and avoid
issues with top bits of plain AT_HWCAP in glibc.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/4] arm64: Add AT_HWCAP3

[Marc Zyngier]

On 2022-07-05 18:53, Mark Brown wrote:
> On Mon, Jun 20, 2022 at 01:54:47PM +0100, Mark Brown wrote:
>> Currently for arm64 we expose hwcaps to userspace using the low 32 
>> bits
>> of AT_HWCAP and AT_HWCAP2. Due to the ever expanding capabilties of 
>> the
>> architecture we have now allocated all the available bits in this 
>> scheme
>> so we need to expand, either using the higher bits or adding a new
>> AT_HWCAP3. Discussions with glibc developers suggested that AT_HWCAP3
>> would be cleaner for them so the series adopts that approach.
> 
> This discussion appears to have ground to a halt.  Szabolcs prefers
> AT_HWCAP3, Catalin using the high bits of AT_HWCAP2 for now and nobody
> else said anything.  I don't have strong preferences either way but
> wonder if it's worth defining AT_HWCAP3 even if we start allocating 
> more
> bits from AT_HWCAP2 so that it's got more time to percolate through
> libcs and so on before it's actually needed.
> 
> What should we do here?

I really don't like the idea of adding a new HWCAP when we still
have free bits lying around. We should use that first, and only
add HWCAP3 once we are at (or close enough to) the limit.

That's assuming that there is no breakage with using these spare
bits. If something is at risk, I'd be good to spell it out.

         M.
-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/4] arm64: Add AT_HWCAP3

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 866 bytes --]

On Wed, Jul 06, 2022 at 11:08:55AM +0100, Marc Zyngier wrote:
> On 2022-07-05 18:53, Mark Brown wrote:

> > else said anything.  I don't have strong preferences either way but
> > wonder if it's worth defining AT_HWCAP3 even if we start allocating more
> > bits from AT_HWCAP2 so that it's got more time to percolate through
> > libcs and so on before it's actually needed.

> I really don't like the idea of adding a new HWCAP when we still
> have free bits lying around. We should use that first, and only
> add HWCAP3 once we are at (or close enough to) the limit.

It depends what you define as close of course - it's only taken us 4
years to fill the low 32 bits in AT_HWCAP2, that doesn't make it a
pressing issue in terms of kernel development but in terms of things
like entrprise distros it's starting to get into the timescales where it
might be relevant.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel