linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
@ 2021-04-20  2:50 Maciej W. Rozycki
  2021-04-20  2:50 ` [PATCH 1/4] lib/math: Add a `do_div' test module Maciej W. Rozycki
                   ` (5 more replies)
  0 siblings, 6 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-20  2:50 UTC (permalink / raw)
  To: Arnd Bergmann, Thomas Bogendoerfer
  Cc: Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch, linux-mips,
	linux-kernel

Hi,

 As Huacai has recently discovered the MIPS backend for `do_div' has been 
broken and inadvertently disabled with commit c21004cd5b4c ("MIPS: Rewrite 
<asm/div64.h> to work with gcc 4.4.0.").  As it is code I have originally 
written myself and Huacai had issues bringing it back to life leading to a 
request to discard it even I have decided to step in.

 In the end I have fixed the code and measured its performance to be ~100% 
better on average than our generic code.  I have decided it would be worth 
having the test module I have prepared for correctness evaluation as well 
as benchmarking, so I have included it with the series, also so that I can 
refer to the results easily.

 In the end I have included four patches on this occasion: 1/4 is the test 
module, 2/4 is an inline documentation fix/clarification for the `do_div' 
wrapper, 3/4 enables the MIPS `__div64_32' backend and 4/4 adds a small 
performance improvement to it.

 I have investigated a fifth change as a potential improvement where I 
replaced the call to `do_div64_32' with a DIVU instruction for cases where 
the high part of the intermediate divident is zero, but it has turned out 
to regress performance a little, so I have discarded it.

 Also a follow-up change might be worth having to reduce the code size and 
place `__div64_32' out of line for CC_OPTIMIZE_FOR_SIZE configurations, 
but I have not fully prepared such a change at this time.  I did use the 
WIP form I have for performance evaluation however; see the figures quoted 
with 4/4.

 These changes have been verified with a DECstation system with an R3400 
MIPS I processor @40MHz and a MTI Malta system with a 5Kc MIPS64 processor 
@160MHz.

 See individual change descriptions and any additional discussions for
further details.

 Questions, comments or concerns?  Otherwise please apply.

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [PATCH 1/4] lib/math: Add a `do_div' test module
  2021-04-20  2:50 [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Maciej W. Rozycki
@ 2021-04-20  2:50 ` Maciej W. Rozycki
  2021-04-20  2:50 ` [PATCH 2/4] div64: Correct inline documentation for `do_div' Maciej W. Rozycki
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-20  2:50 UTC (permalink / raw)
  To: Arnd Bergmann, Thomas Bogendoerfer
  Cc: Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch, linux-mips,
	linux-kernel

Implement a module for correctness and performance evaluation for the 
`do_div' function, often handled in an optimised manner by platform 
code.  Use a somewhat randomly generated set of inputs that is supposed 
to be representative, using the same set of divisors twice, expressed as 
a constant and as a variable each, so as to verify the implementation 
for both cases should they be handled by different code execution paths.  
Reference results were produced with GNU bc.

At the conclusion output the total execution time elapsed.

Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
---
NB there's a `checkpatch.pl' warning for a split line, but I can see no 
gain from joining it.
---
 lib/Kconfig.debug     |   10 ++
 lib/math/Makefile     |    2 
 lib/math/test_div64.c |  249 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 261 insertions(+)

linux-div64-test.diff
Index: linux-3maxp-div64/lib/Kconfig.debug
===================================================================
--- linux-3maxp-div64.orig/lib/Kconfig.debug
+++ linux-3maxp-div64/lib/Kconfig.debug
@@ -2027,6 +2027,16 @@ config TEST_SORT
 
 	  If unsure, say N.
 
+config TEST_DIV64
+	tristate "64bit/32bit division and modulo test"
+	depends on DEBUG_KERNEL || m
+	help
+	  Enable this to turn on 'do_div()' function test. This test is
+	  executed only once during system boot (so affects only boot time),
+	  or at module load time.
+
+	  If unsure, say N.
+
 config KPROBES_SANITY_TEST
 	bool "Kprobes sanity tests"
 	depends on DEBUG_KERNEL
Index: linux-3maxp-div64/lib/math/Makefile
===================================================================
--- linux-3maxp-div64.orig/lib/math/Makefile
+++ linux-3maxp-div64/lib/math/Makefile
@@ -4,3 +4,5 @@ obj-y += div64.o gcd.o lcm.o int_pow.o i
 obj-$(CONFIG_CORDIC)		+= cordic.o
 obj-$(CONFIG_PRIME_NUMBERS)	+= prime_numbers.o
 obj-$(CONFIG_RATIONAL)		+= rational.o
+
+obj-$(CONFIG_TEST_DIV64)	+= test_div64.o
Index: linux-3maxp-div64/lib/math/test_div64.c
===================================================================
--- /dev/null
+++ linux-3maxp-div64/lib/math/test_div64.c
@@ -0,0 +1,249 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021  Maciej W. Rozycki
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/init.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/time64.h>
+#include <linux/types.h>
+
+#include <asm/div64.h>
+
+#define TEST_DIV64_N_ITER 1024
+
+static const u64 test_div64_dividents[] = {
+	0x00000000ab275080,
+	0x0000000fe73c1959,
+	0x000000e54c0a74b1,
+	0x00000d4398ff1ef9,
+	0x0000a18c2ee1c097,
+	0x00079fb80b072e4a,
+	0x0072db27380dd689,
+	0x0842f488162e2284,
+	0xf66745411d8ab063,
+};
+#define SIZE_DIV64_DIVIDENTS ARRAY_SIZE(test_div64_dividents)
+
+#define TEST_DIV64_DIVISOR_0 0x00000009
+#define TEST_DIV64_DIVISOR_1 0x0000007c
+#define TEST_DIV64_DIVISOR_2 0x00000204
+#define TEST_DIV64_DIVISOR_3 0x0000cb5b
+#define TEST_DIV64_DIVISOR_4 0x00010000
+#define TEST_DIV64_DIVISOR_5 0x0008a880
+#define TEST_DIV64_DIVISOR_6 0x003fd3ae
+#define TEST_DIV64_DIVISOR_7 0x0b658fac
+#define TEST_DIV64_DIVISOR_8 0xdc08b349
+
+static const u32 test_div64_divisors[] = {
+	TEST_DIV64_DIVISOR_0,
+	TEST_DIV64_DIVISOR_1,
+	TEST_DIV64_DIVISOR_2,
+	TEST_DIV64_DIVISOR_3,
+	TEST_DIV64_DIVISOR_4,
+	TEST_DIV64_DIVISOR_5,
+	TEST_DIV64_DIVISOR_6,
+	TEST_DIV64_DIVISOR_7,
+	TEST_DIV64_DIVISOR_8,
+};
+#define SIZE_DIV64_DIVISORS ARRAY_SIZE(test_div64_divisors)
+
+static const struct {
+	u64 quotient;
+	u32 remainder;
+} test_div64_results[SIZE_DIV64_DIVISORS][SIZE_DIV64_DIVIDENTS] = {
+	{
+		{ 0x0000000013045e47, 0x00000001 },
+		{ 0x000000000161596c, 0x00000030 },
+		{ 0x000000000054e9d4, 0x00000130 },
+		{ 0x000000000000d776, 0x0000278e },
+		{ 0x000000000000ab27, 0x00005080 },
+		{ 0x00000000000013c4, 0x0004ce80 },
+		{ 0x00000000000002ae, 0x001e143c },
+		{ 0x000000000000000f, 0x0033e56c },
+		{ 0x0000000000000000, 0xab275080 },
+	}, {
+		{ 0x00000001c45c02d1, 0x00000000 },
+		{ 0x0000000020d5213c, 0x00000049 },
+		{ 0x0000000007e3d65f, 0x000001dd },
+		{ 0x0000000000140531, 0x000065ee },
+		{ 0x00000000000fe73c, 0x00001959 },
+		{ 0x000000000001d637, 0x0004e5d9 },
+		{ 0x0000000000003fc9, 0x000713bb },
+		{ 0x0000000000000165, 0x029abe7d },
+		{ 0x0000000000000012, 0x6e9f7e37 },
+	}, {
+		{ 0x000000197a3a0cf7, 0x00000002 },
+		{ 0x00000001d9632e5c, 0x00000021 },
+		{ 0x0000000071c28039, 0x000001cd },
+		{ 0x000000000120a844, 0x0000b885 },
+		{ 0x0000000000e54c0a, 0x000074b1 },
+		{ 0x00000000001a7bb3, 0x00072331 },
+		{ 0x00000000000397ad, 0x0002c61b },
+		{ 0x000000000000141e, 0x06ea2e89 },
+		{ 0x000000000000010a, 0xab002ad7 },
+	}, {
+		{ 0x0000017949e37538, 0x00000001 },
+		{ 0x0000001b62441f37, 0x00000055 },
+		{ 0x0000000694a3391d, 0x00000085 },
+		{ 0x0000000010b2a5d2, 0x0000a753 },
+		{ 0x000000000d4398ff, 0x00001ef9 },
+		{ 0x0000000001882ec6, 0x0005cbf9 },
+		{ 0x000000000035333b, 0x0017abdf },
+		{ 0x00000000000129f1, 0x0ab4520d },
+		{ 0x0000000000000f6e, 0x8ac0ce9b },
+	}, {
+		{ 0x000011f321a74e49, 0x00000006 },
+		{ 0x0000014d8481d211, 0x0000005b },
+		{ 0x0000005025cbd92d, 0x000001e3 },
+		{ 0x00000000cb5e71e3, 0x000043e6 },
+		{ 0x00000000a18c2ee1, 0x0000c097 },
+		{ 0x0000000012a88828, 0x00036c97 },
+		{ 0x000000000287f16f, 0x002c2a25 },
+		{ 0x00000000000e2cc7, 0x02d581e3 },
+		{ 0x000000000000bbf4, 0x1ba08c03 },
+	}, {
+		{ 0x0000d8db8f72935d, 0x00000005 },
+		{ 0x00000fbd5aed7a2e, 0x00000002 },
+		{ 0x000003c84b6ea64a, 0x00000122 },
+		{ 0x0000000998fa8829, 0x000044b7 },
+		{ 0x000000079fb80b07, 0x00002e4a },
+		{ 0x00000000e16b20fa, 0x0002a14a },
+		{ 0x000000001e940d22, 0x00353b2e },
+		{ 0x0000000000ab40ac, 0x06fba6ba },
+		{ 0x000000000008debd, 0x72d98365 },
+	}, {
+		{ 0x000cc3045b8fc281, 0x00000000 },
+		{ 0x0000ed1f48b5c9fc, 0x00000079 },
+		{ 0x000038fb9c63406a, 0x000000e1 },
+		{ 0x000000909705b825, 0x00000a62 },
+		{ 0x00000072db27380d, 0x0000d689 },
+		{ 0x0000000d43fce827, 0x00082b09 },
+		{ 0x00000001ccaba11a, 0x0037e8dd },
+		{ 0x000000000a13f729, 0x0566dffd },
+		{ 0x000000000085a14b, 0x23d36726 },
+	}, {
+		{ 0x00eafeb9c993592b, 0x00000001 },
+		{ 0x00110e5befa9a991, 0x00000048 },
+		{ 0x00041947b4a1d36a, 0x000000dc },
+		{ 0x00000a6679327311, 0x0000c079 },
+		{ 0x00000842f488162e, 0x00002284 },
+		{ 0x000000f4459740fc, 0x00084484 },
+		{ 0x0000002122c47bf9, 0x002ca446 },
+		{ 0x00000000b9936290, 0x004979c4 },
+		{ 0x00000000099ca89d, 0x9db446bf },
+	}, {
+		{ 0x1b60cece589da1d2, 0x00000001 },
+		{ 0x01fcb42be1453f5b, 0x0000004f },
+		{ 0x007a3f2457df0749, 0x0000013f },
+		{ 0x0001363130e3ec7b, 0x000017aa },
+		{ 0x0000f66745411d8a, 0x0000b063 },
+		{ 0x00001c757dfab350, 0x00048863 },
+		{ 0x000003dc4979c652, 0x00224ea7 },
+		{ 0x000000159edc3144, 0x06409ab3 },
+		{ 0x000000011eadfee3, 0xa99c48a8 },
+	},
+};
+
+static inline bool test_div64_verify(u64 quotient, u32 remainder, int i, int j)
+{
+	return (quotient == test_div64_results[i][j].quotient &&
+		remainder == test_div64_results[i][j].remainder);
+}
+
+/*
+ * This needs to be a macro, because we don't want to rely on the compiler
+ * to do constant propagation, and `do_div' may take a different path for
+ * constants, so we do want to verify that as well.
+ */
+#define test_div64_one(divident, divisor, i, j) ({			\
+	bool result = true;						\
+	u64 quotient;							\
+	u32 remainder;							\
+									\
+	quotient = divident;						\
+	remainder = do_div(quotient, divisor);				\
+	if (!test_div64_verify(quotient, remainder, i, j)) {		\
+		pr_err("ERROR: %016llx / %08x => %016llx,%08x\n",	\
+		       divident, divisor, quotient, remainder);		\
+		pr_err("ERROR: expected value=> %016llx,%08x\n",	\
+		       test_div64_results[i][j].quotient,		\
+		       test_div64_results[i][j].remainder);		\
+		result = false;						\
+	}								\
+	result;								\
+})
+
+/*
+ * Run calculation for the same divisor value expressed as a constant
+ * and as a variable, so as to verify the implementation for both cases
+ * should they be handled by different code execution paths.
+ */
+static bool __init test_div64(void)
+{
+	u64 divident;
+	int i, j;
+
+	for (i = 0; i < SIZE_DIV64_DIVIDENTS; i++) {
+		divident = test_div64_dividents[i];
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_0, i, 0))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_1, i, 1))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_2, i, 2))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_3, i, 3))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_4, i, 4))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_5, i, 5))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_6, i, 6))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_7, i, 7))
+			return false;
+		if (!test_div64_one(divident, TEST_DIV64_DIVISOR_8, i, 8))
+			return false;
+		for (j = 0; j < SIZE_DIV64_DIVISORS; j++) {
+			if (!test_div64_one(divident, test_div64_divisors[j],
+					    i, j))
+				return false;
+		}
+	}
+	return true;
+}
+
+static int __init test_div64_init(void)
+{
+	struct timespec64 ts, ts0, ts1;
+	int i;
+
+	pr_info("Starting 64bit/32bit division and modulo test\n");
+	ktime_get_ts64(&ts0);
+
+	for (i = 0; i < TEST_DIV64_N_ITER; i++)
+		if (!test_div64())
+			break;
+
+	ktime_get_ts64(&ts1);
+	ts = timespec64_sub(ts1, ts0);
+	pr_info("Completed 64bit/32bit division and modulo test, "
+		"%llu.%09lus elapsed\n", ts.tv_sec, ts.tv_nsec);
+
+	return 0;
+}
+
+static void __exit test_div64_exit(void)
+{
+}
+
+module_init(test_div64_init);
+module_exit(test_div64_exit);
+
+MODULE_AUTHOR("Maciej W. Rozycki <macro@orcam.me.uk>");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("64bit/32bit division and modulo test module");

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [PATCH 2/4] div64: Correct inline documentation for `do_div'
  2021-04-20  2:50 [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Maciej W. Rozycki
  2021-04-20  2:50 ` [PATCH 1/4] lib/math: Add a `do_div' test module Maciej W. Rozycki
@ 2021-04-20  2:50 ` Maciej W. Rozycki
  2021-04-20  2:50 ` [PATCH 3/4] MIPS: Reinstate platform `__div64_32' handler Maciej W. Rozycki
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-20  2:50 UTC (permalink / raw)
  To: Arnd Bergmann, Thomas Bogendoerfer
  Cc: Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch, linux-mips,
	linux-kernel

Correct inline documentation for `do_div', which is a function-like 
macro the `n' parameter of which has the semantics of a C++ reference: 
it is both read and written in the context of the caller without an 
explicit dereference such as with a pointer.

In the C programming language it has no equivalent for proper functions, 
in terms of which the documentation expresses the semantics of `do_div', 
but substituting a pointer in documentation is misleading, and using the 
C++ notation should at least raise the reader's attention and encourage 
to seek explanation even if the C++ semantics is not readily understood.

While at it observe that "semantics" is an uncountable noun, so refer to 
it with a singular rather than plural verb.

Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
---
NB there's a `checkpatch.pl' warning for tabs preceded by spaces, but that 
is just the style of the piece of code quoted and I can see no gain from 
changing it or worse yet making inconsistent.
---
 include/asm-generic/div64.h |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

linux-div64-doc-fix.diff
Index: linux-3maxp-div64/include/asm-generic/div64.h
===================================================================
--- linux-3maxp-div64.orig/include/asm-generic/div64.h
+++ linux-3maxp-div64/include/asm-generic/div64.h
@@ -8,12 +8,14 @@
  * Optimization for constant divisors on 32-bit machines:
  * Copyright (C) 2006-2015 Nicolas Pitre
  *
- * The semantics of do_div() are:
+ * The semantics of do_div() is, in C++ notation, observing that the name
+ * is a function-like macro and the n parameter has the semantics of a C++
+ * reference:
  *
- * uint32_t do_div(uint64_t *n, uint32_t base)
+ * uint32_t do_div(uint64_t &n, uint32_t base)
  * {
- * 	uint32_t remainder = *n % base;
- * 	*n = *n / base;
+ * 	uint32_t remainder = n % base;
+ * 	n = n / base;
  * 	return remainder;
  * }
  *

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [PATCH 3/4] MIPS: Reinstate platform `__div64_32' handler
  2021-04-20  2:50 [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Maciej W. Rozycki
  2021-04-20  2:50 ` [PATCH 1/4] lib/math: Add a `do_div' test module Maciej W. Rozycki
  2021-04-20  2:50 ` [PATCH 2/4] div64: Correct inline documentation for `do_div' Maciej W. Rozycki
@ 2021-04-20  2:50 ` Maciej W. Rozycki
  2021-04-22 18:36   ` Guenter Roeck
  2021-04-20  2:50 ` [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero Maciej W. Rozycki
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-20  2:50 UTC (permalink / raw)
  To: Arnd Bergmann, Thomas Bogendoerfer
  Cc: Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch, linux-mips,
	linux-kernel, stable

Our current MIPS platform `__div64_32' handler is inactive, because it 
is incorrectly only enabled for 64-bit configurations, for which generic 
`do_div' code does not call it anyway.

The handler is not suitable for being called from there though as it 
only calculates 32 bits of the quotient under the assumption the 64-bit 
divident has been suitably reduced.  Code for such reduction used to be 
there, however it has been incorrectly removed with commit c21004cd5b4c 
("MIPS: Rewrite <asm/div64.h> to work with gcc 4.4.0."), which should 
have only updated an obsoleted constraint for an inline asm involving 
$hi and $lo register outputs, while possibly wiring the original MIPS 
variant of the `do_div' macro as `__div64_32' handler for the generic 
`do_div' implementation

Correct the handler as follows then:

- Revert most of the commit referred, however retaining the current 
  formatting, except for the final two instructions of the inline asm 
  sequence, which the original commit missed.  Omit the original 64-bit 
  parts though.

- Rename the original `do_div' macro to `__div64_32'.  Use the combined 
  `x' constraint referring to the MD accumulator as a whole, replacing 
  the original individual `h' and `l' constraints used for $hi and $lo 
  registers respectively, of which `h' has been obsoleted with GCC 4.4. 
  Update surrounding code accordingly.

  We have since removed support for GCC versions before 4.9, so no need 
  for a special arrangement here; GCC has supported the `x' constraint 
  since forever anyway, or at least going back to 1991.

- Rename the `__base' local variable in `__div64_32' to `__radix' to 
  avoid a conflict with a local variable in `do_div'.

- Actually enable this code for 32-bit rather than 64-bit configurations
  by qualifying it with BITS_PER_LONG being 32 instead of 64.  Include 
  <asm/bitsperlong.h> for this macro rather than <linux/types.h> as we 
  don't need anything else.

- Finally include <asm-generic/div64.h> last rather than first.

This has passed correctness verification with test_div64 and reduced the 
module's average execution time down to 1.0668s and 0.2629s from 2.1529s 
and 0.5647s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz.  
For a reference 64-bit `do_div' code where we have the DDIVU instruction 
available to do the whole calculation right away averages at 0.0660s for 
the latter CPU.

Reported-by: Huacai Chen <chenhuacai@kernel.org>
Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Fixes: c21004cd5b4c ("MIPS: Rewrite <asm/div64.h> to work with gcc 4.4.0.")
Cc: stable@vger.kernel.org # v2.6.30+
---
Our handcrafted handler seems to run at ~25% of the performance of the 
64-bit hardware instruction; not too bad I would say.  Though there's 
likely some overhead from surrounding code that interferes with the 
figures.

Then there are a couple of `checkpatch.pl' nits about trailing whitespace 
in inline asm, which however makes it more readable.  So the change stays 
as it is.
---
 arch/mips/include/asm/div64.h |   57 ++++++++++++++++++++++++++++++------------
 1 file changed, 41 insertions(+), 16 deletions(-)

linux-mips-div64-generic-fix.diff
Index: linux-3maxp-div64/arch/mips/include/asm/div64.h
===================================================================
--- linux-3maxp-div64.orig/arch/mips/include/asm/div64.h
+++ linux-3maxp-div64/arch/mips/include/asm/div64.h
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2000, 2004  Maciej W. Rozycki
+ * Copyright (C) 2000, 2004, 2021  Maciej W. Rozycki
  * Copyright (C) 2003, 07 Ralf Baechle (ralf@linux-mips.org)
  *
  * This file is subject to the terms and conditions of the GNU General Public
@@ -9,25 +9,18 @@
 #ifndef __ASM_DIV64_H
 #define __ASM_DIV64_H
 
-#include <asm-generic/div64.h>
-
-#if BITS_PER_LONG == 64
+#include <asm/bitsperlong.h>
 
-#include <linux/types.h>
+#if BITS_PER_LONG == 32
 
 /*
  * No traps on overflows for any of these...
  */
 
-#define __div64_32(n, base)						\
-({									\
+#define do_div64_32(res, high, low, base) ({				\
 	unsigned long __cf, __tmp, __tmp2, __i;				\
 	unsigned long __quot32, __mod32;				\
-	unsigned long __high, __low;					\
-	unsigned long long __n;						\
 									\
-	__high = *__n >> 32;						\
-	__low = __n;							\
 	__asm__(							\
 	"	.set	push					\n"	\
 	"	.set	noat					\n"	\
@@ -51,18 +44,50 @@
 	"	subu	%0, %0, %z6				\n"	\
 	"	addiu	%2, %2, 1				\n"	\
 	"3:							\n"	\
-	"	bnez	%4, 0b\n\t"					\
-	"	 srl	%5, %1, 0x1f\n\t"				\
+	"	bnez	%4, 0b					\n"	\
+	"	 srl	%5, %1, 0x1f				\n"	\
 	"	.set	pop"						\
 	: "=&r" (__mod32), "=&r" (__tmp),				\
 	  "=&r" (__quot32), "=&r" (__cf),				\
 	  "=&r" (__i), "=&r" (__tmp2)					\
-	: "Jr" (base), "0" (__high), "1" (__low));			\
+	: "Jr" (base), "0" (high), "1" (low));				\
 									\
-	(__n) = __quot32;						\
+	(res) = __quot32;						\
 	__mod32;							\
 })
 
-#endif /* BITS_PER_LONG == 64 */
+#define __div64_32(n, base) ({						\
+	unsigned long __upper, __low, __high, __radix;			\
+	unsigned long long __modquot;					\
+	unsigned long long __quot;					\
+	unsigned long long __div;					\
+	unsigned long __mod;						\
+									\
+	__div = (*n);							\
+	__radix = (base);						\
+									\
+	__high = __div >> 32;						\
+	__low = __div;							\
+	__upper = __high;						\
+									\
+	if (__high) {							\
+		__asm__("divu	$0, %z1, %z2"				\
+		: "=x" (__modquot)					\
+		: "Jr" (__high), "Jr" (__radix));			\
+		__upper = __modquot >> 32;				\
+		__high = __modquot;					\
+	}								\
+									\
+	__mod = do_div64_32(__low, __upper, __low, __radix);		\
+									\
+	__quot = __high;						\
+	__quot = __quot << 32 | __low;					\
+	(*n) = __quot;							\
+	__mod;								\
+})
+
+#endif /* BITS_PER_LONG == 32 */
+
+#include <asm-generic/div64.h>
 
 #endif /* __ASM_DIV64_H */

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-20  2:50 [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Maciej W. Rozycki
                   ` (2 preceding siblings ...)
  2021-04-20  2:50 ` [PATCH 3/4] MIPS: Reinstate platform `__div64_32' handler Maciej W. Rozycki
@ 2021-04-20  2:50 ` Maciej W. Rozycki
  2021-04-21 16:05   ` H. Nikolaus Schaller
  2021-04-22 11:17   ` Andreas Schwab
  2021-04-21 12:01 ` [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Thomas Bogendoerfer
  2021-04-21 16:00 ` H. Nikolaus Schaller
  5 siblings, 2 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-20  2:50 UTC (permalink / raw)
  To: Arnd Bergmann, Thomas Bogendoerfer
  Cc: Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch, linux-mips,
	linux-kernel

We already check the high part of the divident against zero to avoid the 
costly DIVU instruction in that case, needed to reduce the high part of 
the divident, so we may well check against the divisor instead and set 
the high part of the quotient to zero right away.  We need to treat the 
high part the divident in that case though as the remainder that would 
be calculated by the DIVU instruction we avoided.

This has passed correctness verification with test_div64 and reduced the
module's average execution time down to 1.0445s and 0.2619s from 1.0668s
and 0.2629s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz.

Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
---
I have made an experimental change on top of this to put `__div64_32' out 
of line, and that increases the averages respectively up to 1.0785s and 
0.2705s.  Not a terrible loss, especially compared to generic times quoted 
with 3/4, but still, so I think it would best be made where optimising for 
size, as noted in the cover letter.
---
 arch/mips/include/asm/div64.h |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Index: linux-3maxp-div64/arch/mips/include/asm/div64.h
===================================================================
--- linux-3maxp-div64.orig/arch/mips/include/asm/div64.h
+++ linux-3maxp-div64/arch/mips/include/asm/div64.h
@@ -68,9 +68,11 @@
 									\
 	__high = __div >> 32;						\
 	__low = __div;							\
-	__upper = __high;						\
 									\
-	if (__high) {							\
+	if (__high < __radix) {						\
+		__upper = __high;					\
+		__high = 0;						\
+	} else {							\
 		__asm__("divu	$0, %z1, %z2"				\
 		: "=x" (__modquot)					\
 		: "Jr" (__high), "Jr" (__radix));			\

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-20  2:50 [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Maciej W. Rozycki
                   ` (3 preceding siblings ...)
  2021-04-20  2:50 ` [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero Maciej W. Rozycki
@ 2021-04-21 12:01 ` Thomas Bogendoerfer
  2021-04-21 13:12   ` Maciej W. Rozycki
  2021-04-21 16:00 ` H. Nikolaus Schaller
  5 siblings, 1 reply; 25+ messages in thread
From: Thomas Bogendoerfer @ 2021-04-21 12:01 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: Arnd Bergmann, Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch,
	linux-mips, linux-kernel

On Tue, Apr 20, 2021 at 04:50:22AM +0200, Maciej W. Rozycki wrote:
> Hi,
> 
>  As Huacai has recently discovered the MIPS backend for `do_div' has been 
> broken and inadvertently disabled with commit c21004cd5b4c ("MIPS: Rewrite 
> <asm/div64.h> to work with gcc 4.4.0.").  As it is code I have originally 
> written myself and Huacai had issues bringing it back to life leading to a 
> request to discard it even I have decided to step in.
> 
>  In the end I have fixed the code and measured its performance to be ~100% 
> better on average than our generic code.  I have decided it would be worth 
> having the test module I have prepared for correctness evaluation as well 
> as benchmarking, so I have included it with the series, also so that I can 
> refer to the results easily.
> 
>  In the end I have included four patches on this occasion: 1/4 is the test 
> module, 2/4 is an inline documentation fix/clarification for the `do_div' 
> wrapper, 3/4 enables the MIPS `__div64_32' backend and 4/4 adds a small 
> performance improvement to it.
> 
>  I have investigated a fifth change as a potential improvement where I 
> replaced the call to `do_div64_32' with a DIVU instruction for cases where 
> the high part of the intermediate divident is zero, but it has turned out 
> to regress performance a little, so I have discarded it.
> 
>  Also a follow-up change might be worth having to reduce the code size and 
> place `__div64_32' out of line for CC_OPTIMIZE_FOR_SIZE configurations, 
> but I have not fully prepared such a change at this time.  I did use the 
> WIP form I have for performance evaluation however; see the figures quoted 
> with 4/4.
> 
>  These changes have been verified with a DECstation system with an R3400 
> MIPS I processor @40MHz and a MTI Malta system with a 5Kc MIPS64 processor 
> @160MHz.
> 
>  See individual change descriptions and any additional discussions for
> further details.
> 
>  Questions, comments or concerns?  Otherwise please apply.

series applied to mips-next.

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-21 12:01 ` [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Thomas Bogendoerfer
@ 2021-04-21 13:12   ` Maciej W. Rozycki
  0 siblings, 0 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-21 13:12 UTC (permalink / raw)
  To: Thomas Bogendoerfer
  Cc: Arnd Bergmann, Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch,
	linux-mips, linux-kernel

On Wed, 21 Apr 2021, Thomas Bogendoerfer wrote:

> series applied to mips-next.

 Thanks!  I was about to send v2 with a small fix for an issue discovered 
with message formatting in the failure path of the test_div64 module, 
which was supposed to align the expected result with one actually produced 
for easier visual comparison.  Somehow I failed to realise that format 
specifiers will expand to text of a different length and therefore source 
string alignment won't do.

 I'll send an incremental change then.

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-20  2:50 [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Maciej W. Rozycki
                   ` (4 preceding siblings ...)
  2021-04-21 12:01 ` [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Thomas Bogendoerfer
@ 2021-04-21 16:00 ` H. Nikolaus Schaller
  2021-04-21 19:04   ` Maciej W. Rozycki
  5 siblings, 1 reply; 25+ messages in thread
From: H. Nikolaus Schaller @ 2021-04-21 16:00 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

Hi,

> Am 20.04.2021 um 04:50 schrieb Maciej W. Rozycki <macro@orcam.me.uk>:
> 
> Hi,
> 
> As Huacai has recently discovered the MIPS backend for `do_div' has been 
> broken and inadvertently disabled with commit c21004cd5b4c ("MIPS: Rewrite 
> <asm/div64.h> to work with gcc 4.4.0.").  As it is code I have originally 
> written myself and Huacai had issues bringing it back to life leading to a 
> request to discard it even I have decided to step in.
> 
> In the end I have fixed the code and measured its performance to be ~100% 
> better on average than our generic code.

That would be good.

>  I have decided it would be worth 
> having the test module I have prepared for correctness evaluation as well 
> as benchmarking, so I have included it with the series, also so that I can 
> refer to the results easily.
> 
> In the end I have included four patches on this occasion: 1/4 is the test 
> module, 2/4 is an inline documentation fix/clarification for the `do_div' 
> wrapper, 3/4 enables the MIPS `__div64_32' backend and 4/4 adds a small 
> performance improvement to it.

How can I apply them to the kernel? There is something wrong which makes
git am fail.

> 
> I have investigated a fifth change as a potential improvement where I 
> replaced the call to `do_div64_32' with a DIVU instruction for cases where 
> the high part of the intermediate divident is zero, but it has turned out 
> to regress performance a little, so I have discarded it.
> 
> Also a follow-up change might be worth having to reduce the code size and 
> place `__div64_32' out of line for CC_OPTIMIZE_FOR_SIZE configurations, 
> but I have not fully prepared such a change at this time.  I did use the 
> WIP form I have for performance evaluation however; see the figures quoted 
> with 4/4.
> 
> These changes have been verified with a DECstation system with an R3400 
> MIPS I processor @40MHz and a MTI Malta system with a 5Kc MIPS64 processor 
> @160MHz.

I'd like to test on ~320 MHz JZ4730.

> 
> See individual change descriptions and any additional discussions for
> further details.
> 
> Questions, comments or concerns?  Otherwise please apply.
> 
>  Maciej

BR and thanks,
Nikolaus Schaller

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-20  2:50 ` [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero Maciej W. Rozycki
@ 2021-04-21 16:05   ` H. Nikolaus Schaller
  2021-04-21 16:16     ` Maciej W. Rozycki
  2021-04-22 11:17   ` Andreas Schwab
  1 sibling, 1 reply; 25+ messages in thread
From: H. Nikolaus Schaller @ 2021-04-21 16:05 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel


> Am 20.04.2021 um 04:50 schrieb Maciej W. Rozycki <macro@orcam.me.uk>:
> 
> We already check the high part of the divident against zero to avoid the 

nit-picking: s/divident/dividend/

(seems to come from from Latin "dividendum" = the number that is to be divided).

> costly DIVU instruction in that case, needed to reduce the high part of 
> the divident, so we may well check against the divisor instead and set 
> the high part of the quotient to zero right away.  We need to treat the 
> high part the divident in that case though as the remainder that would 
> be calculated by the DIVU instruction we avoided.
> 
> This has passed correctness verification with test_div64 and reduced the
> module's average execution time down to 1.0445s and 0.2619s from 1.0668s
> and 0.2629s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz.

Impressive.

> 
> Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
> ---
> I have made an experimental change on top of this to put `__div64_32' out 
> of line, and that increases the averages respectively up to 1.0785s and 
> 0.2705s.  Not a terrible loss, especially compared to generic times quoted 
> with 3/4, but still, so I think it would best be made where optimising for 
> size, as noted in the cover letter.
> ---
> arch/mips/include/asm/div64.h |    6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
> 
> Index: linux-3maxp-div64/arch/mips/include/asm/div64.h
> ===================================================================
> --- linux-3maxp-div64.orig/arch/mips/include/asm/div64.h
> +++ linux-3maxp-div64/arch/mips/include/asm/div64.h
> @@ -68,9 +68,11 @@
> 									\
> 	__high = __div >> 32;						\
> 	__low = __div;							\
> -	__upper = __high;						\
> 									\
> -	if (__high) {							\
> +	if (__high < __radix) {						\
> +		__upper = __high;					\
> +		__high = 0;						\
> +	} else {							\
> 		__asm__("divu	$0, %z1, %z2"				\
> 		: "=x" (__modquot)					\
> 		: "Jr" (__high), "Jr" (__radix));			\


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-21 16:05   ` H. Nikolaus Schaller
@ 2021-04-21 16:16     ` Maciej W. Rozycki
  2021-04-22  7:56       ` Thomas Bogendoerfer
  0 siblings, 1 reply; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-21 16:16 UTC (permalink / raw)
  To: H. Nikolaus Schaller, Thomas Bogendoerfer
  Cc: Arnd Bergmann, Huacai Chen, Huacai Chen, Jiaxun Yang, linux-arch,
	linux-mips, linux-kernel

On Wed, 21 Apr 2021, H. Nikolaus Schaller wrote:

> > We already check the high part of the divident against zero to avoid the 
> 
> nit-picking: s/divident/dividend/

 Umm, I find this embarassing (and I hesitated reaching for a dictionary 
to double-check the spelling!).  Indeed why would this be different from 
subtrahend or minuend?

 Thomas, as this mistake has spread across three out of these patches,
both in change descriptions and actual code, would you mind dropping the 
series from mips-next altogether and I'll respin the series with all 
these issues corrected?

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-21 16:00 ` H. Nikolaus Schaller
@ 2021-04-21 19:04   ` Maciej W. Rozycki
  2021-04-22  5:53     ` H. Nikolaus Schaller
  0 siblings, 1 reply; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-21 19:04 UTC (permalink / raw)
  To: H. Nikolaus Schaller
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

On Wed, 21 Apr 2021, H. Nikolaus Schaller wrote:

> > In the end I have included four patches on this occasion: 1/4 is the test 
> > module, 2/4 is an inline documentation fix/clarification for the `do_div' 
> > wrapper, 3/4 enables the MIPS `__div64_32' backend and 4/4 adds a small 
> > performance improvement to it.
> 
> How can I apply them to the kernel? There is something wrong which makes
> git am fail.

 I don't know.  The changes were made against vanilla 5.12-rc7, but then 
the pieces affected have not changed for ages.  FWIW I can `git am' the 
series as received back just fine.

> > These changes have been verified with a DECstation system with an R3400 
> > MIPS I processor @40MHz and a MTI Malta system with a 5Kc MIPS64 processor 
> > @160MHz.
> 
> I'd like to test on ~320 MHz JZ4730.

 Sure, I'd love to hear how this code performs with other implementations.  

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-21 19:04   ` Maciej W. Rozycki
@ 2021-04-22  5:53     ` H. Nikolaus Schaller
  2021-04-22 13:39       ` Jiaxun Yang
  0 siblings, 1 reply; 25+ messages in thread
From: H. Nikolaus Schaller @ 2021-04-22  5:53 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, Linux Kernel Mailing List

Hi,

> Am 21.04.2021 um 21:04 schrieb Maciej W. Rozycki <macro@orcam.me.uk>:
> 
> On Wed, 21 Apr 2021, H. Nikolaus Schaller wrote:
> 
>>> In the end I have included four patches on this occasion: 1/4 is the test 
>>> module, 2/4 is an inline documentation fix/clarification for the `do_div' 
>>> wrapper, 3/4 enables the MIPS `__div64_32' backend and 4/4 adds a small 
>>> performance improvement to it.
>> 
>> How can I apply them to the kernel? There is something wrong which makes
>> git am fail.
> 
> I don't know.  The changes were made against vanilla 5.12-rc7, but then 
> the pieces affected have not changed for ages.  FWIW I can `git am' the 
> series as received back just fine.

Please can you point me to some download/pull/gitweb? It seems as if the series
also did not appear at https://patchwork.kernel.org/project/linux-mips/list/

> 
>>> These changes have been verified with a DECstation system with an R3400 
>>> MIPS I processor @40MHz and a MTI Malta system with a 5Kc MIPS64 processor 
>>> @160MHz.
>> 
>> I'd like to test on ~320 MHz JZ4730.
> 
> Sure, I'd love to hear how this code performs with other implementations.  
> 
>  Maciej

BR and thanks,
Nikolaus Schaller

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-21 16:16     ` Maciej W. Rozycki
@ 2021-04-22  7:56       ` Thomas Bogendoerfer
  2021-04-22  9:12         ` Maciej W. Rozycki
  0 siblings, 1 reply; 25+ messages in thread
From: Thomas Bogendoerfer @ 2021-04-22  7:56 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: H. Nikolaus Schaller, Arnd Bergmann, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

On Wed, Apr 21, 2021 at 06:16:18PM +0200, Maciej W. Rozycki wrote:
> On Wed, 21 Apr 2021, H. Nikolaus Schaller wrote:
> 
> > > We already check the high part of the divident against zero to avoid the 
> > 
> > nit-picking: s/divident/dividend/
> 
>  Umm, I find this embarassing (and I hesitated reaching for a dictionary 
> to double-check the spelling!).  Indeed why would this be different from 
> subtrahend or minuend?
> 
>  Thomas, as this mistake has spread across three out of these patches,
> both in change descriptions and actual code, would you mind dropping the 
> series from mips-next altogether and I'll respin the series with all 
> these issues corrected?

I'm sorry, but I don't rebase mips-next and the patches have been pushed
out before I received this mail.

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-22  7:56       ` Thomas Bogendoerfer
@ 2021-04-22  9:12         ` Maciej W. Rozycki
  2021-04-22 11:08           ` Thomas Bogendoerfer
  2021-04-27 12:16           ` Maciej W. Rozycki
  0 siblings, 2 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-22  9:12 UTC (permalink / raw)
  To: Thomas Bogendoerfer
  Cc: H. Nikolaus Schaller, Arnd Bergmann, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

On Thu, 22 Apr 2021, Thomas Bogendoerfer wrote:

> >  Thomas, as this mistake has spread across three out of these patches,
> > both in change descriptions and actual code, would you mind dropping the 
> > series from mips-next altogether and I'll respin the series with all 
> > these issues corrected?
> 
> I'm sorry, but I don't rebase mips-next and the patches have been pushed
> out before I received this mail.

 Hmm, what about changes known to actually break something then?  Like 
with R6 here?  Those will undoubtedly cause someone a headache with 
bisection sometime in the future.  Are you sure your policy is the best 
possible?

 NB, I have benchmarked the update with my DECstation, however my Malta 
has not come up after a reboot last evening and neither it has after a few 
remote power cycles.  I have planned a visit in my lab next week anyway, 
so I'll see what has happened there; hopefully I'm able to bring the board 
back to life as I find it valuable for my purposes.  I had to replace the 
PSU it came with already a couple years back and the new one is supposedly 
high-quality, so I fear it's the board itself.

 Meanwhile I'll be able to give DECstation figures only.  I guess such 
limited results will suffice if I post the fix as an update rather than 
replacement.

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-22  9:12         ` Maciej W. Rozycki
@ 2021-04-22 11:08           ` Thomas Bogendoerfer
  2021-04-22 20:47             ` Maciej W. Rozycki
  2021-04-27 12:16           ` Maciej W. Rozycki
  1 sibling, 1 reply; 25+ messages in thread
From: Thomas Bogendoerfer @ 2021-04-22 11:08 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: H. Nikolaus Schaller, Arnd Bergmann, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

On Thu, Apr 22, 2021 at 11:12:40AM +0200, Maciej W. Rozycki wrote:
> On Thu, 22 Apr 2021, Thomas Bogendoerfer wrote:
> 
> > >  Thomas, as this mistake has spread across three out of these patches,
> > > both in change descriptions and actual code, would you mind dropping the 
> > > series from mips-next altogether and I'll respin the series with all 
> > > these issues corrected?
> > 
> > I'm sorry, but I don't rebase mips-next and the patches have been pushed
> > out before I received this mail.
> 
>  Hmm, what about changes known to actually break something then?  Like 
> with R6 here?  Those will undoubtedly cause someone a headache with 
> bisection sometime in the future.  Are you sure your policy is the best 
> possible?

This is my Linus pull tree, so I'm following 

Documentation/maintainer/rebasing-and-merging.rst

>  Meanwhile I'll be able to give DECstation figures only.  I guess such 
> limited results will suffice if I post the fix as an update rather than 
> replacement.

thank you.

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-20  2:50 ` [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero Maciej W. Rozycki
  2021-04-21 16:05   ` H. Nikolaus Schaller
@ 2021-04-22 11:17   ` Andreas Schwab
  1 sibling, 0 replies; 25+ messages in thread
From: Andreas Schwab @ 2021-04-22 11:17 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

Subject typo: s/is/if/

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-22  5:53     ` H. Nikolaus Schaller
@ 2021-04-22 13:39       ` Jiaxun Yang
  2021-04-22 15:58         ` Maciej W. Rozycki
  2021-04-22 16:00         ` H. Nikolaus Schaller
  0 siblings, 2 replies; 25+ messages in thread
From: Jiaxun Yang @ 2021-04-22 13:39 UTC (permalink / raw)
  To: H. Nikolaus Schaller, Maciej W. Rozycki
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	linux-arch, linux-mips, Linux Kernel Mailing List



On Thu, Apr 22, 2021, at 1:53 PM, H. Nikolaus Schaller wrote:
> Hi,
> 
> > Am 21.04.2021 um 21:04 schrieb Maciej W. Rozycki <macro@orcam.me.uk>:
> > 
> > On Wed, 21 Apr 2021, H. Nikolaus Schaller wrote:
> > 
> >>> In the end I have included four patches on this occasion: 1/4 is the test 
> >>> module, 2/4 is an inline documentation fix/clarification for the `do_div' 
> >>> wrapper, 3/4 enables the MIPS `__div64_32' backend and 4/4 adds a small 
> >>> performance improvement to it.
> >> 
> >> How can I apply them to the kernel? There is something wrong which makes
> >> git am fail.
> > 
> > I don't know.  The changes were made against vanilla 5.12-rc7, but then 
> > the pieces affected have not changed for ages.  FWIW I can `git am' the 
> > series as received back just fine.
> 
> Please can you point me to some download/pull/gitweb? It seems as if the series
> also did not appear at https://patchwork.kernel.org/project/linux-mips/list/
> 

You may try download raw from:

https://lore.kernel.org/linux-mips/E6326E8A-50DA-4F81-9865-F29EE0E298A9@goldelico.com/T/#t


-- 
- Jiaxun

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-22 13:39       ` Jiaxun Yang
@ 2021-04-22 15:58         ` Maciej W. Rozycki
  2021-04-22 16:00         ` H. Nikolaus Schaller
  1 sibling, 0 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-22 15:58 UTC (permalink / raw)
  To: Jiaxun Yang
  Cc: H. Nikolaus Schaller, Arnd Bergmann, Thomas Bogendoerfer,
	Huacai Chen, Huacai Chen, linux-arch, linux-mips,
	Linux Kernel Mailing List

On Thu, 22 Apr 2021, Jiaxun Yang wrote:

> > Please can you point me to some download/pull/gitweb? It seems as if the series
> > also did not appear at https://patchwork.kernel.org/project/linux-mips/list/
> > 
> 
> You may try download raw from:
> 
> https://lore.kernel.org/linux-mips/E6326E8A-50DA-4F81-9865-F29EE0E298A9@goldelico.com/T/#t

 Or you can cherry-pick the commits directly from mips-next: 
<git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux.git>.

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-22 13:39       ` Jiaxun Yang
  2021-04-22 15:58         ` Maciej W. Rozycki
@ 2021-04-22 16:00         ` H. Nikolaus Schaller
  2021-04-22 16:55           ` Maciej W. Rozycki
  1 sibling, 1 reply; 25+ messages in thread
From: H. Nikolaus Schaller @ 2021-04-22 16:00 UTC (permalink / raw)
  To: Jiaxun Yang, Maciej W. Rozycki
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	linux-arch, linux-mips, Linux Kernel Mailing List, Paul Boddie,
	Lubomir Rintel

Hi,

> Am 22.04.2021 um 15:39 schrieb Jiaxun Yang <jiaxun.yang@flygoat.com>:
> 
> 
> 
> On Thu, Apr 22, 2021, at 1:53 PM, H. Nikolaus Schaller wrote:
>> Hi,
>> 
>>> Am 21.04.2021 um 21:04 schrieb Maciej W. Rozycki <macro@orcam.me.uk>:
>>> 
>>> On Wed, 21 Apr 2021, H. Nikolaus Schaller wrote:
>>> 
>>>>> In the end I have included four patches on this occasion: 1/4 is the test 
>>>>> module, 2/4 is an inline documentation fix/clarification for the `do_div' 
>>>>> wrapper, 3/4 enables the MIPS `__div64_32' backend and 4/4 adds a small 
>>>>> performance improvement to it.
>>>> 
>>>> How can I apply them to the kernel? There is something wrong which makes
>>>> git am fail.
>>> 
>>> I don't know.  The changes were made against vanilla 5.12-rc7, but then 
>>> the pieces affected have not changed for ages.  FWIW I can `git am' the 
>>> series as received back just fine.
>> 
>> Please can you point me to some download/pull/gitweb? It seems as if the series
>> also did not appear at https://patchwork.kernel.org/project/linux-mips/list/
>> 
> 
> You may try download raw from:
> 
> https://lore.kernel.org/linux-mips/E6326E8A-50DA-4F81-9865-F29EE0E298A9@goldelico.com/T/#t

I simply tried again and it seems that I had tried to git am it on top of the
patches from Huacai which of course fails.

Now I could run the tests:

from [PATCH 4/4]:

> This has passed correctness verification with test_div64 and reduced the
> module's average execution time down to 1.0445s and 0.2619s from 1.0668s
> and 0.2629s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz.

test only [PATCH 1/4 and 2/4]:

[  256.301140] test_div64: Completed 64bit/32bit division and modulo test, 0.291154944s elapsed

+ [PATCH 3/4]

[ 1698.698920] test_div64: Completed 64bit/32bit division and modulo test, 0.132142865s elapsed

+ [PATCH 4/4]

[  466.818349] test_div64: Completed 64bit/32bit division and modulo test, 0.134429075s elapsed

So the new code is indeed faster than the default implementation.
[PATCH 4/4] has no significant influence (wouldn't say it is slower because timer resolution
isn't very high on this machine and the kernel has some scheduling issue [1]).

Anyways the JZ4730 can boot and works with these patches included.

BR and thanks,
Nikolaus Schaller


[1] we are preparing full support for the JZ4730 based Skytone Alpha machine. Most features
are working except sound/I2S. I2C is a little unreliable and Ethernet has hickups. And scheduling
which indicates some fundamental IRQ or timer issue we could not yet identify.

So if anyone happens to have such a machine and can help debugging, please contact me.


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-22 16:00         ` H. Nikolaus Schaller
@ 2021-04-22 16:55           ` Maciej W. Rozycki
  2021-04-22 17:06             ` H. Nikolaus Schaller
  0 siblings, 1 reply; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-22 16:55 UTC (permalink / raw)
  To: H. Nikolaus Schaller
  Cc: Jiaxun Yang, Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen,
	Huacai Chen, linux-arch, linux-mips, Linux Kernel Mailing List,
	Paul Boddie, Lubomir Rintel

On Thu, 22 Apr 2021, H. Nikolaus Schaller wrote:

> > This has passed correctness verification with test_div64 and reduced the
> > module's average execution time down to 1.0445s and 0.2619s from 1.0668s
> > and 0.2629s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz.
> 
> test only [PATCH 1/4 and 2/4]:
> 
> [  256.301140] test_div64: Completed 64bit/32bit division and modulo test, 0.291154944s elapsed
> 
> + [PATCH 3/4]
> 
> [ 1698.698920] test_div64: Completed 64bit/32bit division and modulo test, 0.132142865s elapsed
> 
> + [PATCH 4/4]
> 
> [  466.818349] test_div64: Completed 64bit/32bit division and modulo test, 0.134429075s elapsed
> 
> So the new code is indeed faster than the default implementation.
> [PATCH 4/4] has no significant influence (wouldn't say it is slower because timer resolution
> isn't very high on this machine and the kernel has some scheduling issue [1]).

 Have you used it as a module or at bootstrap?  I have noticed that at 
bootstrap the initialisation of the random number generator sometimes 
interferes with the benchmark, which happens when there's an intervening 
message produced, e.g.:

test_div64: Starting 64bit/32bit division and modulo test
random: fast init done
test_div64: Completed 64bit/32bit division and modulo test, 1.069906272s elapsed

I think it can be worked around by configuration changes so that more 
stuff is run between the RNG and the test module, but instead I have 
simply inserted:

	mdelay(5000);

at the beginning of `test_div64_init' instead, as for historical reasons I 
haven't got the systems involved set up for modules (beyond Linux 2.4) at 
this time.

 NB I have run the benchmark five times with each change and system and 
with the RNG taken out of the picture results were very stable as any 
fluctuation only started at the fifth decimal digit.  Both the DECstation 
(the model I used anyway) and the Malta have a high-resolution clock 
source though, the I/O ASIC free-running counter register at 25MHz (used 
by David L. Mills, the original author of the NTP suite, for his reference 
implementation) and the CP0 Count register at 80MHz respectively.

 I would expect your JZ4730 device to have the CP0 Count register as well, 
as it has been architectural ever since MIPS III really, or is your system 
SMP with CP0 Count registers out of sync across CPUs due to sleep modes or 
whatever?

 Thanks for sharing your figures.

> [1] we are preparing full support for the JZ4730 based Skytone Alpha machine. Most features
> are working except sound/I2S. I2C is a little unreliable and Ethernet has hickups. And scheduling
> which indicates some fundamental IRQ or timer issue we could not yet identify.

 Good luck with that!

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation
  2021-04-22 16:55           ` Maciej W. Rozycki
@ 2021-04-22 17:06             ` H. Nikolaus Schaller
  0 siblings, 0 replies; 25+ messages in thread
From: H. Nikolaus Schaller @ 2021-04-22 17:06 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: Jiaxun Yang, Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen,
	Huacai Chen, linux-arch, linux-mips, Linux Kernel Mailing List,
	Paul Boddie, Lubomir Rintel


> Am 22.04.2021 um 18:55 schrieb Maciej W. Rozycki <macro@orcam.me.uk>:
> 
> 
> Have you used it as a module or at bootstrap?

I did load it by insmod.

> I would expect your JZ4730 device to have the CP0 Count register as well, 
> as it has been architectural ever since MIPS III really, or is your system 
> SMP with CP0 Count registers out of sync across CPUs due to sleep modes or 
> whatever?

It switches clocksource to some operating system timers on the SoC which
may have an influence on the resolution (or precision).

> Thanks for sharing your figures.

It was a pleasure towards better MIPS support...

> 
>> [1] we are preparing full support for the JZ4730 based Skytone Alpha machine. Most features
>> are working except sound/I2S. I2C is a little unreliable and Ethernet has hickups. And scheduling
>> which indicates some fundamental IRQ or timer issue we could not yet identify.
> 
> Good luck with that!

BR and thanks,
Niklaus


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 3/4] MIPS: Reinstate platform `__div64_32' handler
  2021-04-20  2:50 ` [PATCH 3/4] MIPS: Reinstate platform `__div64_32' handler Maciej W. Rozycki
@ 2021-04-22 18:36   ` Guenter Roeck
  2021-04-22 20:43     ` Maciej W. Rozycki
  0 siblings, 1 reply; 25+ messages in thread
From: Guenter Roeck @ 2021-04-22 18:36 UTC (permalink / raw)
  To: Maciej W. Rozycki
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel, stable

On Tue, Apr 20, 2021 at 04:50:40AM +0200, Maciej W. Rozycki wrote:
> Our current MIPS platform `__div64_32' handler is inactive, because it 
> is incorrectly only enabled for 64-bit configurations, for which generic 
> `do_div' code does not call it anyway.
> 
> The handler is not suitable for being called from there though as it 
> only calculates 32 bits of the quotient under the assumption the 64-bit 
> divident has been suitably reduced.  Code for such reduction used to be 
> there, however it has been incorrectly removed with commit c21004cd5b4c 
> ("MIPS: Rewrite <asm/div64.h> to work with gcc 4.4.0."), which should 
> have only updated an obsoleted constraint for an inline asm involving 
> $hi and $lo register outputs, while possibly wiring the original MIPS 
> variant of the `do_div' macro as `__div64_32' handler for the generic 
> `do_div' implementation
> 
> Correct the handler as follows then:
> 
> - Revert most of the commit referred, however retaining the current 
>   formatting, except for the final two instructions of the inline asm 
>   sequence, which the original commit missed.  Omit the original 64-bit 
>   parts though.
> 
> - Rename the original `do_div' macro to `__div64_32'.  Use the combined 
>   `x' constraint referring to the MD accumulator as a whole, replacing 
>   the original individual `h' and `l' constraints used for $hi and $lo 
>   registers respectively, of which `h' has been obsoleted with GCC 4.4. 
>   Update surrounding code accordingly.
> 
>   We have since removed support for GCC versions before 4.9, so no need 
>   for a special arrangement here; GCC has supported the `x' constraint 
>   since forever anyway, or at least going back to 1991.
> 
> - Rename the `__base' local variable in `__div64_32' to `__radix' to 
>   avoid a conflict with a local variable in `do_div'.
> 
> - Actually enable this code for 32-bit rather than 64-bit configurations
>   by qualifying it with BITS_PER_LONG being 32 instead of 64.  Include 
>   <asm/bitsperlong.h> for this macro rather than <linux/types.h> as we 
>   don't need anything else.
> 
> - Finally include <asm-generic/div64.h> last rather than first.
> 
> This has passed correctness verification with test_div64 and reduced the 
> module's average execution time down to 1.0668s and 0.2629s from 2.1529s 
> and 0.5647s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz.  
> For a reference 64-bit `do_div' code where we have the DDIVU instruction 
> available to do the whole calculation right away averages at 0.0660s for 
> the latter CPU.
> 
This patch results in:

arch/mips/mti-malta/malta-time.c: In function 'plat_time_init':
./arch/mips/include/asm/div64.h:76:3: error: inconsistent operand constraints in an 'asm'

and similar errors when trying to compile malta_qemu_32r6_defconfig.
I tried with gcc 8.3.0, 8.4.0, 9.3.0, and 10.3.0.

Does this need some additional new compile flags ?

Thanks,
Guenter

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 3/4] MIPS: Reinstate platform `__div64_32' handler
  2021-04-22 18:36   ` Guenter Roeck
@ 2021-04-22 20:43     ` Maciej W. Rozycki
  0 siblings, 0 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-22 20:43 UTC (permalink / raw)
  To: Guenter Roeck
  Cc: Arnd Bergmann, Thomas Bogendoerfer, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel, stable

On Thu, 22 Apr 2021, Guenter Roeck wrote:

> This patch results in:
> 
> arch/mips/mti-malta/malta-time.c: In function 'plat_time_init':
> ./arch/mips/include/asm/div64.h:76:3: error: inconsistent operand constraints in an 'asm'
> 
> and similar errors when trying to compile malta_qemu_32r6_defconfig.

 Thanks for the heads-up, however the 0-DAY bot has caught this issue 
already last night and I would have addressed it earlier on if not for a 
failure of my Malta board :( which disrupted my verification.

> I tried with gcc 8.3.0, 8.4.0, 9.3.0, and 10.3.0.
> 
> Does this need some additional new compile flags ?

 MIPSr6 doesn't have the original division instruction along with the MD 
accumulator registers anymore, and consequently GCC cannot fit the 
constraint requested.

 We don't need that asm however.  Maybe we didn't with GCC 2.95 either, 
but I suspect there was something to it.  Anyway I have just posted a fix.

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-22 11:08           ` Thomas Bogendoerfer
@ 2021-04-22 20:47             ` Maciej W. Rozycki
  0 siblings, 0 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-22 20:47 UTC (permalink / raw)
  To: Thomas Bogendoerfer
  Cc: H. Nikolaus Schaller, Arnd Bergmann, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

On Thu, 22 Apr 2021, Thomas Bogendoerfer wrote:

> >  Hmm, what about changes known to actually break something then?  Like 
> > with R6 here?  Those will undoubtedly cause someone a headache with 
> > bisection sometime in the future.  Are you sure your policy is the best 
> > possible?
> 
> This is my Linus pull tree, so I'm following 
> 
> Documentation/maintainer/rebasing-and-merging.rst

 Fair enough.

> >  Meanwhile I'll be able to give DECstation figures only.  I guess such 
> > limited results will suffice if I post the fix as an update rather than 
> > replacement.
> 
> thank you.

 I have requested 3/4 to be backported however, so I think in this case 
4/4 will have to follow as well as 2/2 of the fix series I have just 
posted.  Will you be able to resolve this somehow?

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero
  2021-04-22  9:12         ` Maciej W. Rozycki
  2021-04-22 11:08           ` Thomas Bogendoerfer
@ 2021-04-27 12:16           ` Maciej W. Rozycki
  1 sibling, 0 replies; 25+ messages in thread
From: Maciej W. Rozycki @ 2021-04-27 12:16 UTC (permalink / raw)
  To: Thomas Bogendoerfer
  Cc: H. Nikolaus Schaller, Arnd Bergmann, Huacai Chen, Huacai Chen,
	Jiaxun Yang, linux-arch, linux-mips, linux-kernel

On Thu, 22 Apr 2021, Maciej W. Rozycki wrote:

>  NB, I have benchmarked the update with my DECstation, however my Malta 
> has not come up after a reboot last evening and neither it has after a few 
> remote power cycles.  I have planned a visit in my lab next week anyway, 
> so I'll see what has happened there; hopefully I'm able to bring the board 
> back to life as I find it valuable for my purposes.  I had to replace the 
> PSU it came with already a couple years back and the new one is supposedly 
> high-quality, so I fear it's the board itself.

 For the record I have put the Malta back in service now.

 It was weird: the YAMON status shown on the ASCII display was "E:P_UNKN",
i.e. "Unknown PCI device on board (should never happen)" (according to the 
manual).  Indeed I never saw that message before.  By elimination I have 
tracked down the DEFPA FDDI network interface to be the culprit and upon a 
close inspection found a bent pin with the PFI ASIC (the PCI interface 
chip), a 160-pin 20-mil PQFP part.  It was IDSEL shorting to RST_L, the 
adjacent pin.  No wonder the card wreaked havoc with the system.  Fixed by 
carefully moving the pins apart with a razor blade (always good to have 
one at hand!).

 Now I wonder how this card happened to work with the pin bent for some
15 years and only finally developed a short now.  Weird indeed, and 
weirder even to see a second failure related to IC pins this year only.

  Maciej

^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2021-04-27 12:16 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-20  2:50 [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Maciej W. Rozycki
2021-04-20  2:50 ` [PATCH 1/4] lib/math: Add a `do_div' test module Maciej W. Rozycki
2021-04-20  2:50 ` [PATCH 2/4] div64: Correct inline documentation for `do_div' Maciej W. Rozycki
2021-04-20  2:50 ` [PATCH 3/4] MIPS: Reinstate platform `__div64_32' handler Maciej W. Rozycki
2021-04-22 18:36   ` Guenter Roeck
2021-04-22 20:43     ` Maciej W. Rozycki
2021-04-20  2:50 ` [PATCH 4/4] MIPS: Avoid DIVU in `__div64_32' is result would be zero Maciej W. Rozycki
2021-04-21 16:05   ` H. Nikolaus Schaller
2021-04-21 16:16     ` Maciej W. Rozycki
2021-04-22  7:56       ` Thomas Bogendoerfer
2021-04-22  9:12         ` Maciej W. Rozycki
2021-04-22 11:08           ` Thomas Bogendoerfer
2021-04-22 20:47             ` Maciej W. Rozycki
2021-04-27 12:16           ` Maciej W. Rozycki
2021-04-22 11:17   ` Andreas Schwab
2021-04-21 12:01 ` [PATCH 0/4] Reinstate and improve MIPS `do_div' implementation Thomas Bogendoerfer
2021-04-21 13:12   ` Maciej W. Rozycki
2021-04-21 16:00 ` H. Nikolaus Schaller
2021-04-21 19:04   ` Maciej W. Rozycki
2021-04-22  5:53     ` H. Nikolaus Schaller
2021-04-22 13:39       ` Jiaxun Yang
2021-04-22 15:58         ` Maciej W. Rozycki
2021-04-22 16:00         ` H. Nikolaus Schaller
2021-04-22 16:55           ` Maciej W. Rozycki
2021-04-22 17:06             ` H. Nikolaus Schaller

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).