[PATCH v4 00/14] liblockdep fixes for 5.9-rc1

[PATCH v4 00/14] liblockdep fixes for 5.9-rc1

[Sasha Levin]

Hi Linus,

Please consider applying these patches for liblockdep, or alternatively
pull from:

  git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linux.git tags/liblockdep-fixes-040820

The patches fix up compilation and functionality of liblockdep on 5.8,
they were tested using liblockdep's internal testsuite.

I was unable to get the x86 folks to pull these fixes for the past few
months:

 - https://lkml.org/lkml/2020/2/17/1089
 - https://lkml.org/lkml/2020/4/18/817
 - https://lkml.org/lkml/2020/6/22/1262

Which is why this pull request ends up going straight to you.


Sasha Levin (14):
  tools headers: Add kprobes.h header
  tools headers: Add rcupdate.h header
  tools/kernel.h: extend with dummy RCU functions
  tools bitmap: add bitmap_andnot definition
  tools/lib/lockdep: add definition required for IRQ flag tracing
  tools bitmap: add bitmap_clear definition
  tools/lib/lockdep: Hook up vsprintf, find_bit, hweight libraries
  tools/lib/lockdep: Enable building with CONFIG_TRACE_IRQFLAGS
  tools/lib/lockdep: New stacktrace API
  tools/lib/lockdep: call lockdep_init_task on init
  tools/lib/lockdep: switch to using lockdep_init_map_waits
  tools/kernel.h: hide noinstr
  tools/lib/lockdep: explicitly declare lockdep_init_task()
  tools/kernel.h: hide task_struct.hardirq_chain_key

 tools/include/linux/bitmap.h                  | 10 ++++++
 tools/include/linux/kernel.h                  | 11 ++++++
 tools/include/linux/kprobes.h                 |  7 ++++
 tools/include/linux/lockdep.h                 | 10 ++++++
 tools/include/linux/rcupdate.h                | 12 +++++++
 tools/include/linux/stacktrace.h              |  8 +++++
 tools/lib/bitmap.c                            | 35 +++++++++++++++++++
 tools/lib/lockdep/Build                       |  2 +-
 tools/lib/lockdep/Makefile                    |  2 +-
 tools/lib/lockdep/include/liblockdep/common.h |  4 +--
 tools/lib/lockdep/include/liblockdep/mutex.h  |  2 +-
 tools/lib/lockdep/include/liblockdep/rwlock.h |  2 +-
 tools/lib/lockdep/lockdep.c                   |  4 +--
 tools/lib/lockdep/preload.c                   |  8 ++++-
 14 files changed, 108 insertions(+), 9 deletions(-)
 create mode 100644 tools/include/linux/kprobes.h
 create mode 100644 tools/include/linux/rcupdate.h

-- 
2.25.1

[PATCH v4 01/14] tools headers: Add kprobes.h header

[Sasha Levin]

This is now needed by liblockdep as a result of 2f43c6022d84 ("kprobes:
Prohibit probing on lockdep functions").

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/kprobes.h | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 tools/include/linux/kprobes.h

diff --git a/tools/include/linux/kprobes.h b/tools/include/linux/kprobes.h
new file mode 100644
index 0000000000000..f665725ea4d59
--- /dev/null
+++ b/tools/include/linux/kprobes.h
@@ -0,0 +1,7 @@
+#ifndef _TOOLS_LINUX_KPROBES_H_
+#define _TOOLS_LINUX_KPROBES_H_
+
+#define NOKPROBE_SYMBOL(x)
+#define nokprobe_inline
+
+#endif
-- 
2.25.1

[PATCH v4 02/14] tools headers: Add rcupdate.h header

[Sasha Levin]

This is now needed by liblockdep as a result of a0b0fd53e1e6
("locking/lockdep: Free lock classes that are no longer in use").

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/rcupdate.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 tools/include/linux/rcupdate.h

diff --git a/tools/include/linux/rcupdate.h b/tools/include/linux/rcupdate.h
new file mode 100644
index 0000000000000..1b7abc86f4336
--- /dev/null
+++ b/tools/include/linux/rcupdate.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LIBLOCKDEP_RCUPDATE_H_
+#define _LIBLOCKDEP_RCUPDATE_H_
+
+#define call_rcu(x, y)
+#define init_rcu_head(x)
+
+struct rcu_head {
+	char dummy;
+};
+
+#endif
-- 
2.25.1

[PATCH v4 03/14] tools/kernel.h: extend with dummy RCU functions

[Sasha Levin]

These calls were added by 108c14858b9e ("locking/lockdep: Add support
for dynamic keys") and require no special handling in userspace, so just
add empty function definitions.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/kernel.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/include/linux/kernel.h b/tools/include/linux/kernel.h
index a7e54a08fb54c..902d3b9ab4c17 100644
--- a/tools/include/linux/kernel.h
+++ b/tools/include/linux/kernel.h
@@ -117,4 +117,11 @@ int scnprintf_pad(char * buf, size_t size, const char * fmt, ...);
 #define current_gfp_context(k) 0
 #define synchronize_rcu()
 
+static __maybe_unused int system_state;
+#define SYSTEM_SCHEDULING 0
+
+#define might_sleep()
+#define rcu_read_lock()
+#define rcu_read_unlock()
+
 #endif
-- 
2.25.1

[PATCH v4 04/14] tools bitmap: add bitmap_andnot definition

[Sasha Levin]

Add definition of bitmap_andnot() and wire tools/lib/bitmap.c into
liblockdep.

This is needed as a result of de4643a77356 ("locking/lockdep: Reuse lock
chains that have been freed").

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/bitmap.h | 10 ++++++++++
 tools/lib/bitmap.c           | 15 +++++++++++++++
 tools/lib/lockdep/Build      |  2 +-
 3 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/tools/include/linux/bitmap.h b/tools/include/linux/bitmap.h
index 477a1cae513f2..ab5df035f8eda 100644
--- a/tools/include/linux/bitmap.h
+++ b/tools/include/linux/bitmap.h
@@ -18,6 +18,8 @@ int __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
 int __bitmap_equal(const unsigned long *bitmap1,
 		   const unsigned long *bitmap2, unsigned int bits);
 void bitmap_clear(unsigned long *map, unsigned int start, int len);
+int __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+			const unsigned long *bitmap2, unsigned int bits);
 
 #define BITMAP_FIRST_WORD_MASK(start) (~0UL << ((start) & (BITS_PER_LONG - 1)))
 
@@ -178,4 +180,12 @@ static inline int bitmap_equal(const unsigned long *src1,
 	return __bitmap_equal(src1, src2, nbits);
 }
 
+static inline int bitmap_andnot(unsigned long *dst, const unsigned long *src1,
+                        const unsigned long *src2, unsigned int nbits)
+{
+	if (small_const_nbits(nbits))
+		return (*dst = *src1 & ~(*src2) & BITMAP_LAST_WORD_MASK(nbits)) != 0;
+	return __bitmap_andnot(dst, src1, src2, nbits);
+}
+
 #endif /* _PERF_BITOPS_H */
diff --git a/tools/lib/bitmap.c b/tools/lib/bitmap.c
index 5043747ef6c5f..b6bc037623fc1 100644
--- a/tools/lib/bitmap.c
+++ b/tools/lib/bitmap.c
@@ -86,3 +86,18 @@ int __bitmap_equal(const unsigned long *bitmap1,
 
 	return 1;
 }
+
+int __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+                                const unsigned long *bitmap2, unsigned int bits)
+{
+	unsigned int k;
+	unsigned int lim = bits/BITS_PER_LONG;
+	unsigned long result = 0;
+
+	for (k = 0; k < lim; k++)
+		result |= (dst[k] = bitmap1[k] & ~bitmap2[k]);
+	if (bits % BITS_PER_LONG)
+		result |= (dst[k] = bitmap1[k] & ~bitmap2[k] &
+			BITMAP_LAST_WORD_MASK(bits));
+	return result != 0;
+}
diff --git a/tools/lib/lockdep/Build b/tools/lib/lockdep/Build
index 6f667355b0687..219a9e2d9e0ba 100644
--- a/tools/lib/lockdep/Build
+++ b/tools/lib/lockdep/Build
@@ -1 +1 @@
-liblockdep-y += common.o lockdep.o preload.o rbtree.o
+liblockdep-y += common.o lockdep.o preload.o rbtree.o ../../lib/bitmap.o
-- 
2.25.1

[PATCH v4 05/14] tools/lib/lockdep: add definition required for IRQ flag tracing

[Sasha Levin]

We are going to start building with CONFIG_TRACE_IRQFLAGS defined, so
let's wire up a few dummy variables in our task_struct.

This isn't needed in userspace, but due to some refactoring in
kernel-side lockdep it's easier to just wire it up and enable
CONFIG_TRACE_IRQFLAGS.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/lockdep.h | 8 ++++++++
 tools/lib/lockdep/lockdep.c   | 4 ++--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/tools/include/linux/lockdep.h b/tools/include/linux/lockdep.h
index e56997288f2b0..3a9924d6d3ae8 100644
--- a/tools/include/linux/lockdep.h
+++ b/tools/include/linux/lockdep.h
@@ -23,6 +23,8 @@
 
 #include "../../../include/linux/lockdep.h"
 
+static bool early_boot_irqs_disabled;
+
 struct task_struct {
 	u64 curr_chain_key;
 	int lockdep_depth;
@@ -31,6 +33,12 @@ struct task_struct {
 	gfp_t lockdep_reclaim_gfp;
 	int pid;
 	int state;
+	int softirqs_enabled, hardirqs_enabled, softirqs_disabled, hardirqs_disabled, irq_events;
+	unsigned long softirq_disable_ip, softirq_enable_ip;
+	unsigned int softirq_disable_event, softirq_enable_event;
+	unsigned long hardirq_disable_ip, hardirq_enable_ip;
+	unsigned int hardirq_disable_event, hardirq_enable_event;
+	int softirq_context, hardirq_context;
 	char comm[17];
 };
 
diff --git a/tools/lib/lockdep/lockdep.c b/tools/lib/lockdep/lockdep.c
index 348a9d0fb766a..9be12d233477a 100644
--- a/tools/lib/lockdep/lockdep.c
+++ b/tools/lib/lockdep/lockdep.c
@@ -15,10 +15,10 @@ u32 prandom_u32(void)
 	abort();
 }
 
-void print_irqtrace_events(struct task_struct *curr)
+/*void print_irqtrace_events(struct task_struct *curr)
 {
 	abort();
-}
+}*/
 
 static struct new_utsname *init_utsname(void)
 {
-- 
2.25.1

[PATCH v4 06/14] tools bitmap: add bitmap_clear definition

[Sasha Levin]

This is needed as a result of de4643a77356 ("locking/lockdep: Reuse lock
chains that have been freed").

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/lib/bitmap.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/lib/bitmap.c b/tools/lib/bitmap.c
index b6bc037623fc1..873176c501e43 100644
--- a/tools/lib/bitmap.c
+++ b/tools/lib/bitmap.c
@@ -101,3 +101,23 @@ int __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
 			BITMAP_LAST_WORD_MASK(bits));
 	return result != 0;
 }
+
+void bitmap_clear(unsigned long *map, unsigned int start, int len)
+{
+	unsigned long *p = map + BIT_WORD(start);
+	const unsigned int size = start + len;
+	int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
+	unsigned long mask_to_clear = BITMAP_FIRST_WORD_MASK(start);
+
+	while (len - bits_to_clear >= 0) {
+		*p &= ~mask_to_clear;
+		len -= bits_to_clear;
+		bits_to_clear = BITS_PER_LONG;
+		mask_to_clear = ~0UL;
+		p++;
+	}
+	if (len) {
+		mask_to_clear &= BITMAP_LAST_WORD_MASK(size);
+		*p &= ~mask_to_clear;
+	}
+}
-- 
2.25.1

[PATCH v4 07/14] tools/lib/lockdep: Hook up vsprintf, find_bit, hweight libraries

[Sasha Levin]

They already exist in tools/lib/, and are now required by liblockdep, so
just add them to the build manifest.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/lib/lockdep/Build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/lib/lockdep/Build b/tools/lib/lockdep/Build
index 219a9e2d9e0ba..1e7c25690b8ea 100644
--- a/tools/lib/lockdep/Build
+++ b/tools/lib/lockdep/Build
@@ -1 +1 @@
-liblockdep-y += common.o lockdep.o preload.o rbtree.o ../../lib/bitmap.o
+liblockdep-y += common.o lockdep.o preload.o rbtree.o ../../lib/bitmap.o ../../lib/vsprintf.o ../../lib/find_bit.o ../../lib/hweight.o
-- 
2.25.1

[PATCH v4 08/14] tools/lib/lockdep: Enable building with CONFIG_TRACE_IRQFLAGS

[Sasha Levin]

At this point it's always enabled with CONFIG_PROVE_LOCKING, so it's
easier to enable it in liblockdep as well rather than try and fix up the
lockdep code.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/lib/lockdep/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/lib/lockdep/Makefile b/tools/lib/lockdep/Makefile
index 9dafb8cb752fe..efb6b7980a009 100644
--- a/tools/lib/lockdep/Makefile
+++ b/tools/lib/lockdep/Makefile
@@ -78,7 +78,7 @@ export Q VERBOSE
 INCLUDES = -I. -I./uinclude -I./include -I../../include $(CONFIG_INCLUDES)
 
 # Set compile option CFLAGS if not set elsewhere
-CFLAGS ?= -g -DCONFIG_LOCKDEP -DCONFIG_STACKTRACE -DCONFIG_PROVE_LOCKING -DBITS_PER_LONG=__WORDSIZE -DLIBLOCKDEP_VERSION='"$(LIBLOCKDEP_VERSION)"' -rdynamic -O0 -g
+CFLAGS ?= -g -DCONFIG_LOCKDEP -DCONFIG_STACKTRACE -DCONFIG_PROVE_LOCKING -DCONFIG_TRACE_IRQFLAGS -DBITS_PER_LONG=__WORDSIZE -DLIBLOCKDEP_VERSION='"$(LIBLOCKDEP_VERSION)"' -rdynamic -O0 -g
 CFLAGS += -fPIC
 CFLAGS += -Wall
 
-- 
2.25.1

[PATCH v4 09/14] tools/lib/lockdep: New stacktrace API

[Sasha Levin]

The kernel switched to using kernel/stacktrace.c, and the API slightly
changed.

Adjust it to make stack traces work again.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/stacktrace.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/include/linux/stacktrace.h b/tools/include/linux/stacktrace.h
index ae343ac35bfa7..b290b98d883aa 100644
--- a/tools/include/linux/stacktrace.h
+++ b/tools/include/linux/stacktrace.h
@@ -15,10 +15,18 @@ static inline void print_stack_trace(struct stack_trace *trace, int spaces)
 	backtrace_symbols_fd((void **)trace->entries, trace->nr_entries, 1);
 }
 
+static inline void stack_trace_print(const long unsigned int *entries, unsigned int nr, int spaces)
+{
+	backtrace_symbols_fd((void **)entries, nr, 1);
+}
+
 #define save_stack_trace(trace)	\
 	((trace)->nr_entries =	\
 		backtrace((void **)(trace)->entries, (trace)->max_entries))
 
+#define stack_trace_save(e, m, x) \
+	backtrace((void **)(e), (m))
+
 static inline int dump_stack(void)
 {
 	void *array[64];
-- 
2.25.1

[PATCH v4 10/14] tools/lib/lockdep: call lockdep_init_task on init

[Sasha Levin]

We now have to explicitly call lockdep_init_task() when starting up.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/lib/lockdep/preload.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/lib/lockdep/preload.c b/tools/lib/lockdep/preload.c
index 8f1adbe887b2f..578fdeda9422c 100644
--- a/tools/lib/lockdep/preload.c
+++ b/tools/lib/lockdep/preload.c
@@ -9,6 +9,8 @@
 #include "include/liblockdep/mutex.h"
 #include "../../include/linux/rbtree.h"
 
+extern struct task_struct *__curr(void);
+
 /**
  * struct lock_lookup - liblockdep's view of a single unique lock
  * @orig: pointer to the original pthread lock, used for lookups
@@ -421,6 +423,8 @@ __attribute__((constructor)) static void init_preload(void)
 	if (__init_state == done)
 		return;
 
+	lockdep_init_task(__curr());
+
 #ifndef __GLIBC__
 	__init_state = prepare;
 
-- 
2.25.1

[PATCH v4 11/14] tools/lib/lockdep: switch to using lockdep_init_map_waits

[Sasha Levin]

As of de8f5e4f2dc1 ("lockdep: Introduce wait-type checks") lockdep
exports lockdep_init_map_waits() instead of lockdep_init_map() for
initialization.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/kernel.h                  | 2 ++
 tools/include/linux/lockdep.h                 | 1 +
 tools/lib/lockdep/include/liblockdep/common.h | 4 ++--
 tools/lib/lockdep/include/liblockdep/mutex.h  | 2 +-
 tools/lib/lockdep/include/liblockdep/rwlock.h | 2 +-
 tools/lib/lockdep/preload.c                   | 2 +-
 6 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/tools/include/linux/kernel.h b/tools/include/linux/kernel.h
index 902d3b9ab4c17..5ac493eef4237 100644
--- a/tools/include/linux/kernel.h
+++ b/tools/include/linux/kernel.h
@@ -124,4 +124,6 @@ static __maybe_unused int system_state;
 #define rcu_read_lock()
 #define rcu_read_unlock()
 
+#define in_nmi() 0
+
 #endif
diff --git a/tools/include/linux/lockdep.h b/tools/include/linux/lockdep.h
index 3a9924d6d3ae8..4e5f31f28ca3e 100644
--- a/tools/include/linux/lockdep.h
+++ b/tools/include/linux/lockdep.h
@@ -34,6 +34,7 @@ struct task_struct {
 	int pid;
 	int state;
 	int softirqs_enabled, hardirqs_enabled, softirqs_disabled, hardirqs_disabled, irq_events;
+	int hardirq_threaded, irq_config;
 	unsigned long softirq_disable_ip, softirq_enable_ip;
 	unsigned int softirq_disable_event, softirq_enable_event;
 	unsigned long hardirq_disable_ip, hardirq_enable_ip;
diff --git a/tools/lib/lockdep/include/liblockdep/common.h b/tools/lib/lockdep/include/liblockdep/common.h
index a6d7ee5f18ba9..5f698671f45c2 100644
--- a/tools/lib/lockdep/include/liblockdep/common.h
+++ b/tools/lib/lockdep/include/liblockdep/common.h
@@ -37,8 +37,8 @@ struct lockdep_map {
 #endif
 };
 
-void lockdep_init_map(struct lockdep_map *lock, const char *name,
-			struct lock_class_key *key, int subclass);
+void lockdep_init_map_waits(struct lockdep_map *lock, const char *name,
+	struct lock_class_key *key, int subclass, short inner, short outer);
 void lock_acquire(struct lockdep_map *lock, unsigned int subclass,
 			int trylock, int read, int check,
 			struct lockdep_map *nest_lock, unsigned long ip);
diff --git a/tools/lib/lockdep/include/liblockdep/mutex.h b/tools/lib/lockdep/include/liblockdep/mutex.h
index bd106b82759b7..6106fc73da687 100644
--- a/tools/lib/lockdep/include/liblockdep/mutex.h
+++ b/tools/lib/lockdep/include/liblockdep/mutex.h
@@ -24,7 +24,7 @@ static inline int __mutex_init(liblockdep_pthread_mutex_t *lock,
 				struct lock_class_key *key,
 				const pthread_mutexattr_t *__mutexattr)
 {
-	lockdep_init_map(&lock->dep_map, name, key, 0);
+	lockdep_init_map_waits(&lock->dep_map, name, key, 0, 0, 0);
 	return pthread_mutex_init(&lock->mutex, __mutexattr);
 }
 
diff --git a/tools/lib/lockdep/include/liblockdep/rwlock.h b/tools/lib/lockdep/include/liblockdep/rwlock.h
index 6d5d2932bf4d9..222748d04219d 100644
--- a/tools/lib/lockdep/include/liblockdep/rwlock.h
+++ b/tools/lib/lockdep/include/liblockdep/rwlock.h
@@ -23,7 +23,7 @@ static inline int __rwlock_init(liblockdep_pthread_rwlock_t *lock,
 				struct lock_class_key *key,
 				const pthread_rwlockattr_t *attr)
 {
-	lockdep_init_map(&lock->dep_map, name, key, 0);
+	lockdep_init_map_waits(&lock->dep_map, name, key, 0, 0, 0);
 
 	return pthread_rwlock_init(&lock->rwlock, attr);
 }
diff --git a/tools/lib/lockdep/preload.c b/tools/lib/lockdep/preload.c
index 578fdeda9422c..5bd58c51066c4 100644
--- a/tools/lib/lockdep/preload.c
+++ b/tools/lib/lockdep/preload.c
@@ -199,7 +199,7 @@ static struct lock_lookup *__get_lock(void *lock)
 	 * TODO: Get the real name of the lock using libdwarf
 	 */
 	sprintf(l->name, "%p", lock);
-	lockdep_init_map(&l->dep_map, l->name, &l->key, 0);
+//	lockdep_init_map_waits(&l->dep_map, l->name, &l->key, 0, 0, 0);
 
 	ll_pthread_rwlock_wrlock(&locks_rwlock);
 	/* This might have changed since the last time we fetched it */
-- 
2.25.1

[PATCH v4 12/14] tools/kernel.h: hide noinstr

[Sasha Levin]

After 655389666643 ("vmlinux.lds.h: Create section for protection against
instrumentation") we need to ignore "noinstr" in userspace.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/kernel.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/include/linux/kernel.h b/tools/include/linux/kernel.h
index 5ac493eef4237..39fcbb144a34a 100644
--- a/tools/include/linux/kernel.h
+++ b/tools/include/linux/kernel.h
@@ -126,4 +126,6 @@ static __maybe_unused int system_state;
 
 #define in_nmi() 0
 
+#define noinstr
+
 #endif
-- 
2.25.1

[PATCH v4 13/14] tools/lib/lockdep: explicitly declare lockdep_init_task()

[Sasha Levin]

Avoid a warning as we can't pull the decleration for lockdep's header.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/lib/lockdep/preload.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/lib/lockdep/preload.c b/tools/lib/lockdep/preload.c
index 5bd58c51066c4..2928c712f87f8 100644
--- a/tools/lib/lockdep/preload.c
+++ b/tools/lib/lockdep/preload.c
@@ -92,6 +92,8 @@ static int (*ll_pthread_rwlock_trywrlock)(pthread_rwlock_t *rwlock)	= __pthread_
 static int (*ll_pthread_rwlock_wrlock)(pthread_rwlock_t *rwlock)	= __pthread_rwlock_wrlock;
 static int (*ll_pthread_rwlock_unlock)(pthread_rwlock_t *rwlock)	= __pthread_rwlock_unlock;
 
+extern void lockdep_init_task(struct task_struct *task);
+
 enum { none, prepare, done, } __init_state;
 static void init_preload(void);
 static void try_init_preload(void)
-- 
2.25.1

[PATCH v4 14/14] tools/kernel.h: hide task_struct.hardirq_chain_key

[Sasha Levin]

As a result of c86e9b987cea ("lockdep: Prepare for noinstr sections") we need
to hide hardirq_chain_key from userspace to avoid a build error.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/include/linux/lockdep.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/include/linux/lockdep.h b/tools/include/linux/lockdep.h
index 4e5f31f28ca3e..b6c9e81eb45e5 100644
--- a/tools/include/linux/lockdep.h
+++ b/tools/include/linux/lockdep.h
@@ -40,6 +40,7 @@ struct task_struct {
 	unsigned long hardirq_disable_ip, hardirq_enable_ip;
 	unsigned int hardirq_disable_event, hardirq_enable_event;
 	int softirq_context, hardirq_context;
+	unsigned long hardirq_chain_key;
 	char comm[17];
 };
 
-- 
2.25.1

Re: [PATCH v4 00/14] liblockdep fixes for 5.9-rc1

[Ingo Molnar]

* Sasha Levin <sashal@kernel.org> wrote:

> Hi Linus,
> 
> Please consider applying these patches for liblockdep, or alternatively
> pull from:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linux.git tags/liblockdep-fixes-040820
> 
> The patches fix up compilation and functionality of liblockdep on 5.8,
> they were tested using liblockdep's internal testsuite.
> 
> I was unable to get the x86 folks to pull these fixes for the past few
> months:

So the primary reason I didn't pull is that liblockdep was permanently 
build-broken from February 2019 to around February 2020, despite me 
pinging you multiple times about it.

>  - https://lkml.org/lkml/2020/2/17/1089

This pull request still said that if fixes "most of" liblockdep, not 
"all of", which is the benchmark really after such a long series of 
breakage.

>  - https://lkml.org/lkml/2020/4/18/817

This still said "most of".

>  - https://lkml.org/lkml/2020/6/22/1262

Same 'most of' verbiage.

> Which is why this pull request ends up going straight to you.

So at this point I think we need to ask whether it's worth it: are 
there any actual users of liblockdep, besides the testcases in 
liblockdep itself? I see there's a 'liblockdep-dev' package for 
Debian, but not propagated to Ubuntu or other popular variants AFAICS.

Also, could you please specify whether all bugs are fixed or just 
'most'?

> Sasha Levin (14):
>   tools headers: Add kprobes.h header
>   tools headers: Add rcupdate.h header
>   tools/kernel.h: extend with dummy RCU functions
>   tools bitmap: add bitmap_andnot definition
>   tools/lib/lockdep: add definition required for IRQ flag tracing
>   tools bitmap: add bitmap_clear definition
>   tools/lib/lockdep: Hook up vsprintf, find_bit, hweight libraries
>   tools/lib/lockdep: Enable building with CONFIG_TRACE_IRQFLAGS
>   tools/lib/lockdep: New stacktrace API
>   tools/lib/lockdep: call lockdep_init_task on init
>   tools/lib/lockdep: switch to using lockdep_init_map_waits
>   tools/kernel.h: hide noinstr
>   tools/lib/lockdep: explicitly declare lockdep_init_task()
>   tools/kernel.h: hide task_struct.hardirq_chain_key

Style nits, please use consistent titles for patches:

 - First word should be capitalized consistently, instead of mismash 
   of lower case mixed with upper case.

 - First word should preferably be a verb, i.e. "Add new stacktrace 
   API stubs", not "New stacktrace API"

Also, please always check linux-next whether there's some new upstream 
changes that liblockdep needs to adapt to. Right now there's a new 
build breakage even with all your fixes applied:

  thule:~/tip/tools/lib/lockdep> make
    CC       common.o
  In file included from ../../include/linux/lockdep.h:24,
                   from common.c:5:
   ../../include/linux/../../../include/linux/lockdep.h:13:10: fatal error: linux/lockdep_types.h: No such file or directory
     13 | #include <linux/lockdep_types.h>
        |          ^~~~~~~~~~~~~~~~~~~~~~~

At which point we need to step back and analyze the development model: 
this comparatively high rate of breakage derives from the unorthodox 
direct coupling of a kernel subsystem to a user-space library.

The solution for that would be to use the method how perf syncs to 
kernel space headers, by maintaining a 100% copy in tools/include/ and 
having automatic mechanism that warns about out of sync headers but 
doesn't break functionality.

See tools/perf/check-headers.sh for details.

I believe this same half-automated sync-on-upstream-changes model 
could be used for liblockdep as well, i.e. lets copy kernel/lockdep.c 
and lockdep*.h over to tools/lib/lockdep/, and reuse the perf header 
syncing method to keep it synchronized from that point on.

That would result in a far more maintainable liblockdep end result 
IMO?

Thanks,

	Ingo

Re: [PATCH v4 00/14] liblockdep fixes for 5.9-rc1

[Sasha Levin]

On Thu, Aug 06, 2020 at 10:09:41AM +0200, Ingo Molnar wrote:
>
>* Sasha Levin <sashal@kernel.org> wrote:
>
>> Hi Linus,
>>
>> Please consider applying these patches for liblockdep, or alternatively
>> pull from:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linux.git tags/liblockdep-fixes-040820
>>
>> The patches fix up compilation and functionality of liblockdep on 5.8,
>> they were tested using liblockdep's internal testsuite.
>>
>> I was unable to get the x86 folks to pull these fixes for the past few
>> months:
>
>So the primary reason I didn't pull is that liblockdep was permanently
>build-broken from February 2019 to around February 2020, despite me
>pinging you multiple times about it.

I'm not sure how ignoring me for additional 6 months helped the state of
liblockdep.

>>  - https://lkml.org/lkml/2020/2/17/1089
>
>This pull request still said that if fixes "most of" liblockdep, not
>"all of", which is the benchmark really after such a long series of
>breakage.
>
>>  - https://lkml.org/lkml/2020/4/18/817
>
>This still said "most of".
>
>>  - https://lkml.org/lkml/2020/6/22/1262
>
>Same 'most of' verbiage.

Right, and there are still benign build warnings around unused
variables which doesn't impede liblockdep's functionality.

Should they be cleaned up? sure, but let's get it working again.

>> Which is why this pull request ends up going straight to you.
>
>So at this point I think we need to ask whether it's worth it: are
>there any actual users of liblockdep, besides the testcases in
>liblockdep itself? I see there's a 'liblockdep-dev' package for
>Debian, but not propagated to Ubuntu or other popular variants AFAICS.

Right, I can't really say how many users of this there are out there. I
get an occasional email with a question or comment but I'm not aware of
how many of those users are out there.

I do think that if we keep liblockdep alive we should do a better job at
not taking commits that break it to begin with. The model where someone
is chasing lockdep to fix breakage after breakage isn't sustainable.

>Also, could you please specify whether all bugs are fixed or just
>'most'?

It passes the testsuite, there are compiler warnings that show up on
some gcc versions I'll work on cleaning up.

>> Sasha Levin (14):
>>   tools headers: Add kprobes.h header
>>   tools headers: Add rcupdate.h header
>>   tools/kernel.h: extend with dummy RCU functions
>>   tools bitmap: add bitmap_andnot definition
>>   tools/lib/lockdep: add definition required for IRQ flag tracing
>>   tools bitmap: add bitmap_clear definition
>>   tools/lib/lockdep: Hook up vsprintf, find_bit, hweight libraries
>>   tools/lib/lockdep: Enable building with CONFIG_TRACE_IRQFLAGS
>>   tools/lib/lockdep: New stacktrace API
>>   tools/lib/lockdep: call lockdep_init_task on init
>>   tools/lib/lockdep: switch to using lockdep_init_map_waits
>>   tools/kernel.h: hide noinstr
>>   tools/lib/lockdep: explicitly declare lockdep_init_task()
>>   tools/kernel.h: hide task_struct.hardirq_chain_key
>
>Style nits, please use consistent titles for patches:
>
> - First word should be capitalized consistently, instead of mismash
>   of lower case mixed with upper case.
>
> - First word should preferably be a verb, i.e. "Add new stacktrace
>   API stubs", not "New stacktrace API"

I'll address these.

>Also, please always check linux-next whether there's some new upstream
>changes that liblockdep needs to adapt to. Right now there's a new
>build breakage even with all your fixes applied:
>
>  thule:~/tip/tools/lib/lockdep> make
>    CC       common.o
>  In file included from ../../include/linux/lockdep.h:24,
>                   from common.c:5:
>   ../../include/linux/../../../include/linux/lockdep.h:13:10: fatal error: linux/lockdep_types.h: No such file or directory
>     13 | #include <linux/lockdep_types.h>
>        |          ^~~~~~~~~~~~~~~~~~~~~~~
>
>At which point we need to step back and analyze the development model:
>this comparatively high rate of breakage derives from the unorthodox
>direct coupling of a kernel subsystem to a user-space library.
>
>The solution for that would be to use the method how perf syncs to
>kernel space headers, by maintaining a 100% copy in tools/include/ and
>having automatic mechanism that warns about out of sync headers but
>doesn't break functionality.
>
>See tools/perf/check-headers.sh for details.
>
>I believe this same half-automated sync-on-upstream-changes model
>could be used for liblockdep as well, i.e. lets copy kernel/lockdep.c
>and lockdep*.h over to tools/lib/lockdep/, and reuse the perf header
>syncing method to keep it synchronized from that point on.
>
>That would result in a far more maintainable liblockdep end result
>IMO?

How does perf handle doing changes on top of those headers? As an
example, if you look at patch #12, it basically makes userspace ignore
the new "noinstr" annotation, otherwise compiling code that uses
"noinstr" in userspace breaks badly.

For liblockdep it's not enough to just sync headers, we need to do
changes to those headers as well.

-- 
Thanks,
Sasha