[PATCH 0/3] livepatch: Allow user to specify functions to search for on a stack

[PATCH 0/3] livepatch: Allow user to specify functions to search for on a stack

[Miroslav Benes]

livepatch's consistency model requires that no live patched function
must be found on any task's stack during a transition process after a
live patch is applied. It is achieved by walking through stacks of all
blocked tasks.

The user might also want to define more functions to search for without
them being patched at all. It may either help with preparing a live
patch, which would otherwise require additional touches to achieve the
consistency, or it can be used to overcome deficiencies the stack
checking inherently has. For example, GCC may optimize a function so
that a part of it is moved to a different section and the function would
jump to it. This child function would not be found on a stack in this
case, but it may be important to search for it so that, again, the
consistency is achieved.

The patch set adds a new API which allows the user to specify such
functions.

The first patch is only preparatory. The main work is in the second one.
The third patch adds a test.

Originally, I wanted to add it all to klp_patch struct, but it makes
more sense to do it on klp_object level. It is also easier to reuse as
much of the existing code as possible in that case.

I am not good with naming so bike-shedding is welcome. Reviews even
more.

Miroslav Benes (3):
  livepatch: Move the initialization of old_func to a new function
  livepatch: Allow user to specify functions to search for on a stack
  selftests/livepatch: Test of the API for specifying functions to
    search for on a stack

 include/linux/livepatch.h                     | 11 +++
 kernel/livepatch/core.c                       | 50 ++++++++---
 kernel/livepatch/transition.c                 | 21 +++--
 lib/Kconfig.debug                             |  1 +
 lib/livepatch/Makefile                        |  4 +-
 lib/livepatch/test_klp_funcstack_demo.c       | 61 +++++++++++++
 lib/livepatch/test_klp_funcstack_mod.c        | 72 +++++++++++++++
 tools/testing/selftests/livepatch/Makefile    |  3 +-
 .../selftests/livepatch/test-func-stack.sh    | 88 +++++++++++++++++++
 9 files changed, 293 insertions(+), 18 deletions(-)
 create mode 100644 lib/livepatch/test_klp_funcstack_demo.c
 create mode 100644 lib/livepatch/test_klp_funcstack_mod.c
 create mode 100755 tools/testing/selftests/livepatch/test-func-stack.sh

-- 
2.33.1

[PATCH 1/3] livepatch: Move the initialization of old_func to a new function

[Miroslav Benes]

struct klp_func will be used not only for functions to be patched but
also for functions which must not be found on a stack. Move the
initialization of needed struct members to a separate function, so the
code can be reused.

Signed-off-by: Miroslav Benes <mbenes@suse.cz>
---
 kernel/livepatch/core.c | 34 +++++++++++++++++++++++-----------
 1 file changed, 23 insertions(+), 11 deletions(-)

diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
index 335d988bd811..3d8e3caf9f92 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -767,6 +767,28 @@ static int klp_apply_object_relocs(struct klp_patch *patch,
 	return 0;
 }
 
+static int klp_init_old_func(struct klp_object *obj,
+			     struct klp_func *func)
+{
+	int ret;
+
+	ret = klp_find_object_symbol(obj->name, func->old_name,
+				     func->old_sympos,
+				     (unsigned long *)&func->old_func);
+	if (ret)
+		return ret;
+
+	ret = kallsyms_lookup_size_offset((unsigned long)func->old_func,
+					  &func->old_size, NULL);
+	if (!ret) {
+		pr_err("kallsyms size lookup failed for '%s'\n",
+		       func->old_name);
+		return -ENOENT;
+	}
+
+	return 0;
+}
+
 /* parts of the initialization that is done only when the object is loaded */
 static int klp_init_object_loaded(struct klp_patch *patch,
 				  struct klp_object *obj)
@@ -787,20 +809,10 @@ static int klp_init_object_loaded(struct klp_patch *patch,
 	}
 
 	klp_for_each_func(obj, func) {
-		ret = klp_find_object_symbol(obj->name, func->old_name,
-					     func->old_sympos,
-					     (unsigned long *)&func->old_func);
+		ret = klp_init_old_func(obj, func);
 		if (ret)
 			return ret;
 
-		ret = kallsyms_lookup_size_offset((unsigned long)func->old_func,
-						  &func->old_size, NULL);
-		if (!ret) {
-			pr_err("kallsyms size lookup failed for '%s'\n",
-			       func->old_name);
-			return -ENOENT;
-		}
-
 		if (func->nop)
 			func->new_func = func->old_func;
 
-- 
2.33.1

[PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Miroslav Benes]

livepatch's consistency model requires that no live patched function
must be found on any task's stack during a transition process after a
live patch is applied. It is achieved by walking through stacks of all
blocked tasks.

The user might also want to define more functions to search for without
them being patched at all. It may either help with preparing a live
patch, which would otherwise require additional touches to achieve the
consistency, or it can be used to overcome deficiencies the stack
checking inherently has. For example, GCC may optimize a function so
that a part of it is moved to a different section and the function would
jump to it. This child function would not be found on a stack in this
case, but it may be important to search for it so that, again, the
consistency is achieved.

Allow the user to specify such functions on klp_object level.

Signed-off-by: Miroslav Benes <mbenes@suse.cz>
---
 include/linux/livepatch.h     | 11 +++++++++++
 kernel/livepatch/core.c       | 16 ++++++++++++++++
 kernel/livepatch/transition.c | 21 ++++++++++++++++-----
 3 files changed, 43 insertions(+), 5 deletions(-)

diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
index 2614247a9781..89df578af8c3 100644
--- a/include/linux/livepatch.h
+++ b/include/linux/livepatch.h
@@ -106,9 +106,11 @@ struct klp_callbacks {
  * struct klp_object - kernel object structure for live patching
  * @name:	module name (or NULL for vmlinux)
  * @funcs:	function entries for functions to be patched in the object
+ * @funcs_stack:	function entries for functions to be stack checked
  * @callbacks:	functions to be executed pre/post (un)patching
  * @kobj:	kobject for sysfs resources
  * @func_list:	dynamic list of the function entries
+ * @func_stack_list:	dynamic list of the function entries for stack checking
  * @node:	list node for klp_patch obj_list
  * @mod:	kernel module associated with the patched object
  *		(NULL for vmlinux)
@@ -119,11 +121,13 @@ struct klp_object {
 	/* external */
 	const char *name;
 	struct klp_func *funcs;
+	struct klp_func *funcs_stack;
 	struct klp_callbacks callbacks;
 
 	/* internal */
 	struct kobject kobj;
 	struct list_head func_list;
+	struct list_head func_stack_list;
 	struct list_head node;
 	struct module *mod;
 	bool dynamic;
@@ -187,12 +191,19 @@ struct klp_patch {
 	     func->old_name || func->new_func || func->old_sympos; \
 	     func++)
 
+#define klp_for_each_func_stack_static(obj, func) \
+	for (func = obj->funcs_stack; \
+	     func && (func->old_name || func->old_sympos); func++)
+
 #define klp_for_each_func_safe(obj, func, tmp_func)			\
 	list_for_each_entry_safe(func, tmp_func, &obj->func_list, node)
 
 #define klp_for_each_func(obj, func)	\
 	list_for_each_entry(func, &obj->func_list, node)
 
+#define klp_for_each_func_stack(obj, func)	\
+	list_for_each_entry(func, &obj->func_stack_list, node)
+
 int klp_enable_patch(struct klp_patch *);
 
 /* Called from the module loader during module coming/going states */
diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
index 3d8e3caf9f92..86fc73a06844 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -825,6 +825,12 @@ static int klp_init_object_loaded(struct klp_patch *patch,
 		}
 	}
 
+	klp_for_each_func_stack(obj, func) {
+		ret = klp_init_old_func(obj, func);
+		if (ret)
+			return ret;
+	}
+
 	return 0;
 }
 
@@ -853,6 +859,11 @@ static int klp_init_object(struct klp_patch *patch, struct klp_object *obj)
 			return ret;
 	}
 
+	klp_for_each_func_stack(obj, func) {
+		if (strlen(func->old_name) >= KSYM_NAME_LEN)
+			return -EINVAL;
+	}
+
 	if (klp_is_object_loaded(obj))
 		ret = klp_init_object_loaded(patch, obj);
 
@@ -870,6 +881,7 @@ static void klp_init_object_early(struct klp_patch *patch,
 				  struct klp_object *obj)
 {
 	INIT_LIST_HEAD(&obj->func_list);
+	INIT_LIST_HEAD(&obj->func_stack_list);
 	kobject_init(&obj->kobj, &klp_ktype_object);
 	list_add_tail(&obj->node, &patch->obj_list);
 }
@@ -899,6 +911,10 @@ static int klp_init_patch_early(struct klp_patch *patch)
 		klp_for_each_func_static(obj, func) {
 			klp_init_func_early(obj, func);
 		}
+
+		klp_for_each_func_stack_static(obj, func) {
+			list_add_tail(&func->node, &obj->func_stack_list);
+		}
 	}
 
 	if (!try_module_get(patch->mod))
diff --git a/kernel/livepatch/transition.c b/kernel/livepatch/transition.c
index 5683ac0d2566..be7afc5dc275 100644
--- a/kernel/livepatch/transition.c
+++ b/kernel/livepatch/transition.c
@@ -200,7 +200,10 @@ static int klp_check_stack_func(struct klp_func *func, unsigned long *entries,
 	for (i = 0; i < nr_entries; i++) {
 		address = entries[i];
 
-		if (klp_target_state == KLP_UNPATCHED) {
+		if (!func->new_func) {
+			func_addr = (unsigned long)func->old_func;
+			func_size = func->old_size;
+		} else if (klp_target_state == KLP_UNPATCHED) {
 			 /*
 			  * Check for the to-be-unpatched function
 			  * (the func itself).
@@ -256,14 +259,22 @@ static int klp_check_stack(struct task_struct *task, const char **oldname)
 			continue;
 		klp_for_each_func(obj, func) {
 			ret = klp_check_stack_func(func, entries, nr_entries);
-			if (ret) {
-				*oldname = func->old_name;
-				return -EADDRINUSE;
-			}
+			if (ret)
+				goto err;
+		}
+
+		klp_for_each_func_stack(obj, func) {
+			ret = klp_check_stack_func(func, entries, nr_entries);
+			if (ret)
+				goto err;
 		}
 	}
 
 	return 0;
+
+err:
+	*oldname = func->old_name;
+	return -EADDRINUSE;
 }
 
 static int klp_check_and_switch_task(struct task_struct *task, void *arg)
-- 
2.33.1

Re: [PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Peter Zijlstra]

On Fri, Nov 19, 2021 at 10:03:26AM +0100, Miroslav Benes wrote:
> livepatch's consistency model requires that no live patched function
> must be found on any task's stack during a transition process after a
> live patch is applied. It is achieved by walking through stacks of all
> blocked tasks.
> 
> The user might also want to define more functions to search for without
> them being patched at all. It may either help with preparing a live
> patch, which would otherwise require additional touches to achieve the
> consistency, or it can be used to overcome deficiencies the stack
> checking inherently has. For example, GCC may optimize a function so
> that a part of it is moved to a different section and the function would
> jump to it. This child function would not be found on a stack in this
> case, but it may be important to search for it so that, again, the
> consistency is achieved.
> 
> Allow the user to specify such functions on klp_object level.

Ok, so this relies on the patch generator to DTRT but then it should
work.

Thanks!

Re: [PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Josh Poimboeuf]

Thanks for doing this!  And at peterz-esque speed no less :-)

On Fri, Nov 19, 2021 at 10:03:26AM +0100, Miroslav Benes wrote:
> livepatch's consistency model requires that no live patched function
> must be found on any task's stack during a transition process after a
> live patch is applied. It is achieved by walking through stacks of all
> blocked tasks.
> 
> The user might also want to define more functions to search for without
> them being patched at all. It may either help with preparing a live
> patch, which would otherwise require additional touches to achieve the
> consistency

Do we have any examples of this situation we can add to the commit log?

> or it can be used to overcome deficiencies the stack
> checking inherently has. For example, GCC may optimize a function so
> that a part of it is moved to a different section and the function would
> jump to it. This child function would not be found on a stack in this
> case, but it may be important to search for it so that, again, the
> consistency is achieved.
> 
> Allow the user to specify such functions on klp_object level.
> 
> Signed-off-by: Miroslav Benes <mbenes@suse.cz>
> ---
>  include/linux/livepatch.h     | 11 +++++++++++
>  kernel/livepatch/core.c       | 16 ++++++++++++++++
>  kernel/livepatch/transition.c | 21 ++++++++++++++++-----
>  3 files changed, 43 insertions(+), 5 deletions(-)
> 
> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> index 2614247a9781..89df578af8c3 100644
> --- a/include/linux/livepatch.h
> +++ b/include/linux/livepatch.h
> @@ -106,9 +106,11 @@ struct klp_callbacks {
>   * struct klp_object - kernel object structure for live patching
>   * @name:	module name (or NULL for vmlinux)
>   * @funcs:	function entries for functions to be patched in the object
> + * @funcs_stack:	function entries for functions to be stack checked

So there are two arrays/lists of 'klp_func', and two implied meanings of
what a 'klp_func' is and how it's initialized.

Might it be simpler and more explicit to just add a new external field
to 'klp_func' and continue to have a single 'funcs' array?  Similar to
what we already do with the special-casing of 'nop', except it would be
an external field, e.g. 'no_patch' or 'stack_only'.

Then instead of all the extra klp_for_each_func_stack_static()
incantations, and the special cases in higher-level callers like
klp_init_object() and klp_init_patch_early(), the lower-level functions
like klp_init_func() and klp_init_func_early() can check the field to
determine which initializations need to be made.  Which is kind of nice
IMO as it pushes that detail down more where it belongs.  And makes the
different types of 'klp_func' more explicit.

-- 
Josh

Re: [PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Miroslav Benes]

On Fri, 19 Nov 2021, Josh Poimboeuf wrote:

> Thanks for doing this!  And at peterz-esque speed no less :-)
> 
> On Fri, Nov 19, 2021 at 10:03:26AM +0100, Miroslav Benes wrote:
> > livepatch's consistency model requires that no live patched function
> > must be found on any task's stack during a transition process after a
> > live patch is applied. It is achieved by walking through stacks of all
> > blocked tasks.
> > 
> > The user might also want to define more functions to search for without
> > them being patched at all. It may either help with preparing a live
> > patch, which would otherwise require additional touches to achieve the
> > consistency
> 
> Do we have any examples of this situation we can add to the commit log?

I do not have anything at hand. Joe, do you remember the case you 
mentioned previously about adding a nop to a function?
 
> > or it can be used to overcome deficiencies the stack
> > checking inherently has. For example, GCC may optimize a function so
> > that a part of it is moved to a different section and the function would
> > jump to it. This child function would not be found on a stack in this
> > case, but it may be important to search for it so that, again, the
> > consistency is achieved.
> > 
> > Allow the user to specify such functions on klp_object level.
> > 
> > Signed-off-by: Miroslav Benes <mbenes@suse.cz>
> > ---
> >  include/linux/livepatch.h     | 11 +++++++++++
> >  kernel/livepatch/core.c       | 16 ++++++++++++++++
> >  kernel/livepatch/transition.c | 21 ++++++++++++++++-----
> >  3 files changed, 43 insertions(+), 5 deletions(-)
> > 
> > diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> > index 2614247a9781..89df578af8c3 100644
> > --- a/include/linux/livepatch.h
> > +++ b/include/linux/livepatch.h
> > @@ -106,9 +106,11 @@ struct klp_callbacks {
> >   * struct klp_object - kernel object structure for live patching
> >   * @name:	module name (or NULL for vmlinux)
> >   * @funcs:	function entries for functions to be patched in the object
> > + * @funcs_stack:	function entries for functions to be stack checked
> 
> So there are two arrays/lists of 'klp_func', and two implied meanings of
> what a 'klp_func' is and how it's initialized.
> 
> Might it be simpler and more explicit to just add a new external field
> to 'klp_func' and continue to have a single 'funcs' array?  Similar to
> what we already do with the special-casing of 'nop', except it would be
> an external field, e.g. 'no_patch' or 'stack_only'.
> 
> Then instead of all the extra klp_for_each_func_stack_static()
> incantations, and the special cases in higher-level callers like
> klp_init_object() and klp_init_patch_early(), the lower-level functions
> like klp_init_func() and klp_init_func_early() can check the field to
> determine which initializations need to be made.  Which is kind of nice
> IMO as it pushes that detail down more where it belongs.  And makes the
> different types of 'klp_func' more explicit.

I thought about doing this for a moment but then I was worried there would 
be many places which would require special-casing, so I tried to keep it 
separate. But yes, it would be cleaner, so definitely worth trying for v2.

Thanks

Miroslav

Re: [PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Joe Lawrence]

On 11/22/21 2:57 AM, Miroslav Benes wrote:
> On Fri, 19 Nov 2021, Josh Poimboeuf wrote:
> 
>> Thanks for doing this!  And at peterz-esque speed no less :-)
>>
>> On Fri, Nov 19, 2021 at 10:03:26AM +0100, Miroslav Benes wrote:
>>> livepatch's consistency model requires that no live patched function
>>> must be found on any task's stack during a transition process after a
>>> live patch is applied. It is achieved by walking through stacks of all
>>> blocked tasks.
>>>
>>> The user might also want to define more functions to search for without
>>> them being patched at all. It may either help with preparing a live
>>> patch, which would otherwise require additional touches to achieve the
>>> consistency
>>
>> Do we have any examples of this situation we can add to the commit log?
> 
> I do not have anything at hand. Joe, do you remember the case you 
> mentioned previously about adding a nop to a function?
>  

I went looking in my inbox to see... Unfortunately the closest thing I
found was a kpatchset in which we added nops to coax kpatch-build into
reverting previous patch version changes.  Not applicable here, but I
was certain we entertained the same idea to increase the task stack
check for some other problem.

Maybe adding a hypothetical scenario to the commit log would suffice?

>>> or it can be used to overcome deficiencies the stack
>>> checking inherently has. For example, GCC may optimize a function so
>>> that a part of it is moved to a different section and the function would
>>> jump to it. This child function would not be found on a stack in this
>>> case, but it may be important to search for it so that, again, the
>>> consistency is achieved.
>>>
>>> Allow the user to specify such functions on klp_object level.
>>>
>>> Signed-off-by: Miroslav Benes <mbenes@suse.cz>
>>> ---
>>>  include/linux/livepatch.h     | 11 +++++++++++
>>>  kernel/livepatch/core.c       | 16 ++++++++++++++++
>>>  kernel/livepatch/transition.c | 21 ++++++++++++++++-----
>>>  3 files changed, 43 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>>> index 2614247a9781..89df578af8c3 100644
>>> --- a/include/linux/livepatch.h
>>> +++ b/include/linux/livepatch.h
>>> @@ -106,9 +106,11 @@ struct klp_callbacks {
>>>   * struct klp_object - kernel object structure for live patching
>>>   * @name:	module name (or NULL for vmlinux)
>>>   * @funcs:	function entries for functions to be patched in the object
>>> + * @funcs_stack:	function entries for functions to be stack checked
>>
>> So there are two arrays/lists of 'klp_func', and two implied meanings of
>> what a 'klp_func' is and how it's initialized.
>>
>> Might it be simpler and more explicit to just add a new external field
>> to 'klp_func' and continue to have a single 'funcs' array?  Similar to
>> what we already do with the special-casing of 'nop', except it would be
>> an external field, e.g. 'no_patch' or 'stack_only'.
>>
>> Then instead of all the extra klp_for_each_func_stack_static()
>> incantations, and the special cases in higher-level callers like
>> klp_init_object() and klp_init_patch_early(), the lower-level functions
>> like klp_init_func() and klp_init_func_early() can check the field to
>> determine which initializations need to be made.  Which is kind of nice
>> IMO as it pushes that detail down more where it belongs.  And makes the
>> different types of 'klp_func' more explicit.
> 
> I thought about doing this for a moment but then I was worried there would 
> be many places which would require special-casing, so I tried to keep it 
> separate. But yes, it would be cleaner, so definitely worth trying for v2.
> 

I'll add that the first thing that came to mind when you raised this
feature idea in the other thread was to support existing klp_funcs array
with NULL new_func's.  I didn't go look to see how invasive it would be,
but it will be interesting to see if a single list approach turns out
any simpler for v2.

-- 
Joe

Re: [PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Petr Mladek]

On Mon 2021-11-22 10:53:21, Joe Lawrence wrote:
> On 11/22/21 2:57 AM, Miroslav Benes wrote:
> > On Fri, 19 Nov 2021, Josh Poimboeuf wrote:
> > 
> >> Thanks for doing this!  And at peterz-esque speed no less :-)
> >>
> >> On Fri, Nov 19, 2021 at 10:03:26AM +0100, Miroslav Benes wrote:
> >>> livepatch's consistency model requires that no live patched function
> >>> must be found on any task's stack during a transition process after a
> >>> live patch is applied. It is achieved by walking through stacks of all
> >>> blocked tasks.
> >>>
> >>> The user might also want to define more functions to search for without
> >>> them being patched at all. It may either help with preparing a live
> >>> patch, which would otherwise require additional touches to achieve the
> >>> consistency
> >>
> >> Do we have any examples of this situation we can add to the commit log?
> > 
> > I do not have anything at hand. Joe, do you remember the case you 
> > mentioned previously about adding a nop to a function?
> 
> Maybe adding a hypothetical scenario to the commit log would suffice?

I wonder if we could describe a scenario based on the thread about
.cold code variants, see
https://lore.kernel.org/all/20211112015003.pefl656m3zmir6ov@treble/

This feature would allow to safely livepatch already released
kernels where the unwinder is not able to reliably detect
a newly discovered problems.

> >>> or it can be used to overcome deficiencies the stack
> >>> checking inherently has. For example, GCC may optimize a function so
> >>> that a part of it is moved to a different section and the function would
> >>> jump to it. This child function would not be found on a stack in this
> >>> case, but it may be important to search for it so that, again, the
> >>> consistency is achieved.
> >>>
> >>> Allow the user to specify such functions on klp_object level.
> >>>
> >>> Signed-off-by: Miroslav Benes <mbenes@suse.cz>
> >>> ---
> >>>  include/linux/livepatch.h     | 11 +++++++++++
> >>>  kernel/livepatch/core.c       | 16 ++++++++++++++++
> >>>  kernel/livepatch/transition.c | 21 ++++++++++++++++-----
> >>>  3 files changed, 43 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> >>> index 2614247a9781..89df578af8c3 100644
> >>> --- a/include/linux/livepatch.h
> >>> +++ b/include/linux/livepatch.h
> >>> @@ -106,9 +106,11 @@ struct klp_callbacks {
> >>>   * struct klp_object - kernel object structure for live patching
> >>>   * @name:	module name (or NULL for vmlinux)
> >>>   * @funcs:	function entries for functions to be patched in the object
> >>> + * @funcs_stack:	function entries for functions to be stack checked
> >>
> >> So there are two arrays/lists of 'klp_func', and two implied meanings of
> >> what a 'klp_func' is and how it's initialized.
> >>
> >> Might it be simpler and more explicit to just add a new external field
> >> to 'klp_func' and continue to have a single 'funcs' array?  Similar to
> >> what we already do with the special-casing of 'nop', except it would be
> >> an external field, e.g. 'no_patch' or 'stack_only'.
> 
> I'll add that the first thing that came to mind when you raised this
> feature idea in the other thread was to support existing klp_funcs array
> with NULL new_func's.

Please, solve this with the extra flag, e.g. .stack_only, as
already suggested. It will help to distinguish mistakes and
intentions. Also it will allow to find these symbols by grep.

> I didn't go look to see how invasive it would be,
> but it will be interesting to see if a single list approach turns out
> any simpler for v2.

I am not sure either. But I expect that it will be easier than
the extra array.

Best Regards,
Petr

Re: [PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Miroslav Benes]

On Thu, 25 Nov 2021, Petr Mladek wrote:

> On Mon 2021-11-22 10:53:21, Joe Lawrence wrote:
> > On 11/22/21 2:57 AM, Miroslav Benes wrote:
> > > On Fri, 19 Nov 2021, Josh Poimboeuf wrote:
> > > 
> > >> Thanks for doing this!  And at peterz-esque speed no less :-)
> > >>
> > >> On Fri, Nov 19, 2021 at 10:03:26AM +0100, Miroslav Benes wrote:
> > >>> livepatch's consistency model requires that no live patched function
> > >>> must be found on any task's stack during a transition process after a
> > >>> live patch is applied. It is achieved by walking through stacks of all
> > >>> blocked tasks.
> > >>>
> > >>> The user might also want to define more functions to search for without
> > >>> them being patched at all. It may either help with preparing a live
> > >>> patch, which would otherwise require additional touches to achieve the
> > >>> consistency
> > >>
> > >> Do we have any examples of this situation we can add to the commit log?
> > > 
> > > I do not have anything at hand. Joe, do you remember the case you 
> > > mentioned previously about adding a nop to a function?
> > 
> > Maybe adding a hypothetical scenario to the commit log would suffice?
> 
> I wonder if we could describe a scenario based on the thread about
> .cold code variants, see
> https://lore.kernel.org/all/20211112015003.pefl656m3zmir6ov@treble/
> 
> This feature would allow to safely livepatch already released
> kernels where the unwinder is not able to reliably detect
> a newly discovered problems.

and is described (well, without actually using .cold suffix) a few lines 
below. I'll try to improve the changelog.
 
> > >>> or it can be used to overcome deficiencies the stack
> > >>> checking inherently has. For example, GCC may optimize a function so
> > >>> that a part of it is moved to a different section and the function would
> > >>> jump to it. This child function would not be found on a stack in this
> > >>> case, but it may be important to search for it so that, again, the
> > >>> consistency is achieved.
> > >>>
> > >>> Allow the user to specify such functions on klp_object level.
> > >>>
> > >>> Signed-off-by: Miroslav Benes <mbenes@suse.cz>
> > >>> ---
> > >>>  include/linux/livepatch.h     | 11 +++++++++++
> > >>>  kernel/livepatch/core.c       | 16 ++++++++++++++++
> > >>>  kernel/livepatch/transition.c | 21 ++++++++++++++++-----
> > >>>  3 files changed, 43 insertions(+), 5 deletions(-)
> > >>>
> > >>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> > >>> index 2614247a9781..89df578af8c3 100644
> > >>> --- a/include/linux/livepatch.h
> > >>> +++ b/include/linux/livepatch.h
> > >>> @@ -106,9 +106,11 @@ struct klp_callbacks {
> > >>>   * struct klp_object - kernel object structure for live patching
> > >>>   * @name:	module name (or NULL for vmlinux)
> > >>>   * @funcs:	function entries for functions to be patched in the object
> > >>> + * @funcs_stack:	function entries for functions to be stack checked
> > >>
> > >> So there are two arrays/lists of 'klp_func', and two implied meanings of
> > >> what a 'klp_func' is and how it's initialized.
> > >>
> > >> Might it be simpler and more explicit to just add a new external field
> > >> to 'klp_func' and continue to have a single 'funcs' array?  Similar to
> > >> what we already do with the special-casing of 'nop', except it would be
> > >> an external field, e.g. 'no_patch' or 'stack_only'.
> > 
> > I'll add that the first thing that came to mind when you raised this
> > feature idea in the other thread was to support existing klp_funcs array
> > with NULL new_func's.
> 
> Please, solve this with the extra flag, e.g. .stack_only, as
> already suggested. It will help to distinguish mistakes and
> intentions. Also it will allow to find these symbols by grep.

Indeed, that is what I did for v2. I used new_func being NULL fact even in 
v1, but I do not like it much. Extra flag is definitely more robust.
 
> > I didn't go look to see how invasive it would be,
> > but it will be interesting to see if a single list approach turns out
> > any simpler for v2.
> 
> I am not sure either. But I expect that it will be easier than
> the extra array.

So, extra flag and one array makes certain things (initialization) 
definitely easier. On the other hand, there are suddenly more problems to 
think about (and I haven't solved them yet):

  - I need to make it work with nops functions. Especially if we allow a 
    scenario where there could be klp_object instance with just stack_only 
    functions. Would that be useful? For example, to patch something in a 
    module and add a stack_only for a function in vmlinux.
 
    If yes, then the interaction with nops is not completely 
    straightforward and also some parts of the code would have to be 
    changed (for example how obj->patched flag is handled).

  - klp_func instances are directly mirrored in sysfs. Do we want to keep 
    stack_only functions there too? If not, it makes the whole thing 
    slighly more difficult given how we manage kobjects.

Nothing really difficult to implement if we come up with answers.

Miroslav

Re: [PATCH 2/3] livepatch: Allow user to specify functions to search for on a stack

[Petr Mladek]

On Thu 2021-11-25 11:20:04, Miroslav Benes wrote:
> On Thu, 25 Nov 2021, Petr Mladek wrote:
> > On Mon 2021-11-22 10:53:21, Joe Lawrence wrote:
> > > On 11/22/21 2:57 AM, Miroslav Benes wrote:
> > > > On Fri, 19 Nov 2021, Josh Poimboeuf wrote:
> > > >>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> > > >>> index 2614247a9781..89df578af8c3 100644
> > > >>> --- a/include/linux/livepatch.h
> > > >>> +++ b/include/linux/livepatch.h
> > > >>> @@ -106,9 +106,11 @@ struct klp_callbacks {
> > > >>>   * struct klp_object - kernel object structure for live patching
> > > >>>   * @name:	module name (or NULL for vmlinux)
> > > >>>   * @funcs:	function entries for functions to be patched in the object
> > > >>> + * @funcs_stack:	function entries for functions to be stack checked
> > > >>
> > > >> So there are two arrays/lists of 'klp_func', and two implied meanings of
> > > >> what a 'klp_func' is and how it's initialized.
> > > >>
> > > >> Might it be simpler and more explicit to just add a new external field
> > > >> to 'klp_func' and continue to have a single 'funcs' array?  Similar to
> > > >> what we already do with the special-casing of 'nop', except it would be
> > > >> an external field, e.g. 'no_patch' or 'stack_only'.
> > > 
> > > I'll add that the first thing that came to mind when you raised this
> > > feature idea in the other thread was to support existing klp_funcs array
> > > with NULL new_func's.
> > 
> > Please, solve this with the extra flag, e.g. .stack_only, as
> > already suggested. It will help to distinguish mistakes and
> > intentions. Also it will allow to find these symbols by grep.
> 
> Indeed, that is what I did for v2. I used new_func being NULL fact even in 
> v1, but I do not like it much. Extra flag is definitely more robust.
>  
> > > I didn't go look to see how invasive it would be,
> > > but it will be interesting to see if a single list approach turns out
> > > any simpler for v2.
> > 
> > I am not sure either. But I expect that it will be easier than
> > the extra array.
> 
> So, extra flag and one array makes certain things (initialization) 
> definitely easier. On the other hand, there are suddenly more problems to 
> think about (and I haven't solved them yet):
> 
>   - I need to make it work with nops functions. Especially if we allow a 
>     scenario where there could be klp_object instance with just stack_only 
>     functions. Would that be useful? For example, to patch something in a 
>     module and add a stack_only for a function in vmlinux.

My naive expectation is that the struct klp_func with @stack_only
flag might be handled the same way on most locations, except when:

   + func->new_func is handled
   + ftrace handler is added/removed

Something like:

--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -724,7 +724,7 @@ static int klp_init_func(struct klp_object *obj, struct klp_func *func)
 	 * NOPs get the address later. The patched module must be loaded,
 	 * see klp_init_object_loaded().
 	 */
-	if (!func->new_func && !func->nop)
+	if (!func->new_func && !func->nop && !func->stack_only)
 		return -EINVAL;
 
 	if (strlen(func->old_name) >= KSYM_NAME_LEN)
@@ -801,6 +801,9 @@ static int klp_init_object_loaded(struct klp_patch *patch,
 			return -ENOENT;
 		}
 
+		if (func->stack_only)
+			continue;
+
 		if (func->nop)
 			func->new_func = func->old_func;
 
--- a/kernel/livepatch/patch.c
+++ b/kernel/livepatch/patch.c
@@ -247,6 +247,9 @@ static void __klp_unpatch_object(struct klp_object *obj, bool nops_only)
 	struct klp_func *func;
 
 	klp_for_each_func(obj, func) {
+		if (func->stack_only)
+			continue;
+
 		if (nops_only && !func->nop)
 			continue;
 
@@ -273,6 +276,8 @@ int klp_patch_object(struct klp_object *obj)
 		return -EINVAL;
 
 	klp_for_each_func(obj, func) {
+		if (func->stack_only)
+			continue;
 		ret = klp_patch_func(func);
 		if (ret) {
 			klp_unpatch_object(obj);


>     If yes, then the interaction with nops is not completely 
>     straightforward and also some parts of the code would have to be 
>     changed (for example how obj->patched flag is handled).

I would keep func->patched disabled for @stack_only entries.

But they should not affect obj->patched state. I mean that
obj->patched should be set when the patch is enabled even when
there are only "stack_only" funcs.

It might look a bit unclear. A solution might be to rename the flags:

   + func->patched    ->   func->active   (and set even for @stack_only)
   + obj->patched     ->   obj->active    (same as func)

But I am not sure if it is worth it.


>   - klp_func instances are directly mirrored in sysfs. Do we want to keep 
>     stack_only functions there too? If not, it makes the whole thing 
>     slighly more difficult given how we manage kobjects.

I would keep them in sysfs. It will be easier and it does not harm.

> Nothing really difficult to implement if we come up with answers.

I am sorry for not answering this earlier. I have missed the questions :-(

Best Regards,
Petr

[PATCH 3/3] selftests/livepatch: Test of the API for specifying functions to search for on a stack

[Miroslav Benes]

Add a test for the API which allows the user to specify functions which
are then searched for on any tasks's stack during a transition process.

Signed-off-by: Miroslav Benes <mbenes@suse.cz>
---
 lib/Kconfig.debug                             |  1 +
 lib/livepatch/Makefile                        |  4 +-
 lib/livepatch/test_klp_funcstack_demo.c       | 61 +++++++++++++
 lib/livepatch/test_klp_funcstack_mod.c        | 72 +++++++++++++++
 tools/testing/selftests/livepatch/Makefile    |  3 +-
 .../selftests/livepatch/test-func-stack.sh    | 88 +++++++++++++++++++
 6 files changed, 227 insertions(+), 2 deletions(-)
 create mode 100644 lib/livepatch/test_klp_funcstack_demo.c
 create mode 100644 lib/livepatch/test_klp_funcstack_mod.c
 create mode 100755 tools/testing/selftests/livepatch/test-func-stack.sh

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 9ef7ce18b4f5..aa4c97098f41 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -2529,6 +2529,7 @@ config TEST_LIVEPATCH
 	default n
 	depends on DYNAMIC_DEBUG
 	depends on LIVEPATCH
+	depends on DEBUG_FS
 	depends on m
 	help
 	  Test kernel livepatching features for correctness.  The tests will
diff --git a/lib/livepatch/Makefile b/lib/livepatch/Makefile
index dcc912b3478f..584e3b8b5415 100644
--- a/lib/livepatch/Makefile
+++ b/lib/livepatch/Makefile
@@ -11,4 +11,6 @@ obj-$(CONFIG_TEST_LIVEPATCH) += test_klp_atomic_replace.o \
 				test_klp_shadow_vars.o \
 				test_klp_state.o \
 				test_klp_state2.o \
-				test_klp_state3.o
+				test_klp_state3.o \
+				test_klp_funcstack_mod.o \
+				test_klp_funcstack_demo.o
diff --git a/lib/livepatch/test_klp_funcstack_demo.c b/lib/livepatch/test_klp_funcstack_demo.c
new file mode 100644
index 000000000000..902798077f05
--- /dev/null
+++ b/lib/livepatch/test_klp_funcstack_demo.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2021 Miroslav Benes <mbenes@suse.cz>
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/livepatch.h>
+
+static int funcstack;
+module_param(funcstack, int, 0644);
+MODULE_PARM_DESC(funcstack, "func_stack (default=0)");
+
+static noinline void livepatch_child2_function(void)
+{
+	pr_info("%s\n", __func__);
+}
+
+static struct klp_func funcs[] = {
+	{
+		.old_name = "child2_function",
+		.new_func = livepatch_child2_function,
+	}, {}
+};
+
+static struct klp_func funcs_stack[] = {
+	{
+		.old_name = "parent_function",
+	}, {}
+};
+
+static struct klp_object objs[] = {
+	{
+		.name = "test_klp_funcstack_mod",
+		.funcs = funcs,
+	}, {}
+};
+
+static struct klp_patch patch = {
+	.mod = THIS_MODULE,
+	.objs = objs,
+};
+
+static int test_klp_funcstack_demo_init(void)
+{
+	if (funcstack)
+		objs[0].funcs_stack = funcs_stack;
+
+	return klp_enable_patch(&patch);
+}
+
+static void test_klp_funcstack_demo_exit(void)
+{
+}
+
+module_init(test_klp_funcstack_demo_init);
+module_exit(test_klp_funcstack_demo_exit);
+MODULE_LICENSE("GPL");
+MODULE_INFO(livepatch, "Y");
+MODULE_AUTHOR("Miroslav Benes <mbenes@suse.cz>");
+MODULE_DESCRIPTION("Livepatch test: func_stack demo");
diff --git a/lib/livepatch/test_klp_funcstack_mod.c b/lib/livepatch/test_klp_funcstack_mod.c
new file mode 100644
index 000000000000..127c6093d890
--- /dev/null
+++ b/lib/livepatch/test_klp_funcstack_mod.c
@@ -0,0 +1,72 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2021 Miroslav Benes <mbenes@suse.cz>
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/debugfs.h>
+#include <linux/delay.h>
+
+static int sleep_length = 10000;
+module_param(sleep_length, int, 0644);
+MODULE_PARM_DESC(sleep_length, "length of sleep in seconds (default=10)");
+
+static noinline void child_function(void)
+{
+	pr_info("%s enter\n", __func__);
+	msleep(sleep_length);
+	pr_info("%s exit\n", __func__);
+}
+
+static noinline void child2_function(void)
+{
+	pr_info("%s\n", __func__);
+}
+
+static noinline void parent_function(void)
+{
+	pr_info("%s enter\n", __func__);
+	child_function();
+	child2_function();
+	pr_info("%s exit\n", __func__);
+}
+
+static int parent_function_get(void *data, u64 *val)
+{
+	*val = 0;
+	parent_function();
+
+	return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(fops_parent_function, parent_function_get, NULL, "%llu\n");
+
+static struct dentry *debugfs_dir;
+
+static int test_klp_funcstack_mod_init(void)
+{
+	struct dentry *d;
+
+	debugfs_dir = debugfs_create_dir("test_klp_funcstack", NULL);
+	if (IS_ERR(debugfs_dir))
+		return PTR_ERR(debugfs_dir);
+
+	d = debugfs_create_file("parent_function", 0400, debugfs_dir, NULL,
+				&fops_parent_function);
+	if (IS_ERR(d))
+		debugfs_remove_recursive(debugfs_dir);
+
+	return 0;
+}
+
+static void test_klp_funcstack_mod_exit(void)
+{
+	debugfs_remove_recursive(debugfs_dir);
+}
+
+module_init(test_klp_funcstack_mod_init);
+module_exit(test_klp_funcstack_mod_exit);
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Miroslav Benes <mbenes@suse.cz>");
+MODULE_DESCRIPTION("Livepatch test: func_stack module");
diff --git a/tools/testing/selftests/livepatch/Makefile b/tools/testing/selftests/livepatch/Makefile
index 1acc9e1fa3fb..40f8a3a2e9aa 100644
--- a/tools/testing/selftests/livepatch/Makefile
+++ b/tools/testing/selftests/livepatch/Makefile
@@ -6,7 +6,8 @@ TEST_PROGS := \
 	test-callbacks.sh \
 	test-shadow-vars.sh \
 	test-state.sh \
-	test-ftrace.sh
+	test-ftrace.sh \
+	test-func-stack.sh
 
 TEST_FILES := settings
 
diff --git a/tools/testing/selftests/livepatch/test-func-stack.sh b/tools/testing/selftests/livepatch/test-func-stack.sh
new file mode 100755
index 000000000000..b7da62c9f5a1
--- /dev/null
+++ b/tools/testing/selftests/livepatch/test-func-stack.sh
@@ -0,0 +1,88 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021 Miroslav Benes <mbenes@suse.cz>
+
+. $(dirname $0)/functions.sh
+
+MOD_TARGET=test_klp_funcstack_mod
+MOD_LIVEPATCH=test_klp_funcstack_demo
+
+setup_config
+
+# - load a target module and call its parent_function(). It will sleep in its
+#   child_function() callee.
+# - load a live patch with new child2_function() called from parent_function()
+#   too. The patching does not wait for child_function() to return, because
+#   child2_function() is not on any stack.
+# - clean up afterwards
+
+start_test "non-blocking patching without the function on a stack"
+
+load_mod $MOD_TARGET
+
+(cat /sys/kernel/debug/test_klp_funcstack/parent_function) >/dev/null &
+PID=$!
+
+load_lp $MOD_LIVEPATCH
+
+wait $PID
+
+disable_lp $MOD_LIVEPATCH
+unload_lp $MOD_LIVEPATCH
+unload_mod $MOD_TARGET
+
+check_result "% modprobe $MOD_TARGET
+$MOD_TARGET: parent_function enter
+$MOD_TARGET: child_function enter
+% modprobe $MOD_LIVEPATCH
+livepatch: enabling patch '$MOD_LIVEPATCH'
+livepatch: '$MOD_LIVEPATCH': initializing patching transition
+livepatch: '$MOD_LIVEPATCH': starting patching transition
+livepatch: '$MOD_LIVEPATCH': completing patching transition
+livepatch: '$MOD_LIVEPATCH': patching complete
+$MOD_TARGET: child_function exit
+$MOD_LIVEPATCH: livepatch_child2_function
+$MOD_TARGET: parent_function exit
+% echo 0 > /sys/kernel/livepatch/$MOD_LIVEPATCH/enabled
+livepatch: '$MOD_LIVEPATCH': initializing unpatching transition
+livepatch: '$MOD_LIVEPATCH': starting unpatching transition
+livepatch: '$MOD_LIVEPATCH': completing unpatching transition
+livepatch: '$MOD_LIVEPATCH': unpatching complete
+% rmmod $MOD_LIVEPATCH
+% rmmod $MOD_TARGET"
+
+# Similar to the previous test but now the patching has to wait for
+# child2_function() to return, because parent_function() is also checked for.
+
+start_test "patching delayed due to the function on a stack"
+
+load_mod $MOD_TARGET
+
+(cat /sys/kernel/debug/test_klp_funcstack/parent_function) >/dev/null &
+
+load_lp $MOD_LIVEPATCH funcstack=1
+disable_lp $MOD_LIVEPATCH
+unload_lp $MOD_LIVEPATCH
+unload_mod $MOD_TARGET
+
+check_result "% modprobe $MOD_TARGET
+$MOD_TARGET: parent_function enter
+$MOD_TARGET: child_function enter
+% modprobe $MOD_LIVEPATCH funcstack=1
+livepatch: enabling patch '$MOD_LIVEPATCH'
+livepatch: '$MOD_LIVEPATCH': initializing patching transition
+livepatch: '$MOD_LIVEPATCH': starting patching transition
+$MOD_TARGET: child_function exit
+$MOD_TARGET: child2_function
+$MOD_TARGET: parent_function exit
+livepatch: '$MOD_LIVEPATCH': completing patching transition
+livepatch: '$MOD_LIVEPATCH': patching complete
+% echo 0 > /sys/kernel/livepatch/$MOD_LIVEPATCH/enabled
+livepatch: '$MOD_LIVEPATCH': initializing unpatching transition
+livepatch: '$MOD_LIVEPATCH': starting unpatching transition
+livepatch: '$MOD_LIVEPATCH': completing unpatching transition
+livepatch: '$MOD_LIVEPATCH': unpatching complete
+% rmmod $MOD_LIVEPATCH
+% rmmod $MOD_TARGET"
+
+exit 0
-- 
2.33.1

Re: [PATCH 3/3] selftests/livepatch: Test of the API for specifying functions to search for on a stack

[Petr Mladek]

On Fri 2021-11-19 10:03:27, Miroslav Benes wrote:
> Add a test for the API which allows the user to specify functions which
> are then searched for on any tasks's stack during a transition process.
> 
> --- /dev/null
> +++ b/lib/livepatch/test_klp_funcstack_mod.c
> @@ -0,0 +1,72 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (C) 2021 Miroslav Benes <mbenes@suse.cz>
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/module.h>
> +#include <linux/kernel.h>
> +#include <linux/debugfs.h>
> +#include <linux/delay.h>
> +
> +static int sleep_length = 10000;
> +module_param(sleep_length, int, 0644);
> +MODULE_PARM_DESC(sleep_length, "length of sleep in seconds (default=10)");
> +
> +static noinline void child_function(void)
> +{
> +	pr_info("%s enter\n", __func__);
> +	msleep(sleep_length);

The hardcoded sleep is not ideal. It might be too low or non-necessary high.

If I get it correctly, we are trying to achieve here the same as
busymod_work_func() in test_klp_callbacks_busy.c.

The approach with debugfs is an interesting trick. Though, I slightly
prefer using the scheduled work. The workqueue API looks less tricky
to me than sysfs/debugfs API. Also it does not block the module
in the init() callback[*]. But I might be biased.

Anyway, it might make sense to use the same trick in both situations.
It would make it easier to maintain the test modules.

[*] There is actually a race in the workqueue approach. The module
init() callback should wait until the work is really scheduled
and sleeping. It might be achieved by similar hand-shake like
with @block_transition variable. Or completion API might be
even more elegant.


> +	pr_info("%s exit\n", __func__);
> +}
> +
> +static noinline void child2_function(void)
> +{
> +	pr_info("%s\n", __func__);
> +}
> +
> +static noinline void parent_function(void)
> +{
> +	pr_info("%s enter\n", __func__);
> +	child_function();
> +	child2_function();

This would deserve some explanation what we try to simulate here
and how it is achieved. It is not easy for me even with the background
that I have freshly in my mind.

Also I think about more descriptive names ;-)

What about something like this (using workqueue work and completion):

/*
 * Simulate part of the caller code that is in another .elf section
 * and is reached via jump. It this was really the case then the stack
 * unwinder might not be able to detect that the process is sleeping
 * in the caller.
 */
static void simulate_jump_part(void)
{
	pr_info("%s enter\n", __func__);

	/* Stay in the jump part unless told to leave. */
	wait_for_completion(finish_jump);

	pr_info("%s exit\n", __func__);
}

/*
 * Simulate modified part of the caller code. It should never get
 * livepatched when the caller is sleeping in the just_part().
 */
static void simulate_modified_part(void)
{
	pr_info("%s\n", __func__);
}

static void test_not_on_stack_func_work(struct work_struct *work)
{
	pr_info("%s enter\n", __func__);

	/* Simulation ready */
	complete(work_started);

	simulate_jump_part();
	simulate_modified_part();

	pr_info("%s exit\n", __func__);
}

static int test_klp_no_on_stack_init(void)
{
	pr_info("%s\n", __func__);

	schedule_work(&work);
	wait_for_completion(&work_started);

	return 0;
}

static void test_not_on_stack_exit(void)
{
	complete(&finish_jump);
	flush_work(&work);
	pr_info("%s\n", __func__);
}

module_init(test_klp_not_on_stack_init);
module_exit(test_klp_not_on_stack_exit);

> +	pr_info("%s exit\n", __func__);
> +}
> +

Best Regards,
Petr

Re: [PATCH 3/3] selftests/livepatch: Test of the API for specifying functions to search for on a stack

[Miroslav Benes]

On Thu, 25 Nov 2021, Petr Mladek wrote:

> On Fri 2021-11-19 10:03:27, Miroslav Benes wrote:
> > Add a test for the API which allows the user to specify functions which
> > are then searched for on any tasks's stack during a transition process.
> > 
> > --- /dev/null
> > +++ b/lib/livepatch/test_klp_funcstack_mod.c
> > @@ -0,0 +1,72 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +// Copyright (C) 2021 Miroslav Benes <mbenes@suse.cz>
> > +
> > +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> > +
> > +#include <linux/module.h>
> > +#include <linux/kernel.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/delay.h>
> > +
> > +static int sleep_length = 10000;
> > +module_param(sleep_length, int, 0644);
> > +MODULE_PARM_DESC(sleep_length, "length of sleep in seconds (default=10)");
> > +
> > +static noinline void child_function(void)
> > +{
> > +	pr_info("%s enter\n", __func__);
> > +	msleep(sleep_length);
> 
> The hardcoded sleep is not ideal. It might be too low or non-necessary high.

It is not.
 
> If I get it correctly, we are trying to achieve here the same as
> busymod_work_func() in test_klp_callbacks_busy.c.

Yes.

> The approach with debugfs is an interesting trick. Though, I slightly
> prefer using the scheduled work. The workqueue API looks less tricky
> to me than sysfs/debugfs API. Also it does not block the module
> in the init() callback[*]. But I might be biased.

It seemed to me that debugfs gave us more control over the process than 
workqueues, but I do not really care. Could you explain the blocking in 
the init callback? I do not follow.

> Anyway, it might make sense to use the same trick in both situations.
> It would make it easier to maintain the test modules.

True. So I will rewrite it to workqueues as you are proposing below.

> [*] There is actually a race in the workqueue approach. The module
> init() callback should wait until the work is really scheduled
> and sleeping. It might be achieved by similar hand-shake like
> with @block_transition variable. Or completion API might be
> even more elegant.
> 
> 
> > +	pr_info("%s exit\n", __func__);
> > +}
> > +
> > +static noinline void child2_function(void)
> > +{
> > +	pr_info("%s\n", __func__);
> > +}
> > +
> > +static noinline void parent_function(void)
> > +{
> > +	pr_info("%s enter\n", __func__);
> > +	child_function();
> > +	child2_function();
> 
> This would deserve some explanation what we try to simulate here
> and how it is achieved. It is not easy for me even with the background
> that I have freshly in my mind.
> 
> Also I think about more descriptive names ;-)

Hey, I thought it was self-explaining :). So, yes, I started with the 
example given in the .fixup thread, but it is not really tied to .cold 
section, jumps or whatever. The setup is just used to test a new API. 
Moreover, the .fixup example is just a one scenario the new API tries to 
solve.

What you propose below, that is function names and comments, is a bit 
confusing for me. Especially if I did not know anything about the original 
issue (which will be the case in a couple of weeks when I forget 
everything).

So I think it I will stick to brevity unless you or someone else really 
insist.

I can improve tests description in 
tools/testing/selftests/livepatch/test-func-stack.sh if it helps anything.

Miroslav

> What about something like this (using workqueue work and completion):
> 
> /*
>  * Simulate part of the caller code that is in another .elf section
>  * and is reached via jump. It this was really the case then the stack
>  * unwinder might not be able to detect that the process is sleeping
>  * in the caller.
>  */
> static void simulate_jump_part(void)
> {
> 	pr_info("%s enter\n", __func__);
> 
> 	/* Stay in the jump part unless told to leave. */
> 	wait_for_completion(finish_jump);
> 
> 	pr_info("%s exit\n", __func__);
> }
> 
> /*
>  * Simulate modified part of the caller code. It should never get
>  * livepatched when the caller is sleeping in the just_part().
>  */
> static void simulate_modified_part(void)
> {
> 	pr_info("%s\n", __func__);
> }
> 
> static void test_not_on_stack_func_work(struct work_struct *work)
> {
> 	pr_info("%s enter\n", __func__);
> 
> 	/* Simulation ready */
> 	complete(work_started);
> 
> 	simulate_jump_part();
> 	simulate_modified_part();
> 
> 	pr_info("%s exit\n", __func__);
> }
> 
> static int test_klp_no_on_stack_init(void)
> {
> 	pr_info("%s\n", __func__);
> 
> 	schedule_work(&work);
> 	wait_for_completion(&work_started);
> 
> 	return 0;
> }
> 
> static void test_not_on_stack_exit(void)
> {
> 	complete(&finish_jump);
> 	flush_work(&work);
> 	pr_info("%s\n", __func__);
> }
> 
> module_init(test_klp_not_on_stack_init);
> module_exit(test_klp_not_on_stack_exit);
> 
> > +	pr_info("%s exit\n", __func__);
> > +}
> > +

Re: [PATCH 3/3] selftests/livepatch: Test of the API for specifying functions to search for on a stack

[Petr Mladek]

On Fri 2021-11-26 10:20:54, Miroslav Benes wrote:
> On Thu, 25 Nov 2021, Petr Mladek wrote:
> 
> > On Fri 2021-11-19 10:03:27, Miroslav Benes wrote:
> > > Add a test for the API which allows the user to specify functions which
> > > are then searched for on any tasks's stack during a transition process.
> > > 
> > The approach with debugfs is an interesting trick. Though, I slightly
> > prefer using the scheduled work. The workqueue API looks less tricky
> > to me than sysfs/debugfs API. Also it does not block the module
> > in the init() callback[*]. But I might be biased.
> 
> It seemed to me that debugfs gave us more control over the process than 
> workqueues, but I do not really care. Could you explain the blocking in 
> the init callback? I do not follow.

Good question about the blocking! Please, forget it. I am not sure
why I thought that the module might get blocked in the module_init()
script.


> > Anyway, it might make sense to use the same trick in both situations.
> > It would make it easier to maintain the test modules.
> 
> True. So I will rewrite it to workqueues as you are proposing below.
> 
> > [*] There is actually a race in the workqueue approach. The module
> > init() callback should wait until the work is really scheduled
> > and sleeping. It might be achieved by similar hand-shake like
> > with @block_transition variable. Or completion API might be
> > even more elegant.
> > 
> > 
> > > +	pr_info("%s exit\n", __func__);
> > > +}
> > > +
> > > +static noinline void child2_function(void)
> > > +{
> > > +	pr_info("%s\n", __func__);
> > > +}
> > > +
> > > +static noinline void parent_function(void)
> > > +{
> > > +	pr_info("%s enter\n", __func__);
> > > +	child_function();
> > > +	child2_function();
> > 
> > This would deserve some explanation what we try to simulate here
> > and how it is achieved. It is not easy for me even with the background
> > that I have freshly in my mind.
> > 
> > Also I think about more descriptive names ;-)
> 
> Hey, I thought it was self-explaining :). So, yes, I started with the 
> example given in the .fixup thread, but it is not really tied to .cold 
> section, jumps or whatever. The setup is just used to test a new API. 
> Moreover, the .fixup example is just a one scenario the new API tries to 
> solve.

OK, single child() function should be enough for testing the behavior.
We might sleep/wait in the parent().

I think that I was confused by the two child() functions. It looked
like sleeping in a child function was important. And the "same"
name of the two children did not help much to understand and
distinguish the purpose.

> What you propose below, that is function names and comments, is a bit 
> confusing for me. Especially if I did not know anything about the original 
> issue (which will be the case in a couple of weeks when I forget 
> everything).
> 
> So I think it I will stick to brevity unless you or someone else really 
> insist.

No, I do not resist on the complicated exmaple. When thinking about
it, the easier test case the better. It should be enough to describe
the real-life purpose of the API in the patch that introduces the API.

> I can improve tests description in 
> tools/testing/selftests/livepatch/test-func-stack.sh if it helps anything.

Yes, please. I miss some top-level descriptions of the tested
functionality, something like:

# Tests for "bla bla" feature.
# It allows to block transition of a process when it is running
# parent() function and only the child() function is livepatched.

# This test does not use the feature. The transition finishes even
# before parent() exits.

# In this test case, the livepatch is instructed to check also
# parent() on stack. The transition must not finish before
# parent() exists.

It would be nice to have these high-level descriptions even in
the test modules.

Best Regards,
Petr