/proc/kallsyms shows undefined symbols for livepatch modules

/proc/kallsyms shows undefined symbols for livepatch modules

[Josh Poimboeuf]

Hi Jessica,

I found a bug:

  [root@f25 ~]# modprobe livepatch-sample
  [root@f25 ~]# grep ' u ' /proc/kallsyms
  ffffffff81161080 u klp_enable_patch	[livepatch_sample]
  ffffffff81a01800 u __fentry__	[livepatch_sample]
  ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
  ffffffff81161870 u klp_register_patch	[livepatch_sample]
  ffffffff8131f0b0 u seq_printf	[livepatch_sample]

Notice that livepatch modules' undefined symbols are showing up in
/proc/kallsyms.  This can confuse klp_find_object_symbol() which can
cause subtle bugs in livepatch.

I stared at the module kallsyms code for a bit, but I don't see the bug.
Maybe it has something to do with how we save the symbol table in
copy_module_elf().  Any ideas?

-- 
Josh

Re: /proc/kallsyms shows undefined symbols for livepatch modules

[Jessica Yu]

+++ Josh Poimboeuf [02/06/18 12:32 -0500]:
>Hi Jessica,
>
>I found a bug:
>
>  [root@f25 ~]# modprobe livepatch-sample
>  [root@f25 ~]# grep ' u ' /proc/kallsyms
>  ffffffff81161080 u klp_enable_patch	[livepatch_sample]
>  ffffffff81a01800 u __fentry__	[livepatch_sample]
>  ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
>  ffffffff81161870 u klp_register_patch	[livepatch_sample]
>  ffffffff8131f0b0 u seq_printf	[livepatch_sample]
>
>Notice that livepatch modules' undefined symbols are showing up in
>/proc/kallsyms.  This can confuse klp_find_object_symbol() which can
>cause subtle bugs in livepatch.
>
>I stared at the module kallsyms code for a bit, but I don't see the bug.
>Maybe it has something to do with how we save the symbol table in
>copy_module_elf().  Any ideas?

Hi Josh!

This is because we preserve the entire symbol table for livepatch
modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
still apply relocations properly with apply_relocate_add() after a
to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
symbols for modules so they do not appear in /proc/kallsyms.

I think I see how this can cause bugs, since if we're searching for
seq_printf we want the real seq_printf and not the undef symbol from
the livepatch module. But I thought we renamed these symbols with some
.klp prefix? Or is the livepatch bug you're running into something else?

Thanks,

Jessica

Re: /proc/kallsyms shows undefined symbols for livepatch modules

[Jessica Yu]

+++ Jessica Yu [04/06/18 10:05 +0200]:
>+++ Josh Poimboeuf [02/06/18 12:32 -0500]:
>>Hi Jessica,
>>
>>I found a bug:
>>
>> [root@f25 ~]# modprobe livepatch-sample
>> [root@f25 ~]# grep ' u ' /proc/kallsyms
>> ffffffff81161080 u klp_enable_patch	[livepatch_sample]
>> ffffffff81a01800 u __fentry__	[livepatch_sample]
>> ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
>> ffffffff81161870 u klp_register_patch	[livepatch_sample]
>> ffffffff8131f0b0 u seq_printf	[livepatch_sample]
>>
>>Notice that livepatch modules' undefined symbols are showing up in
>>/proc/kallsyms.  This can confuse klp_find_object_symbol() which can
>>cause subtle bugs in livepatch.
>>
>>I stared at the module kallsyms code for a bit, but I don't see the bug.
>>Maybe it has something to do with how we save the symbol table in
>>copy_module_elf().  Any ideas?
>
>Hi Josh!
>
>This is because we preserve the entire symbol table for livepatch
>modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
>still apply relocations properly with apply_relocate_add() after a
>to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
>symbols for modules so they do not appear in /proc/kallsyms.

Hm, if having the full symtab in kallsyms is causing trouble, one
possibility would be to just have the module kallsyms code simply
skip/ignore undef symbols. That's what we technically do for normal
modules anyway (we normally cut undef syms out of the symtab). Haven't
tested this idea but does that sound like it'd help?

Thanks,

Jessica

[PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Jessica Yu]

+++ Jessica Yu [04/06/18 11:54 +0200]:
>+++ Jessica Yu [04/06/18 10:05 +0200]:
>>+++ Josh Poimboeuf [02/06/18 12:32 -0500]:
>>>Hi Jessica,
>>>
>>>I found a bug:
>>>
>>>[root@f25 ~]# modprobe livepatch-sample
>>>[root@f25 ~]# grep ' u ' /proc/kallsyms
>>>ffffffff81161080 u klp_enable_patch	[livepatch_sample]
>>>ffffffff81a01800 u __fentry__	[livepatch_sample]
>>>ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
>>>ffffffff81161870 u klp_register_patch	[livepatch_sample]
>>>ffffffff8131f0b0 u seq_printf	[livepatch_sample]
>>>
>>>Notice that livepatch modules' undefined symbols are showing up in
>>>/proc/kallsyms.  This can confuse klp_find_object_symbol() which can
>>>cause subtle bugs in livepatch.
>>>
>>>I stared at the module kallsyms code for a bit, but I don't see the bug.
>>>Maybe it has something to do with how we save the symbol table in
>>>copy_module_elf().  Any ideas?
>>
>>Hi Josh!
>>
>>This is because we preserve the entire symbol table for livepatch
>>modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
>>still apply relocations properly with apply_relocate_add() after a
>>to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
>>symbols for modules so they do not appear in /proc/kallsyms.
>
>Hm, if having the full symtab in kallsyms is causing trouble, one
>possibility would be to just have the module kallsyms code simply
>skip/ignore undef symbols. That's what we technically do for normal
>modules anyway (we normally cut undef syms out of the symtab). Haven't
>tested this idea but does that sound like it'd help?

See if the following patch (untested) helps. It does not fix the
/proc/kallsyms lookup, that requires a separate patch. But it should
exclude the undef symbols from module_kallsyms_on_each_symbol() and
thus also from klp_find_object_symbol().

>From 9cfd14675206adf55a85e5f5322b36ea89a523e4 Mon Sep 17 00:00:00 2001
From: Jessica Yu <jeyu@kernel.org>
Date: Mon, 4 Jun 2018 14:35:56 +0200
Subject: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

---
 kernel/module.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/kernel/module.c b/kernel/module.c
index c9bea7f2b43e..dfa61490b37d 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -4070,7 +4070,7 @@ static unsigned long mod_find_symname(struct module *mod, const char *name)
 
 	for (i = 0; i < kallsyms->num_symtab; i++)
 		if (strcmp(name, symname(kallsyms, i)) == 0 &&
-		    kallsyms->symtab[i].st_info != 'U')
+		    kallsyms->symtab[i].st_shndx != SHN_UNDEF)
 			return kallsyms->symtab[i].st_value;
 	return 0;
 }
@@ -4116,6 +4116,10 @@ int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *,
 		if (mod->state == MODULE_STATE_UNFORMED)
 			continue;
 		for (i = 0; i < kallsyms->num_symtab; i++) {
+
+			if (kallsyms->symtab[i].st_shndx == SHN_UNDEF)
+				continue;
+
 			ret = fn(data, symname(kallsyms, i),
 				 mod, kallsyms->symtab[i].st_value);
 			if (ret != 0)
-- 
2.12.3

Re: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Josh Poimboeuf]

On Mon, Jun 04, 2018 at 03:01:31PM +0200, Jessica Yu wrote:
> +++ Jessica Yu [04/06/18 11:54 +0200]:
> > +++ Jessica Yu [04/06/18 10:05 +0200]:
> > > +++ Josh Poimboeuf [02/06/18 12:32 -0500]:
> > > > Hi Jessica,
> > > > 
> > > > I found a bug:
> > > > 
> > > > [root@f25 ~]# modprobe livepatch-sample
> > > > [root@f25 ~]# grep ' u ' /proc/kallsyms
> > > > ffffffff81161080 u klp_enable_patch	[livepatch_sample]
> > > > ffffffff81a01800 u __fentry__	[livepatch_sample]
> > > > ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
> > > > ffffffff81161870 u klp_register_patch	[livepatch_sample]
> > > > ffffffff8131f0b0 u seq_printf	[livepatch_sample]
> > > > 
> > > > Notice that livepatch modules' undefined symbols are showing up in
> > > > /proc/kallsyms.  This can confuse klp_find_object_symbol() which can
> > > > cause subtle bugs in livepatch.
> > > > 
> > > > I stared at the module kallsyms code for a bit, but I don't see the bug.
> > > > Maybe it has something to do with how we save the symbol table in
> > > > copy_module_elf().  Any ideas?
> > > 
> > > Hi Josh!
> > > 
> > > This is because we preserve the entire symbol table for livepatch
> > > modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
> > > still apply relocations properly with apply_relocate_add() after a
> > > to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
> > > symbols for modules so they do not appear in /proc/kallsyms.
> > 
> > Hm, if having the full symtab in kallsyms is causing trouble, one
> > possibility would be to just have the module kallsyms code simply
> > skip/ignore undef symbols. That's what we technically do for normal
> > modules anyway (we normally cut undef syms out of the symtab). Haven't
> > tested this idea but does that sound like it'd help?
> 
> See if the following patch (untested) helps. It does not fix the
> /proc/kallsyms lookup, that requires a separate patch. But it should
> exclude the undef symbols from module_kallsyms_on_each_symbol() and
> thus also from klp_find_object_symbol().

That seems like it would work.  But wouldn't it be more robust if we
don't store the SHN_UNDEF symbols to start with?  Really it's only the
SHN_LIVEPATCH symbols that we need to keep, right?

What do you think about the following (untested)?

diff --git a/kernel/module.c b/kernel/module.c
index c9bea7f2b43e..78ec9de856e3 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -2586,6 +2586,9 @@ static bool is_core_symbol(const Elf_Sym *src, const Elf_Shdr *sechdrs,
 {
 	const Elf_Shdr *sec;
 
+	if (src->st_shndx == SHN_LIVEPATCH)
+		return true;
+
 	if (src->st_shndx == SHN_UNDEF
 	    || src->st_shndx >= shnum
 	    || !src->st_name)
@@ -2632,9 +2635,9 @@ static void layout_symtab(struct module *mod, struct load_info *info)
 
 	/* Compute total space required for the core symbols' strtab. */
 	for (ndst = i = 0; i < nsrc; i++) {
-		if (i == 0 || is_livepatch_module(mod) ||
-		    is_core_symbol(src+i, info->sechdrs, info->hdr->e_shnum,
-				   info->index.pcpu)) {
+		if (i == 0 || is_core_symbol(src+i, info->sechdrs,
+					     info->hdr->e_shnum,
+					     info->index.pcpu)) {
 			strtab_size += strlen(&info->strtab[src[i].st_name])+1;
 			ndst++;
 		}
@@ -2691,9 +2694,9 @@ static void add_kallsyms(struct module *mod, const struct load_info *info)
 	mod->core_kallsyms.strtab = s = mod->core_layout.base + info->stroffs;
 	src = mod->kallsyms->symtab;
 	for (ndst = i = 0; i < mod->kallsyms->num_symtab; i++) {
-		if (i == 0 || is_livepatch_module(mod) ||
-		    is_core_symbol(src+i, info->sechdrs, info->hdr->e_shnum,
-				   info->index.pcpu)) {
+		if (i == 0 || is_core_symbol(src+i, info->sechdrs,
+					     info->hdr->e_shnum,
+					     info->index.pcpu)) {
 			dst[ndst] = src[i];
 			dst[ndst++].st_name = s - mod->core_kallsyms.strtab;
 			s += strlcpy(s, &mod->kallsyms->strtab[src[i].st_name],

Re: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Jessica Yu]

+++ Josh Poimboeuf [04/06/18 08:16 -0500]:
>On Mon, Jun 04, 2018 at 03:01:31PM +0200, Jessica Yu wrote:
>> +++ Jessica Yu [04/06/18 11:54 +0200]:
>> > +++ Jessica Yu [04/06/18 10:05 +0200]:
>> > > +++ Josh Poimboeuf [02/06/18 12:32 -0500]:
>> > > > Hi Jessica,
>> > > >
>> > > > I found a bug:
>> > > >
>> > > > [root@f25 ~]# modprobe livepatch-sample
>> > > > [root@f25 ~]# grep ' u ' /proc/kallsyms
>> > > > ffffffff81161080 u klp_enable_patch	[livepatch_sample]
>> > > > ffffffff81a01800 u __fentry__	[livepatch_sample]
>> > > > ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
>> > > > ffffffff81161870 u klp_register_patch	[livepatch_sample]
>> > > > ffffffff8131f0b0 u seq_printf	[livepatch_sample]
>> > > >
>> > > > Notice that livepatch modules' undefined symbols are showing up in
>> > > > /proc/kallsyms.  This can confuse klp_find_object_symbol() which can
>> > > > cause subtle bugs in livepatch.
>> > > >
>> > > > I stared at the module kallsyms code for a bit, but I don't see the bug.
>> > > > Maybe it has something to do with how we save the symbol table in
>> > > > copy_module_elf().  Any ideas?
>> > >
>> > > Hi Josh!
>> > >
>> > > This is because we preserve the entire symbol table for livepatch
>> > > modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
>> > > still apply relocations properly with apply_relocate_add() after a
>> > > to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
>> > > symbols for modules so they do not appear in /proc/kallsyms.
>> >
>> > Hm, if having the full symtab in kallsyms is causing trouble, one
>> > possibility would be to just have the module kallsyms code simply
>> > skip/ignore undef symbols. That's what we technically do for normal
>> > modules anyway (we normally cut undef syms out of the symtab). Haven't
>> > tested this idea but does that sound like it'd help?
>>
>> See if the following patch (untested) helps. It does not fix the
>> /proc/kallsyms lookup, that requires a separate patch. But it should
>> exclude the undef symbols from module_kallsyms_on_each_symbol() and
>> thus also from klp_find_object_symbol().
>
>That seems like it would work.  But wouldn't it be more robust if we
>don't store the SHN_UNDEF symbols to start with?  Really it's only the
>SHN_LIVEPATCH symbols that we need to keep, right?

Hm, didn't we have a reason for preserving the original symbol tables?
It's not that we're storing the undef symbols on purpose, it's just
that in the case of a livepatch module we keep *all* the symbols.
IIRC, the original reasoning was that we needed to keep the original
symtab indices in place, otherwise apply_relocate() (which uses the
original symbol index encoded in the relocation) won't work.
Selectively copying only SHN_LIVEPATCH symbols when building the
module's symbol table will unfortunately mess the symbol indices up :/
Under these restrictions, it just seemed easier to make kallsyms
ignore undef symbols.

>What do you think about the following (untested)?
>
>diff --git a/kernel/module.c b/kernel/module.c
>index c9bea7f2b43e..78ec9de856e3 100644
>--- a/kernel/module.c
>+++ b/kernel/module.c
>@@ -2586,6 +2586,9 @@ static bool is_core_symbol(const Elf_Sym *src, const Elf_Shdr *sechdrs,
> {
> 	const Elf_Shdr *sec;
>
>+	if (src->st_shndx == SHN_LIVEPATCH)
>+		return true;
>+
> 	if (src->st_shndx == SHN_UNDEF
> 	    || src->st_shndx >= shnum
> 	    || !src->st_name)
>@@ -2632,9 +2635,9 @@ static void layout_symtab(struct module *mod, struct load_info *info)
>
> 	/* Compute total space required for the core symbols' strtab. */
> 	for (ndst = i = 0; i < nsrc; i++) {
>-		if (i == 0 || is_livepatch_module(mod) ||
>-		    is_core_symbol(src+i, info->sechdrs, info->hdr->e_shnum,
>-				   info->index.pcpu)) {
>+		if (i == 0 || is_core_symbol(src+i, info->sechdrs,
>+					     info->hdr->e_shnum,
>+					     info->index.pcpu)) {
> 			strtab_size += strlen(&info->strtab[src[i].st_name])+1;
> 			ndst++;
> 		}
>@@ -2691,9 +2694,9 @@ static void add_kallsyms(struct module *mod, const struct load_info *info)
> 	mod->core_kallsyms.strtab = s = mod->core_layout.base + info->stroffs;
> 	src = mod->kallsyms->symtab;
> 	for (ndst = i = 0; i < mod->kallsyms->num_symtab; i++) {
>-		if (i == 0 || is_livepatch_module(mod) ||
>-		    is_core_symbol(src+i, info->sechdrs, info->hdr->e_shnum,
>-				   info->index.pcpu)) {
>+		if (i == 0 || is_core_symbol(src+i, info->sechdrs,
>+					     info->hdr->e_shnum,
>+					     info->index.pcpu)) {
> 			dst[ndst] = src[i];
> 			dst[ndst++].st_name = s - mod->core_kallsyms.strtab;
> 			s += strlcpy(s, &mod->kallsyms->strtab[src[i].st_name],

Re: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Josh Poimboeuf]

On Mon, Jun 04, 2018 at 04:05:11PM +0200, Jessica Yu wrote:
> +++ Josh Poimboeuf [04/06/18 08:16 -0500]:
> > On Mon, Jun 04, 2018 at 03:01:31PM +0200, Jessica Yu wrote:
> > > +++ Jessica Yu [04/06/18 11:54 +0200]:
> > > > +++ Jessica Yu [04/06/18 10:05 +0200]:
> > > > > +++ Josh Poimboeuf [02/06/18 12:32 -0500]:
> > > > > > Hi Jessica,
> > > > > >
> > > > > > I found a bug:
> > > > > >
> > > > > > [root@f25 ~]# modprobe livepatch-sample
> > > > > > [root@f25 ~]# grep ' u ' /proc/kallsyms
> > > > > > ffffffff81161080 u klp_enable_patch	[livepatch_sample]
> > > > > > ffffffff81a01800 u __fentry__	[livepatch_sample]
> > > > > > ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
> > > > > > ffffffff81161870 u klp_register_patch	[livepatch_sample]
> > > > > > ffffffff8131f0b0 u seq_printf	[livepatch_sample]
> > > > > >
> > > > > > Notice that livepatch modules' undefined symbols are showing up in
> > > > > > /proc/kallsyms.  This can confuse klp_find_object_symbol() which can
> > > > > > cause subtle bugs in livepatch.
> > > > > >
> > > > > > I stared at the module kallsyms code for a bit, but I don't see the bug.
> > > > > > Maybe it has something to do with how we save the symbol table in
> > > > > > copy_module_elf().  Any ideas?
> > > > >
> > > > > Hi Josh!
> > > > >
> > > > > This is because we preserve the entire symbol table for livepatch
> > > > > modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
> > > > > still apply relocations properly with apply_relocate_add() after a
> > > > > to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
> > > > > symbols for modules so they do not appear in /proc/kallsyms.
> > > >
> > > > Hm, if having the full symtab in kallsyms is causing trouble, one
> > > > possibility would be to just have the module kallsyms code simply
> > > > skip/ignore undef symbols. That's what we technically do for normal
> > > > modules anyway (we normally cut undef syms out of the symtab). Haven't
> > > > tested this idea but does that sound like it'd help?
> > > 
> > > See if the following patch (untested) helps. It does not fix the
> > > /proc/kallsyms lookup, that requires a separate patch. But it should
> > > exclude the undef symbols from module_kallsyms_on_each_symbol() and
> > > thus also from klp_find_object_symbol().
> > 
> > That seems like it would work.  But wouldn't it be more robust if we
> > don't store the SHN_UNDEF symbols to start with?  Really it's only the
> > SHN_LIVEPATCH symbols that we need to keep, right?
> 
> Hm, didn't we have a reason for preserving the original symbol tables?
> It's not that we're storing the undef symbols on purpose, it's just
> that in the case of a livepatch module we keep *all* the symbols.
> IIRC, the original reasoning was that we needed to keep the original
> symtab indices in place, otherwise apply_relocate() (which uses the
> original symbol index encoded in the relocation) won't work.
> Selectively copying only SHN_LIVEPATCH symbols when building the
> module's symbol table will unfortunately mess the symbol indices up :/
> Under these restrictions, it just seemed easier to make kallsyms
> ignore undef symbols.

Ah, you're right.  I'll try out your patch.

-- 
Josh

Re: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Josh Poimboeuf]

On Mon, Jun 04, 2018 at 09:54:12AM -0500, Josh Poimboeuf wrote:
> On Mon, Jun 04, 2018 at 04:05:11PM +0200, Jessica Yu wrote:
> > +++ Josh Poimboeuf [04/06/18 08:16 -0500]:
> > > On Mon, Jun 04, 2018 at 03:01:31PM +0200, Jessica Yu wrote:
> > > > +++ Jessica Yu [04/06/18 11:54 +0200]:
> > > > > +++ Jessica Yu [04/06/18 10:05 +0200]:
> > > > > > +++ Josh Poimboeuf [02/06/18 12:32 -0500]:
> > > > > > > Hi Jessica,
> > > > > > >
> > > > > > > I found a bug:
> > > > > > >
> > > > > > > [root@f25 ~]# modprobe livepatch-sample
> > > > > > > [root@f25 ~]# grep ' u ' /proc/kallsyms
> > > > > > > ffffffff81161080 u klp_enable_patch	[livepatch_sample]
> > > > > > > ffffffff81a01800 u __fentry__	[livepatch_sample]
> > > > > > > ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
> > > > > > > ffffffff81161870 u klp_register_patch	[livepatch_sample]
> > > > > > > ffffffff8131f0b0 u seq_printf	[livepatch_sample]
> > > > > > >
> > > > > > > Notice that livepatch modules' undefined symbols are showing up in
> > > > > > > /proc/kallsyms.  This can confuse klp_find_object_symbol() which can
> > > > > > > cause subtle bugs in livepatch.
> > > > > > >
> > > > > > > I stared at the module kallsyms code for a bit, but I don't see the bug.
> > > > > > > Maybe it has something to do with how we save the symbol table in
> > > > > > > copy_module_elf().  Any ideas?
> > > > > >
> > > > > > Hi Josh!
> > > > > >
> > > > > > This is because we preserve the entire symbol table for livepatch
> > > > > > modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
> > > > > > still apply relocations properly with apply_relocate_add() after a
> > > > > > to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
> > > > > > symbols for modules so they do not appear in /proc/kallsyms.
> > > > >
> > > > > Hm, if having the full symtab in kallsyms is causing trouble, one
> > > > > possibility would be to just have the module kallsyms code simply
> > > > > skip/ignore undef symbols. That's what we technically do for normal
> > > > > modules anyway (we normally cut undef syms out of the symtab). Haven't
> > > > > tested this idea but does that sound like it'd help?
> > > > 
> > > > See if the following patch (untested) helps. It does not fix the
> > > > /proc/kallsyms lookup, that requires a separate patch. But it should
> > > > exclude the undef symbols from module_kallsyms_on_each_symbol() and
> > > > thus also from klp_find_object_symbol().
> > > 
> > > That seems like it would work.  But wouldn't it be more robust if we
> > > don't store the SHN_UNDEF symbols to start with?  Really it's only the
> > > SHN_LIVEPATCH symbols that we need to keep, right?
> > 
> > Hm, didn't we have a reason for preserving the original symbol tables?
> > It's not that we're storing the undef symbols on purpose, it's just
> > that in the case of a livepatch module we keep *all* the symbols.
> > IIRC, the original reasoning was that we needed to keep the original
> > symtab indices in place, otherwise apply_relocate() (which uses the
> > original symbol index encoded in the relocation) won't work.
> > Selectively copying only SHN_LIVEPATCH symbols when building the
> > module's symbol table will unfortunately mess the symbol indices up :/
> > Under these restrictions, it just seemed easier to make kallsyms
> > ignore undef symbols.
> 
> Ah, you're right.  I'll try out your patch.

I can confirm that your patch fixes the issue.  Thanks!

-- 
Josh

Re: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Josh Poimboeuf]

On Mon, Jun 04, 2018 at 05:56:05PM -0500, Josh Poimboeuf wrote:
> On Mon, Jun 04, 2018 at 09:54:12AM -0500, Josh Poimboeuf wrote:
> > On Mon, Jun 04, 2018 at 04:05:11PM +0200, Jessica Yu wrote:
> > > +++ Josh Poimboeuf [04/06/18 08:16 -0500]:
> > > > On Mon, Jun 04, 2018 at 03:01:31PM +0200, Jessica Yu wrote:
> > > > > +++ Jessica Yu [04/06/18 11:54 +0200]:
> > > > > > +++ Jessica Yu [04/06/18 10:05 +0200]:
> > > > > > > +++ Josh Poimboeuf [02/06/18 12:32 -0500]:
> > > > > > > > Hi Jessica,
> > > > > > > >
> > > > > > > > I found a bug:
> > > > > > > >
> > > > > > > > [root@f25 ~]# modprobe livepatch-sample
> > > > > > > > [root@f25 ~]# grep ' u ' /proc/kallsyms
> > > > > > > > ffffffff81161080 u klp_enable_patch	[livepatch_sample]
> > > > > > > > ffffffff81a01800 u __fentry__	[livepatch_sample]
> > > > > > > > ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
> > > > > > > > ffffffff81161870 u klp_register_patch	[livepatch_sample]
> > > > > > > > ffffffff8131f0b0 u seq_printf	[livepatch_sample]
> > > > > > > >
> > > > > > > > Notice that livepatch modules' undefined symbols are showing up in
> > > > > > > > /proc/kallsyms.  This can confuse klp_find_object_symbol() which can
> > > > > > > > cause subtle bugs in livepatch.
> > > > > > > >
> > > > > > > > I stared at the module kallsyms code for a bit, but I don't see the bug.
> > > > > > > > Maybe it has something to do with how we save the symbol table in
> > > > > > > > copy_module_elf().  Any ideas?
> > > > > > >
> > > > > > > Hi Josh!
> > > > > > >
> > > > > > > This is because we preserve the entire symbol table for livepatch
> > > > > > > modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
> > > > > > > still apply relocations properly with apply_relocate_add() after a
> > > > > > > to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
> > > > > > > symbols for modules so they do not appear in /proc/kallsyms.
> > > > > >
> > > > > > Hm, if having the full symtab in kallsyms is causing trouble, one
> > > > > > possibility would be to just have the module kallsyms code simply
> > > > > > skip/ignore undef symbols. That's what we technically do for normal
> > > > > > modules anyway (we normally cut undef syms out of the symtab). Haven't
> > > > > > tested this idea but does that sound like it'd help?
> > > > > 
> > > > > See if the following patch (untested) helps. It does not fix the
> > > > > /proc/kallsyms lookup, that requires a separate patch. But it should
> > > > > exclude the undef symbols from module_kallsyms_on_each_symbol() and
> > > > > thus also from klp_find_object_symbol().
> > > > 
> > > > That seems like it would work.  But wouldn't it be more robust if we
> > > > don't store the SHN_UNDEF symbols to start with?  Really it's only the
> > > > SHN_LIVEPATCH symbols that we need to keep, right?
> > > 
> > > Hm, didn't we have a reason for preserving the original symbol tables?
> > > It's not that we're storing the undef symbols on purpose, it's just
> > > that in the case of a livepatch module we keep *all* the symbols.
> > > IIRC, the original reasoning was that we needed to keep the original
> > > symtab indices in place, otherwise apply_relocate() (which uses the
> > > original symbol index encoded in the relocation) won't work.
> > > Selectively copying only SHN_LIVEPATCH symbols when building the
> > > module's symbol table will unfortunately mess the symbol indices up :/
> > > Under these restrictions, it just seemed easier to make kallsyms
> > > ignore undef symbols.
> > 
> > Ah, you're right.  I'll try out your patch.
> 
> I can confirm that your patch fixes the issue.  Thanks!

BTW you can add my

Reported-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Josh Poimboeuf <jpoimboe@redhat.com>

-- 
Josh

Re: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Jessica Yu]

+++ Josh Poimboeuf [04/06/18 18:02 -0500]:
>On Mon, Jun 04, 2018 at 05:56:05PM -0500, Josh Poimboeuf wrote:
>> On Mon, Jun 04, 2018 at 09:54:12AM -0500, Josh Poimboeuf wrote:
>> > On Mon, Jun 04, 2018 at 04:05:11PM +0200, Jessica Yu wrote:
>> > > +++ Josh Poimboeuf [04/06/18 08:16 -0500]:
>> > > > On Mon, Jun 04, 2018 at 03:01:31PM +0200, Jessica Yu wrote:
>> > > > > +++ Jessica Yu [04/06/18 11:54 +0200]:
>> > > > > > +++ Jessica Yu [04/06/18 10:05 +0200]:
>> > > > > > > +++ Josh Poimboeuf [02/06/18 12:32 -0500]:
>> > > > > > > > Hi Jessica,
>> > > > > > > >
>> > > > > > > > I found a bug:
>> > > > > > > >
>> > > > > > > > [root@f25 ~]# modprobe livepatch-sample
>> > > > > > > > [root@f25 ~]# grep ' u ' /proc/kallsyms
>> > > > > > > > ffffffff81161080 u klp_enable_patch	[livepatch_sample]
>> > > > > > > > ffffffff81a01800 u __fentry__	[livepatch_sample]
>> > > > > > > > ffffffff81161250 u klp_unregister_patch	[livepatch_sample]
>> > > > > > > > ffffffff81161870 u klp_register_patch	[livepatch_sample]
>> > > > > > > > ffffffff8131f0b0 u seq_printf	[livepatch_sample]
>> > > > > > > >
>> > > > > > > > Notice that livepatch modules' undefined symbols are showing up in
>> > > > > > > > /proc/kallsyms.  This can confuse klp_find_object_symbol() which can
>> > > > > > > > cause subtle bugs in livepatch.
>> > > > > > > >
>> > > > > > > > I stared at the module kallsyms code for a bit, but I don't see the bug.
>> > > > > > > > Maybe it has something to do with how we save the symbol table in
>> > > > > > > > copy_module_elf().  Any ideas?
>> > > > > > >
>> > > > > > > Hi Josh!
>> > > > > > >
>> > > > > > > This is because we preserve the entire symbol table for livepatch
>> > > > > > > modules, including the SHN_UNDEF symbols. IIRC, this is so that we can
>> > > > > > > still apply relocations properly with apply_relocate_add() after a
>> > > > > > > to-be-patched object is loaded. Normally we don't save these SHN_UNDEF
>> > > > > > > symbols for modules so they do not appear in /proc/kallsyms.
>> > > > > >
>> > > > > > Hm, if having the full symtab in kallsyms is causing trouble, one
>> > > > > > possibility would be to just have the module kallsyms code simply
>> > > > > > skip/ignore undef symbols. That's what we technically do for normal
>> > > > > > modules anyway (we normally cut undef syms out of the symtab). Haven't
>> > > > > > tested this idea but does that sound like it'd help?
>> > > > >
>> > > > > See if the following patch (untested) helps. It does not fix the
>> > > > > /proc/kallsyms lookup, that requires a separate patch. But it should
>> > > > > exclude the undef symbols from module_kallsyms_on_each_symbol() and
>> > > > > thus also from klp_find_object_symbol().
>> > > >
>> > > > That seems like it would work.  But wouldn't it be more robust if we
>> > > > don't store the SHN_UNDEF symbols to start with?  Really it's only the
>> > > > SHN_LIVEPATCH symbols that we need to keep, right?
>> > >
>> > > Hm, didn't we have a reason for preserving the original symbol tables?
>> > > It's not that we're storing the undef symbols on purpose, it's just
>> > > that in the case of a livepatch module we keep *all* the symbols.
>> > > IIRC, the original reasoning was that we needed to keep the original
>> > > symtab indices in place, otherwise apply_relocate() (which uses the
>> > > original symbol index encoded in the relocation) won't work.
>> > > Selectively copying only SHN_LIVEPATCH symbols when building the
>> > > module's symbol table will unfortunately mess the symbol indices up :/
>> > > Under these restrictions, it just seemed easier to make kallsyms
>> > > ignore undef symbols.
>> >
>> > Ah, you're right.  I'll try out your patch.
>>
>> I can confirm that your patch fixes the issue.  Thanks!
>
>BTW you can add my
>
>Reported-by: Josh Poimboeuf <jpoimboe@redhat.com>
>Tested-by: Josh Poimboeuf <jpoimboe@redhat.com>

Thanks for testing! Will clean up the patch and formally repost it.
It'll be queued up in modules-next after the merge window.

Thanks,

Jessica

[PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Jessica Yu]

Livepatch modules are special in that we preserve their entire symbol
tables in order to be able to apply relocations after module load. The
unwanted side effect of this is that undefined (SHN_UNDEF) symbols of
livepatch modules are accessible via the kallsyms api and this can
confuse symbol resolution in livepatch (klp_find_object_symbol()) and
cause subtle bugs in livepatch.

Have the module kallsyms api skip over SHN_UNDEF symbols. These symbols
are usually not available for normal modules anyway as we cut down their
symbol tables to just the core (non-undefined) symbols, so this should
really just affect livepatch modules. Note that this patch doesn't
affect the display of undefined symbols in /proc/kallsyms.

Reported-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Jessica Yu <jeyu@kernel.org>
---
 kernel/module.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/kernel/module.c b/kernel/module.c
index c9bea7f2b43e..dfa61490b37d 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -4070,7 +4070,7 @@ static unsigned long mod_find_symname(struct module *mod, const char *name)
 
 	for (i = 0; i < kallsyms->num_symtab; i++)
 		if (strcmp(name, symname(kallsyms, i)) == 0 &&
-		    kallsyms->symtab[i].st_info != 'U')
+		    kallsyms->symtab[i].st_shndx != SHN_UNDEF)
 			return kallsyms->symtab[i].st_value;
 	return 0;
 }
@@ -4116,6 +4116,10 @@ int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *,
 		if (mod->state == MODULE_STATE_UNFORMED)
 			continue;
 		for (i = 0; i < kallsyms->num_symtab; i++) {
+
+			if (kallsyms->symtab[i].st_shndx == SHN_UNDEF)
+				continue;
+
 			ret = fn(data, symname(kallsyms, i),
 				 mod, kallsyms->symtab[i].st_value);
 			if (ret != 0)
-- 
2.16.3

Re: [PATCH] module: exclude SHN_UNDEF symbols from kallsyms api

[Josh Poimboeuf]

On Tue, Jun 05, 2018 at 10:42:23AM +0200, Jessica Yu wrote:
> Livepatch modules are special in that we preserve their entire symbol
> tables in order to be able to apply relocations after module load. The
> unwanted side effect of this is that undefined (SHN_UNDEF) symbols of
> livepatch modules are accessible via the kallsyms api and this can
> confuse symbol resolution in livepatch (klp_find_object_symbol()) and
> cause subtle bugs in livepatch.
> 
> Have the module kallsyms api skip over SHN_UNDEF symbols. These symbols
> are usually not available for normal modules anyway as we cut down their
> symbol tables to just the core (non-undefined) symbols, so this should
> really just affect livepatch modules. Note that this patch doesn't
> affect the display of undefined symbols in /proc/kallsyms.
> 
> Reported-by: Josh Poimboeuf <jpoimboe@redhat.com>
> Tested-by: Josh Poimboeuf <jpoimboe@redhat.com>
> Signed-off-by: Jessica Yu <jeyu@kernel.org>

Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>

-- 
Josh