[RFC PATCH v3 0/6] (mostly) Arch-independent livepatch

[RFC PATCH v3 0/6] (mostly) Arch-independent livepatch

[Jessica Yu]

This patchset removes livepatch's need for architecture-specific relocation
code by leveraging existing code in the module loader to perform
arch-dependent work. Specifically, instead of duplicating code and
re-implementing what the apply_relocate_add() function in the module loader
already does in livepatch's klp_write_module_reloc(), we reuse
apply_relocate_add() to write relocations. The hope is that this will make
livepatch more easily portable to other architectures and greatly reduce
the amount of arch-specific code required to port livepatch to a particular
architecture.

Background: Why does livepatch need to write its own relocations?
==
A typical livepatch module contains patched versions of functions that can
reference non-exported global symbols and non-included local symbols.
Relocations referencing these types of symbols cannot be left in as-is
since the kernel module loader cannot resolve them and will therefore
reject the livepatch module. Furthermore, we cannot apply relocations that
affect modules not loaded yet at run time (e.g. a patch to a driver). The
current kpatch build system therefore solves this problem by embedding
special "dynrela" (dynamic reloc) sections in the resulting patch module
Elf output. Using these dynrela sections, livepatch can correctly resolve
symbols while taking into account its scope and what module the symbol
belongs to, and then manually apply the dynamic relocations.

Motivation: Why is having arch-dependent relocation code a problem?
==
The original motivation for this patchset stems from the increasing
roadblocks encountered while attempting to port livepatch to s390.
Specifically, there were problems dealing with s390 PLT and GOT relocation
types (R_390_{PLT,GOT}), which are handled differently from x86's
relocation types (which are much simpler to deal with, and a single
livepatch function (klp_write_module_reloc()) has been sufficient enough).
These s390 reloc types cannot be handled by simply performing a calculation
(as in the x86 case). For s390 modules with PLT/GOT relocations, the kernel
module loader allocates and fills in PLT+GOT table entries for every symbol
referenced by a PLT/GOT reloc in module core memory. So the problem of
porting livepatch to s390 became much more complicated than simply writing
an s390-specific klp_write_module_reloc() function. How can livepatch
handle these relocation types if the s390 module loader needs to allocate
and fill PLT/GOT entries ahead of time? The potential solutions were: 1)
have livepatch possibly allocate and maintain its own PLT/GOT tables for
every patch module (requiring even more arch-specific code), 2) modify the
s390 module loader heavily to accommodate livepatch modules (i.e. allocate
all the needed PLT/GOT entries for livepatch in advance but refrain from
applying relocations for to-be-patched modules), or 3) eliminate this
potential mess by leveraging module loader code to do all the relocation
work, letting livepatch off the hook completely. Solution #3 is what this
patchset implements.

How does this patchset remedy these problems?
==
Reusing the module loader code to perform livepatch relocations means that
livepatch no longer needs arch-specific reloc code and the aforementioned
problems with s390 PLT/GOT reloc types disappear (because we let the module
loader do all the relocation work for us). It will enable livepatch to be
more easily ported to other architectures.

Summary of proposed changes
==
This patch series enables livepatch to use the module loader's
apply_relocate_add() function to apply livepatch relocations (i.e. what
used to be dynrelas). apply_relocate_add() requires access to a patch
module's section headers, symbol table, reloc section indices, etc., and all
of these are accessible through the load_info struct used in the module
loader. Therefore we persist module Elf information (copied from load_info)
for livepatch modules.

The ELF-related changes enable livepatch to patch modules that are not yet
loaded (as well as patch vmlinux when kaslr is enabled). In order to use
apply_relocate_add(), we need real SHT_RELA sections to pass in. A
complication here is that relocations for not-yet-loaded modules should not
be applied when the patch module loads; they should only be applied once
the target module is loaded. Thus kpatch build scripts were modified to
output a livepatch module that contains special .klp.rel. sections that
are managed by livepatch and are applied at the appropriate time (i.e. when
target module loads). They are marked with a special SHF_RELA_LIVEPATCH
section flag to indicate to the module loader that livepatch will handle
them. The SHN_LIVEPATCH shndx marks symbols that need to be resolved
once their respective target module loads. So, the module loader ignores
these symbols and does not attempt to resolve them. These ELF constants
were selected from OS-specific ranges according to the definitions from
glibc.

Based on the following branch:
- git://git.kernel.org/pub/scm/linux/kernel/git/jikos/livepatching.git
  for-4.5/core

v3:
 - Remove usage of the klp_reloc_sec struct, since we can simply loop
   through the patch module's section headers.
 - Remove necessity of the "external" flag by prefixing symbol names
   with the object name and extracting this name during symbol
   resolution.
 - Create CONFIG_LIVEPATCH and !CONFIG_LIVEPATCH versions of
   {copy,free}_module_elf(), is_livepatch_module(), and
   check_livepatch_modinfo().
 - Encoded symbol position of a livepatch sym in its st_other field.
 - Various bug fixes from v2

v2:
 - Copy only the minimum required Elf information for livepatch modules to
   make the call to apply_relocate_add(), not the entire load_info struct
   and the redundant copy of the module in memory
 - Add module->klp flag for simple identification of livepatch modules
 - s390: remove redundant vfree() and preserve mod_arch_specific if
   livepatch module
 - Use array format instead of a linked list for klp_reloc_secs
 - Add new documentation describing the format of a livepatch module in
   Documentation/livepatch

Jessica Yu (6):
  Elf: add livepatch-specific Elf constants
  module: preserve Elf information for livepatch modules
  module: s390: keep mod_arch_specific for livepatch modules
  livepatch: reuse module loader code to write relocations
  samples: livepatch: mark as livepatch module
  Documentation: livepatch: outline the Elf format of a livepatch module

 Documentation/livepatch/patch-module-format.txt |  98 ++++++++++++++++
 arch/s390/kernel/module.c                       |  16 ++-
 arch/x86/include/asm/livepatch.h                |   2 -
 arch/x86/kernel/Makefile                        |   1 -
 arch/x86/kernel/livepatch.c                     |  70 -----------
 include/linux/livepatch.h                       |  33 +++---
 include/linux/module.h                          |  31 +++++
 include/uapi/linux/elf.h                        |  10 +-
 kernel/livepatch/core.c                         | 104 ++++++++---------
 kernel/module.c                                 | 149 +++++++++++++++++++++---
 samples/livepatch/livepatch-sample.c            |   1 +
 11 files changed, 347 insertions(+), 168 deletions(-)
 create mode 100644 Documentation/livepatch/patch-module-format.txt
 delete mode 100644 arch/x86/kernel/livepatch.c

-- 
2.4.3

[RFC PATCH v3 0/6] (mostly) Arch-independent livepatch

[Jessica Yu]

This patchset removes livepatch's need for architecture-specific relocation
code by leveraging existing code in the module loader to perform
arch-dependent work. Specifically, instead of duplicating code and
re-implementing what the apply_relocate_add() function in the module loader
already does in livepatch's klp_write_module_reloc(), we reuse
apply_relocate_add() to write relocations. The hope is that this will make
livepatch more easily portable to other architectures and greatly reduce
the amount of arch-specific code required to port livepatch to a particular
architecture.

Background: Why does livepatch need to write its own relocations?
==
A typical livepatch module contains patched versions of functions that can
reference non-exported global symbols and non-included local symbols.
Relocations referencing these types of symbols cannot be left in as-is
since the kernel module loader cannot resolve them and will therefore
reject the livepatch module. Furthermore, we cannot apply relocations that
affect modules not loaded yet at run time (e.g. a patch to a driver). The
current kpatch build system therefore solves this problem by embedding
special "dynrela" (dynamic reloc) sections in the resulting patch module
Elf output. Using these dynrela sections, livepatch can correctly resolve
symbols while taking into account its scope and what module the symbol
belongs to, and then manually apply the dynamic relocations.

Motivation: Why is having arch-dependent relocation code a problem?
==
The original motivation for this patchset stems from the increasing
roadblocks encountered while attempting to port livepatch to s390.
Specifically, there were problems dealing with s390 PLT and GOT relocation
types (R_390_{PLT,GOT}), which are handled differently from x86's
relocation types (which are much simpler to deal with, and a single
livepatch function (klp_write_module_reloc()) has been sufficient enough).
These s390 reloc types cannot be handled by simply performing a calculation
(as in the x86 case). For s390 modules with PLT/GOT relocations, the kernel
module loader allocates and fills in PLT+GOT table entries for every symbol
referenced by a PLT/GOT reloc in module core memory. So the problem of
porting livepatch to s390 became much more complicated than simply writing
an s390-specific klp_write_module_reloc() function. How can livepatch
handle these relocation types if the s390 module loader needs to allocate
and fill PLT/GOT entries ahead of time? The potential solutions were: 1)
have livepatch possibly allocate and maintain its own PLT/GOT tables for
every patch module (requiring even more arch-specific code), 2) modify the
s390 module loader heavily to accommodate livepatch modules (i.e. allocate
all the needed PLT/GOT entries for livepatch in advance but refrain from
applying relocations for to-be-patched modules), or 3) eliminate this
potential mess by leveraging module loader code to do all the relocation
work, letting livepatch off the hook completely. Solution #3 is what this
patchset implements.

How does this patchset remedy these problems?
==
Reusing the module loader code to perform livepatch relocations means that
livepatch no longer needs arch-specific reloc code and the aforementioned
problems with s390 PLT/GOT reloc types disappear (because we let the module
loader do all the relocation work for us). It will enable livepatch to be
more easily ported to other architectures.

Summary of proposed changes
==
This patch series enables livepatch to use the module loader's
apply_relocate_add() function to apply livepatch relocations (i.e. what
used to be dynrelas). apply_relocate_add() requires access to a patch
module's section headers, symbol table, reloc section indices, etc., and all
of these are accessible through the load_info struct used in the module
loader. Therefore we persist module Elf information (copied from load_info)
for livepatch modules.

The ELF-related changes enable livepatch to patch modules that are not yet
loaded (as well as patch vmlinux when kaslr is enabled). In order to use
apply_relocate_add(), we need real SHT_RELA sections to pass in. A
complication here is that relocations for not-yet-loaded modules should not
be applied when the patch module loads; they should only be applied once
the target module is loaded. Thus kpatch build scripts were modified to
output a livepatch module that contains special .klp.rel. sections that
are managed by livepatch and are applied at the appropriate time (i.e. when
target module loads). They are marked with a special SHF_RELA_LIVEPATCH
section flag to indicate to the module loader that livepatch will handle
them. The SHN_LIVEPATCH shndx marks symbols that need to be resolved
once their respective target module loads. So, the module loader ignores
these symbols and does not attempt to resolve them. These ELF constants
were selected from OS-specific ranges according to the definitions from
glibc.

Based on the following branch:
- git://git.kernel.org/pub/scm/linux/kernel/git/jikos/livepatching.git
  for-4.5/core

v3:
 - Remove usage of the klp_reloc_sec struct, since we can simply loop
   through the patch module's section headers.
 - Remove necessity of the "external" flag by prefixing symbol names
   with the object name and extracting this name during symbol
   resolution.
 - Create CONFIG_LIVEPATCH and !CONFIG_LIVEPATCH versions of
   {copy,free}_module_elf(), is_livepatch_module(), and
   check_livepatch_modinfo().
 - Encoded symbol position of a livepatch sym in its st_other field.
 - Various bug fixes from v2

v2:
 - Copy only the minimum required Elf information for livepatch modules to
   make the call to apply_relocate_add(), not the entire load_info struct
   and the redundant copy of the module in memory
 - Add module->klp flag for simple identification of livepatch modules
 - s390: remove redundant vfree() and preserve mod_arch_specific if
   livepatch module
 - Use array format instead of a linked list for klp_reloc_secs
 - Add new documentation describing the format of a livepatch module in
   Documentation/livepatch

Jessica Yu (6):
  Elf: add livepatch-specific Elf constants
  module: preserve Elf information for livepatch modules
  module: s390: keep mod_arch_specific for livepatch modules
  livepatch: reuse module loader code to write relocations
  samples: livepatch: mark as livepatch module
  Documentation: livepatch: outline the Elf format of a livepatch module

 Documentation/livepatch/patch-module-format.txt |  98 ++++++++++++++++
 arch/s390/kernel/module.c                       |  16 ++-
 arch/x86/include/asm/livepatch.h                |   2 -
 arch/x86/kernel/Makefile                        |   1 -
 arch/x86/kernel/livepatch.c                     |  70 -----------
 include/linux/livepatch.h                       |  33 +++---
 include/linux/module.h                          |  31 +++++
 include/uapi/linux/elf.h                        |  10 +-
 kernel/livepatch/core.c                         | 104 ++++++++---------
 kernel/module.c                                 | 149 +++++++++++++++++++++---
 samples/livepatch/livepatch-sample.c            |   1 +
 11 files changed, 347 insertions(+), 168 deletions(-)
 create mode 100644 Documentation/livepatch/patch-module-format.txt
 delete mode 100644 arch/x86/kernel/livepatch.c

-- 
2.4.3

[RFC PATCH v3 1/6] Elf: add livepatch-specific Elf constants

[Jessica Yu]

Add livepatch Elf relocation section flag (SHF_RELA_LIVEPATCH), and symbol
section index (SHN_LIVEPATCH). The values of these Elf constants were
selected from OS-specific ranges according to the definitions from glibc.

Livepatch relocation sections are marked with SHF_RELA_LIVEPATCH to
indicate to the module loader that it should not apply that relocation
section and that livepatch will handle them.

The SHN_LIVEPATCH shndx marks symbols that will be resolved by livepatch.
The module loader ignores these symbols and does not attempt to resolve
them.

Signed-off-by: Jessica Yu <jeyu@redhat.com>
---
 include/uapi/linux/elf.h | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index 71e1d0e..cb4a72f 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -282,16 +282,18 @@ typedef struct elf64_phdr {
 #define SHT_HIUSER	0xffffffff
 
 /* sh_flags */
-#define SHF_WRITE	0x1
-#define SHF_ALLOC	0x2
-#define SHF_EXECINSTR	0x4
-#define SHF_MASKPROC	0xf0000000
+#define SHF_WRITE		0x1
+#define SHF_ALLOC		0x2
+#define SHF_EXECINSTR		0x4
+#define SHF_RELA_LIVEPATCH	0x00100000
+#define SHF_MASKPROC		0xf0000000
 
 /* special section indexes */
 #define SHN_UNDEF	0
 #define SHN_LORESERVE	0xff00
 #define SHN_LOPROC	0xff00
 #define SHN_HIPROC	0xff1f
+#define SHN_LIVEPATCH	0xff20
 #define SHN_ABS		0xfff1
 #define SHN_COMMON	0xfff2
 #define SHN_HIRESERVE	0xffff
-- 
2.4.3

[RFC PATCH v3 2/6] module: preserve Elf information for livepatch modules

[Jessica Yu]

For livepatch modules, copy Elf section, symbol, and string information
from the load_info struct in the module loader.

Livepatch uses special relocation sections in order to be able to patch
modules that are not yet loaded, as well as apply patches to the kernel
when the addresses of symbols cannot be determined at compile time (for
example, when kaslr is enabled). Livepatch modules must preserve Elf
information such as section indices in order to apply the remaining
relocation sections at the appropriate time (i.e. when the target module
loads).

Signed-off-by: Jessica Yu <jeyu@redhat.com>
---
 include/linux/module.h |  31 ++++++++++
 kernel/module.c        | 149 +++++++++++++++++++++++++++++++++++++++++++------
 2 files changed, 164 insertions(+), 16 deletions(-)

diff --git a/include/linux/module.h b/include/linux/module.h
index 4560d8f..ba6052d 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -281,6 +281,21 @@ void *__symbol_get(const char *symbol);
 void *__symbol_get_gpl(const char *symbol);
 #define symbol_get(x) ((typeof(&x))(__symbol_get(VMLINUX_SYMBOL_STR(x))))
 
+struct load_info {
+	Elf_Ehdr *hdr;
+	unsigned long len;
+	Elf_Shdr *sechdrs;
+	char *secstrings, *strtab;
+	unsigned long symoffs, stroffs;
+	struct _ddebug *debug;
+	unsigned int num_debug;
+	bool sig_ok;
+	struct {
+		unsigned int sym, str, mod, vers, info, pcpu;
+	} index;
+};
+
+
 /* modules using other modules: kdb wants to see this. */
 struct module_use {
 	struct list_head source_list;
@@ -455,7 +470,11 @@ struct module {
 #endif
 
 #ifdef CONFIG_LIVEPATCH
+	bool klp; /* Is this a livepatch module? */
 	bool klp_alive;
+
+	/* Elf information */
+	struct load_info *info;
 #endif
 
 #ifdef CONFIG_MODULE_UNLOAD
@@ -798,4 +817,16 @@ static inline bool module_sig_ok(struct module *module)
 }
 #endif	/* CONFIG_MODULE_SIG */
 
+#ifdef CONFIG_LIVEPATCH
+static inline bool is_livepatch_module(struct module *mod)
+{
+	return mod->klp;
+}
+#else
+static inline bool is_livepatch_module(struct module *mod)
+{
+	return false;
+}
+#endif /* CONFIG_LIVEPATCH */
+
 #endif /* _LINUX_MODULE_H */
diff --git a/kernel/module.c b/kernel/module.c
index 912e891..3d14e36 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -294,20 +294,6 @@ int unregister_module_notifier(struct notifier_block *nb)
 }
 EXPORT_SYMBOL(unregister_module_notifier);
 
-struct load_info {
-	Elf_Ehdr *hdr;
-	unsigned long len;
-	Elf_Shdr *sechdrs;
-	char *secstrings, *strtab;
-	unsigned long symoffs, stroffs;
-	struct _ddebug *debug;
-	unsigned int num_debug;
-	bool sig_ok;
-	struct {
-		unsigned int sym, str, mod, vers, info, pcpu;
-	} index;
-};
-
 /* We require a truly strong try_module_get(): 0 means failure due to
    ongoing or failed initialization etc. */
 static inline int strong_try_module_get(struct module *mod)
@@ -1966,6 +1952,89 @@ static void module_enable_nx(const struct module *mod) { }
 static void module_disable_nx(const struct module *mod) { }
 #endif
 
+#ifdef CONFIG_LIVEPATCH
+/*
+ * copy_module_elf - preserve Elf information about a module
+ *
+ * Copy relevant Elf information from the load_info struct.
+ * Note: not all fields from the original load_info are
+ * copied into mod->info.
+ */
+static int copy_module_elf(struct module *mod, struct load_info *info)
+{
+	unsigned int size;
+	int ret = 0;
+
+	mod->info = kmalloc(sizeof(*info), GFP_KERNEL);
+	if (mod->info == NULL)
+		return -ENOMEM;
+
+	/* Elf header */
+	size = sizeof(Elf_Ehdr);
+	mod->info->hdr = kmalloc(size, GFP_KERNEL);
+	if (mod->info->hdr == NULL) {
+		ret = -ENOMEM;
+		goto free_info;
+	}
+	memcpy(mod->info->hdr, info->hdr, size);
+
+	/* Elf section header table */
+	size = sizeof(Elf_Shdr) * info->hdr->e_shnum;
+	mod->info->sechdrs = kmalloc(size, GFP_KERNEL);
+	if (mod->info->sechdrs == NULL) {
+		ret = -ENOMEM;
+		goto free_hdr;
+	}
+	memcpy(mod->info->sechdrs, info->sechdrs, size);
+
+	/* Elf section name string table */
+	size = info->sechdrs[info->hdr->e_shstrndx].sh_size;
+	mod->info->secstrings = kmalloc(size, GFP_KERNEL);
+	if (mod->info->secstrings == NULL) {
+		ret = -ENOMEM;
+		goto free_sechdrs;
+	}
+	memcpy(mod->info->secstrings, info->secstrings, size);
+
+	/* Elf section indices */
+	memcpy(&mod->info->index, &info->index, sizeof(info->index));
+
+	/*
+	 * Update symtab's sh_addr to point to a valid
+	 * symbol table, as the temporary symtab in module
+	 * init memory will be freed
+	 */
+	mod->info->sechdrs[mod->info->index.sym].sh_addr = (unsigned long)mod->core_symtab;
+
+	return ret;
+
+free_sechdrs:
+	kfree(mod->info->sechdrs);
+free_hdr:
+	kfree(mod->info->hdr);
+free_info:
+	kfree(mod->info);
+	return ret;
+}
+
+static void free_module_elf(struct module *mod)
+{
+	kfree(mod->info->sechdrs);
+	kfree(mod->info->hdr);
+	kfree(mod->info->secstrings);
+	kfree(mod->info);
+}
+#else /* !CONFIG_LIVEPATCH */
+static int copy_module_elf(struct module *mod, struct load_info *info)
+{
+	return 0;
+}
+
+static void free_module_elf(struct module *mod)
+{
+}
+#endif /* CONFIG_LIVEPATCH */
+
 void __weak module_memfree(void *module_region)
 {
 	vfree(module_region);
@@ -2004,6 +2073,10 @@ static void free_module(struct module *mod)
 	/* Free any allocated parameters. */
 	destroy_params(mod->kp, mod->num_kp);
 
+	/* Free Elf information if it was saved */
+	if (is_livepatch_module(mod))
+		free_module_elf(mod);
+
 	/* Now we can delete it from the lists */
 	mutex_lock(&module_mutex);
 	/* Unlink carefully: kallsyms could be walking list. */
@@ -2119,6 +2192,10 @@ static int simplify_symbols(struct module *mod, const struct load_info *info)
 			       (long)sym[i].st_value);
 			break;
 
+		case SHN_LIVEPATCH:
+			/* klp symbols are resolved by livepatch */
+			break;
+
 		case SHN_UNDEF:
 			ksym = resolve_symbol_wait(mod, info, name);
 			/* Ok if resolved.  */
@@ -2167,6 +2244,10 @@ static int apply_relocations(struct module *mod, const struct load_info *info)
 		if (!(info->sechdrs[infosec].sh_flags & SHF_ALLOC))
 			continue;
 
+		/* klp relocation sections are applied by livepatch */
+		if (info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH)
+			continue;
+
 		if (info->sechdrs[i].sh_type == SHT_REL)
 			err = apply_relocate(info->sechdrs, info->strtab,
 					     info->index.sym, i, mod);
@@ -2385,6 +2466,8 @@ static char elf_type(const Elf_Sym *sym, const struct load_info *info)
 		return 'U';
 	if (sym->st_shndx == SHN_ABS || sym->st_shndx == info->index.pcpu)
 		return 'a';
+	if (sym->st_shndx == SHN_LIVEPATCH)
+		return 'k';
 	if (sym->st_shndx >= SHN_LORESERVE)
 		return '?';
 	if (sechdrs[sym->st_shndx].sh_flags & SHF_EXECINSTR)
@@ -2462,7 +2545,7 @@ static void layout_symtab(struct module *mod, struct load_info *info)
 
 	/* Compute total space required for the core symbols' strtab. */
 	for (ndst = i = 0; i < nsrc; i++) {
-		if (i == 0 ||
+		if (i == 0 || is_livepatch_module(mod) ||
 		    is_core_symbol(src+i, info->sechdrs, info->hdr->e_shnum,
 				   info->index.pcpu)) {
 			strtab_size += strlen(&info->strtab[src[i].st_name])+1;
@@ -2505,7 +2588,7 @@ static void add_kallsyms(struct module *mod, const struct load_info *info)
 	mod->core_strtab = s = mod->core_layout.base + info->stroffs;
 	src = mod->symtab;
 	for (ndst = i = 0; i < mod->num_symtab; i++) {
-		if (i == 0 ||
+		if (i == 0 || is_livepatch_module(mod) ||
 		    is_core_symbol(src+i, info->sechdrs, info->hdr->e_shnum,
 				   info->index.pcpu)) {
 			dst[ndst] = src[i];
@@ -2672,6 +2755,24 @@ static int copy_module_from_user(const void __user *umod, unsigned long len,
 	return 0;
 }
 
+#ifdef CONFIG_LIVEPATCH
+static int check_livepatch_modinfo(struct module *mod, struct load_info *info)
+{
+	mod->klp = get_modinfo(info, "livepatch") ? true : false;
+
+	return 0;
+}
+#else
+static int check_livepatch_modinfo(struct module *mod, struct load_info *info)
+{
+	if (get_modinfo(info, "livepatch"))
+		return -EINVAL;
+
+	return 0;
+}
+#endif
+
+
 /* Sets info->hdr and info->len. */
 static int copy_module_from_fd(int fd, struct load_info *info)
 {
@@ -2855,6 +2956,10 @@ static int check_modinfo(struct module *mod, struct load_info *info, int flags)
 			"is unknown, you have been warned.\n", mod->name);
 	}
 
+	err = check_livepatch_modinfo(mod, info);
+	if (err)
+		return -ENOEXEC;
+
 	/* Set up license info based on the info section */
 	set_license(mod, get_modinfo(info, "license"));
 
@@ -3511,6 +3616,16 @@ static int load_module(struct load_info *info, const char __user *uargs,
 	if (err < 0)
 		goto bug_cleanup;
 
+	/*
+	 * Save sechdrs, indices, and other data from
+	 * load_info for livepatch modules
+	 */
+	if (is_livepatch_module(mod)) {
+		err = copy_module_elf(mod, info);
+		if (err < 0)
+			goto sysfs_cleanup;
+	}
+
 	/* Get rid of temporary copy. */
 	free_copy(info);
 
@@ -3519,6 +3634,8 @@ static int load_module(struct load_info *info, const char __user *uargs,
 
 	return do_init_module(mod);
 
+ sysfs_cleanup:
+	mod_sysfs_teardown(mod);
  bug_cleanup:
 	/* module_bug_cleanup needs module_mutex protection */
 	mutex_lock(&module_mutex);
-- 
2.4.3

[RFC PATCH v3 3/6] module: s390: keep mod_arch_specific for livepatch modules

[Jessica Yu]

Livepatch needs to utilize the symbol information contained in the
mod_arch_specific struct in order to be able to call the s390
apply_relocate_add() function to apply relocations. Keep a reference to
syminfo if the module is a livepatch module. Remove the redundant vfree()
in module_finalize() since module_arch_freeing_init() (which also frees
those structures) is called in do_init_module(). If the module isn't a
livepatch module, we free the structures in module_arch_freeing_init() as
usual.

Signed-off-by: Jessica Yu <jeyu@redhat.com>
---
 arch/s390/kernel/module.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c
index 7873e17..fbc0789 100644
--- a/arch/s390/kernel/module.c
+++ b/arch/s390/kernel/module.c
@@ -51,6 +51,10 @@ void *module_alloc(unsigned long size)
 
 void module_arch_freeing_init(struct module *mod)
 {
+	if (is_livepatch_module(mod) &&
+	    mod->state == MODULE_STATE_LIVE)
+		return;
+
 	vfree(mod->arch.syminfo);
 	mod->arch.syminfo = NULL;
 }
@@ -425,7 +429,5 @@ int module_finalize(const Elf_Ehdr *hdr,
 		    struct module *me)
 {
 	jump_label_apply_nops(me);
-	vfree(me->arch.syminfo);
-	me->arch.syminfo = NULL;
 	return 0;
 }
-- 
2.4.3

[RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Jessica Yu]

Reuse module loader code to write relocations, thereby eliminating the need
for architecture specific relocation code in livepatch. Namely, we reuse
apply_relocate_add() in the module loader to write relocations instead of
duplicating functionality in livepatch's klp_write_module_reloc(). To apply
relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
are resolved and then apply_relocate_add() is called to apply those
relocations.

In addition, remove x86 livepatch relocation code. It is no longer needed
since symbol resolution and relocation work have been offloaded to module
loader.

Signed-off-by: Jessica Yu <jeyu@redhat.com>
---
 arch/x86/include/asm/livepatch.h |   2 -
 arch/x86/kernel/Makefile         |   1 -
 arch/x86/kernel/livepatch.c      |  70 ---------------------------
 include/linux/livepatch.h        |  33 +++++--------
 kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
 5 files changed, 62 insertions(+), 145 deletions(-)
 delete mode 100644 arch/x86/kernel/livepatch.c

diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
index 19c099a..7312e25 100644
--- a/arch/x86/include/asm/livepatch.h
+++ b/arch/x86/include/asm/livepatch.h
@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
 #endif
 	return 0;
 }
-int klp_write_module_reloc(struct module *mod, unsigned long type,
-			   unsigned long loc, unsigned long value);
 
 static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
 {
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index b1b78ff..c5e9a5c 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
 obj-y				+= apic/
 obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
 obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
 obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
 obj-$(CONFIG_X86_TSC)		+= trace_clock.o
diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
deleted file mode 100644
index 92fc1a5..0000000
--- a/arch/x86/kernel/livepatch.c
+++ /dev/null
@@ -1,70 +0,0 @@
-/*
- * livepatch.c - x86-specific Kernel Live Patching Core
- *
- * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
- * Copyright (C) 2014 SUSE
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version 2
- * of the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <linux/module.h>
-#include <linux/uaccess.h>
-#include <asm/elf.h>
-#include <asm/livepatch.h>
-
-/**
- * klp_write_module_reloc() - write a relocation in a module
- * @mod:	module in which the section to be modified is found
- * @type:	ELF relocation type (see asm/elf.h)
- * @loc:	address that the relocation should be written to
- * @value:	relocation value (sym address + addend)
- *
- * This function writes a relocation to the specified location for
- * a particular module.
- */
-int klp_write_module_reloc(struct module *mod, unsigned long type,
-			   unsigned long loc, unsigned long value)
-{
-	size_t size = 4;
-	unsigned long val;
-	unsigned long core = (unsigned long)mod->core_layout.base;
-	unsigned long core_size = mod->core_layout.size;
-
-	switch (type) {
-	case R_X86_64_NONE:
-		return 0;
-	case R_X86_64_64:
-		val = value;
-		size = 8;
-		break;
-	case R_X86_64_32:
-		val = (u32)value;
-		break;
-	case R_X86_64_32S:
-		val = (s32)value;
-		break;
-	case R_X86_64_PC32:
-		val = (u32)(value - loc);
-		break;
-	default:
-		/* unsupported relocation type */
-		return -EINVAL;
-	}
-
-	if (loc < core || loc >= core + core_size)
-		/* loc does not point to any symbol inside the module */
-		return -EINVAL;
-
-	return probe_kernel_write((void *)loc, &val, size);
-}
diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
index a882865..2f12ce7 100644
--- a/include/linux/livepatch.h
+++ b/include/linux/livepatch.h
@@ -65,27 +65,8 @@ struct klp_func {
 };
 
 /**
- * struct klp_reloc - relocation structure for live patching
- * @loc:	address where the relocation will be written
- * @sympos:	position in kallsyms to disambiguate symbols (optional)
- * @type:	ELF relocation type
- * @name:	name of the referenced symbol (for lookup/verification)
- * @addend:	offset from the referenced symbol
- * @external:	symbol is either exported or within the live patch module itself
- */
-struct klp_reloc {
-	unsigned long loc;
-	unsigned long sympos;
-	unsigned long type;
-	const char *name;
-	int addend;
-	int external;
-};
-
-/**
  * struct klp_object - kernel object structure for live patching
  * @name:	module name (or NULL for vmlinux)
- * @relocs:	relocation entries to be applied at load time
  * @funcs:	function entries for functions to be patched in the object
  * @kobj:	kobject for sysfs resources
  * @mod:	kernel module associated with the patched object
@@ -95,7 +76,6 @@ struct klp_reloc {
 struct klp_object {
 	/* external */
 	const char *name;
-	struct klp_reloc *relocs;
 	struct klp_func *funcs;
 
 	/* internal */
@@ -123,6 +103,19 @@ struct klp_patch {
 	enum klp_state state;
 };
 
+/*
+ * Livepatch-specific symbols and relocation
+ * sections are prefixed with a tag:
+ * .klp.rel. for relocation sections
+ * .klp.sym. for livepatch symbols
+ */
+#define KLP_TAG_LEN 9
+/*
+ * Livepatch-specific bits for specifying symbol
+ * positions in the Elf_Sym st_other field
+ */
+#define KLP_SYMPOS(o) (o >> 2) & 0xff
+
 #define klp_for_each_object(patch, obj) \
 	for (obj = patch->objs; obj->funcs; obj++)
 
diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
index bc2c85c..64536a4 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -28,6 +28,9 @@
 #include <linux/list.h>
 #include <linux/kallsyms.h>
 #include <linux/livepatch.h>
+#include <linux/elf.h>
+#include <linux/string.h>
+#include <linux/moduleloader.h>
 #include <asm/cacheflush.h>
 
 /**
@@ -204,74 +207,70 @@ static int klp_find_object_symbol(const char *objname, const char *name,
 	return -EINVAL;
 }
 
-/*
- * external symbols are located outside the parent object (where the parent
- * object is either vmlinux or the kmod being patched).
- */
-static int klp_find_external_symbol(struct module *pmod, const char *name,
-				    unsigned long *addr)
+static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
 {
-	const struct kernel_symbol *sym;
+	int i, len, ret = 0;
+	Elf_Rela *relas;
+	Elf_Sym *sym;
+	char *symname, *sym_objname;
 
-	/* first, check if it's an exported symbol */
-	preempt_disable();
-	sym = find_symbol(name, NULL, NULL, true, true);
-	if (sym) {
-		*addr = sym->value;
-		preempt_enable();
-		return 0;
+	relas = (Elf_Rela *) relsec->sh_addr;
+	/* For each rela in this .klp.rel. section */
+	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
+		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
+		symname = pmod->core_strtab + sym->st_name;
+
+		len = strcspn(symname + KLP_TAG_LEN, ".");
+		sym_objname = strncmp(symname + KLP_TAG_LEN, "vmlinux", len) ?
+			kstrndup(symname + KLP_TAG_LEN, len, GFP_KERNEL) : NULL;
+		/* .klp.sym.objname.[symbol_name] */
+		symname += KLP_TAG_LEN + len + 1;
+
+		ret = klp_find_object_symbol(sym_objname, symname,
+					     KLP_SYMPOS(sym->st_other),
+					     (unsigned long *) &sym->st_value);
+		kfree(sym_objname);
+		if (ret)
+			return ret;
 	}
-	preempt_enable();
 
-	/*
-	 * Check if it's in another .o within the patch module. This also
-	 * checks that the external symbol is unique.
-	 */
-	return klp_find_object_symbol(pmod->name, name, 0, addr);
+	return ret;
 }
 
 static int klp_write_object_relocations(struct module *pmod,
 					struct klp_object *obj)
 {
-	int ret = 0;
-	unsigned long val;
-	struct klp_reloc *reloc;
+	int i, len, ret = 0;
+	char *secname;
+	const char *objname;
 
 	if (WARN_ON(!klp_is_object_loaded(obj)))
 		return -EINVAL;
 
-	if (WARN_ON(!obj->relocs))
-		return -EINVAL;
+	objname = klp_is_module(obj) ? obj->name : "vmlinux";
 
 	module_disable_ro(pmod);
+	/* For each klp rela section for this object */
+	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
+		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
+			continue;
+
+		secname = pmod->info->secstrings + pmod->info->sechdrs[i].sh_name;
+		/* .klp.rel.[objname].section_name */
+		len = strcspn(secname + KLP_TAG_LEN, ".");
+
+		if (strncmp(objname, secname + KLP_TAG_LEN, len))
+			continue;
 
-	for (reloc = obj->relocs; reloc->name; reloc++) {
-		/* discover the address of the referenced symbol */
-		if (reloc->external) {
-			if (reloc->sympos > 0) {
-				pr_err("non-zero sympos for external reloc symbol '%s' is not supported\n",
-				       reloc->name);
-				ret = -EINVAL;
-				goto out;
-			}
-			ret = klp_find_external_symbol(pmod, reloc->name, &val);
-		} else
-			ret = klp_find_object_symbol(obj->name,
-						     reloc->name,
-						     reloc->sympos,
-						     &val);
+		ret = klp_resolve_symbols(pmod->info->sechdrs + i, pmod);
 		if (ret)
 			goto out;
 
-		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
-					     val + reloc->addend);
-		if (ret) {
-			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
-			       reloc->name, val, ret);
+		ret = apply_relocate_add(pmod->info->sechdrs, pmod->core_strtab,
+					 pmod->info->index.sym, i, pmod);
+		if (ret)
 			goto out;
-		}
 	}
-
 out:
 	module_enable_ro(pmod);
 	return ret;
@@ -703,11 +702,9 @@ static int klp_init_object_loaded(struct klp_patch *patch,
 	struct klp_func *func;
 	int ret;
 
-	if (obj->relocs) {
-		ret = klp_write_object_relocations(patch->mod, obj);
-		if (ret)
-			return ret;
-	}
+	ret = klp_write_object_relocations(patch->mod, obj);
+	if (ret)
+		return ret;
 
 	klp_for_each_func(obj, func) {
 		ret = klp_find_object_symbol(obj->name, func->old_name,
-- 
2.4.3

[RFC PATCH v3 5/6] samples: livepatch: mark as livepatch module

[Jessica Yu]

Mark the module as a livepatch module so that the module loader can
appropriately identify and initialize it.

Signed-off-by: Jessica Yu <jeyu@redhat.com>
---
 samples/livepatch/livepatch-sample.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/samples/livepatch/livepatch-sample.c b/samples/livepatch/livepatch-sample.c
index fb8c861..e34f871 100644
--- a/samples/livepatch/livepatch-sample.c
+++ b/samples/livepatch/livepatch-sample.c
@@ -89,3 +89,4 @@ static void livepatch_exit(void)
 module_init(livepatch_init);
 module_exit(livepatch_exit);
 MODULE_LICENSE("GPL");
+MODULE_INFO(livepatch, "Y");
-- 
2.4.3

[RFC PATCH v3 6/6] Documentation: livepatch: outline the Elf format of a livepatch module

[Jessica Yu]

Document the special Elf sections and constants livepatch modules use.

Signed-off-by: Jessica Yu <jeyu@redhat.com>
---
 Documentation/livepatch/patch-module-format.txt | 106 ++++++++++++++++++++++++
 1 file changed, 106 insertions(+)
 create mode 100644 Documentation/livepatch/patch-module-format.txt

diff --git a/Documentation/livepatch/patch-module-format.txt b/Documentation/livepatch/patch-module-format.txt
new file mode 100644
index 0000000..d825629
--- /dev/null
+++ b/Documentation/livepatch/patch-module-format.txt
@@ -0,0 +1,106 @@
+---------------------------
+Livepatch module Elf format
+---------------------------
+
+This document outlines the special Elf constants and sections livepatch
+uses to patch both modules and the kernel (vmlinux).
+
+--------------------------
+1. Livepatch modinfo field
+--------------------------
+
+Livepatch modules can be identified by users by using the 'modinfo' command
+and looking for the presence of the "livepatch" field. This field is also
+used by the kernel module loader to identify livepatch modules.
+
+Example modinfo output:
+
+% modinfo kpatch-meminfo.ko
+filename:		kpatch-meminfo.ko
+livepatch:		Y
+license:		GPL
+depends:
+vermagic:		4.3.0+ SMP mod_unload
+
+--------------------
+2. Livepatch symbols
+--------------------
+
+These are symbols marked with SHN_LIVEPATCH and their names are prefixed
+with the string ".klp.sym.${objname}.", where ${objname} is the name of the
+"object" where symbol stems from (the name of a module, for example).
+A symbol's position (used to differentiate duplicate symbols within the
+same object) in its object is encoded in the Elf_Sym st_other field
+and accessed with the KLP_SYMPOS macro (see include/linux/livepatch.h)
+
+Livepatch symbols are manually resolved by livepatch, and are used in cases
+where we cannot immediately know the address of a symbol because the
+to-be-patched module is not loaded yet. Livepatch modules keep these
+symbols in their symbol tables, and the symbol table is made accessible
+through module->core_symtab. For livepatch modules, core_symtab will
+contain an exact copy of the original symbol table as opposed to a stripped
+down version containing just the "core" symbols.
+
+-----------------------------------
+3. ".klp.rel." relocation sections
+-----------------------------------
+
+A livepatch module uses special Elf relocation sections to apply
+relocations both for regular vmlinux patches as well as those that should
+be applied as soon as the to-be-patched module is loaded. For example, if a
+patch module patches a driver that is not currently loaded, livepatch will
+apply its corresponding klp relocation section(s) to the driver once it
+loads.
+
+The names of these livepatch relocation sections are formatted
+".klp.rel.${objname}.", where ${objname} is the name of the "object" being
+patched (e.g. vmlinux or name of module). Each object within a patch module
+may have multiple klp sections (e.g. patches to multiple functions within
+the same object). There is a 1-1 correspondence between a klp relocation
+section and the target section (usually the text section for a function) to
+which the relocation(s) apply.
+
+Here's a sample readelf output for a livepatch module that patches vmlinux and
+modules 9p, btrfs, ext4:
+  ...
+  [29] .klp.rel.9p.text.caches.show RELA                    0000000000000000 002d58 0000c0 18 AIo 64   9  8
+  [30] .klp.rel.btrfs.text.btrfs.feature.attr.show RELA     0000000000000000 002e18 000060 18 AIo 64  11  8
+  ...
+  [34] .klp.rel.ext4.text.ext4.attr.store RELA              0000000000000000 002fd8 0000d8 18 AIo 64  13  8
+  [35] .klp.rel.ext4.text.ext4.attr.show RELA               0000000000000000 0030b0 000150 18 AIo 64  15  8
+  [36] .klp.rel.vmlinux.text.cmdline.proc.show RELA         0000000000000000 003200 000018 18 AIo 64  17  8
+  [37] .klp.rel.vmlinux.text.meminfo.proc.show RELA         0000000000000000 003218 0000f0 18 AIo 64  19  8
+  ...
+
+klp relocation sections are SHT_RELA sections but with a few special
+characteristics. Notice that they are marked SHF_ALLOC ("A") so that they
+will not be discarded when the module is loaded into memory, as well as
+with the SHF_RELA_LIVEPATCH flag ("o" - for OS-specific) so the module
+loader can identify them and avoid treating them as regular SHT_RELA
+sections, since they are manually managed by livepatch.
+
+Since Elf information is preserved for livepatch modules (see Section 4), a
+klp relocation section can be applied simply by passing in the appropriate
+section index to apply_relocate_add() (in the module loader code), which
+then uses it to access the relocation section and apply the relocations.
+
+--------------------------------------------------------
+4. How a livepatch module accesses its symbol table and
+its klp relocation sections
+--------------------------------------------------------
+
+The kernel module loader checks whether the module being loaded is a
+livepatch module. If so, it then makes a copy of the module's Elf header,
+section headers, section name string table, and some noteworthy section
+indices (for example, the symtab's section index). It adjusts the symtab's
+sh_addr to point to mod->core_symtab, since the original mod->symtab lies
+in init memory and gets freed once the module finishes initializing. For
+livepatch modules, the core_symtab will be an exact copy of its original
+symbol table (where normally, only "core" symbols are included in this
+symbol table. See is_core_symbol() in kernel/module.c). Livepatch requires
+that the symbols retain their original indices in the symbol table so that
+the klp relocation sections can be applied correctly.
+
+As for the klp relocation sections, since we save Elf information for
+livepatch modules, we can simply iterate through module->info->sechdrs
+and check if the SHF_RELA_LIVEPATCH flag is set for a particular section.
-- 
2.4.3

[RFC PATCH v3 6/6] Documentation: livepatch: outline the Elf format of a livepatch module

[Jessica Yu]

Document the special Elf sections and constants livepatch modules use.

Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 Documentation/livepatch/patch-module-format.txt | 106 ++++++++++++++++++++++++
 1 file changed, 106 insertions(+)
 create mode 100644 Documentation/livepatch/patch-module-format.txt

diff --git a/Documentation/livepatch/patch-module-format.txt b/Documentation/livepatch/patch-module-format.txt
new file mode 100644
index 0000000..d825629
--- /dev/null
+++ b/Documentation/livepatch/patch-module-format.txt
@@ -0,0 +1,106 @@
+---------------------------
+Livepatch module Elf format
+---------------------------
+
+This document outlines the special Elf constants and sections livepatch
+uses to patch both modules and the kernel (vmlinux).
+
+--------------------------
+1. Livepatch modinfo field
+--------------------------
+
+Livepatch modules can be identified by users by using the 'modinfo' command
+and looking for the presence of the "livepatch" field. This field is also
+used by the kernel module loader to identify livepatch modules.
+
+Example modinfo output:
+
+% modinfo kpatch-meminfo.ko
+filename:		kpatch-meminfo.ko
+livepatch:		Y
+license:		GPL
+depends:
+vermagic:		4.3.0+ SMP mod_unload
+
+--------------------
+2. Livepatch symbols
+--------------------
+
+These are symbols marked with SHN_LIVEPATCH and their names are prefixed
+with the string ".klp.sym.${objname}.", where ${objname} is the name of the
+"object" where symbol stems from (the name of a module, for example).
+A symbol's position (used to differentiate duplicate symbols within the
+same object) in its object is encoded in the Elf_Sym st_other field
+and accessed with the KLP_SYMPOS macro (see include/linux/livepatch.h)
+
+Livepatch symbols are manually resolved by livepatch, and are used in cases
+where we cannot immediately know the address of a symbol because the
+to-be-patched module is not loaded yet. Livepatch modules keep these
+symbols in their symbol tables, and the symbol table is made accessible
+through module->core_symtab. For livepatch modules, core_symtab will
+contain an exact copy of the original symbol table as opposed to a stripped
+down version containing just the "core" symbols.
+
+-----------------------------------
+3. ".klp.rel." relocation sections
+-----------------------------------
+
+A livepatch module uses special Elf relocation sections to apply
+relocations both for regular vmlinux patches as well as those that should
+be applied as soon as the to-be-patched module is loaded. For example, if a
+patch module patches a driver that is not currently loaded, livepatch will
+apply its corresponding klp relocation section(s) to the driver once it
+loads.
+
+The names of these livepatch relocation sections are formatted
+".klp.rel.${objname}.", where ${objname} is the name of the "object" being
+patched (e.g. vmlinux or name of module). Each object within a patch module
+may have multiple klp sections (e.g. patches to multiple functions within
+the same object). There is a 1-1 correspondence between a klp relocation
+section and the target section (usually the text section for a function) to
+which the relocation(s) apply.
+
+Here's a sample readelf output for a livepatch module that patches vmlinux and
+modules 9p, btrfs, ext4:
+  ...
+  [29] .klp.rel.9p.text.caches.show RELA                    0000000000000000 002d58 0000c0 18 AIo 64   9  8
+  [30] .klp.rel.btrfs.text.btrfs.feature.attr.show RELA     0000000000000000 002e18 000060 18 AIo 64  11  8
+  ...
+  [34] .klp.rel.ext4.text.ext4.attr.store RELA              0000000000000000 002fd8 0000d8 18 AIo 64  13  8
+  [35] .klp.rel.ext4.text.ext4.attr.show RELA               0000000000000000 0030b0 000150 18 AIo 64  15  8
+  [36] .klp.rel.vmlinux.text.cmdline.proc.show RELA         0000000000000000 003200 000018 18 AIo 64  17  8
+  [37] .klp.rel.vmlinux.text.meminfo.proc.show RELA         0000000000000000 003218 0000f0 18 AIo 64  19  8
+  ...
+
+klp relocation sections are SHT_RELA sections but with a few special
+characteristics. Notice that they are marked SHF_ALLOC ("A") so that they
+will not be discarded when the module is loaded into memory, as well as
+with the SHF_RELA_LIVEPATCH flag ("o" - for OS-specific) so the module
+loader can identify them and avoid treating them as regular SHT_RELA
+sections, since they are manually managed by livepatch.
+
+Since Elf information is preserved for livepatch modules (see Section 4), a
+klp relocation section can be applied simply by passing in the appropriate
+section index to apply_relocate_add() (in the module loader code), which
+then uses it to access the relocation section and apply the relocations.
+
+--------------------------------------------------------
+4. How a livepatch module accesses its symbol table and
+its klp relocation sections
+--------------------------------------------------------
+
+The kernel module loader checks whether the module being loaded is a
+livepatch module. If so, it then makes a copy of the module's Elf header,
+section headers, section name string table, and some noteworthy section
+indices (for example, the symtab's section index). It adjusts the symtab's
+sh_addr to point to mod->core_symtab, since the original mod->symtab lies
+in init memory and gets freed once the module finishes initializing. For
+livepatch modules, the core_symtab will be an exact copy of its original
+symbol table (where normally, only "core" symbols are included in this
+symbol table. See is_core_symbol() in kernel/module.c). Livepatch requires
+that the symbols retain their original indices in the symbol table so that
+the klp relocation sections can be applied correctly.
+
+As for the klp relocation sections, since we save Elf information for
+livepatch modules, we can simply iterate through module->info->sechdrs
+and check if the SHF_RELA_LIVEPATCH flag is set for a particular section.
-- 
2.4.3

Re: [RFC PATCH v3 2/6] module: preserve Elf information for livepatch modules

[Rusty Russell]

Hi Jessica,

        Nice patch series.  Minor issues below, but they're really
just nits which can be patched afterwards if you're getting sick of
rebasing :)

> +#ifdef CONFIG_LIVEPATCH
> +/*
> + * copy_module_elf - preserve Elf information about a module
> + *
> + * Copy relevant Elf information from the load_info struct.
> + * Note: not all fields from the original load_info are
> + * copied into mod->info.

This makes me nervous, to have a struct which is half-initialized.

Better would be to have a separate type for this, eg:

struct livepatch_modinfo {
        Elf_Ehdr hdr;
        Elf_Shdr *sechdrs;
        char *secstrings;
        /* FIXME: Which of these do you need? */
        struct {
        	unsigned int sym, str, mod, vers, info, pcpu;
        } index;
};

This also avoids an extra allocation as hdr is no longer a ptr.

> +	/*
> +	 * Update symtab's sh_addr to point to a valid
> +	 * symbol table, as the temporary symtab in module
> +	 * init memory will be freed
> +	 */
> +	mod->info->sechdrs[mod->info->index.sym].sh_addr = (unsigned long)mod->core_symtab;

This comment is a bit misleading: it's actually pointing into the
temporary module copy, which will be discarded.  The init section is
slightly different...

> +#ifdef CONFIG_LIVEPATCH
> +static int check_livepatch_modinfo(struct module *mod, struct load_info *info)
> +{
> +	mod->klp = get_modinfo(info, "livepatch") ? true : false;
> +
> +	return 0;
> +}
> +#else
> +static int check_livepatch_modinfo(struct module *mod, struct load_info *info)
> +{
> +	if (get_modinfo(info, "livepatch"))
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +#endif

The term "check" implies no side-effects; I'd usually call this
int find_livepatch_modinfo(), and make sure you use the error code in
the caller:

> +	err = check_livepatch_modinfo(mod, info);
> +	if (err)
> +		return -ENOEXEC;
> +

Thanks,
Rusty.

Re: [RFC PATCH v3 2/6] module: preserve Elf information for livepatch modules

[Rusty Russell]

Hi Jessica,

        Nice patch series.  Minor issues below, but they're really
just nits which can be patched afterwards if you're getting sick of
rebasing :)

> +#ifdef CONFIG_LIVEPATCH
> +/*
> + * copy_module_elf - preserve Elf information about a module
> + *
> + * Copy relevant Elf information from the load_info struct.
> + * Note: not all fields from the original load_info are
> + * copied into mod->info.

This makes me nervous, to have a struct which is half-initialized.

Better would be to have a separate type for this, eg:

struct livepatch_modinfo {
        Elf_Ehdr hdr;
        Elf_Shdr *sechdrs;
        char *secstrings;
        /* FIXME: Which of these do you need? */
        struct {
        	unsigned int sym, str, mod, vers, info, pcpu;
        } index;
};

This also avoids an extra allocation as hdr is no longer a ptr.

> +	/*
> +	 * Update symtab's sh_addr to point to a valid
> +	 * symbol table, as the temporary symtab in module
> +	 * init memory will be freed
> +	 */
> +	mod->info->sechdrs[mod->info->index.sym].sh_addr = (unsigned long)mod->core_symtab;

This comment is a bit misleading: it's actually pointing into the
temporary module copy, which will be discarded.  The init section is
slightly different...

> +#ifdef CONFIG_LIVEPATCH
> +static int check_livepatch_modinfo(struct module *mod, struct load_info *info)
> +{
> +	mod->klp = get_modinfo(info, "livepatch") ? true : false;
> +
> +	return 0;
> +}
> +#else
> +static int check_livepatch_modinfo(struct module *mod, struct load_info *info)
> +{
> +	if (get_modinfo(info, "livepatch"))
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +#endif

The term "check" implies no side-effects; I'd usually call this
int find_livepatch_modinfo(), and make sure you use the error code in
the caller:

> +	err = check_livepatch_modinfo(mod, info);
> +	if (err)
> +		return -ENOEXEC;
> +

Thanks,
Rusty.

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Fri 2016-01-08 14:28:22, Jessica Yu wrote:
> Reuse module loader code to write relocations, thereby eliminating the need
> for architecture specific relocation code in livepatch. Namely, we reuse
> apply_relocate_add() in the module loader to write relocations instead of
> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> are resolved and then apply_relocate_add() is called to apply those
> relocations.
> 
> In addition, remove x86 livepatch relocation code. It is no longer needed
> since symbol resolution and relocation work have been offloaded to module
> loader.
> 
> --- a/kernel/livepatch/core.c
> +++ b/kernel/livepatch/core.c
> @@ -204,74 +207,70 @@ static int klp_find_object_symbol(const char *objname, const char *name,
>  	return -EINVAL;
>  }
>  
> -/*
> - * external symbols are located outside the parent object (where the parent
> - * object is either vmlinux or the kmod being patched).
> - */
> -static int klp_find_external_symbol(struct module *pmod, const char *name,
> -				    unsigned long *addr)
> +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
>  {
> -	const struct kernel_symbol *sym;
> +	int i, len, ret = 0;
> +	Elf_Rela *relas;
> +	Elf_Sym *sym;
> +	char *symname, *sym_objname;
>  
> -	/* first, check if it's an exported symbol */
> -	preempt_disable();
> -	sym = find_symbol(name, NULL, NULL, true, true);
> -	if (sym) {
> -		*addr = sym->value;
> -		preempt_enable();
> -		return 0;
> +	relas = (Elf_Rela *) relsec->sh_addr;
> +	/* For each rela in this .klp.rel. section */
> +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
> +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
> +		symname = pmod->core_strtab + sym->st_name;
> +
> +		len = strcspn(symname + KLP_TAG_LEN, ".");

We should check that len is non-zero. Otherwise, sym_objname might
be empty string and symname might overflow below.

Also we should check that symname really starts with .klp.sym. to
avoid invalid memory access.

> +		sym_objname = strncmp(symname + KLP_TAG_LEN, "vmlinux", len) ?
> +			kstrndup(symname + KLP_TAG_LEN, len, GFP_KERNEL) : NULL;
> +		/* .klp.sym.objname.[symbol_name] */
> +		symname += KLP_TAG_LEN + len + 1;
> +
> +		ret = klp_find_object_symbol(sym_objname, symname,
> +					     KLP_SYMPOS(sym->st_other),
> +					     (unsigned long *) &sym->st_value);
> +		kfree(sym_objname);
> +		if (ret)
> +			return ret;
>  	}
> -	preempt_enable();
>  
> -	/*
> -	 * Check if it's in another .o within the patch module. This also
> -	 * checks that the external symbol is unique.
> -	 */
> -	return klp_find_object_symbol(pmod->name, name, 0, addr);
> +	return ret;
>  }
>  
>  static int klp_write_object_relocations(struct module *pmod,
>  					struct klp_object *obj)
>  {
> -	int ret = 0;
> -	unsigned long val;
> -	struct klp_reloc *reloc;
> +	int i, len, ret = 0;
> +	char *secname;
> +	const char *objname;
>  
>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>  		return -EINVAL;
>  
> -	if (WARN_ON(!obj->relocs))
> -		return -EINVAL;
> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>  
>  	module_disable_ro(pmod);
> +	/* For each klp rela section for this object */
> +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
> +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
> +			continue;
> +
> +		secname = pmod->info->secstrings + pmod->info->sechdrs[i].sh_name;
> +		/* .klp.rel.[objname].section_name */
> +		len = strcspn(secname + KLP_TAG_LEN, ".");

We should check that secname realy starts with .klp.rel. string
to avoid access out of memmory.

Otherwise, nice work.

Thanks,
Petr.

> +
> +		if (strncmp(objname, secname + KLP_TAG_LEN, len))
> +			continue;
>  
> -	for (reloc = obj->relocs; reloc->name; reloc++) {
> -		/* discover the address of the referenced symbol */
> -		if (reloc->external) {
> -			if (reloc->sympos > 0) {
> -				pr_err("non-zero sympos for external reloc symbol '%s' is not supported\n",
> -				       reloc->name);
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			ret = klp_find_external_symbol(pmod, reloc->name, &val);
> -		} else
> -			ret = klp_find_object_symbol(obj->name,
> -						     reloc->name,
> -						     reloc->sympos,
> -						     &val);
> +		ret = klp_resolve_symbols(pmod->info->sechdrs + i, pmod);
>  		if (ret)
>  			goto out;
>  
> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
> -					     val + reloc->addend);
> -		if (ret) {
> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
> -			       reloc->name, val, ret);
> +		ret = apply_relocate_add(pmod->info->sechdrs, pmod->core_strtab,
> +					 pmod->info->index.sym, i, pmod);
> +		if (ret)
>  			goto out;
> -		}
>  	}
> -
>  out:
>  	module_enable_ro(pmod);
>  	return ret;

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Fri 2016-01-08 14:28:22, Jessica Yu wrote:
> Reuse module loader code to write relocations, thereby eliminating the need
> for architecture specific relocation code in livepatch. Namely, we reuse
> apply_relocate_add() in the module loader to write relocations instead of
> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> are resolved and then apply_relocate_add() is called to apply those
> relocations.
> 
> In addition, remove x86 livepatch relocation code. It is no longer needed
> since symbol resolution and relocation work have been offloaded to module
> loader.
> 
> --- a/kernel/livepatch/core.c
> +++ b/kernel/livepatch/core.c
> @@ -204,74 +207,70 @@ static int klp_find_object_symbol(const char *objname, const char *name,
>  	return -EINVAL;
>  }
>  
> -/*
> - * external symbols are located outside the parent object (where the parent
> - * object is either vmlinux or the kmod being patched).
> - */
> -static int klp_find_external_symbol(struct module *pmod, const char *name,
> -				    unsigned long *addr)
> +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
>  {
> -	const struct kernel_symbol *sym;
> +	int i, len, ret = 0;
> +	Elf_Rela *relas;
> +	Elf_Sym *sym;
> +	char *symname, *sym_objname;
>  
> -	/* first, check if it's an exported symbol */
> -	preempt_disable();
> -	sym = find_symbol(name, NULL, NULL, true, true);
> -	if (sym) {
> -		*addr = sym->value;
> -		preempt_enable();
> -		return 0;
> +	relas = (Elf_Rela *) relsec->sh_addr;
> +	/* For each rela in this .klp.rel. section */
> +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
> +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
> +		symname = pmod->core_strtab + sym->st_name;
> +
> +		len = strcspn(symname + KLP_TAG_LEN, ".");

We should check that len is non-zero. Otherwise, sym_objname might
be empty string and symname might overflow below.

Also we should check that symname really starts with .klp.sym. to
avoid invalid memory access.

> +		sym_objname = strncmp(symname + KLP_TAG_LEN, "vmlinux", len) ?
> +			kstrndup(symname + KLP_TAG_LEN, len, GFP_KERNEL) : NULL;
> +		/* .klp.sym.objname.[symbol_name] */
> +		symname += KLP_TAG_LEN + len + 1;
> +
> +		ret = klp_find_object_symbol(sym_objname, symname,
> +					     KLP_SYMPOS(sym->st_other),
> +					     (unsigned long *) &sym->st_value);
> +		kfree(sym_objname);
> +		if (ret)
> +			return ret;
>  	}
> -	preempt_enable();
>  
> -	/*
> -	 * Check if it's in another .o within the patch module. This also
> -	 * checks that the external symbol is unique.
> -	 */
> -	return klp_find_object_symbol(pmod->name, name, 0, addr);
> +	return ret;
>  }
>  
>  static int klp_write_object_relocations(struct module *pmod,
>  					struct klp_object *obj)
>  {
> -	int ret = 0;
> -	unsigned long val;
> -	struct klp_reloc *reloc;
> +	int i, len, ret = 0;
> +	char *secname;
> +	const char *objname;
>  
>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>  		return -EINVAL;
>  
> -	if (WARN_ON(!obj->relocs))
> -		return -EINVAL;
> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>  
>  	module_disable_ro(pmod);
> +	/* For each klp rela section for this object */
> +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
> +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
> +			continue;
> +
> +		secname = pmod->info->secstrings + pmod->info->sechdrs[i].sh_name;
> +		/* .klp.rel.[objname].section_name */
> +		len = strcspn(secname + KLP_TAG_LEN, ".");

We should check that secname realy starts with .klp.rel. string
to avoid access out of memmory.

Otherwise, nice work.

Thanks,
Petr.

> +
> +		if (strncmp(objname, secname + KLP_TAG_LEN, len))
> +			continue;
>  
> -	for (reloc = obj->relocs; reloc->name; reloc++) {
> -		/* discover the address of the referenced symbol */
> -		if (reloc->external) {
> -			if (reloc->sympos > 0) {
> -				pr_err("non-zero sympos for external reloc symbol '%s' is not supported\n",
> -				       reloc->name);
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			ret = klp_find_external_symbol(pmod, reloc->name, &val);
> -		} else
> -			ret = klp_find_object_symbol(obj->name,
> -						     reloc->name,
> -						     reloc->sympos,
> -						     &val);
> +		ret = klp_resolve_symbols(pmod->info->sechdrs + i, pmod);
>  		if (ret)
>  			goto out;
>  
> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
> -					     val + reloc->addend);
> -		if (ret) {
> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
> -			       reloc->name, val, ret);
> +		ret = apply_relocate_add(pmod->info->sechdrs, pmod->core_strtab,
> +					 pmod->info->index.sym, i, pmod);
> +		if (ret)
>  			goto out;
> -		}
>  	}
> -
>  out:
>  	module_enable_ro(pmod);
>  	return ret;

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Mon, Jan 11, 2016 at 05:56:13PM +0100, Petr Mladek wrote:
> On Fri 2016-01-08 14:28:22, Jessica Yu wrote:
> > Reuse module loader code to write relocations, thereby eliminating the need
> > for architecture specific relocation code in livepatch. Namely, we reuse
> > apply_relocate_add() in the module loader to write relocations instead of
> > duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> > relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> > are resolved and then apply_relocate_add() is called to apply those
> > relocations.
> > 
> > In addition, remove x86 livepatch relocation code. It is no longer needed
> > since symbol resolution and relocation work have been offloaded to module
> > loader.
> > 
> > --- a/kernel/livepatch/core.c
> > +++ b/kernel/livepatch/core.c
> > @@ -204,74 +207,70 @@ static int klp_find_object_symbol(const char *objname, const char *name,
> >  	return -EINVAL;
> >  }
> >  
> > -/*
> > - * external symbols are located outside the parent object (where the parent
> > - * object is either vmlinux or the kmod being patched).
> > - */
> > -static int klp_find_external_symbol(struct module *pmod, const char *name,
> > -				    unsigned long *addr)
> > +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
> >  {
> > -	const struct kernel_symbol *sym;
> > +	int i, len, ret = 0;
> > +	Elf_Rela *relas;
> > +	Elf_Sym *sym;
> > +	char *symname, *sym_objname;
> >  
> > -	/* first, check if it's an exported symbol */
> > -	preempt_disable();
> > -	sym = find_symbol(name, NULL, NULL, true, true);
> > -	if (sym) {
> > -		*addr = sym->value;
> > -		preempt_enable();
> > -		return 0;
> > +	relas = (Elf_Rela *) relsec->sh_addr;
> > +	/* For each rela in this .klp.rel. section */
> > +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
> > +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
> > +		symname = pmod->core_strtab + sym->st_name;
> > +
> > +		len = strcspn(symname + KLP_TAG_LEN, ".");
> 
> We should check that len is non-zero. Otherwise, sym_objname might
> be empty string and symname might overflow below.
> 
> Also we should check that symname really starts with .klp.sym. to
> avoid invalid memory access.

It would also be good to check for SHN_LIVEPATCH.

-- 
Josh

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
> Reuse module loader code to write relocations, thereby eliminating the need
> for architecture specific relocation code in livepatch. Namely, we reuse
> apply_relocate_add() in the module loader to write relocations instead of
> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> are resolved and then apply_relocate_add() is called to apply those
> relocations.
> 
> In addition, remove x86 livepatch relocation code. It is no longer needed
> since symbol resolution and relocation work have been offloaded to module
> loader.
> 
> Signed-off-by: Jessica Yu <jeyu@redhat.com>
> ---
>  arch/x86/include/asm/livepatch.h |   2 -
>  arch/x86/kernel/Makefile         |   1 -
>  arch/x86/kernel/livepatch.c      |  70 ---------------------------
>  include/linux/livepatch.h        |  33 +++++--------
>  kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
>  5 files changed, 62 insertions(+), 145 deletions(-)
>  delete mode 100644 arch/x86/kernel/livepatch.c
> 
> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> index 19c099a..7312e25 100644
> --- a/arch/x86/include/asm/livepatch.h
> +++ b/arch/x86/include/asm/livepatch.h
> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>  #endif
>  	return 0;
>  }
> -int klp_write_module_reloc(struct module *mod, unsigned long type,
> -			   unsigned long loc, unsigned long value);
>  
>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>  {
> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> index b1b78ff..c5e9a5c 100644
> --- a/arch/x86/kernel/Makefile
> +++ b/arch/x86/kernel/Makefile
> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>  obj-y				+= apic/
>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
> deleted file mode 100644
> index 92fc1a5..0000000
> --- a/arch/x86/kernel/livepatch.c
> +++ /dev/null
> @@ -1,70 +0,0 @@
> -/*
> - * livepatch.c - x86-specific Kernel Live Patching Core
> - *
> - * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
> - * Copyright (C) 2014 SUSE
> - *
> - * This program is free software; you can redistribute it and/or
> - * modify it under the terms of the GNU General Public License
> - * as published by the Free Software Foundation; either version 2
> - * of the License, or (at your option) any later version.
> - *
> - * This program is distributed in the hope that it will be useful,
> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> - * GNU General Public License for more details.
> - *
> - * You should have received a copy of the GNU General Public License
> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
> - */
> -
> -#include <linux/module.h>
> -#include <linux/uaccess.h>
> -#include <asm/elf.h>
> -#include <asm/livepatch.h>
> -
> -/**
> - * klp_write_module_reloc() - write a relocation in a module
> - * @mod:	module in which the section to be modified is found
> - * @type:	ELF relocation type (see asm/elf.h)
> - * @loc:	address that the relocation should be written to
> - * @value:	relocation value (sym address + addend)
> - *
> - * This function writes a relocation to the specified location for
> - * a particular module.
> - */
> -int klp_write_module_reloc(struct module *mod, unsigned long type,
> -			   unsigned long loc, unsigned long value)
> -{
> -	size_t size = 4;
> -	unsigned long val;
> -	unsigned long core = (unsigned long)mod->core_layout.base;
> -	unsigned long core_size = mod->core_layout.size;
> -
> -	switch (type) {
> -	case R_X86_64_NONE:
> -		return 0;
> -	case R_X86_64_64:
> -		val = value;
> -		size = 8;
> -		break;
> -	case R_X86_64_32:
> -		val = (u32)value;
> -		break;
> -	case R_X86_64_32S:
> -		val = (s32)value;
> -		break;
> -	case R_X86_64_PC32:
> -		val = (u32)(value - loc);
> -		break;
> -	default:
> -		/* unsupported relocation type */
> -		return -EINVAL;
> -	}
> -
> -	if (loc < core || loc >= core + core_size)
> -		/* loc does not point to any symbol inside the module */
> -		return -EINVAL;
> -
> -	return probe_kernel_write((void *)loc, &val, size);
> -}
> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> index a882865..2f12ce7 100644
> --- a/include/linux/livepatch.h
> +++ b/include/linux/livepatch.h
> @@ -65,27 +65,8 @@ struct klp_func {
>  };
>  
>  /**
> - * struct klp_reloc - relocation structure for live patching
> - * @loc:	address where the relocation will be written
> - * @sympos:	position in kallsyms to disambiguate symbols (optional)
> - * @type:	ELF relocation type
> - * @name:	name of the referenced symbol (for lookup/verification)
> - * @addend:	offset from the referenced symbol
> - * @external:	symbol is either exported or within the live patch module itself
> - */
> -struct klp_reloc {
> -	unsigned long loc;
> -	unsigned long sympos;
> -	unsigned long type;
> -	const char *name;
> -	int addend;
> -	int external;
> -};
> -
> -/**
>   * struct klp_object - kernel object structure for live patching
>   * @name:	module name (or NULL for vmlinux)
> - * @relocs:	relocation entries to be applied at load time
>   * @funcs:	function entries for functions to be patched in the object
>   * @kobj:	kobject for sysfs resources
>   * @mod:	kernel module associated with the patched object
> @@ -95,7 +76,6 @@ struct klp_reloc {
>  struct klp_object {
>  	/* external */
>  	const char *name;
> -	struct klp_reloc *relocs;
>  	struct klp_func *funcs;
>  
>  	/* internal */
> @@ -123,6 +103,19 @@ struct klp_patch {
>  	enum klp_state state;
>  };
>  
> +/*
> + * Livepatch-specific symbols and relocation
> + * sections are prefixed with a tag:
> + * .klp.rel. for relocation sections
> + * .klp.sym. for livepatch symbols
> + */
> +#define KLP_TAG_LEN 9
> +/*
> + * Livepatch-specific bits for specifying symbol
> + * positions in the Elf_Sym st_other field
> + */
> +#define KLP_SYMPOS(o) (o >> 2) & 0xff
> +

Can st_value be used instead?  I think we ended up deciding that would
be better:

  https://lkml.kernel.org/g/20151210213328.GA6553@packer-debian-8-amd64.digitalocean.com

Because:

- st_value is easily viewable in readelf
- st_other has some arch-specific uses

And another reason not previously discussed:

- st_other is an unsigned char, which limits sympos to values < 64

>  #define klp_for_each_object(patch, obj) \
>  	for (obj = patch->objs; obj->funcs; obj++)
>  
> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
> index bc2c85c..64536a4 100644
> --- a/kernel/livepatch/core.c
> +++ b/kernel/livepatch/core.c
> @@ -28,6 +28,9 @@
>  #include <linux/list.h>
>  #include <linux/kallsyms.h>
>  #include <linux/livepatch.h>
> +#include <linux/elf.h>
> +#include <linux/string.h>
> +#include <linux/moduleloader.h>
>  #include <asm/cacheflush.h>
>  
>  /**
> @@ -204,74 +207,70 @@ static int klp_find_object_symbol(const char *objname, const char *name,
>  	return -EINVAL;
>  }
>  
> -/*
> - * external symbols are located outside the parent object (where the parent
> - * object is either vmlinux or the kmod being patched).
> - */
> -static int klp_find_external_symbol(struct module *pmod, const char *name,
> -				    unsigned long *addr)
> +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
>  {
> -	const struct kernel_symbol *sym;
> +	int i, len, ret = 0;
> +	Elf_Rela *relas;
> +	Elf_Sym *sym;
> +	char *symname, *sym_objname;
>  
> -	/* first, check if it's an exported symbol */
> -	preempt_disable();
> -	sym = find_symbol(name, NULL, NULL, true, true);
> -	if (sym) {
> -		*addr = sym->value;
> -		preempt_enable();
> -		return 0;
> +	relas = (Elf_Rela *) relsec->sh_addr;
> +	/* For each rela in this .klp.rel. section */
> +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
> +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
> +		symname = pmod->core_strtab + sym->st_name;
> +
> +		len = strcspn(symname + KLP_TAG_LEN, ".");

It would be nice to have a comment here similar to the below one:

		/* .klp.sym.[objname].symbol_name */

> +		sym_objname = strncmp(symname + KLP_TAG_LEN, "vmlinux", len) ?

I think there's a bug here in the (unlikely) case where the module's
name is a subset of "vmlinux", e.g.  "vm".

> +			kstrndup(symname + KLP_TAG_LEN, len, GFP_KERNEL) : NULL;
> +		/* .klp.sym.objname.[symbol_name] */
> +		symname += KLP_TAG_LEN + len + 1;
> +
> +		ret = klp_find_object_symbol(sym_objname, symname,
> +					     KLP_SYMPOS(sym->st_other),
> +					     (unsigned long *) &sym->st_value);
> +		kfree(sym_objname);
> +		if (ret)
> +			return ret;
>  	}
> -	preempt_enable();
>  
> -	/*
> -	 * Check if it's in another .o within the patch module. This also
> -	 * checks that the external symbol is unique.
> -	 */
> -	return klp_find_object_symbol(pmod->name, name, 0, addr);
> +	return ret;
>  }
>  
>  static int klp_write_object_relocations(struct module *pmod,
>  					struct klp_object *obj)
>  {
> -	int ret = 0;
> -	unsigned long val;
> -	struct klp_reloc *reloc;
> +	int i, len, ret = 0;
> +	char *secname;
> +	const char *objname;
>  
>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>  		return -EINVAL;
>  
> -	if (WARN_ON(!obj->relocs))
> -		return -EINVAL;
> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>  
>  	module_disable_ro(pmod);
> +	/* For each klp rela section for this object */
> +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
> +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
> +			continue;
> +
> +		secname = pmod->info->secstrings + pmod->info->sechdrs[i].sh_name;
> +		/* .klp.rel.[objname].section_name */
> +		len = strcspn(secname + KLP_TAG_LEN, ".");
> +
> +		if (strncmp(objname, secname + KLP_TAG_LEN, len))
> +			continue;

Same problem here.  For ".klp.rel.foo.sym", an objname of "foobar" will
give a false match.

> -	for (reloc = obj->relocs; reloc->name; reloc++) {
> -		/* discover the address of the referenced symbol */
> -		if (reloc->external) {
> -			if (reloc->sympos > 0) {
> -				pr_err("non-zero sympos for external reloc symbol '%s' is not supported\n",
> -				       reloc->name);
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			ret = klp_find_external_symbol(pmod, reloc->name, &val);
> -		} else
> -			ret = klp_find_object_symbol(obj->name,
> -						     reloc->name,
> -						     reloc->sympos,
> -						     &val);
> +		ret = klp_resolve_symbols(pmod->info->sechdrs + i, pmod);
>  		if (ret)
>  			goto out;
>  
> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
> -					     val + reloc->addend);
> -		if (ret) {
> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
> -			       reloc->name, val, ret);
> +		ret = apply_relocate_add(pmod->info->sechdrs, pmod->core_strtab,
> +					 pmod->info->index.sym, i, pmod);
> +		if (ret)
>  			goto out;
> -		}
>  	}
> -
>  out:
>  	module_enable_ro(pmod);
>  	return ret;
> @@ -703,11 +702,9 @@ static int klp_init_object_loaded(struct klp_patch *patch,
>  	struct klp_func *func;
>  	int ret;
>  
> -	if (obj->relocs) {
> -		ret = klp_write_object_relocations(patch->mod, obj);
> -		if (ret)
> -			return ret;
> -	}
> +	ret = klp_write_object_relocations(patch->mod, obj);
> +	if (ret)
> +		return ret;
>  
>  	klp_for_each_func(obj, func) {
>  		ret = klp_find_object_symbol(obj->name, func->old_name,
> -- 
> 2.4.3
> 

-- 
Josh

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
> Reuse module loader code to write relocations, thereby eliminating the need
> for architecture specific relocation code in livepatch. Namely, we reuse
> apply_relocate_add() in the module loader to write relocations instead of
> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> are resolved and then apply_relocate_add() is called to apply those
> relocations.
> 
> In addition, remove x86 livepatch relocation code. It is no longer needed
> since symbol resolution and relocation work have been offloaded to module
> loader.
> 
> Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> ---
>  arch/x86/include/asm/livepatch.h |   2 -
>  arch/x86/kernel/Makefile         |   1 -
>  arch/x86/kernel/livepatch.c      |  70 ---------------------------
>  include/linux/livepatch.h        |  33 +++++--------
>  kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
>  5 files changed, 62 insertions(+), 145 deletions(-)
>  delete mode 100644 arch/x86/kernel/livepatch.c
> 
> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> index 19c099a..7312e25 100644
> --- a/arch/x86/include/asm/livepatch.h
> +++ b/arch/x86/include/asm/livepatch.h
> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>  #endif
>  	return 0;
>  }
> -int klp_write_module_reloc(struct module *mod, unsigned long type,
> -			   unsigned long loc, unsigned long value);
>  
>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>  {
> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> index b1b78ff..c5e9a5c 100644
> --- a/arch/x86/kernel/Makefile
> +++ b/arch/x86/kernel/Makefile
> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>  obj-y				+= apic/
>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
> deleted file mode 100644
> index 92fc1a5..0000000
> --- a/arch/x86/kernel/livepatch.c
> +++ /dev/null
> @@ -1,70 +0,0 @@
> -/*
> - * livepatch.c - x86-specific Kernel Live Patching Core
> - *
> - * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> - * Copyright (C) 2014 SUSE
> - *
> - * This program is free software; you can redistribute it and/or
> - * modify it under the terms of the GNU General Public License
> - * as published by the Free Software Foundation; either version 2
> - * of the License, or (at your option) any later version.
> - *
> - * This program is distributed in the hope that it will be useful,
> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> - * GNU General Public License for more details.
> - *
> - * You should have received a copy of the GNU General Public License
> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
> - */
> -
> -#include <linux/module.h>
> -#include <linux/uaccess.h>
> -#include <asm/elf.h>
> -#include <asm/livepatch.h>
> -
> -/**
> - * klp_write_module_reloc() - write a relocation in a module
> - * @mod:	module in which the section to be modified is found
> - * @type:	ELF relocation type (see asm/elf.h)
> - * @loc:	address that the relocation should be written to
> - * @value:	relocation value (sym address + addend)
> - *
> - * This function writes a relocation to the specified location for
> - * a particular module.
> - */
> -int klp_write_module_reloc(struct module *mod, unsigned long type,
> -			   unsigned long loc, unsigned long value)
> -{
> -	size_t size = 4;
> -	unsigned long val;
> -	unsigned long core = (unsigned long)mod->core_layout.base;
> -	unsigned long core_size = mod->core_layout.size;
> -
> -	switch (type) {
> -	case R_X86_64_NONE:
> -		return 0;
> -	case R_X86_64_64:
> -		val = value;
> -		size = 8;
> -		break;
> -	case R_X86_64_32:
> -		val = (u32)value;
> -		break;
> -	case R_X86_64_32S:
> -		val = (s32)value;
> -		break;
> -	case R_X86_64_PC32:
> -		val = (u32)(value - loc);
> -		break;
> -	default:
> -		/* unsupported relocation type */
> -		return -EINVAL;
> -	}
> -
> -	if (loc < core || loc >= core + core_size)
> -		/* loc does not point to any symbol inside the module */
> -		return -EINVAL;
> -
> -	return probe_kernel_write((void *)loc, &val, size);
> -}
> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> index a882865..2f12ce7 100644
> --- a/include/linux/livepatch.h
> +++ b/include/linux/livepatch.h
> @@ -65,27 +65,8 @@ struct klp_func {
>  };
>  
>  /**
> - * struct klp_reloc - relocation structure for live patching
> - * @loc:	address where the relocation will be written
> - * @sympos:	position in kallsyms to disambiguate symbols (optional)
> - * @type:	ELF relocation type
> - * @name:	name of the referenced symbol (for lookup/verification)
> - * @addend:	offset from the referenced symbol
> - * @external:	symbol is either exported or within the live patch module itself
> - */
> -struct klp_reloc {
> -	unsigned long loc;
> -	unsigned long sympos;
> -	unsigned long type;
> -	const char *name;
> -	int addend;
> -	int external;
> -};
> -
> -/**
>   * struct klp_object - kernel object structure for live patching
>   * @name:	module name (or NULL for vmlinux)
> - * @relocs:	relocation entries to be applied at load time
>   * @funcs:	function entries for functions to be patched in the object
>   * @kobj:	kobject for sysfs resources
>   * @mod:	kernel module associated with the patched object
> @@ -95,7 +76,6 @@ struct klp_reloc {
>  struct klp_object {
>  	/* external */
>  	const char *name;
> -	struct klp_reloc *relocs;
>  	struct klp_func *funcs;
>  
>  	/* internal */
> @@ -123,6 +103,19 @@ struct klp_patch {
>  	enum klp_state state;
>  };
>  
> +/*
> + * Livepatch-specific symbols and relocation
> + * sections are prefixed with a tag:
> + * .klp.rel. for relocation sections
> + * .klp.sym. for livepatch symbols
> + */
> +#define KLP_TAG_LEN 9
> +/*
> + * Livepatch-specific bits for specifying symbol
> + * positions in the Elf_Sym st_other field
> + */
> +#define KLP_SYMPOS(o) (o >> 2) & 0xff
> +

Can st_value be used instead?  I think we ended up deciding that would
be better:

  https://lkml.kernel.org/g/20151210213328.GA6553-N0bYjD2NfQ6k4hzjq3hgyGTy53QMssKEsp+A89P3RPuQWHG76I6BsA@public.gmane.org

Because:

- st_value is easily viewable in readelf
- st_other has some arch-specific uses

And another reason not previously discussed:

- st_other is an unsigned char, which limits sympos to values < 64

>  #define klp_for_each_object(patch, obj) \
>  	for (obj = patch->objs; obj->funcs; obj++)
>  
> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
> index bc2c85c..64536a4 100644
> --- a/kernel/livepatch/core.c
> +++ b/kernel/livepatch/core.c
> @@ -28,6 +28,9 @@
>  #include <linux/list.h>
>  #include <linux/kallsyms.h>
>  #include <linux/livepatch.h>
> +#include <linux/elf.h>
> +#include <linux/string.h>
> +#include <linux/moduleloader.h>
>  #include <asm/cacheflush.h>
>  
>  /**
> @@ -204,74 +207,70 @@ static int klp_find_object_symbol(const char *objname, const char *name,
>  	return -EINVAL;
>  }
>  
> -/*
> - * external symbols are located outside the parent object (where the parent
> - * object is either vmlinux or the kmod being patched).
> - */
> -static int klp_find_external_symbol(struct module *pmod, const char *name,
> -				    unsigned long *addr)
> +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
>  {
> -	const struct kernel_symbol *sym;
> +	int i, len, ret = 0;
> +	Elf_Rela *relas;
> +	Elf_Sym *sym;
> +	char *symname, *sym_objname;
>  
> -	/* first, check if it's an exported symbol */
> -	preempt_disable();
> -	sym = find_symbol(name, NULL, NULL, true, true);
> -	if (sym) {
> -		*addr = sym->value;
> -		preempt_enable();
> -		return 0;
> +	relas = (Elf_Rela *) relsec->sh_addr;
> +	/* For each rela in this .klp.rel. section */
> +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
> +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
> +		symname = pmod->core_strtab + sym->st_name;
> +
> +		len = strcspn(symname + KLP_TAG_LEN, ".");

It would be nice to have a comment here similar to the below one:

		/* .klp.sym.[objname].symbol_name */

> +		sym_objname = strncmp(symname + KLP_TAG_LEN, "vmlinux", len) ?

I think there's a bug here in the (unlikely) case where the module's
name is a subset of "vmlinux", e.g.  "vm".

> +			kstrndup(symname + KLP_TAG_LEN, len, GFP_KERNEL) : NULL;
> +		/* .klp.sym.objname.[symbol_name] */
> +		symname += KLP_TAG_LEN + len + 1;
> +
> +		ret = klp_find_object_symbol(sym_objname, symname,
> +					     KLP_SYMPOS(sym->st_other),
> +					     (unsigned long *) &sym->st_value);
> +		kfree(sym_objname);
> +		if (ret)
> +			return ret;
>  	}
> -	preempt_enable();
>  
> -	/*
> -	 * Check if it's in another .o within the patch module. This also
> -	 * checks that the external symbol is unique.
> -	 */
> -	return klp_find_object_symbol(pmod->name, name, 0, addr);
> +	return ret;
>  }
>  
>  static int klp_write_object_relocations(struct module *pmod,
>  					struct klp_object *obj)
>  {
> -	int ret = 0;
> -	unsigned long val;
> -	struct klp_reloc *reloc;
> +	int i, len, ret = 0;
> +	char *secname;
> +	const char *objname;
>  
>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>  		return -EINVAL;
>  
> -	if (WARN_ON(!obj->relocs))
> -		return -EINVAL;
> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>  
>  	module_disable_ro(pmod);
> +	/* For each klp rela section for this object */
> +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
> +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
> +			continue;
> +
> +		secname = pmod->info->secstrings + pmod->info->sechdrs[i].sh_name;
> +		/* .klp.rel.[objname].section_name */
> +		len = strcspn(secname + KLP_TAG_LEN, ".");
> +
> +		if (strncmp(objname, secname + KLP_TAG_LEN, len))
> +			continue;

Same problem here.  For ".klp.rel.foo.sym", an objname of "foobar" will
give a false match.

> -	for (reloc = obj->relocs; reloc->name; reloc++) {
> -		/* discover the address of the referenced symbol */
> -		if (reloc->external) {
> -			if (reloc->sympos > 0) {
> -				pr_err("non-zero sympos for external reloc symbol '%s' is not supported\n",
> -				       reloc->name);
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			ret = klp_find_external_symbol(pmod, reloc->name, &val);
> -		} else
> -			ret = klp_find_object_symbol(obj->name,
> -						     reloc->name,
> -						     reloc->sympos,
> -						     &val);
> +		ret = klp_resolve_symbols(pmod->info->sechdrs + i, pmod);
>  		if (ret)
>  			goto out;
>  
> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
> -					     val + reloc->addend);
> -		if (ret) {
> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
> -			       reloc->name, val, ret);
> +		ret = apply_relocate_add(pmod->info->sechdrs, pmod->core_strtab,
> +					 pmod->info->index.sym, i, pmod);
> +		if (ret)
>  			goto out;
> -		}
>  	}
> -
>  out:
>  	module_enable_ro(pmod);
>  	return ret;
> @@ -703,11 +702,9 @@ static int klp_init_object_loaded(struct klp_patch *patch,
>  	struct klp_func *func;
>  	int ret;
>  
> -	if (obj->relocs) {
> -		ret = klp_write_object_relocations(patch->mod, obj);
> -		if (ret)
> -			return ret;
> -	}
> +	ret = klp_write_object_relocations(patch->mod, obj);
> +	if (ret)
> +		return ret;
>  
>  	klp_for_each_func(obj, func) {
>  		ret = klp_find_object_symbol(obj->name, func->old_name,
> -- 
> 2.4.3
> 

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [11/01/16 15:33 -0600]:
>On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the need
>> for architecture specific relocation code in livepatch. Namely, we reuse
>> apply_relocate_add() in the module loader to write relocations instead of
>> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>> are resolved and then apply_relocate_add() is called to apply those
>> relocations.
>>
>> In addition, remove x86 livepatch relocation code. It is no longer needed
>> since symbol resolution and relocation work have been offloaded to module
>> loader.
>>
>> Signed-off-by: Jessica Yu <jeyu@redhat.com>
>> ---
>>  arch/x86/include/asm/livepatch.h |   2 -
>>  arch/x86/kernel/Makefile         |   1 -
>>  arch/x86/kernel/livepatch.c      |  70 ---------------------------
>>  include/linux/livepatch.h        |  33 +++++--------
>>  kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
>>  5 files changed, 62 insertions(+), 145 deletions(-)
>>  delete mode 100644 arch/x86/kernel/livepatch.c
>>
>> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> index 19c099a..7312e25 100644
>> --- a/arch/x86/include/asm/livepatch.h
>> +++ b/arch/x86/include/asm/livepatch.h
>> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>  #endif
>>  	return 0;
>>  }
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value);
>>
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> index b1b78ff..c5e9a5c 100644
>> --- a/arch/x86/kernel/Makefile
>> +++ b/arch/x86/kernel/Makefile
>> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>  obj-y				+= apic/
>>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> deleted file mode 100644
>> index 92fc1a5..0000000
>> --- a/arch/x86/kernel/livepatch.c
>> +++ /dev/null
>> @@ -1,70 +0,0 @@
>> -/*
>> - * livepatch.c - x86-specific Kernel Live Patching Core
>> - *
>> - * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
>> - * Copyright (C) 2014 SUSE
>> - *
>> - * This program is free software; you can redistribute it and/or
>> - * modify it under the terms of the GNU General Public License
>> - * as published by the Free Software Foundation; either version 2
>> - * of the License, or (at your option) any later version.
>> - *
>> - * This program is distributed in the hope that it will be useful,
>> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> - * GNU General Public License for more details.
>> - *
>> - * You should have received a copy of the GNU General Public License
>> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> - */
>> -
>> -#include <linux/module.h>
>> -#include <linux/uaccess.h>
>> -#include <asm/elf.h>
>> -#include <asm/livepatch.h>
>> -
>> -/**
>> - * klp_write_module_reloc() - write a relocation in a module
>> - * @mod:	module in which the section to be modified is found
>> - * @type:	ELF relocation type (see asm/elf.h)
>> - * @loc:	address that the relocation should be written to
>> - * @value:	relocation value (sym address + addend)
>> - *
>> - * This function writes a relocation to the specified location for
>> - * a particular module.
>> - */
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value)
>> -{
>> -	size_t size = 4;
>> -	unsigned long val;
>> -	unsigned long core = (unsigned long)mod->core_layout.base;
>> -	unsigned long core_size = mod->core_layout.size;
>> -
>> -	switch (type) {
>> -	case R_X86_64_NONE:
>> -		return 0;
>> -	case R_X86_64_64:
>> -		val = value;
>> -		size = 8;
>> -		break;
>> -	case R_X86_64_32:
>> -		val = (u32)value;
>> -		break;
>> -	case R_X86_64_32S:
>> -		val = (s32)value;
>> -		break;
>> -	case R_X86_64_PC32:
>> -		val = (u32)(value - loc);
>> -		break;
>> -	default:
>> -		/* unsupported relocation type */
>> -		return -EINVAL;
>> -	}
>> -
>> -	if (loc < core || loc >= core + core_size)
>> -		/* loc does not point to any symbol inside the module */
>> -		return -EINVAL;
>> -
>> -	return probe_kernel_write((void *)loc, &val, size);
>> -}
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index a882865..2f12ce7 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -65,27 +65,8 @@ struct klp_func {
>>  };
>>
>>  /**
>> - * struct klp_reloc - relocation structure for live patching
>> - * @loc:	address where the relocation will be written
>> - * @sympos:	position in kallsyms to disambiguate symbols (optional)
>> - * @type:	ELF relocation type
>> - * @name:	name of the referenced symbol (for lookup/verification)
>> - * @addend:	offset from the referenced symbol
>> - * @external:	symbol is either exported or within the live patch module itself
>> - */
>> -struct klp_reloc {
>> -	unsigned long loc;
>> -	unsigned long sympos;
>> -	unsigned long type;
>> -	const char *name;
>> -	int addend;
>> -	int external;
>> -};
>> -
>> -/**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +76,6 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>>  	struct klp_func *funcs;
>>
>>  	/* internal */
>> @@ -123,6 +103,19 @@ struct klp_patch {
>>  	enum klp_state state;
>>  };
>>
>> +/*
>> + * Livepatch-specific symbols and relocation
>> + * sections are prefixed with a tag:
>> + * .klp.rel. for relocation sections
>> + * .klp.sym. for livepatch symbols
>> + */
>> +#define KLP_TAG_LEN 9
>> +/*
>> + * Livepatch-specific bits for specifying symbol
>> + * positions in the Elf_Sym st_other field
>> + */
>> +#define KLP_SYMPOS(o) (o >> 2) & 0xff
>> +
>
>Can st_value be used instead?  I think we ended up deciding that would
>be better:
>
>  https://lkml.kernel.org/g/20151210213328.GA6553@packer-debian-8-amd64.digitalocean.com
>
>Because:
>
>- st_value is easily viewable in readelf
>- st_other has some arch-specific uses
>
>And another reason not previously discussed:
>
>- st_other is an unsigned char, which limits sympos to values < 64

I originally wanted to encode the symbol position in st_value, but
I've discovered that since st_value is overwritten once the
symbol is (first) resolved, we no longer have the symbol position if we
need to resolve the symbols again (so we wouldn't be able to resolve
them). This could happen when we patch a module that loads and
unloads more than once for example.

I chose st_other since it isn't touched in s390x kernel code nor in
x86 kernel code (it does get used in the x86 userspace reloc tool in
arch/x86/tools, which is why I left the first two bits alone for
ELF_ST_VISIBILITY, and the rest had undefined usage). However this
isn't the best approach and I'm afraid of stepping on other arch's
toes in future patches. Perhaps there is another field where we can
stuff the sympos in? st_size for instance doesn't seem to be touched
anywhere in the module loader. I don't know if I'd want to stuff
sympos in the sym name, there's enough going on there..

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [11/01/16 15:33 -0600]:
>On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the need
>> for architecture specific relocation code in livepatch. Namely, we reuse
>> apply_relocate_add() in the module loader to write relocations instead of
>> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>> are resolved and then apply_relocate_add() is called to apply those
>> relocations.
>>
>> In addition, remove x86 livepatch relocation code. It is no longer needed
>> since symbol resolution and relocation work have been offloaded to module
>> loader.
>>
>> Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> ---
>>  arch/x86/include/asm/livepatch.h |   2 -
>>  arch/x86/kernel/Makefile         |   1 -
>>  arch/x86/kernel/livepatch.c      |  70 ---------------------------
>>  include/linux/livepatch.h        |  33 +++++--------
>>  kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
>>  5 files changed, 62 insertions(+), 145 deletions(-)
>>  delete mode 100644 arch/x86/kernel/livepatch.c
>>
>> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> index 19c099a..7312e25 100644
>> --- a/arch/x86/include/asm/livepatch.h
>> +++ b/arch/x86/include/asm/livepatch.h
>> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>  #endif
>>  	return 0;
>>  }
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value);
>>
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> index b1b78ff..c5e9a5c 100644
>> --- a/arch/x86/kernel/Makefile
>> +++ b/arch/x86/kernel/Makefile
>> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>  obj-y				+= apic/
>>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> deleted file mode 100644
>> index 92fc1a5..0000000
>> --- a/arch/x86/kernel/livepatch.c
>> +++ /dev/null
>> @@ -1,70 +0,0 @@
>> -/*
>> - * livepatch.c - x86-specific Kernel Live Patching Core
>> - *
>> - * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> - * Copyright (C) 2014 SUSE
>> - *
>> - * This program is free software; you can redistribute it and/or
>> - * modify it under the terms of the GNU General Public License
>> - * as published by the Free Software Foundation; either version 2
>> - * of the License, or (at your option) any later version.
>> - *
>> - * This program is distributed in the hope that it will be useful,
>> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> - * GNU General Public License for more details.
>> - *
>> - * You should have received a copy of the GNU General Public License
>> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> - */
>> -
>> -#include <linux/module.h>
>> -#include <linux/uaccess.h>
>> -#include <asm/elf.h>
>> -#include <asm/livepatch.h>
>> -
>> -/**
>> - * klp_write_module_reloc() - write a relocation in a module
>> - * @mod:	module in which the section to be modified is found
>> - * @type:	ELF relocation type (see asm/elf.h)
>> - * @loc:	address that the relocation should be written to
>> - * @value:	relocation value (sym address + addend)
>> - *
>> - * This function writes a relocation to the specified location for
>> - * a particular module.
>> - */
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value)
>> -{
>> -	size_t size = 4;
>> -	unsigned long val;
>> -	unsigned long core = (unsigned long)mod->core_layout.base;
>> -	unsigned long core_size = mod->core_layout.size;
>> -
>> -	switch (type) {
>> -	case R_X86_64_NONE:
>> -		return 0;
>> -	case R_X86_64_64:
>> -		val = value;
>> -		size = 8;
>> -		break;
>> -	case R_X86_64_32:
>> -		val = (u32)value;
>> -		break;
>> -	case R_X86_64_32S:
>> -		val = (s32)value;
>> -		break;
>> -	case R_X86_64_PC32:
>> -		val = (u32)(value - loc);
>> -		break;
>> -	default:
>> -		/* unsupported relocation type */
>> -		return -EINVAL;
>> -	}
>> -
>> -	if (loc < core || loc >= core + core_size)
>> -		/* loc does not point to any symbol inside the module */
>> -		return -EINVAL;
>> -
>> -	return probe_kernel_write((void *)loc, &val, size);
>> -}
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index a882865..2f12ce7 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -65,27 +65,8 @@ struct klp_func {
>>  };
>>
>>  /**
>> - * struct klp_reloc - relocation structure for live patching
>> - * @loc:	address where the relocation will be written
>> - * @sympos:	position in kallsyms to disambiguate symbols (optional)
>> - * @type:	ELF relocation type
>> - * @name:	name of the referenced symbol (for lookup/verification)
>> - * @addend:	offset from the referenced symbol
>> - * @external:	symbol is either exported or within the live patch module itself
>> - */
>> -struct klp_reloc {
>> -	unsigned long loc;
>> -	unsigned long sympos;
>> -	unsigned long type;
>> -	const char *name;
>> -	int addend;
>> -	int external;
>> -};
>> -
>> -/**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +76,6 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>>  	struct klp_func *funcs;
>>
>>  	/* internal */
>> @@ -123,6 +103,19 @@ struct klp_patch {
>>  	enum klp_state state;
>>  };
>>
>> +/*
>> + * Livepatch-specific symbols and relocation
>> + * sections are prefixed with a tag:
>> + * .klp.rel. for relocation sections
>> + * .klp.sym. for livepatch symbols
>> + */
>> +#define KLP_TAG_LEN 9
>> +/*
>> + * Livepatch-specific bits for specifying symbol
>> + * positions in the Elf_Sym st_other field
>> + */
>> +#define KLP_SYMPOS(o) (o >> 2) & 0xff
>> +
>
>Can st_value be used instead?  I think we ended up deciding that would
>be better:
>
>  https://lkml.kernel.org/g/20151210213328.GA6553-N0bYjD2NfQ6k4hzjq3hgyGTy53QMssKEsp+A89P3RPuQWHG76I6BsA@public.gmane.org
>
>Because:
>
>- st_value is easily viewable in readelf
>- st_other has some arch-specific uses
>
>And another reason not previously discussed:
>
>- st_other is an unsigned char, which limits sympos to values < 64

I originally wanted to encode the symbol position in st_value, but
I've discovered that since st_value is overwritten once the
symbol is (first) resolved, we no longer have the symbol position if we
need to resolve the symbols again (so we wouldn't be able to resolve
them). This could happen when we patch a module that loads and
unloads more than once for example.

I chose st_other since it isn't touched in s390x kernel code nor in
x86 kernel code (it does get used in the x86 userspace reloc tool in
arch/x86/tools, which is why I left the first two bits alone for
ELF_ST_VISIBILITY, and the rest had undefined usage). However this
isn't the best approach and I'm afraid of stepping on other arch's
toes in future patches. Perhaps there is another field where we can
stuff the sympos in? st_size for instance doesn't seem to be touched
anywhere in the module loader. I don't know if I'd want to stuff
sympos in the sym name, there's enough going on there..

Jessica

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Mon, Jan 11, 2016 at 05:35:13PM -0500, Jessica Yu wrote:
> +++ Josh Poimboeuf [11/01/16 15:33 -0600]:
> >On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
> >>Reuse module loader code to write relocations, thereby eliminating the need
> >>for architecture specific relocation code in livepatch. Namely, we reuse
> >>apply_relocate_add() in the module loader to write relocations instead of
> >>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> >>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> >>are resolved and then apply_relocate_add() is called to apply those
> >>relocations.
> >>
> >>In addition, remove x86 livepatch relocation code. It is no longer needed
> >>since symbol resolution and relocation work have been offloaded to module
> >>loader.
> >>
> >>Signed-off-by: Jessica Yu <jeyu@redhat.com>
> >>---
> >> arch/x86/include/asm/livepatch.h |   2 -
> >> arch/x86/kernel/Makefile         |   1 -
> >> arch/x86/kernel/livepatch.c      |  70 ---------------------------
> >> include/linux/livepatch.h        |  33 +++++--------
> >> kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
> >> 5 files changed, 62 insertions(+), 145 deletions(-)
> >> delete mode 100644 arch/x86/kernel/livepatch.c
> >>
> >>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> >>index 19c099a..7312e25 100644
> >>--- a/arch/x86/include/asm/livepatch.h
> >>+++ b/arch/x86/include/asm/livepatch.h
> >>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
> >> #endif
> >> 	return 0;
> >> }
> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> >>-			   unsigned long loc, unsigned long value);
> >>
> >> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
> >> {
> >>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> >>index b1b78ff..c5e9a5c 100644
> >>--- a/arch/x86/kernel/Makefile
> >>+++ b/arch/x86/kernel/Makefile
> >>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
> >> obj-y				+= apic/
> >> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
> >> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
> >>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
> >> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
> >> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
> >> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
> >>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
> >>deleted file mode 100644
> >>index 92fc1a5..0000000
> >>--- a/arch/x86/kernel/livepatch.c
> >>+++ /dev/null
> >>@@ -1,70 +0,0 @@
> >>-/*
> >>- * livepatch.c - x86-specific Kernel Live Patching Core
> >>- *
> >>- * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
> >>- * Copyright (C) 2014 SUSE
> >>- *
> >>- * This program is free software; you can redistribute it and/or
> >>- * modify it under the terms of the GNU General Public License
> >>- * as published by the Free Software Foundation; either version 2
> >>- * of the License, or (at your option) any later version.
> >>- *
> >>- * This program is distributed in the hope that it will be useful,
> >>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> >>- * GNU General Public License for more details.
> >>- *
> >>- * You should have received a copy of the GNU General Public License
> >>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
> >>- */
> >>-
> >>-#include <linux/module.h>
> >>-#include <linux/uaccess.h>
> >>-#include <asm/elf.h>
> >>-#include <asm/livepatch.h>
> >>-
> >>-/**
> >>- * klp_write_module_reloc() - write a relocation in a module
> >>- * @mod:	module in which the section to be modified is found
> >>- * @type:	ELF relocation type (see asm/elf.h)
> >>- * @loc:	address that the relocation should be written to
> >>- * @value:	relocation value (sym address + addend)
> >>- *
> >>- * This function writes a relocation to the specified location for
> >>- * a particular module.
> >>- */
> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> >>-			   unsigned long loc, unsigned long value)
> >>-{
> >>-	size_t size = 4;
> >>-	unsigned long val;
> >>-	unsigned long core = (unsigned long)mod->core_layout.base;
> >>-	unsigned long core_size = mod->core_layout.size;
> >>-
> >>-	switch (type) {
> >>-	case R_X86_64_NONE:
> >>-		return 0;
> >>-	case R_X86_64_64:
> >>-		val = value;
> >>-		size = 8;
> >>-		break;
> >>-	case R_X86_64_32:
> >>-		val = (u32)value;
> >>-		break;
> >>-	case R_X86_64_32S:
> >>-		val = (s32)value;
> >>-		break;
> >>-	case R_X86_64_PC32:
> >>-		val = (u32)(value - loc);
> >>-		break;
> >>-	default:
> >>-		/* unsupported relocation type */
> >>-		return -EINVAL;
> >>-	}
> >>-
> >>-	if (loc < core || loc >= core + core_size)
> >>-		/* loc does not point to any symbol inside the module */
> >>-		return -EINVAL;
> >>-
> >>-	return probe_kernel_write((void *)loc, &val, size);
> >>-}
> >>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> >>index a882865..2f12ce7 100644
> >>--- a/include/linux/livepatch.h
> >>+++ b/include/linux/livepatch.h
> >>@@ -65,27 +65,8 @@ struct klp_func {
> >> };
> >>
> >> /**
> >>- * struct klp_reloc - relocation structure for live patching
> >>- * @loc:	address where the relocation will be written
> >>- * @sympos:	position in kallsyms to disambiguate symbols (optional)
> >>- * @type:	ELF relocation type
> >>- * @name:	name of the referenced symbol (for lookup/verification)
> >>- * @addend:	offset from the referenced symbol
> >>- * @external:	symbol is either exported or within the live patch module itself
> >>- */
> >>-struct klp_reloc {
> >>-	unsigned long loc;
> >>-	unsigned long sympos;
> >>-	unsigned long type;
> >>-	const char *name;
> >>-	int addend;
> >>-	int external;
> >>-};
> >>-
> >>-/**
> >>  * struct klp_object - kernel object structure for live patching
> >>  * @name:	module name (or NULL for vmlinux)
> >>- * @relocs:	relocation entries to be applied at load time
> >>  * @funcs:	function entries for functions to be patched in the object
> >>  * @kobj:	kobject for sysfs resources
> >>  * @mod:	kernel module associated with the patched object
> >>@@ -95,7 +76,6 @@ struct klp_reloc {
> >> struct klp_object {
> >> 	/* external */
> >> 	const char *name;
> >>-	struct klp_reloc *relocs;
> >> 	struct klp_func *funcs;
> >>
> >> 	/* internal */
> >>@@ -123,6 +103,19 @@ struct klp_patch {
> >> 	enum klp_state state;
> >> };
> >>
> >>+/*
> >>+ * Livepatch-specific symbols and relocation
> >>+ * sections are prefixed with a tag:
> >>+ * .klp.rel. for relocation sections
> >>+ * .klp.sym. for livepatch symbols
> >>+ */
> >>+#define KLP_TAG_LEN 9
> >>+/*
> >>+ * Livepatch-specific bits for specifying symbol
> >>+ * positions in the Elf_Sym st_other field
> >>+ */
> >>+#define KLP_SYMPOS(o) (o >> 2) & 0xff
> >>+
> >
> >Can st_value be used instead?  I think we ended up deciding that would
> >be better:
> >
> > https://lkml.kernel.org/g/20151210213328.GA6553@packer-debian-8-amd64.digitalocean.com
> >
> >Because:
> >
> >- st_value is easily viewable in readelf
> >- st_other has some arch-specific uses
> >
> >And another reason not previously discussed:
> >
> >- st_other is an unsigned char, which limits sympos to values < 64
> 
> I originally wanted to encode the symbol position in st_value, but
> I've discovered that since st_value is overwritten once the
> symbol is (first) resolved, we no longer have the symbol position if we
> need to resolve the symbols again (so we wouldn't be able to resolve
> them). This could happen when we patch a module that loads and
> unloads more than once for example.

Ah, good point.

> I chose st_other since it isn't touched in s390x kernel code nor in
> x86 kernel code (it does get used in the x86 userspace reloc tool in
> arch/x86/tools, which is why I left the first two bits alone for
> ELF_ST_VISIBILITY, and the rest had undefined usage). However this
> isn't the best approach and I'm afraid of stepping on other arch's
> toes in future patches. Perhaps there is another field where we can
> stuff the sympos in? st_size for instance doesn't seem to be touched
> anywhere in the module loader. I don't know if I'd want to stuff
> sympos in the sym name, there's enough going on there..

I think st_other still isn't going to work because of possible arch
conflicts and because of its small size (6 bits) otherwise.

st_size could work.  It doesn't _seem_ to be used by the module code,
though it's hard to confirm that for all arches.  But it makes me a
little nervous to override that field.  For example, might it confuse
some user-space tools out there, or the linker?

Maybe encoding it in the sym name is the safest bet.  Why not, we've
already got a bunch of other stuff there anyway :-)  One idea would be
to append it with ',<sympos>' to be consistent with the sysfs entries:

  .klp.sym.vmlinux.symname,1

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Mon, Jan 11, 2016 at 05:35:13PM -0500, Jessica Yu wrote:
> +++ Josh Poimboeuf [11/01/16 15:33 -0600]:
> >On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
> >>Reuse module loader code to write relocations, thereby eliminating the need
> >>for architecture specific relocation code in livepatch. Namely, we reuse
> >>apply_relocate_add() in the module loader to write relocations instead of
> >>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> >>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> >>are resolved and then apply_relocate_add() is called to apply those
> >>relocations.
> >>
> >>In addition, remove x86 livepatch relocation code. It is no longer needed
> >>since symbol resolution and relocation work have been offloaded to module
> >>loader.
> >>
> >>Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> >>---
> >> arch/x86/include/asm/livepatch.h |   2 -
> >> arch/x86/kernel/Makefile         |   1 -
> >> arch/x86/kernel/livepatch.c      |  70 ---------------------------
> >> include/linux/livepatch.h        |  33 +++++--------
> >> kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
> >> 5 files changed, 62 insertions(+), 145 deletions(-)
> >> delete mode 100644 arch/x86/kernel/livepatch.c
> >>
> >>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> >>index 19c099a..7312e25 100644
> >>--- a/arch/x86/include/asm/livepatch.h
> >>+++ b/arch/x86/include/asm/livepatch.h
> >>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
> >> #endif
> >> 	return 0;
> >> }
> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> >>-			   unsigned long loc, unsigned long value);
> >>
> >> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
> >> {
> >>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> >>index b1b78ff..c5e9a5c 100644
> >>--- a/arch/x86/kernel/Makefile
> >>+++ b/arch/x86/kernel/Makefile
> >>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
> >> obj-y				+= apic/
> >> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
> >> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
> >>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
> >> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
> >> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
> >> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
> >>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
> >>deleted file mode 100644
> >>index 92fc1a5..0000000
> >>--- a/arch/x86/kernel/livepatch.c
> >>+++ /dev/null
> >>@@ -1,70 +0,0 @@
> >>-/*
> >>- * livepatch.c - x86-specific Kernel Live Patching Core
> >>- *
> >>- * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> >>- * Copyright (C) 2014 SUSE
> >>- *
> >>- * This program is free software; you can redistribute it and/or
> >>- * modify it under the terms of the GNU General Public License
> >>- * as published by the Free Software Foundation; either version 2
> >>- * of the License, or (at your option) any later version.
> >>- *
> >>- * This program is distributed in the hope that it will be useful,
> >>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> >>- * GNU General Public License for more details.
> >>- *
> >>- * You should have received a copy of the GNU General Public License
> >>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
> >>- */
> >>-
> >>-#include <linux/module.h>
> >>-#include <linux/uaccess.h>
> >>-#include <asm/elf.h>
> >>-#include <asm/livepatch.h>
> >>-
> >>-/**
> >>- * klp_write_module_reloc() - write a relocation in a module
> >>- * @mod:	module in which the section to be modified is found
> >>- * @type:	ELF relocation type (see asm/elf.h)
> >>- * @loc:	address that the relocation should be written to
> >>- * @value:	relocation value (sym address + addend)
> >>- *
> >>- * This function writes a relocation to the specified location for
> >>- * a particular module.
> >>- */
> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> >>-			   unsigned long loc, unsigned long value)
> >>-{
> >>-	size_t size = 4;
> >>-	unsigned long val;
> >>-	unsigned long core = (unsigned long)mod->core_layout.base;
> >>-	unsigned long core_size = mod->core_layout.size;
> >>-
> >>-	switch (type) {
> >>-	case R_X86_64_NONE:
> >>-		return 0;
> >>-	case R_X86_64_64:
> >>-		val = value;
> >>-		size = 8;
> >>-		break;
> >>-	case R_X86_64_32:
> >>-		val = (u32)value;
> >>-		break;
> >>-	case R_X86_64_32S:
> >>-		val = (s32)value;
> >>-		break;
> >>-	case R_X86_64_PC32:
> >>-		val = (u32)(value - loc);
> >>-		break;
> >>-	default:
> >>-		/* unsupported relocation type */
> >>-		return -EINVAL;
> >>-	}
> >>-
> >>-	if (loc < core || loc >= core + core_size)
> >>-		/* loc does not point to any symbol inside the module */
> >>-		return -EINVAL;
> >>-
> >>-	return probe_kernel_write((void *)loc, &val, size);
> >>-}
> >>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> >>index a882865..2f12ce7 100644
> >>--- a/include/linux/livepatch.h
> >>+++ b/include/linux/livepatch.h
> >>@@ -65,27 +65,8 @@ struct klp_func {
> >> };
> >>
> >> /**
> >>- * struct klp_reloc - relocation structure for live patching
> >>- * @loc:	address where the relocation will be written
> >>- * @sympos:	position in kallsyms to disambiguate symbols (optional)
> >>- * @type:	ELF relocation type
> >>- * @name:	name of the referenced symbol (for lookup/verification)
> >>- * @addend:	offset from the referenced symbol
> >>- * @external:	symbol is either exported or within the live patch module itself
> >>- */
> >>-struct klp_reloc {
> >>-	unsigned long loc;
> >>-	unsigned long sympos;
> >>-	unsigned long type;
> >>-	const char *name;
> >>-	int addend;
> >>-	int external;
> >>-};
> >>-
> >>-/**
> >>  * struct klp_object - kernel object structure for live patching
> >>  * @name:	module name (or NULL for vmlinux)
> >>- * @relocs:	relocation entries to be applied at load time
> >>  * @funcs:	function entries for functions to be patched in the object
> >>  * @kobj:	kobject for sysfs resources
> >>  * @mod:	kernel module associated with the patched object
> >>@@ -95,7 +76,6 @@ struct klp_reloc {
> >> struct klp_object {
> >> 	/* external */
> >> 	const char *name;
> >>-	struct klp_reloc *relocs;
> >> 	struct klp_func *funcs;
> >>
> >> 	/* internal */
> >>@@ -123,6 +103,19 @@ struct klp_patch {
> >> 	enum klp_state state;
> >> };
> >>
> >>+/*
> >>+ * Livepatch-specific symbols and relocation
> >>+ * sections are prefixed with a tag:
> >>+ * .klp.rel. for relocation sections
> >>+ * .klp.sym. for livepatch symbols
> >>+ */
> >>+#define KLP_TAG_LEN 9
> >>+/*
> >>+ * Livepatch-specific bits for specifying symbol
> >>+ * positions in the Elf_Sym st_other field
> >>+ */
> >>+#define KLP_SYMPOS(o) (o >> 2) & 0xff
> >>+
> >
> >Can st_value be used instead?  I think we ended up deciding that would
> >be better:
> >
> > https://lkml.kernel.org/g/20151210213328.GA6553-N0bYjD2NfQ6k4hzjq3hgyGTy53QMssKEsp+A89P3RPuQWHG76I6BsA@public.gmane.org
> >
> >Because:
> >
> >- st_value is easily viewable in readelf
> >- st_other has some arch-specific uses
> >
> >And another reason not previously discussed:
> >
> >- st_other is an unsigned char, which limits sympos to values < 64
> 
> I originally wanted to encode the symbol position in st_value, but
> I've discovered that since st_value is overwritten once the
> symbol is (first) resolved, we no longer have the symbol position if we
> need to resolve the symbols again (so we wouldn't be able to resolve
> them). This could happen when we patch a module that loads and
> unloads more than once for example.

Ah, good point.

> I chose st_other since it isn't touched in s390x kernel code nor in
> x86 kernel code (it does get used in the x86 userspace reloc tool in
> arch/x86/tools, which is why I left the first two bits alone for
> ELF_ST_VISIBILITY, and the rest had undefined usage). However this
> isn't the best approach and I'm afraid of stepping on other arch's
> toes in future patches. Perhaps there is another field where we can
> stuff the sympos in? st_size for instance doesn't seem to be touched
> anywhere in the module loader. I don't know if I'd want to stuff
> sympos in the sym name, there's enough going on there..

I think st_other still isn't going to work because of possible arch
conflicts and because of its small size (6 bits) otherwise.

st_size could work.  It doesn't _seem_ to be used by the module code,
though it's hard to confirm that for all arches.  But it makes me a
little nervous to override that field.  For example, might it confuse
some user-space tools out there, or the linker?

Maybe encoding it in the sym name is the safest bet.  Why not, we've
already got a bunch of other stuff there anyway :-)  One idea would be
to append it with ',<sympos>' to be consistent with the sysfs entries:

  .klp.sym.vmlinux.symname,1

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Mon 2016-01-11 21:05:52, Josh Poimboeuf wrote:
> On Mon, Jan 11, 2016 at 05:35:13PM -0500, Jessica Yu wrote:
> > +++ Josh Poimboeuf [11/01/16 15:33 -0600]:
> > >On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
> > >>Reuse module loader code to write relocations, thereby eliminating the need
> > >>for architecture specific relocation code in livepatch. Namely, we reuse
> > >>apply_relocate_add() in the module loader to write relocations instead of
> > >>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> > >>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> > >>are resolved and then apply_relocate_add() is called to apply those
> > >>relocations.
> > >>
> > >>In addition, remove x86 livepatch relocation code. It is no longer needed
> > >>since symbol resolution and relocation work have been offloaded to module
> > >>loader.
> > >>
> > >>Signed-off-by: Jessica Yu <jeyu@redhat.com>
> > >>---
> > >> arch/x86/include/asm/livepatch.h |   2 -
> > >> arch/x86/kernel/Makefile         |   1 -
> > >> arch/x86/kernel/livepatch.c      |  70 ---------------------------
> > >> include/linux/livepatch.h        |  33 +++++--------
> > >> kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
> > >> 5 files changed, 62 insertions(+), 145 deletions(-)
> > >> delete mode 100644 arch/x86/kernel/livepatch.c
> > >>
> > >>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> > >>index 19c099a..7312e25 100644
> > >>--- a/arch/x86/include/asm/livepatch.h
> > >>+++ b/arch/x86/include/asm/livepatch.h
> > >>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
> > >> #endif
> > >> 	return 0;
> > >> }
> > >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> > >>-			   unsigned long loc, unsigned long value);
> > >>
> > >> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
> > >> {
> > >>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> > >>index b1b78ff..c5e9a5c 100644
> > >>--- a/arch/x86/kernel/Makefile
> > >>+++ b/arch/x86/kernel/Makefile
> > >>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
> > >> obj-y				+= apic/
> > >> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
> > >> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
> > >>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
> > >> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
> > >> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
> > >> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
> > >>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
> > >>deleted file mode 100644
> > >>index 92fc1a5..0000000
> > >>--- a/arch/x86/kernel/livepatch.c
> > >>+++ /dev/null
> > >>@@ -1,70 +0,0 @@
> > >>-/*
> > >>- * livepatch.c - x86-specific Kernel Live Patching Core
> > >>- *
> > >>- * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
> > >>- * Copyright (C) 2014 SUSE
> > >>- *
> > >>- * This program is free software; you can redistribute it and/or
> > >>- * modify it under the terms of the GNU General Public License
> > >>- * as published by the Free Software Foundation; either version 2
> > >>- * of the License, or (at your option) any later version.
> > >>- *
> > >>- * This program is distributed in the hope that it will be useful,
> > >>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > >>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > >>- * GNU General Public License for more details.
> > >>- *
> > >>- * You should have received a copy of the GNU General Public License
> > >>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > >>- */
> > >>-
> > >>-#include <linux/module.h>
> > >>-#include <linux/uaccess.h>
> > >>-#include <asm/elf.h>
> > >>-#include <asm/livepatch.h>
> > >>-
> > >>-/**
> > >>- * klp_write_module_reloc() - write a relocation in a module
> > >>- * @mod:	module in which the section to be modified is found
> > >>- * @type:	ELF relocation type (see asm/elf.h)
> > >>- * @loc:	address that the relocation should be written to
> > >>- * @value:	relocation value (sym address + addend)
> > >>- *
> > >>- * This function writes a relocation to the specified location for
> > >>- * a particular module.
> > >>- */
> > >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> > >>-			   unsigned long loc, unsigned long value)
> > >>-{
> > >>-	size_t size = 4;
> > >>-	unsigned long val;
> > >>-	unsigned long core = (unsigned long)mod->core_layout.base;
> > >>-	unsigned long core_size = mod->core_layout.size;
> > >>-
> > >>-	switch (type) {
> > >>-	case R_X86_64_NONE:
> > >>-		return 0;
> > >>-	case R_X86_64_64:
> > >>-		val = value;
> > >>-		size = 8;
> > >>-		break;
> > >>-	case R_X86_64_32:
> > >>-		val = (u32)value;
> > >>-		break;
> > >>-	case R_X86_64_32S:
> > >>-		val = (s32)value;
> > >>-		break;
> > >>-	case R_X86_64_PC32:
> > >>-		val = (u32)(value - loc);
> > >>-		break;
> > >>-	default:
> > >>-		/* unsupported relocation type */
> > >>-		return -EINVAL;
> > >>-	}
> > >>-
> > >>-	if (loc < core || loc >= core + core_size)
> > >>-		/* loc does not point to any symbol inside the module */
> > >>-		return -EINVAL;
> > >>-
> > >>-	return probe_kernel_write((void *)loc, &val, size);
> > >>-}
> > >>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> > >>index a882865..2f12ce7 100644
> > >>--- a/include/linux/livepatch.h
> > >>+++ b/include/linux/livepatch.h
> > >>@@ -65,27 +65,8 @@ struct klp_func {
> > >> };
> > >>
> > >> /**
> > >>- * struct klp_reloc - relocation structure for live patching
> > >>- * @loc:	address where the relocation will be written
> > >>- * @sympos:	position in kallsyms to disambiguate symbols (optional)
> > >>- * @type:	ELF relocation type
> > >>- * @name:	name of the referenced symbol (for lookup/verification)
> > >>- * @addend:	offset from the referenced symbol
> > >>- * @external:	symbol is either exported or within the live patch module itself
> > >>- */
> > >>-struct klp_reloc {
> > >>-	unsigned long loc;
> > >>-	unsigned long sympos;
> > >>-	unsigned long type;
> > >>-	const char *name;
> > >>-	int addend;
> > >>-	int external;
> > >>-};
> > >>-
> > >>-/**
> > >>  * struct klp_object - kernel object structure for live patching
> > >>  * @name:	module name (or NULL for vmlinux)
> > >>- * @relocs:	relocation entries to be applied at load time
> > >>  * @funcs:	function entries for functions to be patched in the object
> > >>  * @kobj:	kobject for sysfs resources
> > >>  * @mod:	kernel module associated with the patched object
> > >>@@ -95,7 +76,6 @@ struct klp_reloc {
> > >> struct klp_object {
> > >> 	/* external */
> > >> 	const char *name;
> > >>-	struct klp_reloc *relocs;
> > >> 	struct klp_func *funcs;
> > >>
> > >> 	/* internal */
> > >>@@ -123,6 +103,19 @@ struct klp_patch {
> > >> 	enum klp_state state;
> > >> };
> > >>
> > >>+/*
> > >>+ * Livepatch-specific symbols and relocation
> > >>+ * sections are prefixed with a tag:
> > >>+ * .klp.rel. for relocation sections
> > >>+ * .klp.sym. for livepatch symbols
> > >>+ */
> > >>+#define KLP_TAG_LEN 9
> > >>+/*
> > >>+ * Livepatch-specific bits for specifying symbol
> > >>+ * positions in the Elf_Sym st_other field
> > >>+ */
> > >>+#define KLP_SYMPOS(o) (o >> 2) & 0xff
> > >>+
> > >Can st_value be used instead?  I think we ended up deciding that would
> > >be better:
>
> Maybe encoding it in the sym name is the safest bet.  Why not, we've
> already got a bunch of other stuff there anyway :-)  One idea would be
> to append it with ',<sympos>' to be consistent with the sysfs entries:
>
>   .klp.sym.vmlinux.symname,1

This looks like the most safe solution to me as well.

Best Regards,
Petr

Re: livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Mon 2016-01-11 21:05:52, Josh Poimboeuf wrote:
> On Mon, Jan 11, 2016 at 05:35:13PM -0500, Jessica Yu wrote:
> > +++ Josh Poimboeuf [11/01/16 15:33 -0600]:
> > >On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
> > >>Reuse module loader code to write relocations, thereby eliminating the need
> > >>for architecture specific relocation code in livepatch. Namely, we reuse
> > >>apply_relocate_add() in the module loader to write relocations instead of
> > >>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
> > >>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
> > >>are resolved and then apply_relocate_add() is called to apply those
> > >>relocations.
> > >>
> > >>In addition, remove x86 livepatch relocation code. It is no longer needed
> > >>since symbol resolution and relocation work have been offloaded to module
> > >>loader.
> > >>
> > >>Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> > >>---
> > >> arch/x86/include/asm/livepatch.h |   2 -
> > >> arch/x86/kernel/Makefile         |   1 -
> > >> arch/x86/kernel/livepatch.c      |  70 ---------------------------
> > >> include/linux/livepatch.h        |  33 +++++--------
> > >> kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
> > >> 5 files changed, 62 insertions(+), 145 deletions(-)
> > >> delete mode 100644 arch/x86/kernel/livepatch.c
> > >>
> > >>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> > >>index 19c099a..7312e25 100644
> > >>--- a/arch/x86/include/asm/livepatch.h
> > >>+++ b/arch/x86/include/asm/livepatch.h
> > >>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
> > >> #endif
> > >> 	return 0;
> > >> }
> > >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> > >>-			   unsigned long loc, unsigned long value);
> > >>
> > >> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
> > >> {
> > >>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> > >>index b1b78ff..c5e9a5c 100644
> > >>--- a/arch/x86/kernel/Makefile
> > >>+++ b/arch/x86/kernel/Makefile
> > >>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
> > >> obj-y				+= apic/
> > >> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
> > >> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
> > >>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
> > >> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
> > >> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
> > >> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
> > >>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
> > >>deleted file mode 100644
> > >>index 92fc1a5..0000000
> > >>--- a/arch/x86/kernel/livepatch.c
> > >>+++ /dev/null
> > >>@@ -1,70 +0,0 @@
> > >>-/*
> > >>- * livepatch.c - x86-specific Kernel Live Patching Core
> > >>- *
> > >>- * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> > >>- * Copyright (C) 2014 SUSE
> > >>- *
> > >>- * This program is free software; you can redistribute it and/or
> > >>- * modify it under the terms of the GNU General Public License
> > >>- * as published by the Free Software Foundation; either version 2
> > >>- * of the License, or (at your option) any later version.
> > >>- *
> > >>- * This program is distributed in the hope that it will be useful,
> > >>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > >>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > >>- * GNU General Public License for more details.
> > >>- *
> > >>- * You should have received a copy of the GNU General Public License
> > >>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > >>- */
> > >>-
> > >>-#include <linux/module.h>
> > >>-#include <linux/uaccess.h>
> > >>-#include <asm/elf.h>
> > >>-#include <asm/livepatch.h>
> > >>-
> > >>-/**
> > >>- * klp_write_module_reloc() - write a relocation in a module
> > >>- * @mod:	module in which the section to be modified is found
> > >>- * @type:	ELF relocation type (see asm/elf.h)
> > >>- * @loc:	address that the relocation should be written to
> > >>- * @value:	relocation value (sym address + addend)
> > >>- *
> > >>- * This function writes a relocation to the specified location for
> > >>- * a particular module.
> > >>- */
> > >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
> > >>-			   unsigned long loc, unsigned long value)
> > >>-{
> > >>-	size_t size = 4;
> > >>-	unsigned long val;
> > >>-	unsigned long core = (unsigned long)mod->core_layout.base;
> > >>-	unsigned long core_size = mod->core_layout.size;
> > >>-
> > >>-	switch (type) {
> > >>-	case R_X86_64_NONE:
> > >>-		return 0;
> > >>-	case R_X86_64_64:
> > >>-		val = value;
> > >>-		size = 8;
> > >>-		break;
> > >>-	case R_X86_64_32:
> > >>-		val = (u32)value;
> > >>-		break;
> > >>-	case R_X86_64_32S:
> > >>-		val = (s32)value;
> > >>-		break;
> > >>-	case R_X86_64_PC32:
> > >>-		val = (u32)(value - loc);
> > >>-		break;
> > >>-	default:
> > >>-		/* unsupported relocation type */
> > >>-		return -EINVAL;
> > >>-	}
> > >>-
> > >>-	if (loc < core || loc >= core + core_size)
> > >>-		/* loc does not point to any symbol inside the module */
> > >>-		return -EINVAL;
> > >>-
> > >>-	return probe_kernel_write((void *)loc, &val, size);
> > >>-}
> > >>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
> > >>index a882865..2f12ce7 100644
> > >>--- a/include/linux/livepatch.h
> > >>+++ b/include/linux/livepatch.h
> > >>@@ -65,27 +65,8 @@ struct klp_func {
> > >> };
> > >>
> > >> /**
> > >>- * struct klp_reloc - relocation structure for live patching
> > >>- * @loc:	address where the relocation will be written
> > >>- * @sympos:	position in kallsyms to disambiguate symbols (optional)
> > >>- * @type:	ELF relocation type
> > >>- * @name:	name of the referenced symbol (for lookup/verification)
> > >>- * @addend:	offset from the referenced symbol
> > >>- * @external:	symbol is either exported or within the live patch module itself
> > >>- */
> > >>-struct klp_reloc {
> > >>-	unsigned long loc;
> > >>-	unsigned long sympos;
> > >>-	unsigned long type;
> > >>-	const char *name;
> > >>-	int addend;
> > >>-	int external;
> > >>-};
> > >>-
> > >>-/**
> > >>  * struct klp_object - kernel object structure for live patching
> > >>  * @name:	module name (or NULL for vmlinux)
> > >>- * @relocs:	relocation entries to be applied at load time
> > >>  * @funcs:	function entries for functions to be patched in the object
> > >>  * @kobj:	kobject for sysfs resources
> > >>  * @mod:	kernel module associated with the patched object
> > >>@@ -95,7 +76,6 @@ struct klp_reloc {
> > >> struct klp_object {
> > >> 	/* external */
> > >> 	const char *name;
> > >>-	struct klp_reloc *relocs;
> > >> 	struct klp_func *funcs;
> > >>
> > >> 	/* internal */
> > >>@@ -123,6 +103,19 @@ struct klp_patch {
> > >> 	enum klp_state state;
> > >> };
> > >>
> > >>+/*
> > >>+ * Livepatch-specific symbols and relocation
> > >>+ * sections are prefixed with a tag:
> > >>+ * .klp.rel. for relocation sections
> > >>+ * .klp.sym. for livepatch symbols
> > >>+ */
> > >>+#define KLP_TAG_LEN 9
> > >>+/*
> > >>+ * Livepatch-specific bits for specifying symbol
> > >>+ * positions in the Elf_Sym st_other field
> > >>+ */
> > >>+#define KLP_SYMPOS(o) (o >> 2) & 0xff
> > >>+
> > >Can st_value be used instead?  I think we ended up deciding that would
> > >be better:
>
> Maybe encoding it in the sym name is the safest bet.  Why not, we've
> already got a bunch of other stuff there anyway :-)  One idea would be
> to append it with ',<sympos>' to be consistent with the sysfs entries:
>
>   .klp.sym.vmlinux.symname,1

This looks like the most safe solution to me as well.

Best Regards,
Petr

Re: [RFC PATCH v3 6/6] Documentation: livepatch: outline the Elf format of a livepatch module

[Petr Mladek]

Hi Jessica,

first, thanks a lot for writing a documentation. It is really
appreciated!

To be honest, I am not sure if it makes sense to give feedback
at this stage. It seems that there still will be some changes
in the elf format.

On Fri 2016-01-08 14:28:24, Jessica Yu wrote:
> Document the special Elf sections and constants livepatch modules use.
> 
> Signed-off-by: Jessica Yu <jeyu@redhat.com>
> ---
>  Documentation/livepatch/patch-module-format.txt | 106 ++++++++++++++++++++++++
>  1 file changed, 106 insertions(+)
>  create mode 100644 Documentation/livepatch/patch-module-format.txt

I would call this symbol-relocation.txt or so. It describes only this
part of the patch format.

> 
> diff --git a/Documentation/livepatch/patch-module-format.txt b/Documentation/livepatch/patch-module-format.txt
> new file mode 100644
> index 0000000..d825629
> --- /dev/null
> +++ b/Documentation/livepatch/patch-module-format.txt
> @@ -0,0 +1,106 @@
> +---------------------------
> +Livepatch module Elf format
> +---------------------------

I would start with a description what symbols are relocated and why
it needs to be done a special way.

Also I would switch a bit the order of the sections below.
We have relocation sections for each patched object. They are
used when the object is loaded and patched. Each section has
relocation information for symbols that are accessed from
new versions of functions for the patched object.
Hmm, we should add some generic documentation about
LivePatching that would describe the design and used terms.


> +This document outlines the special Elf constants and sections livepatch
> +uses to patch both modules and the kernel (vmlinux).

> +--------------------------
> +1. Livepatch modinfo field
> +--------------------------
> +

Please, express that livepatch must set the "livepatch" field to be
indentified and loaded correctly.

> +Livepatch modules can be identified by users by using the 'modinfo' command
> +and looking for the presence of the "livepatch" field. This field is also
> +used by the kernel module loader to identify livepatch modules.
> +
> +Example modinfo output:
> +
> +% modinfo kpatch-meminfo.ko

s/kpatch/livepatch/ ;-)

> +filename:		kpatch-meminfo.ko

same here ;-)

> +livepatch:		Y
> +license:		GPL
> +depends:
> +vermagic:		4.3.0+ SMP mod_unload
> +
> +--------------------
> +2. Livepatch symbols
> +--------------------

I would be more precise here. These are symbols that will get
relocated by the livepatch framework.

> +
> +These are symbols marked with SHN_LIVEPATCH and their names are prefixed
> +with the string ".klp.sym.${objname}.", where ${objname} is the

We should describe the entire format. I mean
.klp.sym.${objname}.[symname]. Also we should explain why symname is optional.

> name of the
> +"object" where symbol stems from (the name of a module, for example).
> +A symbol's position (used to differentiate duplicate symbols within the
> +same object) in its object is encoded in the Elf_Sym st_other field
> +and accessed with the KLP_SYMPOS macro (see include/linux/livepatch.h)
> +
> +Livepatch symbols are manually resolved by livepatch, and are used in cases

Please, what do you mean by "manually" resolved?

> +where we cannot immediately know the address of a symbol because the
> +to-be-patched module is not loaded yet. Livepatch modules keep these
> +symbols in their symbol tables, and the symbol table is made accessible
> +through module->core_symtab. For livepatch modules, core_symtab will
> +contain an exact copy of the original symbol table as opposed to a stripped
> +down version containing just the "core" symbols.
> +
> +-----------------------------------
> +3. ".klp.rel." relocation sections
> +-----------------------------------
> +
> +A livepatch module uses special Elf relocation sections to apply
> +relocations both for regular vmlinux patches as well as those that should
> +be applied as soon as the to-be-patched module is loaded. For example, if a
> +patch module patches a driver that is not currently loaded, livepatch will
> +apply its corresponding klp relocation section(s) to the driver once it
> +loads.
> +
> +The names of these livepatch relocation sections are formatted
> +".klp.rel.${objname}.", where ${objname} is the name of the
> "object" being

It seems that the full format is ".klp.rel.[objname].section_name"
and section_name is not explained here.


> +patched (e.g. vmlinux or name of module). Each object within a patch module
> +may have multiple klp sections (e.g. patches to multiple functions within
> +the same object). There is a 1-1 correspondence between a klp relocation
> +section and the target section (usually the text section for a function) to
> +which the relocation(s) apply.
> +
> +Here's a sample readelf output for a livepatch module that patches vmlinux and
> +modules 9p, btrfs, ext4:
> +  ...
> +  [29] .klp.rel.9p.text.caches.show RELA                    0000000000000000 002d58 0000c0 18 AIo 64   9  8
> +  [30] .klp.rel.btrfs.text.btrfs.feature.attr.show RELA     0000000000000000 002e18 000060 18 AIo 64  11  8
> +  ...
> +  [34] .klp.rel.ext4.text.ext4.attr.store RELA              0000000000000000 002fd8 0000d8 18 AIo 64  13  8
> +  [35] .klp.rel.ext4.text.ext4.attr.show RELA               0000000000000000 0030b0 000150 18 AIo 64  15  8
> +  [36] .klp.rel.vmlinux.text.cmdline.proc.show RELA         0000000000000000 003200 000018 18 AIo 64  17  8
> +  [37] .klp.rel.vmlinux.text.meminfo.proc.show RELA         0000000000000000 003218 0000f0 18 AIo 64  19  8
> +  ...
> +
> +klp relocation sections are SHT_RELA sections but with a few special
> +characteristics. Notice that they are marked SHF_ALLOC ("A") so that they
> +will not be discarded when the module is loaded into memory, as well as
> +with the SHF_RELA_LIVEPATCH flag ("o" - for OS-specific) so the module

Hmm, it is SHF_RELA... I wonder if we want to rather use
.klp.rela. prefix to be consistent.

> +loader can identify them and avoid treating them as regular SHT_RELA
> +sections, since they are manually managed by livepatch.
> +
> +Since Elf information is preserved for livepatch modules (see Section 4), a
> +klp relocation section can be applied simply by passing in the appropriate
> +section index to apply_relocate_add() (in the module loader code), which
> +then uses it to access the relocation section and apply the relocations.
> +
> +--------------------------------------------------------
> +4. How a livepatch module accesses its symbol table and
> +its klp relocation sections
> +--------------------------------------------------------
> +
> +The kernel module loader checks whether the module being loaded is a
> +livepatch module. If so, it then makes a copy of the module's Elf header,
> +section headers, section name string table, and some noteworthy section
> +indices (for example, the symtab's section index). It adjusts the symtab's
> +sh_addr to point to mod->core_symtab, since the original mod->symtab lies
> +in init memory and gets freed once the module finishes initializing. For
> +livepatch modules, the core_symtab will be an exact copy of its original
> +symbol table (where normally, only "core" symbols are included in this
> +symbol table. See is_core_symbol() in kernel/module.c). Livepatch requires
> +that the symbols retain their original indices in the symbol table so that
> +the klp relocation sections can be applied correctly.

We should add some notice also to the source code or commit message
aboud why we preserve all the symbols for live patch.

Great work.

Thanks,
Petr

Re: [RFC PATCH v3 6/6] Documentation: livepatch: outline the Elf format of a livepatch module

[Josh Poimboeuf]

On Tue, Jan 12, 2016 at 01:09:51PM +0100, Petr Mladek wrote:
> Hi Jessica,
> 
> first, thanks a lot for writing a documentation. It is really
> appreciated!
> 
> To be honest, I am not sure if it makes sense to give feedback
> at this stage. It seems that there still will be some changes
> in the elf format.
> 
> On Fri 2016-01-08 14:28:24, Jessica Yu wrote:
> > Document the special Elf sections and constants livepatch modules use.
> > 
> > Signed-off-by: Jessica Yu <jeyu@redhat.com>
> > ---
> >  Documentation/livepatch/patch-module-format.txt | 106 ++++++++++++++++++++++++
> >  1 file changed, 106 insertions(+)
> >  create mode 100644 Documentation/livepatch/patch-module-format.txt
> 
> I would call this symbol-relocation.txt or so. It describes only this
> part of the patch format.
> 
> > 
> > diff --git a/Documentation/livepatch/patch-module-format.txt b/Documentation/livepatch/patch-module-format.txt
> > new file mode 100644
> > index 0000000..d825629
> > --- /dev/null
> > +++ b/Documentation/livepatch/patch-module-format.txt
> > @@ -0,0 +1,106 @@
> > +---------------------------
> > +Livepatch module Elf format
> > +---------------------------
> 
> I would start with a description what symbols are relocated and why
> it needs to be done a special way.

The cover letter has this information, and a lot of other useful
background.  It would be good to add all that background to this
document as an introduction.

-- 
Josh

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Miroslav Benes]

Hi Jessica,

I walked through the series and it looks really nice. Others have already 
pointed out the issues I also found, so only few minor things below.

First thing, could you copy&paste the information and reasoning from the 
cover letter to the changelogs where appropriate? It is very detailed and 
it would be a pity to lost it.

On Fri, 8 Jan 2016, Jessica Yu wrote:

> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> index 19c099a..7312e25 100644
> --- a/arch/x86/include/asm/livepatch.h
> +++ b/arch/x86/include/asm/livepatch.h
> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>  #endif
>  	return 0;
>  }
> -int klp_write_module_reloc(struct module *mod, unsigned long type,
> -			   unsigned long loc, unsigned long value);

You left klp_write_module_reloc() in arch/s390/include/asm/livepatch.h I'm 
afraid. Anyway it would be really great if you managed to test the series 
on s390 somehow. Just to know that all the roadblocks are really gone.

> -/*
> - * external symbols are located outside the parent object (where the parent
> - * object is either vmlinux or the kmod being patched).
> - */
> -static int klp_find_external_symbol(struct module *pmod, const char *name,
> -				    unsigned long *addr)
> +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
>  {
> -	const struct kernel_symbol *sym;
> +	int i, len, ret = 0;
> +	Elf_Rela *relas;
> +	Elf_Sym *sym;
> +	char *symname, *sym_objname;
>  
> -	/* first, check if it's an exported symbol */
> -	preempt_disable();
> -	sym = find_symbol(name, NULL, NULL, true, true);
> -	if (sym) {
> -		*addr = sym->value;
> -		preempt_enable();
> -		return 0;
> +	relas = (Elf_Rela *) relsec->sh_addr;
> +	/* For each rela in this .klp.rel. section */
> +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
> +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
> +		symname = pmod->core_strtab + sym->st_name;

Maybe it would be better to use pmod->symtab and pmod->strtab everywhere. 
It should be the same, but core_* versions are only helpers used in 
load_module and friends. There is even a comment in 
include/linux/module.h.

	/*
	 * We keep the symbol and string tables for kallsyms.
	 * The core_* fields below are temporary, loader-only (they
	 * could really be discarded after module init).
	 */

We should respect that.

Thanks,
Miroslav

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Miroslav Benes]

Hi Jessica,

I walked through the series and it looks really nice. Others have already 
pointed out the issues I also found, so only few minor things below.

First thing, could you copy&paste the information and reasoning from the 
cover letter to the changelogs where appropriate? It is very detailed and 
it would be a pity to lost it.

On Fri, 8 Jan 2016, Jessica Yu wrote:

> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
> index 19c099a..7312e25 100644
> --- a/arch/x86/include/asm/livepatch.h
> +++ b/arch/x86/include/asm/livepatch.h
> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>  #endif
>  	return 0;
>  }
> -int klp_write_module_reloc(struct module *mod, unsigned long type,
> -			   unsigned long loc, unsigned long value);

You left klp_write_module_reloc() in arch/s390/include/asm/livepatch.h I'm 
afraid. Anyway it would be really great if you managed to test the series 
on s390 somehow. Just to know that all the roadblocks are really gone.

> -/*
> - * external symbols are located outside the parent object (where the parent
> - * object is either vmlinux or the kmod being patched).
> - */
> -static int klp_find_external_symbol(struct module *pmod, const char *name,
> -				    unsigned long *addr)
> +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
>  {
> -	const struct kernel_symbol *sym;
> +	int i, len, ret = 0;
> +	Elf_Rela *relas;
> +	Elf_Sym *sym;
> +	char *symname, *sym_objname;
>  
> -	/* first, check if it's an exported symbol */
> -	preempt_disable();
> -	sym = find_symbol(name, NULL, NULL, true, true);
> -	if (sym) {
> -		*addr = sym->value;
> -		preempt_enable();
> -		return 0;
> +	relas = (Elf_Rela *) relsec->sh_addr;
> +	/* For each rela in this .klp.rel. section */
> +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
> +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
> +		symname = pmod->core_strtab + sym->st_name;

Maybe it would be better to use pmod->symtab and pmod->strtab everywhere. 
It should be the same, but core_* versions are only helpers used in 
load_module and friends. There is even a comment in 
include/linux/module.h.

	/*
	 * We keep the symbol and string tables for kallsyms.
	 * The core_* fields below are temporary, loader-only (they
	 * could really be discarded after module init).
	 */

We should respect that.

Thanks,
Miroslav

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Miroslav Benes]

On Fri, 8 Jan 2016, Jessica Yu wrote:

>  static int klp_write_object_relocations(struct module *pmod,
>  					struct klp_object *obj)
>  {
> -	int ret = 0;
> -	unsigned long val;
> -	struct klp_reloc *reloc;
> +	int i, len, ret = 0;
> +	char *secname;
> +	const char *objname;
>  
>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>  		return -EINVAL;
>  
> -	if (WARN_ON(!obj->relocs))
> -		return -EINVAL;
> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>  
>  	module_disable_ro(pmod);
> +	/* For each klp rela section for this object */
> +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
> +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
> +			continue;

One more thing. If the module does not specify it is a live patch module 
in modinfo (with MODULE_INFO(livepatch, "Y")), but it is a perfect live 
patch module otherwise (it calls klp_register_patch in its init function), 
the kernel crashes here. pmod->info is not initialized at all. This should 
be fixed. Perhaps the easiest would be to call 
klp_write_object_relocations() in klp_init_object_loaded() only if 
is_livepatch_module() returns true. Similar to a check for obj->relocs 
before.

Miroslav

Re: [RFC PATCH v3 4/6] livepatch: reuse module loader code to write relocations

[Jiri Kosina]

On Wed, 13 Jan 2016, Miroslav Benes wrote:

> >  {
> > -	int ret = 0;
> > -	unsigned long val;
> > -	struct klp_reloc *reloc;
> > +	int i, len, ret = 0;
> > +	char *secname;
> > +	const char *objname;
> >  
> >  	if (WARN_ON(!klp_is_object_loaded(obj)))
> >  		return -EINVAL;
> >  
> > -	if (WARN_ON(!obj->relocs))
> > -		return -EINVAL;
> > +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
> >  
> >  	module_disable_ro(pmod);
> > +	/* For each klp rela section for this object */
> > +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
> > +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
> > +			continue;
> 
> One more thing. If the module does not specify it is a live patch module 
> in modinfo (with MODULE_INFO(livepatch, "Y")), but it is a perfect live 
> patch module otherwise (it calls klp_register_patch in its init function), 

Side note: I think we should at least issue some light warning in such 
case anyway.

-- 
Jiri Kosina
SUSE Labs

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [13/01/16 10:19 +0100]:
>On Fri, 8 Jan 2016, Jessica Yu wrote:
>
>>  static int klp_write_object_relocations(struct module *pmod,
>>  					struct klp_object *obj)
>>  {
>> -	int ret = 0;
>> -	unsigned long val;
>> -	struct klp_reloc *reloc;
>> +	int i, len, ret = 0;
>> +	char *secname;
>> +	const char *objname;
>>
>>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>  		return -EINVAL;
>>
>> -	if (WARN_ON(!obj->relocs))
>> -		return -EINVAL;
>> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>>
>>  	module_disable_ro(pmod);
>> +	/* For each klp rela section for this object */
>> +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
>> +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
>> +			continue;
>
>One more thing. If the module does not specify it is a live patch module
>in modinfo (with MODULE_INFO(livepatch, "Y")), but it is a perfect live
>patch module otherwise (it calls klp_register_patch in its init function),
>the kernel crashes here. pmod->info is not initialized at all. This should
>be fixed. Perhaps the easiest would be to call
>klp_write_object_relocations() in klp_init_object_loaded() only if
>is_livepatch_module() returns true. Similar to a check for obj->relocs
>before.

Hm yes, that's a problem. To remedy this, I think it makes sense to
require all livepatch modules to identify themselves with the modinfo
attribute, since it is a very simple requirement. If some module calls
klp_register_patch() and it does not have the livepatch attribute,
klp_register_patch() can just return an error. We can call
is_livepatch_module() at the beginning of klp_register_patch(), and
proceed only if the check succeeds, since we'll then know that the
required structures have been properly initialized in the module
loader. What do you think?

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [13/01/16 10:19 +0100]:
>On Fri, 8 Jan 2016, Jessica Yu wrote:
>
>>  static int klp_write_object_relocations(struct module *pmod,
>>  					struct klp_object *obj)
>>  {
>> -	int ret = 0;
>> -	unsigned long val;
>> -	struct klp_reloc *reloc;
>> +	int i, len, ret = 0;
>> +	char *secname;
>> +	const char *objname;
>>
>>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>  		return -EINVAL;
>>
>> -	if (WARN_ON(!obj->relocs))
>> -		return -EINVAL;
>> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>>
>>  	module_disable_ro(pmod);
>> +	/* For each klp rela section for this object */
>> +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
>> +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
>> +			continue;
>
>One more thing. If the module does not specify it is a live patch module
>in modinfo (with MODULE_INFO(livepatch, "Y")), but it is a perfect live
>patch module otherwise (it calls klp_register_patch in its init function),
>the kernel crashes here. pmod->info is not initialized at all. This should
>be fixed. Perhaps the easiest would be to call
>klp_write_object_relocations() in klp_init_object_loaded() only if
>is_livepatch_module() returns true. Similar to a check for obj->relocs
>before.

Hm yes, that's a problem. To remedy this, I think it makes sense to
require all livepatch modules to identify themselves with the modinfo
attribute, since it is a very simple requirement. If some module calls
klp_register_patch() and it does not have the livepatch attribute,
klp_register_patch() can just return an error. We can call
is_livepatch_module() at the beginning of klp_register_patch(), and
proceed only if the check succeeds, since we'll then know that the
required structures have been properly initialized in the module
loader. What do you think?

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [12/01/16 17:40 +0100]:
>
>Hi Jessica,
>
>I walked through the series and it looks really nice. Others have already
>pointed out the issues I also found, so only few minor things below.
>
>First thing, could you copy&paste the information and reasoning from the
>cover letter to the changelogs where appropriate? It is very detailed and
>it would be a pity to lost it.

Thanks Miroslav! I'll do that.

>On Fri, 8 Jan 2016, Jessica Yu wrote:
>
>> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> index 19c099a..7312e25 100644
>> --- a/arch/x86/include/asm/livepatch.h
>> +++ b/arch/x86/include/asm/livepatch.h
>> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>  #endif
>>  	return 0;
>>  }
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value);
>
>You left klp_write_module_reloc() in arch/s390/include/asm/livepatch.h I'm
>afraid. Anyway it would be really great if you managed to test the series
>on s390 somehow. Just to know that all the roadblocks are really gone.

Ah, thanks for catching that. I will also try testing the patchset on
s390x and report back.

>> -/*
>> - * external symbols are located outside the parent object (where the parent
>> - * object is either vmlinux or the kmod being patched).
>> - */
>> -static int klp_find_external_symbol(struct module *pmod, const char *name,
>> -				    unsigned long *addr)
>> +static int klp_resolve_symbols(Elf_Shdr *relsec, struct module *pmod)
>>  {
>> -	const struct kernel_symbol *sym;
>> +	int i, len, ret = 0;
>> +	Elf_Rela *relas;
>> +	Elf_Sym *sym;
>> +	char *symname, *sym_objname;
>>
>> -	/* first, check if it's an exported symbol */
>> -	preempt_disable();
>> -	sym = find_symbol(name, NULL, NULL, true, true);
>> -	if (sym) {
>> -		*addr = sym->value;
>> -		preempt_enable();
>> -		return 0;
>> +	relas = (Elf_Rela *) relsec->sh_addr;
>> +	/* For each rela in this .klp.rel. section */
>> +	for (i = 0; i < relsec->sh_size / sizeof(Elf_Rela); i++) {
>> +		sym = pmod->core_symtab + ELF_R_SYM(relas[i].r_info);
>> +		symname = pmod->core_strtab + sym->st_name;
>
>Maybe it would be better to use pmod->symtab and pmod->strtab everywhere.
>It should be the same, but core_* versions are only helpers used in
>load_module and friends. There is even a comment in
>include/linux/module.h.
>
>	/*
>	 * We keep the symbol and string tables for kallsyms.
>	 * The core_* fields below are temporary, loader-only (they
>	 * could really be discarded after module init).
>	 */
>
>We should respect that.

I admit I'm a bit confused by the comment, I can't seem to find where
core_symtab and core_strtab are purportedly discarded after module
init (perhaps I'm missing something?). IMO it sounds more like it's
describing mod->symtab and mod->strtab instead, because these are in
module init memory and are freed later. In any case, my reason for using
core_symtab is that the original symbol table (mod->symtab) is marked
with INIT_OFFSET_MASK in layout_symtab() (see kernel/module.c), and is
therefore in init memory. This memory is freed near the end of
do_init_module() with do_free_init(). Since core_symtab is in module core
memory, for livepatch modules I simply used core_symtab to hold a
full copy of the symbol table instead of the slimmed down version that
it was originally intended to hold.

Alternatively, we can tweak layout_symtab() to *not* mark the symtab
with INIT_OFFSET_MASK and put it in core memory instead. I think
either way will work, but maybe it is cleaner to do it this way
instead.

Jessica

Re: module: preserve Elf information for livepatch modules

[Jessica Yu]

+++ Rusty Russell [11/01/16 11:55 +1030]:
>Hi Jessica,
>
>        Nice patch series.  Minor issues below, but they're really
>just nits which can be patched afterwards if you're getting sick of
>rebasing :)

Thanks Rusty!

>> +#ifdef CONFIG_LIVEPATCH
>> +/*
>> + * copy_module_elf - preserve Elf information about a module
>> + *
>> + * Copy relevant Elf information from the load_info struct.
>> + * Note: not all fields from the original load_info are
>> + * copied into mod->info.
>
>This makes me nervous, to have a struct which is half-initialized.
>
>Better would be to have a separate type for this, eg:
>
>struct livepatch_modinfo {
>        Elf_Ehdr hdr;
>        Elf_Shdr *sechdrs;
>        char *secstrings;
>        /* FIXME: Which of these do you need? */
>        struct {
>        	unsigned int sym, str, mod, vers, info, pcpu;
>        } index;
>};
>
>This also avoids an extra allocation as hdr is no longer a ptr.

Sure, sounds good.

>> +	/*
>> +	 * Update symtab's sh_addr to point to a valid
>> +	 * symbol table, as the temporary symtab in module
>> +	 * init memory will be freed
>> +	 */
>> +	mod->info->sechdrs[mod->info->index.sym].sh_addr = (unsigned long)mod->core_symtab;
>
>This comment is a bit misleading: it's actually pointing into the
>temporary module copy, which will be discarded.  The init section is
>slightly different...

Ah, perhaps I'm misunderstanding something..

Since copy_module_elf() is called after move_module(), my
understanding is that all the section sh_addr's should be pointing
to either module core memory or module init memory (instead of the
initial temporary copy of the module in info->hdr). Since the symbol
table section is marked with INIT_OFFSET_MASK, it will reside in
module init memory (and freed near the end of do_init_module()),
which is why I am updating the sh_addr here to point to core_symtab
instead. For livepatch modules, the core_symtab will be a complete
copy of the symbol table instead of the slimmed down version. Please
correct me if my understanding is incorrect.

Thanks for the patch review,
Jessica

Re: Documentation: livepatch: outline the Elf format of a livepatch module

[Jessica Yu]

+++ Petr Mladek [12/01/16 13:09 +0100]:
>Hi Jessica,
>
>first, thanks a lot for writing a documentation. It is really
>appreciated!
>
>To be honest, I am not sure if it makes sense to give feedback
>at this stage. It seems that there still will be some changes
>in the elf format.

Your feedback is appreciated regardless! :-)

>On Fri 2016-01-08 14:28:24, Jessica Yu wrote:
>> Document the special Elf sections and constants livepatch modules use.
>>
>> Signed-off-by: Jessica Yu <jeyu@redhat.com>
>> ---
>>  Documentation/livepatch/patch-module-format.txt | 106 ++++++++++++++++++++++++
>>  1 file changed, 106 insertions(+)
>>  create mode 100644 Documentation/livepatch/patch-module-format.txt
>
>I would call this symbol-relocation.txt or so. It describes only this
>part of the patch format.
>
>>
>> diff --git a/Documentation/livepatch/patch-module-format.txt b/Documentation/livepatch/patch-module-format.txt
>> new file mode 100644
>> index 0000000..d825629
>> --- /dev/null
>> +++ b/Documentation/livepatch/patch-module-format.txt
>> @@ -0,0 +1,106 @@
>> +---------------------------
>> +Livepatch module Elf format
>> +---------------------------
>
>I would start with a description what symbols are relocated and why
>it needs to be done a special way.
>
>Also I would switch a bit the order of the sections below.
>We have relocation sections for each patched object. They are
>used when the object is loaded and patched. Each section has
>relocation information for symbols that are accessed from
>new versions of functions for the patched object.
>Hmm, we should add some generic documentation about
>LivePatching that would describe the design and used terms.
>
>
>> +This document outlines the special Elf constants and sections livepatch
>> +uses to patch both modules and the kernel (vmlinux).
>
>> +--------------------------
>> +1. Livepatch modinfo field
>> +--------------------------
>> +
>
>Please, express that livepatch must set the "livepatch" field to be
>indentified and loaded correctly.
>
>> +Livepatch modules can be identified by users by using the 'modinfo' command
>> +and looking for the presence of the "livepatch" field. This field is also
>> +used by the kernel module loader to identify livepatch modules.
>> +
>> +Example modinfo output:
>> +
>> +% modinfo kpatch-meminfo.ko
>
>s/kpatch/livepatch/ ;-)
>
>> +filename:		kpatch-meminfo.ko
>
>same here ;-)
>
>> +livepatch:		Y
>> +license:		GPL
>> +depends:
>> +vermagic:		4.3.0+ SMP mod_unload
>> +
>> +--------------------
>> +2. Livepatch symbols
>> +--------------------
>
>I would be more precise here. These are symbols that will get
>relocated by the livepatch framework.
>
>> +
>> +These are symbols marked with SHN_LIVEPATCH and their names are prefixed
>> +with the string ".klp.sym.${objname}.", where ${objname} is the
>
>We should describe the entire format. I mean
>.klp.sym.${objname}.[symname]. Also we should explain why symname is optional.
>
>> name of the
>> +"object" where symbol stems from (the name of a module, for example).
>> +A symbol's position (used to differentiate duplicate symbols within the
>> +same object) in its object is encoded in the Elf_Sym st_other field
>> +and accessed with the KLP_SYMPOS macro (see include/linux/livepatch.h)
>> +
>> +Livepatch symbols are manually resolved by livepatch, and are used in cases
>
>Please, what do you mean by "manually" resolved?

I think I was trying to emphasize that livepatch will take care of
symbol resolution (i.e. using klp_find_object_symbol()), but it
is sufficient enough to just say "livepatch resolves SHN_LIVEPATCH
symbols."

>> +where we cannot immediately know the address of a symbol because the
>> +to-be-patched module is not loaded yet. Livepatch modules keep these
>> +symbols in their symbol tables, and the symbol table is made accessible
>> +through module->core_symtab. For livepatch modules, core_symtab will
>> +contain an exact copy of the original symbol table as opposed to a stripped
>> +down version containing just the "core" symbols.
>> +
>> +-----------------------------------
>> +3. ".klp.rel." relocation sections
>> +-----------------------------------
>> +
>> +A livepatch module uses special Elf relocation sections to apply
>> +relocations both for regular vmlinux patches as well as those that should
>> +be applied as soon as the to-be-patched module is loaded. For example, if a
>> +patch module patches a driver that is not currently loaded, livepatch will
>> +apply its corresponding klp relocation section(s) to the driver once it
>> +loads.
>> +
>> +The names of these livepatch relocation sections are formatted
>> +".klp.rel.${objname}.", where ${objname} is the name of the
>> "object" being
>
>It seems that the full format is ".klp.rel.[objname].section_name"
>and section_name is not explained here.

Ah, thanks for catching that. I meant to say that they are *prefixed*
by the string ".klp.rel.[objname].", but like you said, it is much
better to explain the full format ".klp.rel.[objname].section_name"
here.

>
>> +patched (e.g. vmlinux or name of module). Each object within a patch module
>> +may have multiple klp sections (e.g. patches to multiple functions within
>> +the same object). There is a 1-1 correspondence between a klp relocation
>> +section and the target section (usually the text section for a function) to
>> +which the relocation(s) apply.
>> +
>> +Here's a sample readelf output for a livepatch module that patches vmlinux and
>> +modules 9p, btrfs, ext4:
>> +  ...
>> +  [29] .klp.rel.9p.text.caches.show RELA                    0000000000000000 002d58 0000c0 18 AIo 64   9  8
>> +  [30] .klp.rel.btrfs.text.btrfs.feature.attr.show RELA     0000000000000000 002e18 000060 18 AIo 64  11  8
>> +  ...
>> +  [34] .klp.rel.ext4.text.ext4.attr.store RELA              0000000000000000 002fd8 0000d8 18 AIo 64  13  8
>> +  [35] .klp.rel.ext4.text.ext4.attr.show RELA               0000000000000000 0030b0 000150 18 AIo 64  15  8
>> +  [36] .klp.rel.vmlinux.text.cmdline.proc.show RELA         0000000000000000 003200 000018 18 AIo 64  17  8
>> +  [37] .klp.rel.vmlinux.text.meminfo.proc.show RELA         0000000000000000 003218 0000f0 18 AIo 64  19  8
>> +  ...
>> +
>> +klp relocation sections are SHT_RELA sections but with a few special
>> +characteristics. Notice that they are marked SHF_ALLOC ("A") so that they
>> +will not be discarded when the module is loaded into memory, as well as
>> +with the SHF_RELA_LIVEPATCH flag ("o" - for OS-specific) so the module
>
>Hmm, it is SHF_RELA... I wonder if we want to rather use
>.klp.rela. prefix to be consistent.

Yeah, that looks better. I was actually just trying to make the
prefixes ".klp.sym." and ".klp.rel." the same length so the string
parsing code would be slightly simpler using KLP_TAG_LEN.

>> +loader can identify them and avoid treating them as regular SHT_RELA
>> +sections, since they are manually managed by livepatch.
>> +
>> +Since Elf information is preserved for livepatch modules (see Section 4), a
>> +klp relocation section can be applied simply by passing in the appropriate
>> +section index to apply_relocate_add() (in the module loader code), which
>> +then uses it to access the relocation section and apply the relocations.
>> +
>> +--------------------------------------------------------
>> +4. How a livepatch module accesses its symbol table and
>> +its klp relocation sections
>> +--------------------------------------------------------
>> +
>> +The kernel module loader checks whether the module being loaded is a
>> +livepatch module. If so, it then makes a copy of the module's Elf header,
>> +section headers, section name string table, and some noteworthy section
>> +indices (for example, the symtab's section index). It adjusts the symtab's
>> +sh_addr to point to mod->core_symtab, since the original mod->symtab lies
>> +in init memory and gets freed once the module finishes initializing. For
>> +livepatch modules, the core_symtab will be an exact copy of its original
>> +symbol table (where normally, only "core" symbols are included in this
>> +symbol table. See is_core_symbol() in kernel/module.c). Livepatch requires
>> +that the symbols retain their original indices in the symbol table so that
>> +the klp relocation sections can be applied correctly.
>
>We should add some notice also to the source code or commit message
>aboud why we preserve all the symbols for live patch.
>
>Great work.

Thanks for the review Petr!

Jessica

Re: Documentation: livepatch: outline the Elf format of a livepatch module

[Jessica Yu]

+++ Petr Mladek [12/01/16 13:09 +0100]:
>Hi Jessica,
>
>first, thanks a lot for writing a documentation. It is really
>appreciated!
>
>To be honest, I am not sure if it makes sense to give feedback
>at this stage. It seems that there still will be some changes
>in the elf format.

Your feedback is appreciated regardless! :-)

>On Fri 2016-01-08 14:28:24, Jessica Yu wrote:
>> Document the special Elf sections and constants livepatch modules use.
>>
>> Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> ---
>>  Documentation/livepatch/patch-module-format.txt | 106 ++++++++++++++++++++++++
>>  1 file changed, 106 insertions(+)
>>  create mode 100644 Documentation/livepatch/patch-module-format.txt
>
>I would call this symbol-relocation.txt or so. It describes only this
>part of the patch format.
>
>>
>> diff --git a/Documentation/livepatch/patch-module-format.txt b/Documentation/livepatch/patch-module-format.txt
>> new file mode 100644
>> index 0000000..d825629
>> --- /dev/null
>> +++ b/Documentation/livepatch/patch-module-format.txt
>> @@ -0,0 +1,106 @@
>> +---------------------------
>> +Livepatch module Elf format
>> +---------------------------
>
>I would start with a description what symbols are relocated and why
>it needs to be done a special way.
>
>Also I would switch a bit the order of the sections below.
>We have relocation sections for each patched object. They are
>used when the object is loaded and patched. Each section has
>relocation information for symbols that are accessed from
>new versions of functions for the patched object.
>Hmm, we should add some generic documentation about
>LivePatching that would describe the design and used terms.
>
>
>> +This document outlines the special Elf constants and sections livepatch
>> +uses to patch both modules and the kernel (vmlinux).
>
>> +--------------------------
>> +1. Livepatch modinfo field
>> +--------------------------
>> +
>
>Please, express that livepatch must set the "livepatch" field to be
>indentified and loaded correctly.
>
>> +Livepatch modules can be identified by users by using the 'modinfo' command
>> +and looking for the presence of the "livepatch" field. This field is also
>> +used by the kernel module loader to identify livepatch modules.
>> +
>> +Example modinfo output:
>> +
>> +% modinfo kpatch-meminfo.ko
>
>s/kpatch/livepatch/ ;-)
>
>> +filename:		kpatch-meminfo.ko
>
>same here ;-)
>
>> +livepatch:		Y
>> +license:		GPL
>> +depends:
>> +vermagic:		4.3.0+ SMP mod_unload
>> +
>> +--------------------
>> +2. Livepatch symbols
>> +--------------------
>
>I would be more precise here. These are symbols that will get
>relocated by the livepatch framework.
>
>> +
>> +These are symbols marked with SHN_LIVEPATCH and their names are prefixed
>> +with the string ".klp.sym.${objname}.", where ${objname} is the
>
>We should describe the entire format. I mean
>.klp.sym.${objname}.[symname]. Also we should explain why symname is optional.
>
>> name of the
>> +"object" where symbol stems from (the name of a module, for example).
>> +A symbol's position (used to differentiate duplicate symbols within the
>> +same object) in its object is encoded in the Elf_Sym st_other field
>> +and accessed with the KLP_SYMPOS macro (see include/linux/livepatch.h)
>> +
>> +Livepatch symbols are manually resolved by livepatch, and are used in cases
>
>Please, what do you mean by "manually" resolved?

I think I was trying to emphasize that livepatch will take care of
symbol resolution (i.e. using klp_find_object_symbol()), but it
is sufficient enough to just say "livepatch resolves SHN_LIVEPATCH
symbols."

>> +where we cannot immediately know the address of a symbol because the
>> +to-be-patched module is not loaded yet. Livepatch modules keep these
>> +symbols in their symbol tables, and the symbol table is made accessible
>> +through module->core_symtab. For livepatch modules, core_symtab will
>> +contain an exact copy of the original symbol table as opposed to a stripped
>> +down version containing just the "core" symbols.
>> +
>> +-----------------------------------
>> +3. ".klp.rel." relocation sections
>> +-----------------------------------
>> +
>> +A livepatch module uses special Elf relocation sections to apply
>> +relocations both for regular vmlinux patches as well as those that should
>> +be applied as soon as the to-be-patched module is loaded. For example, if a
>> +patch module patches a driver that is not currently loaded, livepatch will
>> +apply its corresponding klp relocation section(s) to the driver once it
>> +loads.
>> +
>> +The names of these livepatch relocation sections are formatted
>> +".klp.rel.${objname}.", where ${objname} is the name of the
>> "object" being
>
>It seems that the full format is ".klp.rel.[objname].section_name"
>and section_name is not explained here.

Ah, thanks for catching that. I meant to say that they are *prefixed*
by the string ".klp.rel.[objname].", but like you said, it is much
better to explain the full format ".klp.rel.[objname].section_name"
here.

>
>> +patched (e.g. vmlinux or name of module). Each object within a patch module
>> +may have multiple klp sections (e.g. patches to multiple functions within
>> +the same object). There is a 1-1 correspondence between a klp relocation
>> +section and the target section (usually the text section for a function) to
>> +which the relocation(s) apply.
>> +
>> +Here's a sample readelf output for a livepatch module that patches vmlinux and
>> +modules 9p, btrfs, ext4:
>> +  ...
>> +  [29] .klp.rel.9p.text.caches.show RELA                    0000000000000000 002d58 0000c0 18 AIo 64   9  8
>> +  [30] .klp.rel.btrfs.text.btrfs.feature.attr.show RELA     0000000000000000 002e18 000060 18 AIo 64  11  8
>> +  ...
>> +  [34] .klp.rel.ext4.text.ext4.attr.store RELA              0000000000000000 002fd8 0000d8 18 AIo 64  13  8
>> +  [35] .klp.rel.ext4.text.ext4.attr.show RELA               0000000000000000 0030b0 000150 18 AIo 64  15  8
>> +  [36] .klp.rel.vmlinux.text.cmdline.proc.show RELA         0000000000000000 003200 000018 18 AIo 64  17  8
>> +  [37] .klp.rel.vmlinux.text.meminfo.proc.show RELA         0000000000000000 003218 0000f0 18 AIo 64  19  8
>> +  ...
>> +
>> +klp relocation sections are SHT_RELA sections but with a few special
>> +characteristics. Notice that they are marked SHF_ALLOC ("A") so that they
>> +will not be discarded when the module is loaded into memory, as well as
>> +with the SHF_RELA_LIVEPATCH flag ("o" - for OS-specific) so the module
>
>Hmm, it is SHF_RELA... I wonder if we want to rather use
>.klp.rela. prefix to be consistent.

Yeah, that looks better. I was actually just trying to make the
prefixes ".klp.sym." and ".klp.rel." the same length so the string
parsing code would be slightly simpler using KLP_TAG_LEN.

>> +loader can identify them and avoid treating them as regular SHT_RELA
>> +sections, since they are manually managed by livepatch.
>> +
>> +Since Elf information is preserved for livepatch modules (see Section 4), a
>> +klp relocation section can be applied simply by passing in the appropriate
>> +section index to apply_relocate_add() (in the module loader code), which
>> +then uses it to access the relocation section and apply the relocations.
>> +
>> +--------------------------------------------------------
>> +4. How a livepatch module accesses its symbol table and
>> +its klp relocation sections
>> +--------------------------------------------------------
>> +
>> +The kernel module loader checks whether the module being loaded is a
>> +livepatch module. If so, it then makes a copy of the module's Elf header,
>> +section headers, section name string table, and some noteworthy section
>> +indices (for example, the symtab's section index). It adjusts the symtab's
>> +sh_addr to point to mod->core_symtab, since the original mod->symtab lies
>> +in init memory and gets freed once the module finishes initializing. For
>> +livepatch modules, the core_symtab will be an exact copy of its original
>> +symbol table (where normally, only "core" symbols are included in this
>> +symbol table. See is_core_symbol() in kernel/module.c). Livepatch requires
>> +that the symbols retain their original indices in the symbol table so that
>> +the klp relocation sections can be applied correctly.
>
>We should add some notice also to the source code or commit message
>aboud why we preserve all the symbols for live patch.
>
>Great work.

Thanks for the review Petr!

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [11/01/16 21:05 -0600]:
>On Mon, Jan 11, 2016 at 05:35:13PM -0500, Jessica Yu wrote:
>> +++ Josh Poimboeuf [11/01/16 15:33 -0600]:
>> >On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
>> >>Reuse module loader code to write relocations, thereby eliminating the need
>> >>for architecture specific relocation code in livepatch. Namely, we reuse
>> >>apply_relocate_add() in the module loader to write relocations instead of
>> >>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>> >>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>> >>are resolved and then apply_relocate_add() is called to apply those
>> >>relocations.
>> >>
>> >>In addition, remove x86 livepatch relocation code. It is no longer needed
>> >>since symbol resolution and relocation work have been offloaded to module
>> >>loader.
>> >>
>> >>Signed-off-by: Jessica Yu <jeyu@redhat.com>
>> >>---
>> >> arch/x86/include/asm/livepatch.h |   2 -
>> >> arch/x86/kernel/Makefile         |   1 -
>> >> arch/x86/kernel/livepatch.c      |  70 ---------------------------
>> >> include/linux/livepatch.h        |  33 +++++--------
>> >> kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
>> >> 5 files changed, 62 insertions(+), 145 deletions(-)
>> >> delete mode 100644 arch/x86/kernel/livepatch.c
>> >>
>> >>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> >>index 19c099a..7312e25 100644
>> >>--- a/arch/x86/include/asm/livepatch.h
>> >>+++ b/arch/x86/include/asm/livepatch.h
>> >>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>> >> #endif
>> >> 	return 0;
>> >> }
>> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>> >>-			   unsigned long loc, unsigned long value);
>> >>
>> >> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>> >> {
>> >>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> >>index b1b78ff..c5e9a5c 100644
>> >>--- a/arch/x86/kernel/Makefile
>> >>+++ b/arch/x86/kernel/Makefile
>> >>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>> >> obj-y				+= apic/
>> >> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>> >> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> >>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>> >> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>> >> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>> >> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> >>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> >>deleted file mode 100644
>> >>index 92fc1a5..0000000
>> >>--- a/arch/x86/kernel/livepatch.c
>> >>+++ /dev/null
>> >>@@ -1,70 +0,0 @@
>> >>-/*
>> >>- * livepatch.c - x86-specific Kernel Live Patching Core
>> >>- *
>> >>- * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
>> >>- * Copyright (C) 2014 SUSE
>> >>- *
>> >>- * This program is free software; you can redistribute it and/or
>> >>- * modify it under the terms of the GNU General Public License
>> >>- * as published by the Free Software Foundation; either version 2
>> >>- * of the License, or (at your option) any later version.
>> >>- *
>> >>- * This program is distributed in the hope that it will be useful,
>> >>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> >>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> >>- * GNU General Public License for more details.
>> >>- *
>> >>- * You should have received a copy of the GNU General Public License
>> >>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> >>- */
>> >>-
>> >>-#include <linux/module.h>
>> >>-#include <linux/uaccess.h>
>> >>-#include <asm/elf.h>
>> >>-#include <asm/livepatch.h>
>> >>-
>> >>-/**
>> >>- * klp_write_module_reloc() - write a relocation in a module
>> >>- * @mod:	module in which the section to be modified is found
>> >>- * @type:	ELF relocation type (see asm/elf.h)
>> >>- * @loc:	address that the relocation should be written to
>> >>- * @value:	relocation value (sym address + addend)
>> >>- *
>> >>- * This function writes a relocation to the specified location for
>> >>- * a particular module.
>> >>- */
>> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>> >>-			   unsigned long loc, unsigned long value)
>> >>-{
>> >>-	size_t size = 4;
>> >>-	unsigned long val;
>> >>-	unsigned long core = (unsigned long)mod->core_layout.base;
>> >>-	unsigned long core_size = mod->core_layout.size;
>> >>-
>> >>-	switch (type) {
>> >>-	case R_X86_64_NONE:
>> >>-		return 0;
>> >>-	case R_X86_64_64:
>> >>-		val = value;
>> >>-		size = 8;
>> >>-		break;
>> >>-	case R_X86_64_32:
>> >>-		val = (u32)value;
>> >>-		break;
>> >>-	case R_X86_64_32S:
>> >>-		val = (s32)value;
>> >>-		break;
>> >>-	case R_X86_64_PC32:
>> >>-		val = (u32)(value - loc);
>> >>-		break;
>> >>-	default:
>> >>-		/* unsupported relocation type */
>> >>-		return -EINVAL;
>> >>-	}
>> >>-
>> >>-	if (loc < core || loc >= core + core_size)
>> >>-		/* loc does not point to any symbol inside the module */
>> >>-		return -EINVAL;
>> >>-
>> >>-	return probe_kernel_write((void *)loc, &val, size);
>> >>-}
>> >>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> >>index a882865..2f12ce7 100644
>> >>--- a/include/linux/livepatch.h
>> >>+++ b/include/linux/livepatch.h
>> >>@@ -65,27 +65,8 @@ struct klp_func {
>> >> };
>> >>
>> >> /**
>> >>- * struct klp_reloc - relocation structure for live patching
>> >>- * @loc:	address where the relocation will be written
>> >>- * @sympos:	position in kallsyms to disambiguate symbols (optional)
>> >>- * @type:	ELF relocation type
>> >>- * @name:	name of the referenced symbol (for lookup/verification)
>> >>- * @addend:	offset from the referenced symbol
>> >>- * @external:	symbol is either exported or within the live patch module itself
>> >>- */
>> >>-struct klp_reloc {
>> >>-	unsigned long loc;
>> >>-	unsigned long sympos;
>> >>-	unsigned long type;
>> >>-	const char *name;
>> >>-	int addend;
>> >>-	int external;
>> >>-};
>> >>-
>> >>-/**
>> >>  * struct klp_object - kernel object structure for live patching
>> >>  * @name:	module name (or NULL for vmlinux)
>> >>- * @relocs:	relocation entries to be applied at load time
>> >>  * @funcs:	function entries for functions to be patched in the object
>> >>  * @kobj:	kobject for sysfs resources
>> >>  * @mod:	kernel module associated with the patched object
>> >>@@ -95,7 +76,6 @@ struct klp_reloc {
>> >> struct klp_object {
>> >> 	/* external */
>> >> 	const char *name;
>> >>-	struct klp_reloc *relocs;
>> >> 	struct klp_func *funcs;
>> >>
>> >> 	/* internal */
>> >>@@ -123,6 +103,19 @@ struct klp_patch {
>> >> 	enum klp_state state;
>> >> };
>> >>
>> >>+/*
>> >>+ * Livepatch-specific symbols and relocation
>> >>+ * sections are prefixed with a tag:
>> >>+ * .klp.rel. for relocation sections
>> >>+ * .klp.sym. for livepatch symbols
>> >>+ */
>> >>+#define KLP_TAG_LEN 9
>> >>+/*
>> >>+ * Livepatch-specific bits for specifying symbol
>> >>+ * positions in the Elf_Sym st_other field
>> >>+ */
>> >>+#define KLP_SYMPOS(o) (o >> 2) & 0xff
>> >>+
>> >
>> >Can st_value be used instead?  I think we ended up deciding that would
>> >be better:
>> >
>> > https://lkml.kernel.org/g/20151210213328.GA6553@packer-debian-8-amd64.digitalocean.com
>> >
>> >Because:
>> >
>> >- st_value is easily viewable in readelf
>> >- st_other has some arch-specific uses
>> >
>> >And another reason not previously discussed:
>> >
>> >- st_other is an unsigned char, which limits sympos to values < 64
>>
>> I originally wanted to encode the symbol position in st_value, but
>> I've discovered that since st_value is overwritten once the
>> symbol is (first) resolved, we no longer have the symbol position if we
>> need to resolve the symbols again (so we wouldn't be able to resolve
>> them). This could happen when we patch a module that loads and
>> unloads more than once for example.
>
>Ah, good point.
>
>> I chose st_other since it isn't touched in s390x kernel code nor in
>> x86 kernel code (it does get used in the x86 userspace reloc tool in
>> arch/x86/tools, which is why I left the first two bits alone for
>> ELF_ST_VISIBILITY, and the rest had undefined usage). However this
>> isn't the best approach and I'm afraid of stepping on other arch's
>> toes in future patches. Perhaps there is another field where we can
>> stuff the sympos in? st_size for instance doesn't seem to be touched
>> anywhere in the module loader. I don't know if I'd want to stuff
>> sympos in the sym name, there's enough going on there..
>
>I think st_other still isn't going to work because of possible arch
>conflicts and because of its small size (6 bits) otherwise.
>
>st_size could work.  It doesn't _seem_ to be used by the module code,
>though it's hard to confirm that for all arches.  But it makes me a
>little nervous to override that field.  For example, might it confuse
>some user-space tools out there, or the linker?
>
>Maybe encoding it in the sym name is the safest bet.  Why not, we've
>already got a bunch of other stuff there anyway :-)  One idea would be
>to append it with ',<sympos>' to be consistent with the sysfs entries:
>
>  .klp.sym.vmlinux.symname,1

Sure, as long as commas aren't allowed in symbol names, that will work. :-)

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [11/01/16 21:05 -0600]:
>On Mon, Jan 11, 2016 at 05:35:13PM -0500, Jessica Yu wrote:
>> +++ Josh Poimboeuf [11/01/16 15:33 -0600]:
>> >On Fri, Jan 08, 2016 at 02:28:22PM -0500, Jessica Yu wrote:
>> >>Reuse module loader code to write relocations, thereby eliminating the need
>> >>for architecture specific relocation code in livepatch. Namely, we reuse
>> >>apply_relocate_add() in the module loader to write relocations instead of
>> >>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>> >>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>> >>are resolved and then apply_relocate_add() is called to apply those
>> >>relocations.
>> >>
>> >>In addition, remove x86 livepatch relocation code. It is no longer needed
>> >>since symbol resolution and relocation work have been offloaded to module
>> >>loader.
>> >>
>> >>Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> >>---
>> >> arch/x86/include/asm/livepatch.h |   2 -
>> >> arch/x86/kernel/Makefile         |   1 -
>> >> arch/x86/kernel/livepatch.c      |  70 ---------------------------
>> >> include/linux/livepatch.h        |  33 +++++--------
>> >> kernel/livepatch/core.c          | 101 +++++++++++++++++++--------------------
>> >> 5 files changed, 62 insertions(+), 145 deletions(-)
>> >> delete mode 100644 arch/x86/kernel/livepatch.c
>> >>
>> >>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> >>index 19c099a..7312e25 100644
>> >>--- a/arch/x86/include/asm/livepatch.h
>> >>+++ b/arch/x86/include/asm/livepatch.h
>> >>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>> >> #endif
>> >> 	return 0;
>> >> }
>> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>> >>-			   unsigned long loc, unsigned long value);
>> >>
>> >> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>> >> {
>> >>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> >>index b1b78ff..c5e9a5c 100644
>> >>--- a/arch/x86/kernel/Makefile
>> >>+++ b/arch/x86/kernel/Makefile
>> >>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>> >> obj-y				+= apic/
>> >> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>> >> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> >>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>> >> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>> >> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>> >> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> >>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> >>deleted file mode 100644
>> >>index 92fc1a5..0000000
>> >>--- a/arch/x86/kernel/livepatch.c
>> >>+++ /dev/null
>> >>@@ -1,70 +0,0 @@
>> >>-/*
>> >>- * livepatch.c - x86-specific Kernel Live Patching Core
>> >>- *
>> >>- * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> >>- * Copyright (C) 2014 SUSE
>> >>- *
>> >>- * This program is free software; you can redistribute it and/or
>> >>- * modify it under the terms of the GNU General Public License
>> >>- * as published by the Free Software Foundation; either version 2
>> >>- * of the License, or (at your option) any later version.
>> >>- *
>> >>- * This program is distributed in the hope that it will be useful,
>> >>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> >>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> >>- * GNU General Public License for more details.
>> >>- *
>> >>- * You should have received a copy of the GNU General Public License
>> >>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> >>- */
>> >>-
>> >>-#include <linux/module.h>
>> >>-#include <linux/uaccess.h>
>> >>-#include <asm/elf.h>
>> >>-#include <asm/livepatch.h>
>> >>-
>> >>-/**
>> >>- * klp_write_module_reloc() - write a relocation in a module
>> >>- * @mod:	module in which the section to be modified is found
>> >>- * @type:	ELF relocation type (see asm/elf.h)
>> >>- * @loc:	address that the relocation should be written to
>> >>- * @value:	relocation value (sym address + addend)
>> >>- *
>> >>- * This function writes a relocation to the specified location for
>> >>- * a particular module.
>> >>- */
>> >>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>> >>-			   unsigned long loc, unsigned long value)
>> >>-{
>> >>-	size_t size = 4;
>> >>-	unsigned long val;
>> >>-	unsigned long core = (unsigned long)mod->core_layout.base;
>> >>-	unsigned long core_size = mod->core_layout.size;
>> >>-
>> >>-	switch (type) {
>> >>-	case R_X86_64_NONE:
>> >>-		return 0;
>> >>-	case R_X86_64_64:
>> >>-		val = value;
>> >>-		size = 8;
>> >>-		break;
>> >>-	case R_X86_64_32:
>> >>-		val = (u32)value;
>> >>-		break;
>> >>-	case R_X86_64_32S:
>> >>-		val = (s32)value;
>> >>-		break;
>> >>-	case R_X86_64_PC32:
>> >>-		val = (u32)(value - loc);
>> >>-		break;
>> >>-	default:
>> >>-		/* unsupported relocation type */
>> >>-		return -EINVAL;
>> >>-	}
>> >>-
>> >>-	if (loc < core || loc >= core + core_size)
>> >>-		/* loc does not point to any symbol inside the module */
>> >>-		return -EINVAL;
>> >>-
>> >>-	return probe_kernel_write((void *)loc, &val, size);
>> >>-}
>> >>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> >>index a882865..2f12ce7 100644
>> >>--- a/include/linux/livepatch.h
>> >>+++ b/include/linux/livepatch.h
>> >>@@ -65,27 +65,8 @@ struct klp_func {
>> >> };
>> >>
>> >> /**
>> >>- * struct klp_reloc - relocation structure for live patching
>> >>- * @loc:	address where the relocation will be written
>> >>- * @sympos:	position in kallsyms to disambiguate symbols (optional)
>> >>- * @type:	ELF relocation type
>> >>- * @name:	name of the referenced symbol (for lookup/verification)
>> >>- * @addend:	offset from the referenced symbol
>> >>- * @external:	symbol is either exported or within the live patch module itself
>> >>- */
>> >>-struct klp_reloc {
>> >>-	unsigned long loc;
>> >>-	unsigned long sympos;
>> >>-	unsigned long type;
>> >>-	const char *name;
>> >>-	int addend;
>> >>-	int external;
>> >>-};
>> >>-
>> >>-/**
>> >>  * struct klp_object - kernel object structure for live patching
>> >>  * @name:	module name (or NULL for vmlinux)
>> >>- * @relocs:	relocation entries to be applied at load time
>> >>  * @funcs:	function entries for functions to be patched in the object
>> >>  * @kobj:	kobject for sysfs resources
>> >>  * @mod:	kernel module associated with the patched object
>> >>@@ -95,7 +76,6 @@ struct klp_reloc {
>> >> struct klp_object {
>> >> 	/* external */
>> >> 	const char *name;
>> >>-	struct klp_reloc *relocs;
>> >> 	struct klp_func *funcs;
>> >>
>> >> 	/* internal */
>> >>@@ -123,6 +103,19 @@ struct klp_patch {
>> >> 	enum klp_state state;
>> >> };
>> >>
>> >>+/*
>> >>+ * Livepatch-specific symbols and relocation
>> >>+ * sections are prefixed with a tag:
>> >>+ * .klp.rel. for relocation sections
>> >>+ * .klp.sym. for livepatch symbols
>> >>+ */
>> >>+#define KLP_TAG_LEN 9
>> >>+/*
>> >>+ * Livepatch-specific bits for specifying symbol
>> >>+ * positions in the Elf_Sym st_other field
>> >>+ */
>> >>+#define KLP_SYMPOS(o) (o >> 2) & 0xff
>> >>+
>> >
>> >Can st_value be used instead?  I think we ended up deciding that would
>> >be better:
>> >
>> > https://lkml.kernel.org/g/20151210213328.GA6553-N0bYjD2NfQ6k4hzjq3hgyGTy53QMssKEsp+A89P3RPuQWHG76I6BsA@public.gmane.org
>> >
>> >Because:
>> >
>> >- st_value is easily viewable in readelf
>> >- st_other has some arch-specific uses
>> >
>> >And another reason not previously discussed:
>> >
>> >- st_other is an unsigned char, which limits sympos to values < 64
>>
>> I originally wanted to encode the symbol position in st_value, but
>> I've discovered that since st_value is overwritten once the
>> symbol is (first) resolved, we no longer have the symbol position if we
>> need to resolve the symbols again (so we wouldn't be able to resolve
>> them). This could happen when we patch a module that loads and
>> unloads more than once for example.
>
>Ah, good point.
>
>> I chose st_other since it isn't touched in s390x kernel code nor in
>> x86 kernel code (it does get used in the x86 userspace reloc tool in
>> arch/x86/tools, which is why I left the first two bits alone for
>> ELF_ST_VISIBILITY, and the rest had undefined usage). However this
>> isn't the best approach and I'm afraid of stepping on other arch's
>> toes in future patches. Perhaps there is another field where we can
>> stuff the sympos in? st_size for instance doesn't seem to be touched
>> anywhere in the module loader. I don't know if I'd want to stuff
>> sympos in the sym name, there's enough going on there..
>
>I think st_other still isn't going to work because of possible arch
>conflicts and because of its small size (6 bits) otherwise.
>
>st_size could work.  It doesn't _seem_ to be used by the module code,
>though it's hard to confirm that for all arches.  But it makes me a
>little nervous to override that field.  For example, might it confuse
>some user-space tools out there, or the linker?
>
>Maybe encoding it in the sym name is the safest bet.  Why not, we've
>already got a bunch of other stuff there anyway :-)  One idea would be
>to append it with ',<sympos>' to be consistent with the sysfs entries:
>
>  .klp.sym.vmlinux.symname,1

Sure, as long as commas aren't allowed in symbol names, that will work. :-)

Re: livepatch: reuse module loader code to write relocations

[Miroslav Benes]

On Wed, 13 Jan 2016, Jessica Yu wrote:

> > Maybe it would be better to use pmod->symtab and pmod->strtab everywhere.
> > It should be the same, but core_* versions are only helpers used in
> > load_module and friends. There is even a comment in
> > include/linux/module.h.
> > 
> > 	/*
> > 	 * We keep the symbol and string tables for kallsyms.
> > 	 * The core_* fields below are temporary, loader-only (they
> > 	 * could really be discarded after module init).
> > 	 */
> > 
> > We should respect that.
> 
> I admit I'm a bit confused by the comment, I can't seem to find where
> core_symtab and core_strtab are purportedly discarded after module
> init (perhaps I'm missing something?). IMO it sounds more like it's
> describing mod->symtab and mod->strtab instead, because these are in
> module init memory and are freed later. 

I think it just says that core_* symbols are used as temporary tables 
during module loading. They are not discarded anywhere but they could be 
(and maybe they'll be in the future). So it is better not to depend on 
them.

> In any case, my reason for using
> core_symtab is that the original symbol table (mod->symtab) is marked
> with INIT_OFFSET_MASK in layout_symtab() (see kernel/module.c), and is
> therefore in init memory. This memory is freed near the end of
> do_init_module() with do_free_init(). Since core_symtab is in module core
> memory, for livepatch modules I simply used core_symtab to hold a
> full copy of the symbol table instead of the slimmed down version that
> it was originally intended to hold.

But both mod->symtab and mod->strtab are changed to point to their core_ 
versions right before do_free_init is called in do_init_module. So they 
should be the same. My remark was more of an academic question. I believe 
it is not a functional thing, just the matter of taste. But maybe I am 
missing something.

> Alternatively, we can tweak layout_symtab() to *not* mark the symtab
> with INIT_OFFSET_MASK and put it in core memory instead. I think
> either way will work, but maybe it is cleaner to do it this way
> instead.

Yeah, I wouldn't do this. core_* symbols are ok from functional point of 
view.

Thanks,
Miroslav

Re: livepatch: reuse module loader code to write relocations

[Miroslav Benes]

On Wed, 13 Jan 2016, Jessica Yu wrote:

> +++ Miroslav Benes [13/01/16 10:19 +0100]:
> > On Fri, 8 Jan 2016, Jessica Yu wrote:
> > 
> > >  static int klp_write_object_relocations(struct module *pmod,
> > >  					struct klp_object *obj)
> > >  {
> > > -	int ret = 0;
> > > -	unsigned long val;
> > > -	struct klp_reloc *reloc;
> > > +	int i, len, ret = 0;
> > > +	char *secname;
> > > +	const char *objname;
> > > 
> > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
> > >  		return -EINVAL;
> > > 
> > > -	if (WARN_ON(!obj->relocs))
> > > -		return -EINVAL;
> > > +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
> > > 
> > >  	module_disable_ro(pmod);
> > > +	/* For each klp rela section for this object */
> > > +	for (i = 1; i < pmod->info->hdr->e_shnum; i++) {
> > > +		if (!(pmod->info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH))
> > > +			continue;
> > 
> > One more thing. If the module does not specify it is a live patch module
> > in modinfo (with MODULE_INFO(livepatch, "Y")), but it is a perfect live
> > patch module otherwise (it calls klp_register_patch in its init function),
> > the kernel crashes here. pmod->info is not initialized at all. This should
> > be fixed. Perhaps the easiest would be to call
> > klp_write_object_relocations() in klp_init_object_loaded() only if
> > is_livepatch_module() returns true. Similar to a check for obj->relocs
> > before.
> 
> Hm yes, that's a problem. To remedy this, I think it makes sense to
> require all livepatch modules to identify themselves with the modinfo
> attribute, since it is a very simple requirement. If some module calls
> klp_register_patch() and it does not have the livepatch attribute,
> klp_register_patch() can just return an error. We can call
> is_livepatch_module() at the beginning of klp_register_patch(), and
> proceed only if the check succeeds, since we'll then know that the
> required structures have been properly initialized in the module
> loader. What do you think?

This is similar to what Jiri proposed in his mail. It is up to you. Both 
ways (the warning and the check, or what you propose) are fine.

Miroslav

Re: module: preserve Elf information for livepatch modules

[Rusty Russell]

Jessica Yu <jeyu@redhat.com> writes:
> +++ Rusty Russell [11/01/16 11:55 +1030]:
>>This comment is a bit misleading: it's actually pointing into the
>>temporary module copy, which will be discarded.  The init section is
>>slightly different...
>
> Ah, perhaps I'm misunderstanding something..
>
> Since copy_module_elf() is called after move_module(), my
> understanding is that all the section sh_addr's should be pointing
> to either module core memory or module init memory (instead of the
> initial temporary copy of the module in info->hdr).

You're absolutely correct; I'd forgotten that particular kallsyms twist.

Sorry for the noise!
Rusty.

Re: module: preserve Elf information for livepatch modules

[Rusty Russell]

Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
> +++ Rusty Russell [11/01/16 11:55 +1030]:
>>This comment is a bit misleading: it's actually pointing into the
>>temporary module copy, which will be discarded.  The init section is
>>slightly different...
>
> Ah, perhaps I'm misunderstanding something..
>
> Since copy_module_elf() is called after move_module(), my
> understanding is that all the section sh_addr's should be pointing
> to either module core memory or module init memory (instead of the
> initial temporary copy of the module in info->hdr).

You're absolutely correct; I'd forgotten that particular kallsyms twist.

Sorry for the noise!
Rusty.

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Jiri Kosina [21/03/16 23:02 +0100]:
>On Mon, 21 Mar 2016, Josh Poimboeuf wrote:
>
>> According to my test that still results in the literal value of
>> "(64 - 8)".
>
>Alright. But we should be able to special-case it with a two #if checks on
>the __SIZEOF_LONG__ value and BUILD_BUG_ON() when __SIZEOF_LONG__ is not
>of one of the ususal sizes.
>

And while we're at it, we might as well add a BUILD_BUG_ON check for
KSYM_NAME_LEN too, since we are also hard coding that field width, and
we'd like to be alerted if that value ever changes.

Re: livepatch: reuse module loader code to write relocations

[Jiri Kosina]

On Mon, 21 Mar 2016, Josh Poimboeuf wrote:

> According to my test that still results in the literal value of
> "(64 - 8)".

Alright. But we should be able to special-case it with a two #if checks on 
the __SIZEOF_LONG__ value and BUILD_BUG_ON() when __SIZEOF_LONG__ is not 
of one of the ususal sizes.

Thanks,

-- 
Jiri Kosina
SUSE Labs

Re: livepatch: reuse module loader code to write relocations

[Jiri Kosina]

On Mon, 21 Mar 2016, Josh Poimboeuf wrote:

> According to my test that still results in the literal value of
> "(64 - 8)".

Alright. But we should be able to special-case it with a two #if checks on 
the __SIZEOF_LONG__ value and BUILD_BUG_ON() when __SIZEOF_LONG__ is not 
of one of the ususal sizes.

Thanks,

-- 
Jiri Kosina
SUSE Labs

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Mon, Mar 21, 2016 at 10:16:17PM +0100, Jiri Kosina wrote:
> On Mon, 21 Mar 2016, Jessica Yu wrote:
> 
> > Yes, this is a concern and I'm not sure what the best way to fix it
> > is. If both MODULE_NAME_LEN and KSYM_NAME_LEN were straight up
> > constants, then I think Josh's stringify approach would have worked
> > perfectly. However since MODULE_NAME_LEN translates to an expression
> > (64 - sizeof(unsigned long)), which the preprocessor cannot evaluate,
> > we will need another approach. Building the format strings at run time
> > might be messier than we'd like. Alternatively we could just go the
> > simple route and simply be a bit more aggressive on the upper bound
> > for the format width; though the size of long varies on different
> > architectures, afaik the max size it could ever be on any arch is 8
> > bytes, so perhaps 64 - 8 = 56 (then - 1 to make room for \0) might be
> > an appropriate field width. This would deserve a comment as well.
> 
> So how about actually modifying MAX_PARAM_PREFIX_LEN so 
> that it's actually properly evaluable at preprocessing time, 
> i.e. something along the lines of
> 
> diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
> index 52666d9..954dae9 100644
> --- a/include/linux/moduleparam.h
> +++ b/include/linux/moduleparam.h
> @@ -14,7 +14,7 @@
>  #endif
>  
>  /* Chosen so that structs with an unsigned long line up. */
> -#define MAX_PARAM_PREFIX_LEN (64 - sizeof(unsigned long))
> +#define MAX_PARAM_PREFIX_LEN (64 - __SIZEOF_LONG__)
>  
>  #ifdef MODULE
>  #define __MODULE_INFO(tag, name, info)					  \

According to my test that still results in the literal value of
"(64 - 8)".

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Jiri Kosina]

On Mon, 21 Mar 2016, Jessica Yu wrote:

> Yes, this is a concern and I'm not sure what the best way to fix it
> is. If both MODULE_NAME_LEN and KSYM_NAME_LEN were straight up
> constants, then I think Josh's stringify approach would have worked
> perfectly. However since MODULE_NAME_LEN translates to an expression
> (64 - sizeof(unsigned long)), which the preprocessor cannot evaluate,
> we will need another approach. Building the format strings at run time
> might be messier than we'd like. Alternatively we could just go the
> simple route and simply be a bit more aggressive on the upper bound
> for the format width; though the size of long varies on different
> architectures, afaik the max size it could ever be on any arch is 8
> bytes, so perhaps 64 - 8 = 56 (then - 1 to make room for \0) might be
> an appropriate field width. This would deserve a comment as well.

So how about actually modifying MAX_PARAM_PREFIX_LEN so 
that it's actually properly evaluable at preprocessing time, 
i.e. something along the lines of

diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
index 52666d9..954dae9 100644
--- a/include/linux/moduleparam.h
+++ b/include/linux/moduleparam.h
@@ -14,7 +14,7 @@
 #endif
 
 /* Chosen so that structs with an unsigned long line up. */
-#define MAX_PARAM_PREFIX_LEN (64 - sizeof(unsigned long))
+#define MAX_PARAM_PREFIX_LEN (64 - __SIZEOF_LONG__)
 
 #ifdef MODULE
 #define __MODULE_INFO(tag, name, info)					  \

-- 
Jiri Kosina
SUSE Labs

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Mon, Mar 21, 2016 at 03:18:32PM -0400, Jessica Yu wrote:
> +++ Miroslav Benes [21/03/16 14:55 +0100]:
> >On Wed, 16 Mar 2016, Jessica Yu wrote:
> >
> >[...]
> >
> >>+struct klp_buf {
> >>+	char symname[KSYM_SYMBOL_LEN];
> >
> >I think it is better to make this KSYM_NAME_LEN. KSYM_SYMBOL_LEN looks
> >like something different and KSYM_NAME_LEN is 128 which you reference
> >below.
> >
> 
> Ack, I did mean to use KSYM_NAME_LEN, thanks.
> 
> >>+	char objname[MODULE_NAME_LEN];
> >>+};
> >
> >[...]
> >
> >>+static int klp_resolve_symbols(Elf_Shdr *relasec, struct module *pmod)
> >>+{
> >>+	int i, cnt, vmlinux, ret;
> >>+	struct klp_buf bufs = {0};
> >>+	Elf_Rela *relas;
> >>+	Elf_Sym *sym;
> >>+	char *symname;
> >>+	unsigned long sympos;
> >>+
> >>+	relas = (Elf_Rela *) relasec->sh_addr;
> >>+	/* For each rela in this klp relocation section */
> >>+	for (i = 0; i < relasec->sh_size / sizeof(Elf_Rela); i++) {
> >>+		sym = pmod->core_kallsyms.symtab + ELF_R_SYM(relas[i].r_info);
> >>+		if (sym->st_shndx != SHN_LIVEPATCH)
> >>+			return -EINVAL;
> >>+
> >>+		klp_clear_buf(&bufs);
> >>+
> >>+		/* Format: .klp.sym.objname.symbol_name,sympos */
> >>+		symname = pmod->core_kallsyms.strtab + sym->st_name;
> >>+		cnt = sscanf(symname, ".klp.sym.%64[^.].%128[^,],%lu",
> >>+			     bufs.objname, bufs.symname, &sympos);
> >
> >It would be really nice to change actual values for their macro
> >definitions, but this would be a mess which is not worth it. Anyway
> >shouldn't those width modifiers be %63 and %127 to make a room for \0?
> >
> 
> Yes, this is a concern and I'm not sure what the best way to fix it
> is. If both MODULE_NAME_LEN and KSYM_NAME_LEN were straight up
> constants, then I think Josh's stringify approach would have worked
> perfectly. However since MODULE_NAME_LEN translates to an expression
> (64 - sizeof(unsigned long)), which the preprocessor cannot evaluate,
> we will need another approach. Building the format strings at run time
> might be messier than we'd like. Alternatively we could just go the
> simple route and simply be a bit more aggressive on the upper bound
> for the format width; though the size of long varies on different
> architectures, afaik the max size it could ever be on any arch is 8
> bytes, so perhaps 64 - 8 = 56 (then - 1 to make room for \0) might be
> an appropriate field width. This would deserve a comment as well.

I think something like that would be good, along with:

  BUILD_BUG_ON(MODULE_NAME_LEN < 56);

and a comment explaining why.

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [21/03/16 14:55 +0100]:
>On Wed, 16 Mar 2016, Jessica Yu wrote:
>
>[...]
>
>> +struct klp_buf {
>> +	char symname[KSYM_SYMBOL_LEN];
>
>I think it is better to make this KSYM_NAME_LEN. KSYM_SYMBOL_LEN looks
>like something different and KSYM_NAME_LEN is 128 which you reference
>below.
>

Ack, I did mean to use KSYM_NAME_LEN, thanks.

>> +	char objname[MODULE_NAME_LEN];
>> +};
>
>[...]
>
>> +static int klp_resolve_symbols(Elf_Shdr *relasec, struct module *pmod)
>> +{
>> +	int i, cnt, vmlinux, ret;
>> +	struct klp_buf bufs = {0};
>> +	Elf_Rela *relas;
>> +	Elf_Sym *sym;
>> +	char *symname;
>> +	unsigned long sympos;
>> +
>> +	relas = (Elf_Rela *) relasec->sh_addr;
>> +	/* For each rela in this klp relocation section */
>> +	for (i = 0; i < relasec->sh_size / sizeof(Elf_Rela); i++) {
>> +		sym = pmod->core_kallsyms.symtab + ELF_R_SYM(relas[i].r_info);
>> +		if (sym->st_shndx != SHN_LIVEPATCH)
>> +			return -EINVAL;
>> +
>> +		klp_clear_buf(&bufs);
>> +
>> +		/* Format: .klp.sym.objname.symbol_name,sympos */
>> +		symname = pmod->core_kallsyms.strtab + sym->st_name;
>> +		cnt = sscanf(symname, ".klp.sym.%64[^.].%128[^,],%lu",
>> +			     bufs.objname, bufs.symname, &sympos);
>
>It would be really nice to change actual values for their macro
>definitions, but this would be a mess which is not worth it. Anyway
>shouldn't those width modifiers be %63 and %127 to make a room for \0?
>

Yes, this is a concern and I'm not sure what the best way to fix it
is. If both MODULE_NAME_LEN and KSYM_NAME_LEN were straight up
constants, then I think Josh's stringify approach would have worked
perfectly. However since MODULE_NAME_LEN translates to an expression
(64 - sizeof(unsigned long)), which the preprocessor cannot evaluate,
we will need another approach. Building the format strings at run time
might be messier than we'd like. Alternatively we could just go the
simple route and simply be a bit more aggressive on the upper bound
for the format width; though the size of long varies on different
architectures, afaik the max size it could ever be on any arch is 8
bytes, so perhaps 64 - 8 = 56 (then - 1 to make room for \0) might be
an appropriate field width. This would deserve a comment as well.

>> +		if (cnt != 3)
>> +			return -EINVAL;
>> +
>> +		/* klp_find_object_symbol() treats a NULL objname as vmlinux */
>> +		vmlinux = !strcmp(bufs.objname, "vmlinux");
>> +		ret = klp_find_object_symbol(vmlinux ? NULL : bufs.objname,
>> +					     bufs.symname, sympos,
>> +					     (unsigned long *) &sym->st_value);
>> +		if (ret)
>> +			return ret;
>>  	}
>> -	preempt_enable();
>>
>> -	/*
>> -	 * Check if it's in another .o within the patch module. This also
>> -	 * checks that the external symbol is unique.
>> -	 */
>> -	return klp_find_object_symbol(pmod->name, name, 0, addr);
>> +	return 0;
>>  }
>>
>>  static int klp_write_object_relocations(struct module *pmod,
>>  					struct klp_object *obj)
>>  {
>> -	int ret = 0;
>> -	unsigned long val;
>> -	struct klp_reloc *reloc;
>> +	int i, cnt, ret = 0;
>> +	const char *objname, *secname;
>> +	struct klp_buf bufs = {0};
>> +	Elf_Shdr *sec;
>>
>>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>  		return -EINVAL;
>>
>> -	if (WARN_ON(!obj->relocs))
>> -		return -EINVAL;
>> +	objname = klp_is_module(obj) ? obj->name : "vmlinux";
>>
>>  	module_disable_ro(pmod);
>> +	/* For each klp relocation section */
>> +	for (i = 1; i < pmod->klp_info->hdr.e_shnum; i++) {
>> +		sec = pmod->klp_info->sechdrs + i;
>> +		if (!(sec->sh_flags & SHF_RELA_LIVEPATCH))
>> +			continue;
>>
>> -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> -		/* discover the address of the referenced symbol */
>> -		if (reloc->external) {
>> -			if (reloc->sympos > 0) {
>> -				pr_err("non-zero sympos for external reloc symbol '%s' is not supported\n",
>> -				       reloc->name);
>> -				ret = -EINVAL;
>> -				goto out;
>> -			}
>> -			ret = klp_find_external_symbol(pmod, reloc->name, &val);
>> -		} else
>> -			ret = klp_find_object_symbol(obj->name,
>> -						     reloc->name,
>> -						     reloc->sympos,
>> -						     &val);
>> -		if (ret)
>> -			goto out;
>> +		klp_clear_buf(&bufs);
>>
>> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> -					     val + reloc->addend);
>> -		if (ret) {
>> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
>> -			       reloc->name, val, ret);
>> -			goto out;
>> +		/* Check if this klp relocation section belongs to obj */
>> +		secname = pmod->klp_info->secstrings + sec->sh_name;
>> +		cnt = sscanf(secname, ".klp.rela.%64[^.]", bufs.objname);
>
>Same here.
>
>Otherwise it looks really good (which applies for the whole series), so
>after fixing these nits you can add my
>
>Reviewed-by: Miroslav Benes <mbenes@suse.cz>
>
>Cheers,
>Miroslav

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [21/03/16 12:36 -0500]:
>On Mon, Mar 21, 2016 at 11:46:51AM -0500, Josh Poimboeuf wrote:
>> On Mon, Mar 21, 2016 at 05:31:57PM +0100, Petr Mladek wrote:
>> > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> > > index 780f00c..2aa20fa 100644
>> > > --- a/kernel/livepatch/core.c
>> > > +++ b/kernel/livepatch/core.c
>> > > +static int klp_resolve_symbols(Elf_Shdr *relasec, struct module *pmod)
>> > > +{
>> > > +	int i, cnt, vmlinux, ret;
>> > > +	struct klp_buf bufs = {0};
>> > > +	Elf_Rela *relas;
>> > > +	Elf_Sym *sym;
>> > > +	char *symname;
>> > > +	unsigned long sympos;
>> > > +
>> > > +	relas = (Elf_Rela *) relasec->sh_addr;
>> > > +	/* For each rela in this klp relocation section */
>> > > +	for (i = 0; i < relasec->sh_size / sizeof(Elf_Rela); i++) {
>> > > +		sym = pmod->core_kallsyms.symtab + ELF_R_SYM(relas[i].r_info);
>> > > +		if (sym->st_shndx != SHN_LIVEPATCH)
>> > > +			return -EINVAL;
>> > > +
>> > > +		klp_clear_buf(&bufs);
>> > > +
>> > > +		/* Format: .klp.sym.objname.symbol_name,sympos */
>> > > +		symname = pmod->core_kallsyms.strtab + sym->st_name;
>> > > +		cnt = sscanf(symname, ".klp.sym.%64[^.].%128[^,],%lu",
>> > > +			     bufs.objname, bufs.symname, &sympos);
>> >
>> > Note that MODULE_NAME_LEN even is not 64. It is defined by:
>> >
>> > #define MAX_PARAM_PREFIX_LEN (64 - sizeof(unsigned long))
>> >
>> > I strongly suggest to use the proposal from Josh.
>>
>> Hm, looks like my suggestion to use __stringify(MODULE_NAME_LEN) doesn't
>> work.  It results in the string "MODULE_NAME_LEN".  Which surprises me:
>> isn't is supposed to resolve the macro before applying the '#' operation
>> to it?
>
>Turns out I hadn't included module.h.  When I do so,
>__stringify(MODULE_NAME_LEN) becomes "(64 - sizeof(unsigned long))".
>Which is still not going to work :-/
>

Hm, we probably won't be able to make use of preprocessor tricks here,
since I don't think the preprocessor can even evaluate that expression
(esp. with that sizeof there). This might mean building the format
string at runtime, which may be more trouble than it's worth...

>> I was going to suggest another idea: hard-code it at 63 and then do
>> something like
>>
>>   BUILD_BUG_ON(MODULE_NAME_LEN != 64)
>>
>> But you're right... it's not even 64!
>>
>> Need to think on this some more...
>
>-- 
>Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Petr Mladek [09/02/16 15:01 +0100]:
>On Wed 2016-02-03 20:11:09, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the need
>> for architecture specific relocation code in livepatch. Specifically, reuse
>> the apply_relocate_add() function in the module loader to write relocations
>> instead of duplicating functionality in livepatch's arch-dependent
>> klp_write_module_reloc() function.
>>
>> In order to accomplish this, livepatch modules manage their own relocation
>> sections (marked with the SHF_RELA_LIVEPATCH section flag) and
>> livepatch-specific symbols (marked with SHN_LIVEPATCH symbol section
>> index). To apply livepatch relocation sections, livepatch symbols
>> referenced by relocs are resolved and then apply_relocate_add() is called
>> to apply those relocations.
>>
>> In addition, remove x86 livepatch relocation code and the s390
>> klp_write_module_reloc() function stub. They are no longer needed since
>> relocation work has been offloaded to module loader.
>>
>> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> index 7aa975d..c1fe57c 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,9 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <linux/string.h>
>> +#include <linux/moduleloader.h>
>>  #include <asm/cacheflush.h>
>>
>>  /**
>> @@ -87,6 +90,166 @@ static bool klp_is_object_loaded(struct klp_object *obj)
>>  	return !obj->name || obj->mod;
>>  }
>>
>> +/*
>> + * Check if a livepatch symbol is formatted properly.
>> + *
>> + * See Documentation/livepatch/module-elf-format.txt for a
>> + * detailed outline of requirements.
>> + */
>> +static int klp_check_symbol_format(struct module *pmod, Elf_Sym *sym)
>> +{
>> +	size_t len;
>> +	char *s, *objname, *symname;
>> +
>> +	if (sym->st_shndx != SHN_LIVEPATCH)
>> +		return -EINVAL;
>> +
>> +	/*
>> +	 * Livepatch symbol names must follow this format:
>> +	 * .klp.sym.objname.symbol_name,sympos
>> +	 */
>> +	s = pmod->strtab + sym->st_name;
>> +	/* [.klp.sym.]objname.symbol_name,sympos */
>> +	if (!s || strncmp(s, KLP_SYM_PREFIX, KLP_SYM_PREFIX_LEN))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname, ".");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.objname.symbol_name,[sympos] */
>> +	if (!strchr(s, ','))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.objname.[symbol_name],sympos */
>> +	symname = objname + len + 1;
>> +	len = strcspn(symname, ",");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	return 0;
>> +}
>> +
>> +/*
>> + * Check if a livepatch relocation section is formatted properly.
>> + *
>> + * See Documentation/livepatch/module-elf-format.txt for a
>> + * detailed outline of requirements.
>> + */
>> +static int klp_check_relasec_format(struct module *pmod, Elf_Shdr *relasec)
>> +{
>> +	char *secname;
>> +	size_t len;
>> +
>> +	secname = pmod->klp_info->secstrings + relasec->sh_name;
>> +	/* [.klp.rela.]objname.section_name */
>> +	if (!secname || strncmp(secname, KLP_RELASEC_PREFIX,
>> +				KLP_RELASEC_PREFIX_LEN))
>> +		return -EINVAL;
>> +
>> +	/* .klp.rela.[objname].section_name */
>> +	len = strcspn(secname + KLP_RELASEC_PREFIX_LEN, ".");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	return 0;
>> +}
>> +
>> +/*
>> + * Check if obj->name matches the objname encoded in the rela
>> + * section name (.klp.rela.[objname].section_name)
>> + *
>> + * Must pass klp_check_relasec_format() before calling this.
>> + */
>> +static bool klp_relasec_matches_object(struct module *pmod, Elf_Shdr *relasec,
>> +				       struct klp_object *obj)
>> +{
>> +	size_t len;
>> +	const char *obj_objname, *sec_objname, *secname;
>> +
>> +	secname = pmod->klp_info->secstrings + relasec->sh_name;
>> +	/* .klp.rela.[objname].section_name */
>> +	sec_objname = secname + KLP_RELASEC_PREFIX_LEN;
>> +	obj_objname = klp_is_module(obj) ? obj->name : "vmlinux";
>> +
>> +	/* Get length of the objname encoded in the section name */
>> +	len = strcspn(sec_objname, ".");
>> +
>> +	if (strlen(obj_objname) != len)
>> +		return false;
>> +
>> +	return strncmp(sec_objname, obj_objname, len) ? false : true;
>> +}
>> +
>> +/*
>> + * klp_get_* helper functions
>> + *
>> + * klp_get_* functions extract different components of the name
>> + * of a livepatch symbol. The full symbol name from the strtab
>> + * is passed in as parameter @s, and @result is filled in with
>> + * the extracted component.
>> + *
>> + * These functions assume a correctly formatted symbol and the
>> + * klp_check_symbol_format() test *must* pass before calling any
>> + * of these functions.
>> + */
>> +
>> +/* .klp.sym.[objname].symbol_name,sympos */
>> +static int klp_get_sym_objname(char *s, char **result)
>> +{
>> +	size_t len;
>> +	char *objname, *objname_start;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname_start = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname_start, ".");
>> +	objname = kstrndup(objname_start, len, GFP_KERNEL);
>> +	if (objname == NULL)
>> +		return -ENOMEM;
>> +
>> +	/* klp_find_object_symbol() treats NULL as vmlinux */
>> +	if (!strcmp(objname, "vmlinux")) {
>> +		*result = NULL;
>> +		kfree(objname);
>> +	} else
>> +		*result = objname;
>> +
>> +	return 0;
>> +}
>> +
>> +/* .klp.sym.objname.[symbol_name],sympos */
>> +static int klp_get_symbol_name(char *s, char **result)
>> +{
>> +	size_t len;
>> +	char *objname, *symname;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname, ".");
>> +
>> +	/* .klp.sym.objname.[symbol_name],sympos */
>> +	symname = objname + len + 1;
>> +	len = strcspn(symname, ",");
>> +
>> +	*result = kstrndup(symname, len, GFP_KERNEL);
>> +	if (*result == NULL)
>> +		return -ENOMEM;
>> +
>> +	return 0;
>> +}
>> +
>> +/* .klp.sym.objname.symbol_name,[sympos] */
>> +static int klp_get_sympos(char *s, unsigned long *result)
>> +{
>> +	char *sympos;
>> +
>> +	/* .klp.sym.symbol_name,[sympos] */
>> +	sympos = strchr(s, ',') + 1;
>> +	return kstrtol(sympos, 10, result);
>> +}
>
>The usage of the helper function is nicely strightforward.
>Also I like a lot all the comments with the [] brackets
>that highlight what each check or search is for.
>
>But I think that there is a lot of duplicated code in
>the check_ and in the get_ functions. Also there is
>a lot of strdup/free games.

I agree, I do not really like the repetition and all
the strdup/free's myself.

>The length of the elemets is limited by definition.
>The functions are called under klp_mutex.
>
>Therefore it might be easier to maintain a two parse
>function that would fill static buffers and return
>error in case of wrong value.
>
>I mean something like:
>
>static char mod_name[MODULE_NAME_LEN + 1];
>static char symbol_name[KSYM_SYMBOL_LEN + 1];
>
>static int
>klp_parse_symbol_format(struct module *pmod, Elf_Sym *sym,
>			char *mod_name, char *symbol_name,
>			int *sympos)
>{
>	char *substr;
>
>
>	/*
>	 * Livepatch symbol names must follow this format:
>	 * .klp.sym.objname.symbol_name,sympos
>	 */
>	s = pmod->strtab + sym->st_name;
>	/* [.klp.sym.]objname.symbol_name,sympos */
>	if (!s || strncmp(s, KLP_SYM_PREFIX, KLP_SYM_PREFIX_LEN))
>		return -EINVAL;
>
>	/* .klp.sym.[objname].symbol_name,sympos */
>	substr = s + KLP_SYM_PREFIX_LEN;
>	len = strcspn(substr, ".");
>	if (!len || len > MODULE_NAME_LEN)
>		return -EINVAL;
>	strncpy(mod_name, substr, len);
>	mod_name[len] = '\0';
>
>	/* .klp.sym.objname.[symbol_name],sympos */
>	substr = substr + len + 1;
>	len = strcspn(substr, ",");
>	if (!len || len > KSYM_SYMBOL_LEN)
>		return -EINVAL;
>	strncpy(symbol_name, substr, len);
>	mod_name[len] = '\0';
>
>	/* .klp.sym.objname.symbol_name,[sympos] */
>	substr = substr + len + 1;
>	len = strlen(substr);
>	if (!len)
>		return -EINVAL;
>
>	return kstrtol(substr, 10, sympos);
>}
>
>How does that sound, please?

I like this idea, it is a lot cleaner and more concise
than using all those helper functions. :-)

Thanks,
Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [08/02/16 14:26 -0600]:
>On Wed, Feb 03, 2016 at 08:11:09PM -0500, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the need
>> for architecture specific relocation code in livepatch. Specifically, reuse
>> the apply_relocate_add() function in the module loader to write relocations
>> instead of duplicating functionality in livepatch's arch-dependent
>> klp_write_module_reloc() function.
>>
>> In order to accomplish this, livepatch modules manage their own relocation
>> sections (marked with the SHF_RELA_LIVEPATCH section flag) and
>> livepatch-specific symbols (marked with SHN_LIVEPATCH symbol section
>> index). To apply livepatch relocation sections, livepatch symbols
>> referenced by relocs are resolved and then apply_relocate_add() is called
>> to apply those relocations.
>>
>> In addition, remove x86 livepatch relocation code and the s390
>> klp_write_module_reloc() function stub. They are no longer needed since
>> relocation work has been offloaded to module loader.
>>
>> Signed-off-by: Jessica Yu <jeyu@redhat.com>
>> ---
>>  arch/s390/include/asm/livepatch.h |   7 -
>>  arch/x86/include/asm/livepatch.h  |   2 -
>>  arch/x86/kernel/Makefile          |   1 -
>>  arch/x86/kernel/livepatch.c       |  70 ----------
>>  include/linux/livepatch.h         |  30 ++--
>>  kernel/livepatch/core.c           | 283 ++++++++++++++++++++++++++++++--------
>>  6 files changed, 238 insertions(+), 155 deletions(-)
>>  delete mode 100644 arch/x86/kernel/livepatch.c
>>
>> diff --git a/arch/s390/include/asm/livepatch.h b/arch/s390/include/asm/livepatch.h
>> index a52b6cc..350a751 100644
>> --- a/arch/s390/include/asm/livepatch.h
>> +++ b/arch/s390/include/asm/livepatch.h
>> @@ -25,13 +25,6 @@ static inline int klp_check_compiler_support(void)
>>  	return 0;
>>  }
>>
>> -static inline int klp_write_module_reloc(struct module *mod, unsigned long
>> -		type, unsigned long loc, unsigned long value)
>> -{
>> -	/* not supported yet */
>> -	return -ENOSYS;
>> -}
>> -
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>>  	regs->psw.addr = ip;
>> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> index e795f52..d7c2b57 100644
>> --- a/arch/x86/include/asm/livepatch.h
>> +++ b/arch/x86/include/asm/livepatch.h
>> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>  #endif
>>  	return 0;
>>  }
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value);
>>
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> index b1b78ff..c5e9a5c 100644
>> --- a/arch/x86/kernel/Makefile
>> +++ b/arch/x86/kernel/Makefile
>> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>  obj-y				+= apic/
>>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> deleted file mode 100644
>> index 92fc1a5..0000000
>> --- a/arch/x86/kernel/livepatch.c
>> +++ /dev/null
>> @@ -1,70 +0,0 @@
>> -/*
>> - * livepatch.c - x86-specific Kernel Live Patching Core
>> - *
>> - * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
>> - * Copyright (C) 2014 SUSE
>> - *
>> - * This program is free software; you can redistribute it and/or
>> - * modify it under the terms of the GNU General Public License
>> - * as published by the Free Software Foundation; either version 2
>> - * of the License, or (at your option) any later version.
>> - *
>> - * This program is distributed in the hope that it will be useful,
>> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> - * GNU General Public License for more details.
>> - *
>> - * You should have received a copy of the GNU General Public License
>> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> - */
>> -
>> -#include <linux/module.h>
>> -#include <linux/uaccess.h>
>> -#include <asm/elf.h>
>> -#include <asm/livepatch.h>
>> -
>> -/**
>> - * klp_write_module_reloc() - write a relocation in a module
>> - * @mod:	module in which the section to be modified is found
>> - * @type:	ELF relocation type (see asm/elf.h)
>> - * @loc:	address that the relocation should be written to
>> - * @value:	relocation value (sym address + addend)
>> - *
>> - * This function writes a relocation to the specified location for
>> - * a particular module.
>> - */
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value)
>> -{
>> -	size_t size = 4;
>> -	unsigned long val;
>> -	unsigned long core = (unsigned long)mod->core_layout.base;
>> -	unsigned long core_size = mod->core_layout.size;
>> -
>> -	switch (type) {
>> -	case R_X86_64_NONE:
>> -		return 0;
>> -	case R_X86_64_64:
>> -		val = value;
>> -		size = 8;
>> -		break;
>> -	case R_X86_64_32:
>> -		val = (u32)value;
>> -		break;
>> -	case R_X86_64_32S:
>> -		val = (s32)value;
>> -		break;
>> -	case R_X86_64_PC32:
>> -		val = (u32)(value - loc);
>> -		break;
>> -	default:
>> -		/* unsupported relocation type */
>> -		return -EINVAL;
>> -	}
>> -
>> -	if (loc < core || loc >= core + core_size)
>> -		/* loc does not point to any symbol inside the module */
>> -		return -EINVAL;
>> -
>> -	return probe_kernel_write((void *)loc, &val, size);
>> -}
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index fdd5f1c..1a40a72 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -65,27 +65,8 @@ struct klp_func {
>>  };
>>
>>  /**
>> - * struct klp_reloc - relocation structure for live patching
>> - * @loc:	address where the relocation will be written
>> - * @sympos:	position in kallsyms to disambiguate symbols (optional)
>> - * @type:	ELF relocation type
>> - * @name:	name of the referenced symbol (for lookup/verification)
>> - * @addend:	offset from the referenced symbol
>> - * @external:	symbol is either exported or within the live patch module itself
>> - */
>> -struct klp_reloc {
>> -	unsigned long loc;
>> -	unsigned long sympos;
>> -	unsigned long type;
>> -	const char *name;
>> -	int addend;
>> -	int external;
>> -};
>> -
>> -/**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +76,6 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>>  	struct klp_func *funcs;
>>
>>  	/* internal */
>> @@ -123,6 +103,16 @@ struct klp_patch {
>>  	enum klp_state state;
>>  };
>>
>> +/*
>> + * Livepatch symbol and relocation section prefixes:
>> + * ".klp.rela." for relocation sections
>> + * ".klp.sym." for livepatch symbols
>> + */
>> +#define KLP_SYM_PREFIX ".klp.sym."
>> +#define KLP_SYM_PREFIX_LEN 9
>> +#define KLP_RELASEC_PREFIX ".klp.rela."
>> +#define KLP_RELASEC_PREFIX_LEN 10
>> +
>>  #define klp_for_each_object(patch, obj) \
>>  	for (obj = patch->objs; obj->funcs; obj++)
>>
>> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> index 7aa975d..c1fe57c 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,9 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <linux/string.h>
>> +#include <linux/moduleloader.h>
>>  #include <asm/cacheflush.h>
>>
>>  /**
>> @@ -87,6 +90,166 @@ static bool klp_is_object_loaded(struct klp_object *obj)
>>  	return !obj->name || obj->mod;
>>  }
>>
>> +/*
>> + * Check if a livepatch symbol is formatted properly.
>> + *
>> + * See Documentation/livepatch/module-elf-format.txt for a
>> + * detailed outline of requirements.
>> + */
>> +static int klp_check_symbol_format(struct module *pmod, Elf_Sym *sym)
>> +{
>> +	size_t len;
>> +	char *s, *objname, *symname;
>> +
>> +	if (sym->st_shndx != SHN_LIVEPATCH)
>> +		return -EINVAL;
>> +
>> +	/*
>> +	 * Livepatch symbol names must follow this format:
>> +	 * .klp.sym.objname.symbol_name,sympos
>> +	 */
>> +	s = pmod->strtab + sym->st_name;
>> +	/* [.klp.sym.]objname.symbol_name,sympos */
>> +	if (!s || strncmp(s, KLP_SYM_PREFIX, KLP_SYM_PREFIX_LEN))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname, ".");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.objname.symbol_name,[sympos] */
>> +	if (!strchr(s, ','))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.objname.[symbol_name],sympos */
>> +	symname = objname + len + 1;
>> +	len = strcspn(symname, ",");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	return 0;
>> +}
>> +
>> +/*
>> + * Check if a livepatch relocation section is formatted properly.
>> + *
>> + * See Documentation/livepatch/module-elf-format.txt for a
>> + * detailed outline of requirements.
>> + */
>> +static int klp_check_relasec_format(struct module *pmod, Elf_Shdr *relasec)
>> +{
>> +	char *secname;
>> +	size_t len;
>> +
>> +	secname = pmod->klp_info->secstrings + relasec->sh_name;
>> +	/* [.klp.rela.]objname.section_name */
>> +	if (!secname || strncmp(secname, KLP_RELASEC_PREFIX,
>> +				KLP_RELASEC_PREFIX_LEN))
>> +		return -EINVAL;
>> +
>> +	/* .klp.rela.[objname].section_name */
>> +	len = strcspn(secname + KLP_RELASEC_PREFIX_LEN, ".");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	return 0;
>> +}
>> +
>> +/*
>> + * Check if obj->name matches the objname encoded in the rela
>> + * section name (.klp.rela.[objname].section_name)
>> + *
>> + * Must pass klp_check_relasec_format() before calling this.
>> + */
>> +static bool klp_relasec_matches_object(struct module *pmod, Elf_Shdr *relasec,
>> +				       struct klp_object *obj)
>> +{
>> +	size_t len;
>> +	const char *obj_objname, *sec_objname, *secname;
>> +
>> +	secname = pmod->klp_info->secstrings + relasec->sh_name;
>> +	/* .klp.rela.[objname].section_name */
>> +	sec_objname = secname + KLP_RELASEC_PREFIX_LEN;
>> +	obj_objname = klp_is_module(obj) ? obj->name : "vmlinux";
>> +
>> +	/* Get length of the objname encoded in the section name */
>> +	len = strcspn(sec_objname, ".");
>> +
>> +	if (strlen(obj_objname) != len)
>> +		return false;
>> +
>> +	return strncmp(sec_objname, obj_objname, len) ? false : true;
>> +}
>> +
>> +/*
>> + * klp_get_* helper functions
>> + *
>> + * klp_get_* functions extract different components of the name
>> + * of a livepatch symbol. The full symbol name from the strtab
>> + * is passed in as parameter @s, and @result is filled in with
>> + * the extracted component.
>> + *
>> + * These functions assume a correctly formatted symbol and the
>> + * klp_check_symbol_format() test *must* pass before calling any
>> + * of these functions.
>> + */
>> +
>> +/* .klp.sym.[objname].symbol_name,sympos */
>> +static int klp_get_sym_objname(char *s, char **result)
>> +{
>> +	size_t len;
>> +	char *objname, *objname_start;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname_start = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname_start, ".");
>> +	objname = kstrndup(objname_start, len, GFP_KERNEL);
>> +	if (objname == NULL)
>> +		return -ENOMEM;
>> +
>> +	/* klp_find_object_symbol() treats NULL as vmlinux */
>> +	if (!strcmp(objname, "vmlinux")) {
>> +		*result = NULL;
>> +		kfree(objname);
>> +	} else
>> +		*result = objname;
>> +
>> +	return 0;
>> +}
>> +
>> +/* .klp.sym.objname.[symbol_name],sympos */
>> +static int klp_get_symbol_name(char *s, char **result)
>> +{
>> +	size_t len;
>> +	char *objname, *symname;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname, ".");
>> +
>> +	/* .klp.sym.objname.[symbol_name],sympos */
>> +	symname = objname + len + 1;
>> +	len = strcspn(symname, ",");
>> +
>> +	*result = kstrndup(symname, len, GFP_KERNEL);
>> +	if (*result == NULL)
>> +		return -ENOMEM;
>> +
>> +	return 0;
>> +}
>> +
>> +/* .klp.sym.objname.symbol_name,[sympos] */
>> +static int klp_get_sympos(char *s, unsigned long *result)
>> +{
>> +	char *sympos;
>> +
>> +	/* .klp.sym.symbol_name,[sympos] */
>> +	sympos = strchr(s, ',') + 1;
>> +	return kstrtol(sympos, 10, result);
>> +}
>> +
>>  /* sets obj->mod if object is not vmlinux and module is found */
>>  static void klp_find_object_module(struct klp_object *obj)
>>  {
>
>I think all the above string parsing code could be replaced with a
>couple of sscanf() calls.
>
>For example:
>
>	char objname[64], symname[256];
>
>	ret = sscanf(s, ".klp.sym.%63[^.].%255[^,],%u", objname, symname, &sympos);
>	if (ret < 3)
>		// string doesn't match expected format
>
>That would be much simpler.
>
>Only problem is, the kernel version of sscanf() doesn't seem to support
>the '[' conversion specifier.  At least not yet ;-)  Adding support for
>that would be a win-win: less code overall, and the addition of a useful
>scanf feature which could be used by other code.

Ah, I had completely forgotten that we have sscanf() in the kernel :-)
That would look *much* nicer, assuming we can get kernel sscanf() to
support the bracket character classes..

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [08/02/16 14:26 -0600]:
>On Wed, Feb 03, 2016 at 08:11:09PM -0500, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the need
>> for architecture specific relocation code in livepatch. Specifically, reuse
>> the apply_relocate_add() function in the module loader to write relocations
>> instead of duplicating functionality in livepatch's arch-dependent
>> klp_write_module_reloc() function.
>>
>> In order to accomplish this, livepatch modules manage their own relocation
>> sections (marked with the SHF_RELA_LIVEPATCH section flag) and
>> livepatch-specific symbols (marked with SHN_LIVEPATCH symbol section
>> index). To apply livepatch relocation sections, livepatch symbols
>> referenced by relocs are resolved and then apply_relocate_add() is called
>> to apply those relocations.
>>
>> In addition, remove x86 livepatch relocation code and the s390
>> klp_write_module_reloc() function stub. They are no longer needed since
>> relocation work has been offloaded to module loader.
>>
>> Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> ---
>>  arch/s390/include/asm/livepatch.h |   7 -
>>  arch/x86/include/asm/livepatch.h  |   2 -
>>  arch/x86/kernel/Makefile          |   1 -
>>  arch/x86/kernel/livepatch.c       |  70 ----------
>>  include/linux/livepatch.h         |  30 ++--
>>  kernel/livepatch/core.c           | 283 ++++++++++++++++++++++++++++++--------
>>  6 files changed, 238 insertions(+), 155 deletions(-)
>>  delete mode 100644 arch/x86/kernel/livepatch.c
>>
>> diff --git a/arch/s390/include/asm/livepatch.h b/arch/s390/include/asm/livepatch.h
>> index a52b6cc..350a751 100644
>> --- a/arch/s390/include/asm/livepatch.h
>> +++ b/arch/s390/include/asm/livepatch.h
>> @@ -25,13 +25,6 @@ static inline int klp_check_compiler_support(void)
>>  	return 0;
>>  }
>>
>> -static inline int klp_write_module_reloc(struct module *mod, unsigned long
>> -		type, unsigned long loc, unsigned long value)
>> -{
>> -	/* not supported yet */
>> -	return -ENOSYS;
>> -}
>> -
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>>  	regs->psw.addr = ip;
>> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> index e795f52..d7c2b57 100644
>> --- a/arch/x86/include/asm/livepatch.h
>> +++ b/arch/x86/include/asm/livepatch.h
>> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>  #endif
>>  	return 0;
>>  }
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value);
>>
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> index b1b78ff..c5e9a5c 100644
>> --- a/arch/x86/kernel/Makefile
>> +++ b/arch/x86/kernel/Makefile
>> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>  obj-y				+= apic/
>>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> deleted file mode 100644
>> index 92fc1a5..0000000
>> --- a/arch/x86/kernel/livepatch.c
>> +++ /dev/null
>> @@ -1,70 +0,0 @@
>> -/*
>> - * livepatch.c - x86-specific Kernel Live Patching Core
>> - *
>> - * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> - * Copyright (C) 2014 SUSE
>> - *
>> - * This program is free software; you can redistribute it and/or
>> - * modify it under the terms of the GNU General Public License
>> - * as published by the Free Software Foundation; either version 2
>> - * of the License, or (at your option) any later version.
>> - *
>> - * This program is distributed in the hope that it will be useful,
>> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> - * GNU General Public License for more details.
>> - *
>> - * You should have received a copy of the GNU General Public License
>> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> - */
>> -
>> -#include <linux/module.h>
>> -#include <linux/uaccess.h>
>> -#include <asm/elf.h>
>> -#include <asm/livepatch.h>
>> -
>> -/**
>> - * klp_write_module_reloc() - write a relocation in a module
>> - * @mod:	module in which the section to be modified is found
>> - * @type:	ELF relocation type (see asm/elf.h)
>> - * @loc:	address that the relocation should be written to
>> - * @value:	relocation value (sym address + addend)
>> - *
>> - * This function writes a relocation to the specified location for
>> - * a particular module.
>> - */
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value)
>> -{
>> -	size_t size = 4;
>> -	unsigned long val;
>> -	unsigned long core = (unsigned long)mod->core_layout.base;
>> -	unsigned long core_size = mod->core_layout.size;
>> -
>> -	switch (type) {
>> -	case R_X86_64_NONE:
>> -		return 0;
>> -	case R_X86_64_64:
>> -		val = value;
>> -		size = 8;
>> -		break;
>> -	case R_X86_64_32:
>> -		val = (u32)value;
>> -		break;
>> -	case R_X86_64_32S:
>> -		val = (s32)value;
>> -		break;
>> -	case R_X86_64_PC32:
>> -		val = (u32)(value - loc);
>> -		break;
>> -	default:
>> -		/* unsupported relocation type */
>> -		return -EINVAL;
>> -	}
>> -
>> -	if (loc < core || loc >= core + core_size)
>> -		/* loc does not point to any symbol inside the module */
>> -		return -EINVAL;
>> -
>> -	return probe_kernel_write((void *)loc, &val, size);
>> -}
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index fdd5f1c..1a40a72 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -65,27 +65,8 @@ struct klp_func {
>>  };
>>
>>  /**
>> - * struct klp_reloc - relocation structure for live patching
>> - * @loc:	address where the relocation will be written
>> - * @sympos:	position in kallsyms to disambiguate symbols (optional)
>> - * @type:	ELF relocation type
>> - * @name:	name of the referenced symbol (for lookup/verification)
>> - * @addend:	offset from the referenced symbol
>> - * @external:	symbol is either exported or within the live patch module itself
>> - */
>> -struct klp_reloc {
>> -	unsigned long loc;
>> -	unsigned long sympos;
>> -	unsigned long type;
>> -	const char *name;
>> -	int addend;
>> -	int external;
>> -};
>> -
>> -/**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +76,6 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>>  	struct klp_func *funcs;
>>
>>  	/* internal */
>> @@ -123,6 +103,16 @@ struct klp_patch {
>>  	enum klp_state state;
>>  };
>>
>> +/*
>> + * Livepatch symbol and relocation section prefixes:
>> + * ".klp.rela." for relocation sections
>> + * ".klp.sym." for livepatch symbols
>> + */
>> +#define KLP_SYM_PREFIX ".klp.sym."
>> +#define KLP_SYM_PREFIX_LEN 9
>> +#define KLP_RELASEC_PREFIX ".klp.rela."
>> +#define KLP_RELASEC_PREFIX_LEN 10
>> +
>>  #define klp_for_each_object(patch, obj) \
>>  	for (obj = patch->objs; obj->funcs; obj++)
>>
>> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> index 7aa975d..c1fe57c 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,9 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <linux/string.h>
>> +#include <linux/moduleloader.h>
>>  #include <asm/cacheflush.h>
>>
>>  /**
>> @@ -87,6 +90,166 @@ static bool klp_is_object_loaded(struct klp_object *obj)
>>  	return !obj->name || obj->mod;
>>  }
>>
>> +/*
>> + * Check if a livepatch symbol is formatted properly.
>> + *
>> + * See Documentation/livepatch/module-elf-format.txt for a
>> + * detailed outline of requirements.
>> + */
>> +static int klp_check_symbol_format(struct module *pmod, Elf_Sym *sym)
>> +{
>> +	size_t len;
>> +	char *s, *objname, *symname;
>> +
>> +	if (sym->st_shndx != SHN_LIVEPATCH)
>> +		return -EINVAL;
>> +
>> +	/*
>> +	 * Livepatch symbol names must follow this format:
>> +	 * .klp.sym.objname.symbol_name,sympos
>> +	 */
>> +	s = pmod->strtab + sym->st_name;
>> +	/* [.klp.sym.]objname.symbol_name,sympos */
>> +	if (!s || strncmp(s, KLP_SYM_PREFIX, KLP_SYM_PREFIX_LEN))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname, ".");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.objname.symbol_name,[sympos] */
>> +	if (!strchr(s, ','))
>> +		return -EINVAL;
>> +
>> +	/* .klp.sym.objname.[symbol_name],sympos */
>> +	symname = objname + len + 1;
>> +	len = strcspn(symname, ",");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	return 0;
>> +}
>> +
>> +/*
>> + * Check if a livepatch relocation section is formatted properly.
>> + *
>> + * See Documentation/livepatch/module-elf-format.txt for a
>> + * detailed outline of requirements.
>> + */
>> +static int klp_check_relasec_format(struct module *pmod, Elf_Shdr *relasec)
>> +{
>> +	char *secname;
>> +	size_t len;
>> +
>> +	secname = pmod->klp_info->secstrings + relasec->sh_name;
>> +	/* [.klp.rela.]objname.section_name */
>> +	if (!secname || strncmp(secname, KLP_RELASEC_PREFIX,
>> +				KLP_RELASEC_PREFIX_LEN))
>> +		return -EINVAL;
>> +
>> +	/* .klp.rela.[objname].section_name */
>> +	len = strcspn(secname + KLP_RELASEC_PREFIX_LEN, ".");
>> +	if (!(len > 0))
>> +		return -EINVAL;
>> +
>> +	return 0;
>> +}
>> +
>> +/*
>> + * Check if obj->name matches the objname encoded in the rela
>> + * section name (.klp.rela.[objname].section_name)
>> + *
>> + * Must pass klp_check_relasec_format() before calling this.
>> + */
>> +static bool klp_relasec_matches_object(struct module *pmod, Elf_Shdr *relasec,
>> +				       struct klp_object *obj)
>> +{
>> +	size_t len;
>> +	const char *obj_objname, *sec_objname, *secname;
>> +
>> +	secname = pmod->klp_info->secstrings + relasec->sh_name;
>> +	/* .klp.rela.[objname].section_name */
>> +	sec_objname = secname + KLP_RELASEC_PREFIX_LEN;
>> +	obj_objname = klp_is_module(obj) ? obj->name : "vmlinux";
>> +
>> +	/* Get length of the objname encoded in the section name */
>> +	len = strcspn(sec_objname, ".");
>> +
>> +	if (strlen(obj_objname) != len)
>> +		return false;
>> +
>> +	return strncmp(sec_objname, obj_objname, len) ? false : true;
>> +}
>> +
>> +/*
>> + * klp_get_* helper functions
>> + *
>> + * klp_get_* functions extract different components of the name
>> + * of a livepatch symbol. The full symbol name from the strtab
>> + * is passed in as parameter @s, and @result is filled in with
>> + * the extracted component.
>> + *
>> + * These functions assume a correctly formatted symbol and the
>> + * klp_check_symbol_format() test *must* pass before calling any
>> + * of these functions.
>> + */
>> +
>> +/* .klp.sym.[objname].symbol_name,sympos */
>> +static int klp_get_sym_objname(char *s, char **result)
>> +{
>> +	size_t len;
>> +	char *objname, *objname_start;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname_start = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname_start, ".");
>> +	objname = kstrndup(objname_start, len, GFP_KERNEL);
>> +	if (objname == NULL)
>> +		return -ENOMEM;
>> +
>> +	/* klp_find_object_symbol() treats NULL as vmlinux */
>> +	if (!strcmp(objname, "vmlinux")) {
>> +		*result = NULL;
>> +		kfree(objname);
>> +	} else
>> +		*result = objname;
>> +
>> +	return 0;
>> +}
>> +
>> +/* .klp.sym.objname.[symbol_name],sympos */
>> +static int klp_get_symbol_name(char *s, char **result)
>> +{
>> +	size_t len;
>> +	char *objname, *symname;
>> +
>> +	/* .klp.sym.[objname].symbol_name,sympos */
>> +	objname = s + KLP_SYM_PREFIX_LEN;
>> +	len = strcspn(objname, ".");
>> +
>> +	/* .klp.sym.objname.[symbol_name],sympos */
>> +	symname = objname + len + 1;
>> +	len = strcspn(symname, ",");
>> +
>> +	*result = kstrndup(symname, len, GFP_KERNEL);
>> +	if (*result == NULL)
>> +		return -ENOMEM;
>> +
>> +	return 0;
>> +}
>> +
>> +/* .klp.sym.objname.symbol_name,[sympos] */
>> +static int klp_get_sympos(char *s, unsigned long *result)
>> +{
>> +	char *sympos;
>> +
>> +	/* .klp.sym.symbol_name,[sympos] */
>> +	sympos = strchr(s, ',') + 1;
>> +	return kstrtol(sympos, 10, result);
>> +}
>> +
>>  /* sets obj->mod if object is not vmlinux and module is found */
>>  static void klp_find_object_module(struct klp_object *obj)
>>  {
>
>I think all the above string parsing code could be replaced with a
>couple of sscanf() calls.
>
>For example:
>
>	char objname[64], symname[256];
>
>	ret = sscanf(s, ".klp.sym.%63[^.].%255[^,],%u", objname, symname, &sympos);
>	if (ret < 3)
>		// string doesn't match expected format
>
>That would be much simpler.
>
>Only problem is, the kernel version of sscanf() doesn't seem to support
>the '[' conversion specifier.  At least not yet ;-)  Adding support for
>that would be a win-win: less code overall, and the addition of a useful
>scanf feature which could be used by other code.

Ah, I had completely forgotten that we have sscanf() in the kernel :-)
That would look *much* nicer, assuming we can get kernel sscanf() to
support the bracket character classes..

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Petr Mladek [17/12/15 16:45 +0100]:
>On Wed 2015-12-16 00:40:48, Jessica Yu wrote:
>> Turns out the string parsing stuff, even with the help of lib/string.c, doesn't
>> look very pretty. As I'm working on v3, I'm starting to think having
>> klp_write_object_relocations() loop simply through all the elf sections might
>> not be a good idea. Let me explain.
>>
>> I don't like the amount of string manipulation code that would potentially come
>> with this change. Even with a string as simple as ".klp.rela.objname", we'll
>> end up with a bunch of kstrdup's/kmalloc's and kfree's (unless we modify and
>> chop the section name string in place, which I don't think we should do) that
>> are going to be required at every iteration of the loop, all just to be able to
>> call strcmp() and see if we're dealing with a klp rela section that belongs to
>> the object in question. This also leads to more complicated error handling.
>
>I do not think that we need to allocate and free buffers every time
>we compare a substring.
>
>One possibility is to find the position of the substring using
>strchr(). Then you could compare it using strncmp() and pass there
>the pointer where the substring begins.

Hm, yes you're right. Specifically, it looks like strcspn() would also
be useful for this situation (i.e. calculate the length of a substring
that does not contain certain characters); combined with strncmp(),
this should make the string code much simpler, and no more buffer
allocating/freeing. :-)

Jessica

Re: livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Wed 2015-12-16 00:40:48, Jessica Yu wrote:
> Turns out the string parsing stuff, even with the help of lib/string.c, doesn't
> look very pretty. As I'm working on v3, I'm starting to think having
> klp_write_object_relocations() loop simply through all the elf sections might
> not be a good idea. Let me explain.
> 
> I don't like the amount of string manipulation code that would potentially come
> with this change. Even with a string as simple as ".klp.rela.objname", we'll
> end up with a bunch of kstrdup's/kmalloc's and kfree's (unless we modify and
> chop the section name string in place, which I don't think we should do) that
> are going to be required at every iteration of the loop, all just to be able to
> call strcmp() and see if we're dealing with a klp rela section that belongs to
> the object in question. This also leads to more complicated error handling.

I do not think that we need to allocate and free buffers every time
we compare a substring.

One possibility is to find the position of the substring using
strchr(). Then you could compare it using strncmp() and pass there
the pointer where the substring begins.

Alternatively we could use a static buffer or allocate a big enough
one at the beginning. IMHO, we could assume that function name must
fit into 80 characters, otherwise, that code would be unmanageable
(won't deserve patching). If we use a buffer size 256 bytes,
we should be definitely on the safe side.

Just 2 my cents,
Petr

Re: livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Wed 2015-12-16 00:40:48, Jessica Yu wrote:
> Turns out the string parsing stuff, even with the help of lib/string.c, doesn't
> look very pretty. As I'm working on v3, I'm starting to think having
> klp_write_object_relocations() loop simply through all the elf sections might
> not be a good idea. Let me explain.
> 
> I don't like the amount of string manipulation code that would potentially come
> with this change. Even with a string as simple as ".klp.rela.objname", we'll
> end up with a bunch of kstrdup's/kmalloc's and kfree's (unless we modify and
> chop the section name string in place, which I don't think we should do) that
> are going to be required at every iteration of the loop, all just to be able to
> call strcmp() and see if we're dealing with a klp rela section that belongs to
> the object in question. This also leads to more complicated error handling.

I do not think that we need to allocate and free buffers every time
we compare a substring.

One possibility is to find the position of the substring using
strchr(). Then you could compare it using strncmp() and pass there
the pointer where the substring begins.

Alternatively we could use a static buffer or allocate a big enough
one at the beginning. IMHO, we could assume that function name must
fit into 80 characters, otherwise, that code would be unmanageable
(won't deserve patching). If we use a buffer size 256 bytes,
we should be definitely on the safe side.

Just 2 my cents,
Petr

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [16/12/15 13:59 +0100]:
>On Wed, 16 Dec 2015, Jessica Yu wrote:
>
>> +++ Jessica Yu [09/12/15 14:10 -0500]:
>> > +++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>> > >
>> > > There was a lot of discussion for v1, so I'm not sure, but I thought we
>> > > ended up deciding to get rid of the klp_reloc_sec struct?  Instead I
>> > > think the symbol table can be walked directly looking for klp rela
>> > > sections?
>> > >
>> > > The benefits of doing so would be to make the interface simpler -- no
>> > > need to "cache" the section metadata when it's already easily and
>> > > quickly available in the module struct elf section headers.
>> >
>> > Ah, I might have interpreted the conclusions of that last discussion
>> > incorrectly.
>> >
>> > In that case, I think I can just get rid of my klp_for_each_reloc_sec
>> > macro as well as the lreloc scaffolding code from kpatch. The only
>> > potentially "ugly" part of this change is that I'll have to move the
>> > string parsing stuff here to extract the objname of the __klp_rela
>> > section (which may not actually look that bad, we'll see how that
>> > turns out).
>
>Yes, that was a conclusion we made. At least I thought so :)
>
>> Turns out the string parsing stuff, even with the help of lib/string.c,
>> doesn't
>> look very pretty. As I'm working on v3, I'm starting to think having
>> klp_write_object_relocations() loop simply through all the elf sections might
>> not be a good idea. Let me explain.
>
>Hm, I still think it is worth it...
>
>> I don't like the amount of string manipulation code that would potentially
>> come
>> with this change. Even with a string as simple as ".klp.rela.objname", we'll
>> end up with a bunch of kstrdup's/kmalloc's and kfree's (unless we modify and
>> chop the section name string in place, which I don't think we should do) that
>> are going to be required at every iteration of the loop, all just to be able
>> to
>> call strcmp() and see if we're dealing with a klp rela section that belongs to
>> the object in question. This also leads to more complicated error handling.
>
>But this shouldn't be much different from what init function of the patch
>module does now, right? You have a klp_extract_objname() which I thought
>could be moved right to livepatch code.

First, thanks for the comments. I think my only real gripe is that, in the
pseudo code I provided, we are not only doing the for loop for each object, but
also a bunch of kmalloc's/kstrdup's and kfree's just to be able to check if
this klp section belongs to the current object, *and* also kmallocing/kfreeing
just to provide buffers to extract parts of a symbol's name. In the patch
module code, each klp section was parsed just once and sorted by object:
https://github.com/flaming-toast/kpatch/blob/no_dynrela_redux/kmod/patch/livepatch-patch-hook.c#L213
^^^ This was before we decided to get rid of 'external', so only klp sections
had to be parsed. In v3, we are also looking to parse the symbol name (to
eliminate 'external'), so now I'm just trying to reduce all the string stuff to
the bare minimum needed to have this work.

function klp_write_object_relocations(..,struct klp_object *obj,..) {
   for each section in mod->sechdrs { // iterate through all elf sections, no more obj->reloc_secs list
       if not a klp rela section
           continue;
->     sec_objname = extract_objname_from_section(section);  // .klp.rela.[objname].sectionname
->     if (strcmp(sec_objname, obj->name)) { // this klp rela section doesn't belong to this object
->         kfree(sec_objname);
->         continue;
       }

     for each rela in this klp rela section {
        sym = symtab + ELF_R_SYM(rela->r_info);
->      sym_objname = extract_objname_from_symbol(sym) // symname.klp.[objname]
->      if (!sym_objname)
->           goto handle_error; // calls kfree(sec_objname);
->      symname = extract_symname_from_symbol(sym); // [symname].klp.objname
->      if (!symname)
->          goto handle_error; // calls kfree(sec_objname);
        ret = klp_find_object_symbol(sym_objname, symname, &addr)
        if (ret)
->          goto handle_error2; // calls kfree(symname), then kfree(sec_objname)
        sym->st_value = addr;
->      if (sym_objname) // free buffers before proceeding to next iteration of loop
->         kfree(sym_objname);
->      kfree(symname);
     }
[ snipped rest of function ]

I'm trying to figure out a way to simplify all the lines marked with '->', and
make this code as readable and palatable as possible. I think primarily the
kfree's and multiple error-handling goto's bothered me, which is why I
suggested putting the symname.klp.[objname] part in .kpatch.strings and using
those masks I suggested. Then we wouldn't even need to set-up temporary buffers to
hold the sym_objname + symname strings. You're right that this would require
another assumption about the patch module, but I think having the names of
livepatch symbols formatted as 'symname.klp.objname' was already another
assumption, and using .kpatch.strings would just be changing this assumption.

>> In v1, the string parsing was done only *once* for each klp rela section in
>> the
>> patch module code, and each klp rela section is sorted into their respective
>> object with the reloc_secs list. Then all klp_write_object_relocations() had
>> to
>> do is iterate over object->reloc_secs. The tradeoff here is the addition of
>> another struct (klp_reloc_sec), but that is not nearly as bad as string
>> parsing, which is imo way more error prone and is unnecessary extra work.
>
>If it is a problem (and Josh thought it was not if I recall correctly) we
>can cache indices to relevant rela sections in klp_object as we discussed
>in v1.
>
>> Here's some pseudocode to help visualize the potential issues:
>>
>> function klp_write_object_relocations(..,struct klp_object *obj,..) {
>>    for each section in mod->sechdrs { // iterate through all elf sections, no
>> more obj->reloc_secs list
>>        if not a klp rela section
>>            continue;
>>        sec_objname = extract_objname_from_section(section);  //
>> .klp.rela.[objname].sectionname
>>        if (strcmp(sec_objname, obj->name)) { // this klp rela section doesn't
>> belong to this object
>>            kfree(sec_objname);
>>            continue;
>>        }
>>
>>      for each rela in this klp rela section {
>>         sym = symtab + ELF_R_SYM(rela->r_info);
>>         sym_objname = extract_objname_from_symbol(sym) //
>> symname.klp.[objname]
>
>This is because of 'external' stuff, right?

Correct, in the v2 discussion we decided symbol names can have the
names of their objects attached, which will eliminate the need for the
external flag and klp_find_external_symbol().

>>         if (!sym_objname)
>>             goto handle_error; // calls kfree(sec_objname);
>>         symname = extract_symname_from_symbol(sym); // [symname].klp.objname
>
>Can't we use a method from the patch? That is
>
>symname = pmod->core_strtab + sym->st_name;

Since we are removing 'external', we decided to append the objname to the
symbol name. But then extra string parsing is needed to get the real symbol
name. So, pmod->core_strtab + sym->st_name would correspond to the full name,
formatted as 'symname.klp.objname'. We parse to get the real symname and the
objname. Then we no longer need the klp_find_external_symbol() call, it'd just
be one call to klp_find_object_symbol(sym_objname, symname, addr)

>>         if (!symname)
>>             goto handle_error; // calls kfree(sec_objname);
>>         ret = klp_find_object_symbol(sym_objname, symname, &addr)
>>         if (ret)
>>             goto handle_error2; // calls kfree(symname), then
>> kfree(sec_objname)
>>
>> ...etc., you get the idea how messy that is getting, and we haven't even
>> reached the call to apply_relocate_add() yet.
>>
>> So I personally think it is better to keep the old v1 way for applying the klp
>> reloc secs. The sections are still named ".klp.rela.objname" but the parsing
>> is
>> done once in the patch module code and used only to build the patch
>> scaffolding
>> structure.
>>
>> In order to avoid adding any additional string parsing code in v3, I no longer
>> thing we should rename livepatch symbols to include the symbol objname (i.e.
>> 'symbol.klp.objname'). Recall that this change is for getting rid of external
>> dynrelas. Instead of parsing the symbol name, we could just encode 1) the
>> sympos and 2) the index into the .kpatch.strings strtab (that contains the
>> sym_objname) in sym->st_value. We define two masks (KLP_MASK_SYMPOS,
>> KLP_MASK_OBJNAME) that extract these values. st_value has either a 32 bit or
>> 64
>> bit size, both of which are big enough sizes for the sympos and string table
>> index. One perk we lose is being able to see which object a symbol belongs to
>> in readelf, but then again we didn't have this benefit to begin with in
>> v1 nor v2 (where we didn't know the symbol's object and had to use
>> klp_find_external_symbol anyway), so I wouldn't say it's a loss.
>
>Just a note. This would lead to another assumption about a patch
>module... .kpatch.strings. I suspect 'external' symbols are kpatch
>specific (I think I did not deal with it while working on relocations for
>kgraft. But I am not sure now. I haven't finished it too.), but let's try
>to make it "generic".

I think 'external' is kpatch-specific (correct me if  I'm wrong), as I
explained above we're trying to get rid of it.

>> Apologies for the super long explanations, but I think avoiding string parsing
>> in livepatch core code will make the relocation code much more succinct and
>> easier to read. Any objections or thoughts on all this?
>
>My main argument is that the work needs to be done somewhere and there is
>no reason why not do it right in the kernel and not in patch modules. The
>code should be more or less the same and we'd get rid of unnecessary
>layers (klp_reloc_sec) as Josh pointed out.

Right, OK. I agree that this work has to be done somewhere, I am just hesitant
about the new string handling code, as I pointed out above. If people are OK and
agree that this is the better approach, then I will keep it.

Thanks again,
Jessica

Re: livepatch: reuse module loader code to write relocations

[Miroslav Benes]

On Wed, 16 Dec 2015, Jessica Yu wrote:

> +++ Jessica Yu [09/12/15 14:10 -0500]:
> > +++ Josh Poimboeuf [08/12/15 12:38 -0600]:
> > > 
> > > There was a lot of discussion for v1, so I'm not sure, but I thought we
> > > ended up deciding to get rid of the klp_reloc_sec struct?  Instead I
> > > think the symbol table can be walked directly looking for klp rela
> > > sections?
> > > 
> > > The benefits of doing so would be to make the interface simpler -- no
> > > need to "cache" the section metadata when it's already easily and
> > > quickly available in the module struct elf section headers.
> > 
> > Ah, I might have interpreted the conclusions of that last discussion
> > incorrectly.
> > 
> > In that case, I think I can just get rid of my klp_for_each_reloc_sec
> > macro as well as the lreloc scaffolding code from kpatch. The only
> > potentially "ugly" part of this change is that I'll have to move the
> > string parsing stuff here to extract the objname of the __klp_rela
> > section (which may not actually look that bad, we'll see how that
> > turns out).

Yes, that was a conclusion we made. At least I thought so :)

> Turns out the string parsing stuff, even with the help of lib/string.c,
> doesn't
> look very pretty. As I'm working on v3, I'm starting to think having
> klp_write_object_relocations() loop simply through all the elf sections might
> not be a good idea. Let me explain.

Hm, I still think it is worth it...

> I don't like the amount of string manipulation code that would potentially
> come
> with this change. Even with a string as simple as ".klp.rela.objname", we'll
> end up with a bunch of kstrdup's/kmalloc's and kfree's (unless we modify and
> chop the section name string in place, which I don't think we should do) that
> are going to be required at every iteration of the loop, all just to be able
> to
> call strcmp() and see if we're dealing with a klp rela section that belongs to
> the object in question. This also leads to more complicated error handling.

But this shouldn't be much different from what init function of the patch 
module does now, right? You have a klp_extract_objname() which I thought 
could be moved right to livepatch code.

> In v1, the string parsing was done only *once* for each klp rela section in
> the
> patch module code, and each klp rela section is sorted into their respective
> object with the reloc_secs list. Then all klp_write_object_relocations() had
> to
> do is iterate over object->reloc_secs. The tradeoff here is the addition of
> another struct (klp_reloc_sec), but that is not nearly as bad as string
> parsing, which is imo way more error prone and is unnecessary extra work.

If it is a problem (and Josh thought it was not if I recall correctly) we 
can cache indices to relevant rela sections in klp_object as we discussed 
in v1.
 
> Here's some pseudocode to help visualize the potential issues:
> 
> function klp_write_object_relocations(..,struct klp_object *obj,..) {
>    for each section in mod->sechdrs { // iterate through all elf sections, no
> more obj->reloc_secs list
>        if not a klp rela section
>            continue;
>        sec_objname = extract_objname_from_section(section);  //
> .klp.rela.[objname].sectionname
>        if (strcmp(sec_objname, obj->name)) { // this klp rela section doesn't
> belong to this object
>            kfree(sec_objname);
>            continue;
>        }
> 
>      for each rela in this klp rela section {
>         sym = symtab + ELF_R_SYM(rela->r_info);
>         sym_objname = extract_objname_from_symbol(sym) //
> symname.klp.[objname]

This is because of 'external' stuff, right?

>         if (!sym_objname)
>             goto handle_error; // calls kfree(sec_objname);
>         symname = extract_symname_from_symbol(sym); // [symname].klp.objname

Can't we use a method from the patch? That is 

symname = pmod->core_strtab + sym->st_name;

>         if (!symname)
>             goto handle_error; // calls kfree(sec_objname);
>         ret = klp_find_object_symbol(sym_objname, symname, &addr)
>         if (ret)
>             goto handle_error2; // calls kfree(symname), then
> kfree(sec_objname)
> 
> ...etc., you get the idea how messy that is getting, and we haven't even
> reached the call to apply_relocate_add() yet.
> 
> So I personally think it is better to keep the old v1 way for applying the klp
> reloc secs. The sections are still named ".klp.rela.objname" but the parsing
> is
> done once in the patch module code and used only to build the patch
> scaffolding
> structure.
> 
> In order to avoid adding any additional string parsing code in v3, I no longer
> thing we should rename livepatch symbols to include the symbol objname (i.e.
> 'symbol.klp.objname'). Recall that this change is for getting rid of external
> dynrelas. Instead of parsing the symbol name, we could just encode 1) the
> sympos and 2) the index into the .kpatch.strings strtab (that contains the
> sym_objname) in sym->st_value. We define two masks (KLP_MASK_SYMPOS,
> KLP_MASK_OBJNAME) that extract these values. st_value has either a 32 bit or
> 64
> bit size, both of which are big enough sizes for the sympos and string table
> index. One perk we lose is being able to see which object a symbol belongs to
> in readelf, but then again we didn't have this benefit to begin with in
> v1 nor v2 (where we didn't know the symbol's object and had to use
> klp_find_external_symbol anyway), so I wouldn't say it's a loss.

Just a note. This would lead to another assumption about a patch 
module... .kpatch.strings. I suspect 'external' symbols are kpatch 
specific (I think I did not deal with it while working on relocations for 
kgraft. But I am not sure now. I haven't finished it too.), but let's try 
to make it "generic".

> Apologies for the super long explanations, but I think avoiding string parsing
> in livepatch core code will make the relocation code much more succinct and
> easier to read. Any objections or thoughts on all this?

My main argument is that the work needs to be done somewhere and there is 
no reason why not do it right in the kernel and not in patch modules. The 
code should be more or less the same and we'd get rid of unnecessary 
layers (klp_reloc_sec) as Josh pointed out.

Miroslav

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Jessica Yu [09/12/15 14:10 -0500]:
>+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>>On Mon, Nov 30, 2015 at 11:21:17PM -0500, Jessica Yu wrote:
>>>Reuse module loader code to write relocations, thereby eliminating the need
>>>for architecture specific relocation code in livepatch. Namely, we reuse
>>>apply_relocate_add() in the module loader to write relocations instead of
>>>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>>>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>>>are resolved and then apply_relocate_add() is called to apply those
>>>relocations.
>>>
>>>In addition, remove x86 livepatch relocation code. It is no longer needed
>>>since symbol resolution and relocation work have been offloaded to module
>>>loader.
>>>
>>>Signed-off-by: Jessica Yu <jeyu@redhat.com>
>>>---
>>> arch/x86/include/asm/livepatch.h |  2 -
>>> arch/x86/kernel/Makefile         |  1 -
>>> arch/x86/kernel/livepatch.c      | 91 --------------------------------------
>>> include/linux/livepatch.h        | 30 ++++++-------
>>> include/linux/module.h           |  6 +++
>>> kernel/livepatch/core.c          | 94 ++++++++++++++++++++++------------------
>>> 6 files changed, 70 insertions(+), 154 deletions(-)
>>> delete mode 100644 arch/x86/kernel/livepatch.c
>>>
>>>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>>>index 19c099a..7312e25 100644
>>>--- a/arch/x86/include/asm/livepatch.h
>>>+++ b/arch/x86/include/asm/livepatch.h
>>>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>> #endif
>>> 	return 0;
>>> }
>>>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>>>-			   unsigned long loc, unsigned long value);
>>>
>>> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>> {
>>>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>>>index b1b78ff..c5e9a5c 100644
>>>--- a/arch/x86/kernel/Makefile
>>>+++ b/arch/x86/kernel/Makefile
>>>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>> obj-y				+= apic/
>>> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>>>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>>>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>>>deleted file mode 100644
>>>index d1d35cc..0000000
>>>--- a/arch/x86/kernel/livepatch.c
>>>+++ /dev/null
>>>@@ -1,91 +0,0 @@
>>>-/*
>>>- * livepatch.c - x86-specific Kernel Live Patching Core
>>>- *
>>>- * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
>>>- * Copyright (C) 2014 SUSE
>>>- *
>>>- * This program is free software; you can redistribute it and/or
>>>- * modify it under the terms of the GNU General Public License
>>>- * as published by the Free Software Foundation; either version 2
>>>- * of the License, or (at your option) any later version.
>>>- *
>>>- * This program is distributed in the hope that it will be useful,
>>>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>- * GNU General Public License for more details.
>>>- *
>>>- * You should have received a copy of the GNU General Public License
>>>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
>>>- */
>>>-
>>>-#include <linux/module.h>
>>>-#include <linux/uaccess.h>
>>>-#include <asm/cacheflush.h>
>>>-#include <asm/page_types.h>
>>>-#include <asm/elf.h>
>>>-#include <asm/livepatch.h>
>>>-
>>>-/**
>>>- * klp_write_module_reloc() - write a relocation in a module
>>>- * @mod:	module in which the section to be modified is found
>>>- * @type:	ELF relocation type (see asm/elf.h)
>>>- * @loc:	address that the relocation should be written to
>>>- * @value:	relocation value (sym address + addend)
>>>- *
>>>- * This function writes a relocation to the specified location for
>>>- * a particular module.
>>>- */
>>>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>>>-			   unsigned long loc, unsigned long value)
>>>-{
>>>-	int ret, numpages, size = 4;
>>>-	bool readonly;
>>>-	unsigned long val;
>>>-	unsigned long core = (unsigned long)mod->module_core;
>>>-	unsigned long core_size = mod->core_size;
>>>-
>>>-	switch (type) {
>>>-	case R_X86_64_NONE:
>>>-		return 0;
>>>-	case R_X86_64_64:
>>>-		val = value;
>>>-		size = 8;
>>>-		break;
>>>-	case R_X86_64_32:
>>>-		val = (u32)value;
>>>-		break;
>>>-	case R_X86_64_32S:
>>>-		val = (s32)value;
>>>-		break;
>>>-	case R_X86_64_PC32:
>>>-		val = (u32)(value - loc);
>>>-		break;
>>>-	default:
>>>-		/* unsupported relocation type */
>>>-		return -EINVAL;
>>>-	}
>>>-
>>>-	if (loc < core || loc >= core + core_size)
>>>-		/* loc does not point to any symbol inside the module */
>>>-		return -EINVAL;
>>>-
>>>-	readonly = false;
>>>-
>>>-#ifdef CONFIG_DEBUG_SET_MODULE_RONX
>>>-	if (loc < core + mod->core_ro_size)
>>>-		readonly = true;
>>>-#endif
>>>-
>>>-	/* determine if the relocation spans a page boundary */
>>>-	numpages = ((loc & PAGE_MASK) == ((loc + size) & PAGE_MASK)) ? 1 : 2;
>>>-
>>>-	if (readonly)
>>>-		set_memory_rw(loc & PAGE_MASK, numpages);
>>>-
>>>-	ret = probe_kernel_write((void *)loc, &val, size);
>>>-
>>>-	if (readonly)
>>>-		set_memory_ro(loc & PAGE_MASK, numpages);
>>>-
>>>-	return ret;
>>>-}
>>>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>>>index 31db7a0..54f62a6 100644
>>>--- a/include/linux/livepatch.h
>>>+++ b/include/linux/livepatch.h
>>>@@ -64,28 +64,21 @@ struct klp_func {
>>> };
>>>
>>> /**
>>>- * struct klp_reloc - relocation structure for live patching
>>>- * @loc:	address where the relocation will be written
>>>- * @val:	address of the referenced symbol (optional,
>>>- *		vmlinux	patches only)
>>>- * @type:	ELF relocation type
>>>- * @name:	name of the referenced symbol (for lookup/verification)
>>>- * @addend:	offset from the referenced symbol
>>>- * @external:	symbol is either exported or within the live patch module itself
>>>+ * struct klp_reloc_sec - relocation section struct for live patching
>>>+ * @index:	Elf section index of the relocation section
>>>+ * @name:	name of the relocation section
>>>+ * @objname:	name of the object associated with the klp reloc section
>>>  */
>>>-struct klp_reloc {
>>>-	unsigned long loc;
>>>-	unsigned long val;
>>>-	unsigned long type;
>>>-	const char *name;
>>>-	int addend;
>>>-	int external;
>>>+struct klp_reloc_sec {
>>>+	unsigned int index;
>>>+	char *name;
>>>+	char *objname;
>>> };
>>>
>>> /**
>>>  * struct klp_object - kernel object structure for live patching
>>>  * @name:	module name (or NULL for vmlinux)
>>>- * @relocs:	relocation entries to be applied at load time
>>>+ * @reloc_secs:	relocation sections to be applied at load time
>>>  * @funcs:	function entries for functions to be patched in the object
>>>  * @kobj:	kobject for sysfs resources
>>>  * @mod:	kernel module associated with the patched object
>>>@@ -95,7 +88,7 @@ struct klp_reloc {
>>> struct klp_object {
>>> 	/* external */
>>> 	const char *name;
>>>-	struct klp_reloc *relocs;
>>>+	struct klp_reloc_sec *reloc_secs;
>>
>>There was a lot of discussion for v1, so I'm not sure, but I thought we
>>ended up deciding to get rid of the klp_reloc_sec struct?  Instead I
>>think the symbol table can be walked directly looking for klp rela
>>sections?
>>
>>The benefits of doing so would be to make the interface simpler -- no
>>need to "cache" the section metadata when it's already easily and
>>quickly available in the module struct elf section headers.
>
>Ah, I might have interpreted the conclusions of that last discussion
>incorrectly.
>
>In that case, I think I can just get rid of my klp_for_each_reloc_sec
>macro as well as the lreloc scaffolding code from kpatch. The only
>potentially "ugly" part of this change is that I'll have to move the
>string parsing stuff here to extract the objname of the __klp_rela
>section (which may not actually look that bad, we'll see how that
>turns out).

Turns out the string parsing stuff, even with the help of lib/string.c, doesn't
look very pretty. As I'm working on v3, I'm starting to think having
klp_write_object_relocations() loop simply through all the elf sections might
not be a good idea. Let me explain.

I don't like the amount of string manipulation code that would potentially come
with this change. Even with a string as simple as ".klp.rela.objname", we'll
end up with a bunch of kstrdup's/kmalloc's and kfree's (unless we modify and
chop the section name string in place, which I don't think we should do) that
are going to be required at every iteration of the loop, all just to be able to
call strcmp() and see if we're dealing with a klp rela section that belongs to
the object in question. This also leads to more complicated error handling.

In v1, the string parsing was done only *once* for each klp rela section in the
patch module code, and each klp rela section is sorted into their respective
object with the reloc_secs list. Then all klp_write_object_relocations() had to
do is iterate over object->reloc_secs. The tradeoff here is the addition of
another struct (klp_reloc_sec), but that is not nearly as bad as string
parsing, which is imo way more error prone and is unnecessary extra work.

Here's some pseudocode to help visualize the potential issues:

function klp_write_object_relocations(..,struct klp_object *obj,..) {
    for each section in mod->sechdrs { // iterate through all elf sections, no more obj->reloc_secs list
        if not a klp rela section
            continue;
        sec_objname = extract_objname_from_section(section);  // .klp.rela.[objname].sectionname
        if (strcmp(sec_objname, obj->name)) { // this klp rela section doesn't belong to this object
            kfree(sec_objname);
            continue;
        }

      for each rela in this klp rela section {
         sym = symtab + ELF_R_SYM(rela->r_info);
         sym_objname = extract_objname_from_symbol(sym) // symname.klp.[objname]
         if (!sym_objname)
             goto handle_error; // calls kfree(sec_objname);
         symname = extract_symname_from_symbol(sym); // [symname].klp.objname
         if (!symname)
             goto handle_error; // calls kfree(sec_objname);
         ret = klp_find_object_symbol(sym_objname, symname, &addr)
         if (ret)
             goto handle_error2; // calls kfree(symname), then kfree(sec_objname)

...etc., you get the idea how messy that is getting, and we haven't even
reached the call to apply_relocate_add() yet.

So I personally think it is better to keep the old v1 way for applying the klp
reloc secs. The sections are still named ".klp.rela.objname" but the parsing is
done once in the patch module code and used only to build the patch scaffolding
structure.

In order to avoid adding any additional string parsing code in v3, I no longer
thing we should rename livepatch symbols to include the symbol objname (i.e.
'symbol.klp.objname'). Recall that this change is for getting rid of external
dynrelas. Instead of parsing the symbol name, we could just encode 1) the
sympos and 2) the index into the .kpatch.strings strtab (that contains the
sym_objname) in sym->st_value. We define two masks (KLP_MASK_SYMPOS,
KLP_MASK_OBJNAME) that extract these values. st_value has either a 32 bit or 64
bit size, both of which are big enough sizes for the sympos and string table
index. One perk we lose is being able to see which object a symbol belongs to
in readelf, but then again we didn't have this benefit to begin with in
v1 nor v2 (where we didn't know the symbol's object and had to use
klp_find_external_symbol anyway), so I wouldn't say it's a loss.

Apologies for the super long explanations, but I think avoiding string parsing
in livepatch core code will make the relocation code much more succinct and
easier to read. Any objections or thoughts on all this?

Thanks,
Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Jessica Yu [09/12/15 14:10 -0500]:
>+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>>On Mon, Nov 30, 2015 at 11:21:17PM -0500, Jessica Yu wrote:
>>>Reuse module loader code to write relocations, thereby eliminating the need
>>>for architecture specific relocation code in livepatch. Namely, we reuse
>>>apply_relocate_add() in the module loader to write relocations instead of
>>>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>>>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>>>are resolved and then apply_relocate_add() is called to apply those
>>>relocations.
>>>
>>>In addition, remove x86 livepatch relocation code. It is no longer needed
>>>since symbol resolution and relocation work have been offloaded to module
>>>loader.
>>>
>>>Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>>>---
>>> arch/x86/include/asm/livepatch.h |  2 -
>>> arch/x86/kernel/Makefile         |  1 -
>>> arch/x86/kernel/livepatch.c      | 91 --------------------------------------
>>> include/linux/livepatch.h        | 30 ++++++-------
>>> include/linux/module.h           |  6 +++
>>> kernel/livepatch/core.c          | 94 ++++++++++++++++++++++------------------
>>> 6 files changed, 70 insertions(+), 154 deletions(-)
>>> delete mode 100644 arch/x86/kernel/livepatch.c
>>>
>>>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>>>index 19c099a..7312e25 100644
>>>--- a/arch/x86/include/asm/livepatch.h
>>>+++ b/arch/x86/include/asm/livepatch.h
>>>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>> #endif
>>> 	return 0;
>>> }
>>>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>>>-			   unsigned long loc, unsigned long value);
>>>
>>> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>> {
>>>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>>>index b1b78ff..c5e9a5c 100644
>>>--- a/arch/x86/kernel/Makefile
>>>+++ b/arch/x86/kernel/Makefile
>>>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>> obj-y				+= apic/
>>> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>>>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>>>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>>>deleted file mode 100644
>>>index d1d35cc..0000000
>>>--- a/arch/x86/kernel/livepatch.c
>>>+++ /dev/null
>>>@@ -1,91 +0,0 @@
>>>-/*
>>>- * livepatch.c - x86-specific Kernel Live Patching Core
>>>- *
>>>- * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>>>- * Copyright (C) 2014 SUSE
>>>- *
>>>- * This program is free software; you can redistribute it and/or
>>>- * modify it under the terms of the GNU General Public License
>>>- * as published by the Free Software Foundation; either version 2
>>>- * of the License, or (at your option) any later version.
>>>- *
>>>- * This program is distributed in the hope that it will be useful,
>>>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>- * GNU General Public License for more details.
>>>- *
>>>- * You should have received a copy of the GNU General Public License
>>>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
>>>- */
>>>-
>>>-#include <linux/module.h>
>>>-#include <linux/uaccess.h>
>>>-#include <asm/cacheflush.h>
>>>-#include <asm/page_types.h>
>>>-#include <asm/elf.h>
>>>-#include <asm/livepatch.h>
>>>-
>>>-/**
>>>- * klp_write_module_reloc() - write a relocation in a module
>>>- * @mod:	module in which the section to be modified is found
>>>- * @type:	ELF relocation type (see asm/elf.h)
>>>- * @loc:	address that the relocation should be written to
>>>- * @value:	relocation value (sym address + addend)
>>>- *
>>>- * This function writes a relocation to the specified location for
>>>- * a particular module.
>>>- */
>>>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>>>-			   unsigned long loc, unsigned long value)
>>>-{
>>>-	int ret, numpages, size = 4;
>>>-	bool readonly;
>>>-	unsigned long val;
>>>-	unsigned long core = (unsigned long)mod->module_core;
>>>-	unsigned long core_size = mod->core_size;
>>>-
>>>-	switch (type) {
>>>-	case R_X86_64_NONE:
>>>-		return 0;
>>>-	case R_X86_64_64:
>>>-		val = value;
>>>-		size = 8;
>>>-		break;
>>>-	case R_X86_64_32:
>>>-		val = (u32)value;
>>>-		break;
>>>-	case R_X86_64_32S:
>>>-		val = (s32)value;
>>>-		break;
>>>-	case R_X86_64_PC32:
>>>-		val = (u32)(value - loc);
>>>-		break;
>>>-	default:
>>>-		/* unsupported relocation type */
>>>-		return -EINVAL;
>>>-	}
>>>-
>>>-	if (loc < core || loc >= core + core_size)
>>>-		/* loc does not point to any symbol inside the module */
>>>-		return -EINVAL;
>>>-
>>>-	readonly = false;
>>>-
>>>-#ifdef CONFIG_DEBUG_SET_MODULE_RONX
>>>-	if (loc < core + mod->core_ro_size)
>>>-		readonly = true;
>>>-#endif
>>>-
>>>-	/* determine if the relocation spans a page boundary */
>>>-	numpages = ((loc & PAGE_MASK) == ((loc + size) & PAGE_MASK)) ? 1 : 2;
>>>-
>>>-	if (readonly)
>>>-		set_memory_rw(loc & PAGE_MASK, numpages);
>>>-
>>>-	ret = probe_kernel_write((void *)loc, &val, size);
>>>-
>>>-	if (readonly)
>>>-		set_memory_ro(loc & PAGE_MASK, numpages);
>>>-
>>>-	return ret;
>>>-}
>>>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>>>index 31db7a0..54f62a6 100644
>>>--- a/include/linux/livepatch.h
>>>+++ b/include/linux/livepatch.h
>>>@@ -64,28 +64,21 @@ struct klp_func {
>>> };
>>>
>>> /**
>>>- * struct klp_reloc - relocation structure for live patching
>>>- * @loc:	address where the relocation will be written
>>>- * @val:	address of the referenced symbol (optional,
>>>- *		vmlinux	patches only)
>>>- * @type:	ELF relocation type
>>>- * @name:	name of the referenced symbol (for lookup/verification)
>>>- * @addend:	offset from the referenced symbol
>>>- * @external:	symbol is either exported or within the live patch module itself
>>>+ * struct klp_reloc_sec - relocation section struct for live patching
>>>+ * @index:	Elf section index of the relocation section
>>>+ * @name:	name of the relocation section
>>>+ * @objname:	name of the object associated with the klp reloc section
>>>  */
>>>-struct klp_reloc {
>>>-	unsigned long loc;
>>>-	unsigned long val;
>>>-	unsigned long type;
>>>-	const char *name;
>>>-	int addend;
>>>-	int external;
>>>+struct klp_reloc_sec {
>>>+	unsigned int index;
>>>+	char *name;
>>>+	char *objname;
>>> };
>>>
>>> /**
>>>  * struct klp_object - kernel object structure for live patching
>>>  * @name:	module name (or NULL for vmlinux)
>>>- * @relocs:	relocation entries to be applied at load time
>>>+ * @reloc_secs:	relocation sections to be applied at load time
>>>  * @funcs:	function entries for functions to be patched in the object
>>>  * @kobj:	kobject for sysfs resources
>>>  * @mod:	kernel module associated with the patched object
>>>@@ -95,7 +88,7 @@ struct klp_reloc {
>>> struct klp_object {
>>> 	/* external */
>>> 	const char *name;
>>>-	struct klp_reloc *relocs;
>>>+	struct klp_reloc_sec *reloc_secs;
>>
>>There was a lot of discussion for v1, so I'm not sure, but I thought we
>>ended up deciding to get rid of the klp_reloc_sec struct?  Instead I
>>think the symbol table can be walked directly looking for klp rela
>>sections?
>>
>>The benefits of doing so would be to make the interface simpler -- no
>>need to "cache" the section metadata when it's already easily and
>>quickly available in the module struct elf section headers.
>
>Ah, I might have interpreted the conclusions of that last discussion
>incorrectly.
>
>In that case, I think I can just get rid of my klp_for_each_reloc_sec
>macro as well as the lreloc scaffolding code from kpatch. The only
>potentially "ugly" part of this change is that I'll have to move the
>string parsing stuff here to extract the objname of the __klp_rela
>section (which may not actually look that bad, we'll see how that
>turns out).

Turns out the string parsing stuff, even with the help of lib/string.c, doesn't
look very pretty. As I'm working on v3, I'm starting to think having
klp_write_object_relocations() loop simply through all the elf sections might
not be a good idea. Let me explain.

I don't like the amount of string manipulation code that would potentially come
with this change. Even with a string as simple as ".klp.rela.objname", we'll
end up with a bunch of kstrdup's/kmalloc's and kfree's (unless we modify and
chop the section name string in place, which I don't think we should do) that
are going to be required at every iteration of the loop, all just to be able to
call strcmp() and see if we're dealing with a klp rela section that belongs to
the object in question. This also leads to more complicated error handling.

In v1, the string parsing was done only *once* for each klp rela section in the
patch module code, and each klp rela section is sorted into their respective
object with the reloc_secs list. Then all klp_write_object_relocations() had to
do is iterate over object->reloc_secs. The tradeoff here is the addition of
another struct (klp_reloc_sec), but that is not nearly as bad as string
parsing, which is imo way more error prone and is unnecessary extra work.

Here's some pseudocode to help visualize the potential issues:

function klp_write_object_relocations(..,struct klp_object *obj,..) {
    for each section in mod->sechdrs { // iterate through all elf sections, no more obj->reloc_secs list
        if not a klp rela section
            continue;
        sec_objname = extract_objname_from_section(section);  // .klp.rela.[objname].sectionname
        if (strcmp(sec_objname, obj->name)) { // this klp rela section doesn't belong to this object
            kfree(sec_objname);
            continue;
        }

      for each rela in this klp rela section {
         sym = symtab + ELF_R_SYM(rela->r_info);
         sym_objname = extract_objname_from_symbol(sym) // symname.klp.[objname]
         if (!sym_objname)
             goto handle_error; // calls kfree(sec_objname);
         symname = extract_symname_from_symbol(sym); // [symname].klp.objname
         if (!symname)
             goto handle_error; // calls kfree(sec_objname);
         ret = klp_find_object_symbol(sym_objname, symname, &addr)
         if (ret)
             goto handle_error2; // calls kfree(symname), then kfree(sec_objname)

...etc., you get the idea how messy that is getting, and we haven't even
reached the call to apply_relocate_add() yet.

So I personally think it is better to keep the old v1 way for applying the klp
reloc secs. The sections are still named ".klp.rela.objname" but the parsing is
done once in the patch module code and used only to build the patch scaffolding
structure.

In order to avoid adding any additional string parsing code in v3, I no longer
thing we should rename livepatch symbols to include the symbol objname (i.e.
'symbol.klp.objname'). Recall that this change is for getting rid of external
dynrelas. Instead of parsing the symbol name, we could just encode 1) the
sympos and 2) the index into the .kpatch.strings strtab (that contains the
sym_objname) in sym->st_value. We define two masks (KLP_MASK_SYMPOS,
KLP_MASK_OBJNAME) that extract these values. st_value has either a 32 bit or 64
bit size, both of which are big enough sizes for the sympos and string table
index. One perk we lose is being able to see which object a symbol belongs to
in readelf, but then again we didn't have this benefit to begin with in
v1 nor v2 (where we didn't know the symbol's object and had to use
klp_find_external_symbol anyway), so I wouldn't say it's a loss.

Apologies for the super long explanations, but I think avoiding string parsing
in livepatch core code will make the relocation code much more succinct and
easier to read. Any objections or thoughts on all this?

Thanks,
Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [10/12/15 15:41 -0600]:
>On Thu, Dec 10, 2015 at 04:33:29PM -0500, Jessica Yu wrote:
>> +++ Josh Poimboeuf [10/12/15 08:28 -0600]:
>> >On Wed, Dec 09, 2015 at 02:10:14PM -0500, Jessica Yu wrote:
>> >>+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>> >>>>+	/* For each __klp_rela section for this object */
>> >>>>+	klp_for_each_reloc_sec(obj, reloc_sec) {
>> >>>>+		relindex = reloc_sec->index;
>> >>>>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>> >>>>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
>> >>>>+
>> >>>>+		/* For each rela in this __klp_rela section */
>> >>>>+		for (i = 0; i < num_relas; i++, rela++) {
>> >>>>+			sym = symtab + ELF_R_SYM(rela->r_info);
>> >>>>+			symname = pmod->core_strtab + sym->st_name;
>> >>>>+
>> >>>>+			if (sym->st_shndx == SHN_LIVEPATCH) {
>> >>>>+				if (sym->st_info == 'K')
>> >>>>+					ret = klp_find_external_symbol(pmod, symname, &addr);
>> >>>>+				else
>> >>>>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
>> >>>>+				if (ret)
>> >>>>+					return ret;
>> >>>>+				sym->st_value = addr;
>> >>>
>> >>>So I think you're also working on removing the 'external' stuff.  Any
>> >>>idea how this code will handle that?  Specifically I'm wondering how the
>> >>>objname and sympos of the rela's symbol will be specified.  Maybe it can
>> >>>be encoded somehow in one of the symbol fields (st_value)?
>> >>
>> >>Thanks for bringing this up. I think we can just encode the symbol's
>> >>position in kallsyms in the symbol's st_other field. It isn't used
>> >>anywhere and has size char, which is plenty of bits to represent the
>> >>small ints.
>> >
>> >st_other does seem to at least have some trivial usage in the kernel,
>> >see print_absolute_symbols() and sym_visibility() in
>> >arch/x86/tools/relocs.c.  Two of the bits are used to specify the
>> >"visibility" of a symbol.  Also readelf shows a "Vis" column in the
>> >symbol table.
>>
>> Yeah, for x86 it looks like st_other is used only for SHN_ABS symbols
>> in print_absolute_symbols(). Technically SHN_LIVEPATCH symbols
>> shouldn't be affected in this case...but despite its sparse usage in the
>> kernel it does look like using st_other to encode sympos is out of the
>> question as its meaning is architecture specific..
>>
>> >>For objname, the simplest solution might be to append ".klp.objname"
>> >>to the symbol name, and extract out this suffix when resolving the
>> >>symbol. Another way might be to have st_value contain the index into
>> >>the strtab (or .kpatch.strings) that contains the objname. Then we'd
>> >>access the objname just like how we access the symbol's name (strtab +
>> >>sym->st_name). After we find the objname we can then overwrite
>> >>st_value with the real address. I think this second method is cleaner.
>> >>Thoughts?
>> >
>> >Yeah, I guess there are a lot of possibilities for ways to encode it.
>> >
>> >Personally I think it would be nice if the encoding were something that
>> >could easily be seen when dumping the symbol table with readelf.  So,
>> >for example, the objname could be encoded in the symbol name (as you
>> >suggested), and the sympos could be in st_value.
>>
>> Sure, that should be doable. So the new process might look like this:
>>
>> For every livepatch symbol referenced by a rela..
>>
>> 1) Save the sympos encoded in st_value
>>
>> 2) Save the sym_objname that is encoded in the symbol's name with the
>> 'klp' suffix (Just to clarify: the sym_objname specifies the object
>> in which the symbol lives, and recall that we need this to remove the
>> need for the "external" flag)
>>
>> 3) Resolve the symbol by using its name (without the klp suffix),
>> sympos, and sym_objname
>>
>> 4) Set st_value to the found address
>
>Sounds right to me.
>
>> >If we do that, it'd probably be good to keep the naming consistent with
>> >the '__klp_rela_objname.symname' thing.  So something like
>> >'_klp_sym_objname.symname'.
>>
>> How about 'symname.klp.objname', and renaming the klp reloc sections
>> to '.klp.objname.rela.section_name'? Special symbol suffixes and
>> section names seem to always use '.', so maybe this would look better?
>> :-) But we can keep the underscores if people like that more. Both
>> naming methods would work, it is only a matter of preference.
>
>It's your patches, so I'd say you get to pick ;-)  My only request would
>be some consistency between the symbol names and the rela section names.
>
>> >But... would there be any side effects associated with renaming it?  For
>> >example, would it cause problems with the s390 PLT?
>
>Just to verify, did you see this question? :-)

Ah, sorry I missed this! So I looked through the module loader code
for x86, s390, arm (and just grepped in a few other archs) and the
symbol's name (st_name) is only ever used in printing error or debug
messages, or not used at all. As long as livepatch correctly resolves
the symbol, and st_value is set, apply_relocate_add() doesn't really
care about the name.

s390 plt/got information (struct mod_arch_syminfo) for each symbol is
stored in an array in mod->arch.syminfo. Every symbol that is in the
module's symbol table will get an entry, but not every symbol will
have their plt/got offsets filled in. Every symbol referenced by a
PLT/GOT rela will have those offsets filled in. Since our livepatch
symbols are real symbols, with indexes and entries in the symbol
table, they will get their own s390 mod_arch_syminfo struct, and
should any PLT rela reference them, a PLT entry as well, which is
good. Tldr, names of symbols don't affect s390 plt/got handling, so we
are safe.

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [10/12/15 15:41 -0600]:
>On Thu, Dec 10, 2015 at 04:33:29PM -0500, Jessica Yu wrote:
>> +++ Josh Poimboeuf [10/12/15 08:28 -0600]:
>> >On Wed, Dec 09, 2015 at 02:10:14PM -0500, Jessica Yu wrote:
>> >>+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>> >>>>+	/* For each __klp_rela section for this object */
>> >>>>+	klp_for_each_reloc_sec(obj, reloc_sec) {
>> >>>>+		relindex = reloc_sec->index;
>> >>>>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>> >>>>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
>> >>>>+
>> >>>>+		/* For each rela in this __klp_rela section */
>> >>>>+		for (i = 0; i < num_relas; i++, rela++) {
>> >>>>+			sym = symtab + ELF_R_SYM(rela->r_info);
>> >>>>+			symname = pmod->core_strtab + sym->st_name;
>> >>>>+
>> >>>>+			if (sym->st_shndx == SHN_LIVEPATCH) {
>> >>>>+				if (sym->st_info == 'K')
>> >>>>+					ret = klp_find_external_symbol(pmod, symname, &addr);
>> >>>>+				else
>> >>>>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
>> >>>>+				if (ret)
>> >>>>+					return ret;
>> >>>>+				sym->st_value = addr;
>> >>>
>> >>>So I think you're also working on removing the 'external' stuff.  Any
>> >>>idea how this code will handle that?  Specifically I'm wondering how the
>> >>>objname and sympos of the rela's symbol will be specified.  Maybe it can
>> >>>be encoded somehow in one of the symbol fields (st_value)?
>> >>
>> >>Thanks for bringing this up. I think we can just encode the symbol's
>> >>position in kallsyms in the symbol's st_other field. It isn't used
>> >>anywhere and has size char, which is plenty of bits to represent the
>> >>small ints.
>> >
>> >st_other does seem to at least have some trivial usage in the kernel,
>> >see print_absolute_symbols() and sym_visibility() in
>> >arch/x86/tools/relocs.c.  Two of the bits are used to specify the
>> >"visibility" of a symbol.  Also readelf shows a "Vis" column in the
>> >symbol table.
>>
>> Yeah, for x86 it looks like st_other is used only for SHN_ABS symbols
>> in print_absolute_symbols(). Technically SHN_LIVEPATCH symbols
>> shouldn't be affected in this case...but despite its sparse usage in the
>> kernel it does look like using st_other to encode sympos is out of the
>> question as its meaning is architecture specific..
>>
>> >>For objname, the simplest solution might be to append ".klp.objname"
>> >>to the symbol name, and extract out this suffix when resolving the
>> >>symbol. Another way might be to have st_value contain the index into
>> >>the strtab (or .kpatch.strings) that contains the objname. Then we'd
>> >>access the objname just like how we access the symbol's name (strtab +
>> >>sym->st_name). After we find the objname we can then overwrite
>> >>st_value with the real address. I think this second method is cleaner.
>> >>Thoughts?
>> >
>> >Yeah, I guess there are a lot of possibilities for ways to encode it.
>> >
>> >Personally I think it would be nice if the encoding were something that
>> >could easily be seen when dumping the symbol table with readelf.  So,
>> >for example, the objname could be encoded in the symbol name (as you
>> >suggested), and the sympos could be in st_value.
>>
>> Sure, that should be doable. So the new process might look like this:
>>
>> For every livepatch symbol referenced by a rela..
>>
>> 1) Save the sympos encoded in st_value
>>
>> 2) Save the sym_objname that is encoded in the symbol's name with the
>> 'klp' suffix (Just to clarify: the sym_objname specifies the object
>> in which the symbol lives, and recall that we need this to remove the
>> need for the "external" flag)
>>
>> 3) Resolve the symbol by using its name (without the klp suffix),
>> sympos, and sym_objname
>>
>> 4) Set st_value to the found address
>
>Sounds right to me.
>
>> >If we do that, it'd probably be good to keep the naming consistent with
>> >the '__klp_rela_objname.symname' thing.  So something like
>> >'_klp_sym_objname.symname'.
>>
>> How about 'symname.klp.objname', and renaming the klp reloc sections
>> to '.klp.objname.rela.section_name'? Special symbol suffixes and
>> section names seem to always use '.', so maybe this would look better?
>> :-) But we can keep the underscores if people like that more. Both
>> naming methods would work, it is only a matter of preference.
>
>It's your patches, so I'd say you get to pick ;-)  My only request would
>be some consistency between the symbol names and the rela section names.
>
>> >But... would there be any side effects associated with renaming it?  For
>> >example, would it cause problems with the s390 PLT?
>
>Just to verify, did you see this question? :-)

Ah, sorry I missed this! So I looked through the module loader code
for x86, s390, arm (and just grepped in a few other archs) and the
symbol's name (st_name) is only ever used in printing error or debug
messages, or not used at all. As long as livepatch correctly resolves
the symbol, and st_value is set, apply_relocate_add() doesn't really
care about the name.

s390 plt/got information (struct mod_arch_syminfo) for each symbol is
stored in an array in mod->arch.syminfo. Every symbol that is in the
module's symbol table will get an entry, but not every symbol will
have their plt/got offsets filled in. Every symbol referenced by a
PLT/GOT rela will have those offsets filled in. Since our livepatch
symbols are real symbols, with indexes and entries in the symbol
table, they will get their own s390 mod_arch_syminfo struct, and
should any PLT rela reference them, a PLT entry as well, which is
good. Tldr, names of symbols don't affect s390 plt/got handling, so we
are safe.

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Thu, Dec 10, 2015 at 04:33:29PM -0500, Jessica Yu wrote:
> +++ Josh Poimboeuf [10/12/15 08:28 -0600]:
> >On Wed, Dec 09, 2015 at 02:10:14PM -0500, Jessica Yu wrote:
> >>+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
> >>>>+	/* For each __klp_rela section for this object */
> >>>>+	klp_for_each_reloc_sec(obj, reloc_sec) {
> >>>>+		relindex = reloc_sec->index;
> >>>>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
> >>>>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
> >>>>+
> >>>>+		/* For each rela in this __klp_rela section */
> >>>>+		for (i = 0; i < num_relas; i++, rela++) {
> >>>>+			sym = symtab + ELF_R_SYM(rela->r_info);
> >>>>+			symname = pmod->core_strtab + sym->st_name;
> >>>>+
> >>>>+			if (sym->st_shndx == SHN_LIVEPATCH) {
> >>>>+				if (sym->st_info == 'K')
> >>>>+					ret = klp_find_external_symbol(pmod, symname, &addr);
> >>>>+				else
> >>>>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
> >>>>+				if (ret)
> >>>>+					return ret;
> >>>>+				sym->st_value = addr;
> >>>
> >>>So I think you're also working on removing the 'external' stuff.  Any
> >>>idea how this code will handle that?  Specifically I'm wondering how the
> >>>objname and sympos of the rela's symbol will be specified.  Maybe it can
> >>>be encoded somehow in one of the symbol fields (st_value)?
> >>
> >>Thanks for bringing this up. I think we can just encode the symbol's
> >>position in kallsyms in the symbol's st_other field. It isn't used
> >>anywhere and has size char, which is plenty of bits to represent the
> >>small ints.
> >
> >st_other does seem to at least have some trivial usage in the kernel,
> >see print_absolute_symbols() and sym_visibility() in
> >arch/x86/tools/relocs.c.  Two of the bits are used to specify the
> >"visibility" of a symbol.  Also readelf shows a "Vis" column in the
> >symbol table.
> 
> Yeah, for x86 it looks like st_other is used only for SHN_ABS symbols
> in print_absolute_symbols(). Technically SHN_LIVEPATCH symbols
> shouldn't be affected in this case...but despite its sparse usage in the
> kernel it does look like using st_other to encode sympos is out of the
> question as its meaning is architecture specific..
> 
> >>For objname, the simplest solution might be to append ".klp.objname"
> >>to the symbol name, and extract out this suffix when resolving the
> >>symbol. Another way might be to have st_value contain the index into
> >>the strtab (or .kpatch.strings) that contains the objname. Then we'd
> >>access the objname just like how we access the symbol's name (strtab +
> >>sym->st_name). After we find the objname we can then overwrite
> >>st_value with the real address. I think this second method is cleaner.
> >>Thoughts?
> >
> >Yeah, I guess there are a lot of possibilities for ways to encode it.
> >
> >Personally I think it would be nice if the encoding were something that
> >could easily be seen when dumping the symbol table with readelf.  So,
> >for example, the objname could be encoded in the symbol name (as you
> >suggested), and the sympos could be in st_value.
> 
> Sure, that should be doable. So the new process might look like this:
> 
> For every livepatch symbol referenced by a rela..
> 
> 1) Save the sympos encoded in st_value
> 
> 2) Save the sym_objname that is encoded in the symbol's name with the
> 'klp' suffix (Just to clarify: the sym_objname specifies the object
> in which the symbol lives, and recall that we need this to remove the
> need for the "external" flag)
> 
> 3) Resolve the symbol by using its name (without the klp suffix),
> sympos, and sym_objname
> 
> 4) Set st_value to the found address

Sounds right to me.

> >If we do that, it'd probably be good to keep the naming consistent with
> >the '__klp_rela_objname.symname' thing.  So something like
> >'_klp_sym_objname.symname'.
> 
> How about 'symname.klp.objname', and renaming the klp reloc sections
> to '.klp.objname.rela.section_name'? Special symbol suffixes and
> section names seem to always use '.', so maybe this would look better?
> :-) But we can keep the underscores if people like that more. Both
> naming methods would work, it is only a matter of preference.

It's your patches, so I'd say you get to pick ;-)  My only request would
be some consistency between the symbol names and the rela section names.

> >But... would there be any side effects associated with renaming it?  For
> >example, would it cause problems with the s390 PLT?

Just to verify, did you see this question? :-)

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Thu, Dec 10, 2015 at 04:33:29PM -0500, Jessica Yu wrote:
> +++ Josh Poimboeuf [10/12/15 08:28 -0600]:
> >On Wed, Dec 09, 2015 at 02:10:14PM -0500, Jessica Yu wrote:
> >>+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
> >>>>+	/* For each __klp_rela section for this object */
> >>>>+	klp_for_each_reloc_sec(obj, reloc_sec) {
> >>>>+		relindex = reloc_sec->index;
> >>>>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
> >>>>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
> >>>>+
> >>>>+		/* For each rela in this __klp_rela section */
> >>>>+		for (i = 0; i < num_relas; i++, rela++) {
> >>>>+			sym = symtab + ELF_R_SYM(rela->r_info);
> >>>>+			symname = pmod->core_strtab + sym->st_name;
> >>>>+
> >>>>+			if (sym->st_shndx == SHN_LIVEPATCH) {
> >>>>+				if (sym->st_info == 'K')
> >>>>+					ret = klp_find_external_symbol(pmod, symname, &addr);
> >>>>+				else
> >>>>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
> >>>>+				if (ret)
> >>>>+					return ret;
> >>>>+				sym->st_value = addr;
> >>>
> >>>So I think you're also working on removing the 'external' stuff.  Any
> >>>idea how this code will handle that?  Specifically I'm wondering how the
> >>>objname and sympos of the rela's symbol will be specified.  Maybe it can
> >>>be encoded somehow in one of the symbol fields (st_value)?
> >>
> >>Thanks for bringing this up. I think we can just encode the symbol's
> >>position in kallsyms in the symbol's st_other field. It isn't used
> >>anywhere and has size char, which is plenty of bits to represent the
> >>small ints.
> >
> >st_other does seem to at least have some trivial usage in the kernel,
> >see print_absolute_symbols() and sym_visibility() in
> >arch/x86/tools/relocs.c.  Two of the bits are used to specify the
> >"visibility" of a symbol.  Also readelf shows a "Vis" column in the
> >symbol table.
> 
> Yeah, for x86 it looks like st_other is used only for SHN_ABS symbols
> in print_absolute_symbols(). Technically SHN_LIVEPATCH symbols
> shouldn't be affected in this case...but despite its sparse usage in the
> kernel it does look like using st_other to encode sympos is out of the
> question as its meaning is architecture specific..
> 
> >>For objname, the simplest solution might be to append ".klp.objname"
> >>to the symbol name, and extract out this suffix when resolving the
> >>symbol. Another way might be to have st_value contain the index into
> >>the strtab (or .kpatch.strings) that contains the objname. Then we'd
> >>access the objname just like how we access the symbol's name (strtab +
> >>sym->st_name). After we find the objname we can then overwrite
> >>st_value with the real address. I think this second method is cleaner.
> >>Thoughts?
> >
> >Yeah, I guess there are a lot of possibilities for ways to encode it.
> >
> >Personally I think it would be nice if the encoding were something that
> >could easily be seen when dumping the symbol table with readelf.  So,
> >for example, the objname could be encoded in the symbol name (as you
> >suggested), and the sympos could be in st_value.
> 
> Sure, that should be doable. So the new process might look like this:
> 
> For every livepatch symbol referenced by a rela..
> 
> 1) Save the sympos encoded in st_value
> 
> 2) Save the sym_objname that is encoded in the symbol's name with the
> 'klp' suffix (Just to clarify: the sym_objname specifies the object
> in which the symbol lives, and recall that we need this to remove the
> need for the "external" flag)
> 
> 3) Resolve the symbol by using its name (without the klp suffix),
> sympos, and sym_objname
> 
> 4) Set st_value to the found address

Sounds right to me.

> >If we do that, it'd probably be good to keep the naming consistent with
> >the '__klp_rela_objname.symname' thing.  So something like
> >'_klp_sym_objname.symname'.
> 
> How about 'symname.klp.objname', and renaming the klp reloc sections
> to '.klp.objname.rela.section_name'? Special symbol suffixes and
> section names seem to always use '.', so maybe this would look better?
> :-) But we can keep the underscores if people like that more. Both
> naming methods would work, it is only a matter of preference.

It's your patches, so I'd say you get to pick ;-)  My only request would
be some consistency between the symbol names and the rela section names.

> >But... would there be any side effects associated with renaming it?  For
> >example, would it cause problems with the s390 PLT?

Just to verify, did you see this question? :-)

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [10/12/15 08:28 -0600]:
>On Wed, Dec 09, 2015 at 02:10:14PM -0500, Jessica Yu wrote:
>> +++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>> >>+	/* For each __klp_rela section for this object */
>> >>+	klp_for_each_reloc_sec(obj, reloc_sec) {
>> >>+		relindex = reloc_sec->index;
>> >>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>> >>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
>> >>+
>> >>+		/* For each rela in this __klp_rela section */
>> >>+		for (i = 0; i < num_relas; i++, rela++) {
>> >>+			sym = symtab + ELF_R_SYM(rela->r_info);
>> >>+			symname = pmod->core_strtab + sym->st_name;
>> >>+
>> >>+			if (sym->st_shndx == SHN_LIVEPATCH) {
>> >>+				if (sym->st_info == 'K')
>> >>+					ret = klp_find_external_symbol(pmod, symname, &addr);
>> >>+				else
>> >>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
>> >>+				if (ret)
>> >>+					return ret;
>> >>+				sym->st_value = addr;
>> >
>> >So I think you're also working on removing the 'external' stuff.  Any
>> >idea how this code will handle that?  Specifically I'm wondering how the
>> >objname and sympos of the rela's symbol will be specified.  Maybe it can
>> >be encoded somehow in one of the symbol fields (st_value)?
>>
>> Thanks for bringing this up. I think we can just encode the symbol's
>> position in kallsyms in the symbol's st_other field. It isn't used
>> anywhere and has size char, which is plenty of bits to represent the
>> small ints.
>
>st_other does seem to at least have some trivial usage in the kernel,
>see print_absolute_symbols() and sym_visibility() in
>arch/x86/tools/relocs.c.  Two of the bits are used to specify the
>"visibility" of a symbol.  Also readelf shows a "Vis" column in the
>symbol table.

Yeah, for x86 it looks like st_other is used only for SHN_ABS symbols
in print_absolute_symbols(). Technically SHN_LIVEPATCH symbols
shouldn't be affected in this case...but despite its sparse usage in the
kernel it does look like using st_other to encode sympos is out of the
question as its meaning is architecture specific..

>> For objname, the simplest solution might be to append ".klp.objname"
>> to the symbol name, and extract out this suffix when resolving the
>> symbol. Another way might be to have st_value contain the index into
>> the strtab (or .kpatch.strings) that contains the objname. Then we'd
>> access the objname just like how we access the symbol's name (strtab +
>> sym->st_name). After we find the objname we can then overwrite
>> st_value with the real address. I think this second method is cleaner.
>> Thoughts?
>
>Yeah, I guess there are a lot of possibilities for ways to encode it.
>
>Personally I think it would be nice if the encoding were something that
>could easily be seen when dumping the symbol table with readelf.  So,
>for example, the objname could be encoded in the symbol name (as you
>suggested), and the sympos could be in st_value.

Sure, that should be doable. So the new process might look like this:

For every livepatch symbol referenced by a rela..

1) Save the sympos encoded in st_value

2) Save the sym_objname that is encoded in the symbol's name with the
'klp' suffix (Just to clarify: the sym_objname specifies the object
in which the symbol lives, and recall that we need this to remove the
need for the "external" flag)

3) Resolve the symbol by using its name (without the klp suffix),
sympos, and sym_objname

4) Set st_value to the found address

>If we do that, it'd probably be good to keep the naming consistent with
>the '__klp_rela_objname.symname' thing.  So something like
>'_klp_sym_objname.symname'.

How about 'symname.klp.objname', and renaming the klp reloc sections
to '.klp.objname.rela.section_name'? Special symbol suffixes and
section names seem to always use '.', so maybe this would look better?
:-) But we can keep the underscores if people like that more. Both
naming methods would work, it is only a matter of preference.

>But... would there be any side effects associated with renaming it?  For
>example, would it cause problems with the s390 PLT?

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Wed, Dec 09, 2015 at 02:10:14PM -0500, Jessica Yu wrote:
> +++ Josh Poimboeuf [08/12/15 12:38 -0600]:
> >>+	/* For each __klp_rela section for this object */
> >>+	klp_for_each_reloc_sec(obj, reloc_sec) {
> >>+		relindex = reloc_sec->index;
> >>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
> >>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
> >>+
> >>+		/* For each rela in this __klp_rela section */
> >>+		for (i = 0; i < num_relas; i++, rela++) {
> >>+			sym = symtab + ELF_R_SYM(rela->r_info);
> >>+			symname = pmod->core_strtab + sym->st_name;
> >>+
> >>+			if (sym->st_shndx == SHN_LIVEPATCH) {
> >>+				if (sym->st_info == 'K')
> >>+					ret = klp_find_external_symbol(pmod, symname, &addr);
> >>+				else
> >>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
> >>+				if (ret)
> >>+					return ret;
> >>+				sym->st_value = addr;
> >
> >So I think you're also working on removing the 'external' stuff.  Any
> >idea how this code will handle that?  Specifically I'm wondering how the
> >objname and sympos of the rela's symbol will be specified.  Maybe it can
> >be encoded somehow in one of the symbol fields (st_value)?
> 
> Thanks for bringing this up. I think we can just encode the symbol's
> position in kallsyms in the symbol's st_other field. It isn't used
> anywhere and has size char, which is plenty of bits to represent the
> small ints.

st_other does seem to at least have some trivial usage in the kernel,
see print_absolute_symbols() and sym_visibility() in
arch/x86/tools/relocs.c.  Two of the bits are used to specify the
"visibility" of a symbol.  Also readelf shows a "Vis" column in the
symbol table.

> For objname, the simplest solution might be to append ".klp.objname"
> to the symbol name, and extract out this suffix when resolving the
> symbol. Another way might be to have st_value contain the index into
> the strtab (or .kpatch.strings) that contains the objname. Then we'd
> access the objname just like how we access the symbol's name (strtab +
> sym->st_name). After we find the objname we can then overwrite
> st_value with the real address. I think this second method is cleaner.
> Thoughts?

Yeah, I guess there are a lot of possibilities for ways to encode it.

Personally I think it would be nice if the encoding were something that
could easily be seen when dumping the symbol table with readelf.  So,
for example, the objname could be encoded in the symbol name (as you
suggested), and the sympos could be in st_value.

If we do that, it'd probably be good to keep the naming consistent with
the '__klp_rela_objname.symname' thing.  So something like
'_klp_sym_objname.symname'.

But... would there be any side effects associated with renaming it?  For
example, would it cause problems with the s390 PLT?

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Wed, Dec 09, 2015 at 02:10:14PM -0500, Jessica Yu wrote:
> +++ Josh Poimboeuf [08/12/15 12:38 -0600]:
> >>+	/* For each __klp_rela section for this object */
> >>+	klp_for_each_reloc_sec(obj, reloc_sec) {
> >>+		relindex = reloc_sec->index;
> >>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
> >>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
> >>+
> >>+		/* For each rela in this __klp_rela section */
> >>+		for (i = 0; i < num_relas; i++, rela++) {
> >>+			sym = symtab + ELF_R_SYM(rela->r_info);
> >>+			symname = pmod->core_strtab + sym->st_name;
> >>+
> >>+			if (sym->st_shndx == SHN_LIVEPATCH) {
> >>+				if (sym->st_info == 'K')
> >>+					ret = klp_find_external_symbol(pmod, symname, &addr);
> >>+				else
> >>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
> >>+				if (ret)
> >>+					return ret;
> >>+				sym->st_value = addr;
> >
> >So I think you're also working on removing the 'external' stuff.  Any
> >idea how this code will handle that?  Specifically I'm wondering how the
> >objname and sympos of the rela's symbol will be specified.  Maybe it can
> >be encoded somehow in one of the symbol fields (st_value)?
> 
> Thanks for bringing this up. I think we can just encode the symbol's
> position in kallsyms in the symbol's st_other field. It isn't used
> anywhere and has size char, which is plenty of bits to represent the
> small ints.

st_other does seem to at least have some trivial usage in the kernel,
see print_absolute_symbols() and sym_visibility() in
arch/x86/tools/relocs.c.  Two of the bits are used to specify the
"visibility" of a symbol.  Also readelf shows a "Vis" column in the
symbol table.

> For objname, the simplest solution might be to append ".klp.objname"
> to the symbol name, and extract out this suffix when resolving the
> symbol. Another way might be to have st_value contain the index into
> the strtab (or .kpatch.strings) that contains the objname. Then we'd
> access the objname just like how we access the symbol's name (strtab +
> sym->st_name). After we find the objname we can then overwrite
> st_value with the real address. I think this second method is cleaner.
> Thoughts?

Yeah, I guess there are a lot of possibilities for ways to encode it.

Personally I think it would be nice if the encoding were something that
could easily be seen when dumping the symbol table with readelf.  So,
for example, the objname could be encoded in the symbol name (as you
suggested), and the sympos could be in st_value.

If we do that, it'd probably be good to keep the naming consistent with
the '__klp_rela_objname.symname' thing.  So something like
'_klp_sym_objname.symname'.

But... would there be any side effects associated with renaming it?  For
example, would it cause problems with the s390 PLT?

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>On Mon, Nov 30, 2015 at 11:21:17PM -0500, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the need
>> for architecture specific relocation code in livepatch. Namely, we reuse
>> apply_relocate_add() in the module loader to write relocations instead of
>> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>> are resolved and then apply_relocate_add() is called to apply those
>> relocations.
>>
>> In addition, remove x86 livepatch relocation code. It is no longer needed
>> since symbol resolution and relocation work have been offloaded to module
>> loader.
>>
>> Signed-off-by: Jessica Yu <jeyu@redhat.com>
>> ---
>>  arch/x86/include/asm/livepatch.h |  2 -
>>  arch/x86/kernel/Makefile         |  1 -
>>  arch/x86/kernel/livepatch.c      | 91 --------------------------------------
>>  include/linux/livepatch.h        | 30 ++++++-------
>>  include/linux/module.h           |  6 +++
>>  kernel/livepatch/core.c          | 94 ++++++++++++++++++++++------------------
>>  6 files changed, 70 insertions(+), 154 deletions(-)
>>  delete mode 100644 arch/x86/kernel/livepatch.c
>>
>> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> index 19c099a..7312e25 100644
>> --- a/arch/x86/include/asm/livepatch.h
>> +++ b/arch/x86/include/asm/livepatch.h
>> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>  #endif
>>  	return 0;
>>  }
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value);
>>
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> index b1b78ff..c5e9a5c 100644
>> --- a/arch/x86/kernel/Makefile
>> +++ b/arch/x86/kernel/Makefile
>> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>  obj-y				+= apic/
>>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> deleted file mode 100644
>> index d1d35cc..0000000
>> --- a/arch/x86/kernel/livepatch.c
>> +++ /dev/null
>> @@ -1,91 +0,0 @@
>> -/*
>> - * livepatch.c - x86-specific Kernel Live Patching Core
>> - *
>> - * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
>> - * Copyright (C) 2014 SUSE
>> - *
>> - * This program is free software; you can redistribute it and/or
>> - * modify it under the terms of the GNU General Public License
>> - * as published by the Free Software Foundation; either version 2
>> - * of the License, or (at your option) any later version.
>> - *
>> - * This program is distributed in the hope that it will be useful,
>> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> - * GNU General Public License for more details.
>> - *
>> - * You should have received a copy of the GNU General Public License
>> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> - */
>> -
>> -#include <linux/module.h>
>> -#include <linux/uaccess.h>
>> -#include <asm/cacheflush.h>
>> -#include <asm/page_types.h>
>> -#include <asm/elf.h>
>> -#include <asm/livepatch.h>
>> -
>> -/**
>> - * klp_write_module_reloc() - write a relocation in a module
>> - * @mod:	module in which the section to be modified is found
>> - * @type:	ELF relocation type (see asm/elf.h)
>> - * @loc:	address that the relocation should be written to
>> - * @value:	relocation value (sym address + addend)
>> - *
>> - * This function writes a relocation to the specified location for
>> - * a particular module.
>> - */
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value)
>> -{
>> -	int ret, numpages, size = 4;
>> -	bool readonly;
>> -	unsigned long val;
>> -	unsigned long core = (unsigned long)mod->module_core;
>> -	unsigned long core_size = mod->core_size;
>> -
>> -	switch (type) {
>> -	case R_X86_64_NONE:
>> -		return 0;
>> -	case R_X86_64_64:
>> -		val = value;
>> -		size = 8;
>> -		break;
>> -	case R_X86_64_32:
>> -		val = (u32)value;
>> -		break;
>> -	case R_X86_64_32S:
>> -		val = (s32)value;
>> -		break;
>> -	case R_X86_64_PC32:
>> -		val = (u32)(value - loc);
>> -		break;
>> -	default:
>> -		/* unsupported relocation type */
>> -		return -EINVAL;
>> -	}
>> -
>> -	if (loc < core || loc >= core + core_size)
>> -		/* loc does not point to any symbol inside the module */
>> -		return -EINVAL;
>> -
>> -	readonly = false;
>> -
>> -#ifdef CONFIG_DEBUG_SET_MODULE_RONX
>> -	if (loc < core + mod->core_ro_size)
>> -		readonly = true;
>> -#endif
>> -
>> -	/* determine if the relocation spans a page boundary */
>> -	numpages = ((loc & PAGE_MASK) == ((loc + size) & PAGE_MASK)) ? 1 : 2;
>> -
>> -	if (readonly)
>> -		set_memory_rw(loc & PAGE_MASK, numpages);
>> -
>> -	ret = probe_kernel_write((void *)loc, &val, size);
>> -
>> -	if (readonly)
>> -		set_memory_ro(loc & PAGE_MASK, numpages);
>> -
>> -	return ret;
>> -}
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index 31db7a0..54f62a6 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -64,28 +64,21 @@ struct klp_func {
>>  };
>>
>>  /**
>> - * struct klp_reloc - relocation structure for live patching
>> - * @loc:	address where the relocation will be written
>> - * @val:	address of the referenced symbol (optional,
>> - *		vmlinux	patches only)
>> - * @type:	ELF relocation type
>> - * @name:	name of the referenced symbol (for lookup/verification)
>> - * @addend:	offset from the referenced symbol
>> - * @external:	symbol is either exported or within the live patch module itself
>> + * struct klp_reloc_sec - relocation section struct for live patching
>> + * @index:	Elf section index of the relocation section
>> + * @name:	name of the relocation section
>> + * @objname:	name of the object associated with the klp reloc section
>>   */
>> -struct klp_reloc {
>> -	unsigned long loc;
>> -	unsigned long val;
>> -	unsigned long type;
>> -	const char *name;
>> -	int addend;
>> -	int external;
>> +struct klp_reloc_sec {
>> +	unsigned int index;
>> +	char *name;
>> +	char *objname;
>>  };
>>
>>  /**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>> + * @reloc_secs:	relocation sections to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +88,7 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>> +	struct klp_reloc_sec *reloc_secs;
>
>There was a lot of discussion for v1, so I'm not sure, but I thought we
>ended up deciding to get rid of the klp_reloc_sec struct?  Instead I
>think the symbol table can be walked directly looking for klp rela
>sections?
>
>The benefits of doing so would be to make the interface simpler -- no
>need to "cache" the section metadata when it's already easily and
>quickly available in the module struct elf section headers.

Ah, I might have interpreted the conclusions of that last discussion
incorrectly.

In that case, I think I can just get rid of my klp_for_each_reloc_sec
macro as well as the lreloc scaffolding code from kpatch. The only
potentially "ugly" part of this change is that I'll have to move the
string parsing stuff here to extract the objname of the __klp_rela
section (which may not actually look that bad, we'll see how that
turns out).

>>  	struct klp_func *funcs;
>>
>>  	/* internal */
>> @@ -129,6 +122,9 @@ struct klp_patch {
>>  #define klp_for_each_func(obj, func) \
>>  	for (func = obj->funcs; func->old_name; func++)
>>
>> +#define klp_for_each_reloc_sec(obj, reloc_sec) \
>> +	for (reloc_sec = obj->reloc_secs; reloc_sec->name; reloc_sec++)
>> +
>>  int klp_register_patch(struct klp_patch *);
>>  int klp_unregister_patch(struct klp_patch *);
>>  int klp_enable_patch(struct klp_patch *);
>> diff --git a/include/linux/module.h b/include/linux/module.h
>> index 9b46256..ea61147 100644
>> --- a/include/linux/module.h
>> +++ b/include/linux/module.h
>> @@ -777,9 +777,15 @@ extern int module_sysfs_initialized;
>>  #ifdef CONFIG_DEBUG_SET_MODULE_RONX
>>  extern void set_all_modules_text_rw(void);
>>  extern void set_all_modules_text_ro(void);
>> +extern void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages));
>>  #else
>>  static inline void set_all_modules_text_rw(void) { }
>>  static inline void set_all_modules_text_ro(void) { }
>> +static inline void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages)) { }
>>  #endif
>>
>>  #ifdef CONFIG_GENERIC_BUG
>> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> index db545cb..17b7278 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,9 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <asm/cacheflush.h>
>> +#include <linux/moduleloader.h>
>>
>>  /**
>>   * struct klp_ops - structure for tracking registered ftrace ops structs
>> @@ -281,52 +284,48 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
>>  }
>>
>>  static int klp_write_object_relocations(struct module *pmod,
>> -					struct klp_object *obj)
>> +					struct klp_object *obj,
>> +					struct klp_patch *patch)
>>  {
>> -	int ret;
>> -	struct klp_reloc *reloc;
>> +	int relindex, num_relas;
>> +	int i, ret = 0;
>> +	unsigned long addr;
>> +	char *symname;
>> +	struct klp_reloc_sec *reloc_sec;
>> +	Elf_Rela *rela;
>> +	Elf_Sym *sym, *symtab;
>>
>>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>  		return -EINVAL;
>>
>> -	if (WARN_ON(!obj->relocs))
>> -		return -EINVAL;
>> -
>> -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> -		if (!klp_is_module(obj)) {
>> -
>> -#if defined(CONFIG_RANDOMIZE_BASE)
>> -			/* If KASLR has been enabled, adjust old value accordingly */
>> -			if (kaslr_enabled())
>> -				reloc->val += kaslr_offset();
>> -#endif
>> -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> -							reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		} else {
>> -			/* module, reloc->val needs to be discovered */
>> -			if (reloc->external)
>> -				ret = klp_find_external_symbol(pmod,
>> -							       reloc->name,
>> -							       &reloc->val);
>> -			else
>> -				ret = klp_find_object_symbol(obj->mod->name,
>> -							     reloc->name,
>> -							     &reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		}
>> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> -					     reloc->val + reloc->addend);
>> -		if (ret) {
>> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
>> -			       reloc->name, reloc->val, ret);
>> -			return ret;
>> +	symtab = (void *)pmod->core_symtab;
>> +
>> +	/* For each __klp_rela section for this object */
>> +	klp_for_each_reloc_sec(obj, reloc_sec) {
>> +		relindex = reloc_sec->index;
>> +		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>> +		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
>> +
>> +		/* For each rela in this __klp_rela section */
>> +		for (i = 0; i < num_relas; i++, rela++) {
>> +			sym = symtab + ELF_R_SYM(rela->r_info);
>> +			symname = pmod->core_strtab + sym->st_name;
>> +
>> +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> +				if (sym->st_info == 'K')
>> +					ret = klp_find_external_symbol(pmod, symname, &addr);
>> +				else
>> +					ret = klp_find_object_symbol(obj->name, symname, &addr);
>> +				if (ret)
>> +					return ret;
>> +				sym->st_value = addr;
>
>So I think you're also working on removing the 'external' stuff.  Any
>idea how this code will handle that?  Specifically I'm wondering how the
>objname and sympos of the rela's symbol will be specified.  Maybe it can
>be encoded somehow in one of the symbol fields (st_value)?

Thanks for bringing this up. I think we can just encode the symbol's
position in kallsyms in the symbol's st_other field. It isn't used
anywhere and has size char, which is plenty of bits to represent the
small ints.

For objname, the simplest solution might be to append ".klp.objname"
to the symbol name, and extract out this suffix when resolving the
symbol. Another way might be to have st_value contain the index into
the strtab (or .kpatch.strings) that contains the objname. Then we'd
access the objname just like how we access the symbol's name (strtab +
sym->st_name). After we find the objname we can then overwrite
st_value with the real address. I think this second method is cleaner.
Thoughts?

>> +			}
>>  		}
>> +		ret = apply_relocate_add(pmod->sechdrs, pmod->core_strtab,
>> +					 pmod->index.sym, relindex, pmod);
>>  	}
>>
>> -	return 0;
>> +	return ret;
>>  }
>>
>>  static void notrace klp_ftrace_handler(unsigned long ip,
>> @@ -747,13 +746,22 @@ static int klp_init_object_loaded(struct klp_patch *patch,
>>  				  struct klp_object *obj)
>>  {
>>  	struct klp_func *func;
>> +	struct module *pmod;
>>  	int ret;
>>
>> -	if (obj->relocs) {
>> -		ret = klp_write_object_relocations(patch->mod, obj);
>> -		if (ret)
>> -			return ret;
>> -	}
>> +	pmod = patch->mod;
>> +
>> +	set_page_attributes(pmod->module_core,
>> +			    pmod->module_core + pmod->core_text_size,
>> +			    set_memory_rw);
>> +
>> +	ret = klp_write_object_relocations(pmod, obj, patch);
>> +	if (ret)
>> +		return ret;
>> +
>> +	set_page_attributes(pmod->module_core,
>> +			    pmod->module_core + pmod->core_text_size,
>> +			    set_memory_ro);
>>
>>  	klp_for_each_func(obj, func) {
>>  		ret = klp_find_verify_func_addr(obj, func);
>> --
>> 2.4.3
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe live-patching" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>-- 
>Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [08/12/15 12:38 -0600]:
>On Mon, Nov 30, 2015 at 11:21:17PM -0500, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the need
>> for architecture specific relocation code in livepatch. Namely, we reuse
>> apply_relocate_add() in the module loader to write relocations instead of
>> duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>> relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>> are resolved and then apply_relocate_add() is called to apply those
>> relocations.
>>
>> In addition, remove x86 livepatch relocation code. It is no longer needed
>> since symbol resolution and relocation work have been offloaded to module
>> loader.
>>
>> Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> ---
>>  arch/x86/include/asm/livepatch.h |  2 -
>>  arch/x86/kernel/Makefile         |  1 -
>>  arch/x86/kernel/livepatch.c      | 91 --------------------------------------
>>  include/linux/livepatch.h        | 30 ++++++-------
>>  include/linux/module.h           |  6 +++
>>  kernel/livepatch/core.c          | 94 ++++++++++++++++++++++------------------
>>  6 files changed, 70 insertions(+), 154 deletions(-)
>>  delete mode 100644 arch/x86/kernel/livepatch.c
>>
>> diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>> index 19c099a..7312e25 100644
>> --- a/arch/x86/include/asm/livepatch.h
>> +++ b/arch/x86/include/asm/livepatch.h
>> @@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
>>  #endif
>>  	return 0;
>>  }
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value);
>>
>>  static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
>>  {
>> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> index b1b78ff..c5e9a5c 100644
>> --- a/arch/x86/kernel/Makefile
>> +++ b/arch/x86/kernel/Makefile
>> @@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
>>  obj-y				+= apic/
>>  obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
>>  obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>> -obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
>>  obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
>>  obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
>>  obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>> diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>> deleted file mode 100644
>> index d1d35cc..0000000
>> --- a/arch/x86/kernel/livepatch.c
>> +++ /dev/null
>> @@ -1,91 +0,0 @@
>> -/*
>> - * livepatch.c - x86-specific Kernel Live Patching Core
>> - *
>> - * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> - * Copyright (C) 2014 SUSE
>> - *
>> - * This program is free software; you can redistribute it and/or
>> - * modify it under the terms of the GNU General Public License
>> - * as published by the Free Software Foundation; either version 2
>> - * of the License, or (at your option) any later version.
>> - *
>> - * This program is distributed in the hope that it will be useful,
>> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> - * GNU General Public License for more details.
>> - *
>> - * You should have received a copy of the GNU General Public License
>> - * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> - */
>> -
>> -#include <linux/module.h>
>> -#include <linux/uaccess.h>
>> -#include <asm/cacheflush.h>
>> -#include <asm/page_types.h>
>> -#include <asm/elf.h>
>> -#include <asm/livepatch.h>
>> -
>> -/**
>> - * klp_write_module_reloc() - write a relocation in a module
>> - * @mod:	module in which the section to be modified is found
>> - * @type:	ELF relocation type (see asm/elf.h)
>> - * @loc:	address that the relocation should be written to
>> - * @value:	relocation value (sym address + addend)
>> - *
>> - * This function writes a relocation to the specified location for
>> - * a particular module.
>> - */
>> -int klp_write_module_reloc(struct module *mod, unsigned long type,
>> -			   unsigned long loc, unsigned long value)
>> -{
>> -	int ret, numpages, size = 4;
>> -	bool readonly;
>> -	unsigned long val;
>> -	unsigned long core = (unsigned long)mod->module_core;
>> -	unsigned long core_size = mod->core_size;
>> -
>> -	switch (type) {
>> -	case R_X86_64_NONE:
>> -		return 0;
>> -	case R_X86_64_64:
>> -		val = value;
>> -		size = 8;
>> -		break;
>> -	case R_X86_64_32:
>> -		val = (u32)value;
>> -		break;
>> -	case R_X86_64_32S:
>> -		val = (s32)value;
>> -		break;
>> -	case R_X86_64_PC32:
>> -		val = (u32)(value - loc);
>> -		break;
>> -	default:
>> -		/* unsupported relocation type */
>> -		return -EINVAL;
>> -	}
>> -
>> -	if (loc < core || loc >= core + core_size)
>> -		/* loc does not point to any symbol inside the module */
>> -		return -EINVAL;
>> -
>> -	readonly = false;
>> -
>> -#ifdef CONFIG_DEBUG_SET_MODULE_RONX
>> -	if (loc < core + mod->core_ro_size)
>> -		readonly = true;
>> -#endif
>> -
>> -	/* determine if the relocation spans a page boundary */
>> -	numpages = ((loc & PAGE_MASK) == ((loc + size) & PAGE_MASK)) ? 1 : 2;
>> -
>> -	if (readonly)
>> -		set_memory_rw(loc & PAGE_MASK, numpages);
>> -
>> -	ret = probe_kernel_write((void *)loc, &val, size);
>> -
>> -	if (readonly)
>> -		set_memory_ro(loc & PAGE_MASK, numpages);
>> -
>> -	return ret;
>> -}
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index 31db7a0..54f62a6 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -64,28 +64,21 @@ struct klp_func {
>>  };
>>
>>  /**
>> - * struct klp_reloc - relocation structure for live patching
>> - * @loc:	address where the relocation will be written
>> - * @val:	address of the referenced symbol (optional,
>> - *		vmlinux	patches only)
>> - * @type:	ELF relocation type
>> - * @name:	name of the referenced symbol (for lookup/verification)
>> - * @addend:	offset from the referenced symbol
>> - * @external:	symbol is either exported or within the live patch module itself
>> + * struct klp_reloc_sec - relocation section struct for live patching
>> + * @index:	Elf section index of the relocation section
>> + * @name:	name of the relocation section
>> + * @objname:	name of the object associated with the klp reloc section
>>   */
>> -struct klp_reloc {
>> -	unsigned long loc;
>> -	unsigned long val;
>> -	unsigned long type;
>> -	const char *name;
>> -	int addend;
>> -	int external;
>> +struct klp_reloc_sec {
>> +	unsigned int index;
>> +	char *name;
>> +	char *objname;
>>  };
>>
>>  /**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>> + * @reloc_secs:	relocation sections to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +88,7 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>> +	struct klp_reloc_sec *reloc_secs;
>
>There was a lot of discussion for v1, so I'm not sure, but I thought we
>ended up deciding to get rid of the klp_reloc_sec struct?  Instead I
>think the symbol table can be walked directly looking for klp rela
>sections?
>
>The benefits of doing so would be to make the interface simpler -- no
>need to "cache" the section metadata when it's already easily and
>quickly available in the module struct elf section headers.

Ah, I might have interpreted the conclusions of that last discussion
incorrectly.

In that case, I think I can just get rid of my klp_for_each_reloc_sec
macro as well as the lreloc scaffolding code from kpatch. The only
potentially "ugly" part of this change is that I'll have to move the
string parsing stuff here to extract the objname of the __klp_rela
section (which may not actually look that bad, we'll see how that
turns out).

>>  	struct klp_func *funcs;
>>
>>  	/* internal */
>> @@ -129,6 +122,9 @@ struct klp_patch {
>>  #define klp_for_each_func(obj, func) \
>>  	for (func = obj->funcs; func->old_name; func++)
>>
>> +#define klp_for_each_reloc_sec(obj, reloc_sec) \
>> +	for (reloc_sec = obj->reloc_secs; reloc_sec->name; reloc_sec++)
>> +
>>  int klp_register_patch(struct klp_patch *);
>>  int klp_unregister_patch(struct klp_patch *);
>>  int klp_enable_patch(struct klp_patch *);
>> diff --git a/include/linux/module.h b/include/linux/module.h
>> index 9b46256..ea61147 100644
>> --- a/include/linux/module.h
>> +++ b/include/linux/module.h
>> @@ -777,9 +777,15 @@ extern int module_sysfs_initialized;
>>  #ifdef CONFIG_DEBUG_SET_MODULE_RONX
>>  extern void set_all_modules_text_rw(void);
>>  extern void set_all_modules_text_ro(void);
>> +extern void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages));
>>  #else
>>  static inline void set_all_modules_text_rw(void) { }
>>  static inline void set_all_modules_text_ro(void) { }
>> +static inline void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages)) { }
>>  #endif
>>
>>  #ifdef CONFIG_GENERIC_BUG
>> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> index db545cb..17b7278 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,9 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <asm/cacheflush.h>
>> +#include <linux/moduleloader.h>
>>
>>  /**
>>   * struct klp_ops - structure for tracking registered ftrace ops structs
>> @@ -281,52 +284,48 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
>>  }
>>
>>  static int klp_write_object_relocations(struct module *pmod,
>> -					struct klp_object *obj)
>> +					struct klp_object *obj,
>> +					struct klp_patch *patch)
>>  {
>> -	int ret;
>> -	struct klp_reloc *reloc;
>> +	int relindex, num_relas;
>> +	int i, ret = 0;
>> +	unsigned long addr;
>> +	char *symname;
>> +	struct klp_reloc_sec *reloc_sec;
>> +	Elf_Rela *rela;
>> +	Elf_Sym *sym, *symtab;
>>
>>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>  		return -EINVAL;
>>
>> -	if (WARN_ON(!obj->relocs))
>> -		return -EINVAL;
>> -
>> -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> -		if (!klp_is_module(obj)) {
>> -
>> -#if defined(CONFIG_RANDOMIZE_BASE)
>> -			/* If KASLR has been enabled, adjust old value accordingly */
>> -			if (kaslr_enabled())
>> -				reloc->val += kaslr_offset();
>> -#endif
>> -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> -							reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		} else {
>> -			/* module, reloc->val needs to be discovered */
>> -			if (reloc->external)
>> -				ret = klp_find_external_symbol(pmod,
>> -							       reloc->name,
>> -							       &reloc->val);
>> -			else
>> -				ret = klp_find_object_symbol(obj->mod->name,
>> -							     reloc->name,
>> -							     &reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		}
>> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> -					     reloc->val + reloc->addend);
>> -		if (ret) {
>> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
>> -			       reloc->name, reloc->val, ret);
>> -			return ret;
>> +	symtab = (void *)pmod->core_symtab;
>> +
>> +	/* For each __klp_rela section for this object */
>> +	klp_for_each_reloc_sec(obj, reloc_sec) {
>> +		relindex = reloc_sec->index;
>> +		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>> +		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
>> +
>> +		/* For each rela in this __klp_rela section */
>> +		for (i = 0; i < num_relas; i++, rela++) {
>> +			sym = symtab + ELF_R_SYM(rela->r_info);
>> +			symname = pmod->core_strtab + sym->st_name;
>> +
>> +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> +				if (sym->st_info == 'K')
>> +					ret = klp_find_external_symbol(pmod, symname, &addr);
>> +				else
>> +					ret = klp_find_object_symbol(obj->name, symname, &addr);
>> +				if (ret)
>> +					return ret;
>> +				sym->st_value = addr;
>
>So I think you're also working on removing the 'external' stuff.  Any
>idea how this code will handle that?  Specifically I'm wondering how the
>objname and sympos of the rela's symbol will be specified.  Maybe it can
>be encoded somehow in one of the symbol fields (st_value)?

Thanks for bringing this up. I think we can just encode the symbol's
position in kallsyms in the symbol's st_other field. It isn't used
anywhere and has size char, which is plenty of bits to represent the
small ints.

For objname, the simplest solution might be to append ".klp.objname"
to the symbol name, and extract out this suffix when resolving the
symbol. Another way might be to have st_value contain the index into
the strtab (or .kpatch.strings) that contains the objname. Then we'd
access the objname just like how we access the symbol's name (strtab +
sym->st_name). After we find the objname we can then overwrite
st_value with the real address. I think this second method is cleaner.
Thoughts?

>> +			}
>>  		}
>> +		ret = apply_relocate_add(pmod->sechdrs, pmod->core_strtab,
>> +					 pmod->index.sym, relindex, pmod);
>>  	}
>>
>> -	return 0;
>> +	return ret;
>>  }
>>
>>  static void notrace klp_ftrace_handler(unsigned long ip,
>> @@ -747,13 +746,22 @@ static int klp_init_object_loaded(struct klp_patch *patch,
>>  				  struct klp_object *obj)
>>  {
>>  	struct klp_func *func;
>> +	struct module *pmod;
>>  	int ret;
>>
>> -	if (obj->relocs) {
>> -		ret = klp_write_object_relocations(patch->mod, obj);
>> -		if (ret)
>> -			return ret;
>> -	}
>> +	pmod = patch->mod;
>> +
>> +	set_page_attributes(pmod->module_core,
>> +			    pmod->module_core + pmod->core_text_size,
>> +			    set_memory_rw);
>> +
>> +	ret = klp_write_object_relocations(pmod, obj, patch);
>> +	if (ret)
>> +		return ret;
>> +
>> +	set_page_attributes(pmod->module_core,
>> +			    pmod->module_core + pmod->core_text_size,
>> +			    set_memory_ro);
>>
>>  	klp_for_each_func(obj, func) {
>>  		ret = klp_find_verify_func_addr(obj, func);
>> --
>> 2.4.3
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe live-patching" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>-- 
>Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Jessica Yu [30/11/15 23:21 -0500]:
>Reuse module loader code to write relocations, thereby eliminating the need
>for architecture specific relocation code in livepatch. Namely, we reuse
>apply_relocate_add() in the module loader to write relocations instead of
>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>are resolved and then apply_relocate_add() is called to apply those
>relocations.
>
>In addition, remove x86 livepatch relocation code. It is no longer needed
>since symbol resolution and relocation work have been offloaded to module
>loader.
>
>Signed-off-by: Jessica Yu <jeyu@redhat.com>
>---
> arch/x86/include/asm/livepatch.h |  2 -
> arch/x86/kernel/Makefile         |  1 -
> arch/x86/kernel/livepatch.c      | 91 --------------------------------------
> include/linux/livepatch.h        | 30 ++++++-------
> include/linux/module.h           |  6 +++
> kernel/livepatch/core.c          | 94 ++++++++++++++++++++++------------------
> 6 files changed, 70 insertions(+), 154 deletions(-)
> delete mode 100644 arch/x86/kernel/livepatch.c
>
>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>index 19c099a..7312e25 100644
>--- a/arch/x86/include/asm/livepatch.h
>+++ b/arch/x86/include/asm/livepatch.h
>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
> #endif
> 	return 0;
> }
>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>-			   unsigned long loc, unsigned long value);
>
> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
> {
>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>index b1b78ff..c5e9a5c 100644
>--- a/arch/x86/kernel/Makefile
>+++ b/arch/x86/kernel/Makefile
>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
> obj-y				+= apic/
> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>deleted file mode 100644
>index d1d35cc..0000000
>--- a/arch/x86/kernel/livepatch.c
>+++ /dev/null
>@@ -1,91 +0,0 @@
>-/*
>- * livepatch.c - x86-specific Kernel Live Patching Core
>- *
>- * Copyright (C) 2014 Seth Jennings <sjenning@redhat.com>
>- * Copyright (C) 2014 SUSE
>- *
>- * This program is free software; you can redistribute it and/or
>- * modify it under the terms of the GNU General Public License
>- * as published by the Free Software Foundation; either version 2
>- * of the License, or (at your option) any later version.
>- *
>- * This program is distributed in the hope that it will be useful,
>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>- * GNU General Public License for more details.
>- *
>- * You should have received a copy of the GNU General Public License
>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
>- */
>-
>-#include <linux/module.h>
>-#include <linux/uaccess.h>
>-#include <asm/cacheflush.h>
>-#include <asm/page_types.h>
>-#include <asm/elf.h>
>-#include <asm/livepatch.h>
>-
>-/**
>- * klp_write_module_reloc() - write a relocation in a module
>- * @mod:	module in which the section to be modified is found
>- * @type:	ELF relocation type (see asm/elf.h)
>- * @loc:	address that the relocation should be written to
>- * @value:	relocation value (sym address + addend)
>- *
>- * This function writes a relocation to the specified location for
>- * a particular module.
>- */
>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>-			   unsigned long loc, unsigned long value)
>-{
>-	int ret, numpages, size = 4;
>-	bool readonly;
>-	unsigned long val;
>-	unsigned long core = (unsigned long)mod->module_core;
>-	unsigned long core_size = mod->core_size;
>-
>-	switch (type) {
>-	case R_X86_64_NONE:
>-		return 0;
>-	case R_X86_64_64:
>-		val = value;
>-		size = 8;
>-		break;
>-	case R_X86_64_32:
>-		val = (u32)value;
>-		break;
>-	case R_X86_64_32S:
>-		val = (s32)value;
>-		break;
>-	case R_X86_64_PC32:
>-		val = (u32)(value - loc);
>-		break;
>-	default:
>-		/* unsupported relocation type */
>-		return -EINVAL;
>-	}
>-
>-	if (loc < core || loc >= core + core_size)
>-		/* loc does not point to any symbol inside the module */
>-		return -EINVAL;
>-
>-	readonly = false;
>-
>-#ifdef CONFIG_DEBUG_SET_MODULE_RONX
>-	if (loc < core + mod->core_ro_size)
>-		readonly = true;
>-#endif
>-
>-	/* determine if the relocation spans a page boundary */
>-	numpages = ((loc & PAGE_MASK) == ((loc + size) & PAGE_MASK)) ? 1 : 2;
>-
>-	if (readonly)
>-		set_memory_rw(loc & PAGE_MASK, numpages);
>-
>-	ret = probe_kernel_write((void *)loc, &val, size);
>-
>-	if (readonly)
>-		set_memory_ro(loc & PAGE_MASK, numpages);
>-
>-	return ret;
>-}
>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>index 31db7a0..54f62a6 100644
>--- a/include/linux/livepatch.h
>+++ b/include/linux/livepatch.h
>@@ -64,28 +64,21 @@ struct klp_func {
> };
>
> /**
>- * struct klp_reloc - relocation structure for live patching
>- * @loc:	address where the relocation will be written
>- * @val:	address of the referenced symbol (optional,
>- *		vmlinux	patches only)
>- * @type:	ELF relocation type
>- * @name:	name of the referenced symbol (for lookup/verification)
>- * @addend:	offset from the referenced symbol
>- * @external:	symbol is either exported or within the live patch module itself
>+ * struct klp_reloc_sec - relocation section struct for live patching
>+ * @index:	Elf section index of the relocation section
>+ * @name:	name of the relocation section
>+ * @objname:	name of the object associated with the klp reloc section
>  */
>-struct klp_reloc {
>-	unsigned long loc;
>-	unsigned long val;
>-	unsigned long type;
>-	const char *name;
>-	int addend;
>-	int external;
>+struct klp_reloc_sec {
>+	unsigned int index;
>+	char *name;
>+	char *objname;
> };
>
> /**
>  * struct klp_object - kernel object structure for live patching
>  * @name:	module name (or NULL for vmlinux)
>- * @relocs:	relocation entries to be applied at load time
>+ * @reloc_secs:	relocation sections to be applied at load time
>  * @funcs:	function entries for functions to be patched in the object
>  * @kobj:	kobject for sysfs resources
>  * @mod:	kernel module associated with the patched object
>@@ -95,7 +88,7 @@ struct klp_reloc {
> struct klp_object {
> 	/* external */
> 	const char *name;
>-	struct klp_reloc *relocs;
>+	struct klp_reloc_sec *reloc_secs;
> 	struct klp_func *funcs;
>
> 	/* internal */
>@@ -129,6 +122,9 @@ struct klp_patch {
> #define klp_for_each_func(obj, func) \
> 	for (func = obj->funcs; func->old_name; func++)
>
>+#define klp_for_each_reloc_sec(obj, reloc_sec) \
>+	for (reloc_sec = obj->reloc_secs; reloc_sec->name; reloc_sec++)
>+
> int klp_register_patch(struct klp_patch *);
> int klp_unregister_patch(struct klp_patch *);
> int klp_enable_patch(struct klp_patch *);
>diff --git a/include/linux/module.h b/include/linux/module.h
>index 9b46256..ea61147 100644
>--- a/include/linux/module.h
>+++ b/include/linux/module.h
>@@ -777,9 +777,15 @@ extern int module_sysfs_initialized;
> #ifdef CONFIG_DEBUG_SET_MODULE_RONX
> extern void set_all_modules_text_rw(void);
> extern void set_all_modules_text_ro(void);
>+extern void
>+set_page_attributes(void *start, void *end,
>+		    int (*set)(unsigned long start, int num_pages));
> #else
> static inline void set_all_modules_text_rw(void) { }
> static inline void set_all_modules_text_ro(void) { }
>+static inline void
>+set_page_attributes(void *start, void *end,
>+		    int (*set)(unsigned long start, int num_pages)) { }
> #endif
>
> #ifdef CONFIG_GENERIC_BUG
>diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>index db545cb..17b7278 100644
>--- a/kernel/livepatch/core.c
>+++ b/kernel/livepatch/core.c
>@@ -28,6 +28,9 @@
> #include <linux/list.h>
> #include <linux/kallsyms.h>
> #include <linux/livepatch.h>
>+#include <linux/elf.h>
>+#include <asm/cacheflush.h>
>+#include <linux/moduleloader.h>
>
> /**
>  * struct klp_ops - structure for tracking registered ftrace ops structs
>@@ -281,52 +284,48 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
> }
>
> static int klp_write_object_relocations(struct module *pmod,
>-					struct klp_object *obj)
>+					struct klp_object *obj,
>+					struct klp_patch *patch)
> {
>-	int ret;
>-	struct klp_reloc *reloc;
>+	int relindex, num_relas;
>+	int i, ret = 0;
>+	unsigned long addr;
>+	char *symname;
>+	struct klp_reloc_sec *reloc_sec;
>+	Elf_Rela *rela;
>+	Elf_Sym *sym, *symtab;
>
> 	if (WARN_ON(!klp_is_object_loaded(obj)))
> 		return -EINVAL;
>
>-	if (WARN_ON(!obj->relocs))
>-		return -EINVAL;
>-
>-	for (reloc = obj->relocs; reloc->name; reloc++) {
>-		if (!klp_is_module(obj)) {
>-
>-#if defined(CONFIG_RANDOMIZE_BASE)
>-			/* If KASLR has been enabled, adjust old value accordingly */
>-			if (kaslr_enabled())
>-				reloc->val += kaslr_offset();
>-#endif
>-			ret = klp_verify_vmlinux_symbol(reloc->name,
>-							reloc->val);
>-			if (ret)
>-				return ret;
>-		} else {
>-			/* module, reloc->val needs to be discovered */
>-			if (reloc->external)
>-				ret = klp_find_external_symbol(pmod,
>-							       reloc->name,
>-							       &reloc->val);
>-			else
>-				ret = klp_find_object_symbol(obj->mod->name,
>-							     reloc->name,
>-							     &reloc->val);
>-			if (ret)
>-				return ret;
>-		}
>-		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>-					     reloc->val + reloc->addend);
>-		if (ret) {
>-			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
>-			       reloc->name, reloc->val, ret);
>-			return ret;
>+	symtab = (void *)pmod->core_symtab;
>+
>+	/* For each __klp_rela section for this object */
>+	klp_for_each_reloc_sec(obj, reloc_sec) {
>+		relindex = reloc_sec->index;
>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
>+
>+		/* For each rela in this __klp_rela section */
>+		for (i = 0; i < num_relas; i++, rela++) {
>+			sym = symtab + ELF_R_SYM(rela->r_info);
>+			symname = pmod->core_strtab + sym->st_name;
>+
>+			if (sym->st_shndx == SHN_LIVEPATCH) {
>+				if (sym->st_info == 'K')
>+					ret = klp_find_external_symbol(pmod, symname, &addr);
>+				else
>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
>+				if (ret)
>+					return ret;
>+				sym->st_value = addr;

I'll note here that this patchset has the same problem that current
livepatch has in that it will be unable to handle duplicate symbols
within the same object (i.e. sysfs will be unable to create entries
with the same name and livepatch errors out). After the Chris's sympos
patchset gets merged I should be able to rebase this patchset on top
of it to resolve the duplicate symbol issue.

>+			}
> 		}
>+		ret = apply_relocate_add(pmod->sechdrs, pmod->core_strtab,
>+					 pmod->index.sym, relindex, pmod);
> 	}
>
>-	return 0;
>+	return ret;
> }
>
> static void notrace klp_ftrace_handler(unsigned long ip,
>@@ -747,13 +746,22 @@ static int klp_init_object_loaded(struct klp_patch *patch,
> 				  struct klp_object *obj)
> {
> 	struct klp_func *func;
>+	struct module *pmod;
> 	int ret;
>
>-	if (obj->relocs) {
>-		ret = klp_write_object_relocations(patch->mod, obj);
>-		if (ret)
>-			return ret;
>-	}
>+	pmod = patch->mod;
>+
>+	set_page_attributes(pmod->module_core,
>+			    pmod->module_core + pmod->core_text_size,
>+			    set_memory_rw);

Note #2: When Josh's module cleanup patches get merged, I'll swap out these
set_page_attribute() calls for (un)set_module_core_ro_nx().

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Jessica Yu [30/11/15 23:21 -0500]:
>Reuse module loader code to write relocations, thereby eliminating the need
>for architecture specific relocation code in livepatch. Namely, we reuse
>apply_relocate_add() in the module loader to write relocations instead of
>duplicating functionality in livepatch's klp_write_module_reloc(). To apply
>relocation sections, remaining SHN_LIVEPATCH symbols referenced by relocs
>are resolved and then apply_relocate_add() is called to apply those
>relocations.
>
>In addition, remove x86 livepatch relocation code. It is no longer needed
>since symbol resolution and relocation work have been offloaded to module
>loader.
>
>Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>---
> arch/x86/include/asm/livepatch.h |  2 -
> arch/x86/kernel/Makefile         |  1 -
> arch/x86/kernel/livepatch.c      | 91 --------------------------------------
> include/linux/livepatch.h        | 30 ++++++-------
> include/linux/module.h           |  6 +++
> kernel/livepatch/core.c          | 94 ++++++++++++++++++++++------------------
> 6 files changed, 70 insertions(+), 154 deletions(-)
> delete mode 100644 arch/x86/kernel/livepatch.c
>
>diff --git a/arch/x86/include/asm/livepatch.h b/arch/x86/include/asm/livepatch.h
>index 19c099a..7312e25 100644
>--- a/arch/x86/include/asm/livepatch.h
>+++ b/arch/x86/include/asm/livepatch.h
>@@ -33,8 +33,6 @@ static inline int klp_check_compiler_support(void)
> #endif
> 	return 0;
> }
>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>-			   unsigned long loc, unsigned long value);
>
> static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip)
> {
>diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>index b1b78ff..c5e9a5c 100644
>--- a/arch/x86/kernel/Makefile
>+++ b/arch/x86/kernel/Makefile
>@@ -67,7 +67,6 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
> obj-y				+= apic/
> obj-$(CONFIG_X86_REBOOTFIXUPS)	+= reboot_fixups_32.o
> obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
>-obj-$(CONFIG_LIVEPATCH)		+= livepatch.o
> obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
> obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
> obj-$(CONFIG_X86_TSC)		+= trace_clock.o
>diff --git a/arch/x86/kernel/livepatch.c b/arch/x86/kernel/livepatch.c
>deleted file mode 100644
>index d1d35cc..0000000
>--- a/arch/x86/kernel/livepatch.c
>+++ /dev/null
>@@ -1,91 +0,0 @@
>-/*
>- * livepatch.c - x86-specific Kernel Live Patching Core
>- *
>- * Copyright (C) 2014 Seth Jennings <sjenning-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>- * Copyright (C) 2014 SUSE
>- *
>- * This program is free software; you can redistribute it and/or
>- * modify it under the terms of the GNU General Public License
>- * as published by the Free Software Foundation; either version 2
>- * of the License, or (at your option) any later version.
>- *
>- * This program is distributed in the hope that it will be useful,
>- * but WITHOUT ANY WARRANTY; without even the implied warranty of
>- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>- * GNU General Public License for more details.
>- *
>- * You should have received a copy of the GNU General Public License
>- * along with this program; if not, see <http://www.gnu.org/licenses/>.
>- */
>-
>-#include <linux/module.h>
>-#include <linux/uaccess.h>
>-#include <asm/cacheflush.h>
>-#include <asm/page_types.h>
>-#include <asm/elf.h>
>-#include <asm/livepatch.h>
>-
>-/**
>- * klp_write_module_reloc() - write a relocation in a module
>- * @mod:	module in which the section to be modified is found
>- * @type:	ELF relocation type (see asm/elf.h)
>- * @loc:	address that the relocation should be written to
>- * @value:	relocation value (sym address + addend)
>- *
>- * This function writes a relocation to the specified location for
>- * a particular module.
>- */
>-int klp_write_module_reloc(struct module *mod, unsigned long type,
>-			   unsigned long loc, unsigned long value)
>-{
>-	int ret, numpages, size = 4;
>-	bool readonly;
>-	unsigned long val;
>-	unsigned long core = (unsigned long)mod->module_core;
>-	unsigned long core_size = mod->core_size;
>-
>-	switch (type) {
>-	case R_X86_64_NONE:
>-		return 0;
>-	case R_X86_64_64:
>-		val = value;
>-		size = 8;
>-		break;
>-	case R_X86_64_32:
>-		val = (u32)value;
>-		break;
>-	case R_X86_64_32S:
>-		val = (s32)value;
>-		break;
>-	case R_X86_64_PC32:
>-		val = (u32)(value - loc);
>-		break;
>-	default:
>-		/* unsupported relocation type */
>-		return -EINVAL;
>-	}
>-
>-	if (loc < core || loc >= core + core_size)
>-		/* loc does not point to any symbol inside the module */
>-		return -EINVAL;
>-
>-	readonly = false;
>-
>-#ifdef CONFIG_DEBUG_SET_MODULE_RONX
>-	if (loc < core + mod->core_ro_size)
>-		readonly = true;
>-#endif
>-
>-	/* determine if the relocation spans a page boundary */
>-	numpages = ((loc & PAGE_MASK) == ((loc + size) & PAGE_MASK)) ? 1 : 2;
>-
>-	if (readonly)
>-		set_memory_rw(loc & PAGE_MASK, numpages);
>-
>-	ret = probe_kernel_write((void *)loc, &val, size);
>-
>-	if (readonly)
>-		set_memory_ro(loc & PAGE_MASK, numpages);
>-
>-	return ret;
>-}
>diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>index 31db7a0..54f62a6 100644
>--- a/include/linux/livepatch.h
>+++ b/include/linux/livepatch.h
>@@ -64,28 +64,21 @@ struct klp_func {
> };
>
> /**
>- * struct klp_reloc - relocation structure for live patching
>- * @loc:	address where the relocation will be written
>- * @val:	address of the referenced symbol (optional,
>- *		vmlinux	patches only)
>- * @type:	ELF relocation type
>- * @name:	name of the referenced symbol (for lookup/verification)
>- * @addend:	offset from the referenced symbol
>- * @external:	symbol is either exported or within the live patch module itself
>+ * struct klp_reloc_sec - relocation section struct for live patching
>+ * @index:	Elf section index of the relocation section
>+ * @name:	name of the relocation section
>+ * @objname:	name of the object associated with the klp reloc section
>  */
>-struct klp_reloc {
>-	unsigned long loc;
>-	unsigned long val;
>-	unsigned long type;
>-	const char *name;
>-	int addend;
>-	int external;
>+struct klp_reloc_sec {
>+	unsigned int index;
>+	char *name;
>+	char *objname;
> };
>
> /**
>  * struct klp_object - kernel object structure for live patching
>  * @name:	module name (or NULL for vmlinux)
>- * @relocs:	relocation entries to be applied at load time
>+ * @reloc_secs:	relocation sections to be applied at load time
>  * @funcs:	function entries for functions to be patched in the object
>  * @kobj:	kobject for sysfs resources
>  * @mod:	kernel module associated with the patched object
>@@ -95,7 +88,7 @@ struct klp_reloc {
> struct klp_object {
> 	/* external */
> 	const char *name;
>-	struct klp_reloc *relocs;
>+	struct klp_reloc_sec *reloc_secs;
> 	struct klp_func *funcs;
>
> 	/* internal */
>@@ -129,6 +122,9 @@ struct klp_patch {
> #define klp_for_each_func(obj, func) \
> 	for (func = obj->funcs; func->old_name; func++)
>
>+#define klp_for_each_reloc_sec(obj, reloc_sec) \
>+	for (reloc_sec = obj->reloc_secs; reloc_sec->name; reloc_sec++)
>+
> int klp_register_patch(struct klp_patch *);
> int klp_unregister_patch(struct klp_patch *);
> int klp_enable_patch(struct klp_patch *);
>diff --git a/include/linux/module.h b/include/linux/module.h
>index 9b46256..ea61147 100644
>--- a/include/linux/module.h
>+++ b/include/linux/module.h
>@@ -777,9 +777,15 @@ extern int module_sysfs_initialized;
> #ifdef CONFIG_DEBUG_SET_MODULE_RONX
> extern void set_all_modules_text_rw(void);
> extern void set_all_modules_text_ro(void);
>+extern void
>+set_page_attributes(void *start, void *end,
>+		    int (*set)(unsigned long start, int num_pages));
> #else
> static inline void set_all_modules_text_rw(void) { }
> static inline void set_all_modules_text_ro(void) { }
>+static inline void
>+set_page_attributes(void *start, void *end,
>+		    int (*set)(unsigned long start, int num_pages)) { }
> #endif
>
> #ifdef CONFIG_GENERIC_BUG
>diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>index db545cb..17b7278 100644
>--- a/kernel/livepatch/core.c
>+++ b/kernel/livepatch/core.c
>@@ -28,6 +28,9 @@
> #include <linux/list.h>
> #include <linux/kallsyms.h>
> #include <linux/livepatch.h>
>+#include <linux/elf.h>
>+#include <asm/cacheflush.h>
>+#include <linux/moduleloader.h>
>
> /**
>  * struct klp_ops - structure for tracking registered ftrace ops structs
>@@ -281,52 +284,48 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
> }
>
> static int klp_write_object_relocations(struct module *pmod,
>-					struct klp_object *obj)
>+					struct klp_object *obj,
>+					struct klp_patch *patch)
> {
>-	int ret;
>-	struct klp_reloc *reloc;
>+	int relindex, num_relas;
>+	int i, ret = 0;
>+	unsigned long addr;
>+	char *symname;
>+	struct klp_reloc_sec *reloc_sec;
>+	Elf_Rela *rela;
>+	Elf_Sym *sym, *symtab;
>
> 	if (WARN_ON(!klp_is_object_loaded(obj)))
> 		return -EINVAL;
>
>-	if (WARN_ON(!obj->relocs))
>-		return -EINVAL;
>-
>-	for (reloc = obj->relocs; reloc->name; reloc++) {
>-		if (!klp_is_module(obj)) {
>-
>-#if defined(CONFIG_RANDOMIZE_BASE)
>-			/* If KASLR has been enabled, adjust old value accordingly */
>-			if (kaslr_enabled())
>-				reloc->val += kaslr_offset();
>-#endif
>-			ret = klp_verify_vmlinux_symbol(reloc->name,
>-							reloc->val);
>-			if (ret)
>-				return ret;
>-		} else {
>-			/* module, reloc->val needs to be discovered */
>-			if (reloc->external)
>-				ret = klp_find_external_symbol(pmod,
>-							       reloc->name,
>-							       &reloc->val);
>-			else
>-				ret = klp_find_object_symbol(obj->mod->name,
>-							     reloc->name,
>-							     &reloc->val);
>-			if (ret)
>-				return ret;
>-		}
>-		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>-					     reloc->val + reloc->addend);
>-		if (ret) {
>-			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
>-			       reloc->name, reloc->val, ret);
>-			return ret;
>+	symtab = (void *)pmod->core_symtab;
>+
>+	/* For each __klp_rela section for this object */
>+	klp_for_each_reloc_sec(obj, reloc_sec) {
>+		relindex = reloc_sec->index;
>+		num_relas = pmod->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>+		rela = (Elf_Rela *) pmod->sechdrs[relindex].sh_addr;
>+
>+		/* For each rela in this __klp_rela section */
>+		for (i = 0; i < num_relas; i++, rela++) {
>+			sym = symtab + ELF_R_SYM(rela->r_info);
>+			symname = pmod->core_strtab + sym->st_name;
>+
>+			if (sym->st_shndx == SHN_LIVEPATCH) {
>+				if (sym->st_info == 'K')
>+					ret = klp_find_external_symbol(pmod, symname, &addr);
>+				else
>+					ret = klp_find_object_symbol(obj->name, symname, &addr);
>+				if (ret)
>+					return ret;
>+				sym->st_value = addr;

I'll note here that this patchset has the same problem that current
livepatch has in that it will be unable to handle duplicate symbols
within the same object (i.e. sysfs will be unable to create entries
with the same name and livepatch errors out). After the Chris's sympos
patchset gets merged I should be able to rebase this patchset on top
of it to resolve the duplicate symbol issue.

>+			}
> 		}
>+		ret = apply_relocate_add(pmod->sechdrs, pmod->core_strtab,
>+					 pmod->index.sym, relindex, pmod);
> 	}
>
>-	return 0;
>+	return ret;
> }
>
> static void notrace klp_ftrace_handler(unsigned long ip,
>@@ -747,13 +746,22 @@ static int klp_init_object_loaded(struct klp_patch *patch,
> 				  struct klp_object *obj)
> {
> 	struct klp_func *func;
>+	struct module *pmod;
> 	int ret;
>
>-	if (obj->relocs) {
>-		ret = klp_write_object_relocations(patch->mod, obj);
>-		if (ret)
>-			return ret;
>-	}
>+	pmod = patch->mod;
>+
>+	set_page_attributes(pmod->module_core,
>+			    pmod->module_core + pmod->core_text_size,
>+			    set_memory_rw);

Note #2: When Josh's module cleanup patches get merged, I'll swap out these
set_page_attribute() calls for (un)set_module_core_ro_nx().

Jessica

Re: livepatch: reuse module loader code to write relocations

[Miroslav Benes]

On Thu, 12 Nov 2015, Josh Poimboeuf wrote:

> On Thu, Nov 12, 2015 at 03:22:44PM -0500, Jessica Yu wrote:
> > Looking into this more, I think we do need one __klp_rela section per
> > function being patched.  Each rela section is linked to the section to
> > which the relocations apply via the rela section's sh_info field. In
> > SHT_RELA sections, the sh_info field contains the section index to
> > which the relocs apply. We cannot have one single combined rela
> > section per object as the call to apply_relocate_add() simply won't
> > work, because we would have relocs that apply to different functions
> > (and hence different sections).
> > 
> > So I guess instead of a single field in klp_object specifying the
> > __klp_rela section index, we could probably just have an array of
> > section indices.
> 
> Ok, makes sense, sounds like we need multiple klp relas per object.

Ok, it seems so.

> I still don't quite understand the benefit of caching the klp_rela
> section indices.  What problem does it solve?  It seems simpler to just
> iterate over all the sections in klp_write_object_relocations().

It was just my need to be efficient and I think it would have made sense 
with only one dynrela section per object. An array of indices is "ugly" so 
I am all for iteration over all the sections in 
klp_write_object_relocations().

Miroslav

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [12/11/15 14:32 -0600]:
>On Thu, Nov 12, 2015 at 03:22:44PM -0500, Jessica Yu wrote:
>> Looking into this more, I think we do need one __klp_rela section per
>> function being patched.  Each rela section is linked to the section to
>> which the relocations apply via the rela section's sh_info field. In
>> SHT_RELA sections, the sh_info field contains the section index to
>> which the relocs apply. We cannot have one single combined rela
>> section per object as the call to apply_relocate_add() simply won't
>> work, because we would have relocs that apply to different functions
>> (and hence different sections).
>>
>> So I guess instead of a single field in klp_object specifying the
>> __klp_rela section index, we could probably just have an array of
>> section indices.
>
>Ok, makes sense, sounds like we need multiple klp relas per object.
>
>I still don't quite understand the benefit of caching the klp_rela
>section indices.  What problem does it solve?  It seems simpler to just
>iterate over all the sections in klp_write_object_relocations().

I was trying to accomodate Miroslav's dislike of klp_reloc_sec :-)
But now that we are sure that we need multiple __klp_rela sections
per object, I think an array of indices might still be somewhat
complicated (we need to figure out its size and everything, a list
here would be easier). I think it might be fine to leave things the
way they are here.

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [12/11/15 14:32 -0600]:
>On Thu, Nov 12, 2015 at 03:22:44PM -0500, Jessica Yu wrote:
>> Looking into this more, I think we do need one __klp_rela section per
>> function being patched.  Each rela section is linked to the section to
>> which the relocations apply via the rela section's sh_info field. In
>> SHT_RELA sections, the sh_info field contains the section index to
>> which the relocs apply. We cannot have one single combined rela
>> section per object as the call to apply_relocate_add() simply won't
>> work, because we would have relocs that apply to different functions
>> (and hence different sections).
>>
>> So I guess instead of a single field in klp_object specifying the
>> __klp_rela section index, we could probably just have an array of
>> section indices.
>
>Ok, makes sense, sounds like we need multiple klp relas per object.
>
>I still don't quite understand the benefit of caching the klp_rela
>section indices.  What problem does it solve?  It seems simpler to just
>iterate over all the sections in klp_write_object_relocations().

I was trying to accomodate Miroslav's dislike of klp_reloc_sec :-)
But now that we are sure that we need multiple __klp_rela sections
per object, I think an array of indices might still be somewhat
complicated (we need to figure out its size and everything, a list
here would be easier). I think it might be fine to leave things the
way they are here.

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Jessica Yu [12/11/15 14:14 -0500]:
>+++ Miroslav Benes [12/11/15 16:27 +0100]:
>>On Wed, 11 Nov 2015, Jessica Yu wrote:
>>
>>>+++ Miroslav Benes [11/11/15 15:30 +0100]:
>>>> On Mon, 9 Nov 2015, Jessica Yu wrote:
>>>>
>>>> So I guess we don't need klp_reloc anymore.
>>>
>>>Yes, that's correct. I am noticing just now that I forgot to remove
>>>the klp_reloc struct definition from livepatch.h. That change will be
>>>reflected in v2...
>>>
>>>> If true, we should really
>>>> start thinking about proper documentation because there are going to be
>>>> plenty of assumptions about a patch module and we need to have it written
>>>> somewhere. Especially how the relocation sections look like.
>>>
>>>Agreed. As a first step the patch module format can perhaps be
>>>documented somewhere. Perhaps it's time we create
>>>Documentation/livepatch/? :-)
>>
>>Yes, I think so.
>>
>>>> > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>>>> > index 087a8c7..26c419f 100644
>>>> > --- a/kernel/livepatch/core.c
>>>> > +++ b/kernel/livepatch/core.c
>>>> > @@ -28,6 +28,8 @@
>>>> >  #include <linux/list.h>
>>>> >  #include <linux/kallsyms.h>
>>>> >  #include <linux/livepatch.h>
>>>> > +#include <linux/elf.h>
>>>> > +#include <asm/cacheflush.h>
>>>> >
>>>> >  /**
>>>> >   * struct klp_ops - structure for tracking registered ftrace ops structs
>>>> > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
>>>> > *pmod, const char *name,
>>>> >  }
>>>> >
>>>> >  static int klp_write_object_relocations(struct module *pmod,
>>>> > -					struct klp_object *obj)
>>>> > +					struct klp_object *obj,
>>>> > +					struct klp_patch *patch)
>>>> >  {
>>>> > -	int ret;
>>>> > -	struct klp_reloc *reloc;
>>>> > +	int relindex, num_relas;
>>>> > +	int i, ret = 0;
>>>> > +	unsigned long addr;
>>>> > +	unsigned int bind;
>>>> > +	char *symname;
>>>> > +	struct klp_reloc_sec *reloc_sec;
>>>> > +	struct load_info *info;
>>>> > +	Elf_Rela *rela;
>>>> > +	Elf_Sym *sym, *symtab;
>>>> > +	Elf_Shdr *symsect;
>>>> >
>>>> >  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>>> >  		return -EINVAL;
>>>> >
>>>> > -	if (WARN_ON(!obj->relocs))
>>>> > -		return -EINVAL;
>>>> > -
>>>> > -	for (reloc = obj->relocs; reloc->name; reloc++) {
>>>> > -		if (!klp_is_module(obj)) {
>>>> > -			ret = klp_verify_vmlinux_symbol(reloc->name,
>>>> > -							reloc->val);
>>>> > -			if (ret)
>>>> > -				return ret;
>>>> > -		} else {
>>>> > -			/* module, reloc->val needs to be discovered */
>>>> > -			if (reloc->external)
>>>> > -				ret = klp_find_external_symbol(pmod,
>>>> > -							       reloc->name,
>>>> > -							       &reloc->val);
>>>> > -			else
>>>> > -				ret = klp_find_object_symbol(obj->mod->name,
>>>> > -							     reloc->name,
>>>> > -							     &reloc->val);
>>>> > -			if (ret)
>>>> > -				return ret;
>>>> > -		}
>>>> > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>>>> > -					     reloc->val + reloc->addend);
>>>> > -		if (ret) {
>>>> > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
>>>> > (%d)\n",
>>>> > -			       reloc->name, reloc->val, ret);
>>>> > -			return ret;
>>>> > +	info = pmod->info;
>>>> > +	symsect = info->sechdrs + info->index.sym;
>>>> > +	symtab = (void *)info->hdr + symsect->sh_offset;
>>>> > +
>>>> > +	/* For each __klp_rela section for this object */
>>>> > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>>>> > +		relindex = reloc_sec->index;
>>>> > +		num_relas = info->sechdrs[relindex].sh_size /
>>>> > sizeof(Elf_Rela);
>>>> > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>>>> > +
>>>> > +		/* For each rela in this __klp_rela section */
>>>> > +		for (i = 0; i < num_relas; i++, rela++) {
>>>> > +			sym = symtab + ELF_R_SYM(rela->r_info);
>>>> > +			symname = info->strtab + sym->st_name;
>>>> > +			bind = ELF_ST_BIND(sym->st_info);
>>>> > +
>>>> > +			if (sym->st_shndx == SHN_LIVEPATCH) {
>>>> > +				if (bind == STB_LIVEPATCH_EXT)
>>>> > +					ret = klp_find_external_symbol(pmod,
>>>> > symname, &addr);
>>>> > +				else
>>>> > +					ret =
>>>> > klp_find_object_symbol(obj->name, symname, &addr);
>>>> > +				if (ret)
>>>> > +					return ret;
>>>> > +				sym->st_value = addr;
>>>> > +			}
>>>> >  		}
>>>> > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>>>> > +					 info->index.sym, relindex, pmod);
>>>> >  	}
>>>> >
>>>> > -	return 0;
>>>> > +	return ret;
>>>> >  }
>>>>
>>>> Looking at this... do we even need reloc_secs in klp_object? Question is
>>>> whether we need more than one dynrela section for an object. If not then
>>>> the binding between klp_reloc_sec and an object is the only relevant thing
>>>> in the structure, be it index or objname. So we can replace the
>>>> list of structures with just the index in klp_object, or get rid of it
>>>> completely and rely on the name of dynrela section be something like
>>>> __klp_rela_{objname}.
>>>
>>>Hm, you bring up a good point. I think theoretically yes, it is
>>>possible to just have one klp_reloc_sec for each object and therefore
>>>a list is not required (I have not checked yet how difficult it would
>>>be to implement this on the kpatch-build side of things).  However,
>>>considering the final format of the patch module, I think it is
>>>semantically clearer to leave it as a list, and for each object to
>>>possibly have more than one __klp_rela section.
>>>
>>>For example, say we are patching two functions in ext4. In my
>>>resulting kpatch module I will have two __klp_rela_ext4 sections, and
>>>they might look like this when we run readelf --sections:
>>>
>>>[34] __klp_rela_ext4.text.ext4_attr_store RELA ...
>>>[35] __klp_rela_ext4.text.ext4_attr_show RELA ...
>>>
>>>Then these two klp rela sections end up as two elements in the
>>>reloc_secs list for the ext4 patch object. I think this way, we can
>>>better tell which rela is being applied to what function. Might be
>>>easier to understand what's happening from the developer's point of
>>>view.
>>>
>>>> You see, we go through elf sections here which were preserved by module
>>>> loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>>>> maybe all the stuff around klp_reloc_sec is not necessary.
>>>>
>>>> Thoughts?
>>>
>>>Ah, so this is where descriptive comments and documentation might have
>>>been useful :-) So I think we will still need to keep the
>>>klp_reloc_sec struct to help the patch module initialize. Though the
>>>name and objname fields aren't used in this patchset, they are used in
>>>the kpatch patch module code [1], where we iterate through each elf
>>>section, find the ones marked with SHF_RELA_LIVEPATCH, set the
>>>klp_reloc_sec's objname (which we find from the "name" field,
>>>formatted as __klp_rela_{objname}.text..). Once we have the objname
>>>set, we can then find the object to attach the reloc_sec to (i.e. add
>>>it to its list of reloc_secs).
>>>
>>>Hope that clears some things up.
>>
>>Ok, I'll try to explain myself and it is gonna be long. I'll try to
>>describe how we deal with dynrelas in klp today, how you use it in kpatch
>>(and this is the place where my knowledge can be wrong or obsolete), what
>>you propose and what I'd like to propose.
>>
>>1. First let's look on what we have now.
>>
>>We have a patch module in which there is a section with all dynrelas
>>needed to be resolved (it was like this in kpatch some time ago and maybe
>>it is different now so just have a patience, I'll get to it). In the init
>>function of the module kpatch builds all the relevant info from dynrela
>>section. It goes through it, creates an array of klp_reloc for each object
>>and includes each dynrela record with an objname to the array of
>>klp_object with that objname. Later when we need to apply relocations for
>>patched object we just go through the list (array) of its dynrelas in
>>klp_object and call our arch-specific klp_write_module_reloc().
>
>Sounds correct to me.
>
>>Now this was probably changed in kpatch and you do not have one dynrela
>>section but one dynrela section for each patched function. Is that
>>correct? (and can you tell us what the reason for the change was? It might
>>be crucial because I might be missing something.). Which leads us to your
>>proposal...
>
>Your original assumption was correct; current kpatch has one big
>.kpatch.dynrelas section, and each dynrela entry within that single
>section gets sorted to the correct object as you described above. The
>one dynrela section per patched function only came with this patchset,
>but for no particular reason other than to make the kpatch-build code
>for generating the patch module slightly less complicated. But I
>haven't checked how big of a change it would be to do
>one-dynrela-section per object, perhaps (and I hope) it will be an
>easy change.
>
>>2. So we have several dynrela section for one patched object. During init
>>function in a patch module kpatch once again builds needed structures from
>>these sections. Now they are called klp_reloc_sec and contain different
>>kind of info. There is no val, loc and such, only name of the symbol,
>>objname and index to dynrela section in ELF. So when you need to write
>>relocations for the patched object you go through all relevant dynrela
>>sections (because they are stored in the klp_object), all dynrela records
>>in each section and you resolve the undefined symbols. All needed info is
>>stored in ELF. Then you just call apply_relocate_add().
>>
>>3. I propose to go one step further. I think we don't need klp_reloc_sec
>>if there is only one dynrela section for patched object (and I currently
>>cannot see why this is not possible. It is possible even with one dynrela
>>section for whole patch module, that is for all patched objects.).
>
>I think the furthest we can go in terms of simplifying klp rela secs
>is to have one __klp_rela section per object. We can't smush all the
>__klp_rela_objname sections into one big __klp_rela section since we
>could be patching some objects that won't be loaded yet, and
>apply_relocate_add() needs to work with real SHT_RELA sections + their
>corresponding section indices (i.e., we cannot call
>apply_relocate_add() with that single, combined klp rela section).
>
>So, I think I would be OK with one __klp_rela section per object.

I just found a problem with this one __klp_rela section per object
approach, so I have to retract what I said above. I no longer think it
works, and the reason has to do with how apply_relocate_add() uses the
sh_info field of each relocation section to figure out to which
section the reloc section applies. Each relocation section references
the section to which it modifies, so this is a 1-1 correspondance we
cannot break. Thus we cannot simply combine all relocations into a
single __klp_rela section, because they would apply to multiple
different sections. See my reply to Josh.

This isn't a big problem anyway, we can just use an array of section
indices and still get rid of klp_reloc_sec. Plus the current patchset
already expects/implements multiple __klp_rela sections per object.

>>In my proposal there would be nothing done in init function of the patched
>>module (albeit some optimization mentioned later). When you call
>>klp_write_object_relocations you would go through all ELF section and find
>>the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and
>>objname is in the name of the section. It is the same thing you do in 2.
>>in the init function.). Then you go through all dynrela records in the
>>section, you do the same things which you do in the proposed patch above,
>>and call apply_relocate_add.
>
>I'm not sure I like having klp_write_object_relocations() repeatedly
>perform a loop through all the elf sections when we can pre-process
>this information in the patch module's init function, and "cache" the
>relevant klp section indices before passing things off to
>klp_write_object_relocations(). So how about this: we do the
>__klp_rela sec sorting in the init function of the patched module. The
>patch module would iterate through its own elf sections, and when it
>encounters a __klp_rela section, it looks at its objname, find the
>corresponding klp_object, and save the section index of the __klp_rela
>section in that klp_object. Then in klp_write_object_relocations, we
>just have to look at the saved section index for the corresponding
>object and access the klp rela section through that index, do the same
>processing in this patch and call apply_relocate_add().

s/index/indices/, so instead of saving a single section index, just
keep track of all the indices of the __klp_rela_objname sections in
each klp_object.

>>Now it would be crazy to go through all sections each time
>>klp_write_object_relocations is called (maybe it does not even matter but
>>nevertheless). So klp_object could have an index to its ELF dynrela
>>section. This index can be retrieved in the init function the same way you
>>do all the stuff with klp_reloc_sec.
>
>Ah, exactly what I said above :-D
>
>>If you insisted on more than one dynrela section for a patched object we
>>could have an array of indices there. Or whatever.
>>
>>It is almost the same as your proposal but in my opinion somewhat nicer.
>>We just use the info stored in ELF directly without unnecessary middle
>>layer (== klp_reloc_sec).
>>
>>Does it make sense? I hope it does. Would it work?
>
>It does make sense, and I think we can make it work without
>klp_reloc_sec. Thanks for the explanations.

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Jessica Yu [12/11/15 14:14 -0500]:
>+++ Miroslav Benes [12/11/15 16:27 +0100]:
>>On Wed, 11 Nov 2015, Jessica Yu wrote:
>>
>>>+++ Miroslav Benes [11/11/15 15:30 +0100]:
>>>> On Mon, 9 Nov 2015, Jessica Yu wrote:
>>>>
>>>> So I guess we don't need klp_reloc anymore.
>>>
>>>Yes, that's correct. I am noticing just now that I forgot to remove
>>>the klp_reloc struct definition from livepatch.h. That change will be
>>>reflected in v2...
>>>
>>>> If true, we should really
>>>> start thinking about proper documentation because there are going to be
>>>> plenty of assumptions about a patch module and we need to have it written
>>>> somewhere. Especially how the relocation sections look like.
>>>
>>>Agreed. As a first step the patch module format can perhaps be
>>>documented somewhere. Perhaps it's time we create
>>>Documentation/livepatch/? :-)
>>
>>Yes, I think so.
>>
>>>> > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>>>> > index 087a8c7..26c419f 100644
>>>> > --- a/kernel/livepatch/core.c
>>>> > +++ b/kernel/livepatch/core.c
>>>> > @@ -28,6 +28,8 @@
>>>> >  #include <linux/list.h>
>>>> >  #include <linux/kallsyms.h>
>>>> >  #include <linux/livepatch.h>
>>>> > +#include <linux/elf.h>
>>>> > +#include <asm/cacheflush.h>
>>>> >
>>>> >  /**
>>>> >   * struct klp_ops - structure for tracking registered ftrace ops structs
>>>> > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
>>>> > *pmod, const char *name,
>>>> >  }
>>>> >
>>>> >  static int klp_write_object_relocations(struct module *pmod,
>>>> > -					struct klp_object *obj)
>>>> > +					struct klp_object *obj,
>>>> > +					struct klp_patch *patch)
>>>> >  {
>>>> > -	int ret;
>>>> > -	struct klp_reloc *reloc;
>>>> > +	int relindex, num_relas;
>>>> > +	int i, ret = 0;
>>>> > +	unsigned long addr;
>>>> > +	unsigned int bind;
>>>> > +	char *symname;
>>>> > +	struct klp_reloc_sec *reloc_sec;
>>>> > +	struct load_info *info;
>>>> > +	Elf_Rela *rela;
>>>> > +	Elf_Sym *sym, *symtab;
>>>> > +	Elf_Shdr *symsect;
>>>> >
>>>> >  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>>> >  		return -EINVAL;
>>>> >
>>>> > -	if (WARN_ON(!obj->relocs))
>>>> > -		return -EINVAL;
>>>> > -
>>>> > -	for (reloc = obj->relocs; reloc->name; reloc++) {
>>>> > -		if (!klp_is_module(obj)) {
>>>> > -			ret = klp_verify_vmlinux_symbol(reloc->name,
>>>> > -							reloc->val);
>>>> > -			if (ret)
>>>> > -				return ret;
>>>> > -		} else {
>>>> > -			/* module, reloc->val needs to be discovered */
>>>> > -			if (reloc->external)
>>>> > -				ret = klp_find_external_symbol(pmod,
>>>> > -							       reloc->name,
>>>> > -							       &reloc->val);
>>>> > -			else
>>>> > -				ret = klp_find_object_symbol(obj->mod->name,
>>>> > -							     reloc->name,
>>>> > -							     &reloc->val);
>>>> > -			if (ret)
>>>> > -				return ret;
>>>> > -		}
>>>> > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>>>> > -					     reloc->val + reloc->addend);
>>>> > -		if (ret) {
>>>> > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
>>>> > (%d)\n",
>>>> > -			       reloc->name, reloc->val, ret);
>>>> > -			return ret;
>>>> > +	info = pmod->info;
>>>> > +	symsect = info->sechdrs + info->index.sym;
>>>> > +	symtab = (void *)info->hdr + symsect->sh_offset;
>>>> > +
>>>> > +	/* For each __klp_rela section for this object */
>>>> > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>>>> > +		relindex = reloc_sec->index;
>>>> > +		num_relas = info->sechdrs[relindex].sh_size /
>>>> > sizeof(Elf_Rela);
>>>> > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>>>> > +
>>>> > +		/* For each rela in this __klp_rela section */
>>>> > +		for (i = 0; i < num_relas; i++, rela++) {
>>>> > +			sym = symtab + ELF_R_SYM(rela->r_info);
>>>> > +			symname = info->strtab + sym->st_name;
>>>> > +			bind = ELF_ST_BIND(sym->st_info);
>>>> > +
>>>> > +			if (sym->st_shndx == SHN_LIVEPATCH) {
>>>> > +				if (bind == STB_LIVEPATCH_EXT)
>>>> > +					ret = klp_find_external_symbol(pmod,
>>>> > symname, &addr);
>>>> > +				else
>>>> > +					ret =
>>>> > klp_find_object_symbol(obj->name, symname, &addr);
>>>> > +				if (ret)
>>>> > +					return ret;
>>>> > +				sym->st_value = addr;
>>>> > +			}
>>>> >  		}
>>>> > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>>>> > +					 info->index.sym, relindex, pmod);
>>>> >  	}
>>>> >
>>>> > -	return 0;
>>>> > +	return ret;
>>>> >  }
>>>>
>>>> Looking at this... do we even need reloc_secs in klp_object? Question is
>>>> whether we need more than one dynrela section for an object. If not then
>>>> the binding between klp_reloc_sec and an object is the only relevant thing
>>>> in the structure, be it index or objname. So we can replace the
>>>> list of structures with just the index in klp_object, or get rid of it
>>>> completely and rely on the name of dynrela section be something like
>>>> __klp_rela_{objname}.
>>>
>>>Hm, you bring up a good point. I think theoretically yes, it is
>>>possible to just have one klp_reloc_sec for each object and therefore
>>>a list is not required (I have not checked yet how difficult it would
>>>be to implement this on the kpatch-build side of things).  However,
>>>considering the final format of the patch module, I think it is
>>>semantically clearer to leave it as a list, and for each object to
>>>possibly have more than one __klp_rela section.
>>>
>>>For example, say we are patching two functions in ext4. In my
>>>resulting kpatch module I will have two __klp_rela_ext4 sections, and
>>>they might look like this when we run readelf --sections:
>>>
>>>[34] __klp_rela_ext4.text.ext4_attr_store RELA ...
>>>[35] __klp_rela_ext4.text.ext4_attr_show RELA ...
>>>
>>>Then these two klp rela sections end up as two elements in the
>>>reloc_secs list for the ext4 patch object. I think this way, we can
>>>better tell which rela is being applied to what function. Might be
>>>easier to understand what's happening from the developer's point of
>>>view.
>>>
>>>> You see, we go through elf sections here which were preserved by module
>>>> loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>>>> maybe all the stuff around klp_reloc_sec is not necessary.
>>>>
>>>> Thoughts?
>>>
>>>Ah, so this is where descriptive comments and documentation might have
>>>been useful :-) So I think we will still need to keep the
>>>klp_reloc_sec struct to help the patch module initialize. Though the
>>>name and objname fields aren't used in this patchset, they are used in
>>>the kpatch patch module code [1], where we iterate through each elf
>>>section, find the ones marked with SHF_RELA_LIVEPATCH, set the
>>>klp_reloc_sec's objname (which we find from the "name" field,
>>>formatted as __klp_rela_{objname}.text..). Once we have the objname
>>>set, we can then find the object to attach the reloc_sec to (i.e. add
>>>it to its list of reloc_secs).
>>>
>>>Hope that clears some things up.
>>
>>Ok, I'll try to explain myself and it is gonna be long. I'll try to
>>describe how we deal with dynrelas in klp today, how you use it in kpatch
>>(and this is the place where my knowledge can be wrong or obsolete), what
>>you propose and what I'd like to propose.
>>
>>1. First let's look on what we have now.
>>
>>We have a patch module in which there is a section with all dynrelas
>>needed to be resolved (it was like this in kpatch some time ago and maybe
>>it is different now so just have a patience, I'll get to it). In the init
>>function of the module kpatch builds all the relevant info from dynrela
>>section. It goes through it, creates an array of klp_reloc for each object
>>and includes each dynrela record with an objname to the array of
>>klp_object with that objname. Later when we need to apply relocations for
>>patched object we just go through the list (array) of its dynrelas in
>>klp_object and call our arch-specific klp_write_module_reloc().
>
>Sounds correct to me.
>
>>Now this was probably changed in kpatch and you do not have one dynrela
>>section but one dynrela section for each patched function. Is that
>>correct? (and can you tell us what the reason for the change was? It might
>>be crucial because I might be missing something.). Which leads us to your
>>proposal...
>
>Your original assumption was correct; current kpatch has one big
>.kpatch.dynrelas section, and each dynrela entry within that single
>section gets sorted to the correct object as you described above. The
>one dynrela section per patched function only came with this patchset,
>but for no particular reason other than to make the kpatch-build code
>for generating the patch module slightly less complicated. But I
>haven't checked how big of a change it would be to do
>one-dynrela-section per object, perhaps (and I hope) it will be an
>easy change.
>
>>2. So we have several dynrela section for one patched object. During init
>>function in a patch module kpatch once again builds needed structures from
>>these sections. Now they are called klp_reloc_sec and contain different
>>kind of info. There is no val, loc and such, only name of the symbol,
>>objname and index to dynrela section in ELF. So when you need to write
>>relocations for the patched object you go through all relevant dynrela
>>sections (because they are stored in the klp_object), all dynrela records
>>in each section and you resolve the undefined symbols. All needed info is
>>stored in ELF. Then you just call apply_relocate_add().
>>
>>3. I propose to go one step further. I think we don't need klp_reloc_sec
>>if there is only one dynrela section for patched object (and I currently
>>cannot see why this is not possible. It is possible even with one dynrela
>>section for whole patch module, that is for all patched objects.).
>
>I think the furthest we can go in terms of simplifying klp rela secs
>is to have one __klp_rela section per object. We can't smush all the
>__klp_rela_objname sections into one big __klp_rela section since we
>could be patching some objects that won't be loaded yet, and
>apply_relocate_add() needs to work with real SHT_RELA sections + their
>corresponding section indices (i.e., we cannot call
>apply_relocate_add() with that single, combined klp rela section).
>
>So, I think I would be OK with one __klp_rela section per object.

I just found a problem with this one __klp_rela section per object
approach, so I have to retract what I said above. I no longer think it
works, and the reason has to do with how apply_relocate_add() uses the
sh_info field of each relocation section to figure out to which
section the reloc section applies. Each relocation section references
the section to which it modifies, so this is a 1-1 correspondance we
cannot break. Thus we cannot simply combine all relocations into a
single __klp_rela section, because they would apply to multiple
different sections. See my reply to Josh.

This isn't a big problem anyway, we can just use an array of section
indices and still get rid of klp_reloc_sec. Plus the current patchset
already expects/implements multiple __klp_rela sections per object.

>>In my proposal there would be nothing done in init function of the patched
>>module (albeit some optimization mentioned later). When you call
>>klp_write_object_relocations you would go through all ELF section and find
>>the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and
>>objname is in the name of the section. It is the same thing you do in 2.
>>in the init function.). Then you go through all dynrela records in the
>>section, you do the same things which you do in the proposed patch above,
>>and call apply_relocate_add.
>
>I'm not sure I like having klp_write_object_relocations() repeatedly
>perform a loop through all the elf sections when we can pre-process
>this information in the patch module's init function, and "cache" the
>relevant klp section indices before passing things off to
>klp_write_object_relocations(). So how about this: we do the
>__klp_rela sec sorting in the init function of the patched module. The
>patch module would iterate through its own elf sections, and when it
>encounters a __klp_rela section, it looks at its objname, find the
>corresponding klp_object, and save the section index of the __klp_rela
>section in that klp_object. Then in klp_write_object_relocations, we
>just have to look at the saved section index for the corresponding
>object and access the klp rela section through that index, do the same
>processing in this patch and call apply_relocate_add().

s/index/indices/, so instead of saving a single section index, just
keep track of all the indices of the __klp_rela_objname sections in
each klp_object.

>>Now it would be crazy to go through all sections each time
>>klp_write_object_relocations is called (maybe it does not even matter but
>>nevertheless). So klp_object could have an index to its ELF dynrela
>>section. This index can be retrieved in the init function the same way you
>>do all the stuff with klp_reloc_sec.
>
>Ah, exactly what I said above :-D
>
>>If you insisted on more than one dynrela section for a patched object we
>>could have an array of indices there. Or whatever.
>>
>>It is almost the same as your proposal but in my opinion somewhat nicer.
>>We just use the info stored in ELF directly without unnecessary middle
>>layer (== klp_reloc_sec).
>>
>>Does it make sense? I hope it does. Would it work?
>
>It does make sense, and I think we can make it work without
>klp_reloc_sec. Thanks for the explanations.

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Thu, Nov 12, 2015 at 03:22:44PM -0500, Jessica Yu wrote:
> Looking into this more, I think we do need one __klp_rela section per
> function being patched.  Each rela section is linked to the section to
> which the relocations apply via the rela section's sh_info field. In
> SHT_RELA sections, the sh_info field contains the section index to
> which the relocs apply. We cannot have one single combined rela
> section per object as the call to apply_relocate_add() simply won't
> work, because we would have relocs that apply to different functions
> (and hence different sections).
> 
> So I guess instead of a single field in klp_object specifying the
> __klp_rela section index, we could probably just have an array of
> section indices.

Ok, makes sense, sounds like we need multiple klp relas per object.

I still don't quite understand the benefit of caching the klp_rela
section indices.  What problem does it solve?  It seems simpler to just
iterate over all the sections in klp_write_object_relocations().

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Thu, Nov 12, 2015 at 03:22:44PM -0500, Jessica Yu wrote:
> Looking into this more, I think we do need one __klp_rela section per
> function being patched.  Each rela section is linked to the section to
> which the relocations apply via the rela section's sh_info field. In
> SHT_RELA sections, the sh_info field contains the section index to
> which the relocs apply. We cannot have one single combined rela
> section per object as the call to apply_relocate_add() simply won't
> work, because we would have relocs that apply to different functions
> (and hence different sections).
> 
> So I guess instead of a single field in klp_object specifying the
> __klp_rela section index, we could probably just have an array of
> section indices.

Ok, makes sense, sounds like we need multiple klp relas per object.

I still don't quite understand the benefit of caching the klp_rela
section indices.  What problem does it solve?  It seems simpler to just
iterate over all the sections in klp_write_object_relocations().

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [12/11/15 11:40 -0600]:
>On Thu, Nov 12, 2015 at 04:27:01PM +0100, Miroslav Benes wrote:
>> On Wed, 11 Nov 2015, Jessica Yu wrote:
>>
>> > +++ Miroslav Benes [11/11/15 15:30 +0100]:
>> > > On Mon, 9 Nov 2015, Jessica Yu wrote:
>> > >
>> > > So I guess we don't need klp_reloc anymore.
>> >
>> > Yes, that's correct. I am noticing just now that I forgot to remove
>> > the klp_reloc struct definition from livepatch.h. That change will be
>> > reflected in v2...
>> >
>> > > If true, we should really
>> > > start thinking about proper documentation because there are going to be
>> > > plenty of assumptions about a patch module and we need to have it written
>> > > somewhere. Especially how the relocation sections look like.
>> >
>> > Agreed. As a first step the patch module format can perhaps be
>> > documented somewhere. Perhaps it's time we create
>> > Documentation/livepatch/? :-)
>>
>> Yes, I think so.
>>
>> > > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> > > > index 087a8c7..26c419f 100644
>> > > > --- a/kernel/livepatch/core.c
>> > > > +++ b/kernel/livepatch/core.c
>> > > > @@ -28,6 +28,8 @@
>> > > >  #include <linux/list.h>
>> > > >  #include <linux/kallsyms.h>
>> > > >  #include <linux/livepatch.h>
>> > > > +#include <linux/elf.h>
>> > > > +#include <asm/cacheflush.h>
>> > > >
>> > > >  /**
>> > > >   * struct klp_ops - structure for tracking registered ftrace ops structs
>> > > > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
>> > > > *pmod, const char *name,
>> > > >  }
>> > > >
>> > > >  static int klp_write_object_relocations(struct module *pmod,
>> > > > -					struct klp_object *obj)
>> > > > +					struct klp_object *obj,
>> > > > +					struct klp_patch *patch)
>> > > >  {
>> > > > -	int ret;
>> > > > -	struct klp_reloc *reloc;
>> > > > +	int relindex, num_relas;
>> > > > +	int i, ret = 0;
>> > > > +	unsigned long addr;
>> > > > +	unsigned int bind;
>> > > > +	char *symname;
>> > > > +	struct klp_reloc_sec *reloc_sec;
>> > > > +	struct load_info *info;
>> > > > +	Elf_Rela *rela;
>> > > > +	Elf_Sym *sym, *symtab;
>> > > > +	Elf_Shdr *symsect;
>> > > >
>> > > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
>> > > >  		return -EINVAL;
>> > > >
>> > > > -	if (WARN_ON(!obj->relocs))
>> > > > -		return -EINVAL;
>> > > > -
>> > > > -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> > > > -		if (!klp_is_module(obj)) {
>> > > > -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> > > > -							reloc->val);
>> > > > -			if (ret)
>> > > > -				return ret;
>> > > > -		} else {
>> > > > -			/* module, reloc->val needs to be discovered */
>> > > > -			if (reloc->external)
>> > > > -				ret = klp_find_external_symbol(pmod,
>> > > > -							       reloc->name,
>> > > > -							       &reloc->val);
>> > > > -			else
>> > > > -				ret = klp_find_object_symbol(obj->mod->name,
>> > > > -							     reloc->name,
>> > > > -							     &reloc->val);
>> > > > -			if (ret)
>> > > > -				return ret;
>> > > > -		}
>> > > > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> > > > -					     reloc->val + reloc->addend);
>> > > > -		if (ret) {
>> > > > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
>> > > > (%d)\n",
>> > > > -			       reloc->name, reloc->val, ret);
>> > > > -			return ret;
>> > > > +	info = pmod->info;
>> > > > +	symsect = info->sechdrs + info->index.sym;
>> > > > +	symtab = (void *)info->hdr + symsect->sh_offset;
>> > > > +
>> > > > +	/* For each __klp_rela section for this object */
>> > > > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>> > > > +		relindex = reloc_sec->index;
>> > > > +		num_relas = info->sechdrs[relindex].sh_size /
>> > > > sizeof(Elf_Rela);
>> > > > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>> > > > +
>> > > > +		/* For each rela in this __klp_rela section */
>> > > > +		for (i = 0; i < num_relas; i++, rela++) {
>> > > > +			sym = symtab + ELF_R_SYM(rela->r_info);
>> > > > +			symname = info->strtab + sym->st_name;
>> > > > +			bind = ELF_ST_BIND(sym->st_info);
>> > > > +
>> > > > +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> > > > +				if (bind == STB_LIVEPATCH_EXT)
>> > > > +					ret = klp_find_external_symbol(pmod,
>> > > > symname, &addr);
>> > > > +				else
>> > > > +					ret =
>> > > > klp_find_object_symbol(obj->name, symname, &addr);
>> > > > +				if (ret)
>> > > > +					return ret;
>> > > > +				sym->st_value = addr;
>> > > > +			}
>> > > >  		}
>> > > > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>> > > > +					 info->index.sym, relindex, pmod);
>> > > >  	}
>> > > >
>> > > > -	return 0;
>> > > > +	return ret;
>> > > >  }
>> > >
>> > > Looking at this... do we even need reloc_secs in klp_object? Question is
>> > > whether we need more than one dynrela section for an object. If not then
>> > > the binding between klp_reloc_sec and an object is the only relevant thing
>> > > in the structure, be it index or objname. So we can replace the
>> > > list of structures with just the index in klp_object, or get rid of it
>> > > completely and rely on the name of dynrela section be something like
>> > > __klp_rela_{objname}.
>> >
>> > Hm, you bring up a good point. I think theoretically yes, it is
>> > possible to just have one klp_reloc_sec for each object and therefore
>> > a list is not required (I have not checked yet how difficult it would
>> > be to implement this on the kpatch-build side of things).  However,
>> > considering the final format of the patch module, I think it is
>> > semantically clearer to leave it as a list, and for each object to
>> > possibly have more than one __klp_rela section.
>> >
>> > For example, say we are patching two functions in ext4. In my
>> > resulting kpatch module I will have two __klp_rela_ext4 sections, and
>> > they might look like this when we run readelf --sections:
>> >
>> > [34] __klp_rela_ext4.text.ext4_attr_store RELA ...
>> > [35] __klp_rela_ext4.text.ext4_attr_show RELA ...
>> >
>> > Then these two klp rela sections end up as two elements in the
>> > reloc_secs list for the ext4 patch object. I think this way, we can
>> > better tell which rela is being applied to what function. Might be
>> > easier to understand what's happening from the developer's point of
>> > view.
>> >
>> > > You see, we go through elf sections here which were preserved by module
>> > > loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>> > > maybe all the stuff around klp_reloc_sec is not necessary.
>> > >
>> > > Thoughts?
>> >
>> > Ah, so this is where descriptive comments and documentation might have
>> > been useful :-) So I think we will still need to keep the
>> > klp_reloc_sec struct to help the patch module initialize. Though the
>> > name and objname fields aren't used in this patchset, they are used in
>> > the kpatch patch module code [1], where we iterate through each elf
>> > section, find the ones marked with SHF_RELA_LIVEPATCH, set the
>> > klp_reloc_sec's objname (which we find from the "name" field,
>> > formatted as __klp_rela_{objname}.text..). Once we have the objname
>> > set, we can then find the object to attach the reloc_sec to (i.e. add
>> > it to its list of reloc_secs).
>> >
>> > Hope that clears some things up.
>>
>> Ok, I'll try to explain myself and it is gonna be long. I'll try to
>> describe how we deal with dynrelas in klp today, how you use it in kpatch
>> (and this is the place where my knowledge can be wrong or obsolete), what
>> you propose and what I'd like to propose.
>>
>> 1. First let's look on what we have now.
>>
>> We have a patch module in which there is a section with all dynrelas
>> needed to be resolved (it was like this in kpatch some time ago and maybe
>> it is different now so just have a patience, I'll get to it). In the init
>> function of the module kpatch builds all the relevant info from dynrela
>> section. It goes through it, creates an array of klp_reloc for each object
>> and includes each dynrela record with an objname to the array of
>> klp_object with that objname. Later when we need to apply relocations for
>> patched object we just go through the list (array) of its dynrelas in
>> klp_object and call our arch-specific klp_write_module_reloc().
>>
>> Now this was probably changed in kpatch and you do not have one dynrela
>> section but one dynrela section for each patched function. Is that
>> correct? (and can you tell us what the reason for the change was? It might
>> be crucial because I might be missing something.). Which leads us to your
>> proposal...
>>
>> 2. So we have several dynrela section for one patched object. During init
>> function in a patch module kpatch once again builds needed structures from
>> these sections. Now they are called klp_reloc_sec and contain different
>> kind of info. There is no val, loc and such, only name of the symbol,
>> objname and index to dynrela section in ELF. So when you need to write
>> relocations for the patched object you go through all relevant dynrela
>> sections (because they are stored in the klp_object), all dynrela records
>> in each section and you resolve the undefined symbols. All needed info is
>> stored in ELF. Then you just call apply_relocate_add().
>>
>> 3. I propose to go one step further. I think we don't need klp_reloc_sec
>> if there is only one dynrela section for patched object (and I currently
>> cannot see why this is not possible. It is possible even with one dynrela
>> section for whole patch module, that is for all patched objects.).
>
>I think I agree that we don't need klp_reloc_sec, and that klp rela
>sections can presumably be discovered by iterating over the sections.
>
>But I don't think it matters much whether we have one klp rela section
>per object, or multiple rela sections per object.  Either way, can't we
>find them when we iterate over the sections?
>
>For example, for one rela section per object, it could be named:
>
>__klp_rela.objname
>
>Or for multiple rela sections per object, they could be named:
>
>__klp_rela.objname.func1
>__klp_rela.objname.func2
>
>Either way, when iterating over the sections, we could just look for
>"__klp_rela.objname".
>
>All that said, I think I would vote for one rela section per object,
>just because it seems simpler.

Looking into this more, I think we do need one __klp_rela section per
function being patched.  Each rela section is linked to the section to
which the relocations apply via the rela section's sh_info field. In
SHT_RELA sections, the sh_info field contains the section index to
which the relocs apply. We cannot have one single combined rela
section per object as the call to apply_relocate_add() simply won't
work, because we would have relocs that apply to different functions
(and hence different sections).

So I guess instead of a single field in klp_object specifying the
__klp_rela section index, we could probably just have an array of
section indices.

>>
>> In my proposal there would be nothing done in init function of the patched
>> module (albeit some optimization mentioned later). When you call
>> klp_write_object_relocations you would go through all ELF section and find
>> the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and
>> objname is in the name of the section. It is the same thing you do in 2.
>> in the init function.). Then you go through all dynrela records in the
>> section, you do the same things which you do in the proposed patch above,
>> and call apply_relocate_add.
>>
>> Now it would be crazy to go through all sections each time
>> klp_write_object_relocations is called (maybe it does not even matter but
>
>Why would that be crazy?  To me it seems perfectly logical :-)  It
>doesn't seem like a very expensive operation to me.
>
>> nevertheless). So klp_object could have an index to its ELF dynrela
>> section. This index can be retrieved in the init function the same way you
>> do all the stuff with klp_reloc_sec.
>>
>> If you insisted on more than one dynrela section for a patched object we
>> could have an array of indices there. Or whatever.
>>
>> It is almost the same as your proposal but in my opinion somewhat nicer.
>> We just use the info stored in ELF directly without unnecessary middle
>> layer (== klp_reloc_sec).
>>
>> Does it make sense? I hope it does. Would it work?
>
>-- 
>Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Josh Poimboeuf [12/11/15 11:40 -0600]:
>On Thu, Nov 12, 2015 at 04:27:01PM +0100, Miroslav Benes wrote:
>> On Wed, 11 Nov 2015, Jessica Yu wrote:
>>
>> > +++ Miroslav Benes [11/11/15 15:30 +0100]:
>> > > On Mon, 9 Nov 2015, Jessica Yu wrote:
>> > >
>> > > So I guess we don't need klp_reloc anymore.
>> >
>> > Yes, that's correct. I am noticing just now that I forgot to remove
>> > the klp_reloc struct definition from livepatch.h. That change will be
>> > reflected in v2...
>> >
>> > > If true, we should really
>> > > start thinking about proper documentation because there are going to be
>> > > plenty of assumptions about a patch module and we need to have it written
>> > > somewhere. Especially how the relocation sections look like.
>> >
>> > Agreed. As a first step the patch module format can perhaps be
>> > documented somewhere. Perhaps it's time we create
>> > Documentation/livepatch/? :-)
>>
>> Yes, I think so.
>>
>> > > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> > > > index 087a8c7..26c419f 100644
>> > > > --- a/kernel/livepatch/core.c
>> > > > +++ b/kernel/livepatch/core.c
>> > > > @@ -28,6 +28,8 @@
>> > > >  #include <linux/list.h>
>> > > >  #include <linux/kallsyms.h>
>> > > >  #include <linux/livepatch.h>
>> > > > +#include <linux/elf.h>
>> > > > +#include <asm/cacheflush.h>
>> > > >
>> > > >  /**
>> > > >   * struct klp_ops - structure for tracking registered ftrace ops structs
>> > > > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
>> > > > *pmod, const char *name,
>> > > >  }
>> > > >
>> > > >  static int klp_write_object_relocations(struct module *pmod,
>> > > > -					struct klp_object *obj)
>> > > > +					struct klp_object *obj,
>> > > > +					struct klp_patch *patch)
>> > > >  {
>> > > > -	int ret;
>> > > > -	struct klp_reloc *reloc;
>> > > > +	int relindex, num_relas;
>> > > > +	int i, ret = 0;
>> > > > +	unsigned long addr;
>> > > > +	unsigned int bind;
>> > > > +	char *symname;
>> > > > +	struct klp_reloc_sec *reloc_sec;
>> > > > +	struct load_info *info;
>> > > > +	Elf_Rela *rela;
>> > > > +	Elf_Sym *sym, *symtab;
>> > > > +	Elf_Shdr *symsect;
>> > > >
>> > > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
>> > > >  		return -EINVAL;
>> > > >
>> > > > -	if (WARN_ON(!obj->relocs))
>> > > > -		return -EINVAL;
>> > > > -
>> > > > -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> > > > -		if (!klp_is_module(obj)) {
>> > > > -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> > > > -							reloc->val);
>> > > > -			if (ret)
>> > > > -				return ret;
>> > > > -		} else {
>> > > > -			/* module, reloc->val needs to be discovered */
>> > > > -			if (reloc->external)
>> > > > -				ret = klp_find_external_symbol(pmod,
>> > > > -							       reloc->name,
>> > > > -							       &reloc->val);
>> > > > -			else
>> > > > -				ret = klp_find_object_symbol(obj->mod->name,
>> > > > -							     reloc->name,
>> > > > -							     &reloc->val);
>> > > > -			if (ret)
>> > > > -				return ret;
>> > > > -		}
>> > > > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> > > > -					     reloc->val + reloc->addend);
>> > > > -		if (ret) {
>> > > > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
>> > > > (%d)\n",
>> > > > -			       reloc->name, reloc->val, ret);
>> > > > -			return ret;
>> > > > +	info = pmod->info;
>> > > > +	symsect = info->sechdrs + info->index.sym;
>> > > > +	symtab = (void *)info->hdr + symsect->sh_offset;
>> > > > +
>> > > > +	/* For each __klp_rela section for this object */
>> > > > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>> > > > +		relindex = reloc_sec->index;
>> > > > +		num_relas = info->sechdrs[relindex].sh_size /
>> > > > sizeof(Elf_Rela);
>> > > > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>> > > > +
>> > > > +		/* For each rela in this __klp_rela section */
>> > > > +		for (i = 0; i < num_relas; i++, rela++) {
>> > > > +			sym = symtab + ELF_R_SYM(rela->r_info);
>> > > > +			symname = info->strtab + sym->st_name;
>> > > > +			bind = ELF_ST_BIND(sym->st_info);
>> > > > +
>> > > > +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> > > > +				if (bind == STB_LIVEPATCH_EXT)
>> > > > +					ret = klp_find_external_symbol(pmod,
>> > > > symname, &addr);
>> > > > +				else
>> > > > +					ret =
>> > > > klp_find_object_symbol(obj->name, symname, &addr);
>> > > > +				if (ret)
>> > > > +					return ret;
>> > > > +				sym->st_value = addr;
>> > > > +			}
>> > > >  		}
>> > > > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>> > > > +					 info->index.sym, relindex, pmod);
>> > > >  	}
>> > > >
>> > > > -	return 0;
>> > > > +	return ret;
>> > > >  }
>> > >
>> > > Looking at this... do we even need reloc_secs in klp_object? Question is
>> > > whether we need more than one dynrela section for an object. If not then
>> > > the binding between klp_reloc_sec and an object is the only relevant thing
>> > > in the structure, be it index or objname. So we can replace the
>> > > list of structures with just the index in klp_object, or get rid of it
>> > > completely and rely on the name of dynrela section be something like
>> > > __klp_rela_{objname}.
>> >
>> > Hm, you bring up a good point. I think theoretically yes, it is
>> > possible to just have one klp_reloc_sec for each object and therefore
>> > a list is not required (I have not checked yet how difficult it would
>> > be to implement this on the kpatch-build side of things).  However,
>> > considering the final format of the patch module, I think it is
>> > semantically clearer to leave it as a list, and for each object to
>> > possibly have more than one __klp_rela section.
>> >
>> > For example, say we are patching two functions in ext4. In my
>> > resulting kpatch module I will have two __klp_rela_ext4 sections, and
>> > they might look like this when we run readelf --sections:
>> >
>> > [34] __klp_rela_ext4.text.ext4_attr_store RELA ...
>> > [35] __klp_rela_ext4.text.ext4_attr_show RELA ...
>> >
>> > Then these two klp rela sections end up as two elements in the
>> > reloc_secs list for the ext4 patch object. I think this way, we can
>> > better tell which rela is being applied to what function. Might be
>> > easier to understand what's happening from the developer's point of
>> > view.
>> >
>> > > You see, we go through elf sections here which were preserved by module
>> > > loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>> > > maybe all the stuff around klp_reloc_sec is not necessary.
>> > >
>> > > Thoughts?
>> >
>> > Ah, so this is where descriptive comments and documentation might have
>> > been useful :-) So I think we will still need to keep the
>> > klp_reloc_sec struct to help the patch module initialize. Though the
>> > name and objname fields aren't used in this patchset, they are used in
>> > the kpatch patch module code [1], where we iterate through each elf
>> > section, find the ones marked with SHF_RELA_LIVEPATCH, set the
>> > klp_reloc_sec's objname (which we find from the "name" field,
>> > formatted as __klp_rela_{objname}.text..). Once we have the objname
>> > set, we can then find the object to attach the reloc_sec to (i.e. add
>> > it to its list of reloc_secs).
>> >
>> > Hope that clears some things up.
>>
>> Ok, I'll try to explain myself and it is gonna be long. I'll try to
>> describe how we deal with dynrelas in klp today, how you use it in kpatch
>> (and this is the place where my knowledge can be wrong or obsolete), what
>> you propose and what I'd like to propose.
>>
>> 1. First let's look on what we have now.
>>
>> We have a patch module in which there is a section with all dynrelas
>> needed to be resolved (it was like this in kpatch some time ago and maybe
>> it is different now so just have a patience, I'll get to it). In the init
>> function of the module kpatch builds all the relevant info from dynrela
>> section. It goes through it, creates an array of klp_reloc for each object
>> and includes each dynrela record with an objname to the array of
>> klp_object with that objname. Later when we need to apply relocations for
>> patched object we just go through the list (array) of its dynrelas in
>> klp_object and call our arch-specific klp_write_module_reloc().
>>
>> Now this was probably changed in kpatch and you do not have one dynrela
>> section but one dynrela section for each patched function. Is that
>> correct? (and can you tell us what the reason for the change was? It might
>> be crucial because I might be missing something.). Which leads us to your
>> proposal...
>>
>> 2. So we have several dynrela section for one patched object. During init
>> function in a patch module kpatch once again builds needed structures from
>> these sections. Now they are called klp_reloc_sec and contain different
>> kind of info. There is no val, loc and such, only name of the symbol,
>> objname and index to dynrela section in ELF. So when you need to write
>> relocations for the patched object you go through all relevant dynrela
>> sections (because they are stored in the klp_object), all dynrela records
>> in each section and you resolve the undefined symbols. All needed info is
>> stored in ELF. Then you just call apply_relocate_add().
>>
>> 3. I propose to go one step further. I think we don't need klp_reloc_sec
>> if there is only one dynrela section for patched object (and I currently
>> cannot see why this is not possible. It is possible even with one dynrela
>> section for whole patch module, that is for all patched objects.).
>
>I think I agree that we don't need klp_reloc_sec, and that klp rela
>sections can presumably be discovered by iterating over the sections.
>
>But I don't think it matters much whether we have one klp rela section
>per object, or multiple rela sections per object.  Either way, can't we
>find them when we iterate over the sections?
>
>For example, for one rela section per object, it could be named:
>
>__klp_rela.objname
>
>Or for multiple rela sections per object, they could be named:
>
>__klp_rela.objname.func1
>__klp_rela.objname.func2
>
>Either way, when iterating over the sections, we could just look for
>"__klp_rela.objname".
>
>All that said, I think I would vote for one rela section per object,
>just because it seems simpler.

Looking into this more, I think we do need one __klp_rela section per
function being patched.  Each rela section is linked to the section to
which the relocations apply via the rela section's sh_info field. In
SHT_RELA sections, the sh_info field contains the section index to
which the relocs apply. We cannot have one single combined rela
section per object as the call to apply_relocate_add() simply won't
work, because we would have relocs that apply to different functions
(and hence different sections).

So I guess instead of a single field in klp_object specifying the
__klp_rela section index, we could probably just have an array of
section indices.

>>
>> In my proposal there would be nothing done in init function of the patched
>> module (albeit some optimization mentioned later). When you call
>> klp_write_object_relocations you would go through all ELF section and find
>> the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and
>> objname is in the name of the section. It is the same thing you do in 2.
>> in the init function.). Then you go through all dynrela records in the
>> section, you do the same things which you do in the proposed patch above,
>> and call apply_relocate_add.
>>
>> Now it would be crazy to go through all sections each time
>> klp_write_object_relocations is called (maybe it does not even matter but
>
>Why would that be crazy?  To me it seems perfectly logical :-)  It
>doesn't seem like a very expensive operation to me.
>
>> nevertheless). So klp_object could have an index to its ELF dynrela
>> section. This index can be retrieved in the init function the same way you
>> do all the stuff with klp_reloc_sec.
>>
>> If you insisted on more than one dynrela section for a patched object we
>> could have an array of indices there. Or whatever.
>>
>> It is almost the same as your proposal but in my opinion somewhat nicer.
>> We just use the info stored in ELF directly without unnecessary middle
>> layer (== klp_reloc_sec).
>>
>> Does it make sense? I hope it does. Would it work?
>
>-- 
>Josh

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [12/11/15 16:27 +0100]:
>On Wed, 11 Nov 2015, Jessica Yu wrote:
>
>> +++ Miroslav Benes [11/11/15 15:30 +0100]:
>> > On Mon, 9 Nov 2015, Jessica Yu wrote:
>> >
>> > So I guess we don't need klp_reloc anymore.
>>
>> Yes, that's correct. I am noticing just now that I forgot to remove
>> the klp_reloc struct definition from livepatch.h. That change will be
>> reflected in v2...
>>
>> > If true, we should really
>> > start thinking about proper documentation because there are going to be
>> > plenty of assumptions about a patch module and we need to have it written
>> > somewhere. Especially how the relocation sections look like.
>>
>> Agreed. As a first step the patch module format can perhaps be
>> documented somewhere. Perhaps it's time we create
>> Documentation/livepatch/? :-)
>
>Yes, I think so.
>
>> > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> > > index 087a8c7..26c419f 100644
>> > > --- a/kernel/livepatch/core.c
>> > > +++ b/kernel/livepatch/core.c
>> > > @@ -28,6 +28,8 @@
>> > >  #include <linux/list.h>
>> > >  #include <linux/kallsyms.h>
>> > >  #include <linux/livepatch.h>
>> > > +#include <linux/elf.h>
>> > > +#include <asm/cacheflush.h>
>> > >
>> > >  /**
>> > >   * struct klp_ops - structure for tracking registered ftrace ops structs
>> > > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
>> > > *pmod, const char *name,
>> > >  }
>> > >
>> > >  static int klp_write_object_relocations(struct module *pmod,
>> > > -					struct klp_object *obj)
>> > > +					struct klp_object *obj,
>> > > +					struct klp_patch *patch)
>> > >  {
>> > > -	int ret;
>> > > -	struct klp_reloc *reloc;
>> > > +	int relindex, num_relas;
>> > > +	int i, ret = 0;
>> > > +	unsigned long addr;
>> > > +	unsigned int bind;
>> > > +	char *symname;
>> > > +	struct klp_reloc_sec *reloc_sec;
>> > > +	struct load_info *info;
>> > > +	Elf_Rela *rela;
>> > > +	Elf_Sym *sym, *symtab;
>> > > +	Elf_Shdr *symsect;
>> > >
>> > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
>> > >  		return -EINVAL;
>> > >
>> > > -	if (WARN_ON(!obj->relocs))
>> > > -		return -EINVAL;
>> > > -
>> > > -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> > > -		if (!klp_is_module(obj)) {
>> > > -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> > > -							reloc->val);
>> > > -			if (ret)
>> > > -				return ret;
>> > > -		} else {
>> > > -			/* module, reloc->val needs to be discovered */
>> > > -			if (reloc->external)
>> > > -				ret = klp_find_external_symbol(pmod,
>> > > -							       reloc->name,
>> > > -							       &reloc->val);
>> > > -			else
>> > > -				ret = klp_find_object_symbol(obj->mod->name,
>> > > -							     reloc->name,
>> > > -							     &reloc->val);
>> > > -			if (ret)
>> > > -				return ret;
>> > > -		}
>> > > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> > > -					     reloc->val + reloc->addend);
>> > > -		if (ret) {
>> > > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
>> > > (%d)\n",
>> > > -			       reloc->name, reloc->val, ret);
>> > > -			return ret;
>> > > +	info = pmod->info;
>> > > +	symsect = info->sechdrs + info->index.sym;
>> > > +	symtab = (void *)info->hdr + symsect->sh_offset;
>> > > +
>> > > +	/* For each __klp_rela section for this object */
>> > > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>> > > +		relindex = reloc_sec->index;
>> > > +		num_relas = info->sechdrs[relindex].sh_size /
>> > > sizeof(Elf_Rela);
>> > > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>> > > +
>> > > +		/* For each rela in this __klp_rela section */
>> > > +		for (i = 0; i < num_relas; i++, rela++) {
>> > > +			sym = symtab + ELF_R_SYM(rela->r_info);
>> > > +			symname = info->strtab + sym->st_name;
>> > > +			bind = ELF_ST_BIND(sym->st_info);
>> > > +
>> > > +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> > > +				if (bind == STB_LIVEPATCH_EXT)
>> > > +					ret = klp_find_external_symbol(pmod,
>> > > symname, &addr);
>> > > +				else
>> > > +					ret =
>> > > klp_find_object_symbol(obj->name, symname, &addr);
>> > > +				if (ret)
>> > > +					return ret;
>> > > +				sym->st_value = addr;
>> > > +			}
>> > >  		}
>> > > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>> > > +					 info->index.sym, relindex, pmod);
>> > >  	}
>> > >
>> > > -	return 0;
>> > > +	return ret;
>> > >  }
>> >
>> > Looking at this... do we even need reloc_secs in klp_object? Question is
>> > whether we need more than one dynrela section for an object. If not then
>> > the binding between klp_reloc_sec and an object is the only relevant thing
>> > in the structure, be it index or objname. So we can replace the
>> > list of structures with just the index in klp_object, or get rid of it
>> > completely and rely on the name of dynrela section be something like
>> > __klp_rela_{objname}.
>>
>> Hm, you bring up a good point. I think theoretically yes, it is
>> possible to just have one klp_reloc_sec for each object and therefore
>> a list is not required (I have not checked yet how difficult it would
>> be to implement this on the kpatch-build side of things).  However,
>> considering the final format of the patch module, I think it is
>> semantically clearer to leave it as a list, and for each object to
>> possibly have more than one __klp_rela section.
>>
>> For example, say we are patching two functions in ext4. In my
>> resulting kpatch module I will have two __klp_rela_ext4 sections, and
>> they might look like this when we run readelf --sections:
>>
>> [34] __klp_rela_ext4.text.ext4_attr_store RELA ...
>> [35] __klp_rela_ext4.text.ext4_attr_show RELA ...
>>
>> Then these two klp rela sections end up as two elements in the
>> reloc_secs list for the ext4 patch object. I think this way, we can
>> better tell which rela is being applied to what function. Might be
>> easier to understand what's happening from the developer's point of
>> view.
>>
>> > You see, we go through elf sections here which were preserved by module
>> > loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>> > maybe all the stuff around klp_reloc_sec is not necessary.
>> >
>> > Thoughts?
>>
>> Ah, so this is where descriptive comments and documentation might have
>> been useful :-) So I think we will still need to keep the
>> klp_reloc_sec struct to help the patch module initialize. Though the
>> name and objname fields aren't used in this patchset, they are used in
>> the kpatch patch module code [1], where we iterate through each elf
>> section, find the ones marked with SHF_RELA_LIVEPATCH, set the
>> klp_reloc_sec's objname (which we find from the "name" field,
>> formatted as __klp_rela_{objname}.text..). Once we have the objname
>> set, we can then find the object to attach the reloc_sec to (i.e. add
>> it to its list of reloc_secs).
>>
>> Hope that clears some things up.
>
>Ok, I'll try to explain myself and it is gonna be long. I'll try to
>describe how we deal with dynrelas in klp today, how you use it in kpatch
>(and this is the place where my knowledge can be wrong or obsolete), what
>you propose and what I'd like to propose.
>
>1. First let's look on what we have now.
>
>We have a patch module in which there is a section with all dynrelas
>needed to be resolved (it was like this in kpatch some time ago and maybe
>it is different now so just have a patience, I'll get to it). In the init
>function of the module kpatch builds all the relevant info from dynrela
>section. It goes through it, creates an array of klp_reloc for each object
>and includes each dynrela record with an objname to the array of
>klp_object with that objname. Later when we need to apply relocations for
>patched object we just go through the list (array) of its dynrelas in
>klp_object and call our arch-specific klp_write_module_reloc().

Sounds correct to me.

>Now this was probably changed in kpatch and you do not have one dynrela
>section but one dynrela section for each patched function. Is that
>correct? (and can you tell us what the reason for the change was? It might
>be crucial because I might be missing something.). Which leads us to your
>proposal...

Your original assumption was correct; current kpatch has one big
.kpatch.dynrelas section, and each dynrela entry within that single
section gets sorted to the correct object as you described above. The
one dynrela section per patched function only came with this patchset,
but for no particular reason other than to make the kpatch-build code
for generating the patch module slightly less complicated. But I
haven't checked how big of a change it would be to do
one-dynrela-section per object, perhaps (and I hope) it will be an
easy change.

>2. So we have several dynrela section for one patched object. During init
>function in a patch module kpatch once again builds needed structures from
>these sections. Now they are called klp_reloc_sec and contain different
>kind of info. There is no val, loc and such, only name of the symbol,
>objname and index to dynrela section in ELF. So when you need to write
>relocations for the patched object you go through all relevant dynrela
>sections (because they are stored in the klp_object), all dynrela records
>in each section and you resolve the undefined symbols. All needed info is
>stored in ELF. Then you just call apply_relocate_add().
>
>3. I propose to go one step further. I think we don't need klp_reloc_sec
>if there is only one dynrela section for patched object (and I currently
>cannot see why this is not possible. It is possible even with one dynrela
>section for whole patch module, that is for all patched objects.).

I think the furthest we can go in terms of simplifying klp rela secs
is to have one __klp_rela section per object. We can't smush all the
__klp_rela_objname sections into one big __klp_rela section since we
could be patching some objects that won't be loaded yet, and
apply_relocate_add() needs to work with real SHT_RELA sections + their
corresponding section indices (i.e., we cannot call
apply_relocate_add() with that single, combined klp rela section).

So, I think I would be OK with one __klp_rela section per object.

>In my proposal there would be nothing done in init function of the patched
>module (albeit some optimization mentioned later). When you call
>klp_write_object_relocations you would go through all ELF section and find
>the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and
>objname is in the name of the section. It is the same thing you do in 2.
>in the init function.). Then you go through all dynrela records in the
>section, you do the same things which you do in the proposed patch above,
>and call apply_relocate_add.

I'm not sure I like having klp_write_object_relocations() repeatedly
perform a loop through all the elf sections when we can pre-process
this information in the patch module's init function, and "cache" the
relevant klp section indices before passing things off to
klp_write_object_relocations(). So how about this: we do the
__klp_rela sec sorting in the init function of the patched module. The
patch module would iterate through its own elf sections, and when it
encounters a __klp_rela section, it looks at its objname, find the
corresponding klp_object, and save the section index of the __klp_rela
section in that klp_object. Then in klp_write_object_relocations, we
just have to look at the saved section index for the corresponding
object and access the klp rela section through that index, do the same
processing in this patch and call apply_relocate_add().

>Now it would be crazy to go through all sections each time
>klp_write_object_relocations is called (maybe it does not even matter but
>nevertheless). So klp_object could have an index to its ELF dynrela
>section. This index can be retrieved in the init function the same way you
>do all the stuff with klp_reloc_sec.

Ah, exactly what I said above :-D

>If you insisted on more than one dynrela section for a patched object we
>could have an array of indices there. Or whatever.
>
>It is almost the same as your proposal but in my opinion somewhat nicer.
>We just use the info stored in ELF directly without unnecessary middle
>layer (== klp_reloc_sec).
>
>Does it make sense? I hope it does. Would it work?

It does make sense, and I think we can make it work without
klp_reloc_sec. Thanks for the explanations.

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [12/11/15 16:27 +0100]:
>On Wed, 11 Nov 2015, Jessica Yu wrote:
>
>> +++ Miroslav Benes [11/11/15 15:30 +0100]:
>> > On Mon, 9 Nov 2015, Jessica Yu wrote:
>> >
>> > So I guess we don't need klp_reloc anymore.
>>
>> Yes, that's correct. I am noticing just now that I forgot to remove
>> the klp_reloc struct definition from livepatch.h. That change will be
>> reflected in v2...
>>
>> > If true, we should really
>> > start thinking about proper documentation because there are going to be
>> > plenty of assumptions about a patch module and we need to have it written
>> > somewhere. Especially how the relocation sections look like.
>>
>> Agreed. As a first step the patch module format can perhaps be
>> documented somewhere. Perhaps it's time we create
>> Documentation/livepatch/? :-)
>
>Yes, I think so.
>
>> > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> > > index 087a8c7..26c419f 100644
>> > > --- a/kernel/livepatch/core.c
>> > > +++ b/kernel/livepatch/core.c
>> > > @@ -28,6 +28,8 @@
>> > >  #include <linux/list.h>
>> > >  #include <linux/kallsyms.h>
>> > >  #include <linux/livepatch.h>
>> > > +#include <linux/elf.h>
>> > > +#include <asm/cacheflush.h>
>> > >
>> > >  /**
>> > >   * struct klp_ops - structure for tracking registered ftrace ops structs
>> > > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
>> > > *pmod, const char *name,
>> > >  }
>> > >
>> > >  static int klp_write_object_relocations(struct module *pmod,
>> > > -					struct klp_object *obj)
>> > > +					struct klp_object *obj,
>> > > +					struct klp_patch *patch)
>> > >  {
>> > > -	int ret;
>> > > -	struct klp_reloc *reloc;
>> > > +	int relindex, num_relas;
>> > > +	int i, ret = 0;
>> > > +	unsigned long addr;
>> > > +	unsigned int bind;
>> > > +	char *symname;
>> > > +	struct klp_reloc_sec *reloc_sec;
>> > > +	struct load_info *info;
>> > > +	Elf_Rela *rela;
>> > > +	Elf_Sym *sym, *symtab;
>> > > +	Elf_Shdr *symsect;
>> > >
>> > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
>> > >  		return -EINVAL;
>> > >
>> > > -	if (WARN_ON(!obj->relocs))
>> > > -		return -EINVAL;
>> > > -
>> > > -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> > > -		if (!klp_is_module(obj)) {
>> > > -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> > > -							reloc->val);
>> > > -			if (ret)
>> > > -				return ret;
>> > > -		} else {
>> > > -			/* module, reloc->val needs to be discovered */
>> > > -			if (reloc->external)
>> > > -				ret = klp_find_external_symbol(pmod,
>> > > -							       reloc->name,
>> > > -							       &reloc->val);
>> > > -			else
>> > > -				ret = klp_find_object_symbol(obj->mod->name,
>> > > -							     reloc->name,
>> > > -							     &reloc->val);
>> > > -			if (ret)
>> > > -				return ret;
>> > > -		}
>> > > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> > > -					     reloc->val + reloc->addend);
>> > > -		if (ret) {
>> > > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
>> > > (%d)\n",
>> > > -			       reloc->name, reloc->val, ret);
>> > > -			return ret;
>> > > +	info = pmod->info;
>> > > +	symsect = info->sechdrs + info->index.sym;
>> > > +	symtab = (void *)info->hdr + symsect->sh_offset;
>> > > +
>> > > +	/* For each __klp_rela section for this object */
>> > > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>> > > +		relindex = reloc_sec->index;
>> > > +		num_relas = info->sechdrs[relindex].sh_size /
>> > > sizeof(Elf_Rela);
>> > > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>> > > +
>> > > +		/* For each rela in this __klp_rela section */
>> > > +		for (i = 0; i < num_relas; i++, rela++) {
>> > > +			sym = symtab + ELF_R_SYM(rela->r_info);
>> > > +			symname = info->strtab + sym->st_name;
>> > > +			bind = ELF_ST_BIND(sym->st_info);
>> > > +
>> > > +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> > > +				if (bind == STB_LIVEPATCH_EXT)
>> > > +					ret = klp_find_external_symbol(pmod,
>> > > symname, &addr);
>> > > +				else
>> > > +					ret =
>> > > klp_find_object_symbol(obj->name, symname, &addr);
>> > > +				if (ret)
>> > > +					return ret;
>> > > +				sym->st_value = addr;
>> > > +			}
>> > >  		}
>> > > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>> > > +					 info->index.sym, relindex, pmod);
>> > >  	}
>> > >
>> > > -	return 0;
>> > > +	return ret;
>> > >  }
>> >
>> > Looking at this... do we even need reloc_secs in klp_object? Question is
>> > whether we need more than one dynrela section for an object. If not then
>> > the binding between klp_reloc_sec and an object is the only relevant thing
>> > in the structure, be it index or objname. So we can replace the
>> > list of structures with just the index in klp_object, or get rid of it
>> > completely and rely on the name of dynrela section be something like
>> > __klp_rela_{objname}.
>>
>> Hm, you bring up a good point. I think theoretically yes, it is
>> possible to just have one klp_reloc_sec for each object and therefore
>> a list is not required (I have not checked yet how difficult it would
>> be to implement this on the kpatch-build side of things).  However,
>> considering the final format of the patch module, I think it is
>> semantically clearer to leave it as a list, and for each object to
>> possibly have more than one __klp_rela section.
>>
>> For example, say we are patching two functions in ext4. In my
>> resulting kpatch module I will have two __klp_rela_ext4 sections, and
>> they might look like this when we run readelf --sections:
>>
>> [34] __klp_rela_ext4.text.ext4_attr_store RELA ...
>> [35] __klp_rela_ext4.text.ext4_attr_show RELA ...
>>
>> Then these two klp rela sections end up as two elements in the
>> reloc_secs list for the ext4 patch object. I think this way, we can
>> better tell which rela is being applied to what function. Might be
>> easier to understand what's happening from the developer's point of
>> view.
>>
>> > You see, we go through elf sections here which were preserved by module
>> > loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>> > maybe all the stuff around klp_reloc_sec is not necessary.
>> >
>> > Thoughts?
>>
>> Ah, so this is where descriptive comments and documentation might have
>> been useful :-) So I think we will still need to keep the
>> klp_reloc_sec struct to help the patch module initialize. Though the
>> name and objname fields aren't used in this patchset, they are used in
>> the kpatch patch module code [1], where we iterate through each elf
>> section, find the ones marked with SHF_RELA_LIVEPATCH, set the
>> klp_reloc_sec's objname (which we find from the "name" field,
>> formatted as __klp_rela_{objname}.text..). Once we have the objname
>> set, we can then find the object to attach the reloc_sec to (i.e. add
>> it to its list of reloc_secs).
>>
>> Hope that clears some things up.
>
>Ok, I'll try to explain myself and it is gonna be long. I'll try to
>describe how we deal with dynrelas in klp today, how you use it in kpatch
>(and this is the place where my knowledge can be wrong or obsolete), what
>you propose and what I'd like to propose.
>
>1. First let's look on what we have now.
>
>We have a patch module in which there is a section with all dynrelas
>needed to be resolved (it was like this in kpatch some time ago and maybe
>it is different now so just have a patience, I'll get to it). In the init
>function of the module kpatch builds all the relevant info from dynrela
>section. It goes through it, creates an array of klp_reloc for each object
>and includes each dynrela record with an objname to the array of
>klp_object with that objname. Later when we need to apply relocations for
>patched object we just go through the list (array) of its dynrelas in
>klp_object and call our arch-specific klp_write_module_reloc().

Sounds correct to me.

>Now this was probably changed in kpatch and you do not have one dynrela
>section but one dynrela section for each patched function. Is that
>correct? (and can you tell us what the reason for the change was? It might
>be crucial because I might be missing something.). Which leads us to your
>proposal...

Your original assumption was correct; current kpatch has one big
.kpatch.dynrelas section, and each dynrela entry within that single
section gets sorted to the correct object as you described above. The
one dynrela section per patched function only came with this patchset,
but for no particular reason other than to make the kpatch-build code
for generating the patch module slightly less complicated. But I
haven't checked how big of a change it would be to do
one-dynrela-section per object, perhaps (and I hope) it will be an
easy change.

>2. So we have several dynrela section for one patched object. During init
>function in a patch module kpatch once again builds needed structures from
>these sections. Now they are called klp_reloc_sec and contain different
>kind of info. There is no val, loc and such, only name of the symbol,
>objname and index to dynrela section in ELF. So when you need to write
>relocations for the patched object you go through all relevant dynrela
>sections (because they are stored in the klp_object), all dynrela records
>in each section and you resolve the undefined symbols. All needed info is
>stored in ELF. Then you just call apply_relocate_add().
>
>3. I propose to go one step further. I think we don't need klp_reloc_sec
>if there is only one dynrela section for patched object (and I currently
>cannot see why this is not possible. It is possible even with one dynrela
>section for whole patch module, that is for all patched objects.).

I think the furthest we can go in terms of simplifying klp rela secs
is to have one __klp_rela section per object. We can't smush all the
__klp_rela_objname sections into one big __klp_rela section since we
could be patching some objects that won't be loaded yet, and
apply_relocate_add() needs to work with real SHT_RELA sections + their
corresponding section indices (i.e., we cannot call
apply_relocate_add() with that single, combined klp rela section).

So, I think I would be OK with one __klp_rela section per object.

>In my proposal there would be nothing done in init function of the patched
>module (albeit some optimization mentioned later). When you call
>klp_write_object_relocations you would go through all ELF section and find
>the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and
>objname is in the name of the section. It is the same thing you do in 2.
>in the init function.). Then you go through all dynrela records in the
>section, you do the same things which you do in the proposed patch above,
>and call apply_relocate_add.

I'm not sure I like having klp_write_object_relocations() repeatedly
perform a loop through all the elf sections when we can pre-process
this information in the patch module's init function, and "cache" the
relevant klp section indices before passing things off to
klp_write_object_relocations(). So how about this: we do the
__klp_rela sec sorting in the init function of the patched module. The
patch module would iterate through its own elf sections, and when it
encounters a __klp_rela section, it looks at its objname, find the
corresponding klp_object, and save the section index of the __klp_rela
section in that klp_object. Then in klp_write_object_relocations, we
just have to look at the saved section index for the corresponding
object and access the klp rela section through that index, do the same
processing in this patch and call apply_relocate_add().

>Now it would be crazy to go through all sections each time
>klp_write_object_relocations is called (maybe it does not even matter but
>nevertheless). So klp_object could have an index to its ELF dynrela
>section. This index can be retrieved in the init function the same way you
>do all the stuff with klp_reloc_sec.

Ah, exactly what I said above :-D

>If you insisted on more than one dynrela section for a patched object we
>could have an array of indices there. Or whatever.
>
>It is almost the same as your proposal but in my opinion somewhat nicer.
>We just use the info stored in ELF directly without unnecessary middle
>layer (== klp_reloc_sec).
>
>Does it make sense? I hope it does. Would it work?

It does make sense, and I think we can make it work without
klp_reloc_sec. Thanks for the explanations.

Jessica

Re: livepatch: reuse module loader code to write relocations

[Josh Poimboeuf]

On Thu, Nov 12, 2015 at 04:27:01PM +0100, Miroslav Benes wrote:
> On Wed, 11 Nov 2015, Jessica Yu wrote:
> 
> > +++ Miroslav Benes [11/11/15 15:30 +0100]:
> > > On Mon, 9 Nov 2015, Jessica Yu wrote:
> > >
> > > So I guess we don't need klp_reloc anymore.
> > 
> > Yes, that's correct. I am noticing just now that I forgot to remove
> > the klp_reloc struct definition from livepatch.h. That change will be
> > reflected in v2...
> > 
> > > If true, we should really
> > > start thinking about proper documentation because there are going to be
> > > plenty of assumptions about a patch module and we need to have it written
> > > somewhere. Especially how the relocation sections look like.
> > 
> > Agreed. As a first step the patch module format can perhaps be
> > documented somewhere. Perhaps it's time we create
> > Documentation/livepatch/? :-)
> 
> Yes, I think so.
> 
> > > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
> > > > index 087a8c7..26c419f 100644
> > > > --- a/kernel/livepatch/core.c
> > > > +++ b/kernel/livepatch/core.c
> > > > @@ -28,6 +28,8 @@
> > > >  #include <linux/list.h>
> > > >  #include <linux/kallsyms.h>
> > > >  #include <linux/livepatch.h>
> > > > +#include <linux/elf.h>
> > > > +#include <asm/cacheflush.h>
> > > > 
> > > >  /**
> > > >   * struct klp_ops - structure for tracking registered ftrace ops structs
> > > > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
> > > > *pmod, const char *name,
> > > >  }
> > > > 
> > > >  static int klp_write_object_relocations(struct module *pmod,
> > > > -					struct klp_object *obj)
> > > > +					struct klp_object *obj,
> > > > +					struct klp_patch *patch)
> > > >  {
> > > > -	int ret;
> > > > -	struct klp_reloc *reloc;
> > > > +	int relindex, num_relas;
> > > > +	int i, ret = 0;
> > > > +	unsigned long addr;
> > > > +	unsigned int bind;
> > > > +	char *symname;
> > > > +	struct klp_reloc_sec *reloc_sec;
> > > > +	struct load_info *info;
> > > > +	Elf_Rela *rela;
> > > > +	Elf_Sym *sym, *symtab;
> > > > +	Elf_Shdr *symsect;
> > > > 
> > > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
> > > >  		return -EINVAL;
> > > > 
> > > > -	if (WARN_ON(!obj->relocs))
> > > > -		return -EINVAL;
> > > > -
> > > > -	for (reloc = obj->relocs; reloc->name; reloc++) {
> > > > -		if (!klp_is_module(obj)) {
> > > > -			ret = klp_verify_vmlinux_symbol(reloc->name,
> > > > -							reloc->val);
> > > > -			if (ret)
> > > > -				return ret;
> > > > -		} else {
> > > > -			/* module, reloc->val needs to be discovered */
> > > > -			if (reloc->external)
> > > > -				ret = klp_find_external_symbol(pmod,
> > > > -							       reloc->name,
> > > > -							       &reloc->val);
> > > > -			else
> > > > -				ret = klp_find_object_symbol(obj->mod->name,
> > > > -							     reloc->name,
> > > > -							     &reloc->val);
> > > > -			if (ret)
> > > > -				return ret;
> > > > -		}
> > > > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
> > > > -					     reloc->val + reloc->addend);
> > > > -		if (ret) {
> > > > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
> > > > (%d)\n",
> > > > -			       reloc->name, reloc->val, ret);
> > > > -			return ret;
> > > > +	info = pmod->info;
> > > > +	symsect = info->sechdrs + info->index.sym;
> > > > +	symtab = (void *)info->hdr + symsect->sh_offset;
> > > > +
> > > > +	/* For each __klp_rela section for this object */
> > > > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
> > > > +		relindex = reloc_sec->index;
> > > > +		num_relas = info->sechdrs[relindex].sh_size /
> > > > sizeof(Elf_Rela);
> > > > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
> > > > +
> > > > +		/* For each rela in this __klp_rela section */
> > > > +		for (i = 0; i < num_relas; i++, rela++) {
> > > > +			sym = symtab + ELF_R_SYM(rela->r_info);
> > > > +			symname = info->strtab + sym->st_name;
> > > > +			bind = ELF_ST_BIND(sym->st_info);
> > > > +
> > > > +			if (sym->st_shndx == SHN_LIVEPATCH) {
> > > > +				if (bind == STB_LIVEPATCH_EXT)
> > > > +					ret = klp_find_external_symbol(pmod,
> > > > symname, &addr);
> > > > +				else
> > > > +					ret =
> > > > klp_find_object_symbol(obj->name, symname, &addr);
> > > > +				if (ret)
> > > > +					return ret;
> > > > +				sym->st_value = addr;
> > > > +			}
> > > >  		}
> > > > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
> > > > +					 info->index.sym, relindex, pmod);
> > > >  	}
> > > > 
> > > > -	return 0;
> > > > +	return ret;
> > > >  }
> > > 
> > > Looking at this... do we even need reloc_secs in klp_object? Question is
> > > whether we need more than one dynrela section for an object. If not then
> > > the binding between klp_reloc_sec and an object is the only relevant thing
> > > in the structure, be it index or objname. So we can replace the
> > > list of structures with just the index in klp_object, or get rid of it
> > > completely and rely on the name of dynrela section be something like
> > > __klp_rela_{objname}.
> > 
> > Hm, you bring up a good point. I think theoretically yes, it is
> > possible to just have one klp_reloc_sec for each object and therefore
> > a list is not required (I have not checked yet how difficult it would
> > be to implement this on the kpatch-build side of things).  However,
> > considering the final format of the patch module, I think it is
> > semantically clearer to leave it as a list, and for each object to
> > possibly have more than one __klp_rela section.
> > 
> > For example, say we are patching two functions in ext4. In my
> > resulting kpatch module I will have two __klp_rela_ext4 sections, and
> > they might look like this when we run readelf --sections:
> > 
> > [34] __klp_rela_ext4.text.ext4_attr_store RELA ...
> > [35] __klp_rela_ext4.text.ext4_attr_show RELA ...
> > 
> > Then these two klp rela sections end up as two elements in the
> > reloc_secs list for the ext4 patch object. I think this way, we can
> > better tell which rela is being applied to what function. Might be
> > easier to understand what's happening from the developer's point of
> > view.
> > 
> > > You see, we go through elf sections here which were preserved by module
> > > loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
> > > maybe all the stuff around klp_reloc_sec is not necessary.
> > > 
> > > Thoughts?
> > 
> > Ah, so this is where descriptive comments and documentation might have
> > been useful :-) So I think we will still need to keep the
> > klp_reloc_sec struct to help the patch module initialize. Though the
> > name and objname fields aren't used in this patchset, they are used in
> > the kpatch patch module code [1], where we iterate through each elf
> > section, find the ones marked with SHF_RELA_LIVEPATCH, set the
> > klp_reloc_sec's objname (which we find from the "name" field,
> > formatted as __klp_rela_{objname}.text..). Once we have the objname
> > set, we can then find the object to attach the reloc_sec to (i.e. add
> > it to its list of reloc_secs).
> > 
> > Hope that clears some things up.
> 
> Ok, I'll try to explain myself and it is gonna be long. I'll try to 
> describe how we deal with dynrelas in klp today, how you use it in kpatch 
> (and this is the place where my knowledge can be wrong or obsolete), what 
> you propose and what I'd like to propose.
> 
> 1. First let's look on what we have now.
> 
> We have a patch module in which there is a section with all dynrelas 
> needed to be resolved (it was like this in kpatch some time ago and maybe 
> it is different now so just have a patience, I'll get to it). In the init 
> function of the module kpatch builds all the relevant info from dynrela 
> section. It goes through it, creates an array of klp_reloc for each object 
> and includes each dynrela record with an objname to the array of 
> klp_object with that objname. Later when we need to apply relocations for 
> patched object we just go through the list (array) of its dynrelas in 
> klp_object and call our arch-specific klp_write_module_reloc().
> 
> Now this was probably changed in kpatch and you do not have one dynrela 
> section but one dynrela section for each patched function. Is that 
> correct? (and can you tell us what the reason for the change was? It might 
> be crucial because I might be missing something.). Which leads us to your 
> proposal...
> 
> 2. So we have several dynrela section for one patched object. During init 
> function in a patch module kpatch once again builds needed structures from 
> these sections. Now they are called klp_reloc_sec and contain different 
> kind of info. There is no val, loc and such, only name of the symbol, 
> objname and index to dynrela section in ELF. So when you need to write 
> relocations for the patched object you go through all relevant dynrela 
> sections (because they are stored in the klp_object), all dynrela records 
> in each section and you resolve the undefined symbols. All needed info is 
> stored in ELF. Then you just call apply_relocate_add().
> 
> 3. I propose to go one step further. I think we don't need klp_reloc_sec 
> if there is only one dynrela section for patched object (and I currently 
> cannot see why this is not possible. It is possible even with one dynrela 
> section for whole patch module, that is for all patched objects.).

I think I agree that we don't need klp_reloc_sec, and that klp rela
sections can presumably be discovered by iterating over the sections.

But I don't think it matters much whether we have one klp rela section
per object, or multiple rela sections per object.  Either way, can't we
find them when we iterate over the sections?

For example, for one rela section per object, it could be named:

__klp_rela.objname

Or for multiple rela sections per object, they could be named:

__klp_rela.objname.func1
__klp_rela.objname.func2

Either way, when iterating over the sections, we could just look for
"__klp_rela.objname".

All that said, I think I would vote for one rela section per object,
just because it seems simpler.

> 
> In my proposal there would be nothing done in init function of the patched 
> module (albeit some optimization mentioned later). When you call 
> klp_write_object_relocations you would go through all ELF section and find 
> the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and 
> objname is in the name of the section. It is the same thing you do in 2. 
> in the init function.). Then you go through all dynrela records in the 
> section, you do the same things which you do in the proposed patch above, 
> and call apply_relocate_add.
> 
> Now it would be crazy to go through all sections each time 
> klp_write_object_relocations is called (maybe it does not even matter but 

Why would that be crazy?  To me it seems perfectly logical :-)  It
doesn't seem like a very expensive operation to me.

> nevertheless). So klp_object could have an index to its ELF dynrela 
> section. This index can be retrieved in the init function the same way you 
> do all the stuff with klp_reloc_sec.
> 
> If you insisted on more than one dynrela section for a patched object we 
> could have an array of indices there. Or whatever.
> 
> It is almost the same as your proposal but in my opinion somewhat nicer. 
> We just use the info stored in ELF directly without unnecessary middle 
> layer (== klp_reloc_sec).
> 
> Does it make sense? I hope it does. Would it work?

-- 
Josh

Re: livepatch: reuse module loader code to write relocations

[Miroslav Benes]

On Wed, 11 Nov 2015, Jessica Yu wrote:

> +++ Miroslav Benes [11/11/15 15:30 +0100]:
> > On Mon, 9 Nov 2015, Jessica Yu wrote:
> >
> > So I guess we don't need klp_reloc anymore.
> 
> Yes, that's correct. I am noticing just now that I forgot to remove
> the klp_reloc struct definition from livepatch.h. That change will be
> reflected in v2...
> 
> > If true, we should really
> > start thinking about proper documentation because there are going to be
> > plenty of assumptions about a patch module and we need to have it written
> > somewhere. Especially how the relocation sections look like.
> 
> Agreed. As a first step the patch module format can perhaps be
> documented somewhere. Perhaps it's time we create
> Documentation/livepatch/? :-)

Yes, I think so.

> > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
> > > index 087a8c7..26c419f 100644
> > > --- a/kernel/livepatch/core.c
> > > +++ b/kernel/livepatch/core.c
> > > @@ -28,6 +28,8 @@
> > >  #include <linux/list.h>
> > >  #include <linux/kallsyms.h>
> > >  #include <linux/livepatch.h>
> > > +#include <linux/elf.h>
> > > +#include <asm/cacheflush.h>
> > > 
> > >  /**
> > >   * struct klp_ops - structure for tracking registered ftrace ops structs
> > > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
> > > *pmod, const char *name,
> > >  }
> > > 
> > >  static int klp_write_object_relocations(struct module *pmod,
> > > -					struct klp_object *obj)
> > > +					struct klp_object *obj,
> > > +					struct klp_patch *patch)
> > >  {
> > > -	int ret;
> > > -	struct klp_reloc *reloc;
> > > +	int relindex, num_relas;
> > > +	int i, ret = 0;
> > > +	unsigned long addr;
> > > +	unsigned int bind;
> > > +	char *symname;
> > > +	struct klp_reloc_sec *reloc_sec;
> > > +	struct load_info *info;
> > > +	Elf_Rela *rela;
> > > +	Elf_Sym *sym, *symtab;
> > > +	Elf_Shdr *symsect;
> > > 
> > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
> > >  		return -EINVAL;
> > > 
> > > -	if (WARN_ON(!obj->relocs))
> > > -		return -EINVAL;
> > > -
> > > -	for (reloc = obj->relocs; reloc->name; reloc++) {
> > > -		if (!klp_is_module(obj)) {
> > > -			ret = klp_verify_vmlinux_symbol(reloc->name,
> > > -							reloc->val);
> > > -			if (ret)
> > > -				return ret;
> > > -		} else {
> > > -			/* module, reloc->val needs to be discovered */
> > > -			if (reloc->external)
> > > -				ret = klp_find_external_symbol(pmod,
> > > -							       reloc->name,
> > > -							       &reloc->val);
> > > -			else
> > > -				ret = klp_find_object_symbol(obj->mod->name,
> > > -							     reloc->name,
> > > -							     &reloc->val);
> > > -			if (ret)
> > > -				return ret;
> > > -		}
> > > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
> > > -					     reloc->val + reloc->addend);
> > > -		if (ret) {
> > > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
> > > (%d)\n",
> > > -			       reloc->name, reloc->val, ret);
> > > -			return ret;
> > > +	info = pmod->info;
> > > +	symsect = info->sechdrs + info->index.sym;
> > > +	symtab = (void *)info->hdr + symsect->sh_offset;
> > > +
> > > +	/* For each __klp_rela section for this object */
> > > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
> > > +		relindex = reloc_sec->index;
> > > +		num_relas = info->sechdrs[relindex].sh_size /
> > > sizeof(Elf_Rela);
> > > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
> > > +
> > > +		/* For each rela in this __klp_rela section */
> > > +		for (i = 0; i < num_relas; i++, rela++) {
> > > +			sym = symtab + ELF_R_SYM(rela->r_info);
> > > +			symname = info->strtab + sym->st_name;
> > > +			bind = ELF_ST_BIND(sym->st_info);
> > > +
> > > +			if (sym->st_shndx == SHN_LIVEPATCH) {
> > > +				if (bind == STB_LIVEPATCH_EXT)
> > > +					ret = klp_find_external_symbol(pmod,
> > > symname, &addr);
> > > +				else
> > > +					ret =
> > > klp_find_object_symbol(obj->name, symname, &addr);
> > > +				if (ret)
> > > +					return ret;
> > > +				sym->st_value = addr;
> > > +			}
> > >  		}
> > > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
> > > +					 info->index.sym, relindex, pmod);
> > >  	}
> > > 
> > > -	return 0;
> > > +	return ret;
> > >  }
> > 
> > Looking at this... do we even need reloc_secs in klp_object? Question is
> > whether we need more than one dynrela section for an object. If not then
> > the binding between klp_reloc_sec and an object is the only relevant thing
> > in the structure, be it index or objname. So we can replace the
> > list of structures with just the index in klp_object, or get rid of it
> > completely and rely on the name of dynrela section be something like
> > __klp_rela_{objname}.
> 
> Hm, you bring up a good point. I think theoretically yes, it is
> possible to just have one klp_reloc_sec for each object and therefore
> a list is not required (I have not checked yet how difficult it would
> be to implement this on the kpatch-build side of things).  However,
> considering the final format of the patch module, I think it is
> semantically clearer to leave it as a list, and for each object to
> possibly have more than one __klp_rela section.
> 
> For example, say we are patching two functions in ext4. In my
> resulting kpatch module I will have two __klp_rela_ext4 sections, and
> they might look like this when we run readelf --sections:
> 
> [34] __klp_rela_ext4.text.ext4_attr_store RELA ...
> [35] __klp_rela_ext4.text.ext4_attr_show RELA ...
> 
> Then these two klp rela sections end up as two elements in the
> reloc_secs list for the ext4 patch object. I think this way, we can
> better tell which rela is being applied to what function. Might be
> easier to understand what's happening from the developer's point of
> view.
> 
> > You see, we go through elf sections here which were preserved by module
> > loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
> > maybe all the stuff around klp_reloc_sec is not necessary.
> > 
> > Thoughts?
> 
> Ah, so this is where descriptive comments and documentation might have
> been useful :-) So I think we will still need to keep the
> klp_reloc_sec struct to help the patch module initialize. Though the
> name and objname fields aren't used in this patchset, they are used in
> the kpatch patch module code [1], where we iterate through each elf
> section, find the ones marked with SHF_RELA_LIVEPATCH, set the
> klp_reloc_sec's objname (which we find from the "name" field,
> formatted as __klp_rela_{objname}.text..). Once we have the objname
> set, we can then find the object to attach the reloc_sec to (i.e. add
> it to its list of reloc_secs).
> 
> Hope that clears some things up.

Ok, I'll try to explain myself and it is gonna be long. I'll try to 
describe how we deal with dynrelas in klp today, how you use it in kpatch 
(and this is the place where my knowledge can be wrong or obsolete), what 
you propose and what I'd like to propose.

1. First let's look on what we have now.

We have a patch module in which there is a section with all dynrelas 
needed to be resolved (it was like this in kpatch some time ago and maybe 
it is different now so just have a patience, I'll get to it). In the init 
function of the module kpatch builds all the relevant info from dynrela 
section. It goes through it, creates an array of klp_reloc for each object 
and includes each dynrela record with an objname to the array of 
klp_object with that objname. Later when we need to apply relocations for 
patched object we just go through the list (array) of its dynrelas in 
klp_object and call our arch-specific klp_write_module_reloc().

Now this was probably changed in kpatch and you do not have one dynrela 
section but one dynrela section for each patched function. Is that 
correct? (and can you tell us what the reason for the change was? It might 
be crucial because I might be missing something.). Which leads us to your 
proposal...

2. So we have several dynrela section for one patched object. During init 
function in a patch module kpatch once again builds needed structures from 
these sections. Now they are called klp_reloc_sec and contain different 
kind of info. There is no val, loc and such, only name of the symbol, 
objname and index to dynrela section in ELF. So when you need to write 
relocations for the patched object you go through all relevant dynrela 
sections (because they are stored in the klp_object), all dynrela records 
in each section and you resolve the undefined symbols. All needed info is 
stored in ELF. Then you just call apply_relocate_add().

3. I propose to go one step further. I think we don't need klp_reloc_sec 
if there is only one dynrela section for patched object (and I currently 
cannot see why this is not possible. It is possible even with one dynrela 
section for whole patch module, that is for all patched objects.).

In my proposal there would be nothing done in init function of the patched 
module (albeit some optimization mentioned later). When you call 
klp_write_object_relocations you would go through all ELF section and find 
the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and 
objname is in the name of the section. It is the same thing you do in 2. 
in the init function.). Then you go through all dynrela records in the 
section, you do the same things which you do in the proposed patch above, 
and call apply_relocate_add.

Now it would be crazy to go through all sections each time 
klp_write_object_relocations is called (maybe it does not even matter but 
nevertheless). So klp_object could have an index to its ELF dynrela 
section. This index can be retrieved in the init function the same way you 
do all the stuff with klp_reloc_sec.

If you insisted on more than one dynrela section for a patched object we 
could have an array of indices there. Or whatever.

It is almost the same as your proposal but in my opinion somewhat nicer. 
We just use the info stored in ELF directly without unnecessary middle 
layer (== klp_reloc_sec).

Does it make sense? I hope it does. Would it work?

Miroslav

Re: livepatch: reuse module loader code to write relocations

[Miroslav Benes]

On Wed, 11 Nov 2015, Jessica Yu wrote:

> +++ Miroslav Benes [11/11/15 15:30 +0100]:
> > On Mon, 9 Nov 2015, Jessica Yu wrote:
> >
> > So I guess we don't need klp_reloc anymore.
> 
> Yes, that's correct. I am noticing just now that I forgot to remove
> the klp_reloc struct definition from livepatch.h. That change will be
> reflected in v2...
> 
> > If true, we should really
> > start thinking about proper documentation because there are going to be
> > plenty of assumptions about a patch module and we need to have it written
> > somewhere. Especially how the relocation sections look like.
> 
> Agreed. As a first step the patch module format can perhaps be
> documented somewhere. Perhaps it's time we create
> Documentation/livepatch/? :-)

Yes, I think so.

> > > diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
> > > index 087a8c7..26c419f 100644
> > > --- a/kernel/livepatch/core.c
> > > +++ b/kernel/livepatch/core.c
> > > @@ -28,6 +28,8 @@
> > >  #include <linux/list.h>
> > >  #include <linux/kallsyms.h>
> > >  #include <linux/livepatch.h>
> > > +#include <linux/elf.h>
> > > +#include <asm/cacheflush.h>
> > > 
> > >  /**
> > >   * struct klp_ops - structure for tracking registered ftrace ops structs
> > > @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module
> > > *pmod, const char *name,
> > >  }
> > > 
> > >  static int klp_write_object_relocations(struct module *pmod,
> > > -					struct klp_object *obj)
> > > +					struct klp_object *obj,
> > > +					struct klp_patch *patch)
> > >  {
> > > -	int ret;
> > > -	struct klp_reloc *reloc;
> > > +	int relindex, num_relas;
> > > +	int i, ret = 0;
> > > +	unsigned long addr;
> > > +	unsigned int bind;
> > > +	char *symname;
> > > +	struct klp_reloc_sec *reloc_sec;
> > > +	struct load_info *info;
> > > +	Elf_Rela *rela;
> > > +	Elf_Sym *sym, *symtab;
> > > +	Elf_Shdr *symsect;
> > > 
> > >  	if (WARN_ON(!klp_is_object_loaded(obj)))
> > >  		return -EINVAL;
> > > 
> > > -	if (WARN_ON(!obj->relocs))
> > > -		return -EINVAL;
> > > -
> > > -	for (reloc = obj->relocs; reloc->name; reloc++) {
> > > -		if (!klp_is_module(obj)) {
> > > -			ret = klp_verify_vmlinux_symbol(reloc->name,
> > > -							reloc->val);
> > > -			if (ret)
> > > -				return ret;
> > > -		} else {
> > > -			/* module, reloc->val needs to be discovered */
> > > -			if (reloc->external)
> > > -				ret = klp_find_external_symbol(pmod,
> > > -							       reloc->name,
> > > -							       &reloc->val);
> > > -			else
> > > -				ret = klp_find_object_symbol(obj->mod->name,
> > > -							     reloc->name,
> > > -							     &reloc->val);
> > > -			if (ret)
> > > -				return ret;
> > > -		}
> > > -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
> > > -					     reloc->val + reloc->addend);
> > > -		if (ret) {
> > > -			pr_err("relocation failed for symbol '%s' at 0x%016lx
> > > (%d)\n",
> > > -			       reloc->name, reloc->val, ret);
> > > -			return ret;
> > > +	info = pmod->info;
> > > +	symsect = info->sechdrs + info->index.sym;
> > > +	symtab = (void *)info->hdr + symsect->sh_offset;
> > > +
> > > +	/* For each __klp_rela section for this object */
> > > +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
> > > +		relindex = reloc_sec->index;
> > > +		num_relas = info->sechdrs[relindex].sh_size /
> > > sizeof(Elf_Rela);
> > > +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
> > > +
> > > +		/* For each rela in this __klp_rela section */
> > > +		for (i = 0; i < num_relas; i++, rela++) {
> > > +			sym = symtab + ELF_R_SYM(rela->r_info);
> > > +			symname = info->strtab + sym->st_name;
> > > +			bind = ELF_ST_BIND(sym->st_info);
> > > +
> > > +			if (sym->st_shndx == SHN_LIVEPATCH) {
> > > +				if (bind == STB_LIVEPATCH_EXT)
> > > +					ret = klp_find_external_symbol(pmod,
> > > symname, &addr);
> > > +				else
> > > +					ret =
> > > klp_find_object_symbol(obj->name, symname, &addr);
> > > +				if (ret)
> > > +					return ret;
> > > +				sym->st_value = addr;
> > > +			}
> > >  		}
> > > +		ret = apply_relocate_add(info->sechdrs, info->strtab,
> > > +					 info->index.sym, relindex, pmod);
> > >  	}
> > > 
> > > -	return 0;
> > > +	return ret;
> > >  }
> > 
> > Looking at this... do we even need reloc_secs in klp_object? Question is
> > whether we need more than one dynrela section for an object. If not then
> > the binding between klp_reloc_sec and an object is the only relevant thing
> > in the structure, be it index or objname. So we can replace the
> > list of structures with just the index in klp_object, or get rid of it
> > completely and rely on the name of dynrela section be something like
> > __klp_rela_{objname}.
> 
> Hm, you bring up a good point. I think theoretically yes, it is
> possible to just have one klp_reloc_sec for each object and therefore
> a list is not required (I have not checked yet how difficult it would
> be to implement this on the kpatch-build side of things).  However,
> considering the final format of the patch module, I think it is
> semantically clearer to leave it as a list, and for each object to
> possibly have more than one __klp_rela section.
> 
> For example, say we are patching two functions in ext4. In my
> resulting kpatch module I will have two __klp_rela_ext4 sections, and
> they might look like this when we run readelf --sections:
> 
> [34] __klp_rela_ext4.text.ext4_attr_store RELA ...
> [35] __klp_rela_ext4.text.ext4_attr_show RELA ...
> 
> Then these two klp rela sections end up as two elements in the
> reloc_secs list for the ext4 patch object. I think this way, we can
> better tell which rela is being applied to what function. Might be
> easier to understand what's happening from the developer's point of
> view.
> 
> > You see, we go through elf sections here which were preserved by module
> > loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
> > maybe all the stuff around klp_reloc_sec is not necessary.
> > 
> > Thoughts?
> 
> Ah, so this is where descriptive comments and documentation might have
> been useful :-) So I think we will still need to keep the
> klp_reloc_sec struct to help the patch module initialize. Though the
> name and objname fields aren't used in this patchset, they are used in
> the kpatch patch module code [1], where we iterate through each elf
> section, find the ones marked with SHF_RELA_LIVEPATCH, set the
> klp_reloc_sec's objname (which we find from the "name" field,
> formatted as __klp_rela_{objname}.text..). Once we have the objname
> set, we can then find the object to attach the reloc_sec to (i.e. add
> it to its list of reloc_secs).
> 
> Hope that clears some things up.

Ok, I'll try to explain myself and it is gonna be long. I'll try to 
describe how we deal with dynrelas in klp today, how you use it in kpatch 
(and this is the place where my knowledge can be wrong or obsolete), what 
you propose and what I'd like to propose.

1. First let's look on what we have now.

We have a patch module in which there is a section with all dynrelas 
needed to be resolved (it was like this in kpatch some time ago and maybe 
it is different now so just have a patience, I'll get to it). In the init 
function of the module kpatch builds all the relevant info from dynrela 
section. It goes through it, creates an array of klp_reloc for each object 
and includes each dynrela record with an objname to the array of 
klp_object with that objname. Later when we need to apply relocations for 
patched object we just go through the list (array) of its dynrelas in 
klp_object and call our arch-specific klp_write_module_reloc().

Now this was probably changed in kpatch and you do not have one dynrela 
section but one dynrela section for each patched function. Is that 
correct? (and can you tell us what the reason for the change was? It might 
be crucial because I might be missing something.). Which leads us to your 
proposal...

2. So we have several dynrela section for one patched object. During init 
function in a patch module kpatch once again builds needed structures from 
these sections. Now they are called klp_reloc_sec and contain different 
kind of info. There is no val, loc and such, only name of the symbol, 
objname and index to dynrela section in ELF. So when you need to write 
relocations for the patched object you go through all relevant dynrela 
sections (because they are stored in the klp_object), all dynrela records 
in each section and you resolve the undefined symbols. All needed info is 
stored in ELF. Then you just call apply_relocate_add().

3. I propose to go one step further. I think we don't need klp_reloc_sec 
if there is only one dynrela section for patched object (and I currently 
cannot see why this is not possible. It is possible even with one dynrela 
section for whole patch module, that is for all patched objects.).

In my proposal there would be nothing done in init function of the patched 
module (albeit some optimization mentioned later). When you call 
klp_write_object_relocations you would go through all ELF section and find 
the relevant one for the object (it is marked with SHF_RELA_LIVEPATCH and 
objname is in the name of the section. It is the same thing you do in 2. 
in the init function.). Then you go through all dynrela records in the 
section, you do the same things which you do in the proposed patch above, 
and call apply_relocate_add.

Now it would be crazy to go through all sections each time 
klp_write_object_relocations is called (maybe it does not even matter but 
nevertheless). So klp_object could have an index to its ELF dynrela 
section. This index can be retrieved in the init function the same way you 
do all the stuff with klp_reloc_sec.

If you insisted on more than one dynrela section for a patched object we 
could have an array of indices there. Or whatever.

It is almost the same as your proposal but in my opinion somewhat nicer. 
We just use the info stored in ELF directly without unnecessary middle 
layer (== klp_reloc_sec).

Does it make sense? I hope it does. Would it work?

Miroslav

Re: livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Wed 2015-11-11 13:27:18, Jessica Yu wrote:
> Basically, the patch module code is expected to build the reloc_secs
> list for each object that is being patched. For example in kpatch, the
> patch module generates this list in patch_init(). Like you guessed, it
> does go through all the elf sections of the patch module to find the
> reloc sections marked SHF_RELA_LIVEPATCH. We are able to access these
> sections through module->info, which is set up for livepatch modules
> before the module loader calls do_init_module() (and hence before
> patch_init(), See patch 2/5).
> See below for an example of how the reloc_secs list might be built: https://github.com/flaming-toast/kpatch/blob/no_dynrela_redux/kmod/patch/livepatch-patch-hook.c#L213

Thanks a lot for the link and explanation. I think that it would be nice to
incorporate this into the patchset. I would put it into a separate
function, e.g. klp_reloc_secs_init(). It can be called either from
klp_register_patch() or from klp_init_patch().

> Once the patch module has built this list, it is good to go for use in
> livepatch core. All livepatch has to do then is to iterate though the
> list, resolve any outstanding symbols, and call apply_relocate_add(),
> as shown in this patch.
>
> Miroslav mentioned in another email that we should start thinking
> about including documentation about this, including the expected patch
> module format. So perhaps v2 should include some documentation about
> this whole process somewhere.

That would be cool.

Thank you,
Petr

Re: livepatch: reuse module loader code to write relocations

[Petr Mladek]

On Wed 2015-11-11 13:27:18, Jessica Yu wrote:
> Basically, the patch module code is expected to build the reloc_secs
> list for each object that is being patched. For example in kpatch, the
> patch module generates this list in patch_init(). Like you guessed, it
> does go through all the elf sections of the patch module to find the
> reloc sections marked SHF_RELA_LIVEPATCH. We are able to access these
> sections through module->info, which is set up for livepatch modules
> before the module loader calls do_init_module() (and hence before
> patch_init(), See patch 2/5).
> See below for an example of how the reloc_secs list might be built: https://github.com/flaming-toast/kpatch/blob/no_dynrela_redux/kmod/patch/livepatch-patch-hook.c#L213

Thanks a lot for the link and explanation. I think that it would be nice to
incorporate this into the patchset. I would put it into a separate
function, e.g. klp_reloc_secs_init(). It can be called either from
klp_register_patch() or from klp_init_patch().

> Once the patch module has built this list, it is good to go for use in
> livepatch core. All livepatch has to do then is to iterate though the
> list, resolve any outstanding symbols, and call apply_relocate_add(),
> as shown in this patch.
>
> Miroslav mentioned in another email that we should start thinking
> about including documentation about this, including the expected patch
> module format. So perhaps v2 should include some documentation about
> this whole process somewhere.

That would be cool.

Thank you,
Petr

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [11/11/15 15:30 +0100]:
>On Mon, 9 Nov 2015, Jessica Yu wrote:
>
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index 31db7a0..601e892 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -85,7 +85,7 @@ struct klp_reloc {
>>  /**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>> + * @reloc_secs:	relocation sections to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +95,7 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>> +	struct list_head reloc_secs;
>>  	struct klp_func *funcs;
>
>So I guess we don't need klp_reloc anymore.

Yes, that's correct. I am noticing just now that I forgot to remove
the klp_reloc struct definition from livepatch.h. That change will be
reflected in v2...

>If true, we should really
>start thinking about proper documentation because there are going to be
>plenty of assumptions about a patch module and we need to have it written
>somewhere. Especially how the relocation sections look like.

Agreed. As a first step the patch module format can perhaps be
documented somewhere. Perhaps it's time we create
Documentation/livepatch/? :-)

>>  	/* internal */
>> @@ -129,6 +129,13 @@ struct klp_patch {
>>  #define klp_for_each_func(obj, func) \
>>  	for (func = obj->funcs; func->old_name; func++)
>>
>> +struct klp_reloc_sec {
>> +	unsigned int index;
>> +	char *name;
>> +	char *objname;
>> +	struct list_head list;
>> +};
>
>Description of the structure and its members is missing.
>
>> diff --git a/include/linux/module.h b/include/linux/module.h
>> index c8680b1..3c34eb8 100644
>> --- a/include/linux/module.h
>> +++ b/include/linux/module.h
>> @@ -793,9 +793,15 @@ extern int module_sysfs_initialized;
>>  #ifdef CONFIG_DEBUG_SET_MODULE_RONX
>>  extern void set_all_modules_text_rw(void);
>>  extern void set_all_modules_text_ro(void);
>> +extern void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages));
>>  #else
>>  static inline void set_all_modules_text_rw(void) { }
>>  static inline void set_all_modules_text_ro(void) { }
>> +static inline void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages)) { }
>>  #endif
>
>This would be solved after Rusty's and Josh's patches get merged, right?

Yes, correct. When Josh and Rusty's patches get merged, I'll base
subsequent versions of this patchset on them.

>> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> index 087a8c7..26c419f 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,8 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <asm/cacheflush.h>
>>
>>  /**
>>   * struct klp_ops - structure for tracking registered ftrace ops structs
>> @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
>>  }
>>
>>  static int klp_write_object_relocations(struct module *pmod,
>> -					struct klp_object *obj)
>> +					struct klp_object *obj,
>> +					struct klp_patch *patch)
>>  {
>> -	int ret;
>> -	struct klp_reloc *reloc;
>> +	int relindex, num_relas;
>> +	int i, ret = 0;
>> +	unsigned long addr;
>> +	unsigned int bind;
>> +	char *symname;
>> +	struct klp_reloc_sec *reloc_sec;
>> +	struct load_info *info;
>> +	Elf_Rela *rela;
>> +	Elf_Sym *sym, *symtab;
>> +	Elf_Shdr *symsect;
>>
>>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>  		return -EINVAL;
>>
>> -	if (WARN_ON(!obj->relocs))
>> -		return -EINVAL;
>> -
>> -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> -		if (!klp_is_module(obj)) {
>> -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> -							reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		} else {
>> -			/* module, reloc->val needs to be discovered */
>> -			if (reloc->external)
>> -				ret = klp_find_external_symbol(pmod,
>> -							       reloc->name,
>> -							       &reloc->val);
>> -			else
>> -				ret = klp_find_object_symbol(obj->mod->name,
>> -							     reloc->name,
>> -							     &reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		}
>> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> -					     reloc->val + reloc->addend);
>> -		if (ret) {
>> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
>> -			       reloc->name, reloc->val, ret);
>> -			return ret;
>> +	info = pmod->info;
>> +	symsect = info->sechdrs + info->index.sym;
>> +	symtab = (void *)info->hdr + symsect->sh_offset;
>> +
>> +	/* For each __klp_rela section for this object */
>> +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>> +		relindex = reloc_sec->index;
>> +		num_relas = info->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>> +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>> +
>> +		/* For each rela in this __klp_rela section */
>> +		for (i = 0; i < num_relas; i++, rela++) {
>> +			sym = symtab + ELF_R_SYM(rela->r_info);
>> +			symname = info->strtab + sym->st_name;
>> +			bind = ELF_ST_BIND(sym->st_info);
>> +
>> +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> +				if (bind == STB_LIVEPATCH_EXT)
>> +					ret = klp_find_external_symbol(pmod, symname, &addr);
>> +				else
>> +					ret = klp_find_object_symbol(obj->name, symname, &addr);
>> +				if (ret)
>> +					return ret;
>> +				sym->st_value = addr;
>> +			}
>>  		}
>> +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>> +					 info->index.sym, relindex, pmod);
>>  	}
>>
>> -	return 0;
>> +	return ret;
>>  }
>
>Looking at this... do we even need reloc_secs in klp_object? Question is
>whether we need more than one dynrela section for an object. If not then
>the binding between klp_reloc_sec and an object is the only relevant thing
>in the structure, be it index or objname. So we can replace the
>list of structures with just the index in klp_object, or get rid of it
>completely and rely on the name of dynrela section be something like
>__klp_rela_{objname}.

Hm, you bring up a good point. I think theoretically yes, it is
possible to just have one klp_reloc_sec for each object and therefore
a list is not required (I have not checked yet how difficult it would
be to implement this on the kpatch-build side of things).  However,
considering the final format of the patch module, I think it is
semantically clearer to leave it as a list, and for each object to
possibly have more than one __klp_rela section.

For example, say we are patching two functions in ext4. In my
resulting kpatch module I will have two __klp_rela_ext4 sections, and
they might look like this when we run readelf --sections:

[34] __klp_rela_ext4.text.ext4_attr_store RELA ...
[35] __klp_rela_ext4.text.ext4_attr_show RELA ...

Then these two klp rela sections end up as two elements in the
reloc_secs list for the ext4 patch object. I think this way, we can
better tell which rela is being applied to what function. Might be
easier to understand what's happening from the developer's point of
view.

>You see, we go through elf sections here which were preserved by module
>loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>maybe all the stuff around klp_reloc_sec is not necessary.
>
>Thoughts?

Ah, so this is where descriptive comments and documentation might have
been useful :-) So I think we will still need to keep the
klp_reloc_sec struct to help the patch module initialize. Though the
name and objname fields aren't used in this patchset, they are used in
the kpatch patch module code [1], where we iterate through each elf
section, find the ones marked with SHF_RELA_LIVEPATCH, set the
klp_reloc_sec's objname (which we find from the "name" field,
formatted as __klp_rela_{objname}.text..). Once we have the objname
set, we can then find the object to attach the reloc_sec to (i.e. add
it to its list of reloc_secs).

Hope that clears some things up.


[1] https://github.com/flaming-toast/kpatch/blob/no_dynrela_redux/kmod/patch/livepatch-patch-hook.c#L213

>> @@ -741,12 +751,23 @@ static int klp_init_object_loaded(struct klp_patch *patch,
>>  				  struct klp_object *obj)
>>  {
>>  	struct klp_func *func;
>> +	struct module *pmod;
>>  	int ret;
>>
>> -	if (obj->relocs) {
>> -		ret = klp_write_object_relocations(patch->mod, obj);
>> +	pmod = patch->mod;
>> +
>> +	if (!list_empty(&obj->reloc_secs)) {
>> +		set_page_attributes(pmod->module_core,
>> +				    pmod->module_core + pmod->core_text_size,
>> +				    set_memory_rw);
>> +
>> +		ret = klp_write_object_relocations(pmod, obj, patch);
>>  		if (ret)
>>  			return ret;
>> +
>> +		set_page_attributes(pmod->module_core,
>> +				    pmod->module_core + pmod->core_text_size,
>> +				    set_memory_ro);
>>  	}
>
>And this would get solved with different patches as well. I think the
>calls to set_page_attributes() should be hidden in
>klp_write_object_relocations() as it is in Josh's patch IIRC.

Yes, when those patches are merged I will switch over to Josh's
functions for setting/unsetting memory ro.

Thanks,
Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Miroslav Benes [11/11/15 15:30 +0100]:
>On Mon, 9 Nov 2015, Jessica Yu wrote:
>
>> diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
>> index 31db7a0..601e892 100644
>> --- a/include/linux/livepatch.h
>> +++ b/include/linux/livepatch.h
>> @@ -85,7 +85,7 @@ struct klp_reloc {
>>  /**
>>   * struct klp_object - kernel object structure for live patching
>>   * @name:	module name (or NULL for vmlinux)
>> - * @relocs:	relocation entries to be applied at load time
>> + * @reloc_secs:	relocation sections to be applied at load time
>>   * @funcs:	function entries for functions to be patched in the object
>>   * @kobj:	kobject for sysfs resources
>>   * @mod:	kernel module associated with the patched object
>> @@ -95,7 +95,7 @@ struct klp_reloc {
>>  struct klp_object {
>>  	/* external */
>>  	const char *name;
>> -	struct klp_reloc *relocs;
>> +	struct list_head reloc_secs;
>>  	struct klp_func *funcs;
>
>So I guess we don't need klp_reloc anymore.

Yes, that's correct. I am noticing just now that I forgot to remove
the klp_reloc struct definition from livepatch.h. That change will be
reflected in v2...

>If true, we should really
>start thinking about proper documentation because there are going to be
>plenty of assumptions about a patch module and we need to have it written
>somewhere. Especially how the relocation sections look like.

Agreed. As a first step the patch module format can perhaps be
documented somewhere. Perhaps it's time we create
Documentation/livepatch/? :-)

>>  	/* internal */
>> @@ -129,6 +129,13 @@ struct klp_patch {
>>  #define klp_for_each_func(obj, func) \
>>  	for (func = obj->funcs; func->old_name; func++)
>>
>> +struct klp_reloc_sec {
>> +	unsigned int index;
>> +	char *name;
>> +	char *objname;
>> +	struct list_head list;
>> +};
>
>Description of the structure and its members is missing.
>
>> diff --git a/include/linux/module.h b/include/linux/module.h
>> index c8680b1..3c34eb8 100644
>> --- a/include/linux/module.h
>> +++ b/include/linux/module.h
>> @@ -793,9 +793,15 @@ extern int module_sysfs_initialized;
>>  #ifdef CONFIG_DEBUG_SET_MODULE_RONX
>>  extern void set_all_modules_text_rw(void);
>>  extern void set_all_modules_text_ro(void);
>> +extern void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages));
>>  #else
>>  static inline void set_all_modules_text_rw(void) { }
>>  static inline void set_all_modules_text_ro(void) { }
>> +static inline void
>> +set_page_attributes(void *start, void *end,
>> +		    int (*set)(unsigned long start, int num_pages)) { }
>>  #endif
>
>This would be solved after Rusty's and Josh's patches get merged, right?

Yes, correct. When Josh and Rusty's patches get merged, I'll base
subsequent versions of this patchset on them.

>> diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>> index 087a8c7..26c419f 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,8 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <asm/cacheflush.h>
>>
>>  /**
>>   * struct klp_ops - structure for tracking registered ftrace ops structs
>> @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
>>  }
>>
>>  static int klp_write_object_relocations(struct module *pmod,
>> -					struct klp_object *obj)
>> +					struct klp_object *obj,
>> +					struct klp_patch *patch)
>>  {
>> -	int ret;
>> -	struct klp_reloc *reloc;
>> +	int relindex, num_relas;
>> +	int i, ret = 0;
>> +	unsigned long addr;
>> +	unsigned int bind;
>> +	char *symname;
>> +	struct klp_reloc_sec *reloc_sec;
>> +	struct load_info *info;
>> +	Elf_Rela *rela;
>> +	Elf_Sym *sym, *symtab;
>> +	Elf_Shdr *symsect;
>>
>>  	if (WARN_ON(!klp_is_object_loaded(obj)))
>>  		return -EINVAL;
>>
>> -	if (WARN_ON(!obj->relocs))
>> -		return -EINVAL;
>> -
>> -	for (reloc = obj->relocs; reloc->name; reloc++) {
>> -		if (!klp_is_module(obj)) {
>> -			ret = klp_verify_vmlinux_symbol(reloc->name,
>> -							reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		} else {
>> -			/* module, reloc->val needs to be discovered */
>> -			if (reloc->external)
>> -				ret = klp_find_external_symbol(pmod,
>> -							       reloc->name,
>> -							       &reloc->val);
>> -			else
>> -				ret = klp_find_object_symbol(obj->mod->name,
>> -							     reloc->name,
>> -							     &reloc->val);
>> -			if (ret)
>> -				return ret;
>> -		}
>> -		ret = klp_write_module_reloc(pmod, reloc->type, reloc->loc,
>> -					     reloc->val + reloc->addend);
>> -		if (ret) {
>> -			pr_err("relocation failed for symbol '%s' at 0x%016lx (%d)\n",
>> -			       reloc->name, reloc->val, ret);
>> -			return ret;
>> +	info = pmod->info;
>> +	symsect = info->sechdrs + info->index.sym;
>> +	symtab = (void *)info->hdr + symsect->sh_offset;
>> +
>> +	/* For each __klp_rela section for this object */
>> +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>> +		relindex = reloc_sec->index;
>> +		num_relas = info->sechdrs[relindex].sh_size / sizeof(Elf_Rela);
>> +		rela = (Elf_Rela *) info->sechdrs[relindex].sh_addr;
>> +
>> +		/* For each rela in this __klp_rela section */
>> +		for (i = 0; i < num_relas; i++, rela++) {
>> +			sym = symtab + ELF_R_SYM(rela->r_info);
>> +			symname = info->strtab + sym->st_name;
>> +			bind = ELF_ST_BIND(sym->st_info);
>> +
>> +			if (sym->st_shndx == SHN_LIVEPATCH) {
>> +				if (bind == STB_LIVEPATCH_EXT)
>> +					ret = klp_find_external_symbol(pmod, symname, &addr);
>> +				else
>> +					ret = klp_find_object_symbol(obj->name, symname, &addr);
>> +				if (ret)
>> +					return ret;
>> +				sym->st_value = addr;
>> +			}
>>  		}
>> +		ret = apply_relocate_add(info->sechdrs, info->strtab,
>> +					 info->index.sym, relindex, pmod);
>>  	}
>>
>> -	return 0;
>> +	return ret;
>>  }
>
>Looking at this... do we even need reloc_secs in klp_object? Question is
>whether we need more than one dynrela section for an object. If not then
>the binding between klp_reloc_sec and an object is the only relevant thing
>in the structure, be it index or objname. So we can replace the
>list of structures with just the index in klp_object, or get rid of it
>completely and rely on the name of dynrela section be something like
>__klp_rela_{objname}.

Hm, you bring up a good point. I think theoretically yes, it is
possible to just have one klp_reloc_sec for each object and therefore
a list is not required (I have not checked yet how difficult it would
be to implement this on the kpatch-build side of things).  However,
considering the final format of the patch module, I think it is
semantically clearer to leave it as a list, and for each object to
possibly have more than one __klp_rela section.

For example, say we are patching two functions in ext4. In my
resulting kpatch module I will have two __klp_rela_ext4 sections, and
they might look like this when we run readelf --sections:

[34] __klp_rela_ext4.text.ext4_attr_store RELA ...
[35] __klp_rela_ext4.text.ext4_attr_show RELA ...

Then these two klp rela sections end up as two elements in the
reloc_secs list for the ext4 patch object. I think this way, we can
better tell which rela is being applied to what function. Might be
easier to understand what's happening from the developer's point of
view.

>You see, we go through elf sections here which were preserved by module
>loader. We even have relevant sections marked with SHF_RELA_LIVEPATCH. So
>maybe all the stuff around klp_reloc_sec is not necessary.
>
>Thoughts?

Ah, so this is where descriptive comments and documentation might have
been useful :-) So I think we will still need to keep the
klp_reloc_sec struct to help the patch module initialize. Though the
name and objname fields aren't used in this patchset, they are used in
the kpatch patch module code [1], where we iterate through each elf
section, find the ones marked with SHF_RELA_LIVEPATCH, set the
klp_reloc_sec's objname (which we find from the "name" field,
formatted as __klp_rela_{objname}.text..). Once we have the objname
set, we can then find the object to attach the reloc_sec to (i.e. add
it to its list of reloc_secs).

Hope that clears some things up.


[1] https://github.com/flaming-toast/kpatch/blob/no_dynrela_redux/kmod/patch/livepatch-patch-hook.c#L213

>> @@ -741,12 +751,23 @@ static int klp_init_object_loaded(struct klp_patch *patch,
>>  				  struct klp_object *obj)
>>  {
>>  	struct klp_func *func;
>> +	struct module *pmod;
>>  	int ret;
>>
>> -	if (obj->relocs) {
>> -		ret = klp_write_object_relocations(patch->mod, obj);
>> +	pmod = patch->mod;
>> +
>> +	if (!list_empty(&obj->reloc_secs)) {
>> +		set_page_attributes(pmod->module_core,
>> +				    pmod->module_core + pmod->core_text_size,
>> +				    set_memory_rw);
>> +
>> +		ret = klp_write_object_relocations(pmod, obj, patch);
>>  		if (ret)
>>  			return ret;
>> +
>> +		set_page_attributes(pmod->module_core,
>> +				    pmod->module_core + pmod->core_text_size,
>> +				    set_memory_ro);
>>  	}
>
>And this would get solved with different patches as well. I think the
>calls to set_page_attributes() should be hidden in
>klp_write_object_relocations() as it is in Josh's patch IIRC.

Yes, when those patches are merged I will switch over to Josh's
functions for setting/unsetting memory ro.

Thanks,
Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Petr Mladek [11/11/15 16:22 +0100]:
>On Mon 2015-11-09 23:45:53, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the
>> need for architecture specific code in livepatch. Namely, we reuse
>> apply_relocate_add() in the module loader to write relocs instead of
>> duplicating functionality in livepatch's klp_write_module_reloc(). To
>> apply relocation sections, remaining SHN_LIVEPATCH symbols referenced by
>> relocs are resolved and then apply_relocate_add() is called to apply
>> those relocations.
>>
>> Signed-off-by: Jessica Yu <jeyu@redhat.com>
>> ---
>>  include/linux/livepatch.h | 11 ++++--
>>  include/linux/module.h    |  6 ++++
>>  kernel/livepatch/core.c   | 89 +++++++++++++++++++++++++++++------------------
>>  3 files changed, 70 insertions(+), 36 deletions(-)
>>
>> index 087a8c7..26c419f 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,8 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <asm/cacheflush.h>
>>
>>  /**
>>   * struct klp_ops - structure for tracking registered ftrace ops structs
>> @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
>>  }
>>
>>  static int klp_write_object_relocations(struct module *pmod,
>> -					struct klp_object *obj)
>> +					struct klp_object *obj,
>> +					struct klp_patch *patch)
>>  {
>[...]
>
>> +
>> +	/* For each __klp_rela section for this object */
>> +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>
>Who, when, and how will define reloc_secs, please?
>
>I guess that it is just an optimization. It helps to avoid going
>through all elf sections of all livepatch modules. Therefore, I think
>that we might fill this when the livepatch module is loaded. But
>I do not see the code for this.

Thanks for bringing this up, I admit that from this patchset it is
unclear where and how the reloc_secs list is built. 

Basically, the patch module code is expected to build the reloc_secs
list for each object that is being patched. For example in kpatch, the
patch module generates this list in patch_init(). Like you guessed, it
does go through all the elf sections of the patch module to find the
reloc sections marked SHF_RELA_LIVEPATCH. We are able to access these
sections through module->info, which is set up for livepatch modules
before the module loader calls do_init_module() (and hence before
patch_init(), See patch 2/5).
See below for an example of how the reloc_secs list might be built: 
https://github.com/flaming-toast/kpatch/blob/no_dynrela_redux/kmod/patch/livepatch-patch-hook.c#L213

Once the patch module has built this list, it is good to go for use in
livepatch core. All livepatch has to do then is to iterate though the
list, resolve any outstanding symbols, and call apply_relocate_add(),
as shown in this patch.

Miroslav mentioned in another email that we should start thinking
about including documentation about this, including the expected patch
module format. So perhaps v2 should include some documentation about
this whole process somewhere.

Please let me know if anything else is unclear.

Thanks,

Jessica

Re: livepatch: reuse module loader code to write relocations

[Jessica Yu]

+++ Petr Mladek [11/11/15 16:22 +0100]:
>On Mon 2015-11-09 23:45:53, Jessica Yu wrote:
>> Reuse module loader code to write relocations, thereby eliminating the
>> need for architecture specific code in livepatch. Namely, we reuse
>> apply_relocate_add() in the module loader to write relocs instead of
>> duplicating functionality in livepatch's klp_write_module_reloc(). To
>> apply relocation sections, remaining SHN_LIVEPATCH symbols referenced by
>> relocs are resolved and then apply_relocate_add() is called to apply
>> those relocations.
>>
>> Signed-off-by: Jessica Yu <jeyu-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> ---
>>  include/linux/livepatch.h | 11 ++++--
>>  include/linux/module.h    |  6 ++++
>>  kernel/livepatch/core.c   | 89 +++++++++++++++++++++++++++++------------------
>>  3 files changed, 70 insertions(+), 36 deletions(-)
>>
>> index 087a8c7..26c419f 100644
>> --- a/kernel/livepatch/core.c
>> +++ b/kernel/livepatch/core.c
>> @@ -28,6 +28,8 @@
>>  #include <linux/list.h>
>>  #include <linux/kallsyms.h>
>>  #include <linux/livepatch.h>
>> +#include <linux/elf.h>
>> +#include <asm/cacheflush.h>
>>
>>  /**
>>   * struct klp_ops - structure for tracking registered ftrace ops structs
>> @@ -281,46 +283,54 @@ static int klp_find_external_symbol(struct module *pmod, const char *name,
>>  }
>>
>>  static int klp_write_object_relocations(struct module *pmod,
>> -					struct klp_object *obj)
>> +					struct klp_object *obj,
>> +					struct klp_patch *patch)
>>  {
>[...]
>
>> +
>> +	/* For each __klp_rela section for this object */
>> +	list_for_each_entry(reloc_sec, &obj->reloc_secs, list) {
>
>Who, when, and how will define reloc_secs, please?
>
>I guess that it is just an optimization. It helps to avoid going
>through all elf sections of all livepatch modules. Therefore, I think
>that we might fill this when the livepatch module is loaded. But
>I do not see the code for this.

Thanks for bringing this up, I admit that from this patchset it is
unclear where and how the reloc_secs list is built. 

Basically, the patch module code is expected to build the reloc_secs
list for each object that is being patched. For example in kpatch, the
patch module generates this list in patch_init(). Like you guessed, it
does go through all the elf sections of the patch module to find the
reloc sections marked SHF_RELA_LIVEPATCH. We are able to access these
sections through module->info, which is set up for livepatch modules
before the module loader calls do_init_module() (and hence before
patch_init(), See patch 2/5).
See below for an example of how the reloc_secs list might be built: 
https://github.com/flaming-toast/kpatch/blob/no_dynrela_redux/kmod/patch/livepatch-patch-hook.c#L213

Once the patch module has built this list, it is good to go for use in
livepatch core. All livepatch has to do then is to iterate though the
list, resolve any outstanding symbols, and call apply_relocate_add(),
as shown in this patch.

Miroslav mentioned in another email that we should start thinking
about including documentation about this, including the expected patch
module format. So perhaps v2 should include some documentation about
this whole process somewhere.

Please let me know if anything else is unclear.

Thanks,

Jessica