* ARM: relocation out of range (when loading a module)
@ 2011-01-10 18:16 Alexander Holler
2011-01-10 22:26 ` Nicolas Pitre
0 siblings, 1 reply; 33+ messages in thread
From: Alexander Holler @ 2011-01-10 18:16 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, linux
Hello,
I've build a kernel 2.6.37 for an armv5-device which includes an
initramfs of about 16MB (uncompressed, result is a kernel of about 6MB)
and I now getting errors when trying to load a module (which is included
in that initramfs):
----------------------------
# modprobe ipv6
modprobe: 'kernel/net/ipv6/ipv6.ko': invalid module format
# dmesg | tail -n 1
[ 107.327672] ipv6: relocation out of range, section 2 reloc 0 sym
'snmp_mib_free'
----------------------------
To compile the kernel I've used gcc 4.5.2 and binutils 2.21.
If I apply the patch found at
http://lists.arm.linux.org.uk/lurker/message/20100729.090437.85f82ebd.en.html
the output from dmesg will be
----------------------------
[ 58.211776] ipv6: section 2 reloc 0 sym 'snmp_mib_free': relocation
28 out of range (0xbf0000a4 -> 0xc11b4858)
----------------------------
I don't know much about how the memory is organized and the relocation
is done for arm, but reading that output it seems the 32MB limit is
reached. Is that because of the initramfs (just speculating) and is
there a workaround to still include such an initramfs in the kernel?
An excerpt from my config:
----------------------------
CONFIG_ARM_THUMB=y
...
# CONFIG_HIGHMEM is not set
# CONFIG_ARCH_SPARSEMEM_DEFAULT is not set
# CONFIG_ARCH_SELECT_MEMORY_MODEL is not set
# CONFIG_HIGHMEM is not set
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_HAVE_MEMBLOCK=y
CONFIG_PAGEFLAGS_EXTENDED=y
----------------------------
Regards,
Alexander
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-10 18:16 ARM: relocation out of range (when loading a module) Alexander Holler
@ 2011-01-10 22:26 ` Nicolas Pitre
2011-01-11 6:34 ` Alexander Holler
0 siblings, 1 reply; 33+ messages in thread
From: Nicolas Pitre @ 2011-01-10 22:26 UTC (permalink / raw)
To: Alexander Holler; +Cc: linux-kernel, linux-arm-kernel, linux
On Mon, 10 Jan 2011, Alexander Holler wrote:
> Hello,
>
> I've build a kernel 2.6.37 for an armv5-device which includes an initramfs of
> about 16MB (uncompressed, result is a kernel of about 6MB) and I now getting
> errors when trying to load a module (which is included in that initramfs):
Do you really need to have such a bit initramfs?
What you can do to work around this issue is to load your initramfs
separately as a initrd ramdisk image. , and not combine it with the
kernel image.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-10 22:26 ` Nicolas Pitre
@ 2011-01-11 6:34 ` Alexander Holler
2011-01-11 15:17 ` Alexander Holler
` (2 more replies)
0 siblings, 3 replies; 33+ messages in thread
From: Alexander Holler @ 2011-01-11 6:34 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: linux-kernel, linux
Am 10.01.2011 23:26, schrieb Nicolas Pitre:
> On Mon, 10 Jan 2011, Alexander Holler wrote:
>
>> I've build a kernel 2.6.37 for an armv5-device which includes an initramfs of
>> about 16MB (uncompressed, result is a kernel of about 6MB) and I now getting
>> errors when trying to load a module (which is included in that initramfs):
>
> Do you really need to have such a bit initramfs?
Not really, but it is handier to use than a separate file.
> What you can do to work around this issue is to load your initramfs
> separately as a initrd ramdisk image. , and not combine it with the
> kernel image.
Point is that I would like to understand why including an initramfs has
such consequences.
I have to search for some pictures which are explaining the memory
layout (never had any interested in that before).
I assume some memory near the kernel is reserved for the modules and an
initramfs will come inbetween the kernel and the reserved memory for
modules.
Regards,
Alexander
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 6:34 ` Alexander Holler
@ 2011-01-11 15:17 ` Alexander Holler
2011-01-11 15:57 ` Russell King - ARM Linux
2011-01-11 16:02 ` Nicolas Pitre
2011-01-11 15:46 ` Rabin Vincent
2011-01-11 16:01 ` Nicolas Pitre
2 siblings, 2 replies; 33+ messages in thread
From: Alexander Holler @ 2011-01-11 15:17 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: linux-kernel, linux
Am 11.01.2011 07:34, schrieb Alexander Holler:
> Am 10.01.2011 23:26, schrieb Nicolas Pitre:
>> On Mon, 10 Jan 2011, Alexander Holler wrote:
>>
>>> I've build a kernel 2.6.37 for an armv5-device which includes an
>>> initramfs of
>>> about 16MB (uncompressed, result is a kernel of about 6MB) and I now
>>> getting
>>> errors when trying to load a module (which is included in that
>>> initramfs):
>>
>> Do you really need to have such a bit initramfs?
>
> Not really, but it is handier to use than a separate file.
>
>> What you can do to work around this issue is to load your initramfs
>> separately as a initrd ramdisk image. , and not combine it with the
>> kernel image.
>
> Point is that I would like to understand why including an initramfs has
> such consequences.
>
> I have to search for some pictures which are explaining the memory
> layout (never had any interested in that before).
>
> I assume some memory near the kernel is reserved for the modules and an
> initramfs will come inbetween the kernel and the reserved memory for
> modules.
Thinking a bit more about that (for me unexcpected) "relocation out of
range" error when loading a module, I'm curious if there could be other
conditions which would lead to that error. Especially conditions which
could be generated by the userspace or remote from some network.
Regards,
Alexander
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 6:34 ` Alexander Holler
2011-01-11 15:17 ` Alexander Holler
@ 2011-01-11 15:46 ` Rabin Vincent
2011-01-11 15:59 ` Russell King - ARM Linux
2011-01-11 16:01 ` Nicolas Pitre
2 siblings, 1 reply; 33+ messages in thread
From: Rabin Vincent @ 2011-01-11 15:46 UTC (permalink / raw)
To: Alexander Holler; +Cc: Nicolas Pitre, linux-kernel, linux, linux-arm-kernel
On Tue, Jan 11, 2011 at 12:04, Alexander Holler <holler@ahsoftware.de> wrote:
> Point is that I would like to understand why including an initramfs has such
> consequences.
>
> I have to search for some pictures which are explaining the memory layout
> (never had any interested in that before).
>
> I assume some memory near the kernel is reserved for the modules and an
> initramfs will come inbetween the kernel and the reserved memory for
> modules.
Modules get placed 16MiB below PAGE_OFFSET, and when you add another
~16MiB initramfs between them and the kernel symbols, you exceed the
offset range of the branch instruction.
It's possible to hack around this by placing the initramfs at the end of
the kernel image rather than at the beginning with the rest of the init
data. Something like the below should work, although you should also
probably take care of alignment and also have this section freed when
the rest of the init data is freed.
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index b16c079..05ad361 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -46,7 +46,6 @@ SECTIONS
INIT_CALLS
CON_INITCALL
SECURITY_INITCALL
- INIT_RAM_FS
#ifndef CONFIG_XIP_KERNEL
__init_begin = _stext;
@@ -170,6 +169,11 @@ SECTIONS
_edata = .;
}
+
+ .initramfs : {
+ INIT_RAM_FS
+ }
+
_edata_loc = __data_loc + SIZEOF(.data);
#ifdef CONFIG_HAVE_TCM
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 15:17 ` Alexander Holler
@ 2011-01-11 15:57 ` Russell King - ARM Linux
2011-01-11 16:02 ` Nicolas Pitre
1 sibling, 0 replies; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-01-11 15:57 UTC (permalink / raw)
To: Alexander Holler; +Cc: Nicolas Pitre, linux-kernel
On Tue, Jan 11, 2011 at 04:17:24PM +0100, Alexander Holler wrote:
> Thinking a bit more about that (for me unexcpected) "relocation out of
> range" error when loading a module, I'm curious if there could be other
> conditions which would lead to that error. Especially conditions which
> could be generated by the userspace or remote from some network.
It's purely about the relative position of the kernel text to the module
text. Nothing outside of the compilation of the kernel will affect it.
It's all to do with the branch instructions in modules having a limited
range.
We generally place the kernel such that it starts 32K into the kernel's
system RAM mapping. We place modules immediately below this mapping.
The initramfs image sits in the kernel body in the startup data, which
sits between the startup text at the beginning of the kernel, and the
main kernel text which follows.
We _could_ rearrange the placement of these sections within the kernel
image to eliminate the problem, but it's not a simple matter to just
rearrange them - there's various assumptions made about the current
arrangement. There's also considerations over whether that would cause
additional memory to be wasted due to the alignment requirements of the
various sections, and whether it would degrade the ability to get
DMA-able memory on certain platforms.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 15:46 ` Rabin Vincent
@ 2011-01-11 15:59 ` Russell King - ARM Linux
2011-01-12 3:00 ` Alexander Holler
2011-01-12 16:25 ` Matthieu CASTET
0 siblings, 2 replies; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-01-11 15:59 UTC (permalink / raw)
To: Rabin Vincent
Cc: Alexander Holler, linux-kernel, linux-arm-kernel, Nicolas Pitre
On Tue, Jan 11, 2011 at 09:16:38PM +0530, Rabin Vincent wrote:
> It's possible to hack around this by placing the initramfs at the end of
> the kernel image rather than at the beginning with the rest of the init
> data. Something like the below should work, although you should also
> probably take care of alignment and also have this section freed when
> the rest of the init data is freed.
You're then running into problems as _sdata.._edata is copied to RAM on
XIP kernels, and you really don't want to waste time copying the
initramfs to RAM.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 6:34 ` Alexander Holler
2011-01-11 15:17 ` Alexander Holler
2011-01-11 15:46 ` Rabin Vincent
@ 2011-01-11 16:01 ` Nicolas Pitre
2 siblings, 0 replies; 33+ messages in thread
From: Nicolas Pitre @ 2011-01-11 16:01 UTC (permalink / raw)
To: Alexander Holler; +Cc: linux-kernel, linux
On Tue, 11 Jan 2011, Alexander Holler wrote:
> Am 10.01.2011 23:26, schrieb Nicolas Pitre:
> > On Mon, 10 Jan 2011, Alexander Holler wrote:
> >
> > > I've build a kernel 2.6.37 for an armv5-device which includes an initramfs
> > > of
> > > about 16MB (uncompressed, result is a kernel of about 6MB) and I now
> > > getting
> > > errors when trying to load a module (which is included in that initramfs):
> >
> > Do you really need to have such a bit initramfs?
>
> Not really, but it is handier to use than a separate file.
>
> > What you can do to work around this issue is to load your initramfs
> > separately as a initrd ramdisk image. , and not combine it with the
> > kernel image.
>
> Point is that I would like to understand why including an initramfs has such
> consequences.
The layout of the kernel virtual memory goes like this:
0xbf000000 to 0xbfffffff: kernel module area
0xc0008000 and above: kernel image
The module area is located just before the kernel memory because this
allows function calls from modules to be linked directly without any
indirect branches. For that to work, the branch destination has to be
within 32MB of its origin location.
Now the kernel image has its __init section located at the beginning. It
is located there because of some alignment restriction, and the
initramfs data is located in the __init section. So if your initramfs
is about 16MB, plus the size of the module area which is also 16MB you
get a distance from a loaded module to the actual kernel core bigger
than the possible branch fixup.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 15:17 ` Alexander Holler
2011-01-11 15:57 ` Russell King - ARM Linux
@ 2011-01-11 16:02 ` Nicolas Pitre
1 sibling, 0 replies; 33+ messages in thread
From: Nicolas Pitre @ 2011-01-11 16:02 UTC (permalink / raw)
To: Alexander Holler; +Cc: lkml, Russell King - ARM Linux
On Tue, 11 Jan 2011, Alexander Holler wrote:
> Thinking a bit more about that (for me unexcpected) "relocation out of range"
> error when loading a module, I'm curious if there could be other conditions
> which would lead to that error. Especially conditions which could be generated
> by the userspace or remote from some network.
No.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 15:59 ` Russell King - ARM Linux
@ 2011-01-12 3:00 ` Alexander Holler
2011-01-12 16:05 ` Dave Martin
2011-01-12 16:25 ` Matthieu CASTET
1 sibling, 1 reply; 33+ messages in thread
From: Alexander Holler @ 2011-01-12 3:00 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Rabin Vincent, linux-kernel, linux-arm-kernel, Nicolas Pitre
Hello,
Am 11.01.2011 16:59, schrieb Russell King - ARM Linux:
> On Tue, Jan 11, 2011 at 09:16:38PM +0530, Rabin Vincent wrote:
>> It's possible to hack around this by placing the initramfs at the end of
>> the kernel image rather than at the beginning with the rest of the init
>> data. Something like the below should work, although you should also
>> probably take care of alignment and also have this section freed when
>> the rest of the init data is freed.
>
> You're then running into problems as _sdata.._edata is copied to RAM on
> XIP kernels, and you really don't want to waste time copying the
> initramfs to RAM.
Thanks to all for the provided informations. I'm now seeing some light. ;)
However, looking at the vmlinux.lds.S I think I should better not try to
fix that, I would likely do something wrong because of missing knowledge
about all those sections.
I can offer a patch wich adds a TODO to vmlinux.lds.S, but if someone
else feels the need to fix that, feel free to do so. ;)
Regards,
Alexander
-------------------------------------------------------------------------
From 2ce2934b6a55ac34734781d1a49569d79d0fdcda Mon Sep 17 00:00:00 2001
From: Alexander Holler <holler@ahsoftware.de>
Date: Wed, 12 Jan 2011 02:49:52 +0100
Subject: [PATCH] ARM: Add TODO to move INIT_RAM_FS to another point.
Signed-off-by: Alexander Holler <holler@ahsoftware.de>
---
arch/arm/kernel/vmlinux.lds.S | 6 ++++++
1 files changed, 6 insertions(+), 0 deletions(-)
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index cead889..5da3479 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -62,6 +62,12 @@ SECTIONS
INIT_CALLS
CON_INITCALL
SECURITY_INITCALL
+ /*
+ * TODO: The size of INIT_RAM_FS could easily reach a
+ * point (~16MB) when loading modules will fail because
+ * relocations will be out of range. So this place here
+ * isn't the best one.
+ */
INIT_RAM_FS
#ifndef CONFIG_XIP_KERNEL
--
1.7.3.4
-------------------------------------------------------------------------
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 3:00 ` Alexander Holler
@ 2011-01-12 16:05 ` Dave Martin
2011-01-12 16:23 ` Russell King - ARM Linux
2011-01-12 18:28 ` Nicolas Pitre
0 siblings, 2 replies; 33+ messages in thread
From: Dave Martin @ 2011-01-12 16:05 UTC (permalink / raw)
To: Alexander Holler
Cc: Russell King - ARM Linux, Rabin Vincent, linux-kernel,
linux-arm-kernel, Nicolas Pitre
On Tue, Jan 11, 2011 at 9:00 PM, Alexander Holler <holler@ahsoftware.de> wrote:
> Hello,
[...]
> INIT_CALLS
> CON_INITCALL
> SECURITY_INITCALL
> + /*
> + * TODO: The size of INIT_RAM_FS could easily reach a
> + * point (~16MB) when loading modules will fail because
> + * relocations will be out of range. So this place here
> + * isn't the best one.
> + */
> INIT_RAM_FS
In general, do we expect always to be able to avoid the situation
where branches in the kernel may need to cover too large a range ...
and is there any strategy for working aroung it?
If we have problems branching from the modules area into vmlinux, we
could possibly build modules with -fPIC : this would remove the
restriction on branch range, though there would also be some
performance impact for the modules...
Cheers
---Dave
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 16:05 ` Dave Martin
@ 2011-01-12 16:23 ` Russell King - ARM Linux
2011-01-12 18:28 ` Nicolas Pitre
1 sibling, 0 replies; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-01-12 16:23 UTC (permalink / raw)
To: Dave Martin
Cc: Alexander Holler, Rabin Vincent, linux-kernel, linux-arm-kernel,
Nicolas Pitre
On Wed, Jan 12, 2011 at 10:05:19AM -0600, Dave Martin wrote:
> In general, do we expect always to be able to avoid the situation
> where branches in the kernel may need to cover too large a range ...
> and is there any strategy for working aroung it?
It's not that big a problem - 99.999999% of setups never run into the
problem. It's only those who use large initramfs's built into their
kernel image at present, and that is ultimately solvable.
I don't think we need to litter code with TODO comments.
> If we have problems branching from the modules area into vmlinux, we
> could possibly build modules with -fPIC : this would remove the
> restriction on branch range, though there would also be some
> performance impact for the modules...
That also brings in issues with GOT tables and the like, and also requires
different build options for modules and the main kernel. I don't think
kbuild is setup to do that - and I'd argue that it's unnecessary if we
fix the layout of the kernel image.
As I said a few emails ago, shuffling sections around in the image is
not as trivial as it looks on the face of it as we make assumptions
about what is in _stext.._etext, _sdata.._edata, whether there's anything
between _etext.._sdata, and other symbolic ranges.
At the moment, _stext.._etext + _sdata.._edata covers the entire kernel
image - with the init sections at the start of _stext. Putting the init
sections between _etext and _sdata makes a hole in the middle of that,
which may be suboptimal for page allocation. It also means that the
range no longer covers all kernel stuff. Also whether _stext = _text,
_sdata = _data, etc.
Another technicality is that some of these ranges are used for stuff like
DMA API debugging:
if (overlap(addr, len, _text, _etext) ||
overlap(addr, len, __start_rodata, __end_rodata))
err_printk(dev, NULL, "DMA-API: device driver maps memory from kernel text or rodata [addr=%p] [len=%lu]\n", addr, len);
The last pieces of the puzzle is whether we have anything that implicitly
relies on the init section being low down and freeing its pages (eg, as
a way of stopping that memory being used for non-DMA stuff.)
I do have a patch which shuffles some of this stuff around, but I'm not
entirely happy with it yet.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-11 15:59 ` Russell King - ARM Linux
2011-01-12 3:00 ` Alexander Holler
@ 2011-01-12 16:25 ` Matthieu CASTET
2011-01-12 16:38 ` Russell King - ARM Linux
1 sibling, 1 reply; 33+ messages in thread
From: Matthieu CASTET @ 2011-01-12 16:25 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Rabin Vincent, Alexander Holler, linux-kernel, linux-arm-kernel,
Nicolas Pitre
Russell King - ARM Linux a écrit :
> On Tue, Jan 11, 2011 at 09:16:38PM +0530, Rabin Vincent wrote:
>> It's possible to hack around this by placing the initramfs at the end of
>> the kernel image rather than at the beginning with the rest of the init
>> data. Something like the below should work, although you should also
>> probably take care of alignment and also have this section freed when
>> the rest of the init data is freed.
>
> You're then running into problems as _sdata.._edata is copied to RAM on
> XIP kernels, and you really don't want to waste time copying the
> initramfs to RAM.
>
But in this case initramfs is after edata and before bss.
So where is the problem ?
Matthieu
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 16:25 ` Matthieu CASTET
@ 2011-01-12 16:38 ` Russell King - ARM Linux
0 siblings, 0 replies; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-01-12 16:38 UTC (permalink / raw)
To: Matthieu CASTET
Cc: Rabin Vincent, Alexander Holler, linux-kernel, linux-arm-kernel,
Nicolas Pitre
On Wed, Jan 12, 2011 at 05:25:28PM +0100, Matthieu CASTET wrote:
> Russell King - ARM Linux a écrit :
>> On Tue, Jan 11, 2011 at 09:16:38PM +0530, Rabin Vincent wrote:
>>> It's possible to hack around this by placing the initramfs at the end of
>>> the kernel image rather than at the beginning with the rest of the init
>>> data. Something like the below should work, although you should also
>>> probably take care of alignment and also have this section freed when
>>> the rest of the init data is freed.
>>
>> You're then running into problems as _sdata.._edata is copied to RAM on
>> XIP kernels, and you really don't want to waste time copying the
>> initramfs to RAM.
>>
> But in this case initramfs is after edata and before bss.
Right, so that isn't a concern.
> So where is the problem ?
There is a problem with it, as noted by Rabin. The initramfs will
never be freed. Let's try to find a proper solution to this rather
than everyone sticking their half-baked ideas at it. Let's see if
we can rearrange the _entire_ layout in a way that sorts this out
once and for all without making things even more complicated than
they already are today.
We've already got enough variables to think about here without having
yet more freeable sections scattered in the middle of the existing
layout.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 16:05 ` Dave Martin
2011-01-12 16:23 ` Russell King - ARM Linux
@ 2011-01-12 18:28 ` Nicolas Pitre
2011-01-12 18:42 ` Russell King - ARM Linux
1 sibling, 1 reply; 33+ messages in thread
From: Nicolas Pitre @ 2011-01-12 18:28 UTC (permalink / raw)
To: Dave Martin
Cc: Alexander Holler, Russell King - ARM Linux, Rabin Vincent,
linux-kernel, linux-arm-kernel
On Wed, 12 Jan 2011, Dave Martin wrote:
> In general, do we expect always to be able to avoid the situation
> where branches in the kernel may need to cover too large a range ...
> and is there any strategy for working aroung it?
Normally we try to keep the code close together. And at least for the
.text section, the linker is able to insert indirect branch veneers when
the range is too large. But in the case of modules, it's the in-kernel
code that perform the final symbol relocation.
> If we have problems branching from the modules area into vmlinux, we
> could possibly build modules with -fPIC : this would remove the
> restriction on branch range, though there would also be some
> performance impact for the modules...
If we really needed to do such thing, that would be even better to
simply have the kernel create those indirect veneers dynamically. And
in fact, Russell had that working and he posted the corresponding patch
many years ago, but the module placement was made so that the indirect
branches were unnecessary.
Even now, it's probably a better idea to rework the section layout and
preserve the ability to perform direct branches into the kernel from
modules as much as possible.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 18:28 ` Nicolas Pitre
@ 2011-01-12 18:42 ` Russell King - ARM Linux
2011-01-12 18:55 ` Nicolas Pitre
2011-01-13 5:50 ` Alexander Holler
0 siblings, 2 replies; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-01-12 18:42 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Dave Martin, Alexander Holler, Rabin Vincent, linux-kernel,
linux-arm-kernel
On Wed, Jan 12, 2011 at 01:28:23PM -0500, Nicolas Pitre wrote:
> If we really needed to do such thing, that would be even better to
> simply have the kernel create those indirect veneers dynamically. And
> in fact, Russell had that working and he posted the corresponding patch
> many years ago, but the module placement was made so that the indirect
> branches were unnecessary.
Actually, it's something we used to do in 2.2 days when modules were
prepared and linked in userspace before being uploaded into kernel
space. This allowed the module to be inteligently linked - so the
indirect branches were created only when they were necessary.
When the new kernel-based module linker happened, this presented a
chicken and egg problem with that approach, which give us a choice:
either place the module within range of the kernel text and guarantee
that the kernel text is reachable, or _always_ indirect every module
branch through a jump table even if it was reachable from where the
module was placed.
The decision was made to go with the former, so the latter never got
implemented.
Then came along the embedded initrd/initramfs idea which rather buggered
the scheme when large initramfs are embedded into the image.
As the overall feeling at the time was "don't use large initrds" it's
something I've never really cared about - and I'm still of the opinion
that 16MB of compressed initrd/initramfs is rather silly.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 18:42 ` Russell King - ARM Linux
@ 2011-01-12 18:55 ` Nicolas Pitre
2011-01-25 8:44 ` Sachin Verma
2011-01-13 5:50 ` Alexander Holler
1 sibling, 1 reply; 33+ messages in thread
From: Nicolas Pitre @ 2011-01-12 18:55 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Dave Martin, Alexander Holler, Rabin Vincent, lkml, linux-arm-kernel
On Wed, 12 Jan 2011, Russell King - ARM Linux wrote:
> Then came along the embedded initrd/initramfs idea which rather buggered
> the scheme when large initramfs are embedded into the image.
>
> As the overall feeling at the time was "don't use large initrds" it's
> something I've never really cared about - and I'm still of the opinion
> that 16MB of compressed initrd/initramfs is rather silly.
It is... but we have more than 32MB of RAM total now, and people are
running standard distributions on ARM these days, such as Fedora or
Ubuntu, including their corresponding initrd that may contain lots of
modules, splashscreen data, etc. So it might be a good idea to think
about fixing this limitation before it comes back again.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 18:42 ` Russell King - ARM Linux
2011-01-12 18:55 ` Nicolas Pitre
@ 2011-01-13 5:50 ` Alexander Holler
2011-01-13 10:04 ` Russell King - ARM Linux
1 sibling, 1 reply; 33+ messages in thread
From: Alexander Holler @ 2011-01-13 5:50 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Nicolas Pitre, Dave Martin, Rabin Vincent, linux-kernel,
linux-arm-kernel
Hello,
Am 12.01.2011 19:42, schrieb Russell King - ARM Linux:
> As the overall feeling at the time was "don't use large initrds" it's
> something I've never really cared about - and I'm still of the opinion
> that 16MB of compressed initrd/initramfs is rather silly.
Hmm, sorry, I have to disagree. ;)
E.g. 12MB out of the 16MB in the initramfs I've used are modules.
I've also created some larger initramfs wich included X11 and such, and
I find it very convenient to use the initramfs to include a rootfs in
the the kernel and so only have to handle one file for all. E.g. just
load the kernel with the included rootfs via tftp and you are done.
Sure this can all be done using an external initrd, but that is much
less comfortable to use.
At least I like that feature since it is available and won't miss it now. ;)
Regards,
Alexander
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-13 5:50 ` Alexander Holler
@ 2011-01-13 10:04 ` Russell King - ARM Linux
2011-01-13 14:36 ` Alexander Holler
0 siblings, 1 reply; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-01-13 10:04 UTC (permalink / raw)
To: Alexander Holler
Cc: Nicolas Pitre, Dave Martin, Rabin Vincent, linux-kernel,
linux-arm-kernel
On Thu, Jan 13, 2011 at 06:50:05AM +0100, Alexander Holler wrote:
> Hello,
>
> Am 12.01.2011 19:42, schrieb Russell King - ARM Linux:
>
>> As the overall feeling at the time was "don't use large initrds" it's
>> something I've never really cared about - and I'm still of the opinion
>> that 16MB of compressed initrd/initramfs is rather silly.
>
> Hmm, sorry, I have to disagree. ;)
> E.g. 12MB out of the 16MB in the initramfs I've used are modules.
> I've also created some larger initramfs wich included X11 and such, and
> I find it very convenient to use the initramfs to include a rootfs in
> the the kernel and so only have to handle one file for all. E.g. just
> load the kernel with the included rootfs via tftp and you are done.
Oddly, that's why I wrote the bootp wrapper, which predates the
currently broken initrd-in-kernel idea.
> Sure this can all be done using an external initrd, but that is much
> less comfortable to use.
>
> At least I like that feature since it is available and won't miss it now. ;)
The feature isn't available if it doesn't work. It's not going to get
fixed for 2.6.38 - it'll be 2.6.39 due to the timing and complexity of
the problem.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-13 10:04 ` Russell King - ARM Linux
@ 2011-01-13 14:36 ` Alexander Holler
0 siblings, 0 replies; 33+ messages in thread
From: Alexander Holler @ 2011-01-13 14:36 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Nicolas Pitre, Dave Martin, Rabin Vincent, linux-kernel,
linux-arm-kernel
Am 13.01.2011 11:04, schrieb Russell King - ARM Linux:
>> At least I like that feature since it is available and won't miss it now. ;)
>
> The feature isn't available if it doesn't work. It's not going to get
> fixed for 2.6.38 - it'll be 2.6.39 due to the timing and complexity of
> the problem.
I just wanted to express that I like that feature a lot and I didn't
wanted to ask for a quick fix nor did I wanted to make pressure.
I've tried a the patch from Rabin Vincent (along with an alignment) and
it works. It might not be the best solution and I certainly waste some
memory (no free), but I don't care.
Regards,
Alexander
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-12 18:55 ` Nicolas Pitre
@ 2011-01-25 8:44 ` Sachin Verma
2011-01-25 13:23 ` Catalin Marinas
2011-01-27 5:43 ` Nicolas Pitre
0 siblings, 2 replies; 33+ messages in thread
From: Sachin Verma @ 2011-01-25 8:44 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Russell King - ARM Linux, Rabin Vincent, Alexander Holler,
Dave Martin, lkml, linux-arm-kernel, catalin.marinas
Hello,
On Thu, Jan 13, 2011 at 12:25 AM, Nicolas Pitre
<nicolas.pitre@linaro.org> wrote:
> On Wed, 12 Jan 2011, Russell King - ARM Linux wrote:
>
>> Then came along the embedded initrd/initramfs idea which rather buggered
>> the scheme when large initramfs are embedded into the image.
>>
>> As the overall feeling at the time was "don't use large initrds" it's
>> something I've never really cared about - and I'm still of the opinion
>> that 16MB of compressed initrd/initramfs is rather silly.
>
> It is... but we have more than 32MB of RAM total now, and people are
> running standard distributions on ARM these days, such as Fedora or
> Ubuntu, including their corresponding initrd that may contain lots of
> modules, splashscreen data, etc. So it might be a good idea to think
> about fixing this limitation before it comes back again.
>
in fact it is not just the initrd/initramfs size, i am facing issues
with loading modules on MMU-less envt(Cortex M3).
I have 128MB of Physical RAM and i am using a 4MB initrd to boot my
system. when i try to load kernel modules i get relocation errors.
As MMU-less systems are allowing modules to be loaded anywhere from
RAM and not from any predefined limits of MODULES_VADDR and
MODULES_END.
Do you have any pointers on how to tackle this situation for MMU less
environments?
Regards,
Sachin.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-25 8:44 ` Sachin Verma
@ 2011-01-25 13:23 ` Catalin Marinas
2011-01-27 5:43 ` Nicolas Pitre
1 sibling, 0 replies; 33+ messages in thread
From: Catalin Marinas @ 2011-01-25 13:23 UTC (permalink / raw)
To: Sachin Verma
Cc: Nicolas Pitre, Russell King - ARM Linux, Rabin Vincent,
Alexander Holler, Dave Martin, lkml, linux-arm-kernel
On 25 January 2011 08:44, Sachin Verma <imschnvrm@gmail.com> wrote:
> On Thu, Jan 13, 2011 at 12:25 AM, Nicolas Pitre
> <nicolas.pitre@linaro.org> wrote:
>> On Wed, 12 Jan 2011, Russell King - ARM Linux wrote:
>>
>>> Then came along the embedded initrd/initramfs idea which rather buggered
>>> the scheme when large initramfs are embedded into the image.
>>>
>>> As the overall feeling at the time was "don't use large initrds" it's
>>> something I've never really cared about - and I'm still of the opinion
>>> that 16MB of compressed initrd/initramfs is rather silly.
>>
>> It is... but we have more than 32MB of RAM total now, and people are
>> running standard distributions on ARM these days, such as Fedora or
>> Ubuntu, including their corresponding initrd that may contain lots of
>> modules, splashscreen data, etc. So it might be a good idea to think
>> about fixing this limitation before it comes back again.
>>
>
> in fact it is not just the initrd/initramfs size, i am facing issues
> with loading modules on MMU-less envt(Cortex M3).
> I have 128MB of Physical RAM and i am using a 4MB initrd to boot my
> system. when i try to load kernel modules i get relocation errors.
> As MMU-less systems are allowing modules to be loaded anywhere from
> RAM and not from any predefined limits of MODULES_VADDR and
> MODULES_END.
>
> Do you have any pointers on how to tackle this situation for MMU less
> environments?
I don't think there is a simple solution. As a hack, you could define
DMA and Normal Zones in your system and force module allocations from
the DMA zone (which would be close to the kernel).
I'm not sure whether the kernel could be compile with a larger memory
model, but then you have to improve the kernel to handle new
relocation types.
--
Catalin
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-25 8:44 ` Sachin Verma
2011-01-25 13:23 ` Catalin Marinas
@ 2011-01-27 5:43 ` Nicolas Pitre
2011-02-10 15:43 ` Russell King - ARM Linux
1 sibling, 1 reply; 33+ messages in thread
From: Nicolas Pitre @ 2011-01-27 5:43 UTC (permalink / raw)
To: Sachin Verma
Cc: Russell King - ARM Linux, Rabin Vincent, Alexander Holler,
Dave Martin, lkml, linux-arm-kernel, catalin.marinas
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1916 bytes --]
On Tue, 25 Jan 2011, Sachin Verma wrote:
> Hello,
>
> On Thu, Jan 13, 2011 at 12:25 AM, Nicolas Pitre
> <nicolas.pitre@linaro.org> wrote:
> > On Wed, 12 Jan 2011, Russell King - ARM Linux wrote:
> >
> >> As the overall feeling at the time was "don't use large initrds" it's
> >> something I've never really cared about - and I'm still of the opinion
> >> that 16MB of compressed initrd/initramfs is rather silly.
> >
> > It is... but we have more than 32MB of RAM total now, and people are
> > running standard distributions on ARM these days, such as Fedora or
> > Ubuntu, including their corresponding initrd that may contain lots of
> > modules, splashscreen data, etc. So it might be a good idea to think
> > about fixing this limitation before it comes back again.
> >
>
> in fact it is not just the initrd/initramfs size, i am facing issues
> with loading modules on MMU-less envt(Cortex M3).
> I have 128MB of Physical RAM and i am using a 4MB initrd to boot my
> system. when i try to load kernel modules i get relocation errors.
> As MMU-less systems are allowing modules to be loaded anywhere from
> RAM and not from any predefined limits of MODULES_VADDR and
> MODULES_END.
>
> Do you have any pointers on how to tackle this situation for MMU less
> environments?
The MMU-less kernel should still favor allocations close to the kernel
text for modules, and anything else away from the kernel going
downwards.
Otherwise a veneer should be created by the module symbol resolver such
that if the branch distance to reach, say, printk is too large, then the
following code would have to be dynamically generated right next to the
module:
ldr pc, [pc, #-4]
.word <far_away_symbol>
Then, in your module, you patch the branch relocation for printk so that
it branches to the code above instead, and then store the address of
printk at the location represented by the .word directive.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-01-27 5:43 ` Nicolas Pitre
@ 2011-02-10 15:43 ` Russell King - ARM Linux
2011-02-10 19:41 ` Nicolas Pitre
0 siblings, 1 reply; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-02-10 15:43 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Sachin Verma, Rabin Vincent, Alexander Holler, Dave Martin, lkml,
linux-arm-kernel, catalin.marinas
On Thu, Jan 27, 2011 at 12:43:54AM -0500, Nicolas Pitre wrote:
> The MMU-less kernel should still favor allocations close to the kernel
> text for modules, and anything else away from the kernel going
> downwards.
>
> Otherwise a veneer should be created by the module symbol resolver such
> that if the branch distance to reach, say, printk is too large, then the
> following code would have to be dynamically generated right next to the
> module:
>
> ldr pc, [pc, #-4]
> .word <far_away_symbol>
>
> Then, in your module, you patch the branch relocation for printk so that
> it branches to the code above instead, and then store the address of
> printk at the location represented by the .word directive.
What you're suggesting is what we used to do with the old user-space
module tools, which would've been nice to carry forwards to the new
module code. I never found a way to do it.
The problems:
1. Where do you create those veneers?
2. How many veneers do you allocate space for?
3. How do you determine that you need a veneer?
While you can say "next to the module" for (1), you can only do that at
the point in time when the space for the module is allocated, and you
need to know at that point how much space you require.
For (2), you could always allocate space for one veneer per symbol present
in the module, but that's very wasteful.
(3) is almost impossible to know ahead of time as you don't have the
relocations, realistically you have to allocate one veneer per symbol,
and as you don't know whether it's a data or code symbol, you'll have
to allocate one veneer for every symbol in a module.
I really don't like it, and I don't see that this is sanely solvable
without giving architectures much more control over module loading,
which I don't think will ever happen. It's probably simpler to build
modules with whatever that magic option is to tell GCC to always generate
'far call' veneers for everything rather than trying to 'fix' the kernel
module loader.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-10 15:43 ` Russell King - ARM Linux
@ 2011-02-10 19:41 ` Nicolas Pitre
2011-02-11 9:31 ` Dave Martin
0 siblings, 1 reply; 33+ messages in thread
From: Nicolas Pitre @ 2011-02-10 19:41 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Sachin Verma, Rabin Vincent, Alexander Holler, Dave Martin, lkml,
linux-arm-kernel, catalin.marinas
On Thu, 10 Feb 2011, Russell King - ARM Linux wrote:
> On Thu, Jan 27, 2011 at 12:43:54AM -0500, Nicolas Pitre wrote:
> > The MMU-less kernel should still favor allocations close to the kernel
> > text for modules, and anything else away from the kernel going
> > downwards.
> >
> > Otherwise a veneer should be created by the module symbol resolver such
> > that if the branch distance to reach, say, printk is too large, then the
> > following code would have to be dynamically generated right next to the
> > module:
> >
> > ldr pc, [pc, #-4]
> > .word <far_away_symbol>
> >
> > Then, in your module, you patch the branch relocation for printk so that
> > it branches to the code above instead, and then store the address of
> > printk at the location represented by the .word directive.
>
> What you're suggesting is what we used to do with the old user-space
> module tools, which would've been nice to carry forwards to the new
> module code. I never found a way to do it.
>
> The problems:
> 1. Where do you create those veneers?
> 2. How many veneers do you allocate space for?
> 3. How do you determine that you need a veneer?
>
> While you can say "next to the module" for (1), you can only do that at
> the point in time when the space for the module is allocated, and you
> need to know at that point how much space you require.
You would have to guess of course. Having a guess of 1/2 the module
size should be pretty safe. So allocating 3/2 the space in
module_alloc(), and then suffice to free the unused portion in
module_finalize().
> For (2), you could always allocate space for one veneer per symbol present
> in the module, but that's very wasteful.
>
> (3) is almost impossible to know ahead of time as you don't have the
> relocations, realistically you have to allocate one veneer per symbol,
> and as you don't know whether it's a data or code symbol, you'll have
> to allocate one veneer for every symbol in a module.
I don't think you may know the number of symbols in advance either
anyway.
> I really don't like it, and I don't see that this is sanely solvable
> without giving architectures much more control over module loading,
> which I don't think will ever happen. It's probably simpler to build
> modules with whatever that magic option is to tell GCC to always generate
> 'far call' veneers for everything rather than trying to 'fix' the kernel
> module loader.
Well, this is certainly possible indeed to just use -mlong-calls to
build modules which would solve the issue right away, and with a smaller
cost than if each function calls were going through a veneer, but with a
higher cost than a relocatable direct branch. So this could be decided
with a Kconfig option.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-10 19:41 ` Nicolas Pitre
@ 2011-02-11 9:31 ` Dave Martin
2011-02-11 9:38 ` Russell King - ARM Linux
2011-02-11 13:51 ` Nicolas Pitre
0 siblings, 2 replies; 33+ messages in thread
From: Dave Martin @ 2011-02-11 9:31 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Russell King - ARM Linux, Sachin Verma, Rabin Vincent,
Alexander Holler, lkml, linux-arm-kernel, catalin.marinas
On Thu, Feb 10, 2011 at 7:41 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 10 Feb 2011, Russell King - ARM Linux wrote:
>
>> On Thu, Jan 27, 2011 at 12:43:54AM -0500, Nicolas Pitre wrote:
>> > The MMU-less kernel should still favor allocations close to the kernel
>> > text for modules, and anything else away from the kernel going
>> > downwards.
>> >
>> > Otherwise a veneer should be created by the module symbol resolver such
>> > that if the branch distance to reach, say, printk is too large, then the
>> > following code would have to be dynamically generated right next to the
>> > module:
>> >
>> > ldr pc, [pc, #-4]
>> > .word <far_away_symbol>
>> >
>> > Then, in your module, you patch the branch relocation for printk so that
>> > it branches to the code above instead, and then store the address of
>> > printk at the location represented by the .word directive.
>>
>> What you're suggesting is what we used to do with the old user-space
>> module tools, which would've been nice to carry forwards to the new
>> module code. I never found a way to do it.
>>
>> The problems:
>> 1. Where do you create those veneers?
>> 2. How many veneers do you allocate space for?
>> 3. How do you determine that you need a veneer?
>>
>> While you can say "next to the module" for (1), you can only do that at
>> the point in time when the space for the module is allocated, and you
>> need to know at that point how much space you require.
>
> You would have to guess of course. Having a guess of 1/2 the module
> size should be pretty safe. So allocating 3/2 the space in
> module_alloc(), and then suffice to free the unused portion in
> module_finalize().
>
>> For (2), you could always allocate space for one veneer per symbol present
>> in the module, but that's very wasteful.
>>
>> (3) is almost impossible to know ahead of time as you don't have the
>> relocations, realistically you have to allocate one veneer per symbol,
>> and as you don't know whether it's a data or code symbol, you'll have
>> to allocate one veneer for every symbol in a module.
>
> I don't think you may know the number of symbols in advance either
> anyway.
You could probably cook up a good upper bound based on the size of the
kernel and the number of symbols in the module: i.e., assume that
every undefined symbol in the module needs to be fixed up to point at
the most distant symbol in the kernel.
For people with normal-sized kernels, this bound will probably work
out as zero most of the time (i.e., the current situation). For
people with big kernels, or when many modules are already loaded, it
may work out at 100% -- but that's the price to pay for guaranteed
preallocation of the space required for the veneers. And anyway, you
may really need a substantial chunk of those veneers in such cases.
---Dave
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-11 9:31 ` Dave Martin
@ 2011-02-11 9:38 ` Russell King - ARM Linux
2011-02-11 9:45 ` Dave Martin
2011-02-11 13:51 ` Nicolas Pitre
1 sibling, 1 reply; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-02-11 9:38 UTC (permalink / raw)
To: Dave Martin
Cc: Nicolas Pitre, Sachin Verma, Rabin Vincent, Alexander Holler,
lkml, linux-arm-kernel, catalin.marinas
On Fri, Feb 11, 2011 at 09:31:04AM +0000, Dave Martin wrote:
> You could probably cook up a good upper bound based on the size of the
> kernel and the number of symbols in the module: i.e., assume that
> every undefined symbol in the module needs to be fixed up to point at
> the most distant symbol in the kernel.
>
> For people with normal-sized kernels, this bound will probably work
> out as zero most of the time (i.e., the current situation). For
> people with big kernels, or when many modules are already loaded, it
> may work out at 100% -- but that's the price to pay for guaranteed
> preallocation of the space required for the veneers. And anyway, you
> may really need a substantial chunk of those veneers in such cases.
I think it's going to be easier just to re-order the kernel image link
order to solve that problem. That just leaves uclinux...
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-11 9:38 ` Russell King - ARM Linux
@ 2011-02-11 9:45 ` Dave Martin
2011-02-11 13:55 ` Nicolas Pitre
0 siblings, 1 reply; 33+ messages in thread
From: Dave Martin @ 2011-02-11 9:45 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Nicolas Pitre, Sachin Verma, Rabin Vincent, Alexander Holler,
lkml, linux-arm-kernel, catalin.marinas
On Fri, Feb 11, 2011 at 9:38 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Fri, Feb 11, 2011 at 09:31:04AM +0000, Dave Martin wrote:
>> You could probably cook up a good upper bound based on the size of the
>> kernel and the number of symbols in the module: i.e., assume that
>> every undefined symbol in the module needs to be fixed up to point at
>> the most distant symbol in the kernel.
>>
>> For people with normal-sized kernels, this bound will probably work
>> out as zero most of the time (i.e., the current situation). For
>> people with big kernels, or when many modules are already loaded, it
>> may work out at 100% -- but that's the price to pay for guaranteed
>> preallocation of the space required for the veneers. And anyway, you
>> may really need a substantial chunk of those veneers in such cases.
>
> I think it's going to be easier just to re-order the kernel image link
> order to solve that problem. That just leaves uclinux...
>
But if the kernel is big, or there are many modules, changing the
order can't solve the problem surely? Or is the problem purely caused
by having the initramfs in the way, and we think the amount of actual
code will never be big enough to cause a problem?
Cheers
---Dave
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-11 9:31 ` Dave Martin
2011-02-11 9:38 ` Russell King - ARM Linux
@ 2011-02-11 13:51 ` Nicolas Pitre
2011-02-11 14:25 ` Dave Martin
1 sibling, 1 reply; 33+ messages in thread
From: Nicolas Pitre @ 2011-02-11 13:51 UTC (permalink / raw)
To: Dave Martin
Cc: Russell King - ARM Linux, Sachin Verma, Rabin Vincent,
Alexander Holler, lkml, linux-arm-kernel, catalin.marinas
[-- Attachment #1: Type: TEXT/PLAIN, Size: 4079 bytes --]
On Fri, 11 Feb 2011, Dave Martin wrote:
> On Thu, Feb 10, 2011 at 7:41 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Thu, 10 Feb 2011, Russell King - ARM Linux wrote:
> >
> >> On Thu, Jan 27, 2011 at 12:43:54AM -0500, Nicolas Pitre wrote:
> >> > The MMU-less kernel should still favor allocations close to the kernel
> >> > text for modules, and anything else away from the kernel going
> >> > downwards.
> >> >
> >> > Otherwise a veneer should be created by the module symbol resolver such
> >> > that if the branch distance to reach, say, printk is too large, then the
> >> > following code would have to be dynamically generated right next to the
> >> > module:
> >> >
> >> > ldr pc, [pc, #-4]
> >> > .word <far_away_symbol>
> >> >
> >> > Then, in your module, you patch the branch relocation for printk so that
> >> > it branches to the code above instead, and then store the address of
> >> > printk at the location represented by the .word directive.
> >>
> >> What you're suggesting is what we used to do with the old user-space
> >> module tools, which would've been nice to carry forwards to the new
> >> module code. I never found a way to do it.
> >>
> >> The problems:
> >> 1. Where do you create those veneers?
> >> 2. How many veneers do you allocate space for?
> >> 3. How do you determine that you need a veneer?
> >>
> >> While you can say "next to the module" for (1), you can only do that at
> >> the point in time when the space for the module is allocated, and you
> >> need to know at that point how much space you require.
> >
> > You would have to guess of course. Having a guess of 1/2 the module
> > size should be pretty safe. So allocating 3/2 the space in
> > module_alloc(), and then suffice to free the unused portion in
> > module_finalize().
> >
> >> For (2), you could always allocate space for one veneer per symbol present
> >> in the module, but that's very wasteful.
> >>
> >> (3) is almost impossible to know ahead of time as you don't have the
> >> relocations, realistically you have to allocate one veneer per symbol,
> >> and as you don't know whether it's a data or code symbol, you'll have
> >> to allocate one veneer for every symbol in a module.
> >
> > I don't think you may know the number of symbols in advance either
> > anyway.
>
> You could probably cook up a good upper bound based on the size of the
> kernel and the number of symbols in the module: i.e., assume that
> every undefined symbol in the module needs to be fixed up to point at
> the most distant symbol in the kernel.
Sure... It is just that the memory allocation is currently done before
the number of symbols in the module is known. Changing that would
require non trivial changes in the generic module loading code which
potentially would affect all architectures, and therefore I don't think
we want to go there.
The other solution would be to determine the number of objects in need
of a veneer in apply_relocate(), allocate a replacement area for the
module, copy everything over, and then create the veneers close to the
module. But 1) the second allocation may fail, and 2) this will change
the distance from the kernel potentially requiring more veneers than
initially determined, and 3) the generic module code might still have
pointer references into the old allocation area (didn't check but that
can be expected). That's just too messy.
> For people with normal-sized kernels, this bound will probably work
> out as zero most of the time (i.e., the current situation). For
> people with big kernels, or when many modules are already loaded, it
> may work out at 100% -- but that's the price to pay for guaranteed
> preallocation of the space required for the veneers. And anyway, you
> may really need a substantial chunk of those veneers in such cases.
I still think that compiling modules with -mlong-calls, and making that
conditional on a kernel config option so only those who need it will
have it, is the simplest solution. Nothing in the kernel code would
need to be changed.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-11 9:45 ` Dave Martin
@ 2011-02-11 13:55 ` Nicolas Pitre
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Pitre @ 2011-02-11 13:55 UTC (permalink / raw)
To: Dave Martin
Cc: Russell King - ARM Linux, Sachin Verma, Rabin Vincent,
Alexander Holler, lkml, linux-arm-kernel, catalin.marinas
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1617 bytes --]
On Fri, 11 Feb 2011, Dave Martin wrote:
> On Fri, Feb 11, 2011 at 9:38 AM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> > On Fri, Feb 11, 2011 at 09:31:04AM +0000, Dave Martin wrote:
> >> You could probably cook up a good upper bound based on the size of the
> >> kernel and the number of symbols in the module: i.e., assume that
> >> every undefined symbol in the module needs to be fixed up to point at
> >> the most distant symbol in the kernel.
> >>
> >> For people with normal-sized kernels, this bound will probably work
> >> out as zero most of the time (i.e., the current situation). For
> >> people with big kernels, or when many modules are already loaded, it
> >> may work out at 100% -- but that's the price to pay for guaranteed
> >> preallocation of the space required for the veneers. And anyway, you
> >> may really need a substantial chunk of those veneers in such cases.
> >
> > I think it's going to be easier just to re-order the kernel image link
> > order to solve that problem. That just leaves uclinux...
> >
>
> But if the kernel is big, or there are many modules, changing the
> order can't solve the problem surely? Or is the problem purely caused
> by having the initramfs in the way, and we think the amount of actual
> code will never be big enough to cause a problem?
Right now the biggest problem comes from the size of the initramfs. The
solution for that is simply to move it elsewhere towards the end of the
kernel image instead of at the beginning. And I think we're good for a
couple years before the actual kernel code size becomes an issue.
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-11 13:51 ` Nicolas Pitre
@ 2011-02-11 14:25 ` Dave Martin
2011-02-11 14:42 ` Russell King - ARM Linux
0 siblings, 1 reply; 33+ messages in thread
From: Dave Martin @ 2011-02-11 14:25 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Russell King - ARM Linux, Sachin Verma, Rabin Vincent,
Alexander Holler, lkml, linux-arm-kernel, catalin.marinas
On Fri, Feb 11, 2011 at 1:51 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Fri, 11 Feb 2011, Dave Martin wrote:
>
>> On Thu, Feb 10, 2011 at 7:41 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Thu, 10 Feb 2011, Russell King - ARM Linux wrote:
>> >
>> >> On Thu, Jan 27, 2011 at 12:43:54AM -0500, Nicolas Pitre wrote:
>> >> > The MMU-less kernel should still favor allocations close to the kernel
>> >> > text for modules, and anything else away from the kernel going
>> >> > downwards.
>> >> >
>> >> > Otherwise a veneer should be created by the module symbol resolver such
>> >> > that if the branch distance to reach, say, printk is too large, then the
>> >> > following code would have to be dynamically generated right next to the
>> >> > module:
>> >> >
>> >> > ldr pc, [pc, #-4]
>> >> > .word <far_away_symbol>
>> >> >
>> >> > Then, in your module, you patch the branch relocation for printk so that
>> >> > it branches to the code above instead, and then store the address of
>> >> > printk at the location represented by the .word directive.
>> >>
>> >> What you're suggesting is what we used to do with the old user-space
>> >> module tools, which would've been nice to carry forwards to the new
>> >> module code. I never found a way to do it.
>> >>
>> >> The problems:
>> >> 1. Where do you create those veneers?
>> >> 2. How many veneers do you allocate space for?
>> >> 3. How do you determine that you need a veneer?
>> >>
>> >> While you can say "next to the module" for (1), you can only do that at
>> >> the point in time when the space for the module is allocated, and you
>> >> need to know at that point how much space you require.
>> >
>> > You would have to guess of course. Having a guess of 1/2 the module
>> > size should be pretty safe. So allocating 3/2 the space in
>> > module_alloc(), and then suffice to free the unused portion in
>> > module_finalize().
>> >
>> >> For (2), you could always allocate space for one veneer per symbol present
>> >> in the module, but that's very wasteful.
>> >>
>> >> (3) is almost impossible to know ahead of time as you don't have the
>> >> relocations, realistically you have to allocate one veneer per symbol,
>> >> and as you don't know whether it's a data or code symbol, you'll have
>> >> to allocate one veneer for every symbol in a module.
>> >
>> > I don't think you may know the number of symbols in advance either
>> > anyway.
>>
>> You could probably cook up a good upper bound based on the size of the
>> kernel and the number of symbols in the module: i.e., assume that
>> every undefined symbol in the module needs to be fixed up to point at
>> the most distant symbol in the kernel.
>
> Sure... It is just that the memory allocation is currently done before
> the number of symbols in the module is known. Changing that would
> require non trivial changes in the generic module loading code which
> potentially would affect all architectures, and therefore I don't think
> we want to go there.
>
> The other solution would be to determine the number of objects in need
> of a veneer in apply_relocate(), allocate a replacement area for the
> module, copy everything over, and then create the veneers close to the
> module. But 1) the second allocation may fail, and 2) this will change
> the distance from the kernel potentially requiring more veneers than
> initially determined, and 3) the generic module code might still have
> pointer references into the old allocation area (didn't check but that
> can be expected). That's just too messy.
>
>> For people with normal-sized kernels, this bound will probably work
>> out as zero most of the time (i.e., the current situation). For
>> people with big kernels, or when many modules are already loaded, it
>> may work out at 100% -- but that's the price to pay for guaranteed
>> preallocation of the space required for the veneers. And anyway, you
>> may really need a substantial chunk of those veneers in such cases.
>
> I still think that compiling modules with -mlong-calls, and making that
> conditional on a kernel config option so only those who need it will
> have it, is the simplest solution. Nothing in the kernel code would
> need to be changed.
Yes, this sounds like a reasonable solution for now ... since most
people don't need this turned on anyway, and those that do need it
know who they are.
if we can also move the initramfs somewhere less obstructive in the
future, then so much the better.
Cheers
---Dave
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-11 14:25 ` Dave Martin
@ 2011-02-11 14:42 ` Russell King - ARM Linux
2011-02-12 19:52 ` Nicolas Pitre
0 siblings, 1 reply; 33+ messages in thread
From: Russell King - ARM Linux @ 2011-02-11 14:42 UTC (permalink / raw)
To: Dave Martin
Cc: Nicolas Pitre, Sachin Verma, Rabin Vincent, Alexander Holler,
lkml, linux-arm-kernel, catalin.marinas
On Fri, Feb 11, 2011 at 02:25:28PM +0000, Dave Martin wrote:
> if we can also move the initramfs somewhere less obstructive in the
> future, then so much the better.
This may do it, and it might even work. Not particularly well tested
yet. It'd be loads easier if we didn't have to worry about XIP.
arch/arm/kernel/etm.c | 2 +-
arch/arm/kernel/smp.c | 4 +-
arch/arm/kernel/vmlinux.lds.S | 133 +++++++++++++++++++++-------------------
arch/arm/mm/init.c | 2 +-
4 files changed, 74 insertions(+), 67 deletions(-)
diff --git a/arch/arm/kernel/etm.c b/arch/arm/kernel/etm.c
index 11db628..ca477d2 100644
--- a/arch/arm/kernel/etm.c
+++ b/arch/arm/kernel/etm.c
@@ -114,7 +114,7 @@ static int trace_start(struct tracectx *t)
return -EFAULT;
}
- etm_setup_address_range(t, 1, (unsigned long)_stext,
+ etm_setup_address_range(t, 1, (unsigned long)_text,
(unsigned long)_etext, 0, 0);
etm_writel(t, 0, ETMR_TRACEENCTRL2);
etm_writel(t, 0, ETMR_TRACESSCTRL);
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 4539ebc..8ad77d3 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -95,7 +95,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
#ifndef CONFIG_HOTPLUG_CPU
identity_mapping_add(pgd, __pa(__init_begin), __pa(__init_end));
#endif
- identity_mapping_add(pgd, __pa(_stext), __pa(_etext));
+ identity_mapping_add(pgd, __pa(_text), __pa(_etext));
identity_mapping_add(pgd, __pa(_sdata), __pa(_edata));
}
@@ -143,7 +143,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
#ifndef CONFIG_HOTPLUG_CPU
identity_mapping_del(pgd, __pa(__init_begin), __pa(__init_end));
#endif
- identity_mapping_del(pgd, __pa(_stext), __pa(_etext));
+ identity_mapping_del(pgd, __pa(_text), __pa(_etext));
identity_mapping_del(pgd, __pa(_sdata), __pa(_edata));
}
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index 45b5651..9cd098b 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -32,46 +32,98 @@ jiffies = jiffies_64 + 4;
SECTIONS
{
+ /*
+ * unwind exit sections must be discarded before the rest of the
+ * unwind sections get included.
+ */
+ /DISCARD/ : {
+ *(.ARM.exidx.exit.text)
+ *(.ARM.extab.exit.text)
+ ARM_CPU_DISCARD(*(.ARM.exidx.cpuexit.text))
+ ARM_CPU_DISCARD(*(.ARM.extab.cpuexit.text))
+#ifndef CONFIG_HOTPLUG
+ *(.ARM.exidx.devexit.text)
+ *(.ARM.extab.devexit.text)
+#endif
+#ifndef CONFIG_MMU
+ *(.fixup)
+ *(__ex_table)
+#endif
+#ifndef CONFIG_SMP_ON_UP
+ *(.alt.smp.init)
+#endif
+ }
+
#ifdef CONFIG_XIP_KERNEL
. = XIP_VIRT_ADDR(CONFIG_XIP_PHYS_ADDR);
#else
. = PAGE_OFFSET + TEXT_OFFSET;
#endif
- .init : { /* Init code and data */
- _stext = .;
- _sinittext = .;
- HEAD_TEXT
- INIT_TEXT
- _einittext = .;
+ _text = .;
+ HEAD_TEXT_SECTION
+
+ .text : { /* Real text segment */
+ _stext = .; /* Text and read-only data */
+ __exception_text_start = .;
+ *(.exception.text)
+ __exception_text_end = .;
+ IRQENTRY_TEXT
+ TEXT_TEXT
+ SCHED_TEXT
+ LOCK_TEXT
+ KPROBES_TEXT
+#ifdef CONFIG_MMU
+ *(.fixup)
+#endif
+ *(.gnu.warning)
+ *(.glue_7)
+ *(.glue_7t)
+ . = ALIGN(4);
+ *(.got) /* Global offset table */
+ ARM_CPU_KEEP(PROC_INFO)
+ _etext = .; /* End of text and rodata section */
+ }
+
+#ifndef CONFIG_XIP_KERNEL
+ . = ALIGN(PAGE_SIZE);
+ __init_begin = .;
+#endif
+ INIT_TEXT_SECTION(0)
+ .init.proc.info : {
ARM_CPU_DISCARD(PROC_INFO)
+ }
+ .init.arch.info : {
__arch_info_begin = .;
*(.arch.info.init)
__arch_info_end = .;
+ }
+ .init.tagtable : {
__tagtable_begin = .;
*(.taglist.init)
__tagtable_end = .;
+ }
#ifdef CONFIG_SMP_ON_UP
+ .init.smpalt : {
__smpalt_begin = .;
*(.alt.smp.init)
__smpalt_end = .;
+ }
#endif
-
+ .init.pv_table : {
__pv_table_begin = .;
*(.pv_table)
__pv_table_end = .;
-
+ }
+ .init.data : {
+#ifndef CONFIG_XIP_KERNEL
+ INIT_DATA
+#endif
INIT_SETUP(16)
-
INIT_CALLS
CON_INITCALL
SECURITY_INITCALL
INIT_RAM_FS
-
-#ifndef CONFIG_XIP_KERNEL
- __init_begin = _stext;
- INIT_DATA
-#endif
}
PERCPU(PAGE_SIZE)
@@ -79,49 +131,10 @@ SECTIONS
#ifndef CONFIG_XIP_KERNEL
. = ALIGN(PAGE_SIZE);
__init_end = .;
-#endif
- /*
- * unwind exit sections must be discarded before the rest of the
- * unwind sections get included.
- */
- /DISCARD/ : {
- *(.ARM.exidx.exit.text)
- *(.ARM.extab.exit.text)
- ARM_CPU_DISCARD(*(.ARM.exidx.cpuexit.text))
- ARM_CPU_DISCARD(*(.ARM.extab.cpuexit.text))
-#ifndef CONFIG_HOTPLUG
- *(.ARM.exidx.devexit.text)
- *(.ARM.extab.devexit.text)
+ _data = .;
+ _sdata = .;
#endif
-#ifndef CONFIG_MMU
- *(.fixup)
- *(__ex_table)
-#endif
- }
-
- .text : { /* Real text segment */
- _text = .; /* Text and read-only data */
- __exception_text_start = .;
- *(.exception.text)
- __exception_text_end = .;
- IRQENTRY_TEXT
- TEXT_TEXT
- SCHED_TEXT
- LOCK_TEXT
- KPROBES_TEXT
-#ifdef CONFIG_MMU
- *(.fixup)
-#endif
- *(.gnu.warning)
- *(.rodata)
- *(.rodata.*)
- *(.glue_7)
- *(.glue_7t)
- . = ALIGN(4);
- *(.got) /* Global offset table */
- ARM_CPU_KEEP(PROC_INFO)
- }
RO_DATA(PAGE_SIZE)
@@ -142,8 +155,6 @@ SECTIONS
}
#endif
- _etext = .; /* End of text and rodata section */
-
#ifdef CONFIG_XIP_KERNEL
__data_loc = ALIGN(4); /* location in binary */
. = PAGE_OFFSET + TEXT_OFFSET;
@@ -153,8 +164,10 @@ SECTIONS
#endif
.data : AT(__data_loc) {
+#ifdef CONFIG_XIP_KERNEL
_data = .; /* address in memory */
_sdata = .;
+#endif
/*
* first, the init task union, aligned
@@ -259,12 +272,6 @@ SECTIONS
/* Default discards */
DISCARDS
-
-#ifndef CONFIG_SMP_ON_UP
- /DISCARD/ : {
- *(.alt.smp.init)
- }
-#endif
}
/*
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index cddd684..9e3c3c3 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -294,7 +294,7 @@ void __init arm_memblock_init(struct meminfo *mi, struct machine_desc *mdesc)
#ifdef CONFIG_XIP_KERNEL
memblock_reserve(__pa(_sdata), _end - _sdata);
#else
- memblock_reserve(__pa(_stext), _end - _stext);
+ memblock_reserve(__pa(_text), _end - _text);
#endif
#ifdef CONFIG_BLK_DEV_INITRD
if (phys_initrd_size &&
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: ARM: relocation out of range (when loading a module)
2011-02-11 14:42 ` Russell King - ARM Linux
@ 2011-02-12 19:52 ` Nicolas Pitre
0 siblings, 0 replies; 33+ messages in thread
From: Nicolas Pitre @ 2011-02-12 19:52 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: Dave Martin, Sachin Verma, Rabin Vincent, Alexander Holler, lkml,
linux-arm-kernel, Catalin Marinas
On Fri, 11 Feb 2011, Russell King - ARM Linux wrote:
> On Fri, Feb 11, 2011 at 02:25:28PM +0000, Dave Martin wrote:
> > if we can also move the initramfs somewhere less obstructive in the
> > future, then so much the better.
>
> This may do it, and it might even work. Not particularly well tested
> yet. It'd be loads easier if we didn't have to worry about XIP.
Is anyone actually still using XIP these days? Now that NOR flash is
not that much used anymore, if maintaining XIP support is more trouble
than it is worth we could simply deprecate it. The linker script has
become quite more hairy than it used to be, and from a quick look I'm
not even sure if all the stuff in there is properly located for XIP.
Otherwise, for the __init stuff move:
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Nicolas
^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2011-02-12 19:52 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-10 18:16 ARM: relocation out of range (when loading a module) Alexander Holler
2011-01-10 22:26 ` Nicolas Pitre
2011-01-11 6:34 ` Alexander Holler
2011-01-11 15:17 ` Alexander Holler
2011-01-11 15:57 ` Russell King - ARM Linux
2011-01-11 16:02 ` Nicolas Pitre
2011-01-11 15:46 ` Rabin Vincent
2011-01-11 15:59 ` Russell King - ARM Linux
2011-01-12 3:00 ` Alexander Holler
2011-01-12 16:05 ` Dave Martin
2011-01-12 16:23 ` Russell King - ARM Linux
2011-01-12 18:28 ` Nicolas Pitre
2011-01-12 18:42 ` Russell King - ARM Linux
2011-01-12 18:55 ` Nicolas Pitre
2011-01-25 8:44 ` Sachin Verma
2011-01-25 13:23 ` Catalin Marinas
2011-01-27 5:43 ` Nicolas Pitre
2011-02-10 15:43 ` Russell King - ARM Linux
2011-02-10 19:41 ` Nicolas Pitre
2011-02-11 9:31 ` Dave Martin
2011-02-11 9:38 ` Russell King - ARM Linux
2011-02-11 9:45 ` Dave Martin
2011-02-11 13:55 ` Nicolas Pitre
2011-02-11 13:51 ` Nicolas Pitre
2011-02-11 14:25 ` Dave Martin
2011-02-11 14:42 ` Russell King - ARM Linux
2011-02-12 19:52 ` Nicolas Pitre
2011-01-13 5:50 ` Alexander Holler
2011-01-13 10:04 ` Russell King - ARM Linux
2011-01-13 14:36 ` Alexander Holler
2011-01-12 16:25 ` Matthieu CASTET
2011-01-12 16:38 ` Russell King - ARM Linux
2011-01-11 16:01 ` Nicolas Pitre
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).