[RFC PATCH] ARM: add fixmap based earlycon support

[RFC PATCH] ARM: add fixmap based earlycon support

[Ard Biesheuvel]

This adds support for fixmap based earlycon by moving the fixmap
initialization to an earlier stage and introducing the Kconfig symbol
that wires up the existing code.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/Kconfig              |  3 +++
 arch/arm/include/asm/fixmap.h |  6 ++++++
 arch/arm/kernel/setup.c       |  2 ++
 arch/arm/mm/mmu.c             | 16 +++++++++++++---
 4 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 45df48ba0b12..0076330789f5 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -287,6 +287,9 @@ config GENERIC_BUG
 	def_bool y
 	depends on BUG
 
+config FIX_EARLYCON_MEM
+	def_bool y
+
 config PGTABLE_LEVELS
 	int
 	default 3 if ARM_LPAE
diff --git a/arch/arm/include/asm/fixmap.h b/arch/arm/include/asm/fixmap.h
index 0415eae1df27..e0abd3a150ee 100644
--- a/arch/arm/include/asm/fixmap.h
+++ b/arch/arm/include/asm/fixmap.h
@@ -1,6 +1,10 @@
 #ifndef _ASM_FIXMAP_H
 #define _ASM_FIXMAP_H
 
+#include <asm/pgtable.h>
+
+#define FIXMAP_PAGE_IO		L_PTE_PRESENT|L_PTE_YOUNG|L_PTE_DIRTY
+
 #define FIXADDR_START		0xffc00000UL
 #define FIXADDR_END		0xfff00000UL
 #define FIXADDR_TOP		(FIXADDR_END - PAGE_SIZE)
@@ -11,6 +15,8 @@ enum fixed_addresses {
 	FIX_KMAP_BEGIN,
 	FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR * NR_CPUS) - 1,
 
+	FIX_EARLYCON_MEM_BASE,
+
 	/* Support writing RO kernel text via kprobes, jump labels, etc. */
 	FIX_TEXT_POKE0,
 	FIX_TEXT_POKE1,
diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 6c777e908a24..2e4b3c22be4d 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -80,6 +80,7 @@ extern void early_paging_init(const struct machine_desc *,
 extern void sanity_check_meminfo(void);
 extern enum reboot_mode reboot_mode;
 extern void setup_dma_zone(const struct machine_desc *desc);
+extern void early_fixmap_init(void);
 
 unsigned int processor_id;
 EXPORT_SYMBOL(processor_id);
@@ -915,6 +916,7 @@ void __init setup_arch(char **cmdline_p)
 	const struct machine_desc *mdesc;
 
 	setup_processor();
+	early_fixmap_init();
 	mdesc = setup_machine_fdt(__atags_pointer);
 	if (!mdesc)
 		mdesc = setup_machine_tags(__atags_pointer, __machine_arch_type);
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index 4e6ef896c619..d072726c3781 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1231,7 +1231,9 @@ static void __init devicemaps_init(const struct machine_desc *mdesc)
 
 	early_trap_init(vectors);
 
-	for (addr = VMALLOC_START; addr; addr += PMD_SIZE)
+	for (addr = VMALLOC_START; addr < FIXADDR_START; addr += PMD_SIZE)
+		pmd_clear(pmd_off_k(addr));
+	for (addr = round_up(FIXADDR_END, PMD_SIZE); addr; addr += PMD_SIZE)
 		pmd_clear(pmd_off_k(addr));
 
 	/*
@@ -1321,9 +1323,17 @@ static void __init kmap_init(void)
 	pkmap_page_table = early_pte_alloc(pmd_off_k(PKMAP_BASE),
 		PKMAP_BASE, _PAGE_KERNEL_TABLE);
 #endif
+}
+
+void __init early_fixmap_init(void)
+{
+	static u8 fixmap_pte[PTE_HWTABLE_OFF + PTE_HWTABLE_SIZE] __page_aligned_bss;
+	pmd_t *pmd = pmd_off_k(FIXADDR_START);
+
+	if (pmd_none(*pmd))
+		__pmd_populate(pmd, __pa(fixmap_pte), _PAGE_KERNEL_TABLE);
 
-	early_pte_alloc(pmd_off_k(FIXADDR_START), FIXADDR_START,
-			_PAGE_KERNEL_TABLE);
+	BUG_ON(pmd_bad(*pmd));
 }
 
 static void __init map_lowmem(void)
-- 
1.9.1

[RFC PATCH] ARM: add fixmap based earlycon support

[Russell King - ARM Linux]

On Wed, Apr 29, 2015 at 03:51:48PM +0200, Ard Biesheuvel wrote:
> This adds support for fixmap based earlycon by moving the fixmap
> initialization to an earlier stage and introducing the Kconfig symbol
> that wires up the existing code.

I don't see the patches for this fixmap based earlycon.  What's the
status of those, and how are they getting into mainline?

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

[RFC PATCH] ARM: add fixmap based earlycon support

[Ard Biesheuvel]

On 1 May 2015 at 18:18, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
> On Wed, Apr 29, 2015 at 03:51:48PM +0200, Ard Biesheuvel wrote:
>> This adds support for fixmap based earlycon by moving the fixmap
>> initialization to an earlier stage and introducing the Kconfig symbol
>> that wires up the existing code.
>
> I don't see the patches for this fixmap based earlycon.  What's the
> status of those, and how are they getting into mainline?
>

They are already /in/ mainline. This single patch is all that is
required to wire it up for ARM.

However, I did notice after sending this that another implementation
had been proposed already:
http://lkml.iu.edu/hypermail/linux/kernel/1412.3/00855.html

I am not sure which one is more correct, but the general idea is the
same, i.e., to allocate the fixmap PTEs from bss rather than the heap.

[RFC PATCH] ARM: add fixmap based earlycon support

[Ard Biesheuvel]

(thread is here: http://thread.gmane.org/gmane.linux.ports.arm.kernel/409329)

On 2 May 2015 at 10:37, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 1 May 2015 at 18:18, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
>> On Wed, Apr 29, 2015 at 03:51:48PM +0200, Ard Biesheuvel wrote:
>>> This adds support for fixmap based earlycon by moving the fixmap
>>> initialization to an earlier stage and introducing the Kconfig symbol
>>> that wires up the existing code.
>>
>> I don't see the patches for this fixmap based earlycon.  What's the
>> status of those, and how are they getting into mainline?
>>
>
> They are already /in/ mainline. This single patch is all that is
> required to wire it up for ARM.
>
> However, I did notice after sending this that another implementation
> had been proposed already:
> http://lkml.iu.edu/hypermail/linux/kernel/1412.3/00855.html
>
> I am not sure which one is more correct, but the general idea is the
> same, i.e., to allocate the fixmap PTEs from bss rather than the heap.

Anyone else care to comment on this?

DT based earlycon is a *really* useful thing to have if you are
working on early kernel code.
With many of the DTs now having been extended with a
/chosen/stdout-path property, it really only takes a simple 'earlycon'
command line parameter (without any arguments) to get console output
as soon as the early DT scan is concluded. I suppose this is also a
useful thing for multi platform kernels?

BTW the patch still applies to v4.1-rc6

Regards,
Ard.

[RFC PATCH] ARM: add fixmap based earlycon support

[Stefan Agner]

On 2015-06-01 10:26, Ard Biesheuvel wrote:
> (thread is here: http://thread.gmane.org/gmane.linux.ports.arm.kernel/409329)
> 
> On 2 May 2015 at 10:37, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
<snip>
>> However, I did notice after sending this that another implementation
>> had been proposed already:
>> http://lkml.iu.edu/hypermail/linux/kernel/1412.3/00855.html
>>
>> I am not sure which one is more correct, but the general idea is the
>> same, i.e., to allocate the fixmap PTEs from bss rather than the heap.
> 
> Anyone else care to comment on this?
> 
> DT based earlycon is a *really* useful thing to have if you are
> working on early kernel code.
> With many of the DTs now having been extended with a
> /chosen/stdout-path property, it really only takes a simple 'earlycon'
> command line parameter (without any arguments) to get console output
> as soon as the early DT scan is concluded. I suppose this is also a
> useful thing for multi platform kernels?
> 
> BTW the patch still applies to v4.1-rc6

I would appreciate earlycon support too. Serial stuff happen to be
initialized rather late on the HW I work often with, hence I quite often
have to enable/disable earlyprintk at the moment.

The patch I sent is missing a tlb flush, I can send a v2 if that
approach is preferred. On the other hand, your approach seems to be
simpler, judging from the number of changed lines...

--
Stefan

[RFC PATCH] ARM: add fixmap based earlycon support

[Rob Herring]

On Mon, Jun 1, 2015 at 3:26 AM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> (thread is here: http://thread.gmane.org/gmane.linux.ports.arm.kernel/409329)
>
> On 2 May 2015 at 10:37, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>> On 1 May 2015 at 18:18, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
>>> On Wed, Apr 29, 2015 at 03:51:48PM +0200, Ard Biesheuvel wrote:
>>>> This adds support for fixmap based earlycon by moving the fixmap
>>>> initialization to an earlier stage and introducing the Kconfig symbol
>>>> that wires up the existing code.
>>>
>>> I don't see the patches for this fixmap based earlycon.  What's the
>>> status of those, and how are they getting into mainline?
>>>
>>
>> They are already /in/ mainline. This single patch is all that is
>> required to wire it up for ARM.
>>
>> However, I did notice after sending this that another implementation
>> had been proposed already:
>> http://lkml.iu.edu/hypermail/linux/kernel/1412.3/00855.html
>>
>> I am not sure which one is more correct, but the general idea is the
>> same, i.e., to allocate the fixmap PTEs from bss rather than the heap.
>
> Anyone else care to comment on this?

You need to re-write the page table entries because the pte and pmd
flags get adjusted during boot. For example SMP related flags get
configured. I believe Stefan's version (based on mine which was based
on Mark Salter's) did this. In fact, I'm surprised this works after
paging_init, but I may have missed something.

> DT based earlycon is a *really* useful thing to have if you are
> working on early kernel code.
> With many of the DTs now having been extended with a
> /chosen/stdout-path property, it really only takes a simple 'earlycon'
> command line parameter (without any arguments) to get console output
> as soon as the early DT scan is concluded. I suppose this is also a
> useful thing for multi platform kernels?

Certainly, that's the whole point. IMO, we should rip out most of
DEBUG_LL when this is merged.

Rob

[RFC PATCH] ARM: add fixmap based earlycon support

[Russell King - ARM Linux]

On Mon, Jun 01, 2015 at 06:26:14PM -0500, Rob Herring wrote:
> On Mon, Jun 1, 2015 at 3:26 AM, Ard Biesheuvel
> <ard.biesheuvel@linaro.org> wrote:
> > (thread is here: http://thread.gmane.org/gmane.linux.ports.arm.kernel/409329)
> >
> > On 2 May 2015 at 10:37, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >> On 1 May 2015 at 18:18, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
> >>> On Wed, Apr 29, 2015 at 03:51:48PM +0200, Ard Biesheuvel wrote:
> >>>> This adds support for fixmap based earlycon by moving the fixmap
> >>>> initialization to an earlier stage and introducing the Kconfig symbol
> >>>> that wires up the existing code.
> >>>
> >>> I don't see the patches for this fixmap based earlycon.  What's the
> >>> status of those, and how are they getting into mainline?
> >>>
> >>
> >> They are already /in/ mainline. This single patch is all that is
> >> required to wire it up for ARM.
> >>
> >> However, I did notice after sending this that another implementation
> >> had been proposed already:
> >> http://lkml.iu.edu/hypermail/linux/kernel/1412.3/00855.html
> >>
> >> I am not sure which one is more correct, but the general idea is the
> >> same, i.e., to allocate the fixmap PTEs from bss rather than the heap.
> >
> > Anyone else care to comment on this?
> 
> You need to re-write the page table entries because the pte and pmd
> flags get adjusted during boot. For example SMP related flags get
> configured. I believe Stefan's version (based on mine which was based
> on Mark Salter's) did this. In fact, I'm surprised this works after
> paging_init, but I may have missed something.

Any changes to PMD flags for things like SMP related flags must be
done via a remove-the-mapping, flush-tlb, add-new-mapping sequence.

> > DT based earlycon is a *really* useful thing to have if you are
> > working on early kernel code.
> > With many of the DTs now having been extended with a
> > /chosen/stdout-path property, it really only takes a simple 'earlycon'
> > command line parameter (without any arguments) to get console output
> > as soon as the early DT scan is concluded. I suppose this is also a
> > useful thing for multi platform kernels?
> 
> Certainly, that's the whole point. IMO, we should rip out most of
> DEBUG_LL when this is merged.

If you think that, you're wrong.  DEBUG_LL exists to be able to debug
stuff well before the kernel gets anywhere near the C code.  It has
to stay.  It fills the gap where no C code can do any debugging.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

[RFC PATCH] ARM: add fixmap based earlycon support

[Rob Herring]

On Mon, Jun 1, 2015 at 6:31 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Mon, Jun 01, 2015 at 06:26:14PM -0500, Rob Herring wrote:
>> On Mon, Jun 1, 2015 at 3:26 AM, Ard Biesheuvel
>> <ard.biesheuvel@linaro.org> wrote:
>> > (thread is here: http://thread.gmane.org/gmane.linux.ports.arm.kernel/409329)
>> >
>> > On 2 May 2015 at 10:37, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>> >> On 1 May 2015 at 18:18, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
>> >>> On Wed, Apr 29, 2015 at 03:51:48PM +0200, Ard Biesheuvel wrote:
>> >>>> This adds support for fixmap based earlycon by moving the fixmap
>> >>>> initialization to an earlier stage and introducing the Kconfig symbol
>> >>>> that wires up the existing code.
>> >>>
>> >>> I don't see the patches for this fixmap based earlycon.  What's the
>> >>> status of those, and how are they getting into mainline?
>> >>>
>> >>
>> >> They are already /in/ mainline. This single patch is all that is
>> >> required to wire it up for ARM.
>> >>
>> >> However, I did notice after sending this that another implementation
>> >> had been proposed already:
>> >> http://lkml.iu.edu/hypermail/linux/kernel/1412.3/00855.html
>> >>
>> >> I am not sure which one is more correct, but the general idea is the
>> >> same, i.e., to allocate the fixmap PTEs from bss rather than the heap.
>> >
>> > Anyone else care to comment on this?
>>
>> You need to re-write the page table entries because the pte and pmd
>> flags get adjusted during boot. For example SMP related flags get
>> configured. I believe Stefan's version (based on mine which was based
>> on Mark Salter's) did this. In fact, I'm surprised this works after
>> paging_init, but I may have missed something.
>
> Any changes to PMD flags for things like SMP related flags must be
> done via a remove-the-mapping, flush-tlb, add-new-mapping sequence.

Right.

>> > DT based earlycon is a *really* useful thing to have if you are
>> > working on early kernel code.
>> > With many of the DTs now having been extended with a
>> > /chosen/stdout-path property, it really only takes a simple 'earlycon'
>> > command line parameter (without any arguments) to get console output
>> > as soon as the early DT scan is concluded. I suppose this is also a
>> > useful thing for multi platform kernels?
>>
>> Certainly, that's the whole point. IMO, we should rip out most of
>> DEBUG_LL when this is merged.
>
> If you think that, you're wrong.  DEBUG_LL exists to be able to debug
> stuff well before the kernel gets anywhere near the C code.  It has
> to stay.  It fills the gap where no C code can do any debugging.

Right, that is the part we should keep. However, we won't really need
earlyprintk (which I recall you are no fan of) and we don't need all
the kconfig of addresses which is getting quite unwieldy.

Rob

[RFC PATCH] ARM: add fixmap based earlycon support

[Russell King - ARM Linux]

On Mon, Jun 01, 2015 at 06:57:24PM -0500, Rob Herring wrote:
> On Mon, Jun 1, 2015 at 6:31 PM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> >> > DT based earlycon is a *really* useful thing to have if you are
> >> > working on early kernel code.
> >> > With many of the DTs now having been extended with a
> >> > /chosen/stdout-path property, it really only takes a simple 'earlycon'
> >> > command line parameter (without any arguments) to get console output
> >> > as soon as the early DT scan is concluded. I suppose this is also a
> >> > useful thing for multi platform kernels?
> >>
> >> Certainly, that's the whole point. IMO, we should rip out most of
> >> DEBUG_LL when this is merged.
> >
> > If you think that, you're wrong.  DEBUG_LL exists to be able to debug
> > stuff well before the kernel gets anywhere near the C code.  It has
> > to stay.  It fills the gap where no C code can do any debugging.
> 
> Right, that is the part we should keep. However, we won't really need
> earlyprintk (which I recall you are no fan of) and we don't need all
> the kconfig of addresses which is getting quite unwieldy.

Both points are true, but we definitely want to keep the information
concerning where the uarts are.  I certainly don't want to go hunting
in old paper documentation for some of that information... maybe
moving it into Documentation/arm somewhere would make sense -
possibly in a set of per-vendor files or maybe a single file separated
by vendor or something like that to break it up from being one huge
conflict-ridden list.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

[RFC PATCH] ARM: add fixmap based earlycon support

[Ard Biesheuvel]

On 2 June 2015 at 01:31, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Mon, Jun 01, 2015 at 06:26:14PM -0500, Rob Herring wrote:
>> On Mon, Jun 1, 2015 at 3:26 AM, Ard Biesheuvel
>> <ard.biesheuvel@linaro.org> wrote:
>> > (thread is here: http://thread.gmane.org/gmane.linux.ports.arm.kernel/409329)
>> >
>> > On 2 May 2015 at 10:37, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>> >> On 1 May 2015 at 18:18, Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
>> >>> On Wed, Apr 29, 2015 at 03:51:48PM +0200, Ard Biesheuvel wrote:
>> >>>> This adds support for fixmap based earlycon by moving the fixmap
>> >>>> initialization to an earlier stage and introducing the Kconfig symbol
>> >>>> that wires up the existing code.
>> >>>
>> >>> I don't see the patches for this fixmap based earlycon.  What's the
>> >>> status of those, and how are they getting into mainline?
>> >>>
>> >>
>> >> They are already /in/ mainline. This single patch is all that is
>> >> required to wire it up for ARM.
>> >>
>> >> However, I did notice after sending this that another implementation
>> >> had been proposed already:
>> >> http://lkml.iu.edu/hypermail/linux/kernel/1412.3/00855.html
>> >>
>> >> I am not sure which one is more correct, but the general idea is the
>> >> same, i.e., to allocate the fixmap PTEs from bss rather than the heap.
>> >
>> > Anyone else care to comment on this?
>>
>> You need to re-write the page table entries because the pte and pmd
>> flags get adjusted during boot. For example SMP related flags get
>> configured. I believe Stefan's version (based on mine which was based
>> on Mark Salter's) did this. In fact, I'm surprised this works after
>> paging_init, but I may have missed something.
>
> Any changes to PMD flags for things like SMP related flags must be
> done via a remove-the-mapping, flush-tlb, add-new-mapping sequence.
>

OK, thanks for pointing that out. At the time I sent it, I was aware
there were probably some architectural details I hadn't taken into
account, and I hadn't seen Stefan's patch.

So Stefan, could you please send that v2 so that we can get the ball
rolling on this again?

Thanks,
Ard.


>> > DT based earlycon is a *really* useful thing to have if you are
>> > working on early kernel code.
>> > With many of the DTs now having been extended with a
>> > /chosen/stdout-path property, it really only takes a simple 'earlycon'
>> > command line parameter (without any arguments) to get console output
>> > as soon as the early DT scan is concluded. I suppose this is also a
>> > useful thing for multi platform kernels?
>>
>> Certainly, that's the whole point. IMO, we should rip out most of
>> DEBUG_LL when this is merged.
>
> If you think that, you're wrong.  DEBUG_LL exists to be able to debug
> stuff well before the kernel gets anywhere near the C code.  It has
> to stay.  It fills the gap where no C code can do any debugging.
>
> --
> FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
> according to speedtest.net.