[PATCH v3] RISC-V: Fix Maximum Physical Memory 2GiB option for 64bit systems

[PATCH v3] RISC-V: Fix Maximum Physical Memory 2GiB option for 64bit systems

[Anup Patel]

The Maximum Physical Memory 2GiB option for 64bit systems is currently
broken because kernel hangs at boot-time when this option is enabled
and the underlying system has more than 2GiB memory.

This issue can be easily reproduced on SiFive Unleashed board where
we have 8GiB of memory.

This patch fixes above issue by removing unusable memory region in
setup_bootmem().

Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
Changes since v2:
- Drop brackets in if() condition added by this patch
Changes since v1:
- Use memblock_remove() instead of memblock_reserve()
---
 arch/riscv/mm/init.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 5fd8c922e1c2..bc7b77e34d09 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -121,6 +121,14 @@ void __init setup_bootmem(void)
 			 */
 			memblock_reserve(reg->base, vmlinux_end - reg->base);
 			mem_size = min(reg->size, (phys_addr_t)-PAGE_OFFSET);
+
+			/*
+			 * Remove memblock from the end of usable area to the
+			 * end of region
+			 */
+			if (reg->base + mem_size < end)
+				memblock_remove(reg->base + mem_size,
+						end - reg->base - mem_size);
 		}
 	}
 	BUG_ON(mem_size == 0);
--
2.17.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Fix Maximum Physical Memory 2GiB option for 64bit systems

[Christoph Hellwig]

On Fri, Apr 05, 2019 at 05:49:34AM +0000, Anup Patel wrote:
> The Maximum Physical Memory 2GiB option for 64bit systems is currently
> broken because kernel hangs at boot-time when this option is enabled
> and the underlying system has more than 2GiB memory.
> 
> This issue can be easily reproduced on SiFive Unleashed board where
> we have 8GiB of memory.
> 
> This patch fixes above issue by removing unusable memory region in
> setup_bootmem().
> 
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>

Btw, what is the rationale behind even offering the 2GiB option and
the medlow model on 64-bit?  Do we reall have use cases where the
slightly more effient generated code matters so much to keep up
the support burden of this mostly unused and unusual configuration?

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Fix Maximum Physical Memory 2GiB option for 64bit systems

[Anup Patel]

On Fri, Apr 5, 2019 at 11:24 AM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Fri, Apr 05, 2019 at 05:49:34AM +0000, Anup Patel wrote:
> > The Maximum Physical Memory 2GiB option for 64bit systems is currently
> > broken because kernel hangs at boot-time when this option is enabled
> > and the underlying system has more than 2GiB memory.
> >
> > This issue can be easily reproduced on SiFive Unleashed board where
> > we have 8GiB of memory.
> >
> > This patch fixes above issue by removing unusable memory region in
> > setup_bootmem().
> >
> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> > Reviewed-by: Christoph Hellwig <hch@lst.de>
>
> Btw, what is the rationale behind even offering the 2GiB option and
> the medlow model on 64-bit?  Do we reall have use cases where the
> slightly more effient generated code matters so much to keep up
> the support burden of this mostly unused and unusual configuration?

Yes, if we want to use medlow with 64bit then max physical memory
can be 2GiB. Otherwise, I don't any practical use of restricting max
physical memory to 2GiB on 64bit systems.

BTW, as-per latest revision of RISC-V priviledge spece the MMU SV32
mode will be able to access 34bit physical memory (i.e. 16GiB) so
max physical memory 2GiB will also be required for 32bit system for
medlow.

Regards,
Anup

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Fix Maximum Physical Memory 2GiB option for 64bit systems

[Palmer Dabbelt]

On Thu, 04 Apr 2019 23:01:11 PDT (-0700), anup@brainfault.org wrote:
> On Fri, Apr 5, 2019 at 11:24 AM Christoph Hellwig <hch@infradead.org> wrote:
>>
>> On Fri, Apr 05, 2019 at 05:49:34AM +0000, Anup Patel wrote:
>> > The Maximum Physical Memory 2GiB option for 64bit systems is currently
>> > broken because kernel hangs at boot-time when this option is enabled
>> > and the underlying system has more than 2GiB memory.
>> >
>> > This issue can be easily reproduced on SiFive Unleashed board where
>> > we have 8GiB of memory.
>> >
>> > This patch fixes above issue by removing unusable memory region in
>> > setup_bootmem().
>> >
>> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
>> > Reviewed-by: Christoph Hellwig <hch@lst.de>
>>
>> Btw, what is the rationale behind even offering the 2GiB option and
>> the medlow model on 64-bit?  Do we reall have use cases where the
>> slightly more effient generated code matters so much to keep up
>> the support burden of this mostly unused and unusual configuration?

I'd be OK dropping medlow on rv64.  The performance impact of medany is pretty minor 
these days, and even back when we added the option it was mostly paranoia about 
the linker being correct.  medany has been solid on rv64 for a while, but we 
did recently find a pretty major toolchain bug in medany/rv32 so if we were to 
drop that we'd need to enforce a pretty recent binutils version (the latest 
release is OK, I'd have to check on the previous one).  I'd prefer to avoid 
that, and since medany doesn't really buy us anything on rv32 it seems fine to 
leave it that way.

Given how many issues we've found here recently I doubt anyone is flipping this 
option away from the default for their base ISA.

> Yes, if we want to use medlow with 64bit then max physical memory
> can be 2GiB. Otherwise, I don't any practical use of restricting max
> physical memory to 2GiB on 64bit systems.
>
> BTW, as-per latest revision of RISC-V priviledge spece the MMU SV32
> mode will be able to access 34bit physical memory (i.e. 16GiB) so
> max physical memory 2GiB will also be required for 32bit system for
> medlow.

It's a bit more complicated that than.  The 34-bit physical addresses have been 
in the spec for a long time, but we've never bothered supporting them in Linux.  
Right now we assume we can map all physical memory into a contiguous region, 
which is why we tie PAGE_OFFSET to the maximum physical memory.  It's 
impossible to map all physical memory with 34-bit physical addresses on a 
32-bit system, so we'll need to break that assumption to move forward with 
34-bit physical addresses anyway.

Additionally: medlow can map everything on 32-bit systems anyway, so moving to 
medany doesn't buy you anything.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: Fix Maximum Physical Memory 2GiB option for 64bit systems

[Anup Patel]

On Sat, Apr 6, 2019 at 2:47 AM Palmer Dabbelt <palmer@sifive.com> wrote:
>
> On Thu, 04 Apr 2019 23:01:11 PDT (-0700), anup@brainfault.org wrote:
> > On Fri, Apr 5, 2019 at 11:24 AM Christoph Hellwig <hch@infradead.org> wrote:
> >>
> >> On Fri, Apr 05, 2019 at 05:49:34AM +0000, Anup Patel wrote:
> >> > The Maximum Physical Memory 2GiB option for 64bit systems is currently
> >> > broken because kernel hangs at boot-time when this option is enabled
> >> > and the underlying system has more than 2GiB memory.
> >> >
> >> > This issue can be easily reproduced on SiFive Unleashed board where
> >> > we have 8GiB of memory.
> >> >
> >> > This patch fixes above issue by removing unusable memory region in
> >> > setup_bootmem().
> >> >
> >> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> >> > Reviewed-by: Christoph Hellwig <hch@lst.de>
> >>
> >> Btw, what is the rationale behind even offering the 2GiB option and
> >> the medlow model on 64-bit?  Do we reall have use cases where the
> >> slightly more effient generated code matters so much to keep up
> >> the support burden of this mostly unused and unusual configuration?
>
> I'd be OK dropping medlow on rv64.  The performance impact of medany is pretty minor
> these days, and even back when we added the option it was mostly paranoia about
> the linker being correct.  medany has been solid on rv64 for a while, but we
> did recently find a pretty major toolchain bug in medany/rv32 so if we were to
> drop that we'd need to enforce a pretty recent binutils version (the latest
> release is OK, I'd have to check on the previous one).  I'd prefer to avoid
> that, and since medany doesn't really buy us anything on rv32 it seems fine to
> leave it that way.
>
> Given how many issues we've found here recently I doubt anyone is flipping this
> option away from the default for their base ISA.

Actually, this issue was reported by someone trying out Maximum Physical
Memory 2GiB option.

I agree that "Maximum Physical Memory 2GiB option" is not that useful and
does not provide much gain over cmodel medany but I would suggest that we
take this patch as a RC fix and remove "Maximum Physical Memory 2GiB
option" as separate patch because:

1. This option has been part of few kernel releases. Even if don't
recommend someone might flip it and report an issue.

2. This fix also takes care of the situation where we are using "Maximum
Physical Memory 128GiB" option (default) and underlying host has more
than 128 GiB memory (assuming a server-class RISC-V system).

>
> > Yes, if we want to use medlow with 64bit then max physical memory
> > can be 2GiB. Otherwise, I don't any practical use of restricting max
> > physical memory to 2GiB on 64bit systems.
> >
> > BTW, as-per latest revision of RISC-V priviledge spece the MMU SV32
> > mode will be able to access 34bit physical memory (i.e. 16GiB) so
> > max physical memory 2GiB will also be required for 32bit system for
> > medlow.
>
> It's a bit more complicated that than.  The 34-bit physical addresses have been
> in the spec for a long time, but we've never bothered supporting them in Linux.
> Right now we assume we can map all physical memory into a contiguous region,
> which is why we tie PAGE_OFFSET to the maximum physical memory.  It's
> impossible to map all physical memory with 34-bit physical addresses on a
> 32-bit system, so we'll need to break that assumption to move forward with
> 34-bit physical addresses anyway.
>
> Additionally: medlow can map everything on 32-bit systems anyway, so moving to
> medany doesn't buy you anything.

Currently, QEMU emulates the legacy SV32 with 32-bit physical address so
first QEMU has to be updated.

Further, I think we will need a kconfig option for new SV32 with 34bit physical
address to support existing RV32 hardware which implements SV32 with
32bit physical address.

Regards,
Anup

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv