Re: arm64: copy_from_user access the last page of ddr has problem on 4.14 kernel

Re: arm64: copy_from_user access the last page of ddr has problem on 4.14 kernel

[Will Deacon]

[I'm due to get on a long flight shortly, so I've added LAKML and a few
others to CC]

On Wed, Jan 16, 2019 at 11:25:15AM +0800, Kassey wrote:
> Hi, Will and team:
> 
>    we met a issue when copy_from_user to access the last page of DDR
> on 4.14 kenrel, below is the detail steps,
>    can you help to suggest if there is know fix or debug something ?
> 
> 1. we mmap ( in userspace)  a region of  phy address that is not
> continous  but include the last page of ddr
> 
>   for example our ddr end is 0x200000000
>   the last page is fall in below addr:
>   0x1fffff000  to  0x200000000
> 
> 2. we using copy_from_user to copy these mmap address to kernel buffer
> 
> 3. and we find everytime when trying to copy_from_user  the last page
> in phy of ddr,
>     the dst kernel buffer is looks overwrite by some same patten start
> with "mmap" in this last page ,but the src in the last page of ddr is
> still correct.
> 
> is there any know issue for copy_from_user to accces the last page of
> phy ddr mmaped by userspace  ?

Not that I'm aware of. Are you using defconfig and can you reproduce the
problem with mainline (e.g. v5.0-rc2)? What exactly do you mean by:

  "the fst kernel buffer is looks overwrite by some same patten start with
  "mmap""?

Does the copy_from_user fail (what does it return?) or does it succeed but
the data copied appears to be junk? Is it deterministic? Could you share an
example of what the corrupted data looks like, please?

Cheers,

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: arm64: copy_from_user access the last page of ddr has problem on 4.14 kernel

[Kassey]

hi, Will
      it is hard to try on v5.0-rc2 kernel, since there is much port
job to be done.
     dst kernel buffer is looks overwriten by some same(fix) patter
start with "mmap"    (0x6d6d7061)  see below code (data from vmalloc),
and file is mmaped (include the last phy page of ddr.)
     see below pattern and pieces of code.
     not sure if there is boundary issue for copy_from_user, please
help to suggest if you got some idea from the pattern, thanks.

0079c00 6d6d 7061 0000 0000 0848 fd8f 0001 0000
0079c10 0048 fd8f 0001 0000 0001 0000 0003 0000
0079c20 2000 fd83 0001 0000 1fff fd86 0001 0000
0079c30 0000 0000 0000 0000 700f 0000 0000 0000
0079c40 6d6d 7061 0000 0000 f448 ffff 0001 0000
0079c50 f748 ffff 0001 0000 0001 0000 0004 0000
0079c60 3000 fd8c 0001 0000 2fff fd8e 0001 0000
0079c70 0000 0000 0000 0000 700f 0000 0000 0000
0079c80 c103 0606 0100 be00 1009 3b00 3b07 0607
0079c90 0100 5700 1006 e800 8c03 3103 0100 0a00
0079ca0 0000 cf00 bf08 0a00 0100 5700 1006 3700
0079cb0 3906 0606 0100 1600 0004 4700 9902 0207
0079cc0 6d6d 7061 0000 0000 f808 ffff 0001 0000
0079cd0 f1c8 ffff 0001 0000 0001 0000 0005 0000
0079ce0 d000 fff8 0001 0000 efff fffa 0001 0000
0079cf0 0000 0000 0000 0000 700f 0000 0000 0000
0079d00 6d6d 7061 0000 0000 f1c8 ffff 0001 0000
0079d10 f388 ffff 0001 0000 0001 0000 0003 0000
0079d20 c000 ffdf 0001 0000 6fff fff8 0001 0000
0079d30 0000 0000 0000 0000 700f 0000 0000 0000
0079d40 9407 0901 0100 5300 0204 b400 d503 0a04
0079d50 0100 0000 0001 0200 7309 0202 0100 0200
0079d60 5000 0200 7309 0202 0400 ff00 f7ff 94ff
0079d70 b400 0208 0100 dc00 0000 b400 5803 0607
0079d80 6d6d 7061 0000 0000 f7c8 ffff 0001 0000


static ssize_t sel_write_load(struct file *file, const char __user *buf,
     size_t count, loff_t *ppos)

{
ssize_t length;
void *data = NULL;

mutex_lock(&sel_mutex);

length = avc_has_perm(current_sid(), SECINITSID_SECURITY,
     SECCLASS_SECURITY, SECURITY__LOAD_POLICY, NULL);
if (length)
goto out;

/* No partial writes. */
length = -EINVAL;
if (*ppos != 0)
goto out;

length = -EFBIG;
if (count > 64 * 1024 * 1024)
goto out;

length = -ENOMEM;
data = vmalloc(count);            // dst buffer in kernel
if (!data)
goto out;

length = -EFAULT;
if (copy_from_user(data, buf, count) != 0)
goto out;




BR
Kassey

Will Deacon <will.deacon@arm.com> 于2019年1月17日周四 上午12:48写道：
>
> [I'm due to get on a long flight shortly, so I've added LAKML and a few
> others to CC]
>
> On Wed, Jan 16, 2019 at 11:25:15AM +0800, Kassey wrote:
> > Hi, Will and team:
> >
> >    we met a issue when copy_from_user to access the last page of DDR
> > on 4.14 kenrel, below is the detail steps,
> >    can you help to suggest if there is know fix or debug something ?
> >
> > 1. we mmap ( in userspace)  a region of  phy address that is not
> > continous  but include the last page of ddr
> >
> >   for example our ddr end is 0x200000000
> >   the last page is fall in below addr:
> >   0x1fffff000  to  0x200000000
> >
> > 2. we using copy_from_user to copy these mmap address to kernel buffer
> >
> > 3. and we find everytime when trying to copy_from_user  the last page
> > in phy of ddr,
> >     the dst kernel buffer is looks overwrite by some same patten start
> > with "mmap" in this last page ,but the src in the last page of ddr is
> > still correct.
> >
> > is there any know issue for copy_from_user to accces the last page of
> > phy ddr mmaped by userspace  ?
>
> Not that I'm aware of. Are you using defconfig and can you reproduce the
> problem with mainline (e.g. v5.0-rc2)? What exactly do you mean by:
>
>   "the fst kernel buffer is looks overwrite by some same patten start with
>   "mmap""?
>
> Does the copy_from_user fail (what does it return?) or does it succeed but
> the data copied appears to be junk? Is it deterministic? Could you share an
> example of what the corrupted data looks like, please?
>
> Cheers,
>
> Will



-- 
Best regards
Kassey

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: arm64: copy_from_user access the last page of ddr has problem on 4.14 kernel

[Will Deacon]

On Thu, Jan 17, 2019 at 09:57:17AM +0800, Kassey wrote:
> hi, Will
>       it is hard to try on v5.0-rc2 kernel, since there is much port
> job to be done.
>      dst kernel buffer is looks overwriten by some same(fix) patter
> start with "mmap"    (0x6d6d7061)  see below code (data from vmalloc),
> and file is mmaped (include the last phy page of ddr.)
>      see below pattern and pieces of code.

Weird!

>      not sure if there is boundary issue for copy_from_user, please
> help to suggest if you got some idea from the pattern, thanks.

copy_from_user() doesn't care about the physical address, so I can't see why
it would matter (assuming we haven't done something nuts elsewhere, like
double-allocate the page).

The corruption you have is reasonably regular:

> 0079c00 6d6d 7061 0000 0000 0848 fd8f 0001 0000
> 0079c10 0048 fd8f 0001 0000 0001 0000 0003 0000
> 0079c20 2000 fd83 0001 0000 1fff fd86 0001 0000
> 0079c30 0000 0000 0000 0000 700f 0000 0000 0000

Here's "mmap" again, but the record is twice as long:

> 0079c40 6d6d 7061 0000 0000 f448 ffff 0001 0000
> 0079c50 f748 ffff 0001 0000 0001 0000 0004 0000
> 0079c60 3000 fd8c 0001 0000 2fff fd8e 0001 0000
> 0079c70 0000 0000 0000 0000 700f 0000 0000 0000
> 0079c80 c103 0606 0100 be00 1009 3b00 3b07 0607
> 0079c90 0100 5700 1006 e800 8c03 3103 0100 0a00
> 0079ca0 0000 cf00 bf08 0a00 0100 5700 1006 3700
> 0079cb0 3906 0606 0100 1600 0004 4700 9902 0207

And then the whole structure repeats:

> 0079cc0 6d6d 7061 0000 0000 f808 ffff 0001 0000
> 0079cd0 f1c8 ffff 0001 0000 0001 0000 0005 0000
> 0079ce0 d000 fff8 0001 0000 efff fffa 0001 0000
> 0079cf0 0000 0000 0000 0000 700f 0000 0000 0000

> 0079d00 6d6d 7061 0000 0000 f1c8 ffff 0001 0000
> 0079d10 f388 ffff 0001 0000 0001 0000 0003 0000
> 0079d20 c000 ffdf 0001 0000 6fff fff8 0001 0000
> 0079d30 0000 0000 0000 0000 700f 0000 0000 0000
> 0079d40 9407 0901 0100 5300 0204 b400 d503 0a04
> 0079d50 0100 0000 0001 0200 7309 0202 0100 0200
> 0079d60 5000 0200 7309 0202 0400 ff00 f7ff 94ff
> 0079d70 b400 0208 0100 dc00 0000 b400 5803 0607
> 0079d80 6d6d 7061 0000 0000 f7c8 ffff 0001 0000

Do you have any applications running with the name "mmap"?

Also, have you booted with "memtest" on the command-line, so that we can
rule out any dram aliasing issues and the like?

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Re: arm64: copy_from_user access the last page of ddr has problem on 4.14 kernel

[kassey]

hi Will:
     sorry for late response. 
     we did have process in userspace doing mmap. 
     mult device can reprocued this issue, so we do not suspect the ddr not stable. 
     can you help review  below patch to against with such issue, because we find if enable kasan , issue not seen. 

      https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.0-rc2&id=c8a43c18a97845e7f94ed7d181c11f41964976a2

BR
Kassey 

 
From: Will Deacon
Date: 2019-01-20 10:08
To: Kassey
CC: linux-kernel; willy; kassey; linux-arm-kernel; catalin.marinas; mark.rutland; robin.murphy; ard.biesheuvel
Subject: Re: arm64: copy_from_user access the last page of ddr has problem on 4.14 kernel
On Thu, Jan 17, 2019 at 09:57:17AM +0800, Kassey wrote:
> hi, Will
>       it is hard to try on v5.0-rc2 kernel, since there is much port
> job to be done.
>      dst kernel buffer is looks overwriten by some same(fix) patter
> start with "mmap"    (0x6d6d7061)  see below code (data from vmalloc),
> and file is mmaped (include the last phy page of ddr.)
>      see below pattern and pieces of code.
 
Weird!
 
>      not sure if there is boundary issue for copy_from_user, please
> help to suggest if you got some idea from the pattern, thanks.
 
copy_from_user() doesn't care about the physical address, so I can't see why
it would matter (assuming we haven't done something nuts elsewhere, like
double-allocate the page).
 
The corruption you have is reasonably regular:
 
> 0079c00 6d6d 7061 0000 0000 0848 fd8f 0001 0000
> 0079c10 0048 fd8f 0001 0000 0001 0000 0003 0000
> 0079c20 2000 fd83 0001 0000 1fff fd86 0001 0000
> 0079c30 0000 0000 0000 0000 700f 0000 0000 0000
 
Here's "mmap" again, but the record is twice as long:
 
> 0079c40 6d6d 7061 0000 0000 f448 ffff 0001 0000
> 0079c50 f748 ffff 0001 0000 0001 0000 0004 0000
> 0079c60 3000 fd8c 0001 0000 2fff fd8e 0001 0000
> 0079c70 0000 0000 0000 0000 700f 0000 0000 0000
> 0079c80 c103 0606 0100 be00 1009 3b00 3b07 0607
> 0079c90 0100 5700 1006 e800 8c03 3103 0100 0a00
> 0079ca0 0000 cf00 bf08 0a00 0100 5700 1006 3700
> 0079cb0 3906 0606 0100 1600 0004 4700 9902 0207
 
And then the whole structure repeats:
 
> 0079cc0 6d6d 7061 0000 0000 f808 ffff 0001 0000
> 0079cd0 f1c8 ffff 0001 0000 0001 0000 0005 0000
> 0079ce0 d000 fff8 0001 0000 efff fffa 0001 0000
> 0079cf0 0000 0000 0000 0000 700f 0000 0000 0000
 
> 0079d00 6d6d 7061 0000 0000 f1c8 ffff 0001 0000
> 0079d10 f388 ffff 0001 0000 0001 0000 0003 0000
> 0079d20 c000 ffdf 0001 0000 6fff fff8 0001 0000
> 0079d30 0000 0000 0000 0000 700f 0000 0000 0000
> 0079d40 9407 0901 0100 5300 0204 b400 d503 0a04
> 0079d50 0100 0000 0001 0200 7309 0202 0100 0200
> 0079d60 5000 0200 7309 0202 0400 ff00 f7ff 94ff
> 0079d70 b400 0208 0100 dc00 0000 b400 5803 0607
> 0079d80 6d6d 7061 0000 0000 f7c8 ffff 0001 0000
 
Do you have any applications running with the name "mmap"?
 
Also, have you booted with "memtest" on the command-line, so that we can
rule out any dram aliasing issues and the like?
 
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Re: arm64: copy_from_user access the last page of ddr has problem on 4.14 kernel

[Will Deacon]

On Fri, Feb 01, 2019 at 05:46:44PM +0800, kassey@126.com wrote:
>      sorry for late response. 
>      we did have process in userspace doing mmap. 
>      mult device can reprocued this issue, so we do not suspect the ddr
> not stable. 

Multiple devices of the same SoC?

>      can you help review  below patch to against with such issue, because
> we find if enable kasan , issue not seen.

Unfortunately, I'm not sure what more we can do with this unless we have a
reliable way to reproduce the problem. What is the output that you're
expecting to see in the page? You said that you're seeing the problem in
sel_write_load(), but afaict that /is/ dealing with binary policy data for
SELinux, which may well look like the corruption you describe.

Did you run memtest as I suggested?

Will
 
> The corruption you have is reasonably regular:
>  
> > 0079c00 6d6d 7061 0000 0000 0848 fd8f 0001 0000
> > 0079c10 0048 fd8f 0001 0000 0001 0000 0003 0000
> > 0079c20 2000 fd83 0001 0000 1fff fd86 0001 0000
> > 0079c30 0000 0000 0000 0000 700f 0000 0000 0000
>  
> Here's "mmap" again, but the record is twice as long:
>  
> > 0079c40 6d6d 7061 0000 0000 f448 ffff 0001 0000
> > 0079c50 f748 ffff 0001 0000 0001 0000 0004 0000
> > 0079c60 3000 fd8c 0001 0000 2fff fd8e 0001 0000
> > 0079c70 0000 0000 0000 0000 700f 0000 0000 0000
> > 0079c80 c103 0606 0100 be00 1009 3b00 3b07 0607
> > 0079c90 0100 5700 1006 e800 8c03 3103 0100 0a00
> > 0079ca0 0000 cf00 bf08 0a00 0100 5700 1006 3700
> > 0079cb0 3906 0606 0100 1600 0004 4700 9902 0207
>  
> And then the whole structure repeats:
>  
> > 0079cc0 6d6d 7061 0000 0000 f808 ffff 0001 0000
> > 0079cd0 f1c8 ffff 0001 0000 0001 0000 0005 0000
> > 0079ce0 d000 fff8 0001 0000 efff fffa 0001 0000
> > 0079cf0 0000 0000 0000 0000 700f 0000 0000 0000
>  
> > 0079d00 6d6d 7061 0000 0000 f1c8 ffff 0001 0000
> > 0079d10 f388 ffff 0001 0000 0001 0000 0003 0000
> > 0079d20 c000 ffdf 0001 0000 6fff fff8 0001 0000
> > 0079d30 0000 0000 0000 0000 700f 0000 0000 0000
> > 0079d40 9407 0901 0100 5300 0204 b400 d503 0a04
> > 0079d50 0100 0000 0001 0200 7309 0202 0100 0200
> > 0079d60 5000 0200 7309 0202 0400 ff00 f7ff 94ff
> > 0079d70 b400 0208 0100 dc00 0000 b400 5803 0607
> > 0079d80 6d6d 7061 0000 0000 f7c8 ffff 0001 0000

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel