arm kernel oops in highmem.c with 4.2

arm kernel oops in highmem.c with 4.2

[Peter Robinson]

Hi All,

On Fedora 23 with recent 4.2 kernels we're seeing a crash (below) in
highmem.c on a fairly regular occurrence across a number of different
SoCs, I've seen it with at least AllWinner A20, i.MX6Q, Tegra2 and 124
with both a LPAE and non LPAE kernel, seen it happen when doing a
number of different things but regenerating a initrd, applying updates
(dnf/yum) and starting X are all pretty good triggers.

I've not seen anyone else report anything similar so I'm wondering if
anyone can give me some assistance and direction in getting to the
bottom of this.

Peter

[71751.658105] ------------[ cut here ]------------
[71751.658153] kernel BUG at arch/arm/mm/highmem.c:114!
[71751.658176] Internal error: Oops - BUG: 0 [#1] SMP ARM
[71751.658200] Modules linked in: fuse ofpart snd_soc_tegra30_i2s
snd_soc_tegra_pcm m25p80 spi_nor mtd snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc videobuf2_memops videobuf2_core v4l2_common videodev
snd_usb_audio snd_usbmidi_lib media snd_rawmidi snd_hda_tegra
snd_soc_rt5640 snd_soc_tegra_rt5640 snd_soc_tegra_utils snd_hda_codec
snd_soc_rl6231 snd_hda_core snd_hwdep snd_soc_tegra30_ahub
snd_soc_core snd_compress snd_pcm_dmaengine ac97_bus snd_seq
snd_seq_device snd_pcm snd_timer snd soundcore tegra_soctherm r8169
nouveau lm90 mii at24 i2c_algo_bit spi_tegra114 ttm tegra_devfreq nfsd
uas usb_storage mmc_block as3722_regulator ehci_tegra gpio_keys
phy_tegra_usb sdhci_tegra sdhci_pltfm sdhci mmc_core ahci_tegra
libahci_platform rtc_tegra i2c_tegra tegra_drm drm_kms_helper
syscopyarea sysfillrect sysimgblt
[71751.659048]  drm host1x
[71751.659099] CPU: 0 PID: 7723 Comm: applydeltarpm Not tainted
4.2.0-0.rc5.git0.1.fc23.armv7hl #1
[71751.659122] Hardware name: NVIDIA Tegra SoC (Flattened Device Tree)
[71751.659148] task: e3482280 ti: e33a8000 task.ti: e33a8000
[71751.659189] PC is at __kunmap_atomic+0x54/0x178
[71751.659222] LR is at copy_page_to_iter+0x15c/0x258
[71751.659246] pc : [<c0222f84>]    lr : [<c052dcf0>]    psr: 800d0013
               sp : e33a9e58  ip : 0000000c  fp : 014791d4
[71751.659269] r10: e33a9f0c  r9 : e33a9f14  r8 : 00001000
[71751.659290] r7 : ffee0000  r6 : 00000000  r5 : c0d48554  r4 : ffedf000
[71751.659310] r3 : fffff000  r2 : fff00000  r1 : 2d84a000  r0 : ffedf000
[71751.659335] Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[71751.659360] Control: 10c5387d  Table: ab7fc06a  DAC: 00000015
[71751.659385] Process applydeltarpm (pid: 7723, stack limit = 0xe33a8220)
[71751.659406] Stack: (0xe33a9e58 to 0xe33aa000)
[71751.659428] 9e40:
    e33a9f0c 0000152c
[71751.659454] 9e60: 00000000 c052dcf0 ef43fd1c ffedf000 00001000
00000002 e3451e40 dcc22164
[71751.659482] 9e80: ef43fd1c 00000000 00001000 e33a9f28 0000052c
c032ad54 00000008 00000002
[71751.659506] 9ea0: e33a9f14 00000001 00000000 dcc22078 e3451ea8
00000004 0000f51e 00000000
[71751.659528] 9ec0: ecc67c38 c08d8fd4 c0514dcc c02a6040 ec856e88
00000000 ee5a8480 e3451e40
[71751.659548] 9ee0: 00000000 e33a9f88 00002000 c020fae4 e33a8000
00000000 00000000 c037fe88
[71751.659571] 9f00: 00002000 c020fae4 e33a8000 01478ca8 00002000
00000000 0000052c 00001ad4
[71751.659593] 9f20: e33a9f0c 00000001 e3451e40 00000000 00001ad4
00000000 00000000 00000000
[71751.659615] 9f40: 00000000 00000000 00da0d20 01478ca8 e3451e40
e33a9f88 00002000 c0380550
[71751.659638] 9f60: e3451e40 01478ca8 00002000 e3451e40 e3451e40
01478ca8 00002000 c020fae4
[71751.659660] 9f80: e33a8000 c0380e04 00001ad4 00000000 00002000
01434260 01478ca8 00002000
[71751.659682] 9fa0: 00000003 c020f960 01434260 01478ca8 0000000d
01478ca8 00002000 01478c78
[71751.659706] 9fc0: 01434260 01478ca8 00002000 00000003 00001ad4
00000000 00024000 00000000
[71751.659729] 9fe0: 00000000 be984f14 00015724 b6e435b0 600f0010
0000000d 00000000 00000000
[71751.659782] [<c0222f84>] (__kunmap_atomic) from [<c052dcf0>]
(copy_page_to_iter+0x15c/0x258)
[71751.659822] [<c052dcf0>] (copy_page_to_iter) from [<c032ad54>]
(generic_file_read_iter+0x370/0x5dc)
[71751.659857] [<c032ad54>] (generic_file_read_iter) from [<c037fe88>]
(__vfs_read+0xb0/0xd8)
[71751.659885] [<c037fe88>] (__vfs_read) from [<c0380550>] (vfs_read+0x8c/0x13c)
[71751.659906] [<c0380550>] (vfs_read) from [<c0380e04>] (SyS_read+0x48/0x88)
[71751.659938] [<c0380e04>] (SyS_read) from [<c020f960>]
(ret_fast_syscall+0x0/0x3c)
[71751.659965] Code: e1a03603 e0632002 e1540002 0a000000 (e7f001f2)
[71751.659992] ---[ end trace 701c1dbef773d4cc ]---
[71756.135495] ------------[ cut here ]------------
[71756.135540] kernel BUG at arch/arm/mm/highmem.c:114!
[71756.135562] Internal error: Oops - BUG: 0 [#2] SMP ARM
[71756.135582] Modules linked in: fuse ofpart snd_soc_tegra30_i2s
snd_soc_tegra_pcm m25p80 spi_nor mtd snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc videobuf2_memops videobuf2_core v4l2_common videodev
snd_usb_audio snd_usbmidi_lib media snd_rawmidi snd_hda_tegra
snd_soc_rt5640 snd_soc_tegra_rt5640 snd_soc_tegra_utils snd_hda_codec
snd_soc_rl6231 snd_hda_core snd_hwdep snd_soc_tegra30_ahub
snd_soc_core snd_compress snd_pcm_dmaengine ac97_bus snd_seq
snd_seq_device snd_pcm snd_timer snd soundcore tegra_soctherm r8169
nouveau lm90 mii at24 i2c_algo_bit spi_tegra114 ttm tegra_devfreq nfsd
uas usb_storage mmc_block as3722_regulator ehci_tegra gpio_keys
phy_tegra_usb sdhci_tegra sdhci_pltfm sdhci mmc_core ahci_tegra
libahci_platform rtc_tegra i2c_tegra tegra_drm drm_kms_helper
syscopyarea sysfillrect sysimgblt
[71756.136307]  drm host1x
[71756.136347] CPU: 1 PID: 7707 Comm: dnf Tainted: G      D
4.2.0-0.rc5.git0.1.fc23.armv7hl #1
[71756.136365] Hardware name: NVIDIA Tegra SoC (Flattened Device Tree)
[71756.136386] task: ea187300 ti: e3386000 task.ti: e3386000
[71756.136439] PC is at __kunmap_atomic+0x54/0x178
[71756.136470] LR is at copy_page_to_iter+0x15c/0x258
[71756.136490] pc : [<c0222f84>]    lr : [<c052dcf0>]    psr: 800d0013
               sp : e3387e58  ip : 000000ef  fp : be81f8a4
[71756.136511] r10: e3387f0c  r9 : e3387f14  r8 : 000002cc
[71756.136530] r7 : ffecf2cc  r6 : 00000000  r5 : c0d48554  r4 : ffecf000
[71756.136545] r3 : 0000f000  r2 : ffef0000  r1 : 2d856000  r0 : ffecf000
[71756.136567] Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[71756.136587] Control: 10c5387d  Table: a33cc06a  DAC: 00000015
[71756.136605] Process dnf (pid: 7707, stack limit = 0xe3386220)
[71756.136620] Stack: (0xe3387e58 to 0xe3388000)
[71756.136638] 7e40:
    e3387f0c 000002cc
[71756.136660] 7e60: 00000000 c052dcf0 ef4047c0 ffecf000 000002cc
00000041 ea5a5300 dcc98184
[71756.136682] 7e80: ef4047c0 00000000 000002cc e3387f28 00000000
c032ad54 edfb61c0 c04b6784
[71756.136703] 7ea0: e3387f14 00000040 00000000 dcc98098 ea5a5368
00000042 000412cb 00000000
[71756.136724] 7ec0: 00000000 00000000 ee5b8180 0000b661 edfb6208
00000000 edfb61c0 ea5a5300
[71756.136744] 7ee0: 00000000 e3387f88 00000800 c020fae4 e3386000
00000000 00000004 c037fe88
[71756.136762] 7f00: 00000800 c020fae4 e3386000 be81f8a4 00000800
00000000 00000000 00000800
[71756.136781] 7f20: e3387f0c 00000001 ea5a5300 00000000 00041000
00000000 00000000 00000000
[71756.136803] 7f40: 00000000 00000000 dcc98098 be81f8a4 ea5a5300
e3387f88 00000800 c0380550
[71756.136821] 7f60: ea5a5300 be81f8a4 00000800 ea5a5300 ea5a5302
be81f8a4 00000800 c020fae4
[71756.136841] 7f80: e3386000 c0380e04 00041000 00000000 00000800
be81f8a4 800cb0b8 00000015
[71756.136861] 7fa0: 00000003 c020f960 be81f8a4 800cb0b8 00000015
be81f8a4 00000800 b84c5a6d
[71756.136879] 7fc0: be81f8a4 800cb0b8 00000015 00000003 00000001
b6f97c80 7fe6d3e8 00000004
[71756.136899] 7fe0: 00000000 be81f850 b6f94860 b6d0cad8 800f0010
00000015 00000000 00000000
[71756.136944] [<c0222f84>] (__kunmap_atomic) from [<c052dcf0>]
(copy_page_to_iter+0x15c/0x258)
[71756.136985] [<c052dcf0>] (copy_page_to_iter) from [<c032ad54>]
(generic_file_read_iter+0x370/0x5dc)
[71756.137020] [<c032ad54>] (generic_file_read_iter) from [<c037fe88>]
(__vfs_read+0xb0/0xd8)
[71756.137045] [<c037fe88>] (__vfs_read) from [<c0380550>] (vfs_read+0x8c/0x13c)
[71756.137069] [<c0380550>] (vfs_read) from [<c0380e04>] (SyS_read+0x48/0x88)
[71756.137102] [<c0380e04>] (SyS_read) from [<c020f960>]
(ret_fast_syscall+0x0/0x3c)
[71756.137127] Code: e1a03603 e0632002 e1540002 0a000000 (e7f001f2)
[71756.137151] ---[ end trace 701c1dbef773d4cd ]---

arm kernel oops in highmem.c with 4.2

[Russell King - ARM Linux]

On Wed, Aug 05, 2015 at 11:01:01AM +0100, Peter Robinson wrote:
> Hi All,
> 
> On Fedora 23 with recent 4.2 kernels we're seeing a crash (below) in
> highmem.c on a fairly regular occurrence across a number of different
> SoCs, I've seen it with at least AllWinner A20, i.MX6Q, Tegra2 and 124
> with both a LPAE and non LPAE kernel, seen it happen when doing a
> number of different things but regenerating a initrd, applying updates
> (dnf/yum) and starting X are all pretty good triggers.

I've yet to see any problems with mainline 4.2-rc5 kernels on any of my
iMX6 platforms, which includes initramfs regeneration, apt-get updates
and X.

> [71751.658105] ------------[ cut here ]------------
> [71751.658153] kernel BUG at arch/arm/mm/highmem.c:114!

Well, in mainline kernels, the BUG is on line 113, not line 114.  So at
least this file is modified from mainline kernels.  Maybe the problem is
caused by patches applied to Fedora kernels?

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

arm kernel oops in highmem.c with 4.2

[Peter Robinson]

On Wed, Aug 5, 2015 at 11:07 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Wed, Aug 05, 2015 at 11:01:01AM +0100, Peter Robinson wrote:
>> Hi All,
>>
>> On Fedora 23 with recent 4.2 kernels we're seeing a crash (below) in
>> highmem.c on a fairly regular occurrence across a number of different
>> SoCs, I've seen it with at least AllWinner A20, i.MX6Q, Tegra2 and 124
>> with both a LPAE and non LPAE kernel, seen it happen when doing a
>> number of different things but regenerating a initrd, applying updates
>> (dnf/yum) and starting X are all pretty good triggers.
>
> I've yet to see any problems with mainline 4.2-rc5 kernels on any of my
> iMX6 platforms, which includes initramfs regeneration, apt-get updates
> and X.
>
>> [71751.658105] ------------[ cut here ]------------
>> [71751.658153] kernel BUG at arch/arm/mm/highmem.c:114!
>
> Well, in mainline kernels, the BUG is on line 113, not line 114.  So at
> least this file is modified from mainline kernels.  Maybe the problem is
> caused by patches applied to Fedora kernels?

We apply a crash driver patch [1] which has been there forever (long
enough that I'd forgotten it) but other than that for arm kernels we
currently don't apply any arm specific patches in 4.2.

Peter

[1] http://pkgs.fedoraproject.org/cgit/kernel.git/tree/crash-driver.patch

arm kernel oops in highmem.c with 4.2

[Russell King - ARM Linux]

On Wed, Aug 05, 2015 at 11:13:07AM +0100, Peter Robinson wrote:
> On Wed, Aug 5, 2015 at 11:07 AM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> > On Wed, Aug 05, 2015 at 11:01:01AM +0100, Peter Robinson wrote:
> >> Hi All,
> >>
> >> On Fedora 23 with recent 4.2 kernels we're seeing a crash (below) in
> >> highmem.c on a fairly regular occurrence across a number of different
> >> SoCs, I've seen it with at least AllWinner A20, i.MX6Q, Tegra2 and 124
> >> with both a LPAE and non LPAE kernel, seen it happen when doing a
> >> number of different things but regenerating a initrd, applying updates
> >> (dnf/yum) and starting X are all pretty good triggers.
> >
> > I've yet to see any problems with mainline 4.2-rc5 kernels on any of my
> > iMX6 platforms, which includes initramfs regeneration, apt-get updates
> > and X.
> >
> >> [71751.658105] ------------[ cut here ]------------
> >> [71751.658153] kernel BUG at arch/arm/mm/highmem.c:114!
> >
> > Well, in mainline kernels, the BUG is on line 113, not line 114.  So at
> > least this file is modified from mainline kernels.  Maybe the problem is
> > caused by patches applied to Fedora kernels?
> 
> We apply a crash driver patch [1] which has been there forever (long
> enough that I'd forgotten it) but other than that for arm kernels we
> currently don't apply any arm specific patches in 4.2.

It helps if I look at 4.2 rather than an older kernel :)

However, I've checked that I have DEBUG_HIGHMEM enabled, which I do, and
I'm unable to reproduce this here.  My kernels are built with gcc 4.7.4.

What it looks like from your oops is that the address which was passed
in was 0xffedf000, but the address we calculated via the following for
the current index was 0xfff00000:

type = kmap_atomic_idx();
idx = type + KM_TYPE_NR * smp_processor_id();
__fix_to_virt(idx)

Doing a bit of maths... the address 0xffedf000 corresponds to a fixmap
index of... (0xffeff000 - 0xffedf000) >> 12 = 32.  KM_TYPE_NR is 16 on
ARM, so the mapping was created by CPU 2, and type was zero.

On unmap, 0xfff00000 gives... (0xffeff000 - 0xfff00000) >> 12 = -1.
That suggests we're on CPU 0, and type is -1 - in other words, there
are no atomically mapped mappings on CPU 0.

Since kmap_atomic() disables preemption and page faults, how did your
kernel migrate this thread from CPU 2 to CPU 0... and I can't see how
that happened.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

arm kernel oops in highmem.c with 4.2

[Mark Salter]

On Wed, 2015-08-05 at 12:27 +0100, Russell King - ARM Linux wrote:
> On Wed, Aug 05, 2015 at 11:13:07AM +0100, Peter Robinson wrote:
> > On Wed, Aug 5, 2015 at 11:07 AM, Russell King - ARM Linux
> > <linux@arm.linux.org.uk> wrote:
> > > On Wed, Aug 05, 2015 at 11:01:01AM +0100, Peter Robinson wrote:
> > > > Hi All,
> > > > 
> > > > On Fedora 23 with recent 4.2 kernels we're seeing a crash (below) 
> > > > in
> > > > highmem.c on a fairly regular occurrence across a number of 
> > > > different
> > > > SoCs, I've seen it with at least AllWinner A20, i.MX6Q, Tegra2 and 
> > > > 124
> > > > with both a LPAE and non LPAE kernel, seen it happen when doing a
> > > > number of different things but regenerating a initrd, applying 
> > > > updates
> > > > (dnf/yum) and starting X are all pretty good triggers.
> > > 
> > > I've yet to see any problems with mainline 4.2-rc5 kernels on any of 
> > > my
> > > iMX6 platforms, which includes initramfs regeneration, apt-get 
> > > updates
> > > and X.
> > > 
> > > > [71751.658105] ------------[ cut here ]------------
> > > > [71751.658153] kernel BUG at arch/arm/mm/highmem.c:114!
> > > 
> > > Well, in mainline kernels, the BUG is on line 113, not line 114.  So 
> > > at
> > > least this file is modified from mainline kernels.  Maybe the problem 
> > > is
> > > caused by patches applied to Fedora kernels?
> > 
> > We apply a crash driver patch [1] which has been there forever (long
> > enough that I'd forgotten it) but other than that for arm kernels we
> > currently don't apply any arm specific patches in 4.2.
> 
> It helps if I look at 4.2 rather than an older kernel :)
> 
> However, I've checked that I have DEBUG_HIGHMEM enabled, which I do, and
> I'm unable to reproduce this here.  My kernels are built with gcc 4.7.4.
> 
> What it looks like from your oops is that the address which was passed
> in was 0xffedf000, but the address we calculated via the following for
> the current index was 0xfff00000:
> 
> type = kmap_atomic_idx();
> idx = type + KM_TYPE_NR * smp_processor_id();
> __fix_to_virt(idx)
> 
> Doing a bit of maths... the address 0xffedf000 corresponds to a fixmap
> index of... (0xffeff000 - 0xffedf000) >> 12 = 32.  KM_TYPE_NR is 16 on
> ARM, so the mapping was created by CPU 2, and type was zero.
> 
> On unmap, 0xfff00000 gives... (0xffeff000 - 0xfff00000) >> 12 = -1.
> That suggests we're on CPU 0, and type is -1 - in other words, there
> are no atomically mapped mappings on CPU 0.
> 
> Since kmap_atomic() disables preemption and page faults, how did your
> kernel migrate this thread from CPU 2 to CPU 0... and I can't see how
> that happened.
> 

The fedora kernel is using PREEMPT_VOLUNTARY with !PREEMPT and
!PREEMPT_CPOUNT. So preempt_disable() is a nop. I added some code
to catch the kernel scheduling between kmap_atomic() and
kunmap_atomic() and got this straightaway:

[    2.958651] ------------[ cut here ]------------
[    2.963263] kernel BUG at arch/arm/mm/highmem.c:61!
[    2.968132] Internal error: Oops - BUG: 0 [#1] SMP ARM
[    2.973261] Modules linked in:
[    2.976313] CPU: 0 PID: 199 Comm: systemd-udevd Tainted: G        W       4.2.0-rc5 #9
[    2.984218] Hardware name: Highbank
[    2.987699] task: ecb9bf40 ti: eccce000 task.ti: eccce000
[    2.993097] PC is at check_kmap_atomic+0x20/0x2c
[    2.997710] LR is at __schedule+0x254/0x60c
[    3.001885] pc : [<c022318c>]    lr : [<c08d49a0>]    psr: 200d0093
[    3.001885] sp : ecccfdd8  ip : 00000000  fp : ecccfe1c
[    3.013350] r10: c0d508a0  r9 : ecb9c244  r8 : ffeff000
[    3.018565] r7 : c0d4a140  r6 : ec90a280  r5 : ed3b7140  r4 : ecb9bf40
[    3.025081] r3 : 00000001  r2 : 2c66d000  r1 : eccce000  r0 : 00000000
[    3.031599] Flags: nzCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment user
[    3.038810] Control: 10c5387d  Table: 2cc9c04a  DAC: 00000015
[    3.044546] Process systemd-udevd (pid: 199, stack limit = 0xeccce220)
[    3.051063] Stack: (0xecccfdd8 to 0xeccd0000)
[    3.055410] fdc0:                                                       eccad500 00000000
[    3.063580] fde0: ecc83e80 eccbc488 ecc83e80 c08d5030 2c66d000 00000000 00000002 eccce000
[    3.071749] fe00: 00000000 0000012a b6fe0000 ffeff000 ecccff14 ecccff0c ecccfe2c c08d5030
[    3.079918] fe20: ecb5f538 00000000 ecccfe34 c08d5088 b6fe0000 c08d6a14 00000000 c05285a0
[    3.088086] fe40: b6fe0000 ffeff000 0000012a 0000012a 00000000 ec3da524 ffeff000 0000012a
[    3.096255] fe60: ecccff14 c052dcec ef7b1678 ffeff000 0000012a 00000000 eccbc480 ec3da524
[    3.104423] fe80: ef7b1678 00000000 0000012a ecccff28 00000000 c032add4 00000000 c04b6804
[    3.112592] fea0: ecccff14 ffffffff 00000fff ec3da438 eccbc4e8 00000001 00000129 00000000
[    3.120760] fec0: ecc72c18 00000000 00000000 000b6fe0 00000000 00000000 b6fe1000 eccbc480
[    3.128929] fee0: 00000000 ecccff88 00001000 c020fae4 eccce000 00000200 00000000 c037ff08
[    3.137098] ff00: 00001000 c020fae4 eccce000 b6fe0000 00001000 00000000 00000000 00001000
[    3.145266] ff20: ecccff0c 00000001 eccbc480 00000000 00000000 00000000 00000000 00000000
[    3.153435] ff40: 00000000 00000000 00000000 b6fe0000 eccbc480 ecccff88 00001000 c03805d0
[    3.161603] ff60: eccbc480 b6fe0000 00001000 eccbc480 eccbc480 b6fe0000 00001000 c020fae4
[    3.169772] ff80: eccce000 c0380e84 00000000 00000000 00001000 8066e1c0 00003ffe 8066e1c0
[    3.177940] ffa0: 00000003 c020fad0 8066e1c0 00003ffe 00000006 b6fe0000 00001000 00000040
[    3.186109] ffc0: 8066e1c0 00003ffe 8066e1c0 00000003 0000000a bee856e4 00000000 00000000
[    3.194277] ffe0: 00000000 bee8528c b6daa470 b6e0f5b0 600d0010 00000006 00000000 00000000
[    3.202454] [<c022318c>] (check_kmap_atomic) from [<c08d49a0>] (__schedule+0x254/0x60c)
[    3.210454] [<c08d49a0>] (__schedule) from [<c08d5030>] (preempt_schedule_common+0x24/0x40)
[    3.218799] [<c08d5030>] (preempt_schedule_common) from [<c08d5088>] (_cond_resched+0x3c/0x4c)
[    3.227404] [<c08d5088>] (_cond_resched) from [<c08d6a14>] (down_read+0x14/0x48)
[    3.234799] [<c08d6a14>] (down_read) from [<c05285a0>] (__copy_to_user_memcpy+0x54/0x17c)
[    3.242974] [<c05285a0>] (__copy_to_user_memcpy) from [<c052dcec>] (copy_page_to_iter+0xd8/0x258)
[    3.251844] [<c052dcec>] (copy_page_to_iter) from [<c032add4>] (generic_file_read_iter+0x370/0x5dc)
[    3.260885] [<c032add4>] (generic_file_read_iter) from [<c037ff08>] (__vfs_read+0xb0/0xd8)
[    3.269142] [<c037ff08>] (__vfs_read) from [<c03805d0>] (vfs_read+0x8c/0x13c)
[    3.276270] [<c03805d0>] (vfs_read) from [<c0380e84>] (SyS_read+0x48/0x88)
[    3.283141] [<c0380e84>] (SyS_read) from [<c020fad0>] (__sys_trace_return+0x0/0x10)
[    3.290790] Code: e7922100 e7923003 e3530000 012fff1e (e7f001f2) 
[    3.296876] ---[ end trace cb88537fdc8fa202 ]---
[    3.301485] note: systemd-udevd[199] exited with preempt_count 2097152

arm kernel oops in highmem.c with 4.2

[Russell King - ARM Linux]

On Tue, Aug 11, 2015 at 01:48:10PM -0400, Mark Salter wrote:
> On Wed, 2015-08-05 at 12:27 +0100, Russell King - ARM Linux wrote:
> > It helps if I look at 4.2 rather than an older kernel :)
> > 
> > However, I've checked that I have DEBUG_HIGHMEM enabled, which I do, and
> > I'm unable to reproduce this here.  My kernels are built with gcc 4.7.4.
> > 
> > What it looks like from your oops is that the address which was passed
> > in was 0xffedf000, but the address we calculated via the following for
> > the current index was 0xfff00000:
> > 
> > type = kmap_atomic_idx();
> > idx = type + KM_TYPE_NR * smp_processor_id();
> > __fix_to_virt(idx)
> > 
> > Doing a bit of maths... the address 0xffedf000 corresponds to a fixmap
> > index of... (0xffeff000 - 0xffedf000) >> 12 = 32.  KM_TYPE_NR is 16 on
> > ARM, so the mapping was created by CPU 2, and type was zero.
> > 
> > On unmap, 0xfff00000 gives... (0xffeff000 - 0xfff00000) >> 12 = -1.
> > That suggests we're on CPU 0, and type is -1 - in other words, there
> > are no atomically mapped mappings on CPU 0.
> > 
> > Since kmap_atomic() disables preemption and page faults, how did your
> > kernel migrate this thread from CPU 2 to CPU 0... and I can't see how
> > that happened.
> > 
> 
> The fedora kernel is using PREEMPT_VOLUNTARY with !PREEMPT and
> !PREEMPT_CPOUNT. So preempt_disable() is a nop. I added some code
> to catch the kernel scheduling between kmap_atomic() and
> kunmap_atomic() and got this straightaway:

Looking at the backtrace, and grepping for __copy_to_user_memcpy, it
seems to imply that you're using the uaccess-with-memcpy code.

This code is relatively unmaintained, and probably mostly unused by
people today, so I doubt it gets much in the way of testing - and
you've certainly found a bug in there.

        /* the mmap semaphore is taken only if not in an atomic context */
        atomic = in_atomic();

        if (!atomic)
                down_read(&current->mm->mmap_sem);

is not sufficient to tell whether we can take the semaphore.

We _could_ replace the above with:

	int ret;

	ret = down_read_trylock(&current->mm->mmap_sem);
	if (!ret) {
		__copy_to_user_std(to, from, n);
		return;
	}

but that's just a guess.  I'm not a big fan of this code, and given
that it probably doesn't get much use, we may be better off deleting
it so it doesn't sit around rotting...  Code like this really needs
regular testing.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

arm kernel oops in highmem.c with 4.2

[Nicolas Pitre]

On Tue, 11 Aug 2015, Russell King - ARM Linux wrote:

> On Tue, Aug 11, 2015 at 01:48:10PM -0400, Mark Salter wrote:
> > On Wed, 2015-08-05 at 12:27 +0100, Russell King - ARM Linux wrote:
> > > It helps if I look at 4.2 rather than an older kernel :)
> > > 
> > > However, I've checked that I have DEBUG_HIGHMEM enabled, which I do, and
> > > I'm unable to reproduce this here.  My kernels are built with gcc 4.7.4.
> > > 
> > > What it looks like from your oops is that the address which was passed
> > > in was 0xffedf000, but the address we calculated via the following for
> > > the current index was 0xfff00000:
> > > 
> > > type = kmap_atomic_idx();
> > > idx = type + KM_TYPE_NR * smp_processor_id();
> > > __fix_to_virt(idx)
> > > 
> > > Doing a bit of maths... the address 0xffedf000 corresponds to a fixmap
> > > index of... (0xffeff000 - 0xffedf000) >> 12 = 32.  KM_TYPE_NR is 16 on
> > > ARM, so the mapping was created by CPU 2, and type was zero.
> > > 
> > > On unmap, 0xfff00000 gives... (0xffeff000 - 0xfff00000) >> 12 = -1.
> > > That suggests we're on CPU 0, and type is -1 - in other words, there
> > > are no atomically mapped mappings on CPU 0.
> > > 
> > > Since kmap_atomic() disables preemption and page faults, how did your
> > > kernel migrate this thread from CPU 2 to CPU 0... and I can't see how
> > > that happened.
> > > 
> > 
> > The fedora kernel is using PREEMPT_VOLUNTARY with !PREEMPT and
> > !PREEMPT_CPOUNT. So preempt_disable() is a nop. I added some code
> > to catch the kernel scheduling between kmap_atomic() and
> > kunmap_atomic() and got this straightaway:
> 
> Looking at the backtrace, and grepping for __copy_to_user_memcpy, it
> seems to imply that you're using the uaccess-with-memcpy code.
> 
> This code is relatively unmaintained, and probably mostly unused by
> people today, so I doubt it gets much in the way of testing - and
> you've certainly found a bug in there.
> 
>         /* the mmap semaphore is taken only if not in an atomic context */
>         atomic = in_atomic();
> 
>         if (!atomic)
>                 down_read(&current->mm->mmap_sem);
> 
> is not sufficient to tell whether we can take the semaphore.
> 
> We _could_ replace the above with:
> 
> 	int ret;
> 
> 	ret = down_read_trylock(&current->mm->mmap_sem);
> 	if (!ret) {
> 		__copy_to_user_std(to, from, n);
> 		return;
> 	}
> 
> but that's just a guess.  I'm not a big fan of this code, and given
> that it probably doesn't get much use, we may be better off deleting
> it so it doesn't sit around rotting...  Code like this really needs
> regular testing.

I'd agree.  But first I'd like to know why the fedora kernel is using 
CONFIG_UACCESS_WITH_MEMCPY?  If it's just because it sounded cool then 
that's a bad reason.

That code was created to work around inneficiency in the Orion CPU core 
that didn't coalesce writes from STRT instructions, and by using plain 
STR and/or STM the actual throughput was more than doubled.  This was 
fixed in later cores. However Orion users (if they still exist) might 
like the added performance. I don't have Orion-based targets anymore 
(they took the way of the recycling facility a while ago).

To be sure you really need it, you may try something like:

	dd if=/dev/zero of=/dev/null bs=4k

and compare the reported speed with and without 
CONFIG_UACCESS_WITH_MEMCPY.


Nicolas

arm kernel oops in highmem.c with 4.2

[Russell King - ARM Linux]

On Tue, Aug 11, 2015 at 03:41:52PM -0400, Nicolas Pitre wrote:
> I'd agree.  But first I'd like to know why the fedora kernel is using 
> CONFIG_UACCESS_WITH_MEMCPY?  If it's just because it sounded cool then 
> that's a bad reason.
> 
> That code was created to work around inneficiency in the Orion CPU core 
> that didn't coalesce writes from STRT instructions, and by using plain 
> STR and/or STM the actual throughput was more than doubled.  This was 
> fixed in later cores. However Orion users (if they still exist) might 
> like the added performance. I don't have Orion-based targets anymore 
> (they took the way of the recycling facility a while ago).

Irrespective of that, what has been found is that the implementation is
unsafe - it is taking a semaphore when page faults are disabled.  In
other words, notwithstanding the above, it's buggy no matter if it's
an Orion CPU core, or highmem, or what.  Any place in the kernel which
uses pagefault_disable() and then calls into this code will be buggy.

It needs fixing or removing.

Even if it is fixed, making it _depend_ on CPU_FEROCEON sounds like a
good idea if non-orion stuff isn't supposed to be enabling it.  Or
something like that.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

arm kernel oops in highmem.c with 4.2

[Nicolas Pitre]

On Tue, 11 Aug 2015, Russell King - ARM Linux wrote:

> On Tue, Aug 11, 2015 at 03:41:52PM -0400, Nicolas Pitre wrote:
> > I'd agree.  But first I'd like to know why the fedora kernel is using 
> > CONFIG_UACCESS_WITH_MEMCPY?  If it's just because it sounded cool then 
> > that's a bad reason.
> > 
> > That code was created to work around inneficiency in the Orion CPU core 
> > that didn't coalesce writes from STRT instructions, and by using plain 
> > STR and/or STM the actual throughput was more than doubled.  This was 
> > fixed in later cores. However Orion users (if they still exist) might 
> > like the added performance. I don't have Orion-based targets anymore 
> > (they took the way of the recycling facility a while ago).
> 
> Irrespective of that, what has been found is that the implementation is
> unsafe - it is taking a semaphore when page faults are disabled.  In
> other words, notwithstanding the above, it's buggy no matter if it's
> an Orion CPU core, or highmem, or what.  Any place in the kernel which
> uses pagefault_disable() and then calls into this code will be buggy.
> 
> It needs fixing or removing.

Absolutely. I'm not disputing that. I'm only asking so we can wisely 
choose between fixing or removing.  Personally I'd be inclined towards 
the later, unless the following is sufficient to fix it:

diff --git a/arch/arm/lib/uaccess_with_memcpy.c b/arch/arm/lib/uaccess_with_memcpy.c
index 3e58d71001..4b39af2dfd 100644
--- a/arch/arm/lib/uaccess_with_memcpy.c
+++ b/arch/arm/lib/uaccess_with_memcpy.c
@@ -96,7 +96,7 @@ __copy_to_user_memcpy(void __user *to, const void *from, unsigned long n)
 	}
 
 	/* the mmap semaphore is taken only if not in an atomic context */
-	atomic = in_atomic();
+	atomic = faulthandler_disabled();
 
 	if (!atomic)
 		down_read(&current->mm->mmap_sem);

> Even if it is fixed, making it _depend_ on CPU_FEROCEON sounds like a
> good idea if non-orion stuff isn't supposed to be enabling it.  Or
> something like that.

It is not clear to me exactly which cores are affected.  That's why the 
Kconfig entry was left open to all, in case it could benefit others.
In theory it shouldn't be harmful to anyone notwithstanding the caveat
mentioned in the help text.


Nicolas

arm kernel oops in highmem.c with 4.2

[Mark Salter]

On Tue, 2015-08-11 at 16:20 -0400, Nicolas Pitre wrote:
> On Tue, 11 Aug 2015, Russell King - ARM Linux wrote:
> 
> > On Tue, Aug 11, 2015 at 03:41:52PM -0400, Nicolas Pitre wrote:
> > > I'd agree.  But first I'd like to know why the fedora kernel is using 
> > > CONFIG_UACCESS_WITH_MEMCPY?  If it's just because it sounded cool 
> > > then 
> > > that's a bad reason.
> > > 
> > > That code was created to work around inneficiency in the Orion CPU 
> > > core 
> > > that didn't coalesce writes from STRT instructions, and by using 
> > > plain 
> > > STR and/or STM the actual throughput was more than doubled.  This was 
> > > fixed in later cores. However Orion users (if they still exist) might 
> > > like the added performance. I don't have Orion-based targets anymore 
> > > (they took the way of the recycling facility a while ago).
> > 
> > Irrespective of that, what has been found is that the implementation is
> > unsafe - it is taking a semaphore when page faults are disabled.  In
> > other words, notwithstanding the above, it's buggy no matter if it's
> > an Orion CPU core, or highmem, or what.  Any place in the kernel which
> > uses pagefault_disable() and then calls into this code will be buggy.
> > 
> > It needs fixing or removing.
> 
> Absolutely. I'm not disputing that. I'm only asking so we can wisely 
> choose between fixing or removing.  Personally I'd be inclined towards 
> the later, unless the following is sufficient to fix it:
> 
> diff --git a/arch/arm/lib/uaccess_with_memcpy.c 
> b/arch/arm/lib/uaccess_with_memcpy.c
> index 3e58d71001..4b39af2dfd 100644
> --- a/arch/arm/lib/uaccess_with_memcpy.c
> +++ b/arch/arm/lib/uaccess_with_memcpy.c
> @@ -96,7 +96,7 @@ __copy_to_user_memcpy(void __user *to, const void 
> *from, unsigned long n)
>  	}
>  
>  	/* the mmap semaphore is taken only if not in an atomic context 
> */
> -	atomic = in_atomic();
> +	atomic = faulthandler_disabled();
>  
>  	if (!atomic)
>  		down_read(&current->mm->mmap_sem);

Yeah, that fixes the problem I was seeing.

> 
> > Even if it is fixed, making it _depend_ on CPU_FEROCEON sounds like a
> > good idea if non-orion stuff isn't supposed to be enabling it.  Or
> > something like that.
> 
> It is not clear to me exactly which cores are affected.  That's why the 
> Kconfig entry was left open to all, in case it could benefit others.
> In theory it shouldn't be harmful to anyone notwithstanding the caveat
> mentioned in the help text.

This was on a calxeda highbank.

With CONFIG_UACCESS_WITH_MEMCPY, the dd copy test reported 1.8GB/s
Without CONFIG_UACCESS_WITH_MEMCPY, 2.1GB/s

arm kernel oops in highmem.c with 4.2

[Nicolas Pitre]

On Tue, 11 Aug 2015, Mark Salter wrote:

> On Tue, 2015-08-11 at 16:20 -0400, Nicolas Pitre wrote:
> > On Tue, 11 Aug 2015, Russell King - ARM Linux wrote:
> > 
> > > It needs fixing or removing.
> > 
> > Absolutely. I'm not disputing that. I'm only asking so we can wisely 
> > choose between fixing or removing.  Personally I'd be inclined towards 
> > the later, unless the following is sufficient to fix it:
> > 
> > diff --git a/arch/arm/lib/uaccess_with_memcpy.c 
> > b/arch/arm/lib/uaccess_with_memcpy.c
> > index 3e58d71001..4b39af2dfd 100644
> > --- a/arch/arm/lib/uaccess_with_memcpy.c
> > +++ b/arch/arm/lib/uaccess_with_memcpy.c
> > @@ -96,7 +96,7 @@ __copy_to_user_memcpy(void __user *to, const void 
> > *from, unsigned long n)
> >  	}
> >  
> >  	/* the mmap semaphore is taken only if not in an atomic context 
> > */
> > -	atomic = in_atomic();
> > +	atomic = faulthandler_disabled();
> >  
> >  	if (!atomic)
> >  		down_read(&current->mm->mmap_sem);
> 
> Yeah, that fixes the problem I was seeing.

Good!  Then I'll add it to RMK's patch system.  May I add a tested-by 
tag with your name?

> This was on a calxeda highbank.

Any idea why this option was set?

> With CONFIG_UACCESS_WITH_MEMCPY, the dd copy test reported 1.8GB/s
> Without CONFIG_UACCESS_WITH_MEMCPY, 2.1GB/s

Therefore this is of no benefit to you.


Nicolas

arm kernel oops in highmem.c with 4.2

[Mark Salter]

On Tue, 2015-08-11 at 22:18 -0400, Nicolas Pitre wrote:
> On Tue, 11 Aug 2015, Mark Salter wrote:
> 
> > On Tue, 2015-08-11 at 16:20 -0400, Nicolas Pitre wrote:
> > > On Tue, 11 Aug 2015, Russell King - ARM Linux wrote:
> > > 
> > > > It needs fixing or removing.
> > > 
> > > Absolutely. I'm not disputing that. I'm only asking so we can wisely 
> > > choose between fixing or removing.  Personally I'd be inclined 
> > > towards 
> > > the later, unless the following is sufficient to fix it:
> > > 
> > > diff --git a/arch/arm/lib/uaccess_with_memcpy.c 
> > > b/arch/arm/lib/uaccess_with_memcpy.c
> > > index 3e58d71001..4b39af2dfd 100644
> > > --- a/arch/arm/lib/uaccess_with_memcpy.c
> > > +++ b/arch/arm/lib/uaccess_with_memcpy.c
> > > @@ -96,7 +96,7 @@ __copy_to_user_memcpy(void __user *to, const void 
> > > *from, unsigned long n)
> > >  	}
> > >  
> > >  	/* the mmap semaphore is taken only if not in an atomic 
> > > context 
> > > */
> > > -	atomic = in_atomic();
> > > +	atomic = faulthandler_disabled();
> > >  
> > >  	if (!atomic)
> > >  		down_read(&current->mm->mmap_sem);
> > 
> > Yeah, that fixes the problem I was seeing.
> 
> Good!  Then I'll add it to RMK's patch system.  May I add a tested-by 
> tag with your name?

Yes

> 
> > This was on a calxeda highbank.
> 
> Any idea why this option was set?

None. It seems to have been there in the original fedora config
for v7 kernels. But nothing in the config turns on CPU_FEROCEON
so there is no real need for it.

> 
> > With CONFIG_UACCESS_WITH_MEMCPY, the dd copy test reported 1.8GB/s
> > Without CONFIG_UACCESS_WITH_MEMCPY, 2.1GB/s
> 
> Therefore this is of no benefit to you.
> 
> 
> Nicolas

arm kernel oops in highmem.c with 4.2

[Nicolas Pitre]

On Wed, 12 Aug 2015, Mark Salter wrote:

> On Tue, 2015-08-11 at 22:18 -0400, Nicolas Pitre wrote:
> > On Tue, 11 Aug 2015, Mark Salter wrote:
> > 
> > > On Tue, 2015-08-11 at 16:20 -0400, Nicolas Pitre wrote:
> > > > diff --git a/arch/arm/lib/uaccess_with_memcpy.c 
> > > > b/arch/arm/lib/uaccess_with_memcpy.c
> > > > index 3e58d71001..4b39af2dfd 100644
> > > > --- a/arch/arm/lib/uaccess_with_memcpy.c
> > > > +++ b/arch/arm/lib/uaccess_with_memcpy.c
> > > > @@ -96,7 +96,7 @@ __copy_to_user_memcpy(void __user *to, const void 
> > > > *from, unsigned long n)
> > > >  	}
> > > >  
> > > >  	/* the mmap semaphore is taken only if not in an atomic 
> > > > context 
> > > > */
> > > > -	atomic = in_atomic();
> > > > +	atomic = faulthandler_disabled();
> > > >  
> > > >  	if (!atomic)
> > > >  		down_read(&current->mm->mmap_sem);
> > > 
> > > Yeah, that fixes the problem I was seeing.
> > 
> > Good!  Then I'll add it to RMK's patch system.  May I add a tested-by 
> > tag with your name?
> 
> Yes

Queued as patch #8414/1.


Nicolas