Re: [PATCH net-next] scm: optimize put_cmsg()

Re: [PATCH net-next] scm: optimize put_cmsg()

[Naresh Kamboju]

Hi Eric,

On Fri, 16 Apr 2021 at 23:27, Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu, 15 Apr 2021 10:37:53 -0700 Eric Dumazet wrote:
> > From: Eric Dumazet <edumazet@google.com>
> >
> > Calling two copy_to_user() for very small regions has very high overhead.
> >
> > Switch to inlined unsafe_put_user() to save one stac/clac sequence,
> > and avoid copy_to_user().
> >
> > Signed-off-by: Eric Dumazet <edumazet@google.com>
> > Cc: Soheil Hassas Yeganeh <soheil@google.com>
>
> Hi Eric!
>
> This appears to break boot on my systems.

I have been noticing this problem.

>
> IDK how exactly, looks like systemd gets stuck waiting for nondescript
> services to start in initramfs. I have lots of debug enabled and didn't
> spot anything of note in kernel logs.

We (LKFT) are still seeing this problem only on arm architecture on
next-20210416 tag onwards. our bisect script points to this commit.

Steps to reproduce:
- build linux next latest next-20210423 tag with below config
     -  kernel-config:
https://builds.tuxbuild.com/1reqrnNLnHEX9uEZFngRfaoJa9E/config
- boot qemu-arm with below command
   -  /usr/bin/qemu-system-aarch64 -cpu host,aarch64=off -machine
virt-2.10,accel=kvm -nographic -net
nic,model=virtio,macaddr=BA:DD:AD:CC:09:04 -net tap -m 2048 -monitor
none -kernel zImage --append "console=ttyAMA0 root=/dev/vda rw" -hda
rpb-console-image-lkft-am57xx-evm-20201022181203-3085.rootfs.ext4 -m
4096 -smp 2 -nographic

- After the mount rootfs - the systemd gets stuck

>
> I'll try to poke at this more, but LMK if you have any ideas. The
> commit looks "obviously correct" :S

May I request to investigate this on arm architecture.
The qemu_arm boot failed link,
https://lkft.validation.linaro.org/scheduler/job/2565371#L540

The qemu_arm boot pass after the reverting this patch,
    commit 38ebcf5096a86762b82262e96b2c8b170fe79040
    scm: optimize put_cmsg()

on the latest linux next tags i have to revert two commits.
   "scm: fix a typo in put_cmsg()"
   "scm: optimize put_cmsg()"

Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org>

--
Linaro LKFT
https://lkft.linaro.org

Re: [PATCH net-next] scm: optimize put_cmsg()

[Eric Dumazet]

On Sun, Apr 25, 2021 at 9:59 PM Naresh Kamboju
<naresh.kamboju@linaro.org> wrote:
>
> Hi Eric,
>
> On Fri, 16 Apr 2021 at 23:27, Jakub Kicinski <kuba@kernel.org> wrote:
> >
> > On Thu, 15 Apr 2021 10:37:53 -0700 Eric Dumazet wrote:
> > > From: Eric Dumazet <edumazet@google.com>
> > >
> > > Calling two copy_to_user() for very small regions has very high overhead.
> > >
> > > Switch to inlined unsafe_put_user() to save one stac/clac sequence,
> > > and avoid copy_to_user().
> > >
> > > Signed-off-by: Eric Dumazet <edumazet@google.com>
> > > Cc: Soheil Hassas Yeganeh <soheil@google.com>
> >
> > Hi Eric!
> >
> > This appears to break boot on my systems.
>
> I have been noticing this problem.
>
> >
> > IDK how exactly, looks like systemd gets stuck waiting for nondescript
> > services to start in initramfs. I have lots of debug enabled and didn't
> > spot anything of note in kernel logs.
>
> We (LKFT) are still seeing this problem only on arm architecture on
> next-20210416 tag onwards. our bisect script points to this commit.
>
> Steps to reproduce:
> - build linux next latest next-20210423 tag with below config
>      -  kernel-config:
> https://builds.tuxbuild.com/1reqrnNLnHEX9uEZFngRfaoJa9E/config
> - boot qemu-arm with below command
>    -  /usr/bin/qemu-system-aarch64 -cpu host,aarch64=off -machine
> virt-2.10,accel=kvm -nographic -net
> nic,model=virtio,macaddr=BA:DD:AD:CC:09:04 -net tap -m 2048 -monitor
> none -kernel zImage --append "console=ttyAMA0 root=/dev/vda rw" -hda
> rpb-console-image-lkft-am57xx-evm-20201022181203-3085.rootfs.ext4 -m
> 4096 -smp 2 -nographic
>
> - After the mount rootfs - the systemd gets stuck
>
> >
> > I'll try to poke at this more, but LMK if you have any ideas. The
> > commit looks "obviously correct" :S
>
> May I request to investigate this on arm architecture.
> The qemu_arm boot failed link,
> https://lkft.validation.linaro.org/scheduler/job/2565371#L540
>
> The qemu_arm boot pass after the reverting this patch,
>     commit 38ebcf5096a86762b82262e96b2c8b170fe79040
>     scm: optimize put_cmsg()

Well, as already reported, this patch had an obvious typo.

Fixed later by "scm: fix a typo in put_cmsg()"

Can you trace put_cmsg() and check that systemd passes an aligned
control buffer ?

Kernel was indeed able to handle arbitrary alignment, but why the
application would
force slow copyout() (alignment mismatch between source/destination buffers)
is quite strange.



>
> on the latest linux next tags i have to revert two commits.
>    "scm: fix a typo in put_cmsg()"
>    "scm: optimize put_cmsg()"
>
> Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org>
>
> --
> Linaro LKFT
> https://lkft.linaro.org