Commit 8ae030c34 breaks USB on the Raspberry Pi 4

Commit 8ae030c34 breaks USB on the Raspberry Pi 4

[Michael Stapelberg]

[re-sending in plain text mode]

Hey!

Linux 5.14 broke USB on the Raspberry Pi 4 for me.

See https://github.com/gokrazy/kernel/commit/e06b78841ba4d2ce99a3f368cb8407951267e082
for more details regarding symptoms and a dmesg diff.

I used git-bisect to find the culprit, and it identified this commit:
https://github.com/torvalds/linux/commit/8ae030c34dce4f5764e945b325e8dc4d2adef044

Indeed, when I compare the kernel config before/with the commit, I see:

--- /tmp/config-WORKING 2021-09-14 23:04:03.180316634 +0200
+++ /tmp/config-BROKEN 2021-09-14 23:05:57.475549129 +0200
@@ -8101,7 +8101,7 @@
 CONFIG_RESET_MESON_AUDIO_ARB=m
 CONFIG_RESET_QCOM_AOSS=y
 CONFIG_RESET_QCOM_PDC=m
-CONFIG_RESET_RASPBERRYPI=y
+CONFIG_RESET_RASPBERRYPI=m
 CONFIG_RESET_SCMI=y
 CONFIG_RESET_SIMPLE=y
 CONFIG_RESET_SUNXI=y

So it seems like this innocuous-looking arm64 defconfig change has
accidentally (?) demoted CONFIG_RESET_RASPBERRYPI=y to
CONFIG_RESET_RASPBERRYPI=m in the arm64 defconfig.

Was this intentional? If not, I’d be happy if you could send a fix for
this change.

Thank you!
Best regards
Michael

PS: I’m building the Linux kernel using this config addition to the
arm64 defconfig:
https://github.com/gokrazy/kernel/blob/master/cmd/gokr-build-kernel/build.go#L19,
just in case it matters…

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Commit 8ae030c34 breaks USB on the Raspberry Pi 4

[Florian Fainelli]

+Nicolas, Stefan,

On 9/14/21 2:28 PM, Michael Stapelberg wrote:
> [re-sending in plain text mode]
> 
> Hey!
> 
> Linux 5.14 broke USB on the Raspberry Pi 4 for me.
> 
> See https://github.com/gokrazy/kernel/commit/e06b78841ba4d2ce99a3f368cb8407951267e082
> for more details regarding symptoms and a dmesg diff.
> 
> I used git-bisect to find the culprit, and it identified this commit:
> https://github.com/torvalds/linux/commit/8ae030c34dce4f5764e945b325e8dc4d2adef044
> 
> Indeed, when I compare the kernel config before/with the commit, I see:
> 
> --- /tmp/config-WORKING 2021-09-14 23:04:03.180316634 +0200
> +++ /tmp/config-BROKEN 2021-09-14 23:05:57.475549129 +0200
> @@ -8101,7 +8101,7 @@
>  CONFIG_RESET_MESON_AUDIO_ARB=m
>  CONFIG_RESET_QCOM_AOSS=y
>  CONFIG_RESET_QCOM_PDC=m
> -CONFIG_RESET_RASPBERRYPI=y
> +CONFIG_RESET_RASPBERRYPI=m
>  CONFIG_RESET_SCMI=y
>  CONFIG_RESET_SIMPLE=y
>  CONFIG_RESET_SUNXI=y
> 
> So it seems like this innocuous-looking arm64 defconfig change has
> accidentally (?) demoted CONFIG_RESET_RASPBERRYPI=y to
> CONFIG_RESET_RASPBERRYPI=m in the arm64 defconfig.
> 
> Was this intentional? If not, I’d be happy if you could send a fix for
> this change.

From a running system, can you see if reset-raspberry.ko is included in
your root filesystem? If not, this would definitively be an issue since
that driver is needed in order for xHCI to work on the Pi 4. There does
not appear to be any explicit dependency between reset-raspberry and
xhci-pci.ko, so without manually "front" loading the former before the
latter, I don't think we can have automatic dependencies.

> 
> Thank you!
> Best regards
> Michael
> 
> PS: I’m building the Linux kernel using this config addition to the
> arm64 defconfig:
> https://github.com/gokrazy/kernel/blob/master/cmd/gokr-build-kernel/build.go#L19,
> just in case it matters…
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 


-- 
Florian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Commit 8ae030c34 breaks USB on the Raspberry Pi 4

[Michael Stapelberg]

Thanks for your mail.

My system does not use any Linux kernel modules (=m), all required
code is compiled into the kernel itself (=y).
The root file system does not contain any .ko files.


On Tue, 14 Sept 2021 at 23:47, Florian Fainelli <f.fainelli@gmail.com> wrote:
>
> +Nicolas, Stefan,
>
> On 9/14/21 2:28 PM, Michael Stapelberg wrote:
> > [re-sending in plain text mode]
> >
> > Hey!
> >
> > Linux 5.14 broke USB on the Raspberry Pi 4 for me.
> >
> > See https://github.com/gokrazy/kernel/commit/e06b78841ba4d2ce99a3f368cb8407951267e082
> > for more details regarding symptoms and a dmesg diff.
> >
> > I used git-bisect to find the culprit, and it identified this commit:
> > https://github.com/torvalds/linux/commit/8ae030c34dce4f5764e945b325e8dc4d2adef044
> >
> > Indeed, when I compare the kernel config before/with the commit, I see:
> >
> > --- /tmp/config-WORKING 2021-09-14 23:04:03.180316634 +0200
> > +++ /tmp/config-BROKEN 2021-09-14 23:05:57.475549129 +0200
> > @@ -8101,7 +8101,7 @@
> >  CONFIG_RESET_MESON_AUDIO_ARB=m
> >  CONFIG_RESET_QCOM_AOSS=y
> >  CONFIG_RESET_QCOM_PDC=m
> > -CONFIG_RESET_RASPBERRYPI=y
> > +CONFIG_RESET_RASPBERRYPI=m
> >  CONFIG_RESET_SCMI=y
> >  CONFIG_RESET_SIMPLE=y
> >  CONFIG_RESET_SUNXI=y
> >
> > So it seems like this innocuous-looking arm64 defconfig change has
> > accidentally (?) demoted CONFIG_RESET_RASPBERRYPI=y to
> > CONFIG_RESET_RASPBERRYPI=m in the arm64 defconfig.
> >
> > Was this intentional? If not, I’d be happy if you could send a fix for
> > this change.
>
> From a running system, can you see if reset-raspberry.ko is included in
> your root filesystem? If not, this would definitively be an issue since
> that driver is needed in order for xHCI to work on the Pi 4. There does
> not appear to be any explicit dependency between reset-raspberry and
> xhci-pci.ko, so without manually "front" loading the former before the
> latter, I don't think we can have automatic dependencies.
>
> >
> > Thank you!
> > Best regards
> > Michael
> >
> > PS: I’m building the Linux kernel using this config addition to the
> > arm64 defconfig:
> > https://github.com/gokrazy/kernel/blob/master/cmd/gokr-build-kernel/build.go#L19,
> > just in case it matters…
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> >
>
>
> --
> Florian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Commit 8ae030c34 breaks USB on the Raspberry Pi 4

[nicolas saenz julienne]

Hi Michael,

On Wed, 2021-09-15 at 08:33 +0200, Michael Stapelberg wrote:
> Thanks for your mail.
> 
> My system does not use any Linux kernel modules (=m), all required
> code is compiled into the kernel itself (=y).
> The root file system does not contain any .ko files.

Are you building your own kernels? Then I suggest to configure the
raspberrypi-reset driver as builtin (=y) so it's available in your kernel
image.

Could you provide more information on how your system/rootfs is built and
setup?

Regards,
Nicolas


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Commit 8ae030c34 breaks USB on the Raspberry Pi 4

[Michael Stapelberg]

Hey Nicolas

On Wed, 15 Sept 2021 at 09:42, nicolas saenz julienne <nsaenz@kernel.org> wrote:
>
> Hi Michael,
>
> On Wed, 2021-09-15 at 08:33 +0200, Michael Stapelberg wrote:
> > Thanks for your mail.
> >
> > My system does not use any Linux kernel modules (=m), all required
> > code is compiled into the kernel itself (=y).
> > The root file system does not contain any .ko files.
>
> Are you building your own kernels? Then I suggest to configure the
> raspberrypi-reset driver as builtin (=y) so it's available in your kernel
> image.

Yes, I have already done that. I was more interested in fixing this
problem for others, and/or avoiding similar problems in the future.

>
> Could you provide more information on how your system/rootfs is built and
> setup?

Please see https://gokrazy.org/, specifically https://gokrazy.org/quickstart/.
The source for the gokr-packer tool is at
https://github.com/gokrazy/tools/tree/master/cmd/gokr-packer

In summary, my tool builds a read-only SquashFS root file system that
directly starts processes written in Go.
The boot partition (FAT32 for the Raspberry Pi) contains only the
Linux kernel and Raspberry Pi firmware files.

>
> Regards,
> Nicolas
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Commit 8ae030c34 breaks USB on the Raspberry Pi 4

[nicolas saenz julienne]

On Wed, 2021-09-15 at 09:55 +0200, Michael Stapelberg wrote:
> Hey Nicolas
> 
> On Wed, 15 Sept 2021 at 09:42, nicolas saenz julienne <nsaenz@kernel.org> wrote:
> > 
> > Hi Michael,
> > 
> > On Wed, 2021-09-15 at 08:33 +0200, Michael Stapelberg wrote:
> > > Thanks for your mail.
> > > 
> > > My system does not use any Linux kernel modules (=m), all required
> > > code is compiled into the kernel itself (=y).
> > > The root file system does not contain any .ko files.
> > 
> > Are you building your own kernels? Then I suggest to configure the
> > raspberrypi-reset driver as builtin (=y) so it's available in your kernel
> > image.
> 
> Yes, I have already done that. I was more interested in fixing this
> problem for others, and/or avoiding similar problems in the future.
> 
> > 
> > Could you provide more information on how your system/rootfs is built and
> > setup?
> 
> Please see https://gokrazy.org/, specifically https://gokrazy.org/quickstart/.
> The source for the gokr-packer tool is at
> https://github.com/gokrazy/tools/tree/master/cmd/gokr-packer
> 
> In summary, my tool builds a read-only SquashFS root file system that
> directly starts processes written in Go.
> The boot partition (FAT32 for the Raspberry Pi) contains only the
> Linux kernel and Raspberry Pi firmware files.

Thanks for the info.

IMO the solution is to maintain your own configuration. The decision not to use
kernel modules, even if reasonable given the scope of gokrazy, is a big
deviation from how the kernel is regularly consumed. The majority of the arm64
defconfig users need the kernel image to be as small as possible to avoid
bloating the boot partition, and allowing for reasonable boot speeds (imagine
having to load a 1GB kernel image from the SD card).

Regards,
Nicolas


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Commit 8ae030c34 breaks USB on the Raspberry Pi 4

[Michael Stapelberg]

On Wed, 15 Sept 2021 at 10:54, nicolas saenz julienne <nsaenz@kernel.org> wrote:
>
> On Wed, 2021-09-15 at 09:55 +0200, Michael Stapelberg wrote:
> > Hey Nicolas
> >
> > On Wed, 15 Sept 2021 at 09:42, nicolas saenz julienne <nsaenz@kernel.org> wrote:
> > >
> > > Hi Michael,
> > >
> > > On Wed, 2021-09-15 at 08:33 +0200, Michael Stapelberg wrote:
> > > > Thanks for your mail.
> > > >
> > > > My system does not use any Linux kernel modules (=m), all required
> > > > code is compiled into the kernel itself (=y).
> > > > The root file system does not contain any .ko files.
> > >
> > > Are you building your own kernels? Then I suggest to configure the
> > > raspberrypi-reset driver as builtin (=y) so it's available in your kernel
> > > image.
> >
> > Yes, I have already done that. I was more interested in fixing this
> > problem for others, and/or avoiding similar problems in the future.
> >
> > >
> > > Could you provide more information on how your system/rootfs is built and
> > > setup?
> >
> > Please see https://gokrazy.org/, specifically https://gokrazy.org/quickstart/.
> > The source for the gokr-packer tool is at
> > https://github.com/gokrazy/tools/tree/master/cmd/gokr-packer
> >
> > In summary, my tool builds a read-only SquashFS root file system that
> > directly starts processes written in Go.
> > The boot partition (FAT32 for the Raspberry Pi) contains only the
> > Linux kernel and Raspberry Pi firmware files.
>
> Thanks for the info.
>
> IMO the solution is to maintain your own configuration. The decision not to use
> kernel modules, even if reasonable given the scope of gokrazy, is a big
> deviation from how the kernel is regularly consumed. The majority of the arm64
> defconfig users need the kernel image to be as small as possible to avoid
> bloating the boot partition, and allowing for reasonable boot speeds (imagine
> having to load a 1GB kernel image from the SD card).

Yep, effectively I am maintaining my own configuration, but I still
run defconfig before applying my own additions to .config.

I think that in general I do want the changes that defconfig brings,
so defconfig+additions is the only way I see, and I thought this was
best practice?

Are you saying there might be a better option? Or perhaps the best
move is to just support modules in gokrazy after all.

Thanks for your advice,

>
> Regards,
> Nicolas
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel