I think I know what happened. When I was rebooting, I still had the older
kernel running (without the fix), to actually test the new one, so probably the
bug still kicked in after reboot.

I now rebooted a few more times already from the patched kernel, and I don't
see the issue anymore. It also doesn't happen with boot from complete off
state.

Thanks!