The case we had is large physical machines with around 1000 disks. We did not see the issue on the smaller cpu/disked physicals and/or vm's. It seemed like both high cpu counts and high disk counts were needed, but in our environment both of those are usually together. The smallest machines that had the issues had 72 threads (36 actual cores). And the disk devices were all SSD SAN luns so I would expect all of the devices to respond to and return IO requests in under .3ms under normal conditions. They were also all partitioned and multipath'ed. 90% of the disk would not have had any LVM on them at all but would have been at least initial scanned by something, but the systemd LVM parts where what was timing out, and based on the time udev was getting in the 90-120 seconds (90 minutes of time) it very much seemed to be having serious cpu time issues doing something.
I have done some simple tests forking a bunch of tests forking off a bunch of /usr/sbin/lvm pvscan --cache major:minor in the background and in parallel rapidly and cannot get it to really act badly except with numbers that are >20000.
And if I am reading the direct case pvscan that is fast about the only thing that differs is that it does not spawn off lots of processes and events and just does the pvscan once. Between udev and systemd I am not clear on how many different events have to be handled and how many of those events need to spawn new threads and/or fork new processes off. Something doing one of those 2 things or both would seem to have been the cause of the issue I saw in the past.
When it has difficult booting up like this what does ps axuS | grep udev look like time wise?