The patch was not enough, and my comments on that bug have been ignored - the
key bit is:

========================================================================

I've tracked part of the problem down to
xfsettingsd/displays.c:xfce_displays_helper_screen_on_event, L462
xfce_displays_helper_disable_crtc call - when this call is commented out, the
monitor is always configured fine (and looking at xfsettingsd's debug output,
in both the success and failure case there are plenty of events happening where
all monitors are connected at the end to allow for proper configuration).

========================================================================

Commenting this out and I have had no further issues. The current state is good
enough for me - you could counter and say that there should be no disconnect
event in the first place, but I don't plan to look into anything further
currently.