OPNSense running in domU has no network connectivity on 5.15.29+

OPNSense running in domU has no network connectivity on 5.15.29+

[Colton Reeder]

Hello,

I am running the FreeBSD-based router OS OPNSense in a domU. I
recently upgraded my dom0 kernel from 5.15.26 to 5.15.32 and with the
new kernel, OPNSense had no connectivity. I downloaded from kernel.org
5.15.26-32, built and installed each version and booted them
consecutively until I found the version that no longer worked. It
turned out to be 5.15.29.

I looked through the change log of 5.15.29 and found two commits for xen-netback

commit 2708ceb4e5cc84ef179bad25a2d7890573ef78be commit
fe39ab30dcc204e321c2670cc1cf55904af35d01

I reverted these changes (a revert of a revert, yes)  in 5.15.32,
built and installed. Now the network works. Now I dont know enough to
know thats for sure the right fix. Maybe I have a config issue, I dont
know, but reverting that change fixes the problem. What should I do?
I was asked to provide xenstore -ls https://pastebin.com/hHPWgrEy

Re: OPNSense running in domU has no network connectivity on 5.15.29+

[Roger Pau Monné]

On Tue, May 03, 2022 at 12:36:43PM -0700, Colton Reeder wrote:
> Hello,
> 
> I am running the FreeBSD-based router OS OPNSense in a domU. I
> recently upgraded my dom0 kernel from 5.15.26 to 5.15.32 and with the
> new kernel, OPNSense had no connectivity. I downloaded from kernel.org
> 5.15.26-32, built and installed each version and booted them
> consecutively until I found the version that no longer worked. It
> turned out to be 5.15.29.
> 
> I looked through the change log of 5.15.29 and found two commits for xen-netback
> 
> commit 2708ceb4e5cc84ef179bad25a2d7890573ef78be commit
> fe39ab30dcc204e321c2670cc1cf55904af35d01
> 
> I reverted these changes (a revert of a revert, yes)  in 5.15.32,
> built and installed. Now the network works. Now I dont know enough to
> know thats for sure the right fix.

Could you try if reverting only one of those fixes your issue?

> Maybe I have a config issue, I dont
> know, but reverting that change fixes the problem. What should I do?
> I was asked to provide xenstore -ls https://pastebin.com/hHPWgrEy

It's better to post the output of `xenstore-ls -fp`, as that's way
easier to read.

So it's only OPNSense that's affected, other VMs run fine?

Do you get any output from Linux dmesg?

From the output of xenstore that you pasted, you do have another
guests that seems to be running fine regarding network, are there any
differences in the configuration file?

FWIW, it seems like the netback instances are stuck in state 2.

I think we need the guest config file, plus the output from `xl
-vvv create config_file.cfg`

Thanks, Roger.

Re: OPNSense running in domU has no network connectivity on 5.15.29+

[Colton Reeder]

https://pastebin.com/uqME2J8W Theres the configuration from both VMs.
You are correct, Ive never had any issues with the Linux PV guest "Services"

I will work on getting the other information you requested as soon as I can.

On Tue, May 10, 2022 at 4:18 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
>
> On Tue, May 03, 2022 at 12:36:43PM -0700, Colton Reeder wrote:
> > Hello,
> >
> > I am running the FreeBSD-based router OS OPNSense in a domU. I
> > recently upgraded my dom0 kernel from 5.15.26 to 5.15.32 and with the
> > new kernel, OPNSense had no connectivity. I downloaded from kernel.org
> > 5.15.26-32, built and installed each version and booted them
> > consecutively until I found the version that no longer worked. It
> > turned out to be 5.15.29.
> >
> > I looked through the change log of 5.15.29 and found two commits for xen-netback
> >
> > commit 2708ceb4e5cc84ef179bad25a2d7890573ef78be commit
> > fe39ab30dcc204e321c2670cc1cf55904af35d01
> >
> > I reverted these changes (a revert of a revert, yes)  in 5.15.32,
> > built and installed. Now the network works. Now I dont know enough to
> > know thats for sure the right fix.
>
> Could you try if reverting only one of those fixes your issue?
>
> > Maybe I have a config issue, I dont
> > know, but reverting that change fixes the problem. What should I do?
> > I was asked to provide xenstore -ls https://pastebin.com/hHPWgrEy
>
> It's better to post the output of `xenstore-ls -fp`, as that's way
> easier to read.
>
> So it's only OPNSense that's affected, other VMs run fine?
>
> Do you get any output from Linux dmesg?
>
> From the output of xenstore that you pasted, you do have another
> guests that seems to be running fine regarding network, are there any
> differences in the configuration file?
>
> FWIW, it seems like the netback instances are stuck in state 2.
>
> I think we need the guest config file, plus the output from `xl
> -vvv create config_file.cfg`
>
> Thanks, Roger.