Hyperv netvsc - regression for 32-PAE kernel

Hyperv netvsc - regression for 32-PAE kernel

[Juliana Rodrigueiro]

Hi all.

I have a custom linux OS vm running kernel 3.14 (32b with or without PAE) in 
windows 2012 R2. The vm has one Network Adapter and is generation 1. With this 
setup everything runs fine.

The problem started when I tried to update to kernel 4.19. The Synthetic 
network adapter driver does not successfully loads during boot and then the 
machine gets stuck.

If I remove the Network Adapter and add a Legacy one instead, the system runs 
normally. However, this implies an unacceptable performance regression for my 
use case.

I manage to boot the vm with the Network Adapter by adding "hv_netvsc" to the 
blacklist, so I can inspect the system. Manually running "modprobe -v 
hv_netvsc" doesn't show any errors, just the "instmod" for ucs2_string and 
hv_netvsc, and then hangs forever. The "dmesg" output shows the following 
problems:

[  994.830251] hv_netvsc 0969e9e1-1392-4ed6-a230-d5db70c76a3c (unnamed 
net_device) (uninitialized): 0x0 (len 0)
[  994.830306] hv_netvsc 0969e9e1-1392-4ed6-a230-d5db70c76a3c (unnamed 
net_device) (uninitialized): unhandled rndis message (type 0 len 0)
[  994.830435] hv_netvsc 0969e9e1-1392-4ed6-a230-d5db70c76a3c (unnamed 
net_device) (uninitialized): 0x0 (len 0)
[  994.830440] hv_netvsc 0969e9e1-1392-4ed6-a230-d5db70c76a3c (unnamed 
net_device) (uninitialized): unhandled rndis message (type 0 len 0)


The Network Adapter was "Not connected" during these error messages, but when 
connected to a Virtual Switch the errors are the same, except doubled, so I 
would have four "unhandled rndis message".

I tested kernel 4.19 without PAE, the module is loaded without problems and 
those error messages never appear. 

I also tested other stable kernel versions, for example 4.14.114, and this one 
actually works fine with PAE. At this point, it looked like a bisect could 
help me to get to the offending changes and to understand the problem.

So I got to the following commit:

commit 6ba34171bcbd10321c6cf554e0c1144d170f9d1a
Author: Michael Kelley <mikelley@microsoft.com>
Date:   Thu Aug 2 03:08:24 2018 +0000

    Drivers: hv: vmbus: Remove use of slow_virt_to_phys()
    
    slow_virt_to_phys() is only implemented for arch/x86.
    Remove its use in arch independent Hyper-V drivers, and
    replace with test for vmalloc() address followed by
    appropriate v-to-p function. This follows the typical
    pattern of other drivers and avoids the need to implement
    slow_virt_to_phys() for Hyper-V on ARM64.
    
    Signed-off-by: Michael Kelley <mikelley@microsoft.com>
    Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>


The catch is that slow_virt_to_phys has a special trick implemented in order 
to keep specifically 32-PAE kernel working, it is explained in a comment 
inside the function.

Reverting this commit makes the kernel 4.19 32-bit PAE work again. However I 
believe a better solution might exist.

Comments are very much appreciated.

Cheers!
Julie R.

RE: Hyperv netvsc - regression for 32-PAE kernel

[Michael Kelley]

From: Juliana Rodrigueiro <juliana.rodrigueiro@intra2net.com> Sent: Thursday, May 2, 2019 9:14 AM
> 
> So I got to the following commit:
> 
> commit 6ba34171bcbd10321c6cf554e0c1144d170f9d1a
> Author: Michael Kelley <mikelley@microsoft.com>
> Date:   Thu Aug 2 03:08:24 2018 +0000
> 
>     Drivers: hv: vmbus: Remove use of slow_virt_to_phys()
> 
>     slow_virt_to_phys() is only implemented for arch/x86.
>     Remove its use in arch independent Hyper-V drivers, and
>     replace with test for vmalloc() address followed by
>     appropriate v-to-p function. This follows the typical
>     pattern of other drivers and avoids the need to implement
>     slow_virt_to_phys() for Hyper-V on ARM64.
> 
>     Signed-off-by: Michael Kelley <mikelley@microsoft.com>
>     Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
>     Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> The catch is that slow_virt_to_phys has a special trick implemented in order
> to keep specifically 32-PAE kernel working, it is explained in a comment
> inside the function.
> 
> Reverting this commit makes the kernel 4.19 32-bit PAE work again. However I
> believe a better solution might exist.
> 
> Comments are very much appreciated.
> 

Julie -- thanks for tracking down the cause of the issue.  I'll try to
look at this tomorrow and propose a solution.

Michael Kelley

RE: Hyperv netvsc - regression for 32-PAE kernel

[Dexuan Cui]

> From: linux-hyperv-owner@vger.kernel.org
> <linux-hyperv-owner@vger.kernel.org> On Behalf Of Michael Kelley
> Sent: Thursday, May 2, 2019 3:24 PM
> To: Juliana Rodrigueiro <juliana.rodrigueiro@intra2net.com>;
> linux-hyperv@vger.kernel.org
> Cc: netdev@vger.kernel.org
> Subject: RE: Hyperv netvsc - regression for 32-PAE kernel
> 
> From: Juliana Rodrigueiro <juliana.rodrigueiro@intra2net.com> Sent: Thursday,
> May 2, 2019 9:14 AM
> >
> > So I got to the following commit:
> >
> > commit 6ba34171bcbd10321c6cf554e0c1144d170f9d1a
> > Author: Michael Kelley <mikelley@microsoft.com>
> > Date:   Thu Aug 2 03:08:24 2018 +0000
> >
> >     Drivers: hv: vmbus: Remove use of slow_virt_to_phys()
> >
> >     slow_virt_to_phys() is only implemented for arch/x86.
> >     Remove its use in arch independent Hyper-V drivers, and
> >     replace with test for vmalloc() address followed by
> >     appropriate v-to-p function. This follows the typical
> >     pattern of other drivers and avoids the need to implement
> >     slow_virt_to_phys() for Hyper-V on ARM64.
> >
> >     Signed-off-by: Michael Kelley <mikelley@microsoft.com>
> >     Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> >     Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >
> > The catch is that slow_virt_to_phys has a special trick implemented in order
> > to keep specifically 32-PAE kernel working, it is explained in a comment
> > inside the function.
> >
> > Reverting this commit makes the kernel 4.19 32-bit PAE work again. However
> I
> > believe a better solution might exist.
> >
> > Comments are very much appreciated.
> >
> 
> Julie -- thanks for tracking down the cause of the issue.  I'll try to
> look at this tomorrow and propose a solution.
> 
> Michael Kelley

Hi Juliana,
Can you please try the below one-line patch? 

It should fix the issue.

Thanks,
-- Dexuan

diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
index 23381c4..aaaee5f 100644
--- a/drivers/hv/channel.c
+++ b/drivers/hv/channel.c
@@ -38,7 +38,7 @@

 static unsigned long virt_to_hvpfn(void *addr)
 {
-       unsigned long paddr;
+       phys_addr_t paddr;

        if (is_vmalloc_addr(addr))
                paddr = page_to_phys(vmalloc_to_page(addr)) +

Re: Hyperv netvsc - regression for 32-PAE kernel

[Juliana Rodrigueiro]

Hi Dexuan.

> Can you please try the below one-line patch?

Nice, easy one liner and it works well for me.

I hope this patch will be applied.

Thank you!

Julie R.

RE: Hyperv netvsc - regression for 32-PAE kernel

[Dexuan Cui]

> From: Juliana Rodrigueiro <juliana.rodrigueiro@intra2net.com>
> Sent: Monday, May 6, 2019 3:56 AM
> To: Dexuan Cui <decui@microsoft.com>
> Cc: linux-hyperv@vger.kernel.org
> Subject: Re: Hyperv netvsc - regression for 32-PAE kernel
> 
> Hi Dexuan.
> 
> > Can you please try the below one-line patch?
> 
> Nice, easy one liner and it works well for me.
> 
> I hope this patch will be applied.
> 
> Thank you!
> 
> Julie R.

Hi Julie,

Thanks for testing the patch!

I'm going to formally send the patch to the list.

Can I add a "Reported-and-tested-by" tag for you?

Thanks,
-- Dexuan