* Wintun NeighborDiscoverySupported @ 2021-09-09 14:32 Brad Spencer 2021-09-09 17:42 ` Jason A. Donenfeld 0 siblings, 1 reply; 6+ messages in thread From: Brad Spencer @ 2021-09-09 14:32 UTC (permalink / raw) To: WireGuard mailing list We have noticed that Windows seems to try to send ARP requests over Wintun interfaces. In our configurations, these don't go anywhere and get no responses, but the ARP table fills up with addresses. For example: $ arp -a -N 10.0.0.100 |wc -l 387 Our Wintun interfaces are created with these properties: PS C:\> Get-NetIPInterface -InterfaceIndex 69 |Select-Object -Property AddressFamily,NeighborDiscoverySupported,NeighborUnreachabilityDetection |fl AddressFamily : IPv6 NeighborDiscoverySupported : Yes NeighborUnreachabilityDetection : Enabled AddressFamily : IPv4 NeighborDiscoverySupported : Yes NeighborUnreachabilityDetection : Enabled We _think_ that the NeighborDiscoverySupported property being Yes means that Windows issues ARP requests for addresses on the Wintun interface. (The second property controls another neighbour behaviour that perhaps could also be disabled for Wintun interfaces.) Microsoft's documentation for SetIpInterfaceEntry() claims that these properties are set "by the network stack", but offers no hint as to what part of the network stack does this. It does not seem possible to change these properties on an existing interface. Are these properties that either the Wintun driver or user-space API is able to control? Should they be set to No and Disabled? I don't see any use of the driver variant of SetIpInterfaceEntry() or MIB_IPINTERFACE_ROW in Wintun's driver or API code, so I'm not sure when these properties get determined. -- Brad Spencer ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Wintun NeighborDiscoverySupported 2021-09-09 14:32 Wintun NeighborDiscoverySupported Brad Spencer @ 2021-09-09 17:42 ` Jason A. Donenfeld 2021-09-09 18:15 ` Brad Spencer 0 siblings, 1 reply; 6+ messages in thread From: Jason A. Donenfeld @ 2021-09-09 17:42 UTC (permalink / raw) To: Brad Spencer; +Cc: WireGuard mailing list Hi Brad, That sure is interesting. Indeed UseNeighborUnreachabilityDetection and SupportsNeighborDiscovery can't be set with SetIpInterfaceEntry. I haven't (yet?) found any way for the driver itself to indicate that these should be false, either, via OIDs or similar. This might require some frustrating reverse engineering. I remember noticing this a long time ago, but I deemed it "annoying but harmless". However, you now mention: On Thu, Sep 9, 2021 at 4:33 PM Brad Spencer <bspencer@blackberry.com> wrote: > but the ARP table fills up with addresses. For example: > > $ arp -a -N 10.0.0.100 |wc -l > 387 If that grows indefinitely, that sounds... bad. So this might be worth looking into again. One thing about your message, though, raised a question in my mind, but I'm not sure whether its an artifact of your wording or a real thing you observed: > We have noticed that Windows seems to try to send ARP requests over > Wintun interfaces. In our configurations, these don't go anywhere and > get no responses, Indeed the ARP table fills up, as shown above, but I'm wondering how you're observing ARP requests exactly. ARP is a layer 2 protocol, and Wintun (and WireGuardNT) are layer 3 devices that should, in theory at least, not have anything to do with ARP packets. So how exactly were you "seeing" the ARP requests on the Wintun interface? Did wireshark show it? Or did you read from the Wintun ring and actually see an ARP frame? Or something else? Or was this just a manner of speaking and you didn't actually observe ARP frames themselves? Another small question: > We _think_ that the NeighborDiscoverySupported property being Yes means > that Windows issues ARP requests for addresses on the Wintun interface. That seems like a good intuition. I'm wondering whether that's something you're assuming or something you read on a Microsoft website. I ask because this might provide a good entry point for whatever reverse engineering I wind up doing to fix this. Regards, Jason ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Wintun NeighborDiscoverySupported 2021-09-09 17:42 ` Jason A. Donenfeld @ 2021-09-09 18:15 ` Brad Spencer 2021-09-09 20:23 ` Alan Graham 0 siblings, 1 reply; 6+ messages in thread From: Brad Spencer @ 2021-09-09 18:15 UTC (permalink / raw) To: Jason A. Donenfeld; +Cc: WireGuard mailing list On 2021-09-09 2:42 p.m., Jason A. Donenfeld wrote: > So how exactly were > you "seeing" the ARP requests on the Wintun interface? Did wireshark > show it? Or did you read from the Wintun ring and actually see an ARP > frame? Or something else? Or was this just a manner of speaking and > you didn't actually observe ARP frames themselves? You're right to suspect that I was speaking imprecisely here. We have never seen an ARP request appear on the Wintun interface! I meant to say that we have noticed the ARP table for the Wintun interface accumulating entries. >> We _think_ that the NeighborDiscoverySupported property being Yes means >> that Windows issues ARP requests for addresses on the Wintun interface. > That seems like a good intuition. I'm wondering whether that's > something you're assuming or something you read on a Microsoft > website. I ask because this might provide a good entry point for > whatever reverse engineering I wind up doing to fix this. I pieced this together from a few scraps. On the MIB_IPINTERFACE_ROW page[1], the docs only tersely say: "A value that specifies if the IP interface support neighbor discovery." Microsoft seems to use the same terminology when documenting SetIpNetEntry2()[2]: "The SetIpNetEntry2 function sets the physical address of an existing neighbor IP address entry on the local computer." And then, most importantly, MIB_IPNET_ROW2, the structure used by that function says this in its Remarks section[3]: "For IPv4, this includes addresses determined used the Address Resolution Protocol (ARP). For IPv6, this includes addresses determined using the Neighbor Discovery (ND) protocol for IPv6 as specified in RFC 2461. " So, it seems that the "NetEntry" APIs are those that deal with ARP (and ND) entries, and the term Microsoft uses for that is "neighbor discovery". I don't know if the SupportsNeighborDiscovery field of MIB_INTERFACE_ROW is implied by other properties of the network interface (such as "all Ethernet interfaces support ARP") or whether it can be individually set at all. One other detail is that we have the gateway for the tunnel's routes set to 0.0.0.0 (or "::"). I presume that also influences how Windows decides which addresses might be on-link neighbours. 1. https://docs.microsoft.com/en-us/windows/win32/api/netioapi/ns-netioapi-mib_ipinterface_row 2. https://docs.microsoft.com/en-us/windows/win32/api/netioapi/nf-netioapi-setipnetentry2 3. https://docs.microsoft.com/en-us/windows/win32/api/netioapi/ns-netioapi-mib_ipnet_table2 -- Brad Spencer ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Wintun NeighborDiscoverySupported 2021-09-09 18:15 ` Brad Spencer @ 2021-09-09 20:23 ` Alan Graham 2021-09-10 17:19 ` Brad Spencer 0 siblings, 1 reply; 6+ messages in thread From: Alan Graham @ 2021-09-09 20:23 UTC (permalink / raw) To: Brad Spencer; +Cc: Jason A. Donenfeld, WireGuard mailing list Hi Brad, Jason, Adding the 0.0.0.0/0 (or ":::") to the config is what is causing some growth of the arp table, but it is not growing indefinitely. After looking around for SupportsNeighborDiscovery and finding nothing, I decided to check the repro. When not routing all traffic through an interface the arp cache is basically static: PS C:\> arp -a -N 100.0.0.2 Interface: 100.0.0.2 --- 0x38 Internet Address Physical Address Type 100.0.0.4 dynamic 224.0.0.22 static 224.0.0.251 static 224.0.0.252 static 239.255.255.250 static When routing all traffic through it grows more or less like an arp cache should: PS C:\> arp -a -N 100.0.0.2 Interface: 100.0.0.2 --- 0x38 Internet Address Physical Address Type 0.0.0.0 static 8.8.4.4 dynamic 8.8.8.8 dynamic 13.64.180.106 dynamic 18.211.21.156 dynamic 20.98.103.204 dynamic 23.6.164.43 dynamic 23.40.197.163 dynamic 23.204.103.147 dynamic 35.185.199.42 dynamic 40.83.247.108 dynamic 51.143.14.218 dynamic 54.214.97.217 dynamic 65.8.17.107 dynamic 67.160.11.228 dynamic 69.1.20.242 dynamic 73.118.184.4 dynamic 74.125.142.188 dynamic 74.125.197.188 dynamic 75.75.75.75 dynamic 100.0.0.2 dynamic 100.0.0.4 dynamic 142.250.69.202 dynamic 142.250.217.99 dynamic 142.251.33.74 dynamic 142.251.33.106 dynamic 173.194.202.189 dynamic 192.168.200.2 dynamic 192.228.79.201 dynamic 224.0.0.22 static 224.0.0.251 static 224.0.0.252 static 239.255.255.250 static So I tend to agree with Jason that this is "harmless" and shouldn't cause any serious problems. It would be nice for Microsoft to fix Set-NetIPInterface, it looks like a bug that SupportsNeighborDiscovery can't be set. Best regards, Alan Graham On Thu, Sep 9, 2021 at 11:17 AM Brad Spencer <bspencer@blackberry.com> wrote: > > On 2021-09-09 2:42 p.m., Jason A. Donenfeld wrote: > > So how exactly were > > you "seeing" the ARP requests on the Wintun interface? Did wireshark > > show it? Or did you read from the Wintun ring and actually see an ARP > > frame? Or something else? Or was this just a manner of speaking and > > you didn't actually observe ARP frames themselves? > You're right to suspect that I was speaking imprecisely here. We have > never seen an ARP request appear on the Wintun interface! I meant to > say that we have noticed the ARP table for the Wintun interface > accumulating entries. > > >> We _think_ that the NeighborDiscoverySupported property being Yes means > >> that Windows issues ARP requests for addresses on the Wintun interface. > > That seems like a good intuition. I'm wondering whether that's > > something you're assuming or something you read on a Microsoft > > website. I ask because this might provide a good entry point for > > whatever reverse engineering I wind up doing to fix this. > I pieced this together from a few scraps. > > On the MIB_IPINTERFACE_ROW page[1], the docs only tersely say: > > "A value that specifies if the IP interface support neighbor discovery." > > Microsoft seems to use the same terminology when documenting > SetIpNetEntry2()[2]: > > "The SetIpNetEntry2 function sets the physical address of an existing > neighbor IP address entry on the local computer." > > And then, most importantly, MIB_IPNET_ROW2, the structure used by that > function says this in its Remarks section[3]: > > "For IPv4, this includes addresses determined used the Address > Resolution Protocol (ARP). For IPv6, this includes addresses determined > using the Neighbor Discovery (ND) protocol for IPv6 as specified in RFC > 2461. " > > So, it seems that the "NetEntry" APIs are those that deal with ARP (and > ND) entries, and the term Microsoft uses for that is "neighbor discovery". > > I don't know if the SupportsNeighborDiscovery field of MIB_INTERFACE_ROW > is implied by other properties of the network interface (such as "all > Ethernet interfaces support ARP") or whether it can be individually set > at all. > > One other detail is that we have the gateway for the tunnel's routes set > to 0.0.0.0 (or "::"). I presume that also influences how Windows > decides which addresses might be on-link neighbours. > > 1. > https://docs.microsoft.com/en-us/windows/win32/api/netioapi/ns-netioapi-mib_ipinterface_row > 2. > https://docs.microsoft.com/en-us/windows/win32/api/netioapi/nf-netioapi-setipnetentry2 > 3. > https://docs.microsoft.com/en-us/windows/win32/api/netioapi/ns-netioapi-mib_ipnet_table2 > > -- > Brad Spencer > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Wintun NeighborDiscoverySupported 2021-09-09 20:23 ` Alan Graham @ 2021-09-10 17:19 ` Brad Spencer 2021-09-10 20:56 ` Alan Graham 0 siblings, 1 reply; 6+ messages in thread From: Brad Spencer @ 2021-09-10 17:19 UTC (permalink / raw) To: Alan Graham; +Cc: Jason A. Donenfeld, WireGuard mailing list On 2021-09-09 5:23 p.m., Alan Graham wrote: > Adding the 0.0.0.0/0 (or ":::") to the config is what is causing some > growth of the arp table, but it is not growing indefinitely. After > looking around for SupportsNeighborDiscovery and finding nothing, I > decided to check the repro. When not routing all traffic through an > interface the arp cache is basically static: Thanks for looking into this, too. I also suspect having the gateway set like this is probably necessary for Windows to start adding ARP entries. How were you able to determine that it is also sufficient? My guess is that if it is possible to indicate to Windows that the interface does not support neighbour discovery in general, doing so likely prevents ARP entries regardless of the gateway values. BTW, how did you determine that it does not grow indefinitely? > So I tend to agree with Jason that this is "harmless" and shouldn't > cause any serious problems. It would be nice for Microsoft to fix > Set-NetIPInterface, it looks like a bug that SupportsNeighborDiscovery > can't be set. One "harm" might be that the OS keeping an easy-to-query list of all (recent?) destinations in the ARP table, which could be undesirable. I'm not sure it's a bug, per se. It seems by design that you cannot change the value of SupportsNeighborDiscovery after the interface is created. The documentation for SetIpInterfaceEntry()[1] says: > The MaxReassemblySize, MinRouterAdvertisementInterval, > MaxRouterAdvertisementInterval , Connected, SupportsWakeUpPatterns, > SupportsNeighborDiscovery, SupportsRouterDiscovery, ReachableTime, > TransmitOffload, and ReceiveOffload members of the MIB_IPINTERFACE_ROW > structure pointed to by the Row are ignored when the > SetIpInterfaceEntry function is called. These members are set by the > network stack and cannot be changed using the SetIpInterfaceEntry > function. I noticed that the Cmdlet in PowerShell seems to treat the -NeighborDiscoverySupported option as an input filter vs. a value that you can set. While this surprised me, it is at least consistent with the Win32 API docs. 1. https://docs.microsoft.com/en-us/windows/win32/api/netioapi/nf-netioapi-setipinterfaceentry#remarks -- Brad Spencer ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Wintun NeighborDiscoverySupported 2021-09-10 17:19 ` Brad Spencer @ 2021-09-10 20:56 ` Alan Graham 0 siblings, 0 replies; 6+ messages in thread From: Alan Graham @ 2021-09-10 20:56 UTC (permalink / raw) To: Brad Spencer; +Cc: Jason A. Donenfeld, WireGuard mailing list I mainly reached the conclusion that it's not growing indefinitely by looking at the results in the arp table. There weren't duplicates or other errors which might suggest the arp table wasn't working properly. I believe it is, in that it should be aging out entries, etc. I also stopped and restarted the wireguard service and it cleared the cache as it should. I also noticed the Powershell Cmdlet used the wrong filter parameters when trying to set NeighborDiscoverySupported, as you did. My expectation would be to get an error invalid parameter like with NeighborUnreachabilityDetection, not that the filter returned zero results because it used the wrong query. Best regards, Alan Graham On Fri, Sep 10, 2021 at 10:19 AM Brad Spencer <bspencer@blackberry.com> wrote: > > On 2021-09-09 5:23 p.m., Alan Graham wrote: > > Adding the 0.0.0.0/0 (or ":::") to the config is what is causing some > > growth of the arp table, but it is not growing indefinitely. After > > looking around for SupportsNeighborDiscovery and finding nothing, I > > decided to check the repro. When not routing all traffic through an > > interface the arp cache is basically static: > > Thanks for looking into this, too. > > I also suspect having the gateway set like this is probably necessary > for Windows to start adding ARP entries. How were you able to determine > that it is also sufficient? My guess is that if it is possible to > indicate to Windows that the interface does not support neighbour > discovery in general, doing so likely prevents ARP entries regardless of > the gateway values. > > BTW, how did you determine that it does not grow indefinitely? > > > So I tend to agree with Jason that this is "harmless" and shouldn't > > cause any serious problems. It would be nice for Microsoft to fix > > Set-NetIPInterface, it looks like a bug that SupportsNeighborDiscovery > > can't be set. > > One "harm" might be that the OS keeping an easy-to-query list of all > (recent?) destinations in the ARP table, which could be undesirable. > > I'm not sure it's a bug, per se. It seems by design that you cannot > change the value of SupportsNeighborDiscovery after the interface is > created. The documentation for SetIpInterfaceEntry()[1] says: > > > The MaxReassemblySize, MinRouterAdvertisementInterval, > > MaxRouterAdvertisementInterval , Connected, SupportsWakeUpPatterns, > > SupportsNeighborDiscovery, SupportsRouterDiscovery, ReachableTime, > > TransmitOffload, and ReceiveOffload members of the MIB_IPINTERFACE_ROW > > structure pointed to by the Row are ignored when the > > SetIpInterfaceEntry function is called. These members are set by the > > network stack and cannot be changed using the SetIpInterfaceEntry > > function. > > I noticed that the Cmdlet in PowerShell seems to treat the > -NeighborDiscoverySupported option as an input filter vs. a value that > you can set. While this surprised me, it is at least consistent with > the Win32 API docs. > > 1. > https://docs.microsoft.com/en-us/windows/win32/api/netioapi/nf-netioapi-setipinterfaceentry#remarks > > > -- > Brad Spencer > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-09-10 20:56 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-09 14:32 Wintun NeighborDiscoverySupported Brad Spencer 2021-09-09 17:42 ` Jason A. Donenfeld 2021-09-09 18:15 ` Brad Spencer 2021-09-09 20:23 ` Alan Graham 2021-09-10 17:19 ` Brad Spencer 2021-09-10 20:56 ` Alan Graham
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).