[Query] Delayed vxlan socket creation?

[Query] Delayed vxlan socket creation?

[Du, Fan]

Hi

I'm interested to one Docker issue[1] which looks like related to kernel vxlan socket creation
as described in the thread. From my limited knowledge here, socket creation is synchronous ,
and after the *socket* syscall, the sock handle will be valid and ready to linkup.

Somehow I'm not sure the detailed scenario here, and which/how possible commit fix?
Thanks!

Quoted analysis:
--------------------------------------------------------------------------
(Found in kernel 3.13)
The issue happens because in older kernels when a vxlan interface is created, 
the socket creation is queued up in a worker thread which actually creates 
the socket. But this needs to happen before we bring up the link on the vxlan interface. 
If for some chance, the worker thread hasn't completed the creation of the socket 
before we did link up then when we do link up the kernel checks if the socket was 
created and if not it will return ENOTCONN. This was a bug in the kernel which got fixed
in later kernels. That is why retrying with a timer fixes the issue.

[1]: https://github.com/docker/libnetwork/issues/1247

Re: [Query] Delayed vxlan socket creation?

[Jiri Benc]

On Wed, 14 Dec 2016 07:49:24 +0000, Du, Fan wrote:
> I'm interested to one Docker issue[1] which looks like related to kernel vxlan socket creation
> as described in the thread. From my limited knowledge here, socket creation is synchronous ,
> and after the *socket* syscall, the sock handle will be valid and ready to linkup.
> 
> Somehow I'm not sure the detailed scenario here, and which/how possible commit fix?

baf606d9c9b1^..56ef9c909b40

 Jiri

Re: [Query] Delayed vxlan socket creation?

[Cong Wang]

On Tue, Dec 13, 2016 at 11:49 PM, Du, Fan <fan.du@intel.com> wrote:
> Hi
>
> I'm interested to one Docker issue[1] which looks like related to kernel vxlan socket creation
> as described in the thread. From my limited knowledge here, socket creation is synchronous ,
> and after the *socket* syscall, the sock handle will be valid and ready to linkup.

You need to read the code. vxlan tunnel is a UDP tunnel, it needs a kernel
socket (and a port) to setup UDP communication, unlike GRE tunnel etc.


>
> Somehow I'm not sure the detailed scenario here, and which/how possible commit fix?
> Thanks!
>
> Quoted analysis:
> --------------------------------------------------------------------------
> (Found in kernel 3.13)
> The issue happens because in older kernels when a vxlan interface is created,
> the socket creation is queued up in a worker thread which actually creates
> the socket. But this needs to happen before we bring up the link on the vxlan interface.
> If for some chance, the worker thread hasn't completed the creation of the socket
> before we did link up then when we do link up the kernel checks if the socket was
> created and if not it will return ENOTCONN. This was a bug in the kernel which got fixed
> in later kernels. That is why retrying with a timer fixes the issue.


This was introduced by commit 1c51a9159ddefa5119724a4c7da3fd3ef44b68d5
and later fixed by commit 56ef9c909b40483d2c8cb63fcbf83865f162d5ec.

Re: [Query] Delayed vxlan socket creation?

[Du, Fan]

在 2016年12月14日 17:29, Jiri Benc 写道:
> On Wed, 14 Dec 2016 07:49:24 +0000, Du, Fan wrote:
>> I'm interested to one Docker issue[1] which looks like related to kernel vxlan socket creation
>> as described in the thread. From my limited knowledge here, socket creation is synchronous ,
>> and after the *socket* syscall, the sock handle will be valid and ready to linkup.
>>
>> Somehow I'm not sure the detailed scenario here, and which/how possible commit fix?
> baf606d9c9b1^..56ef9c909b40
>
>   Jiri

Thanks a lot Jiri!

Re: [Query] Delayed vxlan socket creation?

[Du, Fan]

在 2016年12月15日 01:24, Cong Wang 写道:
> On Tue, Dec 13, 2016 at 11:49 PM, Du, Fan <fan.du@intel.com> wrote:
>> Hi
>>
>> I'm interested to one Docker issue[1] which looks like related to kernel vxlan socket creation
>> as described in the thread. From my limited knowledge here, socket creation is synchronous ,
>> and after the *socket* syscall, the sock handle will be valid and ready to linkup.
> You need to read the code. vxlan tunnel is a UDP tunnel, it needs a kernel
> socket (and a port) to setup UDP communication, unlike GRE tunnel etc.
I check the fix is merged in 4.0, my code base is pretty new,
so somehow I failed to see the work queue stuff in drver/net/vxlan.c
>> Somehow I'm not sure the detailed scenario here, and which/how possible commit fix?
>> Thanks!
>>
>> Quoted analysis:
>> --------------------------------------------------------------------------
>> (Found in kernel 3.13)
>> The issue happens because in older kernels when a vxlan interface is created,
>> the socket creation is queued up in a worker thread which actually creates
>> the socket. But this needs to happen before we bring up the link on the vxlan interface.
>> If for some chance, the worker thread hasn't completed the creation of the socket
>> before we did link up then when we do link up the kernel checks if the socket was
>> created and if not it will return ENOTCONN. This was a bug in the kernel which got fixed
>> in later kernels. That is why retrying with a timer fixes the issue.
>
> This was introduced by commit 1c51a9159ddefa5119724a4c7da3fd3ef44b68d5
> and later fixed by commit 56ef9c909b40483d2c8cb63fcbf83865f162d5ec.
信聪哥，得永生。
Thanks for the offending commit id！