[PATCH 0/2] bring UP loopback device at initialziation

[PATCH 0/2] bring UP loopback device at initialziation

[Mahesh Bandewar]

From: Mahesh Bandewar <maheshb@google.com>

In almost every scenario the loopback device is brought UP after
initialization. So there is no point of bringing up the device in
DOWN state followed by device UP operation. This change exposed
another issue of fib-trie initialization which is corrected in the 
first path.

Mahesh Bandewar (2):
  ipv4: initialize fib_trie prior to register_netdev_notifier call.
  loopback: bringup 'lo' by default at initialization

 drivers/net/loopback.c  | 6 ++++++
 net/ipv4/fib_frontend.c | 4 ++--
 2 files changed, 8 insertions(+), 2 deletions(-)

-- 
2.13.2.725.g09c95d1e9-goog

Re: [PATCH 0/2] bring UP loopback device at initialziation

[David Miller]

From: Mahesh Bandewar <mahesh@bandewar.net>
Date: Tue,  4 Jul 2017 12:16:15 -0700

> In almost every scenario the loopback device is brought UP after
> initialization. So there is no point of bringing up the device in
> DOWN state followed by device UP operation. This change exposed
> another issue of fib-trie initialization which is corrected in the 
> first path.

You use the word almost, which supports my position that someone may
not want this.

I also don't see it as so much of a burdon to bring the lo device up
explicitly.  Systems have been having to do that since the beginning
of time.

Sorry I'm not applying this.

Re: [PATCH 0/2] bring UP loopback device at initialziation

[Mahesh Bandewar (महेश बंडेवार)]

On Wed, Jul 5, 2017 at 1:20 AM, David Miller <davem@davemloft.net> wrote:
> From: Mahesh Bandewar <mahesh@bandewar.net>
> Date: Tue,  4 Jul 2017 12:16:15 -0700
>
>> In almost every scenario the loopback device is brought UP after
>> initialization. So there is no point of bringing up the device in
>> DOWN state followed by device UP operation. This change exposed
>> another issue of fib-trie initialization which is corrected in the
>> first path.
>
> You use the word almost, which supports my position that someone may
> not want this.
>
> I also don't see it as so much of a burdon to bring the lo device up
> explicitly.  Systems have been having to do that since the beginning
> of time.
>
Systems have only one lo device (since ages) and that is usually taken
care at the boot time. Now with the namespaces it's not just one
device as it's per namespace and though not much this patch will
benefit a little. Probably we should ask a question - is it going to
have any bad effects? I couldn't find any and my RFC patch did not get
me any such feedback. As far as the good effects are concerned, it has
already found a bug (another patch in this series)! Also sometime back
I did experience weird behavior inside net-namespace if you forget to
bring-up the loopback device. I didn't pay too much attention as
bringing up the lo device fixed it.

> Sorry I'm not applying this.

Re: [PATCH 0/2] bring UP loopback device at initialziation

[David Miller]

From: Mahesh Bandewar (महेश बंडेवार) <maheshb@google.com>
Date: Wed, 5 Jul 2017 08:59:37 -0700

> On Wed, Jul 5, 2017 at 1:20 AM, David Miller <davem@davemloft.net> wrote:
>> From: Mahesh Bandewar <mahesh@bandewar.net>
>> Date: Tue,  4 Jul 2017 12:16:15 -0700
>>
>>> In almost every scenario the loopback device is brought UP after
>>> initialization. So there is no point of bringing up the device in
>>> DOWN state followed by device UP operation. This change exposed
>>> another issue of fib-trie initialization which is corrected in the
>>> first path.
>>
>> You use the word almost, which supports my position that someone may
>> not want this.
>>
>> I also don't see it as so much of a burdon to bring the lo device up
>> explicitly.  Systems have been having to do that since the beginning
>> of time.
>>
> Systems have only one lo device (since ages) and that is usually taken
> care at the boot time. Now with the namespaces it's not just one
> device as it's per namespace and though not much this patch will
> benefit a little. Probably we should ask a question - is it going to
> have any bad effects? I couldn't find any and my RFC patch did not get
> me any such feedback. As far as the good effects are concerned, it has
> already found a bug (another patch in this series)! Also sometime back
> I did experience weird behavior inside net-namespace if you forget to
> bring-up the loopback device. I didn't pay too much attention as
> bringing up the lo device fixed it.

You're not talking at all about why specifically you need this
(ie. your use case) when you are spinning up namespaces for users.

I do happen to know those details, but you need to talk about this
explicitly in your commit log messages and in this discussion so that
everyone else understands this as well.

Thank you.

Re: [PATCH 0/2] bring UP loopback device at initialziation

[Cong Wang]

On Wed, Jul 5, 2017 at 8:59 AM, Mahesh Bandewar (महेश बंडेवार)
<maheshb@google.com> wrote:
> Systems have only one lo device (since ages) and that is usually taken
> care at the boot time. Now with the namespaces it's not just one
> device as it's per namespace and though not much this patch will
> benefit a little. Probably we should ask a question - is it going to
> have any bad effects? I couldn't find any and my RFC patch did not get
> me any such feedback. As far as the good effects are concerned, it has
> already found a bug (another patch in this series)! Also sometime back
> I did experience weird behavior inside net-namespace if you forget to
> bring-up the loopback device. I didn't pay too much attention as
> bringing up the lo device fixed it.
>

I wonder if it is too late to change this since this behavior is probably
from the beginning of network namespace. A networkless netns is also
useful at least for testing purpose, we do use it as a sandbox.

Re: [PATCH 0/2] bring UP loopback device at initialziation

[Mahesh Bandewar (महेश बंडेवार)]

On Wed, Jul 5, 2017 at 9:05 AM, David Miller <davem@davemloft.net> wrote:
> From: Mahesh Bandewar (महेश बंडेवार) <maheshb@google.com>
> Date: Wed, 5 Jul 2017 08:59:37 -0700
>
>> On Wed, Jul 5, 2017 at 1:20 AM, David Miller <davem@davemloft.net> wrote:
>>> From: Mahesh Bandewar <mahesh@bandewar.net>
>>> Date: Tue,  4 Jul 2017 12:16:15 -0700
>>>
>>>> In almost every scenario the loopback device is brought UP after
>>>> initialization. So there is no point of bringing up the device in
>>>> DOWN state followed by device UP operation. This change exposed
>>>> another issue of fib-trie initialization which is corrected in the
>>>> first path.
>>>
>>> You use the word almost, which supports my position that someone may
>>> not want this.
>>>
>>> I also don't see it as so much of a burdon to bring the lo device up
>>> explicitly.  Systems have been having to do that since the beginning
>>> of time.
>>>
>> Systems have only one lo device (since ages) and that is usually taken
>> care at the boot time. Now with the namespaces it's not just one
>> device as it's per namespace and though not much this patch will
>> benefit a little. Probably we should ask a question - is it going to
>> have any bad effects? I couldn't find any and my RFC patch did not get
>> me any such feedback. As far as the good effects are concerned, it has
>> already found a bug (another patch in this series)! Also sometime back
>> I did experience weird behavior inside net-namespace if you forget to
>> bring-up the loopback device. I didn't pay too much attention as
>> bringing up the lo device fixed it.
>
> You're not talking at all about why specifically you need this
> (ie. your use case) when you are spinning up namespaces for users.
>
> I do happen to know those details, but you need to talk about this
> explicitly in your commit log messages and in this discussion so that
> everyone else understands this as well.
>
Well, I can make this commit message long-winded but most of the (so
called) issues are well known and I thought I wont add any additional
value repeating them here hence kept it simple. I can spin up the next
rev with the long-winded commit message. ;)

> Thank you.

Re: [PATCH 0/2] bring UP loopback device at initialziation

[Mahesh Bandewar (महेश बंडेवार)]

> I wonder if it is too late to change this since this behavior is probably
> from the beginning of network namespace. A networkless netns is also
> useful at least for testing purpose, we do use it as a sandbox.
>
Sandbox is my use case too but i'm worried about all other things that
a process inside that namespace can do (tasks' capability mask) and
want to eliminate a need to have a capability just to bing-up the
loopback device (more in the commit message update)


> If you can please include the analysis that describes why no one will
> care. Especially applications such as vsftpd that create network
> namespaces as a way to sandbox themselves and not have a network stack
> available.
>
Well, I wasn't sure hence probed with the RFC patch.
I'm not familiar with the vsftpd use-case but I assume it's expecting
a loopback device to be in DOWN state.

Now that you have made me aware of some use cases that do want the
loopback device to be DOWN, could we use a global sysctl to dictate
the loopback behavior during init? e.g.

net.core.netdev_loopback_state = {0|1}

    where 0: is current behavior i.e. DOWN by default.
               1: is the proposed new behavior i.e. UP by default.

We can keep the default value of this sysctl to be '1' so that when a
host boots, the loopback is UP by default.
 In a running system that prefers the loopback device to come up in
DOWN state for the namespaces, can change the value of this sysctl to
'0' so every namespace creation will come-up with loopback in DOWN
state. Or we could reverse the behavior by changing the default value
of this sysctl (to '0').

Thanks,
--mahesh..

Re: [PATCH 0/2] bring UP loopback device at initialziation

[Eric W. Biederman]

"Mahesh Bandewar (महेश बंडेवार)" <maheshb@google.com> writes:

>> I wonder if it is too late to change this since this behavior is probably
>> from the beginning of network namespace. A networkless netns is also
>> useful at least for testing purpose, we do use it as a sandbox.
>>
> Sandbox is my use case too but i'm worried about all other things that
> a process inside that namespace can do (tasks' capability mask) and
> want to eliminate a need to have a capability just to bing-up the
> loopback device (more in the commit message update)
>
>
>> If you can please include the analysis that describes why no one will
>> care. Especially applications such as vsftpd that create network
>> namespaces as a way to sandbox themselves and not have a network stack
>> available.
>>
> Well, I wasn't sure hence probed with the RFC patch.
> I'm not familiar with the vsftpd use-case but I assume it's expecting
> a loopback device to be in DOWN state.

The expectation is that it is up to the poster of a patch to convince
the rest of us it is worth while.

The obvious pro that you don't have go through the extra step
of setting the loopback device up.

I listed vsftpd as it is widely available and was the the most obviously
different from the normal use case.  I don't think you will break
vsftpd.  I know all vsftpd cares about is ensuring it's process can not
communicate with the outside world except through it's one socket. I
know somethings are different with lo up vs lo down but I don't know
them well enough to imagine what will break.

I know on any other interface you want to delay bringing the interface
up because things like ipv6 stateless address autoconfiguration can be
problematic for debugging if you aren't using ipv6.  Just lots of extra
packet noise.

The only argument I can see for leaving the loopback interface down by
default is that if bringing the interface up makes it difficult to
disable something (such as tcp to localhost).

Hmm...   The one disadvantage that I can see with bringing the loopback
device up is that, bringing the loopback device up will create
routing entries.  Those routing entries will persist for at least an rcu
grace period.  That will make it more expensive to destroy a network
namespace if you don't every use those routes.  (see vsftpd above)

I personally think it would be handy to have loopback devices always
up.  I don't see loopback devices as that expensive to bring up so I
don't much care.  So right now I personally am persuaded that he
aditional expense for applications like vsftpd makes bringing up the
loopback device undesirable.

The less magic at namespace creation time also seems like a compelling
argument when I am debugging things.

Among non-container things I don't see anything except an initial
ramdisk as being able to care.  As the initial ramdisk is very likely
going to bring up the loopback device.

Does the in kernel nfsroot support bring up the loopback device?
I know it needs to bring up networking otherwise.

> Now that you have made me aware of some use cases that do want the
> loopback device to be DOWN, could we use a global sysctl to dictate
> the loopback behavior during init? e.g.
>
> net.core.netdev_loopback_state = {0|1}
>
>     where 0: is current behavior i.e. DOWN by default.
>                1: is the proposed new behavior i.e. UP by default.
>
> We can keep the default value of this sysctl to be '1' so that when a
> host boots, the loopback is UP by default.
>  In a running system that prefers the loopback device to come up in
> DOWN state for the namespaces, can change the value of this sysctl to
> '0' so every namespace creation will come-up with loopback in DOWN
> state. Or we could reverse the behavior by changing the default value
> of this sysctl (to '0').

I don't think it is worth the complexity to have a sysctl to change the
default behavior.  Especially as we can't make it a per network
namespace sysctl.  That will just make the kernel more complex and less
predictable to userspace.

Anything that is useful enough to implement a sysctl would convince me
it is worth it to change the default.


We do need your bug fix.  Please send that one separately so David can
merge it.  Although I suspect at this point it will need to wait until
David reopens the net-next.


Eric

Re: [PATCH 0/2] bring UP loopback device at initialziation

[Mahesh Bandewar (महेश बंडेवार)]

On Thu, Jul 6, 2017 at 5:18 AM, Eric W. Biederman <ebiederm@xmission.com> wrote:
> "Mahesh Bandewar (महेश बंडेवार)" <maheshb@google.com> writes:
>
>>> I wonder if it is too late to change this since this behavior is probably
>>> from the beginning of network namespace. A networkless netns is also
>>> useful at least for testing purpose, we do use it as a sandbox.
>>>
>> Sandbox is my use case too but i'm worried about all other things that
>> a process inside that namespace can do (tasks' capability mask) and
>> want to eliminate a need to have a capability just to bing-up the
>> loopback device (more in the commit message update)
>>
>>
>>> If you can please include the analysis that describes why no one will
>>> care. Especially applications such as vsftpd that create network
>>> namespaces as a way to sandbox themselves and not have a network stack
>>> available.
>>>
>> Well, I wasn't sure hence probed with the RFC patch.
>> I'm not familiar with the vsftpd use-case but I assume it's expecting
>> a loopback device to be in DOWN state.
>
> The expectation is that it is up to the poster of a patch to convince
> the rest of us it is worth while.
>
> The obvious pro that you don't have go through the extra step
> of setting the loopback device up.
>
> I listed vsftpd as it is widely available and was the the most obviously
> different from the normal use case.  I don't think you will break
> vsftpd.  I know all vsftpd cares about is ensuring it's process can not
> communicate with the outside world except through it's one socket. I
> know somethings are different with lo up vs lo down but I don't know
> them well enough to imagine what will break.
>
> I know on any other interface you want to delay bringing the interface
> up because things like ipv6 stateless address autoconfiguration can be
> problematic for debugging if you aren't using ipv6.  Just lots of extra
> packet noise.
>
> The only argument I can see for leaving the loopback interface down by
> default is that if bringing the interface up makes it difficult to
> disable something (such as tcp to localhost).
>
> Hmm...   The one disadvantage that I can see with bringing the loopback
> device up is that, bringing the loopback device up will create
> routing entries.  Those routing entries will persist for at least an rcu
> grace period.  That will make it more expensive to destroy a network
> namespace if you don't every use those routes.  (see vsftpd above)
>
However note that this is small use case while most use cases do bring up
loopback and add other networking too. So these small savings are not really
applicable to the most common use cases.

> I personally think it would be handy to have loopback devices always
> up.  I don't see loopback devices as that expensive to bring up so I
> don't much care.  So right now I personally am persuaded that he
> aditional expense for applications like vsftpd makes bringing up the
> loopback device undesirable.
>
> The less magic at namespace creation time also seems like a compelling
> argument when I am debugging things.
>
> Among non-container things I don't see anything except an initial
> ramdisk as being able to care.  As the initial ramdisk is very likely
> going to bring up the loopback device.
>
> Does the in kernel nfsroot support bring up the loopback device?
> I know it needs to bring up networking otherwise.
>
>> Now that you have made me aware of some use cases that do want the
>> loopback device to be DOWN, could we use a global sysctl to dictate
>> the loopback behavior during init? e.g.
>>
>> net.core.netdev_loopback_state = {0|1}
>>
>>     where 0: is current behavior i.e. DOWN by default.
>>                1: is the proposed new behavior i.e. UP by default.
>>
>> We can keep the default value of this sysctl to be '1' so that when a
>> host boots, the loopback is UP by default.
>>  In a running system that prefers the loopback device to come up in
>> DOWN state for the namespaces, can change the value of this sysctl to
>> '0' so every namespace creation will come-up with loopback in DOWN
>> state. Or we could reverse the behavior by changing the default value
>> of this sysctl (to '0').
>
> I don't think it is worth the complexity to have a sysctl to change the
> default behavior.  Especially as we can't make it a per network
> namespace sysctl.  That will just make the kernel more complex and less
> predictable to userspace.
>
> Anything that is useful enough to implement a sysctl would convince me
> it is worth it to change the default.
>
>
> We do need your bug fix.  Please send that one separately so David can
> merge it.  Although I suspect at this point it will need to wait until
> David reopens the net-next.
>
OK I'll decouple these two patches and send them separately.

>
> Eric
>

Re: [PATCH 0/2] bring UP loopback device at initialziation

[Cong Wang]

On Wed, Jul 5, 2017 at 4:01 PM, Mahesh Bandewar (महेश बंडेवार)
<maheshb@google.com> wrote:
>
> Now that you have made me aware of some use cases that do want the
> loopback device to be DOWN, could we use a global sysctl to dictate
> the loopback behavior during init? e.g.

Yeah, it is never about which way is better, it is all about if we could break
the existing assumption. This is a similar situation to the
inconsistent behavior
of the per-netns sysctl's between IPv4 and IPv6, there were multiple fixes
or complains about it in this mailing list, but we still don't change it yet.