Re: [MPTCP] Socket layer changes to MPTCP

Re: [MPTCP] Socket layer changes to MPTCP

[Mat Martineau]

[-- Attachment #1: Type: text/plain, Size: 1796 bytes --]


Hi Rao -

On Wed, 28 Mar 2018, Rao Shoaib wrote:

> I have recently noticed that the group is working on socket level 
> changes to MPTCP.
>
> Can a summary of the issues and how the changes will solve them be 
> posted to the list?  That would help any new participants as well. If 
> they have already been discussed please point me to the thread.

I proposed moving away from the metasocket architecture last summer:
https://lists.01.org/pipermail/mptcp/2017-July/000064.html

When I reviewed the multipath-tcp.org implementation of MPTCP, my 
assessment was (and is) that using metasockets resulted in more intrusive 
code changes for TCP. The more partitioned approach of KCM and RDS (using 
internal kernel sockets) shows an example of in-kernel features that the 
maintainers have merged. The approach seemed feasible and I proposed it as 
a direction to go.

> It will also help if it can be made clear how these changes will 
> expedite implementation of a basic MPTCP implementation. I believe that 
> is the current goal, but my understanding may be incorrect.

I think it will expedite upstreaming in the sense that metasockets would 
never get merged upstream, and that a more partitioned approach with 
dedicated subflow sockets has a reasonable shot. That's my opinion based 
on what I've seen from the maintainers (in terms of mailing list 
discussions, conversations, and what they have merged in the past).

Now, Christoph has raised the topic for more detailed discussion with 
regard to the specifics of the multipath-tcp.org implementation. I don't 
think anything is a foregone conclusion in terms of this aspect of the 
upstreaming project, you can propose a different approach to discuss.


Thanks,

--
Mat Martineau
Intel OTC

Re: [MPTCP] Socket layer changes to MPTCP

[Rao Shoaib]

[-- Attachment #1: Type: text/plain, Size: 2593 bytes --]



On 04/04/2018 11:56 AM, Mat Martineau wrote:
>
> On Wed, 4 Apr 2018, Rao Shoaib wrote:
>
>>
>>
>> On 04/03/2018 06:20 PM, Mat Martineau wrote:
>>>
>>> Hi Rao,
>>>
>>> On Mon, 2 Apr 2018, Rao Shoaib wrote:
>>>
>>>> Hi Mat,
>>>>
>>>> We have an implementation that implements MPTCP with much much less 
>>>> intrusiveness than the current implementation. It's review has not 
>>>> revealed any serious issues. If it's interaction with TCP can be 
>>>> improved further that is great and patches should be submitted.
>>>>
>>>> I have yet to see any other proposal that would result in any 
>>>> better implementation and contrary to individual belief's that some 
>>>> folks have, no one knows what upstream will accept. I bet even 
>>>> David Miller and Eric do not know what they will accept. That is 
>>>> why they asked for an implementation to start a discussion, which 
>>>> we have now.
>>>>
>>>> There is no reason to keep working in the dark. Once we have the 
>>>> discussion there will be ample opportunity to improve the 
>>>> implementation based on some guideline.
>>>
>>> I do see what you're saying about not knowing with total certainty 
>>> what the maintainers will say until they say it, and I also think 
>>> there are benefits to be had from our smaller MPTCP upstreaming 
>>> community in helping each other shape a proposal.
>>>
>>> I think the call on Thursday will be a great opportunity to discuss 
>>> the current proposals and patch sets. The real-time discussions 
>>> there have been very helpful - without the latency of email I think 
>>> we can more quickly understand where there is consensus.
>>>
>> Sorry I can not make it to the call -- I have been asked to attend a 
>> meeting at the same time. I know that this is disruptive to everyone 
>> but it is beyond my control. We can instead meet on Friday or Monday 
>> or wait till the next call.
>>
>> Sorry for the inconvenience.
>
> No problem, given the complexity of group scheduling and time zones I 
> think the April 12th call is the straightforward solution.
>
We can also start the discussion on line and continue on the conference 
call.
My question is why would we not want to ask netdev. I already have asked 
them and they are willing to allow refactoring of TCP code but want to 
look at it first.
My plan is to post a few examples of the refactoring and ask for input. 
I do not have any plans to post the complete patch just yet.

Discussing via email also keeps detailed records.

Shoaib
> -- 
> Mat Martineau
> Intel OTC

Re: [MPTCP] Socket layer changes to MPTCP

[Mat Martineau]

[-- Attachment #1: Type: text/plain, Size: 2038 bytes --]


On Wed, 4 Apr 2018, Rao Shoaib wrote:

>
>
> On 04/03/2018 06:20 PM, Mat Martineau wrote:
>> 
>> Hi Rao,
>> 
>> On Mon, 2 Apr 2018, Rao Shoaib wrote:
>> 
>>> Hi Mat,
>>> 
>>> We have an implementation that implements MPTCP with much much less 
>>> intrusiveness than the current implementation. It's review has not 
>>> revealed any serious issues. If it's interaction with TCP can be improved 
>>> further that is great and patches should be submitted.
>>> 
>>> I have yet to see any other proposal that would result in any better 
>>> implementation and contrary to individual belief's that some folks have, 
>>> no one knows what upstream will accept. I bet even David Miller and Eric 
>>> do not know what they will accept. That is why they asked for an 
>>> implementation to start a discussion, which we have now.
>>> 
>>> There is no reason to keep working in the dark. Once we have the 
>>> discussion there will be ample opportunity to improve the implementation 
>>> based on some guideline.
>> 
>> I do see what you're saying about not knowing with total certainty what the 
>> maintainers will say until they say it, and I also think there are benefits 
>> to be had from our smaller MPTCP upstreaming community in helping each 
>> other shape a proposal.
>> 
>> I think the call on Thursday will be a great opportunity to discuss the 
>> current proposals and patch sets. The real-time discussions there have been 
>> very helpful - without the latency of email I think we can more quickly 
>> understand where there is consensus.
>> 
> Sorry I can not make it to the call -- I have been asked to attend a meeting 
> at the same time. I know that this is disruptive to everyone but it is beyond 
> my control. We can instead meet on Friday or Monday or wait till the next 
> call.
>
> Sorry for the inconvenience.

No problem, given the complexity of group scheduling and time zones I 
think the April 12th call is the straightforward solution.

--
Mat Martineau
Intel OTC

Re: [MPTCP] Socket layer changes to MPTCP

[Rao Shoaib]

[-- Attachment #1: Type: text/plain, Size: 1840 bytes --]



On 04/03/2018 06:20 PM, Mat Martineau wrote:
>
> Hi Rao,
>
> On Mon, 2 Apr 2018, Rao Shoaib wrote:
>
>> Hi Mat,
>>
>> We have an implementation that implements MPTCP with much much less 
>> intrusiveness than the current implementation. It's review has not 
>> revealed any serious issues. If it's interaction with TCP can be 
>> improved further that is great and patches should be submitted.
>>
>> I have yet to see any other proposal that would result in any better 
>> implementation and contrary to individual belief's that some folks 
>> have, no one knows what upstream will accept. I bet even David Miller 
>> and Eric do not know what they will accept. That is why they asked 
>> for an implementation to start a discussion, which we have now.
>>
>> There is no reason to keep working in the dark. Once we have the 
>> discussion there will be ample opportunity to improve the 
>> implementation based on some guideline.
>
> I do see what you're saying about not knowing with total certainty 
> what the maintainers will say until they say it, and I also think 
> there are benefits to be had from our smaller MPTCP upstreaming 
> community in helping each other shape a proposal.
>
> I think the call on Thursday will be a great opportunity to discuss 
> the current proposals and patch sets. The real-time discussions there 
> have been very helpful - without the latency of email I think we can 
> more quickly understand where there is consensus.
>
Sorry I can not make it to the call -- I have been asked to attend a 
meeting at the same time. I know that this is disruptive to everyone but 
it is beyond my control. We can instead meet on Friday or Monday or wait 
till the next call.

Sorry for the inconvenience.

Shoaib


> Thanks,
>
> -- 
> Mat Martineau
> Intel OTC

Re: [MPTCP] Socket layer changes to MPTCP

[Mat Martineau]

[-- Attachment #1: Type: text/plain, Size: 1448 bytes --]


Hi Rao,

On Mon, 2 Apr 2018, Rao Shoaib wrote:

> Hi Mat,
>
> We have an implementation that implements MPTCP with much much less 
> intrusiveness than the current implementation. It's review has not revealed 
> any serious issues. If it's interaction with TCP can be improved further that 
> is great and patches should be submitted.
>
> I have yet to see any other proposal that would result in any better 
> implementation and contrary to individual belief's that some folks have, no 
> one knows what upstream will accept. I bet even David Miller and Eric do not 
> know what they will accept. That is why they asked for an implementation to 
> start a discussion, which we have now.
>
> There is no reason to keep working in the dark. Once we have the discussion 
> there will be ample opportunity to improve the implementation based on some 
> guideline.

I do see what you're saying about not knowing with total certainty what 
the maintainers will say until they say it, and I also think there are 
benefits to be had from our smaller MPTCP upstreaming community in helping 
each other shape a proposal.

I think the call on Thursday will be a great opportunity to discuss the 
current proposals and patch sets. The real-time discussions there have 
been very helpful - without the latency of email I think we can more 
quickly understand where there is consensus.

Thanks,

--
Mat Martineau
Intel OTC

Re: [MPTCP] Socket layer changes to MPTCP

[Rao Shoaib]

[-- Attachment #1: Type: text/plain, Size: 813 bytes --]

Hi Mat,

We have an implementation that implements MPTCP with much much less 
intrusiveness than the current implementation. It's review has not 
revealed any serious issues. If it's interaction with TCP can be 
improved further that is great and patches should be submitted.

I have yet to see any other proposal that would result in any better 
implementation and contrary to individual belief's that some folks have, 
no one knows what upstream will accept. I bet even David Miller and Eric 
do not know what they will accept. That is why they asked for an 
implementation to start a discussion, which we have now.

There is no reason to keep working in the dark. Once we have the 
discussion there will be ample opportunity to improve the implementation 
based on some guideline.

Shoaib.

Re: [MPTCP] Socket layer changes to MPTCP

[Mat Martineau]

[-- Attachment #1: Type: text/plain, Size: 6462 bytes --]


Hi Rao,

On Wed, 28 Mar 2018, Rao Shoaib wrote:

>
>
> On 03/28/2018 02:39 PM, Mat Martineau wrote:
>> 
>> Hi Rao -
>> 
>> On Wed, 28 Mar 2018, Rao Shoaib wrote:
>> 
>>> I have recently noticed that the group is working on socket level changes 
>>> to MPTCP.
>>> 
>>> Can a summary of the issues and how the changes will solve them be posted 
>>> to the list?  That would help any new participants as well. If they have 
>>> already been discussed please point me to the thread.
>> 
>> I proposed moving away from the metasocket architecture last summer:
>> https://lists.01.org/pipermail/mptcp/2017-July/000064.html
>> 
>> When I reviewed the multipath-tcp.org implementation of MPTCP, my 
>> assessment was (and is) that using metasockets resulted in more intrusive 
>> code changes for TCP. The more partitioned approach of KCM and RDS (using 
>> internal kernel sockets) shows an example of in-kernel features that the 
>> maintainers have merged. The approach seemed feasible and I proposed it as 
>> a direction to go.
> Thanks for pointing the thread out. That was a proposal we never agreed on 
> it  ?

Yes, I think it's accurate to call it a proposal. The discussion trailed 
off without any specific objection to or endorsement of the metasocket 
point.

> Here is my response to it.
>
> """""
> I agree with the above approach but want to expand on the initial goal.
>
> Our initial goal must be to put a minimal (bare bones) MPTCP implementation 
> in main stream Linux. That could mean no fancy scheduling schemes, just 
> simple round robin or just active/standby. Implementing minimal MPTCP 
> functionality will pretty much expose how main TCP code will be impacted. Any 
> future work to add features will be confined to changes within MPTCP and 
> should not be a concern right now. Such an implementation will also be fully 
> RFC compliant.
>
> I agree with you that upstream folks would want an opt-in option. I am in 
> favor of a completely different socket family as that would leave tcp socket 
> code untouched. However we should talk to upstream folks once again.
>
> I should have been more specific, I was referring to the socket layer 
> changes. Now that I have implemented MPTCP on Linux and understand the design 
> better I don't think using a separate
>
> I would like us to use as much code as possible from the current 
> implementation. In fact if for some reason (I don't see one) we can not ship 
> a minimal implementation than I prefer that we just port the current 
> implementation and worry about architectural issues later.
>
> In short, I would like our focus to be on putting a minimal MPTCP 
> implementation in Linux so that we have a stake in the ground. Performance 
> and Features can come later. That does not mean that the quality of our 
> implementation is so bad that it is unusable.
>
> Shoaib
>
> """"""""""""""
>
> I was OK using KCM or a different socket family, but nothing else, as I say 
> using a socket family would only save changes to the socket code.

Ok - appreciate the clarification.

>
> Since our initial goal was not adhered to and I have had more experience,
> I would like a technical analysis or better yet an implementation that supports 
> the said claims. Any mention of upstream should have pointers to the email 
> threads or should not be used.

I think what people seem to agree on is that an initial upstreamable 
implementation will function with a minimal feature set. My understanding 
is that we have yet to settle on what the steps are to get to that goal, 
and we're working to agree on the steps.

> KCM and RDS are completely different, they do 
> not add options to TCP header or deal with TCP state machine.

I completely agree that they aren't just like MPTCP! And that MPTCP is 
harder to integrate because of exactly the reasons you mention. My 
intent was to point to them as kernel-internal users of TCP with some 
limited design parallels that we can learn from.

>
> I have submitted a patch that provides a complete working implementation 
> based on the current design. It can be used as a reference to point out how 
> the proposed solution would be better.
>
>> 
>>> It will also help if it can be made clear how these changes will 
>>> expedite implementation of a basic MPTCP implementation. I believe 
>>> that is the current goal, but my understanding may be incorrect.
>> 
>> I think it will expedite upstreaming in the sense that metasockets would 
>> never get merged upstream, and that a more partitioned approach with 
>> dedicated subflow sockets has a reasonable shot. That's my opinion based on 
>> what I've seen from the maintainers (in terms of mailing list discussions, 
>> conversations, and what they have merged in the past).
>> 
>> Now, Christoph has raised the topic for more detailed discussion with 
>> regard to the specifics of the multipath-tcp.org implementation. I don't 
>> think anything is a foregone conclusion in terms of this aspect of the 
>> upstreaming project, you can propose a different approach to discuss.
> I want to point out that folks propose a lot of enhancements on Netdev and 
> IETF with valid uses cases. 99.99% get rejected. There has to be very valid 
> reasons -- We are not going to build a kitchen sink of options that will make 
> the design complex and unmaintainable. If a vendor has certain requirements 
> they need to provide us a list that we evaluate and find out the best way to 
> implement it if we choose to. We are not there yet and talking about any kind 
> of options is premature.

To me, the socket-layer changes brought up for discussion are first and 
foremost about modifying the design and architecture of MPTCP to reduce 
the impact on the TCP code. I haven't seen that anyone is putting current 
development effort toward adding bells & whistles - some topics came up 
about possible future APIs that might fit well (like exposing additional 
fds), but I don't see those as in-scope for initial upstreaming.

Part of what we discussed at the last meeting was using a wiki to better 
document what's going on in our MPTCP upstreaming community, I think that 
will do a lot to communicate where contributors are putting their current 
efforts and also help keep track of ideas for the future.


Regards,

--
Mat Martineau
Intel OTC

Re: [MPTCP] Socket layer changes to MPTCP

[Rao Shoaib]

[-- Attachment #1: Type: text/plain, Size: 4827 bytes --]



On 03/28/2018 02:39 PM, Mat Martineau wrote:
>
> Hi Rao -
>
> On Wed, 28 Mar 2018, Rao Shoaib wrote:
>
>> I have recently noticed that the group is working on socket level 
>> changes to MPTCP.
>>
>> Can a summary of the issues and how the changes will solve them be 
>> posted to the list?  That would help any new participants as well. If 
>> they have already been discussed please point me to the thread.
>
> I proposed moving away from the metasocket architecture last summer:
> https://lists.01.org/pipermail/mptcp/2017-July/000064.html
>
> When I reviewed the multipath-tcp.org implementation of MPTCP, my 
> assessment was (and is) that using metasockets resulted in more 
> intrusive code changes for TCP. The more partitioned approach of KCM 
> and RDS (using internal kernel sockets) shows an example of in-kernel 
> features that the maintainers have merged. The approach seemed 
> feasible and I proposed it as a direction to go.
Thanks for pointing the thread out. That was a proposal we never agreed 
on it  ?
Here is my response to it.

"""""
I agree with the above approach but want to expand on the initial goal.

Our initial goal must be to put a minimal (bare bones) MPTCP 
implementation in main stream Linux. That could mean no fancy scheduling 
schemes, just simple round robin or just active/standby. Implementing 
minimal MPTCP functionality will pretty much expose how main TCP code 
will be impacted. Any future work to add features will be confined to 
changes within MPTCP and should not be a concern right now. Such an 
implementation will also be fully RFC compliant.

I agree with you that upstream folks would want an opt-in option. I am 
in favor of a completely different socket family as that would leave tcp 
socket code untouched. However we should talk to upstream folks once again.

I should have been more specific, I was referring to the socket layer 
changes. Now that I have implemented MPTCP on Linux and understand the 
design better I don't think using a separate

I would like us to use as much code as possible from the current 
implementation. In fact if for some reason (I don't see one) we can not 
ship a minimal implementation than I prefer that we just port the 
current implementation and worry about architectural issues later.

In short, I would like our focus to be on putting a minimal MPTCP 
implementation in Linux so that we have a stake in the ground. 
Performance and Features can come later. That does not mean that the 
quality of our implementation is so bad that it is unusable.

Shoaib

""""""""""""""

I was OK using KCM or a different socket family, but nothing else, as I 
say using a socket family would only save changes to the socket code.

Since our initial goal was not adhered to and I have had more 
experience, I would like a technical analysis or better yet an 
implementation that supports the said claims. Any mention of upstream 
should have pointers to the email threads or should not be used. KCM and 
RDS are completely different, they do not add options to TCP header or 
deal with TCP state machine.

I have submitted a patch that provides a complete working implementation 
based on the current design. It can be used as a reference to point out 
how the proposed solution would be better.

>
>> It will also help if it can be made clear how these changes will 
>> expedite implementation of a basic MPTCP implementation. I believe 
>> that is the current goal, but my understanding may be incorrect.
>
> I think it will expedite upstreaming in the sense that metasockets 
> would never get merged upstream, and that a more partitioned approach 
> with dedicated subflow sockets has a reasonable shot. That's my 
> opinion based on what I've seen from the maintainers (in terms of 
> mailing list discussions, conversations, and what they have merged in 
> the past).
>
> Now, Christoph has raised the topic for more detailed discussion with 
> regard to the specifics of the multipath-tcp.org implementation. I 
> don't think anything is a foregone conclusion in terms of this aspect 
> of the upstreaming project, you can propose a different approach to 
> discuss.
I want to point out that folks propose a lot of enhancements on Netdev 
and IETF with valid uses cases. 99.99% get rejected. There has to be 
very valid reasons -- We are not going to build a kitchen sink of 
options that will make the design complex and unmaintainable. If a 
vendor has certain requirements they need to provide us a list that we 
evaluate and find out the best way to implement it if we choose to. We 
are not there yet and talking about any kind of options is premature.

Thanks,

Shoaib
> Thanks,
>
> -- 
> Mat Martineau
> Intel OTC

[MPTCP] Socket layer changes to MPTCP

[Rao Shoaib]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

I have recently noticed that the group is working on socket level 
changes to MPTCP.

Can a summary of the issues and how the changes will solve them be 
posted to the list?  That would help any new participants as well. If 
they have already been discussed please point me to the thread.

It will also help if it can be made clear how these changes will 
expedite implementation of a basic MPTCP implementation. I believe that 
is the current goal, but my understanding may be incorrect.

Thanks,

Shoaib