* Multi-homing question
@ 2013-11-04 1:59 tsoi andrew
2013-11-04 12:09 ` Neil Horman
` (13 more replies)
0 siblings, 14 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-04 1:59 UTC (permalink / raw)
To: linux-sctp
Dear Sir,
I got a problem when set multi-homing to active/active.
That mean, It is expected that our lksctp server have 2 IP to connect
2 IP of destination client with total 4 connections.
The flow is that client send "INIT" message with 2 (primary and
secondary) IP to our server and lksctp server return INIT_ACK. But
when second client send 'heartbeat' to us, lksctp server cannot return
ACK.
Moreover, we already proved our connectivity correct because when our
lksctp server send heartbeat to both destination client and they can
return ACK.
Would you mind sharing if lksctp lib already configure those
multi-homing purpose.
Best Regards,
Andrew Tsoi
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
@ 2013-11-04 12:09 ` Neil Horman
2013-11-07 2:36 ` tsoi andrew
` (12 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Neil Horman @ 2013-11-04 12:09 UTC (permalink / raw)
To: linux-sctp
On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
> Dear Sir,
>
> I got a problem when set multi-homing to active/active.
> That mean, It is expected that our lksctp server have 2 IP to connect
> 2 IP of destination client with total 4 connections.
> The flow is that client send "INIT" message with 2 (primary and
> secondary) IP to our server and lksctp server return INIT_ACK. But
> when second client send 'heartbeat' to us, lksctp server cannot return
> ACK.
> Moreover, we already proved our connectivity correct because when our
> lksctp server send heartbeat to both destination client and they can
> return ACK.
> Would you mind sharing if lksctp lib already configure those
> multi-homing purpose.
>
Can you provide a network diagram and a tcpdump of your connection?
Neil
>
>
> Best Regards,
> Andrew Tsoi
> --
> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
2013-11-04 12:09 ` Neil Horman
@ 2013-11-07 2:36 ` tsoi andrew
2013-11-07 2:43 ` tsoi andrew
` (11 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-07 2:36 UTC (permalink / raw)
To: linux-sctp
[-- Attachment #1: Type: text/plain, Size: 1338 bytes --]
Dear Neil,
Please find my network diagram first.
Now at server side, can't see our lksctp client 4 connection but only
2 2 connection. The reason is that heartbeat ACK not found.
Regards,
Andrew
On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>> Dear Sir,
>>
>> I got a problem when set multi-homing to active/active.
>> That mean, It is expected that our lksctp server have 2 IP to connect
>> 2 IP of destination client with total 4 connections.
>> The flow is that client send "INIT" message with 2 (primary and
>> secondary) IP to our server and lksctp server return INIT_ACK. But
>> when second client send 'heartbeat' to us, lksctp server cannot return
>> ACK.
>> Moreover, we already proved our connectivity correct because when our
>> lksctp server send heartbeat to both destination client and they can
>> return ACK.
>> Would you mind sharing if lksctp lib already configure those
>> multi-homing purpose.
>>
> Can you provide a network diagram and a tcpdump of your connection?
> Neil
>
>>
>>
>> Best Regards,
>> Andrew Tsoi
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
[-- Attachment #2: networkDiagram.jpg --]
[-- Type: image/jpeg, Size: 26784 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
2013-11-04 12:09 ` Neil Horman
2013-11-07 2:36 ` tsoi andrew
@ 2013-11-07 2:43 ` tsoi andrew
2013-11-07 3:45 ` tsoi andrew
` (10 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-07 2:43 UTC (permalink / raw)
To: linux-sctp
Dear Neil,
For more information, our secondary IP doesn't response COOKIE_ECHO
also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
Regards,
Andrew
On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> Dear Neil,
>
> Please find my network diagram first.
> Now at server side, can't see our lksctp client 4 connection but only
> 2 2 connection. The reason is that heartbeat ACK not found.
>
> Regards,
> Andrew
>
> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>>> Dear Sir,
>>>
>>> I got a problem when set multi-homing to active/active.
>>> That mean, It is expected that our lksctp server have 2 IP to connect
>>> 2 IP of destination client with total 4 connections.
>>> The flow is that client send "INIT" message with 2 (primary and
>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>>> when second client send 'heartbeat' to us, lksctp server cannot return
>>> ACK.
>>> Moreover, we already proved our connectivity correct because when our
>>> lksctp server send heartbeat to both destination client and they can
>>> return ACK.
>>> Would you mind sharing if lksctp lib already configure those
>>> multi-homing purpose.
>>>
>> Can you provide a network diagram and a tcpdump of your connection?
>> Neil
>>
>>>
>>>
>>> Best Regards,
>>> Andrew Tsoi
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (2 preceding siblings ...)
2013-11-07 2:43 ` tsoi andrew
@ 2013-11-07 3:45 ` tsoi andrew
2013-11-07 14:16 ` Neil Horman
` (9 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-07 3:45 UTC (permalink / raw)
To: linux-sctp
[-- Attachment #1: Type: text/plain, Size: 1998 bytes --]
Dear Neil,
Attached file is our tcpdump.
Now our secondary IP doesn't work.
Please advise. Thanks.
IP info:
lksctp primary IP is 172.28.129.49
lksctp secondary IP is 172.28.129.176
Client IP is 10.82.29.240/10.82.29.241
Regards,
Andrew
On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> Dear Neil,
>
> For more information, our secondary IP doesn't response COOKIE_ECHO
> also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
>
> Regards,
> Andrew
>
> On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> Dear Neil,
>>
>> Please find my network diagram first.
>> Now at server side, can't see our lksctp client 4 connection but only
>> 2 2 connection. The reason is that heartbeat ACK not found.
>>
>> Regards,
>> Andrew
>>
>> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>>>> Dear Sir,
>>>>
>>>> I got a problem when set multi-homing to active/active.
>>>> That mean, It is expected that our lksctp server have 2 IP to connect
>>>> 2 IP of destination client with total 4 connections.
>>>> The flow is that client send "INIT" message with 2 (primary and
>>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>>>> when second client send 'heartbeat' to us, lksctp server cannot return
>>>> ACK.
>>>> Moreover, we already proved our connectivity correct because when our
>>>> lksctp server send heartbeat to both destination client and they can
>>>> return ACK.
>>>> Would you mind sharing if lksctp lib already configure those
>>>> multi-homing purpose.
>>>>
>>> Can you provide a network diagram and a tcpdump of your connection?
>>> Neil
>>>
>>>>
>>>>
>>>> Best Regards,
>>>> Andrew Tsoi
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>
[-- Attachment #2: sctpmultiHomeNow.pcap --]
[-- Type: application/octet-stream, Size: 5444 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (3 preceding siblings ...)
2013-11-07 3:45 ` tsoi andrew
@ 2013-11-07 14:16 ` Neil Horman
2013-11-08 3:35 ` tsoi andrew
` (8 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Neil Horman @ 2013-11-07 14:16 UTC (permalink / raw)
To: linux-sctp
On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
> Dear Neil,
>
> Attached file is our tcpdump.
> Now our secondary IP doesn't work.
> Please advise. Thanks.
>
> IP info:
> lksctp primary IP is 172.28.129.49
> lksctp secondary IP is 172.28.129.176
>
> Client IP is 10.82.29.240/10.82.29.241
>
>
I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
Looking at this, its not just heartbeats that arent getting answered, nothing is
getting responded to via the 176 address. The only data that I see come out of
that address are two abort frames (not sure what the cause is there yet).
I would start by verifying basic connectivity via that address. I'd try
something like:
ping -I 172.28.129.176 10.82.29.240
and
ping -I 172.28.129.176 10.82.29.241
while running a tcpdump on the interface that holds the 176 address specifically
to make sure that you're routing tables locally and network in generally can
pass traffic over that address
Neil
>
>
>
> Regards,
> Andrew
>
> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> > Dear Neil,
> >
> > For more information, our secondary IP doesn't response COOKIE_ECHO
> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
> >
> > Regards,
> > Andrew
> >
> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> >> Dear Neil,
> >>
> >> Please find my network diagram first.
> >> Now at server side, can't see our lksctp client 4 connection but only
> >> 2 2 connection. The reason is that heartbeat ACK not found.
> >>
> >> Regards,
> >> Andrew
> >>
> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
> >>>> Dear Sir,
> >>>>
> >>>> I got a problem when set multi-homing to active/active.
> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
> >>>> 2 IP of destination client with total 4 connections.
> >>>> The flow is that client send "INIT" message with 2 (primary and
> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
> >>>> ACK.
> >>>> Moreover, we already proved our connectivity correct because when our
> >>>> lksctp server send heartbeat to both destination client and they can
> >>>> return ACK.
> >>>> Would you mind sharing if lksctp lib already configure those
> >>>> multi-homing purpose.
> >>>>
> >>> Can you provide a network diagram and a tcpdump of your connection?
> >>> Neil
> >>>
> >>>>
> >>>>
> >>>> Best Regards,
> >>>> Andrew Tsoi
> >>>> --
> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
> >>>> the body of a message to majordomo@vger.kernel.org
> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >>>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (4 preceding siblings ...)
2013-11-07 14:16 ` Neil Horman
@ 2013-11-08 3:35 ` tsoi andrew
2013-11-08 4:18 ` tsoi andrew
` (7 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-08 3:35 UTC (permalink / raw)
To: linux-sctp
Dear Neil,
The 176 routing is ok because when I shift 172.28.129.176 to primary address.
The 176 can be establish but 48 failed.
I guess it is related to primary and secondary setting. Can secondary
IP be established also?
sctp 172.28.129.176:9082
LISTEN
172.28.129.48:9082
sctp 0 0 172.28.129.176:9082 10.82.29.240:9082
ESTABLISHED
Regards,
Andrew
On Thu, Nov 7, 2013 at 10:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
>> Dear Neil,
>>
>> Attached file is our tcpdump.
>> Now our secondary IP doesn't work.
>> Please advise. Thanks.
>>
>> IP info:
>> lksctp primary IP is 172.28.129.49
>> lksctp secondary IP is 172.28.129.176
>>
>> Client IP is 10.82.29.240/10.82.29.241
>>
>>
> I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
>
> Looking at this, its not just heartbeats that arent getting answered, nothing is
> getting responded to via the 176 address. The only data that I see come out of
> that address are two abort frames (not sure what the cause is there yet).
>
> I would start by verifying basic connectivity via that address. I'd try
> something like:
> ping -I 172.28.129.176 10.82.29.240
> and
> ping -I 172.28.129.176 10.82.29.241
>
> while running a tcpdump on the interface that holds the 176 address specifically
> to make sure that you're routing tables locally and network in generally can
> pass traffic over that address
>
> Neil
>
>>
>>
>>
>> Regards,
>> Andrew
>>
>> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> > Dear Neil,
>> >
>> > For more information, our secondary IP doesn't response COOKIE_ECHO
>> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
>> >
>> > Regards,
>> > Andrew
>> >
>> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> >> Dear Neil,
>> >>
>> >> Please find my network diagram first.
>> >> Now at server side, can't see our lksctp client 4 connection but only
>> >> 2 2 connection. The reason is that heartbeat ACK not found.
>> >>
>> >> Regards,
>> >> Andrew
>> >>
>> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>> >>>> Dear Sir,
>> >>>>
>> >>>> I got a problem when set multi-homing to active/active.
>> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
>> >>>> 2 IP of destination client with total 4 connections.
>> >>>> The flow is that client send "INIT" message with 2 (primary and
>> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
>> >>>> ACK.
>> >>>> Moreover, we already proved our connectivity correct because when our
>> >>>> lksctp server send heartbeat to both destination client and they can
>> >>>> return ACK.
>> >>>> Would you mind sharing if lksctp lib already configure those
>> >>>> multi-homing purpose.
>> >>>>
>> >>> Can you provide a network diagram and a tcpdump of your connection?
>> >>> Neil
>> >>>
>> >>>>
>> >>>>
>> >>>> Best Regards,
>> >>>> Andrew Tsoi
>> >>>> --
>> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>> >>>> the body of a message to majordomo@vger.kernel.org
>> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> >>>>
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (5 preceding siblings ...)
2013-11-08 3:35 ` tsoi andrew
@ 2013-11-08 4:18 ` tsoi andrew
2013-11-08 7:47 ` tsoi andrew
` (6 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-08 4:18 UTC (permalink / raw)
To: linux-sctp
Dear Neil,
For more information, our application make use of
lksctp-tools-1.0.9-1.src in redhat Linux 6 64 bit environment
Regards,
Andrew
On Fri, Nov 8, 2013 at 11:35 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> Dear Neil,
>
> The 176 routing is ok because when I shift 172.28.129.176 to primary address.
> The 176 can be establish but 48 failed.
> I guess it is related to primary and secondary setting. Can secondary
> IP be established also?
>
> sctp 172.28.129.176:9082
> LISTEN
> 172.28.129.48:9082
>
> sctp 0 0 172.28.129.176:9082 10.82.29.240:9082
> ESTABLISHED
> Regards,
> Andrew
>
> On Thu, Nov 7, 2013 at 10:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
>>> Dear Neil,
>>>
>>> Attached file is our tcpdump.
>>> Now our secondary IP doesn't work.
>>> Please advise. Thanks.
>>>
>>> IP info:
>>> lksctp primary IP is 172.28.129.49
>>> lksctp secondary IP is 172.28.129.176
>>>
>>> Client IP is 10.82.29.240/10.82.29.241
>>>
>>>
>> I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
>>
>> Looking at this, its not just heartbeats that arent getting answered, nothing is
>> getting responded to via the 176 address. The only data that I see come out of
>> that address are two abort frames (not sure what the cause is there yet).
>>
>> I would start by verifying basic connectivity via that address. I'd try
>> something like:
>> ping -I 172.28.129.176 10.82.29.240
>> and
>> ping -I 172.28.129.176 10.82.29.241
>>
>> while running a tcpdump on the interface that holds the 176 address specifically
>> to make sure that you're routing tables locally and network in generally can
>> pass traffic over that address
>>
>> Neil
>>
>>>
>>>
>>>
>>> Regards,
>>> Andrew
>>>
>>> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>>> > Dear Neil,
>>> >
>>> > For more information, our secondary IP doesn't response COOKIE_ECHO
>>> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
>>> >
>>> > Regards,
>>> > Andrew
>>> >
>>> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>>> >> Dear Neil,
>>> >>
>>> >> Please find my network diagram first.
>>> >> Now at server side, can't see our lksctp client 4 connection but only
>>> >> 2 2 connection. The reason is that heartbeat ACK not found.
>>> >>
>>> >> Regards,
>>> >> Andrew
>>> >>
>>> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>>> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>>> >>>> Dear Sir,
>>> >>>>
>>> >>>> I got a problem when set multi-homing to active/active.
>>> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
>>> >>>> 2 IP of destination client with total 4 connections.
>>> >>>> The flow is that client send "INIT" message with 2 (primary and
>>> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>>> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
>>> >>>> ACK.
>>> >>>> Moreover, we already proved our connectivity correct because when our
>>> >>>> lksctp server send heartbeat to both destination client and they can
>>> >>>> return ACK.
>>> >>>> Would you mind sharing if lksctp lib already configure those
>>> >>>> multi-homing purpose.
>>> >>>>
>>> >>> Can you provide a network diagram and a tcpdump of your connection?
>>> >>> Neil
>>> >>>
>>> >>>>
>>> >>>>
>>> >>>> Best Regards,
>>> >>>> Andrew Tsoi
>>> >>>> --
>>> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>>> >>>> the body of a message to majordomo@vger.kernel.org
>>> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>> >>>>
>>
>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (6 preceding siblings ...)
2013-11-08 4:18 ` tsoi andrew
@ 2013-11-08 7:47 ` tsoi andrew
2013-11-08 12:12 ` Neil Horman
` (5 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-08 7:47 UTC (permalink / raw)
To: linux-sctp
Dear Neil,
Function related to Accept and Bind are listed below.
int SctpAcceptor::accept(
Handle *newHandle,
SctpAssociation *newChannel,
InetAddr *remoteAddrCopy)
{
void *addrPtr = NULL;
SOCKLEN_T peersize = 0;
if (remoteAddrCopy) {
addrPtr = remoteAddrCopy->getAddr();
peersize = remoteAddrCopy->size();
}
SOCKET newSockFd = ::accept((SOCKET)handle.fd, (struct
sockaddr *)addrPtr, &peersize);
if (newSockFd = INVALID_SOCKET) {
int errorCode = errno;
if (errorCode) perror("accept ");
return -1;
} else {
// We should check the validity of the new socket here
//make sure we receive MSG_NOTIFICATION (without this,
we can't read sinfo_ppid correctly)
struct sctp_event_subscribe event;
memset(&event, 0, sizeof(event));
event.sctp_data_io_event = 1;
event.sctp_association_event = 1;
event.sctp_address_event = 1;
event.sctp_send_failure_event = 1;
event.sctp_peer_error_event = 1;
event.sctp_partial_delivery_event = 1;
event.sctp_shutdown_event = 1;
// Must hardcode sizeof(sctp_event_subscribe) to 8;
otherwise will fail
if (::setsockopt(newSockFd, IPPROTO_SCTP, SCTP_EVENTS,
&event, 8) < 0) {
perror("setsockopt(SCTP_EVENTS): ");
// Unable to set socket option
CLOSE_SOCKET(newSockFd);
return -3;
}
Handle h((int *)newSockFd, Handle::SOCKET);
if (newHandle) {
*newHandle = h;
}
if (newChannel) {
newChannel->setHandle(&h); // Once the
handle is set, we can retrieve the local & remote addr
newChannel->setReadStatus(StreamConnection::STATUS_READ_READY);
}
if (!newHandle && !newChannel) {
// Both pointers are null is disallowed.
CLOSE_SOCKET(newSockFd);
return -2;
}
return 0;
}
}
int SctpAcceptor::bind(SOCKET fd, const MultiHomedInetAddr &addr)
{
char ipStr[128];
int rc;
int secondaryAddrCount = addr.getSecondaryAddrCount();
InetAddr primaryAddr = addr; // Get the
primary address first
if (secondaryAddrCount = 0 && primaryAddr.getPort() = 0) {
return -1; // Acceptor must have a port to bind
}
primaryAddr.getIp(ipStr);
printf("fd=%d. Binding to primary addr %s:%d...\n", (int)fd,
ipStr, primaryAddr.getPort());
// We must call bind() first before calling bindx()
if ((rc = ::bind(fd, (struct sockaddr *)primaryAddr.getAddr(),
primaryAddr.size())) < 0) {
int errorCode = errno;
if (errorCode) perror("bind ");
}
if (!rc && secondaryAddrCount > 0) {
SOCKADDR_IN *addrList = new SOCKADDR_IN[secondaryAddrCount];
addr.getSecondaryAddr(addrList, secondaryAddrCount);
for(int i=0; i<secondaryAddrCount;i++) {
char ip[32];
inet_ntop(addrList[i].sin_family, &addrList[i].sin_addr, ip,
(addrList[i].sin_family = AF_INET) ?
INET_ADDRSTRLEN : INET6_ADDRSTRLEN);
unsigned port = ntohs(addrList[i].sin_port);
printf("Bind to secondary (%s : %d)\n", ip, port);
}
rc = sctp_bindx(fd, (struct sockaddr *)addrList,
secondaryAddrCount, SCTP_BINDX_ADD_ADDR);
delete [] addrList;
if (rc < 0) {
int errorCode = errno;
if (errorCode) perror("bindx ");
}
}
// No need to close the socket if rc < 0, because outer
function will close it for us
return rc;
}
Regards,
Andrew
On Fri, Nov 8, 2013 at 12:18 PM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> Dear Neil,
>
> For more information, our application make use of
> lksctp-tools-1.0.9-1.src in redhat Linux 6 64 bit environment
>
> Regards,
> Andrew
>
> On Fri, Nov 8, 2013 at 11:35 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> Dear Neil,
>>
>> The 176 routing is ok because when I shift 172.28.129.176 to primary address.
>> The 176 can be establish but 48 failed.
>> I guess it is related to primary and secondary setting. Can secondary
>> IP be established also?
>>
>> sctp 172.28.129.176:9082
>> LISTEN
>> 172.28.129.48:9082
>>
>> sctp 0 0 172.28.129.176:9082 10.82.29.240:9082
>> ESTABLISHED
>> Regards,
>> Andrew
>>
>> On Thu, Nov 7, 2013 at 10:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>>> On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
>>>> Dear Neil,
>>>>
>>>> Attached file is our tcpdump.
>>>> Now our secondary IP doesn't work.
>>>> Please advise. Thanks.
>>>>
>>>> IP info:
>>>> lksctp primary IP is 172.28.129.49
>>>> lksctp secondary IP is 172.28.129.176
>>>>
>>>> Client IP is 10.82.29.240/10.82.29.241
>>>>
>>>>
>>> I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
>>>
>>> Looking at this, its not just heartbeats that arent getting answered, nothing is
>>> getting responded to via the 176 address. The only data that I see come out of
>>> that address are two abort frames (not sure what the cause is there yet).
>>>
>>> I would start by verifying basic connectivity via that address. I'd try
>>> something like:
>>> ping -I 172.28.129.176 10.82.29.240
>>> and
>>> ping -I 172.28.129.176 10.82.29.241
>>>
>>> while running a tcpdump on the interface that holds the 176 address specifically
>>> to make sure that you're routing tables locally and network in generally can
>>> pass traffic over that address
>>>
>>> Neil
>>>
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Andrew
>>>>
>>>> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>>>> > Dear Neil,
>>>> >
>>>> > For more information, our secondary IP doesn't response COOKIE_ECHO
>>>> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
>>>> >
>>>> > Regards,
>>>> > Andrew
>>>> >
>>>> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>>>> >> Dear Neil,
>>>> >>
>>>> >> Please find my network diagram first.
>>>> >> Now at server side, can't see our lksctp client 4 connection but only
>>>> >> 2 2 connection. The reason is that heartbeat ACK not found.
>>>> >>
>>>> >> Regards,
>>>> >> Andrew
>>>> >>
>>>> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>>>> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>>>> >>>> Dear Sir,
>>>> >>>>
>>>> >>>> I got a problem when set multi-homing to active/active.
>>>> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
>>>> >>>> 2 IP of destination client with total 4 connections.
>>>> >>>> The flow is that client send "INIT" message with 2 (primary and
>>>> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>>>> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
>>>> >>>> ACK.
>>>> >>>> Moreover, we already proved our connectivity correct because when our
>>>> >>>> lksctp server send heartbeat to both destination client and they can
>>>> >>>> return ACK.
>>>> >>>> Would you mind sharing if lksctp lib already configure those
>>>> >>>> multi-homing purpose.
>>>> >>>>
>>>> >>> Can you provide a network diagram and a tcpdump of your connection?
>>>> >>> Neil
>>>> >>>
>>>> >>>>
>>>> >>>>
>>>> >>>> Best Regards,
>>>> >>>> Andrew Tsoi
>>>> >>>> --
>>>> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>>>> >>>> the body of a message to majordomo@vger.kernel.org
>>>> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>> >>>>
>>>
>>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (7 preceding siblings ...)
2013-11-08 7:47 ` tsoi andrew
@ 2013-11-08 12:12 ` Neil Horman
2013-11-09 2:44 ` tsoi andrew
` (4 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Neil Horman @ 2013-11-08 12:12 UTC (permalink / raw)
To: linux-sctp
On Fri, Nov 08, 2013 at 11:35:51AM +0800, tsoi andrew wrote:
> Dear Neil,
>
> The 176 routing is ok because when I shift 172.28.129.176 to primary address.
> The 176 can be establish but 48 failed.
> I guess it is related to primary and secondary setting. Can secondary
> IP be established also?
>
> sctp 172.28.129.176:9082
> LISTEN
> 172.28.129.48:9082
>
> sctp 0 0 172.28.129.176:9082 10.82.29.240:9082
> ESTABLISHED
> Regards,
> Andrew
>
If thats the case I would suspect that something is wrong with either your
sysctl settings or your application. I can use the sctp_darn utility included
in lksctp-tools to set up a multihomed connection here without issue
Neil
> On Thu, Nov 7, 2013 at 10:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> > On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
> >> Dear Neil,
> >>
> >> Attached file is our tcpdump.
> >> Now our secondary IP doesn't work.
> >> Please advise. Thanks.
> >>
> >> IP info:
> >> lksctp primary IP is 172.28.129.49
> >> lksctp secondary IP is 172.28.129.176
> >>
> >> Client IP is 10.82.29.240/10.82.29.241
> >>
> >>
> > I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
> >
> > Looking at this, its not just heartbeats that arent getting answered, nothing is
> > getting responded to via the 176 address. The only data that I see come out of
> > that address are two abort frames (not sure what the cause is there yet).
> >
> > I would start by verifying basic connectivity via that address. I'd try
> > something like:
> > ping -I 172.28.129.176 10.82.29.240
> > and
> > ping -I 172.28.129.176 10.82.29.241
> >
> > while running a tcpdump on the interface that holds the 176 address specifically
> > to make sure that you're routing tables locally and network in generally can
> > pass traffic over that address
> >
> > Neil
> >
> >>
> >>
> >>
> >> Regards,
> >> Andrew
> >>
> >> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> >> > Dear Neil,
> >> >
> >> > For more information, our secondary IP doesn't response COOKIE_ECHO
> >> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
> >> >
> >> > Regards,
> >> > Andrew
> >> >
> >> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> >> >> Dear Neil,
> >> >>
> >> >> Please find my network diagram first.
> >> >> Now at server side, can't see our lksctp client 4 connection but only
> >> >> 2 2 connection. The reason is that heartbeat ACK not found.
> >> >>
> >> >> Regards,
> >> >> Andrew
> >> >>
> >> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> >> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
> >> >>>> Dear Sir,
> >> >>>>
> >> >>>> I got a problem when set multi-homing to active/active.
> >> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
> >> >>>> 2 IP of destination client with total 4 connections.
> >> >>>> The flow is that client send "INIT" message with 2 (primary and
> >> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
> >> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
> >> >>>> ACK.
> >> >>>> Moreover, we already proved our connectivity correct because when our
> >> >>>> lksctp server send heartbeat to both destination client and they can
> >> >>>> return ACK.
> >> >>>> Would you mind sharing if lksctp lib already configure those
> >> >>>> multi-homing purpose.
> >> >>>>
> >> >>> Can you provide a network diagram and a tcpdump of your connection?
> >> >>> Neil
> >> >>>
> >> >>>>
> >> >>>>
> >> >>>> Best Regards,
> >> >>>> Andrew Tsoi
> >> >>>> --
> >> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
> >> >>>> the body of a message to majordomo@vger.kernel.org
> >> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >> >>>>
> >
> >
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (8 preceding siblings ...)
2013-11-08 12:12 ` Neil Horman
@ 2013-11-09 2:44 ` tsoi andrew
2013-11-11 12:56 ` Neil Horman
` (3 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-09 2:44 UTC (permalink / raw)
To: linux-sctp
Dear Neil,
I agree with you so that I already review code related to Accept and
Bind are listed below but still can't find the problem at this moment.
Would you give me some hint?
Moreover, in your utility, can your secondary IP return heartbeat ACK to client?
int SctpAcceptor::accept(
Handle *newHandle,
SctpAssociation *newChannel,
InetAddr *remoteAddrCopy)
{
void *addrPtr = NULL;
SOCKLEN_T peersize = 0;
if (remoteAddrCopy) {
addrPtr = remoteAddrCopy->getAddr();
peersize = remoteAddrCopy->size();
}
SOCKET newSockFd = ::accept((SOCKET)handle.fd, (struct
sockaddr *)addrPtr, &peersize);
if (newSockFd = INVALID_SOCKET) {
int errorCode = errno;
if (errorCode) perror("accept ");
return -1;
} else {
// We should check the validity of the new socket here
//make sure we receive MSG_NOTIFICATION (without this,
we can't read sinfo_ppid correctly)
struct sctp_event_subscribe event;
memset(&event, 0, sizeof(event));
event.sctp_data_io_event = 1;
event.sctp_association_event = 1;
event.sctp_address_event = 1;
event.sctp_send_failure_event = 1;
event.sctp_peer_error_event = 1;
event.sctp_partial_delivery_event = 1;
event.sctp_shutdown_event = 1;
// Must hardcode sizeof(sctp_event_subscribe) to 8;
otherwise will fail
if (::setsockopt(newSockFd, IPPROTO_SCTP, SCTP_EVENTS,
& event, 8) < 0) {
perror("setsockopt(SCTP_EVENTS): ");
// Unable to set socket option
CLOSE_SOCKET(newSockFd);
return -3;
}
Handle h((int *)newSockFd, Handle::SOCKET);
if (newHandle) {
*newHandle = h;
}
if (newChannel) {
newChannel->setHandle(&h); // Once the
handle is set, we can retrieve the local & remote addr
newChannel->setReadStatus(StreamConnection::STATUS_READ_READY);
}
if (!newHandle && !newChannel) {
// Both pointers are null is disallowed.
CLOSE_SOCKET(newSockFd);
return -2;
}
return 0;
}
}
int SctpAcceptor::bind(SOCKET fd, const MultiHomedInetAddr &addr)
{
char ipStr[128];
int rc;
int secondaryAddrCount = addr.getSecondaryAddrCount();
InetAddr primaryAddr = addr; // Get the
primary address first
if (secondaryAddrCount = 0 && primaryAddr.getPort() = 0) {
return -1; // Acceptor must have a port to bind
}
primaryAddr.getIp(ipStr);
printf("fd=%d. Binding to primary addr %s:%d...\n", (int)fd,
ipStr, primaryAddr.getPort());
// We must call bind() first before calling bindx()
if ((rc = ::bind(fd, (struct sockaddr *)primaryAddr.getAddr(),
primaryAddr.size())) < 0) {
int errorCode = errno;
if (errorCode) perror("bind ");
}
if (!rc && secondaryAddrCount > 0) {
SOCKADDR_IN *addrList = new SOCKADDR_IN[secondaryAddrCount];
addr.getSecondaryAddr(addrList, secondaryAddrCount);
for(int i=0; i<secondaryAddrCount;i++) {
char ip[32];
inet_ntop(addrList[i].sin_family, &addrList[i].sin_addr, ip,
(addrList[i].sin_family = AF_INET) ?
INET_ADDRSTRLEN : INET6_ADDRSTRLEN);
unsigned port = ntohs(addrList[i].sin_port);
printf("Bind to secondary (%s : %d)\n", ip, port);
}
rc = sctp_bindx(fd, (struct sockaddr *)addrList,
secondaryAddrCount, SCTP_BINDX_ADD_ADDR);
delete [] addrList;
if (rc < 0) {
int errorCode = errno;
if (errorCode) perror("bindx ");
}
}
// No need to close the socket if rc < 0, because outer
function will close it for us
return rc;
}
Regards,
Andrew
On Fri, Nov 8, 2013 at 8:12 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> On Fri, Nov 08, 2013 at 11:35:51AM +0800, tsoi andrew wrote:
>> Dear Neil,
>>
>> The 176 routing is ok because when I shift 172.28.129.176 to primary address.
>> The 176 can be establish but 48 failed.
>> I guess it is related to primary and secondary setting. Can secondary
>> IP be established also?
>>
>> sctp 172.28.129.176:9082
>> LISTEN
>> 172.28.129.48:9082
>>
>> sctp 0 0 172.28.129.176:9082 10.82.29.240:9082
>> ESTABLISHED
>> Regards,
>> Andrew
>>
> If thats the case I would suspect that something is wrong with either your
> sysctl settings or your application. I can use the sctp_darn utility included
> in lksctp-tools to set up a multihomed connection here without issue
> Neil
>
>> On Thu, Nov 7, 2013 at 10:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> > On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
>> >> Dear Neil,
>> >>
>> >> Attached file is our tcpdump.
>> >> Now our secondary IP doesn't work.
>> >> Please advise. Thanks.
>> >>
>> >> IP info:
>> >> lksctp primary IP is 172.28.129.49
>> >> lksctp secondary IP is 172.28.129.176
>> >>
>> >> Client IP is 10.82.29.240/10.82.29.241
>> >>
>> >>
>> > I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
>> >
>> > Looking at this, its not just heartbeats that arent getting answered, nothing is
>> > getting responded to via the 176 address. The only data that I see come out of
>> > that address are two abort frames (not sure what the cause is there yet).
>> >
>> > I would start by verifying basic connectivity via that address. I'd try
>> > something like:
>> > ping -I 172.28.129.176 10.82.29.240
>> > and
>> > ping -I 172.28.129.176 10.82.29.241
>> >
>> > while running a tcpdump on the interface that holds the 176 address specifically
>> > to make sure that you're routing tables locally and network in generally can
>> > pass traffic over that address
>> >
>> > Neil
>> >
>> >>
>> >>
>> >>
>> >> Regards,
>> >> Andrew
>> >>
>> >> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> >> > Dear Neil,
>> >> >
>> >> > For more information, our secondary IP doesn't response COOKIE_ECHO
>> >> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
>> >> >
>> >> > Regards,
>> >> > Andrew
>> >> >
>> >> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> >> >> Dear Neil,
>> >> >>
>> >> >> Please find my network diagram first.
>> >> >> Now at server side, can't see our lksctp client 4 connection but only
>> >> >> 2 2 connection. The reason is that heartbeat ACK not found.
>> >> >>
>> >> >> Regards,
>> >> >> Andrew
>> >> >>
>> >> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> >> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>> >> >>>> Dear Sir,
>> >> >>>>
>> >> >>>> I got a problem when set multi-homing to active/active.
>> >> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
>> >> >>>> 2 IP of destination client with total 4 connections.
>> >> >>>> The flow is that client send "INIT" message with 2 (primary and
>> >> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>> >> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
>> >> >>>> ACK.
>> >> >>>> Moreover, we already proved our connectivity correct because when our
>> >> >>>> lksctp server send heartbeat to both destination client and they can
>> >> >>>> return ACK.
>> >> >>>> Would you mind sharing if lksctp lib already configure those
>> >> >>>> multi-homing purpose.
>> >> >>>>
>> >> >>> Can you provide a network diagram and a tcpdump of your connection?
>> >> >>> Neil
>> >> >>>
>> >> >>>>
>> >> >>>>
>> >> >>>> Best Regards,
>> >> >>>> Andrew Tsoi
>> >> >>>> --
>> >> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>> >> >>>> the body of a message to majordomo@vger.kernel.org
>> >> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> >> >>>>
>> >
>> >
>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (9 preceding siblings ...)
2013-11-09 2:44 ` tsoi andrew
@ 2013-11-11 12:56 ` Neil Horman
2013-11-12 2:05 ` tsoi andrew
` (2 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Neil Horman @ 2013-11-11 12:56 UTC (permalink / raw)
To: linux-sctp
On Sat, Nov 09, 2013 at 10:44:48AM +0800, tsoi andrew wrote:
> Dear Neil,
>
> I agree with you so that I already review code related to Accept and
> Bind are listed below but still can't find the problem at this moment.
> Would you give me some hint?
I really don't know, and can't guarantee that the problem is in these functions.
If I had the whole client and server program, perhaps, but I don't see any
problem here. The problem may also be on the connect side of the application.
> Moreover, in your utility, can your secondary IP return heartbeat ACK to client?
>
Yes, Heartbeak ACKs always need to be sent on the same transport that received
the heartbeat in the first place.
Neil
> int SctpAcceptor::accept(
> Handle *newHandle,
> SctpAssociation *newChannel,
> InetAddr *remoteAddrCopy)
> {
> void *addrPtr = NULL;
> SOCKLEN_T peersize = 0;
> if (remoteAddrCopy) {
> addrPtr = remoteAddrCopy->getAddr();
> peersize = remoteAddrCopy->size();
> }
> SOCKET newSockFd = ::accept((SOCKET)handle.fd, (struct
> sockaddr *)addrPtr, &peersize);
> if (newSockFd = INVALID_SOCKET) {
> int errorCode = errno;
> if (errorCode) perror("accept ");
> return -1;
> } else {
> // We should check the validity of the new socket here
> //make sure we receive MSG_NOTIFICATION (without this,
> we can't read sinfo_ppid correctly)
> struct sctp_event_subscribe event;
> memset(&event, 0, sizeof(event));
> event.sctp_data_io_event = 1;
> event.sctp_association_event = 1;
> event.sctp_address_event = 1;
> event.sctp_send_failure_event = 1;
> event.sctp_peer_error_event = 1;
> event.sctp_partial_delivery_event = 1;
> event.sctp_shutdown_event = 1;
> // Must hardcode sizeof(sctp_event_subscribe) to 8;
> otherwise will fail
> if (::setsockopt(newSockFd, IPPROTO_SCTP, SCTP_EVENTS,
> & event, 8) < 0) {
> perror("setsockopt(SCTP_EVENTS): ");
> // Unable to set socket option
> CLOSE_SOCKET(newSockFd);
> return -3;
> }
> Handle h((int *)newSockFd, Handle::SOCKET);
> if (newHandle) {
> *newHandle = h;
> }
> if (newChannel) {
> newChannel->setHandle(&h); // Once the
> handle is set, we can retrieve the local & remote addr
>
> newChannel->setReadStatus(StreamConnection::STATUS_READ_READY);
> }
> if (!newHandle && !newChannel) {
> // Both pointers are null is disallowed.
> CLOSE_SOCKET(newSockFd);
> return -2;
> }
> return 0;
> }
> }
> int SctpAcceptor::bind(SOCKET fd, const MultiHomedInetAddr &addr)
> {
> char ipStr[128];
> int rc;
> int secondaryAddrCount = addr.getSecondaryAddrCount();
> InetAddr primaryAddr = addr; // Get the
> primary address first
> if (secondaryAddrCount = 0 && primaryAddr.getPort() = 0) {
> return -1; // Acceptor must have a port to bind
> }
> primaryAddr.getIp(ipStr);
> printf("fd=%d. Binding to primary addr %s:%d...\n", (int)fd,
> ipStr, primaryAddr.getPort());
> // We must call bind() first before calling bindx()
> if ((rc = ::bind(fd, (struct sockaddr *)primaryAddr.getAddr(),
> primaryAddr.size())) < 0) {
> int errorCode = errno;
> if (errorCode) perror("bind ");
> }
>
> if (!rc && secondaryAddrCount > 0) {
> SOCKADDR_IN *addrList = new SOCKADDR_IN[secondaryAddrCount];
> addr.getSecondaryAddr(addrList, secondaryAddrCount);
> for(int i=0; i<secondaryAddrCount;i++) {
> char ip[32];
> inet_ntop(addrList[i].sin_family, &addrList[i].sin_addr, ip,
> (addrList[i].sin_family = AF_INET) ?
> INET_ADDRSTRLEN : INET6_ADDRSTRLEN);
> unsigned port = ntohs(addrList[i].sin_port);
> printf("Bind to secondary (%s : %d)\n", ip, port);
> }
> rc = sctp_bindx(fd, (struct sockaddr *)addrList,
> secondaryAddrCount, SCTP_BINDX_ADD_ADDR);
> delete [] addrList;
> if (rc < 0) {
> int errorCode = errno;
> if (errorCode) perror("bindx ");
> }
> }
> // No need to close the socket if rc < 0, because outer
> function will close it for us
> return rc;
> }
>
> Regards,
> Andrew
>
> On Fri, Nov 8, 2013 at 8:12 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> > On Fri, Nov 08, 2013 at 11:35:51AM +0800, tsoi andrew wrote:
> >> Dear Neil,
> >>
> >> The 176 routing is ok because when I shift 172.28.129.176 to primary address.
> >> The 176 can be establish but 48 failed.
> >> I guess it is related to primary and secondary setting. Can secondary
> >> IP be established also?
> >>
> >> sctp 172.28.129.176:9082
> >> LISTEN
> >> 172.28.129.48:9082
> >>
> >> sctp 0 0 172.28.129.176:9082 10.82.29.240:9082
> >> ESTABLISHED
> >> Regards,
> >> Andrew
> >>
> > If thats the case I would suspect that something is wrong with either your
> > sysctl settings or your application. I can use the sctp_darn utility included
> > in lksctp-tools to set up a multihomed connection here without issue
> > Neil
> >
> >> On Thu, Nov 7, 2013 at 10:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> >> > On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
> >> >> Dear Neil,
> >> >>
> >> >> Attached file is our tcpdump.
> >> >> Now our secondary IP doesn't work.
> >> >> Please advise. Thanks.
> >> >>
> >> >> IP info:
> >> >> lksctp primary IP is 172.28.129.49
> >> >> lksctp secondary IP is 172.28.129.176
> >> >>
> >> >> Client IP is 10.82.29.240/10.82.29.241
> >> >>
> >> >>
> >> > I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
> >> >
> >> > Looking at this, its not just heartbeats that arent getting answered, nothing is
> >> > getting responded to via the 176 address. The only data that I see come out of
> >> > that address are two abort frames (not sure what the cause is there yet).
> >> >
> >> > I would start by verifying basic connectivity via that address. I'd try
> >> > something like:
> >> > ping -I 172.28.129.176 10.82.29.240
> >> > and
> >> > ping -I 172.28.129.176 10.82.29.241
> >> >
> >> > while running a tcpdump on the interface that holds the 176 address specifically
> >> > to make sure that you're routing tables locally and network in generally can
> >> > pass traffic over that address
> >> >
> >> > Neil
> >> >
> >> >>
> >> >>
> >> >>
> >> >> Regards,
> >> >> Andrew
> >> >>
> >> >> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> >> >> > Dear Neil,
> >> >> >
> >> >> > For more information, our secondary IP doesn't response COOKIE_ECHO
> >> >> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
> >> >> >
> >> >> > Regards,
> >> >> > Andrew
> >> >> >
> >> >> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
> >> >> >> Dear Neil,
> >> >> >>
> >> >> >> Please find my network diagram first.
> >> >> >> Now at server side, can't see our lksctp client 4 connection but only
> >> >> >> 2 2 connection. The reason is that heartbeat ACK not found.
> >> >> >>
> >> >> >> Regards,
> >> >> >> Andrew
> >> >> >>
> >> >> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> >> >> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
> >> >> >>>> Dear Sir,
> >> >> >>>>
> >> >> >>>> I got a problem when set multi-homing to active/active.
> >> >> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
> >> >> >>>> 2 IP of destination client with total 4 connections.
> >> >> >>>> The flow is that client send "INIT" message with 2 (primary and
> >> >> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
> >> >> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
> >> >> >>>> ACK.
> >> >> >>>> Moreover, we already proved our connectivity correct because when our
> >> >> >>>> lksctp server send heartbeat to both destination client and they can
> >> >> >>>> return ACK.
> >> >> >>>> Would you mind sharing if lksctp lib already configure those
> >> >> >>>> multi-homing purpose.
> >> >> >>>>
> >> >> >>> Can you provide a network diagram and a tcpdump of your connection?
> >> >> >>> Neil
> >> >> >>>
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> Best Regards,
> >> >> >>>> Andrew Tsoi
> >> >> >>>> --
> >> >> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
> >> >> >>>> the body of a message to majordomo@vger.kernel.org
> >> >> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >> >> >>>>
> >> >
> >> >
> >>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (10 preceding siblings ...)
2013-11-11 12:56 ` Neil Horman
@ 2013-11-12 2:05 ` tsoi andrew
2013-11-12 13:43 ` Neil Horman
2013-11-15 21:36 ` Vlad Yasevich
13 siblings, 0 replies; 15+ messages in thread
From: tsoi andrew @ 2013-11-12 2:05 UTC (permalink / raw)
To: linux-sctp
Hi Neil,
Many thanks.
> Moreover, in your utility, can your secondary IP return heartbeat ACK to client?
>
Yes, Heartbeak ACKs always need to be sent on the same transport that received
the heartbeat in the first place.
Do you mean the lksctp server secondary IP can be connected to primary
IP and secondary IP for the client at the same time?
Regards,
Andrew
On Mon, Nov 11, 2013 at 8:56 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> On Sat, Nov 09, 2013 at 10:44:48AM +0800, tsoi andrew wrote:
>> Dear Neil,
>>
>> I agree with you so that I already review code related to Accept and
>> Bind are listed below but still can't find the problem at this moment.
>> Would you give me some hint?
> I really don't know, and can't guarantee that the problem is in these functions.
> If I had the whole client and server program, perhaps, but I don't see any
> problem here. The problem may also be on the connect side of the application.
>
>> Moreover, in your utility, can your secondary IP return heartbeat ACK to client?
>>
> Yes, Heartbeak ACKs always need to be sent on the same transport that received
> the heartbeat in the first place.
> Neil
>
>> int SctpAcceptor::accept(
>> Handle *newHandle,
>> SctpAssociation *newChannel,
>> InetAddr *remoteAddrCopy)
>> {
>> void *addrPtr = NULL;
>> SOCKLEN_T peersize = 0;
>> if (remoteAddrCopy) {
>> addrPtr = remoteAddrCopy->getAddr();
>> peersize = remoteAddrCopy->size();
>> }
>> SOCKET newSockFd = ::accept((SOCKET)handle.fd, (struct
>> sockaddr *)addrPtr, &peersize);
>> if (newSockFd = INVALID_SOCKET) {
>> int errorCode = errno;
>> if (errorCode) perror("accept ");
>> return -1;
>> } else {
>> // We should check the validity of the new socket here
>> //make sure we receive MSG_NOTIFICATION (without this,
>> we can't read sinfo_ppid correctly)
>> struct sctp_event_subscribe event;
>> memset(&event, 0, sizeof(event));
>> event.sctp_data_io_event = 1;
>> event.sctp_association_event = 1;
>> event.sctp_address_event = 1;
>> event.sctp_send_failure_event = 1;
>> event.sctp_peer_error_event = 1;
>> event.sctp_partial_delivery_event = 1;
>> event.sctp_shutdown_event = 1;
>> // Must hardcode sizeof(sctp_event_subscribe) to 8;
>> otherwise will fail
>> if (::setsockopt(newSockFd, IPPROTO_SCTP, SCTP_EVENTS,
>> & event, 8) < 0) {
>> perror("setsockopt(SCTP_EVENTS): ");
>> // Unable to set socket option
>> CLOSE_SOCKET(newSockFd);
>> return -3;
>> }
>> Handle h((int *)newSockFd, Handle::SOCKET);
>> if (newHandle) {
>> *newHandle = h;
>> }
>> if (newChannel) {
>> newChannel->setHandle(&h); // Once the
>> handle is set, we can retrieve the local & remote addr
>>
>> newChannel->setReadStatus(StreamConnection::STATUS_READ_READY);
>> }
>> if (!newHandle && !newChannel) {
>> // Both pointers are null is disallowed.
>> CLOSE_SOCKET(newSockFd);
>> return -2;
>> }
>> return 0;
>> }
>> }
>> int SctpAcceptor::bind(SOCKET fd, const MultiHomedInetAddr &addr)
>> {
>> char ipStr[128];
>> int rc;
>> int secondaryAddrCount = addr.getSecondaryAddrCount();
>> InetAddr primaryAddr = addr; // Get the
>> primary address first
>> if (secondaryAddrCount = 0 && primaryAddr.getPort() = 0) {
>> return -1; // Acceptor must have a port to bind
>> }
>> primaryAddr.getIp(ipStr);
>> printf("fd=%d. Binding to primary addr %s:%d...\n", (int)fd,
>> ipStr, primaryAddr.getPort());
>> // We must call bind() first before calling bindx()
>> if ((rc = ::bind(fd, (struct sockaddr *)primaryAddr.getAddr(),
>> primaryAddr.size())) < 0) {
>> int errorCode = errno;
>> if (errorCode) perror("bind ");
>> }
>>
>> if (!rc && secondaryAddrCount > 0) {
>> SOCKADDR_IN *addrList = new SOCKADDR_IN[secondaryAddrCount];
>> addr.getSecondaryAddr(addrList, secondaryAddrCount);
>> for(int i=0; i<secondaryAddrCount;i++) {
>> char ip[32];
>> inet_ntop(addrList[i].sin_family, &addrList[i].sin_addr, ip,
>> (addrList[i].sin_family = AF_INET) ?
>> INET_ADDRSTRLEN : INET6_ADDRSTRLEN);
>> unsigned port = ntohs(addrList[i].sin_port);
>> printf("Bind to secondary (%s : %d)\n", ip, port);
>> }
>> rc = sctp_bindx(fd, (struct sockaddr *)addrList,
>> secondaryAddrCount, SCTP_BINDX_ADD_ADDR);
>> delete [] addrList;
>> if (rc < 0) {
>> int errorCode = errno;
>> if (errorCode) perror("bindx ");
>> }
>> }
>> // No need to close the socket if rc < 0, because outer
>> function will close it for us
>> return rc;
>> }
>>
>> Regards,
>> Andrew
>>
>> On Fri, Nov 8, 2013 at 8:12 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> > On Fri, Nov 08, 2013 at 11:35:51AM +0800, tsoi andrew wrote:
>> >> Dear Neil,
>> >>
>> >> The 176 routing is ok because when I shift 172.28.129.176 to primary address.
>> >> The 176 can be establish but 48 failed.
>> >> I guess it is related to primary and secondary setting. Can secondary
>> >> IP be established also?
>> >>
>> >> sctp 172.28.129.176:9082
>> >> LISTEN
>> >> 172.28.129.48:9082
>> >>
>> >> sctp 0 0 172.28.129.176:9082 10.82.29.240:9082
>> >> ESTABLISHED
>> >> Regards,
>> >> Andrew
>> >>
>> > If thats the case I would suspect that something is wrong with either your
>> > sysctl settings or your application. I can use the sctp_darn utility included
>> > in lksctp-tools to set up a multihomed connection here without issue
>> > Neil
>> >
>> >> On Thu, Nov 7, 2013 at 10:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> >> > On Thu, Nov 07, 2013 at 11:45:25AM +0800, tsoi andrew wrote:
>> >> >> Dear Neil,
>> >> >>
>> >> >> Attached file is our tcpdump.
>> >> >> Now our secondary IP doesn't work.
>> >> >> Please advise. Thanks.
>> >> >>
>> >> >> IP info:
>> >> >> lksctp primary IP is 172.28.129.49
>> >> >> lksctp secondary IP is 172.28.129.176
>> >> >>
>> >> >> Client IP is 10.82.29.240/10.82.29.241
>> >> >>
>> >> >>
>> >> > I think you mean the lksctp primary is 172.28.129.48, not 49, but no matter.
>> >> >
>> >> > Looking at this, its not just heartbeats that arent getting answered, nothing is
>> >> > getting responded to via the 176 address. The only data that I see come out of
>> >> > that address are two abort frames (not sure what the cause is there yet).
>> >> >
>> >> > I would start by verifying basic connectivity via that address. I'd try
>> >> > something like:
>> >> > ping -I 172.28.129.176 10.82.29.240
>> >> > and
>> >> > ping -I 172.28.129.176 10.82.29.241
>> >> >
>> >> > while running a tcpdump on the interface that holds the 176 address specifically
>> >> > to make sure that you're routing tables locally and network in generally can
>> >> > pass traffic over that address
>> >> >
>> >> > Neil
>> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> Regards,
>> >> >> Andrew
>> >> >>
>> >> >> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> >> >> > Dear Neil,
>> >> >> >
>> >> >> > For more information, our secondary IP doesn't response COOKIE_ECHO
>> >> >> > also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
>> >> >> >
>> >> >> > Regards,
>> >> >> > Andrew
>> >> >> >
>> >> >> > On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> >> >> >> Dear Neil,
>> >> >> >>
>> >> >> >> Please find my network diagram first.
>> >> >> >> Now at server side, can't see our lksctp client 4 connection but only
>> >> >> >> 2 2 connection. The reason is that heartbeat ACK not found.
>> >> >> >>
>> >> >> >> Regards,
>> >> >> >> Andrew
>> >> >> >>
>> >> >> >> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>> >> >> >>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>> >> >> >>>> Dear Sir,
>> >> >> >>>>
>> >> >> >>>> I got a problem when set multi-homing to active/active.
>> >> >> >>>> That mean, It is expected that our lksctp server have 2 IP to connect
>> >> >> >>>> 2 IP of destination client with total 4 connections.
>> >> >> >>>> The flow is that client send "INIT" message with 2 (primary and
>> >> >> >>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>> >> >> >>>> when second client send 'heartbeat' to us, lksctp server cannot return
>> >> >> >>>> ACK.
>> >> >> >>>> Moreover, we already proved our connectivity correct because when our
>> >> >> >>>> lksctp server send heartbeat to both destination client and they can
>> >> >> >>>> return ACK.
>> >> >> >>>> Would you mind sharing if lksctp lib already configure those
>> >> >> >>>> multi-homing purpose.
>> >> >> >>>>
>> >> >> >>> Can you provide a network diagram and a tcpdump of your connection?
>> >> >> >>> Neil
>> >> >> >>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> Best Regards,
>> >> >> >>>> Andrew Tsoi
>> >> >> >>>> --
>> >> >> >>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>> >> >> >>>> the body of a message to majordomo@vger.kernel.org
>> >> >> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> >> >> >>>>
>> >> >
>> >> >
>> >>
>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (11 preceding siblings ...)
2013-11-12 2:05 ` tsoi andrew
@ 2013-11-12 13:43 ` Neil Horman
2013-11-15 21:36 ` Vlad Yasevich
13 siblings, 0 replies; 15+ messages in thread
From: Neil Horman @ 2013-11-12 13:43 UTC (permalink / raw)
To: linux-sctp
On Tue, Nov 12, 2013 at 10:05:25AM +0800, tsoi andrew wrote:
> Hi Neil,
>
> Many thanks.
> > Moreover, in your utility, can your secondary IP return heartbeat ACK to client?
> >
> Yes, Heartbeak ACKs always need to be sent on the same transport that received
> the heartbeat in the first place.
You seem like you know this. Why did you ask the question then?
> Do you mean the lksctp server secondary IP can be connected to primary
> IP and secondary IP for the client at the same time?
>
I never made that assertion previously. You can have two transports, one
between each peer address from a single local address, but you will only wind up
using a single transport at any one time (unless you are using round robin
transmission).
Neil
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Multi-homing question
2013-11-04 1:59 Multi-homing question tsoi andrew
` (12 preceding siblings ...)
2013-11-12 13:43 ` Neil Horman
@ 2013-11-15 21:36 ` Vlad Yasevich
13 siblings, 0 replies; 15+ messages in thread
From: Vlad Yasevich @ 2013-11-15 21:36 UTC (permalink / raw)
To: linux-sctp
On 11/06/2013 10:45 PM, tsoi andrew wrote:
> Dear Neil,
>
> Attached file is our tcpdump.
> Now our secondary IP doesn't work.
> Please advise. Thanks.
>
> IP info:
> lksctp primary IP is 172.28.129.49
> lksctp secondary IP is 172.28.129.176
>
> Client IP is 10.82.29.240/10.82.29.241
>
>
Since it appears that you are using multiple interfaces
in the same subnet, can you please make sure that:
1) you have arp_ignore set correctly.
2) you have route rules that specify that different source
addresses should be used for different destinations.
Without this, linux will just pick the first/best address
and interface to use and you will not have real multi-homing.
You alternative of course is to set up secondary ip in a different
subnet. This way you would have more appropriate multi-homing
configuration.
-vlad
>
>
>
> Regards,
> Andrew
>
> On Thu, Nov 7, 2013 at 10:43 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>> Dear Neil,
>>
>> For more information, our secondary IP doesn't response COOKIE_ECHO
>> also. Only our primary IP can response COOKIE_ECHO with COOKIE_ACK.
>>
>> Regards,
>> Andrew
>>
>> On Thu, Nov 7, 2013 at 10:36 AM, tsoi andrew <tsoiandrew@gmail.com> wrote:
>>> Dear Neil,
>>>
>>> Please find my network diagram first.
>>> Now at server side, can't see our lksctp client 4 connection but only
>>> 2 2 connection. The reason is that heartbeat ACK not found.
>>>
>>> Regards,
>>> Andrew
>>>
>>> On Mon, Nov 4, 2013 at 8:09 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>>>> On Mon, Nov 04, 2013 at 09:59:44AM +0800, tsoi andrew wrote:
>>>>> Dear Sir,
>>>>>
>>>>> I got a problem when set multi-homing to active/active.
>>>>> That mean, It is expected that our lksctp server have 2 IP to connect
>>>>> 2 IP of destination client with total 4 connections.
>>>>> The flow is that client send "INIT" message with 2 (primary and
>>>>> secondary) IP to our server and lksctp server return INIT_ACK. But
>>>>> when second client send 'heartbeat' to us, lksctp server cannot return
>>>>> ACK.
>>>>> Moreover, we already proved our connectivity correct because when our
>>>>> lksctp server send heartbeat to both destination client and they can
>>>>> return ACK.
>>>>> Would you mind sharing if lksctp lib already configure those
>>>>> multi-homing purpose.
>>>>>
>>>> Can you provide a network diagram and a tcpdump of your connection?
>>>> Neil
>>>>
>>>>>
>>>>>
>>>>> Best Regards,
>>>>> Andrew Tsoi
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2013-11-15 21:36 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-11-04 1:59 Multi-homing question tsoi andrew
2013-11-04 12:09 ` Neil Horman
2013-11-07 2:36 ` tsoi andrew
2013-11-07 2:43 ` tsoi andrew
2013-11-07 3:45 ` tsoi andrew
2013-11-07 14:16 ` Neil Horman
2013-11-08 3:35 ` tsoi andrew
2013-11-08 4:18 ` tsoi andrew
2013-11-08 7:47 ` tsoi andrew
2013-11-08 12:12 ` Neil Horman
2013-11-09 2:44 ` tsoi andrew
2013-11-11 12:56 ` Neil Horman
2013-11-12 2:05 ` tsoi andrew
2013-11-12 13:43 ` Neil Horman
2013-11-15 21:36 ` Vlad Yasevich
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.