From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A83EDC282C0 for ; Wed, 23 Jan 2019 08:09:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4F55921019 for ; Wed, 23 Jan 2019 08:09:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=mcafee.com header.i=@mcafee.com header.b="SYybLGYC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726253AbfAWIJV (ORCPT ); Wed, 23 Jan 2019 03:09:21 -0500 Received: from dnvwsmailout1.mcafee.com ([161.69.31.173]:51269 "EHLO DNVWSMAILOUT1.mcafee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726000AbfAWIJU (ORCPT ); Wed, 23 Jan 2019 03:09:20 -0500 X-NAI-Header: Modified by McAfee Email Gateway (5500) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mcafee.com; s=s_mcafee; t=1548230891; h=From: To:CC:Subject:Thread-Topic:Thread-Index:Date: Message-ID:References:In-Reply-To:Accept-Language: Content-Language:X-MS-Has-Attach:X-MS-TNEF-Correlator: dlp-product:dlp-version:dlp-reaction:authentication-results: x-originating-ip:x-ms-publictraffictype:x-microsoft-exchange-diagnostics: x-ms-office365-filtering-correlation-id:x-microsoft-antispam: x-ms-traffictypediagnostic:x-microsoft-antispam-prvs: x-forefront-prvs:x-forefront-antispam-report: received-spf:x-ms-exchange-senderadcheck:x-microsoft-antispam-message-info: spamdiagnosticoutput:spamdiagnosticmetadata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-Transport-CrossTenantHeadersStamped: X-OriginatorOrg:X-NAI-Spam-Flag:X-NAI-Spam-Level: X-NAI-Spam-Threshold:X-NAI-Spam-Score:X-NAI-Spam-Version; bh=cxYHv8JAzBCcTfgzgg4IXkBvWL5F1cz0DmkwIH ht1iU=; b=SYybLGYC7V9OQgBR6smXCIrJTIxoZZgykqVH2Owr Cg+xv4fH+jjUS3OhKkIqpvDlaEn4uuS08xOoKWJk23BO0VxB6K oDPYlDVXw51btTCdM7A46wrKDsmOQbSDidIX8c5FY50CgRUi0u lHU67dLbkhNxF+esa6JpMH4N9tU7HU4= Received: from DNVEXAPP1N05.corpzone.internalzone.com (unknown [10.44.48.89]) by DNVWSMAILOUT1.mcafee.com with smtp (TLS: TLSv1/SSLv3,256bits,ECDHE-RSA-AES256-SHA384) id 5aee_ab38_89f3f1f9_0954_4d17_aa52_9b6de6cac0d0; Wed, 23 Jan 2019 01:08:11 -0700 Received: from DNVEXAPP1N05.corpzone.internalzone.com (10.44.48.89) by DNVEXAPP1N05.corpzone.internalzone.com (10.44.48.89) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Wed, 23 Jan 2019 01:07:50 -0700 Received: from DNVO365EDGE1.corpzone.internalzone.com (10.44.176.66) by DNVEXAPP1N05.corpzone.internalzone.com (10.44.48.89) with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Frontend Transport; Wed, 23 Jan 2019 01:07:50 -0700 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (10.44.176.242) by edge.mcafee.com (10.44.176.66) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Wed, 23 Jan 2019 01:07:50 -0700 Received: from MWHPR16MB1502.namprd16.prod.outlook.com (10.175.4.148) by MWHPR16MB1744.namprd16.prod.outlook.com (10.172.60.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1537.25; Wed, 23 Jan 2019 08:07:49 +0000 Received: from MWHPR16MB1502.namprd16.prod.outlook.com ([fe80::7823:6864:dd5d:59e7]) by MWHPR16MB1502.namprd16.prod.outlook.com ([fe80::7823:6864:dd5d:59e7%8]) with mapi id 15.20.1558.016; Wed, 23 Jan 2019 08:07:48 +0000 From: "Mohandass, Roobesh" To: Willy Tarreau , Lukas Tribus CC: Florian Westphal , "netdev@vger.kernel.org" Subject: RE: [NETDEV]: getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, &salen) is in fact sometimes returning the source IP instead the destination IP Thread-Topic: [NETDEV]: getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, &salen) is in fact sometimes returning the source IP instead the destination IP Thread-Index: AdSdH1A1+nnKpsXqSO65La6qqj5o4wAlbbAgAMaDbHAAAHGG4AFqcb6AAQJq5oAAAwg0AAAC4oAAAAE4oAAAAR+YAADf0kBwATNwPWA= Date: Wed, 23 Jan 2019 08:07:48 +0000 Message-ID: References: <20190107111753.aiabujukqx3eteqb@breakpoint.cc> <2e83651c-df8b-8341-4170-df328e3d756a@ltri.eu> <20190112160400.dblitzk2ftlfzryd@breakpoint.cc> <20190112172636.GA26639@1wt.eu> <20190112183344.GA26847@1wt.eu> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.1.100.18 dlp-reaction: no-action authentication-results: spf=none (sender IP is ) smtp.mailfrom=Roobesh_Mohandass@McAfee.com; x-originating-ip: [103.245.47.20] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;MWHPR16MB1744;6:mq4wdDTNrfESxAeSdda3fkaNxpbn6eBlBrn+WzEjG77s1GT9Yyoj5EMAYxRI1isSHgfN02DYq4PPULOsft7PoiSsQvBcmZveKzsTFznd+92KKiwlPDQ9fEps0yXLp3uKtjkiF+GWK3zPKv0qjoEpJpBJs0KBqmh5j1MjBKmQxlg5jVDunGdRJv1CZSz5fKIWERlgTEPg2Jv1591LBu937dX86DpyT7r9UUIEjVUGW3+xSbM/Smqr2oqua1C83vw6iXxhyTUxvOgBx5k5XVaDSd9X+xEevxZ7iJu4F9id1+WCIoATH5RxtUaL3OeZly+fAY6c+oFtseAxwQwPKWS1xLQ3Uz2uT6Vxd+xZyDunyH33BJEArL2wGRw6i7jHy/ELi5McKmVZTfNj3XC/TtKy5UxdvYrDFxV+B2Vyxvj6Nn/8j9CuA1ePzbCohrbMweNzMReR5sTlK5xH5zTyTFKiQw==;5:yQr4cwMNalKx7frKGqowh/m6RYhSfEuZR/mVPpseery5VL3w5vrv5Medjs0mHqtE0z1ZXq38J64askNrbYnHmc8k5ly01kH1hbSoJJ+bz3SW1gsiSb1M5nA8shXwixZGP2p8HEO7/de7HiQuhGIKcuFQnvp8zR+nhr/GA5JdElJHt+k/m5hiC65MWdMZwXarUERcWhSuWqtXwsxRVrwyyQ==;7:ew3Sh2qlQ41BC/a7Y9Kto3coD8q7859Y51Z/fIXy1DYoh6R2a8lhxL8p+/URdQV20z7SdbiuGWZ9UACWbvzYp67IXuGEmi0sOljPfUid55mLe9d2LkxUlbAaO8Qtr/lH7sdIXSGjxfi9oO/4WfVoYQ== x-ms-office365-filtering-correlation-id: 2aba7d3a-83c3-4db9-5b44-08d68109dd63 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600110)(711020)(4605077)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020);SRVR:MWHPR16MB1744; x-ms-traffictypediagnostic: MWHPR16MB1744: x-microsoft-antispam-prvs: x-forefront-prvs: 0926B0E013 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(376002)(366004)(346002)(136003)(396003)(39860400002)(13464003)(40764003)(32952001)(199004)(189003)(11346002)(446003)(72206003)(14454004)(99286004)(486006)(26005)(6246003)(478600001)(102836004)(966005)(97736004)(5024004)(74316002)(316002)(14444005)(7736002)(256004)(2906002)(305945005)(476003)(3846002)(6116002)(76176011)(33656002)(229853002)(68736007)(54906003)(93886005)(80792005)(55016002)(106356001)(6306002)(8676002)(6436002)(66066001)(81156014)(86362001)(71200400001)(71190400001)(6506007)(53936002)(53546011)(25786009)(186003)(4326008)(81166006)(110136005)(7696005)(105586002)(9686003)(8936002)(781001)(85282002);DIR:OUT;SFP:1101;SCL:1;SRVR:MWHPR16MB1744;H:MWHPR16MB1502.namprd16.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: McAfee.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: hQIE9rbwgJIlV6/ezvPxfvypOHJpuCSlOKvFuO08en9yNz3wdEOl9UNabE/V2bFoTBjMPp1K1TZzk1W788ZxNDhEYnUHFgAyFbzX7hYjy2req00b9VXPUdIx2OCNSirmEQASSRQwWdPIVFl9Pv+5R8yfWh3SXij0pyDze4POsCd+5btTpuC/qNv1m7xRhZNXIWPInkoDmPjmkHCWaQUmzdyxAsyHHwRipWaAYSBHT44n3fKMQaIOHmXmrR8T6FsSf2ue7nlHy+6+p/26ud9/PjvoyBWOYgNXkXvJWM8rqFb1Lxjmi8H4kvhEwASEUHNzNF5VPzBvfL09JCmV8+fKSOv/67nXLAQrpxbznW5SbbeLXMcbuPBZkiGa7JGoOt1P4OwK2P0eP+dOvwYJqlWmKdjDh6qFBi+ffNGCqDCoMZU= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 2aba7d3a-83c3-4db9-5b44-08d68109dd63 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jan 2019 08:07:48.7077 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4943e38c-6dd4-428c-886d-24932bc2d5de X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR16MB1744 X-OriginatorOrg: mcafee.com X-NAI-Spam-Flag: NO X-NAI-Spam-Level: X-NAI-Spam-Threshold: 15 X-NAI-Spam-Score: 0.5 X-NAI-Spam-Version: 2.3.0.9418 : core <6466> : inlines <6998> : streams <1810896> : uri <2784259> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Hi Willy/Florian/Lukas, disabling nf_conntrack_tcp_loose (solved the problem) and for last 24 hour= s, we are not evening seeing a single wrong connection data by send-proxy. Worth a note in your documentation related to this, as users might be aware= off. Thanks very much for your help and support, I will report back if I see any= thing again related to this. -Roobesh G M -----Original Message----- From: Mohandass, Roobesh=20 Sent: Thursday, January 17, 2019 10:54 AM To: Willy Tarreau ; Lukas Tribus Cc: Florian Westphal ; netdev@vger.kernel.org Subject: RE: [NETDEV]: getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, &salen) = is in fact sometimes returning the source IP instead the destination IP Hi Willy/Florian/Lukas, Thanks for your help around this concern, sorry for the delayed response. I= will test this out and get back to you. -Roobesh G M -----Original Message----- From: Willy Tarreau Sent: Sunday, January 13, 2019 12:04 AM To: Lukas Tribus Cc: Mohandass, Roobesh ; Florian Westphal ; netdev@vger.kernel.org Subject: Re: [NETDEV]: getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, &salen) = is in fact sometimes returning the source IP instead the destination IP This email originated from outside of the organization. Do not click links = or open attachments unless you recognize the sender and know the content is= safe. Hi Lukas, On Sat, Jan 12, 2019 at 07:01:34PM +0100, Lukas Tribus wrote: > > Roobesh, do you use the destination address only for logging or=20 > > anywhere else in the request path ? And could you check if you have=20 > > nf_conntrack_tcp_loose set as Florian suggests ? I really think he=20 > > figured it right. >=20 > It's about what we send with the PROXY protocol to the backend server,=20 > Roobesh reported things like that (src and dst is the same): >=20 > PROXY TCP4 192.220.26.39 192.220.26.39 45066 45066 PROXY TCP4 > 192.220.26.39 192.220.26.39 45075 45075 >=20 > So the call would actually happen at the beginning of the TCP connection. That sounds quite shocking to me then. Maybe we're facing such a sequence: 1) first session comes from this port, and client closes first (FIN) 2) haproxy then closes with (FIN) 3) client doesn't respond with the final ACK (e.g. blocked by another firewall in between or the client's own conntrack) 4) the socket on the haproxy side remains in LAST_ACK state and ACKs are periodically resent 5) local conntrack is in TIME_WAIT and expires faster than the largest interval between two such ACKs 6) one of these retransmitted ACKs reopens the connection in reverse direction due to nf_conntrack_tcp_loose. The connection is then seen in ESTABLISHED state and might be kept a bit longer. 8) the connection finally expires in the local TCP stack but not yet in conntrack. 7) later the client reuses the same source port while the connection is still present in the conntrack table. 8) assuming tcp_be_liberal is also set, the packets can pass through conntrack and establish a new connection to haproxy. 9) haproxy calls getsockopt(SO_ORIGINAL_DST) and gets the other end point since the connection was created at step 6 above in the other direction. I could be wrong on certain specific points above but it looks plausible.=20 > Initial report is here: > https://discourse.haproxy.org/t/send-proxy-not-modifying-some-traffic- > with-proxy-ip-port-details/3336 Ah cool, I didn't have all this, thank you! > Let's see if disabling nf_conntrack_tcp_loose changes things. Yes this really is the only thing I can think of, and in this case noone is= wrong in this chain (neither kernel nor haproxy). We'll need to document i= t in my opinion. Thanks, Willy