From mboxrd@z Thu Jan 1 00:00:00 1970 From: Florian Westphal Subject: Re: : getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, &salen) is in fact sometimes returning the source IP instead the destination IP Date: Sat, 12 Jan 2019 17:04:00 +0100 Message-ID: <20190112160400.dblitzk2ftlfzryd@breakpoint.cc> References: <20190107111753.aiabujukqx3eteqb@breakpoint.cc> <2e83651c-df8b-8341-4170-df328e3d756a@ltri.eu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Florian Westphal , Mohandass Roobesh , Willy Tarreau , netdev@vger.kernel.org To: Lukas Tribus Return-path: Received: from Chamillionaire.breakpoint.cc ([146.0.238.67]:34736 "EHLO Chamillionaire.breakpoint.cc" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725839AbfALQEH (ORCPT ); Sat, 12 Jan 2019 11:04:07 -0500 Content-Disposition: inline In-Reply-To: <2e83651c-df8b-8341-4170-df328e3d756a@ltri.eu> Sender: netdev-owner@vger.kernel.org List-ID: Lukas Tribus wrote: > The application (haproxy) needs to know the original destination IP > address, however it does not know whether -j REDIRECT was used or not. > Because of this the application always queries SO_ORIGINAL_DST, and > this includes configurations without -j REDIRECT. > > Are you saying the behavior of SO_ORIGINAL_DST is undefined when not > used with -j REDIRECT and that this issue does not happen when -j > REDIRECT is actually used? No, thats not what I said. Because OP provided a link that mentions TPROXY, I concluded OP was using TPROXY, so I pointed out that the error source can be completely avoided by not using SO_ORIGINAL_DST. As I said, SO_ORIGINAL_DST returns the dst address of the original direction *as seen by conntrack*. In case REDIRECT or DNAT was used, the address returned is the on-wire one, before DNAT rewrite took place. Therefore, SO_ORIGINAL_DST is only needed when REDIRECT or DNAT was used. If no DNAT rewrite takes place, sockaddr returned by accept or getsockname can be used directly and SO_ORIGINAL_DST isn't needed. The returned address should be identical to the one given by accept(). If SO_ORIGINAL_DST returns the reply, then conntrack picked up a reply packet as the first packet of the connection, so it believes originator is the responder and vice versa. One case where this can happen is when nf_conntrack_tcp_loose (mid-stream pickup) is enabled. This is not a haproxy bug. Only thing that haproxy could is to provide a knob to make it only use addresses returned by accept, rather than relying on SO_ORIGINAL_DST for those that use TPROXY to do MITM interception. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_NEOMUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4035BC43387 for ; Sat, 12 Jan 2019 16:11:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0FB8220836 for ; Sat, 12 Jan 2019 16:11:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725847AbfALQEH (ORCPT ); Sat, 12 Jan 2019 11:04:07 -0500 Received: from Chamillionaire.breakpoint.cc ([146.0.238.67]:34736 "EHLO Chamillionaire.breakpoint.cc" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725839AbfALQEH (ORCPT ); Sat, 12 Jan 2019 11:04:07 -0500 Received: from fw by Chamillionaire.breakpoint.cc with local (Exim 4.89) (envelope-from ) id 1giLlQ-0006b7-PF; Sat, 12 Jan 2019 17:04:00 +0100 Date: Sat, 12 Jan 2019 17:04:00 +0100 From: Florian Westphal To: Lukas Tribus Cc: Florian Westphal , Mohandass Roobesh , Willy Tarreau , netdev@vger.kernel.org Subject: Re: [NETDEV]: getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, &salen) is in fact sometimes returning the source IP instead the destination IP Message-ID: <20190112160400.dblitzk2ftlfzryd@breakpoint.cc> References: <20190107111753.aiabujukqx3eteqb@breakpoint.cc> <2e83651c-df8b-8341-4170-df328e3d756a@ltri.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2e83651c-df8b-8341-4170-df328e3d756a@ltri.eu> User-Agent: NeoMutt/20170113 (1.7.2) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Message-ID: <20190112160400.pgAYI78lJl2Blv-D58BgUSvqMoueczgvbscl4lgxv_0@z> Lukas Tribus wrote: > The application (haproxy) needs to know the original destination IP > address, however it does not know whether -j REDIRECT was used or not. > Because of this the application always queries SO_ORIGINAL_DST, and > this includes configurations without -j REDIRECT. > > Are you saying the behavior of SO_ORIGINAL_DST is undefined when not > used with -j REDIRECT and that this issue does not happen when -j > REDIRECT is actually used? No, thats not what I said. Because OP provided a link that mentions TPROXY, I concluded OP was using TPROXY, so I pointed out that the error source can be completely avoided by not using SO_ORIGINAL_DST. As I said, SO_ORIGINAL_DST returns the dst address of the original direction *as seen by conntrack*. In case REDIRECT or DNAT was used, the address returned is the on-wire one, before DNAT rewrite took place. Therefore, SO_ORIGINAL_DST is only needed when REDIRECT or DNAT was used. If no DNAT rewrite takes place, sockaddr returned by accept or getsockname can be used directly and SO_ORIGINAL_DST isn't needed. The returned address should be identical to the one given by accept(). If SO_ORIGINAL_DST returns the reply, then conntrack picked up a reply packet as the first packet of the connection, so it believes originator is the responder and vice versa. One case where this can happen is when nf_conntrack_tcp_loose (mid-stream pickup) is enabled. This is not a haproxy bug. Only thing that haproxy could is to provide a knob to make it only use addresses returned by accept, rather than relying on SO_ORIGINAL_DST for those that use TPROXY to do MITM interception.