All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [PATCH net-next 2/3] tcp: implement coalescing on backlog queue
@ 2019-04-07 21:28 richard.purdie
  2019-04-24 14:51 ` Bruno Prémont
  0 siblings, 1 reply; 14+ messages in thread
From: richard.purdie @ 2019-04-07 21:28 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Neal Cardwell, Yuchung Cheng, David S. Miller, netdev,
	Alexander Kanavin, Bruce Ashfield

Hi,

I've been chasing down why a python test from the python3 testsuite
started failing and it seems to point to this kernel change in the
networking stack.

In kernels beyond commit 4f693b55c3d2d2239b8a0094b518a1e533cf75d5 the
test hangs about 90% of the time (I've reproduced with 5.1-rc3, 5.0.7,
5.0-rc1 but not 4.18, 4.19 or 4.20). The reproducer is:

$ python3 -m test test_httplib -v
== CPython 3.7.2 (default, Apr 5 2019, 15:17:15) [GCC 8.3.0]
== Linux-5.0.0-yocto-standard-x86_64-with-glibc2.2.5 little-endian
== cwd: /var/volatile/tmp/test_python_288
== CPU count: 1
== encodings: locale=UTF-8, FS=utf-8
[...]
test_response_fileno (test.test_httplib.BasicTest) ... 

and it hangs in test_response_fileno.

The test in question comes from Lib/test/test_httplib.py in the python
source tree and the code is:

    def test_response_fileno(self):
        # Make sure fd returned by fileno is valid.
        serv = socket.socket(
            socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP)
        self.addCleanup(serv.close)
        serv.bind((HOST, 0))
        serv.listen()

        result = None
        def run_server():
            [conn, address] = serv.accept()
            with conn, conn.makefile("rb") as reader:
                # Read the request header until a blank line
                while True:
                    line = reader.readline()
                    if not line.rstrip(b"\r\n"):
                        break
                conn.sendall(b"HTTP/1.1 200 Connection established\r\n\r\n")
                nonlocal result
                result = reader.read()

        thread = threading.Thread(target=run_server)
        thread.start()
        self.addCleanup(thread.join, float(1))
        conn = client.HTTPConnection(*serv.getsockname())
        conn.request("CONNECT", "dummy:1234")
        response = conn.getresponse()
        try:
            self.assertEqual(response.status, client.OK)
            s = socket.socket(fileno=response.fileno())
            try:
                s.sendall(b"proxied data\n")
            finally:
                s.detach()
        finally:
            response.close()
            conn.close()
        thread.join()
        self.assertEqual(result, b"proxied data\n")

I was hoping someone with more understanding of the networking stack
could look at this and tell whether its a bug in the python test, the
kernel change or otherwise give a pointer to where the problem might
be? I'll freely admit this is not an area I know much about.

Cheers,

Richard


^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCH net-next 0/3] tcp: take a bit more care of backlog stress
@ 2018-11-21 17:52 Eric Dumazet
  2018-11-21 17:52 ` [PATCH net-next 2/3] tcp: implement coalescing on backlog queue Eric Dumazet
  0 siblings, 1 reply; 14+ messages in thread
From: Eric Dumazet @ 2018-11-21 17:52 UTC (permalink / raw)
  To: David S . Miller
  Cc: netdev, Jean-Louis Dupond, Neal Cardwell, Yuchung Cheng,
	Eric Dumazet, Eric Dumazet

While working on the SACK compression issue Jean-Louis Dupond
reported, we found that his linux box was suffering very hard
from tail drops on the socket backlog queue, because the opposite
TCP stack was ont implementing latest RFC recommendations.

First patch is a cleanup

Second patch is attempting coalescing when a new packet must
be added to the backlog queue. Cooking bigger skbs helps
to keep backlog list smaller and speeds its handling when
user thread finally releases the socket lock.

Third patch is implementing head drop as a last resort.
Head drops are generally better for optimal TCP behavior.

Eric Dumazet (3):
  tcp: remove hdrlen argument from tcp_queue_rcv()
  tcp: implement coalescing on backlog queue
  tcp: implement head drops in backlog queue

 include/uapi/linux/snmp.h |  1 +
 net/ipv4/proc.c           |  1 +
 net/ipv4/tcp_input.c      | 13 +++---
 net/ipv4/tcp_ipv4.c       | 89 ++++++++++++++++++++++++++++++++++++---
 4 files changed, 91 insertions(+), 13 deletions(-)

-- 
2.19.1.1215.g8438c0b245-goog

^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2019-04-25 14:16 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-07 21:28 [PATCH net-next 2/3] tcp: implement coalescing on backlog queue richard.purdie
2019-04-24 14:51 ` Bruno Prémont
2019-04-24 15:47   ` Eric Dumazet
2019-04-25  7:55     ` Bruno Prémont
2019-04-25 13:13       ` Bruno Prémont
2019-04-25 13:30         ` Eric Dumazet
2019-04-25 14:16           ` Bruno Prémont
  -- strict thread matches above, loose matches on Subject: below --
2018-11-21 17:52 [PATCH net-next 0/3] tcp: take a bit more care of backlog stress Eric Dumazet
2018-11-21 17:52 ` [PATCH net-next 2/3] tcp: implement coalescing on backlog queue Eric Dumazet
2018-11-21 22:31   ` Yuchung Cheng
2018-11-21 22:40     ` Eric Dumazet
2018-11-22 16:34       ` Yuchung Cheng
2018-11-22 18:01   ` Neal Cardwell
2018-11-22 18:16     ` Eric Dumazet
2018-11-22 18:21       ` Eric Dumazet

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.