Re: [PATCH v2] nvme-tcp: send quota for nvme_tcp_send_all()

[Daniel Wagner <dwagner@suse.de>]

On Tue, Oct 25, 2022 at 12:54:43AM +0000, Chaitanya Kulkarni wrote:
> On 10/24/22 08:03, Daniel Wagner wrote:
> > Add a send quota in nvme_tcp_send_all() to avoid stalls when sending
> > large amounts of requests.
> > 
> > Cc: Hannes Reinecke <hare@suse.de>
> > Signed-off-by: Daniel Wagner <dwagner@suse.de>
> > ---
> > 
> > IMO, this patch might still be a good idea to add. At least in my test
> > setup where I only have one ethernet port it makes a big difference
> > when accessing the system via ssh. When nvme-tcp is pushing a lot of
> > data via the network, the ssh session is completely blocked by the
> > storage traffic. With it, the ssh session stays responsive. >
> 
> I'm not sure whether it is possible but is there a way to gather
> some form of quantitative data and present it here to we all know
> exactly which aspect is improving by this patch in context of
> "ssd session is completely blocked" ?

Before starting a fio test run, the remote shell works fine and when fio
runs, no keystroke gets echoed until the fio stops again. Obviously,
this is heavily depending on the workload. So my observation is that
limiting an unbound send loop prevents a 'fair' usage of the bandwidth.

> > +	unsigned long deadline = jiffies + msecs_to_jiffies(1);
> 
> We need to provide some flexibility to set this value as one specified
> value may not work with all the setups and H/W.

Note, this is the identically approach we have in nvme_tcp_io_work()
already. So nothing new. Though I was wondering too why 1 jiffy.

> Can we make it tunable and not statically coded ?

I would really like to avoid having a tunable knob like a sysfs
entry. None will get it right. That means this would need to be
algorithm which auto adapts, though I don't have any good idea how this
algorithm should operate. Any ideas?