From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-fsdevel-owner@vger.kernel.org>
Received: from mail-qk0-f196.google.com ([209.85.220.196]:45869 "EHLO
        mail-qk0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1754417AbeF0TUu (ORCPT
        <rfc822;linux-fsdevel@vger.kernel.org>);
        Wed, 27 Jun 2018 15:20:50 -0400
Received: by mail-qk0-f196.google.com with SMTP id c198-v6so1667397qkg.12
        for <linux-fsdevel@vger.kernel.org>; Wed, 27 Jun 2018 12:20:49 -0700 (PDT)
Date: Wed, 27 Jun 2018 15:20:48 -0400
From: Josef Bacik <josef@toxicpanda.com>
To: Jens Axboe <axboe@kernel.dk>
Cc: Josef Bacik <josef@toxicpanda.com>, linux-block@vger.kernel.org,
        kernel-team@fb.com, akpm@linux-foundation.org, hannes@cmpxchg.org,
        linux-kernel@vger.kernel.org, tj@kernel.org,
        linux-fsdevel@vger.kernel.org, Josef Bacik <jbacik@fb.com>
Subject: Re: [PATCH 12/15] block: introduce blk-iolatency io controller
Message-ID: <20180627192046.ieqncfl6ioy37mof@destiny>
References: <20180625151243.2132-1-josef@toxicpanda.com>
 <20180625151243.2132-13-josef@toxicpanda.com>
 <05a581ed-8f21-9d89-a813-a03d802d3469@kernel.dk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <05a581ed-8f21-9d89-a813-a03d802d3469@kernel.dk>
Sender: linux-fsdevel-owner@vger.kernel.org
List-ID: <linux-fsdevel.vger.kernel.org>

On Wed, Jun 27, 2018 at 01:06:31PM -0600, Jens Axboe wrote:
> On 6/25/18 9:12 AM, Josef Bacik wrote:
> > +static void __blkcg_iolatency_throttle(struct rq_qos *rqos,
> > +				       struct iolatency_grp *iolat,
> > +				       spinlock_t *lock, bool issue_as_root,
> > +				       bool use_memdelay)
> > +	__releases(lock)
> > +	__acquires(lock)
> > +{
> > +	struct rq_wait *rqw = &iolat->rq_wait;
> > +	unsigned use_delay = atomic_read(&lat_to_blkg(iolat)->use_delay);
> > +	DEFINE_WAIT(wait);
> > +	bool first_block = true;
> > +
> > +	if (use_delay)
> > +		blkcg_schedule_throttle(rqos->q, use_memdelay);
> > +
> > +	/*
> > +	 * To avoid priority inversions we want to just take a slot if we are
> > +	 * issuing as root.  If we're being killed off there's no point in
> > +	 * delaying things, we may have been killed by OOM so throttling may
> > +	 * make recovery take even longer, so just let the IO's through so the
> > +	 * task can go away.
> > +	 */
> > +	if (issue_as_root || fatal_signal_pending(current)) {
> > +		atomic_inc(&rqw->inflight);
> > +		return;
> > +	}
> > +
> > +	if (iolatency_may_queue(iolat, &wait, first_block))
> > +		return;
> > +
> > +	do {
> > +		prepare_to_wait_exclusive(&rqw->wait, &wait,
> > +					  TASK_UNINTERRUPTIBLE);
> > +
> > +		iolatency_may_queue(iolat, &wait, first_block);
> > +		first_block = false;
> > +
> > +		if (lock) {
> > +			spin_unlock_irq(lock);
> > +			io_schedule();
> > +			spin_lock_irq(lock);
> > +		} else {
> > +			io_schedule();
> > +		}
> > +	} while (1);
> 
> So how does this wait loop ever exit?
>

Sigh, I cleaned this up from what we're using in production and did it poorly,
I'll fix it up.  Thanks,

Josef