From: Paolo Valente <paolo.valente@linaro.org> To: Jens Axboe <axboe@fb.com> Cc: Jens Axboe <axboe@kernel.dk>, linux-block@vger.kernel.org, Linux-Kernal <linux-kernel@vger.kernel.org>, Omar Sandoval <osandov@fb.com>, Linus Walleij <linus.walleij@linaro.org>, Ulf Hansson <ulf.hansson@linaro.org>, Mark Brown <broonie@kernel.org> Subject: Re: [PATCHSET v4] blk-mq-scheduling framework Date: Thu, 26 Jan 2017 15:23:39 +0100 [thread overview] Message-ID: <CAC36074-9BB6-4A35-94EA-73ACCFD6C86E@linaro.org> (raw) In-Reply-To: <0e047692-5ff2-d404-a400-8484e060218b@fb.com> > Il giorno 25 gen 2017, alle ore 17:13, Jens Axboe <axboe@fb.com> ha = scritto: >=20 > On 01/25/2017 01:46 AM, Paolo Valente wrote: >>=20 >>> Il giorno 23 gen 2017, alle ore 18:42, Jens Axboe <axboe@fb.com> ha = scritto: >>>=20 >>> On 01/23/2017 10:04 AM, Paolo Valente wrote: >>>>=20 >>>>> Il giorno 18 gen 2017, alle ore 17:21, Jens Axboe <axboe@fb.com> = ha scritto: >>>>>=20 >>>>> On 01/18/2017 08:14 AM, Paolo Valente wrote: >>>>>> according to the function blk_mq_sched_put_request, the >>>>>> mq.completed_request hook seems to always be invoked (if set) for = a >>>>>> request for which the mq.put_rq_priv is invoked (if set). >>>>>=20 >>>>> Correct, any request that came out of blk_mq_sched_get_request() >>>>> will always have completed called on it, regardless of whether it >>>>> had IO started on it or not. >>>>>=20 >>>>=20 >>>> It seems that some request, after being dispatched, happens to have = no >>>> mq.put_rq_priv invoked on it now or then. Is it expected? If it = is, >>>> could you point me to the path through which the end of the life of >>>> such a request is handled? >>>=20 >>> I'm guessing that's a flush request. I added RQF_QUEUED to check for >>> that, if RQF_QUEUED is set, you know it has come from your = get_request >>> handler. >>>=20 >>=20 >> Exactly, the completion-without-put_rq_priv pattern seems to occur >> only for requests coming from the flusher, precisely because they = have >> the flag RQF_ELVPRIV unset. Just to understand: why is this flag >> unset for these requests, if they do have private elevator (bfq) >> data attached? What am I misunderstanding? >>=20 >> Just to be certain: this should be the only case where the >> completed_request hook is invoked while the put_rq_priv is not, = right? >=20 > They must NOT have scheduler data attached. In your get_request > function, you must bypass if blk_mq_sched_bypass_insert() returns = true. Yes, sorry. I'm already using blk_mq_sched_bypass_insert() to bypass insertion in the insert hook, as done in mq-deadline, and I have no get_request defined (see below). The source of my confusion was that I assumed that flush requests had not to leave any trace in the scheduler, since the scheduler does not decide anything for them. Accordingly, I thought they did not trigger any put or completion hook. In contrast, these requests get the flag QUEUED set, in case the get_request hook is set, and then trigger both a put_request and a completed_request. In this respect, in bfq-mq I'm not using any of these three hooks (they are all NULL). I hope I'm not doing something unexpected or incoherent. UPDATE: bfq-mq now survives for minutes. I'm debugging two occasional failures, which (un)fortunately become more and more occasional as I go on with debugging and instrumenting the code. Thanks, Paolo > See how mq-deadline does that. This is important, or you will get = hangs > with flushes as well, since the IO scheduler private data and the = flush > data is unionized in the request. >=20 > --=20 > Jens Axboe
WARNING: multiple messages have this Message-ID (diff)
From: Paolo Valente <paolo.valente@linaro.org> To: Jens Axboe <axboe@fb.com> Cc: Jens Axboe <axboe@kernel.dk>, linux-block@vger.kernel.org, Linux-Kernal <linux-kernel@vger.kernel.org>, Omar Sandoval <osandov@fb.com>, Linus Walleij <linus.walleij@linaro.org>, Ulf Hansson <ulf.hansson@linaro.org>, Mark Brown <broonie@kernel.org> Subject: Re: [PATCHSET v4] blk-mq-scheduling framework Date: Thu, 26 Jan 2017 15:23:39 +0100 [thread overview] Message-ID: <CAC36074-9BB6-4A35-94EA-73ACCFD6C86E@linaro.org> (raw) In-Reply-To: <0e047692-5ff2-d404-a400-8484e060218b@fb.com> > Il giorno 25 gen 2017, alle ore 17:13, Jens Axboe <axboe@fb.com> ha scritto: > > On 01/25/2017 01:46 AM, Paolo Valente wrote: >> >>> Il giorno 23 gen 2017, alle ore 18:42, Jens Axboe <axboe@fb.com> ha scritto: >>> >>> On 01/23/2017 10:04 AM, Paolo Valente wrote: >>>> >>>>> Il giorno 18 gen 2017, alle ore 17:21, Jens Axboe <axboe@fb.com> ha scritto: >>>>> >>>>> On 01/18/2017 08:14 AM, Paolo Valente wrote: >>>>>> according to the function blk_mq_sched_put_request, the >>>>>> mq.completed_request hook seems to always be invoked (if set) for a >>>>>> request for which the mq.put_rq_priv is invoked (if set). >>>>> >>>>> Correct, any request that came out of blk_mq_sched_get_request() >>>>> will always have completed called on it, regardless of whether it >>>>> had IO started on it or not. >>>>> >>>> >>>> It seems that some request, after being dispatched, happens to have no >>>> mq.put_rq_priv invoked on it now or then. Is it expected? If it is, >>>> could you point me to the path through which the end of the life of >>>> such a request is handled? >>> >>> I'm guessing that's a flush request. I added RQF_QUEUED to check for >>> that, if RQF_QUEUED is set, you know it has come from your get_request >>> handler. >>> >> >> Exactly, the completion-without-put_rq_priv pattern seems to occur >> only for requests coming from the flusher, precisely because they have >> the flag RQF_ELVPRIV unset. Just to understand: why is this flag >> unset for these requests, if they do have private elevator (bfq) >> data attached? What am I misunderstanding? >> >> Just to be certain: this should be the only case where the >> completed_request hook is invoked while the put_rq_priv is not, right? > > They must NOT have scheduler data attached. In your get_request > function, you must bypass if blk_mq_sched_bypass_insert() returns true. Yes, sorry. I'm already using blk_mq_sched_bypass_insert() to bypass insertion in the insert hook, as done in mq-deadline, and I have no get_request defined (see below). The source of my confusion was that I assumed that flush requests had not to leave any trace in the scheduler, since the scheduler does not decide anything for them. Accordingly, I thought they did not trigger any put or completion hook. In contrast, these requests get the flag QUEUED set, in case the get_request hook is set, and then trigger both a put_request and a completed_request. In this respect, in bfq-mq I'm not using any of these three hooks (they are all NULL). I hope I'm not doing something unexpected or incoherent. UPDATE: bfq-mq now survives for minutes. I'm debugging two occasional failures, which (un)fortunately become more and more occasional as I go on with debugging and instrumenting the code. Thanks, Paolo > See how mq-deadline does that. This is important, or you will get hangs > with flushes as well, since the IO scheduler private data and the flush > data is unionized in the request. > > -- > Jens Axboe
next prev parent reply other threads:[~2017-01-26 14:23 UTC|newest] Thread overview: 115+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-12-17 0:12 [PATCHSET v4] blk-mq-scheduling framework Jens Axboe 2016-12-17 0:12 ` [PATCH 1/8] block: move existing elevator ops to union Jens Axboe 2016-12-17 0:12 ` [PATCH 2/8] blk-mq: make mq_ops a const pointer Jens Axboe 2016-12-17 0:12 ` [PATCH 3/8] block: move rq_ioc() to blk.h Jens Axboe 2016-12-20 10:12 ` Paolo Valente 2016-12-20 10:12 ` Paolo Valente 2016-12-20 15:46 ` Jens Axboe 2016-12-20 22:14 ` Jens Axboe 2016-12-17 0:12 ` [PATCH 4/8] blk-mq: un-export blk_mq_free_hctx_request() Jens Axboe 2016-12-17 0:12 ` [PATCH 5/8] blk-mq: export some helpers we need to the scheduling framework Jens Axboe 2016-12-17 0:12 ` [PATCH 6/8] blk-mq-sched: add framework for MQ capable IO schedulers Jens Axboe 2016-12-20 11:55 ` Paolo Valente 2016-12-20 11:55 ` Paolo Valente 2016-12-20 15:45 ` Jens Axboe 2016-12-21 2:22 ` Jens Axboe 2016-12-22 15:20 ` Paolo Valente 2016-12-22 15:20 ` Paolo Valente 2016-12-22 9:59 ` Paolo Valente 2016-12-22 9:59 ` Paolo Valente 2016-12-22 11:13 ` Paolo Valente 2016-12-22 11:13 ` Paolo Valente 2017-01-17 2:47 ` Jens Axboe 2017-01-17 2:47 ` Jens Axboe 2017-01-17 10:13 ` Paolo Valente 2017-01-17 10:13 ` Paolo Valente 2017-01-17 12:38 ` Jens Axboe 2017-01-17 12:38 ` Jens Axboe 2016-12-23 10:12 ` Paolo Valente 2016-12-23 10:12 ` Paolo Valente 2017-01-17 2:47 ` Jens Axboe 2017-01-17 9:17 ` Paolo Valente 2017-01-17 9:17 ` Paolo Valente 2016-12-17 0:12 ` [PATCH 7/8] mq-deadline: add blk-mq adaptation of the deadline IO scheduler Jens Axboe 2016-12-20 9:34 ` Paolo Valente 2016-12-20 9:34 ` Paolo Valente 2016-12-20 15:46 ` Jens Axboe 2016-12-21 11:59 ` Bart Van Assche 2016-12-21 11:59 ` Bart Van Assche 2016-12-21 14:22 ` Jens Axboe 2016-12-22 16:07 ` Paolo Valente 2016-12-22 16:07 ` Paolo Valente 2017-01-17 2:47 ` Jens Axboe 2017-01-17 2:47 ` Jens Axboe 2016-12-22 16:49 ` Paolo Valente 2016-12-22 16:49 ` Paolo Valente 2017-01-17 2:47 ` Jens Axboe 2017-01-20 11:07 ` Paolo Valente 2017-01-20 11:07 ` Paolo Valente 2017-01-20 14:26 ` Jens Axboe 2017-01-20 13:14 ` Paolo Valente 2017-01-20 13:14 ` Paolo Valente 2017-01-20 13:18 ` Paolo Valente 2017-01-20 13:18 ` Paolo Valente 2017-01-20 14:28 ` Jens Axboe 2017-01-20 14:28 ` Jens Axboe 2017-02-01 11:11 ` Paolo Valente 2017-02-01 11:11 ` Paolo Valente 2017-02-02 5:19 ` Jens Axboe 2017-02-02 9:19 ` Paolo Valente 2017-02-02 9:19 ` Paolo Valente 2017-02-02 15:30 ` Jens Axboe 2017-02-02 15:30 ` Jens Axboe 2017-02-02 21:15 ` Paolo Valente 2017-02-02 21:15 ` Paolo Valente 2017-02-02 21:32 ` Jens Axboe 2017-02-02 21:32 ` Jens Axboe 2017-02-07 17:27 ` Paolo Valente 2017-02-07 17:27 ` Paolo Valente 2017-02-01 11:56 ` Paolo Valente 2017-02-01 11:56 ` Paolo Valente 2017-02-02 5:20 ` Jens Axboe 2017-02-16 10:46 ` Paolo Valente 2017-02-16 10:46 ` Paolo Valente 2017-02-16 15:35 ` Jens Axboe 2017-02-16 15:35 ` Jens Axboe 2016-12-17 0:12 ` [PATCH 8/8] blk-mq-sched: allow setting of default " Jens Axboe 2016-12-19 11:32 ` [PATCHSET v4] blk-mq-scheduling framework Paolo Valente 2016-12-19 11:32 ` Paolo Valente 2016-12-19 15:20 ` Jens Axboe 2016-12-19 15:20 ` Jens Axboe 2016-12-19 15:33 ` Jens Axboe 2016-12-19 18:21 ` Paolo Valente 2016-12-19 18:21 ` Paolo Valente 2016-12-19 21:05 ` Jens Axboe 2016-12-19 21:05 ` Jens Axboe 2016-12-22 15:28 ` Paolo Valente 2016-12-22 15:28 ` Paolo Valente 2017-01-17 2:47 ` Jens Axboe 2017-01-17 10:43 ` Paolo Valente 2017-01-17 10:44 ` Paolo Valente 2017-01-17 10:47 ` Paolo Valente 2017-01-17 10:49 ` Paolo Valente 2017-01-17 10:49 ` Paolo Valente 2017-01-18 16:14 ` Paolo Valente 2017-01-18 16:14 ` Paolo Valente 2017-01-18 16:21 ` Jens Axboe 2017-01-18 16:21 ` Jens Axboe 2017-01-23 17:04 ` Paolo Valente 2017-01-23 17:04 ` Paolo Valente 2017-01-23 17:42 ` Jens Axboe 2017-01-23 17:42 ` Jens Axboe 2017-01-25 8:46 ` Paolo Valente 2017-01-25 8:46 ` Paolo Valente 2017-01-25 16:13 ` Jens Axboe 2017-01-25 16:13 ` Jens Axboe 2017-01-26 14:23 ` Paolo Valente [this message] 2017-01-26 14:23 ` Paolo Valente 2016-12-22 16:23 ` Bart Van Assche 2016-12-22 16:52 ` Omar Sandoval 2016-12-22 16:52 ` Omar Sandoval 2016-12-22 16:57 ` Bart Van Assche 2016-12-22 16:57 ` Bart Van Assche 2016-12-22 17:12 ` Omar Sandoval 2016-12-22 17:39 ` Bart Van Assche 2016-12-22 17:39 ` Bart Van Assche
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CAC36074-9BB6-4A35-94EA-73ACCFD6C86E@linaro.org \ --to=paolo.valente@linaro.org \ --cc=axboe@fb.com \ --cc=axboe@kernel.dk \ --cc=broonie@kernel.org \ --cc=linus.walleij@linaro.org \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=osandov@fb.com \ --cc=ulf.hansson@linaro.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.