[PATCH] Documentation/scsi: Documentation about scsi_cmnd lifecycle

[PATCH] Documentation/scsi: Documentation about scsi_cmnd lifecycle

[Rajat Jain]

Add documentation to describe the various scenarios that the scsi_cmnd
may go through in its life time in the mid level driver - aborts,
failures, retries, error handling etc. The documentation has lots of
details including examples.
---
Hello,

I have been reading the SCSI for the past few weeks and decided to turn all
my notes from past few weeks into a documentation, in the hope that this may
be helpful to others. I'd appreciate if you could please review & provide your
valuable feedback.

Thanks,

Rajat

 Documentation/scsi/life_of_a_scsi_cmnd.txt | 667 +++++++++++++++++++++++++++++
 1 file changed, 667 insertions(+)
 create mode 100644 Documentation/scsi/life_of_a_scsi_cmnd.txt

diff --git a/Documentation/scsi/life_of_a_scsi_cmnd.txt b/Documentation/scsi/life_of_a_scsi_cmnd.txt
new file mode 100644
index 0000000..b09b2a2
--- /dev/null
+++ b/Documentation/scsi/life_of_a_scsi_cmnd.txt
@@ -0,0 +1,667 @@
+                 ==================================
+                 Life of a SCSI Command (scsi_cmnd)
+                 ==================================
+
+              Rajat Jain <rajatja@google.com> on 12-May-2015
+
+(This document roughly matches the Linux kernel 4.0)
+
+This documents describes the various phases of a SCSI command (struct scsi_cmnd)
+lifecycle, as it flows though different parts of the SCSI mid level driver. It
+describes under what conditions and how a scsi_cmnd may be aborted, or retried,
+or scheduled for error handling, and how is it recovered, and in general how a
+block request is handled by the SCSI mid level driver. It goes into detail about
+what functions get called and the purpose for each one of them etc.
+
+To help explain with an example, it takes example of a scsi_cmnd that goes
+through it all - timeout, abort, error handling, retry (also results in
+CHECK_CONDITION and gets sense info). The last section traces the path taken by
+this example scsi_cmnd in its lifetime.
+
+TABLE OF CONTENTS
+
+[1] Lifecycle of a scsi_cmnd
+[2] How does a scsi_cmnd get queued to the LLD for processing?
+[3] How does a scsi_cmnd complete?
+    [3.1] Command completing via scsi_softirq_done()
+    [3.2] Command completing via scsi_times_out()$
+[4] SCSI Error Handling
+    [4.1] How did we Get here?
+    [4.2] When does Error Handling actually run?
+    [4.3] SCSI Error Handler thread
+[5] SCSI Commands can be "hijacked"
+[6] SCSI Command Aborts
+    [6.1] When would mid level try to abort a command?
+    [6.2] How SCSI command abort works?
+    [6.3] Aborts can fail too
+[7] SCSI command Retries
+    [7.1] When would mid level retry a command?
+    [7.2] Eligibility criteria for Retry
+[8] Example: Following a scsi_cmnd (that results in CHECK_CONDITION)
+    [8.1] High level view of path taken by example scsi_cmnd
+    [8.2] Actual Path taken
+[9] References
+
+1. Lifecycle of a scsi_cmnd
+   ========================
+   SCSI Mid level interfaces with the block layer just like any other block
+   driver. For each block device that SCSI ML adds to the system, it indicates
+   a bunch of functions to serve the corresponding request queue.
+
+   The following functions are relevant to the scsi_cmnd in its lifetime. Note
+   that depending on the situations, it may not go thourgh some of these
+   stages, or may have to go through some stages multiple times.
+
+   scsi_prep_fn()
+     is called by the blocklayer to prepare the request. This
+     function actually allocates a new scsi_cmnd for the request (from
+     scsi_host->cmd_pool) and sets it up. This is where a scsi_smnd is "born".
+     Note, a new scsi_cmnd is allocated only if the blk req did not already have
+     one associated with it (req->special != NULL). A req may already have a
+     scsi_cmnd if the req was tried by SCSI earlier, and it resulted in a
+     decision to retry later (and hence req was put back on the queue).
+
+   scsi_request_fn()
+     is the actual function to serve the request queue. It basically checks
+     whether the host is ready for new commands, and if so, it submits it to the
+     LLD:
+     scsi_request_fn()
+       ->scsi_dispatch_cmd()
+           ->hostt->queue_command()
+     In case a scsi_cmnd could not be queued to LLD for some reason, the req
+     is put back on the original request queue (for retry later).
+
+   scsi_softirq_done()
+     is the handler that gets called once the LLD indicates command completed.
+     scsi_done()
+       ->blk_complete_request()
+           ->causes softirq
+               ->blk_done_softirq()
+                   ->scsi_softirq_done()
+     The most important goal of this function is to determine the course of
+     further action for this req (based on the scsi_cmnd->result and sense data
+     if present), and take that course. The options could be to finish off the
+     request to block layer, requeue it to block layer, or schedule it for error
+     handling (if that is deemed necessary). This is discussed in much detail
+     later.
+
+   scsi_times_out()
+     is the function that gets called if the LLD does not respond with the
+     result of a scsi_cmnd for a long time, and a time out happens. It tries
+     to see if the situation can be fixed by LLD timeout handlers (if available)
+     or aborting the commands. If not, it schedules the commands for EH
+     (discussed at length later).
+
+   scsi_unprep_fn()
+     is the function that gets called to unprepare the request. It is supposed
+     to undo whatever scsi_prep_fn() does.
+
+2. How does a scsi_cmnd get queued to the LLD for processing?
+   ==========================================================
+   The submission part is very simple. Once the scsi_request_fn() gets called
+   for a block request and it picks up a new block request via
+   blk_peek_request(), the scsi_cmnd has already been setup and is ready to be
+   sent to the LLD:
+    scsi_request_fn()
+        ->scsi_dispatch_cmd()
+              ->hostt->queue_command()
+
+3. How does a scsi_cmnd complete?
+   ==============================
+   Once a scsi_cmnd is submitted to the LLD, there are only 2 ways it can get
+   completed:
+
+   a. Either the LLD responds in time.
+      (i.e. resulting in scsi_softirq_done() for the command)
+
+   b. Or, the LLD does not respond in time and a timeout out occurred
+      (i.e. resulting in scsi_times_out() for the command)
+
+   We discuss both these cases below.
+
+   Note 1: There may be scsi_cmnd(s) that are re-tried. But completion of a
+      re-tried scsi_cmnd is not any different than the completion of a new
+      scsi_cmnd. Thus irrespective of retries, the scsi_cmnds will always end up
+      in using one of the above 2 scenarios.
+
+   Note 2: A scsi_cmnd may be "highjacked" during error handling in
+      scsi_send_eh_cmnd(), to send one of the EH commands (TUR / STU /
+      REQUEST_SENSE). However, the completion of these EH commands does not land up
+      in the above two scenarios. This is the only exception. Once the scsi_cmnd is
+      "un-hijacked", the result of this original scsi_cmnd will still go through
+      the same 2 scenarios.
+
+3.1 Command completing via scsi_softirq_done()
+    ==========================================
+    This is the case when the LLD responded in time i.e. completed the command.
+    Note that here "completed" does not mean that the command was successfully
+    completed. In fact it could have been the case, that the SCSI host hardware
+    may have failed without even accepting the command. However, the fact that
+    scsi_softir_done() was called, indicates that there is a "result" available
+    in a timely fashion. And we'll have to examine this result in order to
+    decide the next course of action.
+
+    scsi_softirq_done()
+    |
+    +---> scsi_decide_disposition()
+    |      Takes a look at the scsi_cmnd->result and sense data to determine
+    |      what is the best course of action to take. While reading this
+    |      function code, one should not confuse SUCCESS as meaning the command
+    |      was successful, or FAILED to mean the command failed etc. The return
+    |      value of this function merely indicates the course of action to take
+    |
+    +---> case SUCCESS:
+    |      (Finish off the command to block layer. For e.g, the device may be
+    |      offline, and hence complete the command - the block layer may retry
+    |      on its own later, but that doesn't concern the SCSI ML)
+    |      |
+    |      +---> scsi_finish_command()
+    |            |
+    |            +---> scsi_io_completion() (*see note below)
+    |                  |
+    |                  +---> blk_finish_request()
+    |
+    +---> case RETRY/ADD_TO_MLQUEUE:
+    |     (Requeue the command to request queue. For e.g. the device HW was
+    |      busy, and thus SCSI ML knows that retrying may help)
+    |      |
+    |      +---> scsi_queue_insert()
+    |            |
+    |            +---> blk_requeue_request()
+    |
+    +---> case FAILED/default:
+          (Schedule the scsi_cmnd for EH. For e.g. there was a bus error that
+          might need bus reset. Or we got CHECK_CONDITION and we need to issue
+          REQ_SENSE to get more info about the failure. etc)
+          |
+          +---> scsi_eh_scmd_add()
+                Add scsi_cmnd to the host EH queue
+                    scsi_eh_wakeup()
+
+    Note 3:
+       The scsi_io_completion() has a secondary logic similar to
+       scsi_decide_disposition() in that it also looks at result and sense data
+       and figures out what to do with request. It makes similar choices on the
+       course of action to take. There is a special case in this function that
+       involves "unprepping" a scsi_cmnd before requeuing it, and we'll discuss
+       it in sections below.
+
+3.2 Command completing via scsi_times_out()
+    =======================================
+    This happens when the LLD does not repond in time, the block layer times
+    out, and as a result calls the timeout function for the request queue for
+    the SCSI device in question.
+
+    scsi_times_out()
+    |
+    +---> scsi_transport_template->eh_timed_out() - Successful? If not...
+    |     (Gives transportt a chance to deal with it)
+    |
+    +---> scsi_host_template->eh_timed_out() - Successful? If not...
+    |     (Gives hostt a chance to deal with it)
+    |
+    +---> scsi_abort_command() - Successful? If not...
+    |     (Schedule an ABORT of the scsi_cmnd. The abort handler will also
+    |      requeue it if needed)
+    |
+    +---> scsi_eh_scmd_add()
+          (Schedule the scsi_cmnd for EH. This'll definitely work. Because if it
+           doesn't work, the EH handler will mark the device as offline, which
+           counts as a good fix :-))
+
+4. SCSI Error Handling
+   ===================
+
+   SCSI Error handling should be thought of the action the mid level decides to
+   take when it knows that merely retrying a request may not help, and it needs
+   to do something else (possibly disruptive) in order to fix the issue. For
+   e.g. a stalled host may require a host reset, and only after that a retry of
+   the request may complete.
+
+   Note 4:
+      (Random thoughts): Contrast the "Error Handling" with "Retries". A Retry
+      is a normal thing to do, when the mid level believes that it has seen an
+      error which is transient in nature, and will go away on its own without
+      explicitly doing anything. Thus a retry of a request again makes sense in
+      this case. (On the other hand a cmnd is scheduled for EH, when it knows
+      that it needs to do "something" before a retrying a cmnd can give good
+      results).
+
+   Note 5:
+      The SCSI mid level maintains a (per-host) list of all the scsi_cmnd(s)
+      that have been scheduled for EH at that host using scsi_host->eh_cmd_q.
+      This is the list that gets processed by the EH thread, when it runs.
+
+4.1 How did we Get here?
+    --------------------
+
+    A scsi_cmnd could be marked for EH in the following cases:
+
+    * The command "error completed" i.e. scsi_decide_disposition() returned
+      FAILED or something that indicates a failure that requires some sort of
+      error recovery. E.g. device hardware failed, or we have a CHECK_CONDITION.
+      scsi_softirq_done()
+        ->scsi_decide_disposition = FAILED
+            ->scsi_eh_scmd_add()
+
+    * A scsi_cmnd timed out, and attempt to abort it fails.
+      scsi_times_out()
+        ->scsi_abort_command() != SUCCESS
+            ->scsi_eh_scmd_add()
+
+4.2 When does Error Handling actually run?
+    -------------------------------------
+
+    A SCSI error handler thread is scheduled whenever there is a scsi_smnd that
+    is marked for EH (inserted in the Scsi_Host->eh_cmd_q). Once a scsi_cmnd is
+    marked for EH, the ML does not accept any more scsi_cmnds for that
+    particular Scsi_Host. However, the EH thread does not actually run until all
+    the pending IOs to the LLD for that particular Scsi_Host have either
+    completed or failed. In other words, the only commands pending at the LLD
+    for that host are the ones that need EH (host_busy == host_failed).
+
+    The idea is to quiesce the bus, so that EH thread can recover the devices,
+    as it may require to reset different components in order to do its job.
+
+4.3 SCSI Error Handler thread
+    -------------------------
+
+    scsi_error_handler()
+    |
+    +---> transportt->eh_strategy_handler() if exists, else...
+    |     (Use transportt's own error recovery handler, if available)
+    |
+    +---> scsi_unjam_host()
+    |     (The SCSI ML error handler described below. Also described in
+    |      Documentation/scsi/scsi_eh.txt. Basic goal is to do whatever
+    |      needs to recover from the current error condition. And requeue the
+    |      eligible commands after recovery)
+    |
+    +---> scsi_restart_operations()
+          (Restart the operations of the SCSI request queue)
+          |
+          +---> scsi_run_host_queues()
+                |
+                +---> scsi_run_queue()
+                      |
+                      +---> blk_run_queue()
+
+    scsi_unjam_host()
+    -----------------
+    The idea is to create 2 lists: work_q, done_q.
+    Initially, work_q = <All EH scsi cmds>, done_q = NULL
+    And then error handle all the requests in work_q by taking sequentially
+    higher severity action items that may recover the cmnd or device. Keep
+    moving the requests from work_q to done_q and in the end finish them all
+    in one go rather than individually finishing them up.
+
+    scsi_unjam_host()
+    |
+    +--> Create 2 lists: work_q, done_q
+    |    work_q = <All EH scsi cmds>, done_q = NULL
+    |
+    +--> scsi_eh_get_sense() - Are we done? if not...
+    |   (For the commands that have CHECK_CONDITION, get sense_info)
+    |    |
+    |    +--> scsi_request_sense()
+    |    |   (Use scsi_send_eh_cmnd() to send a "hijacked" REQ_SENSE cmnd)
+    |    |
+    |    +--> scsi_decide_disposition()
+    |    |
+    |    +--> Arrange to finish the scsi_cmnd if SUCCESS (by setting
+    |         retries=allowed)
+    |
+    +--> scsi_eh_abort_cmds() - Are we done? If not...
+    |   (Abort the commands that had timed out)
+    |    |
+    |    +--> scsi_try_to_abort_cmd()
+    |    |    (Results in call to hostt->eh_abort_handler() which is responsible
+    |    |     making the LLD and the HW forget about the scsi_cmnd)
+    |    |
+    |    +--> scsi_eh_test_devices()
+    |         (Test if the device is responding now by sending appropriate EH
+    |          commands (STU / TEST_UNIT_READY). Again, sending these EH
+    |          commands involves highjacking the original scsi_cmnd, and later
+    |          restoring the context)
+    |
+    +--> scsi_eh_ready_devs() - Are we done? if not...
+    |   (Take increasing order of higher severity actions in order to recover)
+    |    |
+    |    +--> scsi_eh_bus_device_reset()
+    |    |   (Reset the scsi_device. Results in call to
+    |    |    hostt->eh_device_reset_handler())
+    |    |
+    |    +--> scsi_eh_target_reset()
+    |    |   (Reset the scsi_target. Results in call to
+    |    |    hostt->eh_target_reset_handler())
+    |    |
+    |    +--> scsi_eh_bus_reset()
+    |    |   (Reset the scsi_device. Results in call to
+    |    |    hostt->eh_bus_reset_handler())
+    |    |
+    |    +--> scsi_eh_host_reset()
+    |    |   (Reset the Scsi_Host. Results in call to
+    |    |    hostt->eh_host_reset_handler())
+    |    |
+    |    +--> If nothing has worked - scsi_eh_offline_sdevs()
+    |         (The device is not recoverable, put it offline)
+    |
+    +--> scsi_eh_flush_done_q()
+        (For all the EH commands on the done_q, either requeue them (via
+         scsi_queue_insert()) if eligible, or finish them up to block layer
+         (via scsi_finish_command())
+
+   Note 6:
+      At each recovery stage we test if we are done (using
+      scsi_eh_test_devices()), and take the next severity action only if needed.
+
+   Note 7:
+      The error handler takes care that for multiple scsi_cmnds that can be
+      recovered by resetting the same component (e.g. same scsi_device), the
+      device is reset only once.
+
+5. SCSI Commands can be "hijacked"
+   ===============================
+
+   As seen above, the EH thread may need to send some EH commands in order to
+   check the health and responsiveness of the SCSI device:
+   * TUR - Test Unit Ready
+   * STU - Start / Stop Unit
+   * REQUEST_SENSE - To get the Sense data in response to CHECK_CONDITION
+
+   However instead of allocating and setting up a new scsi_cmnd for such
+   temporary purposes, the EH thread hijacks- the current scsi_cmnd that it is
+   trying to recover, in order to send the EH commands. This whole process is
+   done in scsi_send_eh_cmnd().
+
+   The scsi_send_eh_cmnd saves a context of the current command before hijacking
+   it, replaces the scsi_done ptr with its own before dipatching it to the LLD,
+   and restores the context later once it is done. The EH commands sent in this
+   manner are subject to the same problems of timeouts / abort failures /
+   completions - but they do not take the route taken by normal commands (i.e.
+   don't take the scsi_softirq_done() or scsi_times_out() route). Every
+   thing is handled within scsi_send_eh_cmnd(). This is discussed in following
+   sections.
+
+6. SCSI Command Aborts
+   ===================
+
+   It refers to the scenario where the SCSI mid level wants to have the LLD
+   driver and the hardware below it forget everything about a scsi_cmnd that
+   was given to the LLD earlier. The most common reason is that the LLD failed
+   to respond in time.
+
+6.1 When would mid level try to abort a command?
+    --------------------------------------------
+    The SCSI ML may try to abort a scsi_cmnd in the following conditions:
+
+    1. SCSI mid layer times out on a command, and tried to abort it.
+       scsi_times_out()
+         -> scsi_abort_command()
+        What happens if this abort fails? Schedule the command for EH.
+
+    2. The EH thread tried to abort all the pending commands while trying to
+       unjam a host.
+       scsi_unjam_host()
+         -> scsi_eh_abort_cmds()
+
+       What happens if this abort fails? We move to higher severity recovery
+       steps (start resetting HW components etc) because that is likely to cause
+       both LLD and the HW forget aout those commands.
+
+    3. This is a nasty one. During error recovery, the EH thread may "hijack"
+       a scsi_cmnd to send a EH command (TUR/STU/REQ_SENSE) to LLD using
+       scsi_send_eh_cmnd(). If such a "hijacked" EH command times out, the SCSI
+       EH thread will try to abort it.
+       scsi_send_eh_cmnd()
+         -> scsi_abort_eh_cmnd()
+              -> scsi_try_to_abort_cmd()
+
+       What happens if this abort fails? Similar to the previous case, the
+       scsi_abort_eh_cmnd() will try to take higher severity actions (reset bus
+       etc) but will not send EH commands such as TUR etc again in order to
+       verify if the devices started to respond.
+
+6.2 How SCSI command abort works?
+    -----------------------------
+    Unlike EH command like TUR, the ABORT is not a SCSI command that mid layer
+    driver sends to LLD. The LLD provides an eh_abort_handler() function
+    pointer that is used to abort the command. It is up to the LLD to do
+    whatever is needed to abort the command. It may require to send some
+    proprietary command to the HW, or fiddle some bits, or do whatever magic
+    is necessary.
+
+6.3 Aborts can fail too
+    --------------------
+
+    As with other things, abort attempts can also fail. The SCSI mid layer does
+    the right thing in such situations as depicted in the section above.
+
+    Note 8:
+       Once a block layer hands off a command to the SCSI subsystem, there is no
+       way currently for the block layer to cancel / abort a request. This needs
+       some work.
+
+7. SCSI command Retries
+   ====================
+
+   The SCSI mid level maintains no queues for the SCSI commands it is processing
+   (other than the EH command queue). Thus whenever the SCSI ML thinks it needs
+   to retry a command, it requeues the request back to the corresponding request
+   queue, so that the retries will be made "naturally" when the request function
+   picks up the next request for processing.
+
+   When requing such requests back to the request queue, they are put at the
+   head so that they go before the other (existing) requests in that request
+   queue.
+
+7.1 When would mid level retry a command?
+    -------------------------------------
+
+    Following are the conditions that will cause a SCSI command to be retried
+    (by putting the blk request back at the request queue):
+
+    1. Mid layer times out on a scsi_cmnd, aborts it successfully, and requeues
+       it.
+       scsi_times_out()
+         -> scsi_abort_command()
+              -> schedules scmd_eh_abort_handler()
+                   -> scsi_queue_insert()
+                        -> blk_requeue_request()
+
+    2. EH thread, after recovering a host, requeues back all the scsi_cmnds that
+       are eligible for a retry:
+       scsi_error_handler()
+         -> scsi_unjam_host()
+              -> scsi_eh_flush_done_q()
+                   -> scsi_queue_insert()
+                        -> blk_requeue_request()
+
+    3. LLD completes the scsi_cmnd, and scsi_decide_disposition() looks at the
+       scsi_cmnd->result and thinks it needs to be retried (For e.g. because the
+       bus was busy).
+       scsi_softirq_done()
+         -> scsi_decide_disposition() returns NEEDS_RETRY
+              -> scsi_queue_insert()
+                   -> blk_requeue_request()
+
+    4. In the scsi_request_fn(), the SCSI ML finds out that the host is busy and
+       the scsi_cmnd could not be sent to the LLD, hence it requeues the req
+       back on the queue.
+       scsi_request_fn()
+         -> case note_ready:
+              -> blk_requeue_request()
+
+    5. scsi_finish_command() that is called from a variety of places to finish
+       off a request to the block level. However, it calls scsi_io_completion()
+       that may look at the request and decide to retry it (if it qualifies).
+       scsi_finish_command()
+         -> scsi_io_completion()
+              -> __scsi_queue_insert()
+                   -> blk_requeue_request()
+
+    Note 9:
+       The case 5 above has a very special case. There may be some cases where
+       the scsi_io_completion() decides that a blk request has to be retried,
+       however the scsi_cmnd for this req should be relased and instead a new
+       scsi_cmnd should be allocated and used for this request at the next
+       retry. This can be the case for e.g. if it sees an ILLEGAL REQUEST as a
+       response to a READ10 command, and thinks that it may be because the
+       device supports only READ6. Thus it may make sense to switch to READ6
+       (hence a new scsi_cmnd) at the time of next retry.
+
+7.2 Eligibility criteria for Retry
+    ------------------------------
+
+    Note that SCSI mid level always checks for retry eligibility before it goes
+    ahead and requeues the command for retries. The eligibility criteria for a
+    scsi_cmnd includes (some of these may not apply in all situations described
+    above):
+
+    * retries < allowed (Num of retries should be less than allowed retries)
+    * no more than host->eh_deadline jiffies spent in EH.
+    * scsi_noretry_cmd() should return 0 for the command.
+    * scsi_device must be online
+    * req->timeout must not have expired
+    * etc.
+
+8. Example: Following a scsi_cmnd
+   ==============================
+
+8.1 High level view of path taken by example scsi_cmnd
+    --------------------------------------------------
+    We take the example of a block request that for example wants to read a
+    block off a scsi disk, how ever the LBA address is out of range for the
+    current device (hypothetically). The ML submits it to LLD, but the HW takes
+    the command and chokes on it (again hypothetically to trace through the
+    abort sequence). So the timeout happens and the ML aborts the
+    command, and requeues it. In the next run, the LLD completes the command
+    with CHECK_CONDITION.  We assume that the SCSI host does not automatically
+    get the sense info. The ML schedules the cmnd for EH. The EH thread sends
+    the REQUEST_SENSE to get sense info ILLEGAL_REQUEST, and based on it
+    completes the request to the block layer.
+
+8.2 Actual Path taken
+    -----------------
+
+    Dispatched:
+
+    scsi_request_fn()
+    |
+    +---> blk_peek_request()
+    |     |
+    |     +---> scsi_prep_fn()
+    |           (Allocate and setup scsi_cmnd)
+    |
+    +---> scsi_dispatch_cmd()
+          |
+          +---> hostt->queue_command()
+
+    Times out:
+
+    scsi_times_out()
+    |
+    +---> scsi_abort_command() - returns SUCCESS
+          |
+          +---> queue_delayed_work(abort_work)
+
+    Abort Handler:
+
+    scmd_eh_abort_handler()
+    |
+    +---> scsi_try_to_abort_cmd() - returns SUCCESS
+    |     |
+    |     +---> hostt->eh_abort_handler()
+    |
+    +---> scsi_queue_insert()
+          |
+          +---> __scsi_queue_insert()
+                |
+                +---> blk_requeue_request()
+                      (the req is requeued, with req->special pointing
+                       to scsi_cmnd)
+
+    Request picked up again:
+
+    scsi_request_fn()
+    |
+    +---> blk_peek_request()
+    |     (req->cmd_flags has REQ_DONTPREP set, so does not call
+    |      scsi_prep_fn() again)
+    |
+    +---> scsi_dispatch_cmd()
+          |
+          +---> hostt->queue_command()
+
+    Command is completed with a CHECK_CONDITION:
+
+    scsi_softirq_done()
+    |
+    +---> scsi_decide_disposition()
+    |     (Sees the CHECK_CONDITION)
+    |     |
+    |     +---> scsi_check_sense() - returns FAILED
+    |           |
+    |           +---> scsi_command_normalize_sense()
+    |                 (Fails to find a valid sense data)
+    |
+    +---> case FAILED:
+          |
+          +---> scsi_eh_scmd_add()
+                Add scsi_cmnd to the host EH queue
+                |
+                +---> scsi_eh_wakeup()
+
+    The SCSI Error handler thread runs to get the sense info, and completes the
+    request once it is done.
+
+    scsi_error_handler()
+    |
+    +---> scsi_unjam_host()
+          |
+          +---> scsi_eh_get_sense()
+          |     |
+          |     +---> scsi_request_sense()
+          |     |     |
+          |     |     +---> scsi_send_eh_cmnd()
+          |     |          (Highjacks the smnd to send EH command)
+          |     |           |
+          |     |           +--> scsi_eh_prep_cmnd()
+          |     |           |    (save context of the existing scsi_cmndi,
+          |     |           |    allocates a sense buffer, and sets up the
+          |     |           |    scsi_cmnd for REQUEST_SENSE)
+          |     |           |
+          |     |           +--> hostt->queuecommand(), and then wait...
+          |     |           |    (gets the sense data for the cmnd)
+          |     |           |
+          |     |           +--> scsi_eh_completed_normally() - returns SUCCESS
+          |     |           |
+          |     |           +--> scsi_eh_restore_cmnd()
+          |     |                (restores the context of original scsi_cmnd)
+          |     |
+          |     +---> scsi_decide_disposition() - returns SUCCESS
+          |     |      (This time can see the sense info)
+          |     |
+          |     +---> Set scmd->retries = scmd->allowed (to avoid retries)
+          |     |
+          |     +---> scsi_eh_finish_cmd()
+          |           (Puts the scsi_cmnd on the done_q)
+          |
+          +---> scsi_eh_flush_done_q()
+                (Sees that scsi_cmnd is not eligible for retries)
+                |
+                +---> scsi_finish_command()
+                      |
+                      +---> scsi_io_completion()
+                            |
+                            +---> scsi_end_request()
+                                  |
+                                  +---> scsi_put_command()
+                                       (Releases the scsi_cmnd)
+
+9. References
+   ==========
+   The following are excellent sources of references:
+   Documentation/scsi/scsi_eh.txt
+   http://events.linuxfoundation.org/sites/events/files/slides/SCSI-EH.pdf
+--
-- 
2.2.0.rc0.207.ga3a616c

Re: [PATCH] Documentation/scsi: Documentation about scsi_cmnd lifecycle

[Jonathan Corbet]

On Tue, 12 May 2015 14:33:46 -0700
Rajat Jain <rajatja@google.com> wrote:

> Add documentation to describe the various scenarios that the scsi_cmnd
> may go through in its life time in the mid level driver - aborts,
> failures, retries, error handling etc. The documentation has lots of
> details including examples.

So this seems like useful stuff.  I'd be happy to take it through the
docs tree, but it definitely needs an ack or two from the SCSI camp first.

Taking another pass through to clean up typographical errors and trivial
language issues would probably be a good thing to do as well.

Thanks,

jon

Re: [PATCH] Documentation/scsi: Documentation about scsi_cmnd lifecycle

[Rajat Jain]

Hello James / linux-scsi folks,

On Wed, May 13, 2015 at 12:36 PM, Jonathan Corbet <corbet@lwn.net> wrote:
> On Tue, 12 May 2015 14:33:46 -0700
> Rajat Jain <rajatja@google.com> wrote:
>
>> Add documentation to describe the various scenarios that the scsi_cmnd
>> may go through in its life time in the mid level driver - aborts,
>> failures, retries, error handling etc. The documentation has lots of
>> details including examples.
>
> So this seems like useful stuff.  I'd be happy to take it through the
> docs tree, but it definitely needs an ack or two from the SCSI camp first.

I was wondering if someone could please help in reviewing this SCSI
documentation?

Thanks,

Rajat

Re: [PATCH] Documentation/scsi: Documentation about scsi_cmnd lifecycle

[Rajat Jain]

Hello James,

On Tue, May 26, 2015 at 11:15 AM, Rajat Jain <rajatja@google.com> wrote:
> Hello James / linux-scsi folks,
>
> On Wed, May 13, 2015 at 12:36 PM, Jonathan Corbet <corbet@lwn.net> wrote:
>> On Tue, 12 May 2015 14:33:46 -0700
>> Rajat Jain <rajatja@google.com> wrote:
>>
>>> Add documentation to describe the various scenarios that the scsi_cmnd
>>> may go through in its life time in the mid level driver - aborts,
>>> failures, retries, error handling etc. The documentation has lots of
>>> details including examples.
>>
>> So this seems like useful stuff.  I'd be happy to take it through the
>> docs tree, but it definitely needs an ack or two from the SCSI camp first.
>
> I was wondering if someone could please help in reviewing this SCSI
> documentation?
>

Can you please help review this SCSI documentation?

Thanks,

Rajat