* [PATCH 1/17]: Integration of SCST into the Linux kernel tree
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
@ 2010-09-14 14:37 ` Vladislav Bolkhovitin
2010-09-14 14:38 ` [PATCH 2/17]: SCST core's Makefile and Kconfig Vladislav Bolkhovitin
` (16 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:37 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains changes in drivers/Kconfig and drivers/Makefile to integrate
SCST into the Linux kernel tree.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Kconfig | 2 ++
Makefile | 1 +
2 files changed, 3 insertions(+)
Signed-off-by:
diff -upkr -X linux-2.6.35/Documentation/dontdiff linux-2.6.35/drivers/Kconfig linux-2.6.35/drivers/Kconfig
--- orig/linux-2.6.35/drivers/Kconfig 01:51:29.000000000 +0400
+++ linux-2.6.35/drivers/Kconfig 14:14:46.000000000 +0400
@@ -22,6 +22,8 @@ source "drivers/ide/Kconfig"
source "drivers/scsi/Kconfig"
+source "drivers/scst/Kconfig"
+
source "drivers/ata/Kconfig"
source "drivers/md/Kconfig"
diff -upkr -X linux-2.6.35/Documentation/dontdiff linux-2.6.35/drivers/Makefile linux-2.6.35/drivers/Makefile
--- orig/linux-2.6.35/drivers/Makefile 01:51:29.000000000 +0400
+++ linux-2.6.35/drivers/Makefile 14:15:29.000000000 +0400
@@ -43,6 +43,7 @@ obj-$(CONFIG_ATM) += atm/
obj-y += macintosh/
obj-$(CONFIG_IDE) += ide/
obj-$(CONFIG_SCSI) += scsi/
+obj-$(CONFIG_SCST) += scst/
obj-$(CONFIG_ATA) += ata/
obj-y += net/
obj-$(CONFIG_ATM) += atm/
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 2/17]: SCST core's Makefile and Kconfig
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
2010-09-14 14:37 ` [PATCH 1/17]: Integration of SCST into the Linux kernel tree Vladislav Bolkhovitin
@ 2010-09-14 14:38 ` Vladislav Bolkhovitin
2010-09-14 14:39 ` [PATCH 3/17]: SCST public headers Vladislav Bolkhovitin
` (15 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:38 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST core's Makefile and Kconfig.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Kconfig | 243 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Makefile | 11 ++
2 files changed, 254 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/Kconfig linux-2.6.35/drivers/scst/Kconfig
--- orig/linux-2.6.35/drivers/scst/Kconfig
+++ linux-2.6.35/drivers/scst/Kconfig
@@ -0,0 +1,243 @@
+menu "SCSI target (SCST) support"
+
+config SCST
+ tristate "SCSI target (SCST) support"
+ depends on SCSI
+ help
+ SCSI target (SCST) is designed to provide unified, consistent
+ interface between SCSI target drivers and Linux kernel and
+ simplify target drivers development as much as possible. Visit
+ http://scst.sourceforge.net for more info about it.
+
+config SCST_DISK
+ tristate "SCSI target disk support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for disk device.
+
+config SCST_TAPE
+ tristate "SCSI target tape support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for tape device.
+
+config SCST_CDROM
+ tristate "SCSI target CDROM support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for CDROM device.
+
+config SCST_MODISK
+ tristate "SCSI target MO disk support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for MO disk device.
+
+config SCST_CHANGER
+ tristate "SCSI target changer support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for changer device.
+
+config SCST_PROCESSOR
+ tristate "SCSI target processor support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for processor device.
+
+config SCST_RAID
+ tristate "SCSI target storage array controller (RAID) support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for raid storage array controller (RAID) device.
+
+config SCST_VDISK
+ tristate "SCSI target virtual disk and/or CDROM support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST device handler for virtual disk and/or CDROM device.
+
+config SCST_STRICT_SERIALIZING
+ bool "Strict serialization"
+ depends on SCST
+ help
+ Enable strict SCSI command serialization. When enabled, SCST sends
+ all SCSI commands to the underlying SCSI device synchronously, one
+ after one. This makes task management more reliable, at the cost of
+ a performance penalty. This is most useful for stateful SCSI devices
+ like tapes, where the result of the execution of a command
+ depends on the device settings configured by previous commands. Disk
+ and RAID devices are stateless in most cases. The current SCSI core
+ in Linux doesn't allow to abort all commands reliably if they have
+ been sent asynchronously to a stateful device.
+ Enable this option if you use stateful device(s) and need as much
+ error recovery reliability as possible.
+
+ If unsure, say "N".
+
+config SCST_STRICT_SECURITY
+ bool "Strict security"
+ depends on SCST
+ help
+ Makes SCST clear (zero-fill) allocated data buffers. Note: this has a
+ significant performance penalty.
+
+ If unsure, say "N".
+
+config SCST_TEST_IO_IN_SIRQ
+ bool "Allow test I/O from soft-IRQ context"
+ depends on SCST
+ help
+ Allows SCST to submit selected SCSI commands (TUR and
+ READ/WRITE) from soft-IRQ context (tasklets). Enabling it will
+ decrease amount of context switches and slightly improve
+ performance. The goal of this option is to be able to measure
+ overhead of the context switches. See more info about it in
+ README.scst.
+
+ WARNING! Improperly used, this option can lead you to a kernel crash!
+
+ If unsure, say "N".
+
+config SCST_ABORT_CONSIDER_FINISHED_TASKS_AS_NOT_EXISTING
+ bool "Send back UNKNOWN TASK when an already finished task is aborted"
+ depends on SCST
+ help
+ Controls which response is sent by SCST to the initiator in case
+ the initiator attempts to abort (ABORT TASK) an already finished
+ request. If this option is enabled, the response UNKNOWN TASK is
+ sent back to the initiator. However, some initiators, particularly
+ the VMware iSCSI initiator, interpret the UNKNOWN TASK response as
+ if the target got crazy and try to RESET it. Then sometimes the
+ initiator gets crazy itself.
+
+ If unsure, say "N".
+
+config SCST_USE_EXPECTED_VALUES
+ bool "Prefer initiator-supplied SCSI command attributes"
+ depends on SCST
+ help
+ When SCST receives a SCSI command from an initiator, such a SCSI
+ command has both data transfer length and direction attributes.
+ There are two possible sources for these attributes: either the
+ values computed by SCST from its internal command translation table
+ or the values supplied by the initiator. The former are used by
+ default because of security reasons. Invalid initiator-supplied
+ attributes can crash the target, especially in pass-through mode.
+ Only consider enabling this option when SCST logs the following
+ message: "Unknown opcode XX for YY. Should you update
+ scst_scsi_op_table?" and when the initiator complains. Please
+ report any unrecognized commands to scst-devel@lists.sourceforge.net.
+
+ If unsure, say "N".
+
+config SCST_EXTRACHECKS
+ bool "Extra consistency checks"
+ depends on SCST
+ help
+ Enable additional consistency checks in the SCSI middle level target
+ code. This may be helpful for SCST developers. Enable it if you have
+ any problems.
+
+ If unsure, say "N".
+
+config SCST_TRACING
+ bool "Tracing support"
+ depends on SCST
+ default y
+ help
+ Enable SCSI middle level tracing support. Tracing can be controlled
+ dynamically via sysfs interface. The traced information
+ is sent to the kernel log and may be very helpful when analyzing
+ the cause of a communication problem between initiator and target.
+
+ If unsure, say "Y".
+
+config SCST_DEBUG
+ bool "Debugging support"
+ depends on SCST
+ select DEBUG_BUGVERBOSE
+ help
+ Enables support for debugging SCST. This may be helpful for SCST
+ developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_OOM
+ bool "Out-of-memory debugging support"
+ depends on SCST
+ help
+ Let SCST's internal memory allocation function
+ (scst_alloc_sg_entries()) fail about once in every 10000 calls, at
+ least if the flag __GFP_NOFAIL has not been set. This allows SCST
+ developers to test the behavior of SCST in out-of-memory conditions.
+ This may be helpful for SCST developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_RETRY
+ bool "SCSI command retry debugging support"
+ depends on SCST
+ help
+ Let SCST's internal SCSI command transfer function
+ (scst_rdy_to_xfer()) fail about once in every 100 calls. This allows
+ SCST developers to test the behavior of SCST when SCSI queues fill
+ up. This may be helpful for SCST developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_SN
+ bool "SCSI sequence number debugging support"
+ depends on SCST
+ help
+ Allows to test SCSI command ordering via sequence numbers by
+ randomly changing the type of SCSI commands into
+ SCST_CMD_QUEUE_ORDERED, SCST_CMD_QUEUE_HEAD_OF_QUEUE or
+ SCST_CMD_QUEUE_SIMPLE for about one in 300 SCSI commands.
+ This may be helpful for SCST developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_TM
+ bool "Task management debugging support"
+ depends on SCST_DEBUG
+ help
+ Enables support for debugging of SCST's task management functions.
+ When enabled, some of the commands on LUN 0 in the default access
+ control group will be delayed for about 60 seconds. This will
+ cause the remote initiator send SCSI task management functions,
+ e.g. ABORT TASK and TARGET RESET.
+
+ If unsure, say "N".
+
+config SCST_TM_DBG_GO_OFFLINE
+ bool "Let devices become completely unresponsive"
+ depends on SCST_DEBUG_TM
+ help
+ Enable this option if you want that the device eventually becomes
+ completely unresponsive. When disabled, the device will receive
+ ABORT and RESET commands.
+
+config SCST_MEASURE_LATENCY
+ bool "Commands processing latency measurement facility"
+ depends on SCST
+ help
+ This option enables commands processing latency measurement
+ facility in SCST. It will provide in the sysfs interface
+ average commands processing latency statistics. You can clear
+ already measured results by writing 0 in the corresponding sysfs file.
+ Note, you need a non-preemtible kernel to have correct results.
+
+ If unsure, say "N".
+
+source "drivers/scst/scst_local/Kconfig"
+source "drivers/scst/srpt/Kconfig"
+
+endmenu
diff -uprN orig/linux-2.6.35/drivers/scst/Makefile linux-2.6.35/drivers/scst/Makefile
--- orig/linux-2.6.35/drivers/scst/Makefile
+++ linux-2.6.35/drivers/scst/Makefile
@@ -0,0 +1,11 @@
+ccflags-y += -Wno-unused-parameter
+
+scst-y += scst_main.o
+scst-y += scst_pres.o
+scst-y += scst_targ.o
+scst-y += scst_lib.o
+scst-y += scst_sysfs.o
+scst-y += scst_mem.o
+scst-y += scst_debug.o
+
+obj-$(CONFIG_SCST) += scst.o dev_handlers/ srpt/ scst_local/
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 3/17]: SCST public headers
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
2010-09-14 14:37 ` [PATCH 1/17]: Integration of SCST into the Linux kernel tree Vladislav Bolkhovitin
2010-09-14 14:38 ` [PATCH 2/17]: SCST core's Makefile and Kconfig Vladislav Bolkhovitin
@ 2010-09-14 14:39 ` Vladislav Bolkhovitin
2010-09-14 14:40 ` [PATCH 4/17]: SCST main management files and private headers Vladislav Bolkhovitin
` (14 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:39 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains declarations of all externally visible SCST constants,
types and functions prototypes.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst.h | 3562 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
scst_const.h | 370 ++++++
2 files changed, 3932 insertions(+)
diff -uprN orig/linux-2.6.35/include/scst/scst_const.h linux-2.6.35/include/scst/scst_const.h
--- orig/linux-2.6.35/include/scst/scst_const.h
+++ linux-2.6.35/include/scst/scst_const.h
@@ -0,0 +1,370 @@
+/*
+ * include/scst_const.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Contains common SCST constants.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_CONST_H
+#define __SCST_CONST_H
+
+#include <scsi/scsi.h>
+
+#define SCST_CONST_VERSION "$Revision: 1864 $"
+
+/*** Shared constants between user and kernel spaces ***/
+
+/* Max size of CDB */
+#define SCST_MAX_CDB_SIZE 16
+
+/* Max size of various names */
+#define SCST_MAX_NAME 50
+
+/* Max size of external names, like initiator name */
+#define SCST_MAX_EXTERNAL_NAME 256
+
+/*
+ * Size of sense sufficient to carry standard sense data.
+ * Warning! It's allocated on stack!
+ */
+#define SCST_STANDARD_SENSE_LEN 18
+
+/* Max size of sense */
+#define SCST_SENSE_BUFFERSIZE 96
+
+/*************************************************************
+ ** Allowed delivery statuses for cmd's delivery_status
+ *************************************************************/
+
+#define SCST_CMD_DELIVERY_SUCCESS 0
+#define SCST_CMD_DELIVERY_FAILED -1
+#define SCST_CMD_DELIVERY_ABORTED -2
+
+/*************************************************************
+ ** Values for task management functions
+ *************************************************************/
+#define SCST_ABORT_TASK 0
+#define SCST_ABORT_TASK_SET 1
+#define SCST_CLEAR_ACA 2
+#define SCST_CLEAR_TASK_SET 3
+#define SCST_LUN_RESET 4
+#define SCST_TARGET_RESET 5
+
+/** SCST extensions **/
+
+/*
+ * Notifies about I_T nexus loss event in the corresponding session.
+ * Aborts all tasks there, resets the reservation, if any, and sets
+ * up the I_T Nexus loss UA.
+ */
+#define SCST_NEXUS_LOSS_SESS 6
+
+/* Aborts all tasks in the corresponding session */
+#define SCST_ABORT_ALL_TASKS_SESS 7
+
+/*
+ * Notifies about I_T nexus loss event. Aborts all tasks in all sessions
+ * of the tgt, resets the reservations, if any, and sets up the I_T Nexus
+ * loss UA.
+ */
+#define SCST_NEXUS_LOSS 8
+
+/* Aborts all tasks in all sessions of the tgt */
+#define SCST_ABORT_ALL_TASKS 9
+
+/*
+ * Internal TM command issued by SCST in scst_unregister_session(). It is the
+ * same as SCST_NEXUS_LOSS_SESS, except:
+ * - it doesn't call task_mgmt_affected_cmds_done()
+ * - it doesn't call task_mgmt_fn_done()
+ * - it doesn't queue NEXUS LOSS UA.
+ *
+ * Target drivers must NEVER use it!!
+ */
+#define SCST_UNREG_SESS_TM 10
+
+/*
+ * Internal TM command issued by SCST in scst_pr_abort_reg(). It aborts all
+ * tasks from mcmd->origin_pr_cmd->tgt_dev, except mcmd->origin_pr_cmd.
+ * Additionally:
+ * - it signals pr_aborting_cmpl completion when all affected
+ * commands marked as aborted.
+ * - it doesn't call task_mgmt_affected_cmds_done()
+ * - it doesn't call task_mgmt_fn_done()
+ * - it calls mcmd->origin_pr_cmd->scst_cmd_done() when all affected
+ * commands aborted.
+ *
+ * Target drivers must NEVER use it!!
+ */
+#define SCST_PR_ABORT_ALL 11
+
+/*************************************************************
+ ** Values for mgmt cmd's status field. Codes taken from iSCSI
+ *************************************************************/
+#define SCST_MGMT_STATUS_SUCCESS 0
+#define SCST_MGMT_STATUS_TASK_NOT_EXIST -1
+#define SCST_MGMT_STATUS_LUN_NOT_EXIST -2
+#define SCST_MGMT_STATUS_FN_NOT_SUPPORTED -5
+#define SCST_MGMT_STATUS_REJECTED -255
+#define SCST_MGMT_STATUS_FAILED -129
+
+/*************************************************************
+ ** SCSI task attribute queue types
+ *************************************************************/
+enum scst_cmd_queue_type {
+ SCST_CMD_QUEUE_UNTAGGED = 0,
+ SCST_CMD_QUEUE_SIMPLE,
+ SCST_CMD_QUEUE_ORDERED,
+ SCST_CMD_QUEUE_HEAD_OF_QUEUE,
+ SCST_CMD_QUEUE_ACA
+};
+
+/*************************************************************
+ ** CDB flags
+ *************************************************************/
+enum scst_cdb_flags {
+ SCST_TRANSFER_LEN_TYPE_FIXED = 0x0001,
+ SCST_SMALL_TIMEOUT = 0x0002,
+ SCST_LONG_TIMEOUT = 0x0004,
+ SCST_UNKNOWN_LENGTH = 0x0008,
+ SCST_INFO_VALID = 0x0010, /* must be single bit */
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED = 0x0020,
+ SCST_IMPLICIT_HQ = 0x0040,
+ SCST_SKIP_UA = 0x0080,
+ SCST_WRITE_MEDIUM = 0x0100,
+ SCST_LOCAL_CMD = 0x0200,
+ SCST_FULLY_LOCAL_CMD = 0x0400,
+ SCST_REG_RESERVE_ALLOWED = 0x0800,
+ SCST_WRITE_EXCL_ALLOWED = 0x1000,
+ SCST_EXCL_ACCESS_ALLOWED = 0x2000,
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED = 0x8000,
+#endif
+};
+
+/*************************************************************
+ ** Data direction aliases. Changing it don't forget to change
+ ** scst_to_tgt_dma_dir as well!!
+ *************************************************************/
+#define SCST_DATA_UNKNOWN 0
+#define SCST_DATA_WRITE 1
+#define SCST_DATA_READ 2
+#define SCST_DATA_BIDI (SCST_DATA_WRITE | SCST_DATA_READ)
+#define SCST_DATA_NONE 4
+
+/*************************************************************
+ ** Default suffix for targets with NULL names
+ *************************************************************/
+#define SCST_DEFAULT_TGT_NAME_SUFFIX "_target_"
+
+/*************************************************************
+ ** Sense manipulation and examination
+ *************************************************************/
+#define SCST_LOAD_SENSE(key_asc_ascq) key_asc_ascq
+
+#define SCST_SENSE_VALID(sense) ((sense != NULL) && \
+ ((((const uint8_t *)(sense))[0] & 0x70) == 0x70))
+
+#define SCST_NO_SENSE(sense) ((sense != NULL) && \
+ (((const uint8_t *)(sense))[2] == 0))
+
+/*************************************************************
+ ** Sense data for the appropriate errors. Can be used with
+ ** scst_set_cmd_error()
+ *************************************************************/
+#define scst_sense_no_sense NO_SENSE, 0x00, 0
+#define scst_sense_hardw_error HARDWARE_ERROR, 0x44, 0
+#define scst_sense_aborted_command ABORTED_COMMAND, 0x00, 0
+#define scst_sense_invalid_opcode ILLEGAL_REQUEST, 0x20, 0
+#define scst_sense_invalid_field_in_cdb ILLEGAL_REQUEST, 0x24, 0
+#define scst_sense_invalid_field_in_parm_list ILLEGAL_REQUEST, 0x26, 0
+#define scst_sense_parameter_value_invalid ILLEGAL_REQUEST, 0x26, 2
+#define scst_sense_invalid_release ILLEGAL_REQUEST, 0x26, 4
+#define scst_sense_parameter_list_length_invalid \
+ ILLEGAL_REQUEST, 0x1A, 0
+#define scst_sense_reset_UA UNIT_ATTENTION, 0x29, 0
+#define scst_sense_nexus_loss_UA UNIT_ATTENTION, 0x29, 0x7
+#define scst_sense_saving_params_unsup ILLEGAL_REQUEST, 0x39, 0
+#define scst_sense_lun_not_supported ILLEGAL_REQUEST, 0x25, 0
+#define scst_sense_data_protect DATA_PROTECT, 0x00, 0
+#define scst_sense_miscompare_error MISCOMPARE, 0x1D, 0
+#define scst_sense_block_out_range_error ILLEGAL_REQUEST, 0x21, 0
+#define scst_sense_medium_changed_UA UNIT_ATTENTION, 0x28, 0
+#define scst_sense_read_error MEDIUM_ERROR, 0x11, 0
+#define scst_sense_write_error MEDIUM_ERROR, 0x03, 0
+#define scst_sense_not_ready NOT_READY, 0x04, 0x10
+#define scst_sense_invalid_message ILLEGAL_REQUEST, 0x49, 0
+#define scst_sense_cleared_by_another_ini_UA UNIT_ATTENTION, 0x2F, 0
+#define scst_sense_capacity_data_changed UNIT_ATTENTION, 0x2A, 0x9
+#define scst_sense_reservation_preempted UNIT_ATTENTION, 0x2A, 0x03
+#define scst_sense_reservation_released UNIT_ATTENTION, 0x2A, 0x04
+#define scst_sense_registrations_preempted UNIT_ATTENTION, 0x2A, 0x05
+#define scst_sense_reported_luns_data_changed UNIT_ATTENTION, 0x3F, 0xE
+#define scst_sense_inquery_data_changed UNIT_ATTENTION, 0x3F, 0x3
+
+/*************************************************************
+ * SCSI opcodes not listed anywhere else
+ *************************************************************/
+#define REPORT_DEVICE_IDENTIFIER 0xA3
+#define INIT_ELEMENT_STATUS 0x07
+#define INIT_ELEMENT_STATUS_RANGE 0x37
+#define PREVENT_ALLOW_MEDIUM 0x1E
+#define READ_ATTRIBUTE 0x8C
+#define REQUEST_VOLUME_ADDRESS 0xB5
+#define WRITE_ATTRIBUTE 0x8D
+#define WRITE_VERIFY_16 0x8E
+#define VERIFY_6 0x13
+#ifndef VERIFY_12
+#define VERIFY_12 0xAF
+#endif
+
+/*************************************************************
+ ** SCSI Architecture Model (SAM) Status codes. Taken from SAM-3 draft
+ ** T10/1561-D Revision 4 Draft dated 7th November 2002.
+ *************************************************************/
+#define SAM_STAT_GOOD 0x00
+#define SAM_STAT_CHECK_CONDITION 0x02
+#define SAM_STAT_CONDITION_MET 0x04
+#define SAM_STAT_BUSY 0x08
+#define SAM_STAT_INTERMEDIATE 0x10
+#define SAM_STAT_INTERMEDIATE_CONDITION_MET 0x14
+#define SAM_STAT_RESERVATION_CONFLICT 0x18
+#define SAM_STAT_COMMAND_TERMINATED 0x22 /* obsolete in SAM-3 */
+#define SAM_STAT_TASK_SET_FULL 0x28
+#define SAM_STAT_ACA_ACTIVE 0x30
+#define SAM_STAT_TASK_ABORTED 0x40
+
+/*************************************************************
+ ** Control byte field in CDB
+ *************************************************************/
+#define CONTROL_BYTE_LINK_BIT 0x01
+#define CONTROL_BYTE_NACA_BIT 0x04
+
+/*************************************************************
+ ** Byte 1 in INQUIRY CDB
+ *************************************************************/
+#define SCST_INQ_EVPD 0x01
+
+/*************************************************************
+ ** Byte 3 in Standard INQUIRY data
+ *************************************************************/
+#define SCST_INQ_BYTE3 3
+
+#define SCST_INQ_NORMACA_BIT 0x20
+
+/*************************************************************
+ ** Byte 2 in RESERVE_10 CDB
+ *************************************************************/
+#define SCST_RES_3RDPTY 0x10
+#define SCST_RES_LONGID 0x02
+
+/*************************************************************
+ ** Values for the control mode page TST field
+ *************************************************************/
+#define SCST_CONTR_MODE_ONE_TASK_SET 0
+#define SCST_CONTR_MODE_SEP_TASK_SETS 1
+
+/*******************************************************************
+ ** Values for the control mode page QUEUE ALGORITHM MODIFIER field
+ *******************************************************************/
+#define SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER 0
+#define SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER 1
+
+/*************************************************************
+ ** Values for the control mode page D_SENSE field
+ *************************************************************/
+#define SCST_CONTR_MODE_FIXED_SENSE 0
+#define SCST_CONTR_MODE_DESCR_SENSE 1
+
+/*************************************************************
+ ** TransportID protocol identifiers
+ *************************************************************/
+
+#define SCSI_TRANSPORTID_PROTOCOLID_FCP2 0
+#define SCSI_TRANSPORTID_PROTOCOLID_SPI5 1
+#define SCSI_TRANSPORTID_PROTOCOLID_SRP 4
+#define SCSI_TRANSPORTID_PROTOCOLID_ISCSI 5
+#define SCSI_TRANSPORTID_PROTOCOLID_SAS 6
+
+/*************************************************************
+ ** Misc SCSI constants
+ *************************************************************/
+#define SCST_SENSE_ASC_UA_RESET 0x29
+#define BYTCHK 0x02
+#define POSITION_LEN_SHORT 20
+#define POSITION_LEN_LONG 32
+
+/*************************************************************
+ ** Various timeouts
+ *************************************************************/
+#define SCST_DEFAULT_TIMEOUT (60 * HZ)
+
+#define SCST_GENERIC_CHANGER_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_CHANGER_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_PROCESSOR_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_PROCESSOR_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_TAPE_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_TAPE_REG_TIMEOUT (900 * HZ)
+#define SCST_GENERIC_TAPE_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_MODISK_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_MODISK_REG_TIMEOUT (900 * HZ)
+#define SCST_GENERIC_MODISK_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_DISK_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_DISK_REG_TIMEOUT (60 * HZ)
+#define SCST_GENERIC_DISK_LONG_TIMEOUT (3600 * HZ)
+
+#define SCST_GENERIC_RAID_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_RAID_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_CDROM_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_CDROM_REG_TIMEOUT (900 * HZ)
+#define SCST_GENERIC_CDROM_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_MAX_OTHER_TIMEOUT (14000 * HZ)
+
+/*************************************************************
+ ** I/O grouping attribute string values. Must match constants
+ ** w/o '_STR' suffix!
+ *************************************************************/
+#define SCST_IO_GROUPING_AUTO_STR "auto"
+#define SCST_IO_GROUPING_THIS_GROUP_ONLY_STR "this_group_only"
+#define SCST_IO_GROUPING_NEVER_STR "never"
+
+/*************************************************************
+ ** Threads pool type attribute string values.
+ ** Must match scst_dev_type_threads_pool_type!
+ *************************************************************/
+#define SCST_THREADS_POOL_PER_INITIATOR_STR "per_initiator"
+#define SCST_THREADS_POOL_SHARED_STR "shared"
+
+/*************************************************************
+ ** Misc constants
+ *************************************************************/
+#define SCST_SYSFS_BLOCK_SIZE PAGE_SIZE
+
+#define SCST_PR_DIR "/var/lib/scst/pr"
+
+#define TID_COMMON_SIZE 24
+
+#define SCST_SYSFS_KEY_MARK "[key]"
+
+#define SCST_MIN_REL_TGT_ID 1
+#define SCST_MAX_REL_TGT_ID 65535
+
+#endif /* __SCST_CONST_H */
diff -uprN orig/linux-2.6.35/include/scst/scst.h linux-2.6.35/include/scst/scst.h
--- orig/linux-2.6.35/include/scst/scst.h
+++ linux-2.6.35/include/scst/scst.h
@@ -0,0 +1,3562 @@
+/*
+ * include/scst.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Main SCSI target mid-level include file.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_H
+#define __SCST_H
+
+#include <linux/types.h>
+#include <linux/version.h>
+#include <linux/blkdev.h>
+#include <linux/interrupt.h>
+#include <linux/wait.h>
+
+
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_eh.h>
+#include <scsi/scsi.h>
+
+#include <scst/scst_const.h>
+
+#include <scst/scst_sgv.h>
+
+/*
+ * Version numbers, the same as for the kernel.
+ *
+ * Changing it don't forget to change SCST_FIO_REV in scst_vdisk.c
+ * and FIO_REV in usr/fileio/common.h as well.
+ */
+#define SCST_VERSION(a, b, c, d) (((a) << 24) + ((b) << 16) + ((c) << 8) + d)
+#define SCST_VERSION_CODE SCST_VERSION(2, 0, 0, 0)
+#define SCST_VERSION_STRING_SUFFIX
+#define SCST_VERSION_STRING "2.1.0-pre1" SCST_VERSION_STRING_SUFFIX
+#define SCST_INTERFACE_VERSION \
+ SCST_VERSION_STRING "$Revision: 2115 $" SCST_CONST_VERSION
+
+#define SCST_LOCAL_NAME "scst_local"
+
+/*************************************************************
+ ** States of command processing state machine. At first,
+ ** "active" states, then - "passive" ones. This is to have
+ ** more efficient generated code of the corresponding
+ ** "switch" statements.
+ *************************************************************/
+
+/* Internal parsing */
+#define SCST_CMD_STATE_PRE_PARSE 0
+
+/* Dev handler's parse() is going to be called */
+#define SCST_CMD_STATE_DEV_PARSE 1
+
+/* Allocation of the cmd's data buffer */
+#define SCST_CMD_STATE_PREPARE_SPACE 2
+
+/* Calling preprocessing_done() */
+#define SCST_CMD_STATE_PREPROCESSING_DONE 3
+
+/* Target driver's rdy_to_xfer() is going to be called */
+#define SCST_CMD_STATE_RDY_TO_XFER 4
+
+/* Target driver's pre_exec() is going to be called */
+#define SCST_CMD_STATE_TGT_PRE_EXEC 5
+
+/* Cmd is going to be sent for execution */
+#define SCST_CMD_STATE_SEND_FOR_EXEC 6
+
+/* Cmd is being checked if it should be executed locally */
+#define SCST_CMD_STATE_LOCAL_EXEC 7
+
+/* Cmd is ready for execution */
+#define SCST_CMD_STATE_REAL_EXEC 8
+
+/* Internal post-exec checks */
+#define SCST_CMD_STATE_PRE_DEV_DONE 9
+
+/* Internal MODE SELECT pages related checks */
+#define SCST_CMD_STATE_MODE_SELECT_CHECKS 10
+
+/* Dev handler's dev_done() is going to be called */
+#define SCST_CMD_STATE_DEV_DONE 11
+
+/* Target driver's xmit_response() is going to be called */
+#define SCST_CMD_STATE_PRE_XMIT_RESP 12
+
+/* Target driver's xmit_response() is going to be called */
+#define SCST_CMD_STATE_XMIT_RESP 13
+
+/* Cmd finished */
+#define SCST_CMD_STATE_FINISHED 14
+
+/* Internal cmd finished */
+#define SCST_CMD_STATE_FINISHED_INTERNAL 15
+
+#define SCST_CMD_STATE_LAST_ACTIVE (SCST_CMD_STATE_FINISHED_INTERNAL+100)
+
+/* A cmd is created, but scst_cmd_init_done() not called */
+#define SCST_CMD_STATE_INIT_WAIT (SCST_CMD_STATE_LAST_ACTIVE+1)
+
+/* LUN translation (cmd->tgt_dev assignment) */
+#define SCST_CMD_STATE_INIT (SCST_CMD_STATE_LAST_ACTIVE+2)
+
+/* Waiting for scst_restart_cmd() */
+#define SCST_CMD_STATE_PREPROCESSING_DONE_CALLED (SCST_CMD_STATE_LAST_ACTIVE+3)
+
+/* Waiting for data from the initiator (until scst_rx_data() called) */
+#define SCST_CMD_STATE_DATA_WAIT (SCST_CMD_STATE_LAST_ACTIVE+4)
+
+/* Waiting for CDB's execution finish */
+#define SCST_CMD_STATE_REAL_EXECUTING (SCST_CMD_STATE_LAST_ACTIVE+5)
+
+/* Waiting for response's transmission finish */
+#define SCST_CMD_STATE_XMIT_WAIT (SCST_CMD_STATE_LAST_ACTIVE+6)
+
+/*************************************************************
+ * Can be retuned instead of cmd's state by dev handlers'
+ * functions, if the command's state should be set by default
+ *************************************************************/
+#define SCST_CMD_STATE_DEFAULT 500
+
+/*************************************************************
+ * Can be retuned instead of cmd's state by dev handlers'
+ * functions, if it is impossible to complete requested
+ * task in atomic context. The cmd will be restarted in thread
+ * context.
+ *************************************************************/
+#define SCST_CMD_STATE_NEED_THREAD_CTX 1000
+
+/*************************************************************
+ * Can be retuned instead of cmd's state by dev handlers'
+ * parse function, if the cmd processing should be stopped
+ * for now. The cmd will be restarted by dev handlers itself.
+ *************************************************************/
+#define SCST_CMD_STATE_STOP 1001
+
+/*************************************************************
+ ** States of mgmt command processing state machine
+ *************************************************************/
+
+/* LUN translation (mcmd->tgt_dev assignment) */
+#define SCST_MCMD_STATE_INIT 0
+
+/* Mgmt cmd is being processed */
+#define SCST_MCMD_STATE_EXEC 1
+
+/* Waiting for affected commands done */
+#define SCST_MCMD_STATE_WAITING_AFFECTED_CMDS_DONE 2
+
+/* Post actions when affected commands done */
+#define SCST_MCMD_STATE_AFFECTED_CMDS_DONE 3
+
+/* Waiting for affected local commands finished */
+#define SCST_MCMD_STATE_WAITING_AFFECTED_CMDS_FINISHED 4
+
+/* Target driver's task_mgmt_fn_done() is going to be called */
+#define SCST_MCMD_STATE_DONE 5
+
+/* The mcmd finished */
+#define SCST_MCMD_STATE_FINISHED 6
+
+/*************************************************************
+ ** Constants for "atomic" parameter of SCST's functions
+ *************************************************************/
+#define SCST_NON_ATOMIC 0
+#define SCST_ATOMIC 1
+
+/*************************************************************
+ ** Values for pref_context parameter of scst_cmd_init_done(),
+ ** scst_rx_data(), scst_restart_cmd(), scst_tgt_cmd_done()
+ ** and scst_cmd_done()
+ *************************************************************/
+
+enum scst_exec_context {
+ /*
+ * Direct cmd's processing (i.e. regular function calls in the current
+ * context) sleeping is not allowed
+ */
+ SCST_CONTEXT_DIRECT_ATOMIC,
+
+ /*
+ * Direct cmd's processing (i.e. regular function calls in the current
+ * context), sleeping is allowed, no restrictions
+ */
+ SCST_CONTEXT_DIRECT,
+
+ /* Tasklet or thread context required for cmd's processing */
+ SCST_CONTEXT_TASKLET,
+
+ /* Thread context required for cmd's processing */
+ SCST_CONTEXT_THREAD,
+
+ /*
+ * Context is the same as it was in previous call of the corresponding
+ * callback. For example, if dev handler's exec() does sync. data
+ * reading this value should be used for scst_cmd_done(). The same is
+ * true if scst_tgt_cmd_done() called directly from target driver's
+ * xmit_response(). Not allowed in scst_cmd_init_done() and
+ * scst_cmd_init_stage1_done().
+ */
+ SCST_CONTEXT_SAME
+};
+
+/*************************************************************
+ ** Values for status parameter of scst_rx_data()
+ *************************************************************/
+
+/* Success */
+#define SCST_RX_STATUS_SUCCESS 0
+
+/*
+ * Data receiving finished with error, so set the sense and
+ * finish the command, including xmit_response() call
+ */
+#define SCST_RX_STATUS_ERROR 1
+
+/*
+ * Data receiving finished with error and the sense is set,
+ * so finish the command, including xmit_response() call
+ */
+#define SCST_RX_STATUS_ERROR_SENSE_SET 2
+
+/*
+ * Data receiving finished with fatal error, so finish the command,
+ * but don't call xmit_response()
+ */
+#define SCST_RX_STATUS_ERROR_FATAL 3
+
+/*************************************************************
+ ** Values for status parameter of scst_restart_cmd()
+ *************************************************************/
+
+/* Success */
+#define SCST_PREPROCESS_STATUS_SUCCESS 0
+
+/*
+ * Command's processing finished with error, so set the sense and
+ * finish the command, including xmit_response() call
+ */
+#define SCST_PREPROCESS_STATUS_ERROR 1
+
+/*
+ * Command's processing finished with error and the sense is set,
+ * so finish the command, including xmit_response() call
+ */
+#define SCST_PREPROCESS_STATUS_ERROR_SENSE_SET 2
+
+/*
+ * Command's processing finished with fatal error, so finish the command,
+ * but don't call xmit_response()
+ */
+#define SCST_PREPROCESS_STATUS_ERROR_FATAL 3
+
+/*************************************************************
+ ** Values for AEN functions
+ *************************************************************/
+
+/*
+ * SCSI Asynchronous Event. Parameter contains SCSI sense
+ * (Unit Attention). AENs generated only for 2 the following UAs:
+ * CAPACITY DATA HAS CHANGED and REPORTED LUNS DATA HAS CHANGED.
+ * Other UAs reported regularly as CHECK CONDITION status,
+ * because it doesn't look safe to report them using AENs, since
+ * reporting using AENs opens delivery race windows even in case of
+ * untagged commands.
+ */
+#define SCST_AEN_SCSI 0
+
+/*************************************************************
+ ** Allowed return/status codes for report_aen() callback and
+ ** scst_set_aen_delivery_status() function
+ *************************************************************/
+
+/* Success */
+#define SCST_AEN_RES_SUCCESS 0
+
+/* Not supported */
+#define SCST_AEN_RES_NOT_SUPPORTED -1
+
+/* Failure */
+#define SCST_AEN_RES_FAILED -2
+
+/*************************************************************
+ ** Allowed return codes for xmit_response(), rdy_to_xfer()
+ *************************************************************/
+
+/* Success */
+#define SCST_TGT_RES_SUCCESS 0
+
+/* Internal device queue is full, retry again later */
+#define SCST_TGT_RES_QUEUE_FULL -1
+
+/*
+ * It is impossible to complete requested task in atomic context.
+ * The cmd will be restarted in thread context.
+ */
+#define SCST_TGT_RES_NEED_THREAD_CTX -2
+
+/*
+ * Fatal error, if returned by xmit_response() the cmd will
+ * be destroyed, if by any other function, xmit_response()
+ * will be called with HARDWARE ERROR sense data
+ */
+#define SCST_TGT_RES_FATAL_ERROR -3
+
+/*************************************************************
+ ** Allowed return codes for dev handler's exec()
+ *************************************************************/
+
+/* The cmd is done, go to other ones */
+#define SCST_EXEC_COMPLETED 0
+
+/* The cmd should be sent to SCSI mid-level */
+#define SCST_EXEC_NOT_COMPLETED 1
+
+/*
+ * Set if cmd is finished and there is status/sense to be sent.
+ * The status should be not sent (i.e. the flag not set) if the
+ * possibility to perform a command in "chunks" (i.e. with multiple
+ * xmit_response()/rdy_to_xfer()) is used (not implemented yet).
+ * Obsolete, use scst_cmd_get_is_send_status() instead.
+ */
+#define SCST_TSC_FLAG_STATUS 0x2
+
+/*************************************************************
+ ** Additional return code for dev handler's task_mgmt_fn()
+ *************************************************************/
+
+/* Regular standard actions for the command should be done */
+#define SCST_DEV_TM_NOT_COMPLETED 1
+
+/*************************************************************
+ ** Session initialization phases
+ *************************************************************/
+
+/* Set if session is being initialized */
+#define SCST_SESS_IPH_INITING 0
+
+/* Set if the session is successfully initialized */
+#define SCST_SESS_IPH_SUCCESS 1
+
+/* Set if the session initialization failed */
+#define SCST_SESS_IPH_FAILED 2
+
+/* Set if session is initialized and ready */
+#define SCST_SESS_IPH_READY 3
+
+/*************************************************************
+ ** Session shutdown phases
+ *************************************************************/
+
+/* Set if session is initialized and ready */
+#define SCST_SESS_SPH_READY 0
+
+/* Set if session is shutting down */
+#define SCST_SESS_SPH_SHUTDOWN 1
+
+/* Set if session is shutting down */
+#define SCST_SESS_SPH_UNREG_DONE_CALLING 2
+
+/*************************************************************
+ ** Session's async (atomic) flags
+ *************************************************************/
+
+/* Set if the sess's hw pending work is scheduled */
+#define SCST_SESS_HW_PENDING_WORK_SCHEDULED 0
+
+/*************************************************************
+ ** Cmd's async (atomic) flags
+ *************************************************************/
+
+/* Set if the cmd is aborted and ABORTED sense will be sent as the result */
+#define SCST_CMD_ABORTED 0
+
+/* Set if the cmd is aborted by other initiator */
+#define SCST_CMD_ABORTED_OTHER 1
+
+/* Set if no response should be sent to the target about this cmd */
+#define SCST_CMD_NO_RESP 2
+
+/* Set if the cmd is dead and can be destroyed at any time */
+#define SCST_CMD_CAN_BE_DESTROYED 3
+
+/*
+ * Set if the cmd's device has TAS flag set. Used only when aborted by
+ * other initiator.
+ */
+#define SCST_CMD_DEVICE_TAS 4
+
+/*************************************************************
+ ** Tgt_dev's async. flags (tgt_dev_flags)
+ *************************************************************/
+
+/* Set if tgt_dev has Unit Attention sense */
+#define SCST_TGT_DEV_UA_PENDING 0
+
+/* Set if tgt_dev is RESERVED by another session */
+#define SCST_TGT_DEV_RESERVED 1
+
+/* Set if the corresponding context is atomic */
+#define SCST_TGT_DEV_AFTER_INIT_WR_ATOMIC 5
+#define SCST_TGT_DEV_AFTER_EXEC_ATOMIC 6
+
+#define SCST_TGT_DEV_CLUST_POOL 11
+
+/*************************************************************
+ ** I/O groupping types. Changing them don't forget to change
+ ** the corresponding *_STR values in scst_const.h!
+ *************************************************************/
+
+/*
+ * All initiators with the same name connected to this group will have
+ * shared IO context, for each name own context. All initiators with
+ * different names will have own IO context.
+ */
+#define SCST_IO_GROUPING_AUTO 0
+
+/* All initiators connected to this group will have shared IO context */
+#define SCST_IO_GROUPING_THIS_GROUP_ONLY -1
+
+/* Each initiator connected to this group will have own IO context */
+#define SCST_IO_GROUPING_NEVER -2
+
+/*************************************************************
+ ** Kernel cache creation helper
+ *************************************************************/
+#ifndef KMEM_CACHE
+#define KMEM_CACHE(__struct, __flags) kmem_cache_create(#__struct,\
+ sizeof(struct __struct), __alignof__(struct __struct),\
+ (__flags), NULL, NULL)
+#endif
+
+/*************************************************************
+ ** Vlaid_mask constants for scst_analyze_sense()
+ *************************************************************/
+
+#define SCST_SENSE_KEY_VALID 1
+#define SCST_SENSE_ASC_VALID 2
+#define SCST_SENSE_ASCQ_VALID 4
+
+#define SCST_SENSE_ASCx_VALID (SCST_SENSE_ASC_VALID | \
+ SCST_SENSE_ASCQ_VALID)
+
+#define SCST_SENSE_ALL_VALID (SCST_SENSE_KEY_VALID | \
+ SCST_SENSE_ASC_VALID | \
+ SCST_SENSE_ASCQ_VALID)
+
+/*************************************************************
+ * TYPES
+ *************************************************************/
+
+struct scst_tgt;
+struct scst_session;
+struct scst_cmd;
+struct scst_mgmt_cmd;
+struct scst_device;
+struct scst_tgt_dev;
+struct scst_dev_type;
+struct scst_acg;
+struct scst_acg_dev;
+struct scst_acn;
+struct scst_aen;
+
+/*
+ * SCST uses 64-bit numbers to represent LUN's internally. The value
+ * NO_SUCH_LUN is guaranteed to be different of every valid LUN.
+ */
+#define NO_SUCH_LUN ((uint64_t)-1)
+
+typedef enum dma_data_direction scst_data_direction;
+
+/*
+ * SCST target template: defines target driver's parameters and callback
+ * functions.
+ *
+ * MUST HAVEs define functions that are expected to be defined in order to
+ * work. OPTIONAL says that there is a choice.
+ */
+struct scst_tgt_template {
+ /* public: */
+
+ /*
+ * SG tablesize allows to check whether scatter/gather can be used
+ * or not.
+ */
+ int sg_tablesize;
+
+ /*
+ * True, if this target adapter uses unchecked DMA onto an ISA bus.
+ */
+ unsigned unchecked_isa_dma:1;
+
+ /*
+ * True, if this target adapter can benefit from using SG-vector
+ * clustering (i.e. smaller number of segments).
+ */
+ unsigned use_clustering:1;
+
+ /*
+ * True, if this target adapter doesn't support SG-vector clustering
+ */
+ unsigned no_clustering:1;
+
+ /*
+ * True, if corresponding function supports execution in
+ * the atomic (non-sleeping) context
+ */
+ unsigned xmit_response_atomic:1;
+ unsigned rdy_to_xfer_atomic:1;
+
+ /* True, if this target doesn't need "enabled" attribute */
+ unsigned enabled_attr_not_needed:1;
+
+ /*
+ * The maximum time in seconds cmd can stay inside the target
+ * hardware, i.e. after rdy_to_xfer() and xmit_response(), before
+ * on_hw_pending_cmd_timeout() will be called, if defined.
+ *
+ * In the current implementation a cmd will be aborted in time t
+ * max_hw_pending_time <= t < 2*max_hw_pending_time.
+ */
+ int max_hw_pending_time;
+
+ /*
+ * This function is equivalent to the SCSI
+ * queuecommand. The target should transmit the response
+ * buffer and the status in the scst_cmd struct.
+ * The expectation is that this executing this command is NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * After the response is actually transmitted, the target
+ * should call the scst_tgt_cmd_done() function of the
+ * mid-level, which will allow it to free up the command.
+ * Returns one of the SCST_TGT_RES_* constants.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * MUST HAVE
+ */
+ int (*xmit_response) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that data
+ * buffer corresponding to the said command have now been
+ * allocated and it is OK to receive data for this command.
+ * This function is necessary because a SCSI target does not
+ * have any control over the commands it receives. Most lower
+ * level protocols have a corresponding function which informs
+ * the initiator that buffers have been allocated e.g., XFER_
+ * RDY in Fibre Channel. After the data is actually received
+ * the low-level driver needs to call scst_rx_data() in order to
+ * continue processing this command.
+ * Returns one of the SCST_TGT_RES_* constants.
+ *
+ * This command is expected to be NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * OPTIONAL
+ */
+ int (*rdy_to_xfer) (struct scst_cmd *cmd);
+
+ /*
+ * Called if cmd stays inside the target hardware, i.e. after
+ * rdy_to_xfer() and xmit_response(), more than max_hw_pending_time
+ * time. The target driver supposed to cleanup this command and
+ * resume cmd's processing.
+ *
+ * OPTIONAL
+ */
+ void (*on_hw_pending_cmd_timeout) (struct scst_cmd *cmd);
+
+ /*
+ * Called to notify the driver that the command is about to be freed.
+ * Necessary, because for aborted commands xmit_response() could not
+ * be called. Could be called on IRQ context.
+ *
+ * OPTIONAL
+ */
+ void (*on_free_cmd) (struct scst_cmd *cmd);
+
+ /*
+ * This function allows target driver to handle data buffer
+ * allocations on its own.
+ *
+ * Target driver doesn't have to always allocate buffer in this
+ * function, but if it decide to do it, it must check that
+ * scst_cmd_get_data_buff_alloced() returns 0, otherwise to avoid
+ * double buffer allocation and memory leaks alloc_data_buf() shall
+ * fail.
+ *
+ * Shall return 0 in case of success or < 0 (preferrably -ENOMEM)
+ * in case of error, or > 0 if the regular SCST allocation should be
+ * done. In case of returning successfully,
+ * scst_cmd->tgt_data_buf_alloced will be set by SCST.
+ *
+ * It is possible that both target driver and dev handler request own
+ * memory allocation. In this case, data will be memcpy() between
+ * buffers, where necessary.
+ *
+ * If allocation in atomic context - cf. scst_cmd_atomic() - is not
+ * desired or fails and consequently < 0 is returned, this function
+ * will be re-called in thread context.
+ *
+ * Please note that the driver will have to handle itself all relevant
+ * details such as scatterlist setup, highmem, freeing the allocated
+ * memory, etc.
+ *
+ * OPTIONAL.
+ */
+ int (*alloc_data_buf) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that data
+ * buffer corresponding to the said command have now been
+ * allocated and other preprocessing tasks have been done.
+ * A target driver could need to do some actions at this stage.
+ * After the target driver done the needed actions, it shall call
+ * scst_restart_cmd() in order to continue processing this command.
+ * In case of preliminary the command completion, this function will
+ * also be called before xmit_response().
+ *
+ * Called only if the cmd is queued using scst_cmd_init_stage1_done()
+ * instead of scst_cmd_init_done().
+ *
+ * Returns void, the result is expected to be returned using
+ * scst_restart_cmd().
+ *
+ * This command is expected to be NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * OPTIONAL.
+ */
+ void (*preprocessing_done) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that the said command is about
+ * to be executed.
+ *
+ * Returns one of the SCST_PREPROCESS_* constants.
+ *
+ * This command is expected to be NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * OPTIONAL
+ */
+ int (*pre_exec) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that all affected by the
+ * corresponding task management function commands have beed completed.
+ * No return value expected.
+ *
+ * This function is expected to be NON-BLOCKING.
+ *
+ * Called without any locks held from a thread context.
+ *
+ * OPTIONAL
+ */
+ void (*task_mgmt_affected_cmds_done) (struct scst_mgmt_cmd *mgmt_cmd);
+
+ /*
+ * This function informs the driver that the corresponding task
+ * management function has been completed, i.e. all the corresponding
+ * commands completed and freed. No return value expected.
+ *
+ * This function is expected to be NON-BLOCKING.
+ *
+ * Called without any locks held from a thread context.
+ *
+ * MUST HAVE if the target supports task management.
+ */
+ void (*task_mgmt_fn_done) (struct scst_mgmt_cmd *mgmt_cmd);
+
+ /*
+ * This function should detect the target adapters that
+ * are present in the system. The function should return a value
+ * >= 0 to signify the number of detected target adapters.
+ * A negative value should be returned whenever there is
+ * an error.
+ *
+ * MUST HAVE
+ */
+ int (*detect) (struct scst_tgt_template *tgt_template);
+
+ /*
+ * This function should free up the resources allocated to the device.
+ * The function should return 0 to indicate successful release
+ * or a negative value if there are some issues with the release.
+ * In the current version the return value is ignored.
+ *
+ * MUST HAVE
+ */
+ int (*release) (struct scst_tgt *tgt);
+
+ /*
+ * This function is used for Asynchronous Event Notifications.
+ *
+ * Returns one of the SCST_AEN_RES_* constants.
+ * After AEN is sent, target driver must call scst_aen_done() and,
+ * optionally, scst_set_aen_delivery_status().
+ *
+ * This function is expected to be NON-BLOCKING, but can sleep.
+ *
+ * This function must be prepared to handle AENs between calls for the
+ * corresponding session of scst_unregister_session() and
+ * unreg_done_fn() callback called or before scst_unregister_session()
+ * returned, if its called in the blocking mode. AENs for such sessions
+ * should be ignored.
+ *
+ * MUST HAVE, if low-level protocol supports AENs.
+ */
+ int (*report_aen) (struct scst_aen *aen);
+
+ /*
+ * This function returns in tr_id the corresponding to sess initiator
+ * port TransporID in the form as it's used by PR commands, see
+ * "Transport Identifiers" in SPC. Space for the initiator port
+ * TransporID must be allocated via kmalloc(). Caller supposed to
+ * kfree() it, when it isn't needed anymore.
+ *
+ * If sess is NULL, this function must return TransportID PROTOCOL
+ * IDENTIFIER of this transport.
+ *
+ * Returns 0 on success or negative error code otherwise.
+ *
+ * SHOULD HAVE, because it's required for Persistent Reservations.
+ */
+ int (*get_initiator_port_transport_id) (struct scst_session *sess,
+ uint8_t **transport_id);
+
+ /*
+ * This function allows to enable or disable particular target.
+ * A disabled target doesn't receive and process any SCSI commands.
+ *
+ * SHOULD HAVE to avoid race when there are connected initiators,
+ * while target not yet completed the initial configuration. In this
+ * case the too early connected initiators would see not those devices,
+ * which they intended to see.
+ *
+ * If you are sure your target driver doesn't need enabling target,
+ * you should set enabled_attr_not_needed in 1.
+ */
+ int (*enable_target) (struct scst_tgt *tgt, bool enable);
+
+ /*
+ * This function shows if particular target is enabled or not.
+ *
+ * SHOULD HAVE, see above why.
+ */
+ bool (*is_target_enabled) (struct scst_tgt *tgt);
+
+ /*
+ * This function adds a virtual target.
+ *
+ * If both add_target and del_target callbacks defined, then this
+ * target driver supposed to support virtual targets. In this case
+ * an "mgmt" entry will be created in the sysfs root for this driver.
+ * The "mgmt" entry will support 2 commands: "add_target" and
+ * "del_target", for which the corresponding callbacks will be called.
+ * Also target driver can define own commands for the "mgmt" entry, see
+ * mgmt_cmd and mgmt_cmd_help below.
+ *
+ * This approach allows uniform targets management to simplify external
+ * management tools like scstadmin. See README for more details.
+ *
+ * Either both add_target and del_target must be defined, or none.
+ *
+ * MUST HAVE if virtual targets are supported.
+ */
+ ssize_t (*add_target) (const char *target_name, char *params);
+
+ /*
+ * This function deletes a virtual target. See comment for add_target
+ * above.
+ *
+ * MUST HAVE if virtual targets are supported.
+ */
+ ssize_t (*del_target) (const char *target_name);
+
+ /*
+ * This function called if not "add_target" or "del_target" command is
+ * sent to the mgmt entry (see comment for add_target above). In this
+ * case the command passed to this function as is in a string form.
+ *
+ * OPTIONAL.
+ */
+ ssize_t (*mgmt_cmd) (char *cmd);
+
+ /*
+ * Should return physical transport version. Used in the corresponding
+ * INQUIRY version descriptor. See SPC for the list of available codes.
+ *
+ * OPTIONAL
+ */
+ uint16_t (*get_phys_transport_version) (struct scst_tgt *tgt);
+
+ /*
+ * Should return SCSI transport version. Used in the corresponding
+ * INQUIRY version descriptor. See SPC for the list of available codes.
+ *
+ * OPTIONAL
+ */
+ uint16_t (*get_scsi_transport_version) (struct scst_tgt *tgt);
+
+ /*
+ * Name of the template. Must be unique to identify
+ * the template. MUST HAVE
+ */
+ const char name[SCST_MAX_NAME];
+
+ /*
+ * Number of additional threads to the pool of dedicated threads.
+ * Used if xmit_response() or rdy_to_xfer() is blocking.
+ * It is the target driver's duty to ensure that not more, than that
+ * number of threads, are blocked in those functions at any time.
+ */
+ int threads_num;
+
+ /* Optional default log flags */
+ const unsigned long default_trace_flags;
+
+ /* Optional pointer to trace flags */
+ unsigned long *trace_flags;
+
+ /* Optional local trace table */
+ struct scst_trace_log *trace_tbl;
+
+ /* Optional local trace table help string */
+ const char *trace_tbl_help;
+
+ /* sysfs attributes, if any */
+ const struct attribute **tgtt_attrs;
+
+ /* sysfs target attributes, if any */
+ const struct attribute **tgt_attrs;
+
+ /* sysfs session attributes, if any */
+ const struct attribute **sess_attrs;
+
+ /* Optional help string for mgmt_cmd commands */
+ const char *mgmt_cmd_help;
+
+ /* List of parameters for add_target command, if any */
+ const char *add_target_parameters;
+
+ /*
+ * List of optional, i.e. which could be added by add_attribute command
+ * and deleted by del_attribute command, sysfs attributes, if any.
+ * Helpful for scstadmin to work correctly.
+ */
+ const char *tgtt_optional_attributes;
+
+ /*
+ * List of optional, i.e. which could be added by add_target_attribute
+ * command and deleted by del_target_attribute command, sysfs
+ * attributes, if any. Helpful for scstadmin to work correctly.
+ */
+ const char *tgt_optional_attributes;
+
+ /** Private, must be inited to 0 by memset() **/
+
+ /* List of targets per template, protected by scst_mutex */
+ struct list_head tgt_list;
+
+ /* List entry of global templates list */
+ struct list_head scst_template_list_entry;
+
+ struct kobject tgtt_kobj; /* kobject for this struct */
+
+ /* Number of currently active sysfs mgmt works (scst_sysfs_work_item) */
+ int tgtt_active_sysfs_works_count;
+
+ /* sysfs release completion */
+ struct completion tgtt_kobj_release_cmpl;
+
+};
+
+/*
+ * Threads pool types. Changing them don't forget to change
+ * the corresponding *_STR values in scst_const.h!
+ */
+enum scst_dev_type_threads_pool_type {
+ /* Each initiator will have dedicated threads pool. */
+ SCST_THREADS_POOL_PER_INITIATOR = 0,
+
+ /* All connected initiators will use shared threads pool */
+ SCST_THREADS_POOL_SHARED,
+
+ /* Invalid value for scst_parse_threads_pool_type() */
+ SCST_THREADS_POOL_TYPE_INVALID,
+};
+
+/*
+ * SCST dev handler template: defines dev handler's parameters and callback
+ * functions.
+ *
+ * MUST HAVEs define functions that are expected to be defined in order to
+ * work. OPTIONAL says that there is a choice.
+ */
+struct scst_dev_type {
+ /* SCSI type of the supported device. MUST HAVE */
+ int type;
+
+ /*
+ * True, if corresponding function supports execution in
+ * the atomic (non-sleeping) context
+ */
+ unsigned parse_atomic:1;
+ unsigned alloc_data_buf_atomic:1;
+ unsigned dev_done_atomic:1;
+
+ /*
+ * Should be true, if exec() is synchronous. This is a hint to SCST core
+ * to optimize commands order management.
+ */
+ unsigned exec_sync:1;
+
+ /*
+ * Should be set if the device wants to receive notification of
+ * Persistent Reservation commands (PR OUT only)
+ * Note: The notification will not be send if the command failed
+ */
+ unsigned pr_cmds_notifications:1;
+
+ /*
+ * Called to parse CDB from the cmd and initialize
+ * cmd->bufflen and cmd->data_direction (both - REQUIRED).
+ *
+ * Returns the command's next state or SCST_CMD_STATE_DEFAULT,
+ * if the next default state should be used, or
+ * SCST_CMD_STATE_NEED_THREAD_CTX if the function called in atomic
+ * context, but requires sleeping, or SCST_CMD_STATE_STOP if the
+ * command should not be further processed for now. In the
+ * SCST_CMD_STATE_NEED_THREAD_CTX case the function
+ * will be recalled in the thread context, where sleeping is allowed.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * MUST HAVE
+ */
+ int (*parse) (struct scst_cmd *cmd);
+
+ /*
+ * This function allows dev handler to handle data buffer
+ * allocations on its own.
+ *
+ * Returns the command's next state or SCST_CMD_STATE_DEFAULT,
+ * if the next default state should be used, or
+ * SCST_CMD_STATE_NEED_THREAD_CTX if the function called in atomic
+ * context, but requires sleeping, or SCST_CMD_STATE_STOP if the
+ * command should not be further processed for now. In the
+ * SCST_CMD_STATE_NEED_THREAD_CTX case the function
+ * will be recalled in the thread context, where sleeping is allowed.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * OPTIONAL
+ */
+ int (*alloc_data_buf) (struct scst_cmd *cmd);
+
+ /*
+ * Called to execute CDB. Useful, for instance, to implement
+ * data caching. The result of CDB execution is reported via
+ * cmd->scst_cmd_done() callback.
+ * Returns:
+ * - SCST_EXEC_COMPLETED - the cmd is done, go to other ones
+ * - SCST_EXEC_NOT_COMPLETED - the cmd should be sent to SCSI
+ * mid-level.
+ *
+ * If this function provides sync execution, you should set
+ * exec_sync flag and consider to setup dedicated threads by
+ * setting threads_num > 0.
+ *
+ * !! If this function is implemented, scst_check_local_events() !!
+ * !! shall be called inside it just before the actual command's !!
+ * !! execution. !!
+ *
+ * OPTIONAL, if not set, the commands will be sent directly to SCSI
+ * device.
+ */
+ int (*exec) (struct scst_cmd *cmd);
+
+ /*
+ * Called to notify dev handler about the result of cmd execution
+ * and perform some post processing. Cmd's fields is_send_status and
+ * resp_data_len should be set by this function, but SCST offers good
+ * defaults.
+ * Returns the command's next state or SCST_CMD_STATE_DEFAULT,
+ * if the next default state should be used, or
+ * SCST_CMD_STATE_NEED_THREAD_CTX if the function called in atomic
+ * context, but requires sleeping. In the last case, the function
+ * will be recalled in the thread context, where sleeping is allowed.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * OPTIONAL
+ */
+ int (*dev_done) (struct scst_cmd *cmd);
+
+ /*
+ * Called to notify dev hander that the command is about to be freed.
+ *
+ * Could be called on IRQ context.
+ *
+ * OPTIONAL
+ */
+ void (*on_free_cmd) (struct scst_cmd *cmd);
+
+ /*
+ * Called to execute a task management command.
+ * Returns:
+ * - SCST_MGMT_STATUS_SUCCESS - the command is done with success,
+ * no firther actions required
+ * - The SCST_MGMT_STATUS_* error code if the command is failed and
+ * no further actions required
+ * - SCST_DEV_TM_NOT_COMPLETED - regular standard actions for the
+ * command should be done
+ *
+ * Called without any locks held from a thread context.
+ *
+ * OPTIONAL
+ */
+ int (*task_mgmt_fn) (struct scst_mgmt_cmd *mgmt_cmd,
+ struct scst_tgt_dev *tgt_dev);
+
+ /*
+ * Called to notify dev handler that its sg_tablesize is too low to
+ * satisfy this command's data transfer requirements. Should return
+ * true if exec() callback will split this command's CDB on smaller
+ * transfers, false otherwise.
+ *
+ * Could be called on SIRQ context.
+ *
+ * MUST HAVE, if dev handler supports CDB splitting.
+ */
+ bool (*on_sg_tablesize_low) (struct scst_cmd *cmd);
+
+ /*
+ * Called when new device is attaching to the dev handler
+ * Returns 0 on success, error code otherwise.
+ *
+ * OPTIONAL
+ */
+ int (*attach) (struct scst_device *dev);
+
+ /*
+ * Called when a device is detaching from the dev handler.
+ *
+ * OPTIONAL
+ */
+ void (*detach) (struct scst_device *dev);
+
+ /*
+ * Called when new tgt_dev (session) is attaching to the dev handler.
+ * Returns 0 on success, error code otherwise.
+ *
+ * OPTIONAL
+ */
+ int (*attach_tgt) (struct scst_tgt_dev *tgt_dev);
+
+ /*
+ * Called when tgt_dev (session) is detaching from the dev handler.
+ *
+ * OPTIONAL
+ */
+ void (*detach_tgt) (struct scst_tgt_dev *tgt_dev);
+
+ /*
+ * This function adds a virtual device.
+ *
+ * If both add_device and del_device callbacks defined, then this
+ * dev handler supposed to support adding/deleting virtual devices.
+ * In this case an "mgmt" entry will be created in the sysfs root for
+ * this handler. The "mgmt" entry will support 2 commands: "add_device"
+ * and "del_device", for which the corresponding callbacks will be called.
+ * Also dev handler can define own commands for the "mgmt" entry, see
+ * mgmt_cmd and mgmt_cmd_help below.
+ *
+ * This approach allows uniform devices management to simplify external
+ * management tools like scstadmin. See README for more details.
+ *
+ * Either both add_device and del_device must be defined, or none.
+ *
+ * MUST HAVE if virtual devices are supported.
+ */
+ ssize_t (*add_device) (const char *device_name, char *params);
+
+ /*
+ * This function deletes a virtual device. See comment for add_device
+ * above.
+ *
+ * MUST HAVE if virtual devices are supported.
+ */
+ ssize_t (*del_device) (const char *device_name);
+
+ /*
+ * This function called if not "add_device" or "del_device" command is
+ * sent to the mgmt entry (see comment for add_device above). In this
+ * case the command passed to this function as is in a string form.
+ *
+ * OPTIONAL.
+ */
+ ssize_t (*mgmt_cmd) (char *cmd);
+
+ /*
+ * Name of the dev handler. Must be unique. MUST HAVE.
+ *
+ * It's SCST_MAX_NAME + few more bytes to match scst_user expectations.
+ */
+ char name[SCST_MAX_NAME + 10];
+
+ /*
+ * Number of threads in this handler's devices' threads pools.
+ * If 0 - no threads will be created, if <0 - creation of the threads
+ * pools is prohibited. Also pay attention to threads_pool_type below.
+ */
+ int threads_num;
+
+ /* Threads pool type. Valid only if threads_num > 0. */
+ enum scst_dev_type_threads_pool_type threads_pool_type;
+
+ /* Optional default log flags */
+ const unsigned long default_trace_flags;
+
+ /* Optional pointer to trace flags */
+ unsigned long *trace_flags;
+
+ /* Optional local trace table */
+ struct scst_trace_log *trace_tbl;
+
+ /* Optional local trace table help string */
+ const char *trace_tbl_help;
+
+ /* Optional help string for mgmt_cmd commands */
+ const char *mgmt_cmd_help;
+
+ /* List of parameters for add_device command, if any */
+ const char *add_device_parameters;
+
+ /*
+ * List of optional, i.e. which could be added by add_attribute command
+ * and deleted by del_attribute command, sysfs attributes, if any.
+ * Helpful for scstadmin to work correctly.
+ */
+ const char *devt_optional_attributes;
+
+ /*
+ * List of optional, i.e. which could be added by add_device_attribute
+ * command and deleted by del_device_attribute command, sysfs
+ * attributes, if any. Helpful for scstadmin to work correctly.
+ */
+ const char *dev_optional_attributes;
+
+ /* sysfs attributes, if any */
+ const struct attribute **devt_attrs;
+
+ /* sysfs device attributes, if any */
+ const struct attribute **dev_attrs;
+
+ /* Pointer to dev handler's private data */
+ void *devt_priv;
+
+ /* Pointer to parent dev type in the sysfs hierarchy */
+ struct scst_dev_type *parent;
+
+ struct module *module;
+
+ /** Private, must be inited to 0 by memset() **/
+
+ /* list entry in scst_(virtual_)dev_type_list */
+ struct list_head dev_type_list_entry;
+
+ struct kobject devt_kobj; /* main handlers/driver */
+
+ /* Number of currently active sysfs mgmt works (scst_sysfs_work_item) */
+ int devt_active_sysfs_works_count;
+
+ /* To wait until devt_kobj released */
+ struct completion devt_kobj_release_compl;
+};
+
+/*
+ * An SCST target, analog of SCSI target port.
+ */
+struct scst_tgt {
+ /* List of remote sessions per target, protected by scst_mutex */
+ struct list_head sess_list;
+
+ /* List entry of targets per template (tgts_list) */
+ struct list_head tgt_list_entry;
+
+ struct scst_tgt_template *tgtt; /* corresponding target template */
+
+ struct scst_acg *default_acg; /* default acg for this target */
+
+ struct list_head tgt_acg_list; /* target ACG groups */
+
+ /*
+ * Maximum SG table size. Needed here, since different cards on the
+ * same target template can have different SG table limitations.
+ */
+ int sg_tablesize;
+
+ /* Used for storage of target driver private stuff */
+ void *tgt_priv;
+
+ /*
+ * The following fields used to store and retry cmds if target's
+ * internal queue is full, so the target is unable to accept
+ * the cmd returning QUEUE FULL.
+ * They protected by tgt_lock, where necessary.
+ */
+ bool retry_timer_active;
+ struct timer_list retry_timer;
+ atomic_t finished_cmds;
+ int retry_cmds;
+ spinlock_t tgt_lock;
+ struct list_head retry_cmd_list;
+
+ /* Used to wait until session finished to unregister */
+ wait_queue_head_t unreg_waitQ;
+
+ /* Name of the target */
+ char *tgt_name;
+
+ uint16_t rel_tgt_id;
+
+ /* sysfs release completion */
+ struct completion tgt_kobj_release_cmpl;
+
+ struct kobject tgt_kobj; /* main targets/target kobject */
+ struct kobject *tgt_sess_kobj; /* target/sessions/ */
+ struct kobject *tgt_luns_kobj; /* target/luns/ */
+ struct kobject *tgt_ini_grp_kobj; /* target/ini_groups/ */
+};
+
+/* Hash size and hash fn for hash based lun translation */
+#define TGT_DEV_HASH_SHIFT 5
+#define TGT_DEV_HASH_SIZE (1 << TGT_DEV_HASH_SHIFT)
+#define HASH_VAL(_val) (_val & (TGT_DEV_HASH_SIZE - 1))
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+/* Defines extended latency statistics */
+struct scst_ext_latency_stat {
+ uint64_t scst_time_rd, tgt_time_rd, dev_time_rd;
+ unsigned int processed_cmds_rd;
+ uint64_t min_scst_time_rd, min_tgt_time_rd, min_dev_time_rd;
+ uint64_t max_scst_time_rd, max_tgt_time_rd, max_dev_time_rd;
+
+ uint64_t scst_time_wr, tgt_time_wr, dev_time_wr;
+ unsigned int processed_cmds_wr;
+ uint64_t min_scst_time_wr, min_tgt_time_wr, min_dev_time_wr;
+ uint64_t max_scst_time_wr, max_tgt_time_wr, max_dev_time_wr;
+};
+
+#define SCST_IO_SIZE_THRESHOLD_SMALL (8*1024)
+#define SCST_IO_SIZE_THRESHOLD_MEDIUM (32*1024)
+#define SCST_IO_SIZE_THRESHOLD_LARGE (128*1024)
+#define SCST_IO_SIZE_THRESHOLD_VERY_LARGE (512*1024)
+
+#define SCST_LATENCY_STAT_INDEX_SMALL 0
+#define SCST_LATENCY_STAT_INDEX_MEDIUM 1
+#define SCST_LATENCY_STAT_INDEX_LARGE 2
+#define SCST_LATENCY_STAT_INDEX_VERY_LARGE 3
+#define SCST_LATENCY_STAT_INDEX_OTHER 4
+#define SCST_LATENCY_STATS_NUM (SCST_LATENCY_STAT_INDEX_OTHER + 1)
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
+
+/*
+ * SCST session, analog of SCSI I_T nexus
+ */
+struct scst_session {
+ /*
+ * Initialization phase, one of SCST_SESS_IPH_* constants, protected by
+ * sess_list_lock
+ */
+ int init_phase;
+
+ struct scst_tgt *tgt; /* corresponding target */
+
+ /* Used for storage of target driver private stuff */
+ void *tgt_priv;
+
+ /* session's async flags */
+ unsigned long sess_aflags;
+
+ /*
+ * Hash list of tgt_dev's for this session, protected by scst_mutex
+ * and suspended activity
+ */
+ struct list_head sess_tgt_dev_list_hash[TGT_DEV_HASH_SIZE];
+
+ /*
+ * List of cmds in this session. Protected by sess_list_lock.
+ *
+ * We must always keep commands in the sess list from the
+ * very beginning, because otherwise they can be missed during
+ * TM processing.
+ */
+ struct list_head sess_cmd_list;
+
+ spinlock_t sess_list_lock; /* protects sess_cmd_list, etc */
+
+ atomic_t refcnt; /* get/put counter */
+
+ /*
+ * Alive commands for this session. ToDo: make it part of the common
+ * IO flow control.
+ */
+ atomic_t sess_cmd_count;
+
+ /* Access control for this session and list entry there */
+ struct scst_acg *acg;
+
+ /* Initiator port transport id */
+ uint8_t *transport_id;
+
+ /* List entry for the sessions list inside ACG */
+ struct list_head acg_sess_list_entry;
+
+ struct delayed_work hw_pending_work;
+
+ /* Name of attached initiator */
+ const char *initiator_name;
+
+ /* List entry of sessions per target */
+ struct list_head sess_list_entry;
+
+ /* List entry for the list that keeps session, waiting for the init */
+ struct list_head sess_init_list_entry;
+
+ /*
+ * List entry for the list that keeps session, waiting for the shutdown
+ */
+ struct list_head sess_shut_list_entry;
+
+ /*
+ * Lists of deferred during session initialization commands.
+ * Protected by sess_list_lock.
+ */
+ struct list_head init_deferred_cmd_list;
+ struct list_head init_deferred_mcmd_list;
+
+ /*
+ * Shutdown phase, one of SCST_SESS_SPH_* constants, unprotected.
+ * Async. relating to init_phase, must be a separate variable, because
+ * session could be unregistered before async. registration is finished.
+ */
+ unsigned long shut_phase;
+
+ /* Used if scst_unregister_session() called in wait mode */
+ struct completion *shutdown_compl;
+
+ /* sysfs release completion */
+ struct completion sess_kobj_release_cmpl;
+
+ unsigned int sess_kobj_ready:1;
+
+ struct kobject sess_kobj; /* kobject for this struct */
+
+ /*
+ * Functions and data for user callbacks from scst_register_session()
+ * and scst_unregister_session()
+ */
+ void *reg_sess_data;
+ void (*init_result_fn) (struct scst_session *sess, void *data,
+ int result);
+ void (*unreg_done_fn) (struct scst_session *sess);
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ /*
+ * Must be the last to allow to work with drivers who don't know
+ * about this config time option.
+ */
+ spinlock_t lat_lock;
+ uint64_t scst_time, tgt_time, dev_time;
+ unsigned int processed_cmds;
+ uint64_t min_scst_time, min_tgt_time, min_dev_time;
+ uint64_t max_scst_time, max_tgt_time, max_dev_time;
+ struct scst_ext_latency_stat sess_latency_stat[SCST_LATENCY_STATS_NUM];
+#endif
+};
+
+/*
+ * SCST_PR_ABORT_ALL TM function helper structure
+ */
+struct scst_pr_abort_all_pending_mgmt_cmds_counter {
+ /*
+ * How many there are pending for this cmd SCST_PR_ABORT_ALL TM
+ * commands.
+ */
+ atomic_t pr_abort_pending_cnt;
+
+ /* Saved completition routine */
+ void (*saved_cmd_done) (struct scst_cmd *cmd, int next_state,
+ enum scst_exec_context pref_context);
+
+ /*
+ * How many there are pending for this cmd SCST_PR_ABORT_ALL TM
+ * commands, which not yet aborted all affected commands and
+ * a completion to signal, when it's done.
+ */
+ atomic_t pr_aborting_cnt;
+ struct completion pr_aborting_cmpl;
+};
+
+/*
+ * Structure to control commands' queuing and threads pool processing the queue
+ */
+struct scst_cmd_threads {
+ spinlock_t cmd_list_lock;
+ struct list_head active_cmd_list; /* commands queue */
+ wait_queue_head_t cmd_list_waitQ;
+
+ struct io_context *io_context; /* IO context of the threads pool */
+
+ bool io_context_ready;
+
+ int nr_threads; /* number of processing threads */
+ struct list_head threads_list; /* processing threads */
+
+ struct list_head lists_list_entry;
+};
+
+/*
+ * SCST command, analog of I_T_L_Q nexus or task
+ */
+struct scst_cmd {
+ /* List entry for below *_cmd_threads */
+ struct list_head cmd_list_entry;
+
+ /* Pointer to lists of commands with the lock */
+ struct scst_cmd_threads *cmd_threads;
+
+ atomic_t cmd_ref;
+
+ struct scst_session *sess; /* corresponding session */
+
+ /* Cmd state, one of SCST_CMD_STATE_* constants */
+ int state;
+
+ /*************************************************************
+ ** Cmd's flags
+ *************************************************************/
+
+ /*
+ * Set if expected_sn should be incremented, i.e. cmd was sent
+ * for execution
+ */
+ unsigned int sent_for_exec:1;
+
+ /* Set if the cmd's action is completed */
+ unsigned int completed:1;
+
+ /* Set if we should ignore Unit Attention in scst_check_sense() */
+ unsigned int ua_ignore:1;
+
+ /* Set if cmd is being processed in atomic context */
+ unsigned int atomic:1;
+
+ /* Set if this command was sent in double UA possible state */
+ unsigned int double_ua_possible:1;
+
+ /* Set if this command contains status */
+ unsigned int is_send_status:1;
+
+ /* Set if cmd is being retried */
+ unsigned int retry:1;
+
+ /* Set if cmd is internally generated */
+ unsigned int internal:1;
+
+ /* Set if the device was blocked by scst_inc_on_dev_cmd() (for debug) */
+ unsigned int inc_blocking:1;
+
+ /* Set if the device should be unblocked after cmd's finish */
+ unsigned int needs_unblocking:1;
+
+ /* Set if scst_dec_on_dev_cmd() call is needed on the cmd's finish */
+ unsigned int dec_on_dev_needed:1;
+
+ /* Set if cmd is queued as hw pending */
+ unsigned int cmd_hw_pending:1;
+
+ /*
+ * Set if the target driver wants to alloc data buffers on its own.
+ * In this case alloc_data_buf() must be provided in the target driver
+ * template.
+ */
+ unsigned int tgt_need_alloc_data_buf:1;
+
+ /*
+ * Set by SCST if the custom data buffer allocation by the target driver
+ * succeeded.
+ */
+ unsigned int tgt_data_buf_alloced:1;
+
+ /* Set if custom data buffer allocated by dev handler */
+ unsigned int dh_data_buf_alloced:1;
+
+ /* Set if the target driver called scst_set_expected() */
+ unsigned int expected_values_set:1;
+
+ /*
+ * Set if the SG buffer was modified by scst_adjust_sg()
+ */
+ unsigned int sg_buff_modified:1;
+
+ /*
+ * Set if cmd buffer was vmallocated and copied from more
+ * then one sg chunk
+ */
+ unsigned int sg_buff_vmallocated:1;
+
+ /*
+ * Set if scst_cmd_init_stage1_done() called and the target
+ * want that preprocessing_done() will be called
+ */
+ unsigned int preprocessing_only:1;
+
+ /* Set if cmd's SN was set */
+ unsigned int sn_set:1;
+
+ /* Set if hq_cmd_count was incremented */
+ unsigned int hq_cmd_inced:1;
+
+ /*
+ * Set if scst_cmd_init_stage1_done() called and the target wants
+ * that the SN for the cmd won't be assigned until scst_restart_cmd()
+ */
+ unsigned int set_sn_on_restart_cmd:1;
+
+ /* Set if the cmd's must not use sgv cache for data buffer */
+ unsigned int no_sgv:1;
+
+ /*
+ * Set if target driver may need to call dma_sync_sg() or similar
+ * function before transferring cmd' data to the target device
+ * via DMA.
+ */
+ unsigned int may_need_dma_sync:1;
+
+ /* Set if the cmd was done or aborted out of its SN */
+ unsigned int out_of_sn:1;
+
+ /* Set if increment expected_sn in cmd->scst_cmd_done() */
+ unsigned int inc_expected_sn_on_done:1;
+
+ /* Set if tgt_sn field is valid */
+ unsigned int tgt_sn_set:1;
+
+ /* Set if any direction residual is possible */
+ unsigned int resid_possible:1;
+
+ /* Set if cmd is done */
+ unsigned int done:1;
+
+ /* Set if cmd is finished */
+ unsigned int finished:1;
+
+#ifdef CONFIG_SCST_DEBUG_TM
+ /* Set if the cmd was delayed by task management debugging code */
+ unsigned int tm_dbg_delayed:1;
+
+ /* Set if the cmd must be ignored by task management debugging code */
+ unsigned int tm_dbg_immut:1;
+#endif
+
+ /**************************************************************/
+
+ /* cmd's async flags */
+ unsigned long cmd_flags;
+
+ /* Keeps status of cmd's status/data delivery to remote initiator */
+ int delivery_status;
+
+ struct scst_tgt_template *tgtt; /* to save extra dereferences */
+ struct scst_tgt *tgt; /* to save extra dereferences */
+ struct scst_device *dev; /* to save extra dereferences */
+
+ struct scst_tgt_dev *tgt_dev; /* corresponding device for this cmd */
+
+ uint64_t lun; /* LUN for this cmd */
+
+ unsigned long start_time;
+
+ /* List entry for tgt_dev's SN related lists */
+ struct list_head sn_cmd_list_entry;
+
+ /* Cmd's serial number, used to execute cmd's in order of arrival */
+ unsigned int sn;
+
+ /* The corresponding sn_slot in tgt_dev->sn_slots */
+ atomic_t *sn_slot;
+
+ /* List entry for sess's sess_cmd_list */
+ struct list_head sess_cmd_list_entry;
+
+ /*
+ * Used to found the cmd by scst_find_cmd_by_tag(). Set by the
+ * target driver on the cmd's initialization time
+ */
+ uint64_t tag;
+
+ uint32_t tgt_sn; /* SN set by target driver (for TM purposes) */
+
+ /* CDB and its len */
+ uint8_t cdb[SCST_MAX_CDB_SIZE];
+ unsigned short cdb_len;
+ unsigned short ext_cdb_len;
+ uint8_t *ext_cdb;
+
+ enum scst_cdb_flags op_flags;
+ const char *op_name;
+
+ enum scst_cmd_queue_type queue_type;
+
+ int timeout; /* CDB execution timeout in seconds */
+ int retries; /* Amount of retries that will be done by SCSI mid-level */
+
+ /* SCSI data direction, one of SCST_DATA_* constants */
+ scst_data_direction data_direction;
+
+ /* Remote initiator supplied values, if any */
+ scst_data_direction expected_data_direction;
+ int expected_transfer_len;
+ int expected_out_transfer_len; /* for bidi writes */
+
+ /*
+ * Cmd data length. Could be different from bufflen for commands like
+ * VERIFY, which transfer different amount of data (if any), than
+ * processed.
+ */
+ int data_len;
+
+ /* Completition routine */
+ void (*scst_cmd_done) (struct scst_cmd *cmd, int next_state,
+ enum scst_exec_context pref_context);
+
+ struct sgv_pool_obj *sgv; /* sgv object */
+ int bufflen; /* cmd buffer length */
+ struct scatterlist *sg; /* cmd data buffer SG vector */
+ int sg_cnt; /* SG segments count */
+
+ /*
+ * Response data length in data buffer. Must not be set
+ * directly, use scst_set_resp_data_len() for that.
+ */
+ int resp_data_len;
+
+ /*
+ * Response data length adjusted on residual, i.e.
+ * min(expected_len, resp_len), if expected len set.
+ */
+ int adjusted_resp_data_len;
+
+ /*
+ * Data length to write, i.e. transfer from the initiator. Might be
+ * different from (out_)bufflen, if the initiator asked too big or too
+ * small expected(_out_)transfer_len.
+ */
+ int write_len;
+
+ /*
+ * Write sg and sg_cnt to point out either on sg/sg_cnt, or on
+ * out_sg/out_sg_cnt.
+ */
+ struct scatterlist **write_sg;
+ int *write_sg_cnt;
+
+ /* scst_get_sg_buf_[first,next]() support */
+ struct scatterlist *get_sg_buf_cur_sg_entry;
+ int get_sg_buf_entry_num;
+
+ /* Bidirectional transfers support */
+ int out_bufflen; /* WRITE buffer length */
+ struct sgv_pool_obj *out_sgv; /* WRITE sgv object */
+ struct scatterlist *out_sg; /* WRITE data buffer SG vector */
+ int out_sg_cnt; /* WRITE SG segments count */
+
+ /*
+ * Used if both target driver and dev handler request own memory
+ * allocation. In other cases, both are equal to sg and sg_cnt
+ * correspondingly.
+ *
+ * If target driver requests own memory allocations, it MUST use
+ * functions scst_cmd_get_tgt_sg*() to get sg and sg_cnt! Otherwise,
+ * it may use functions scst_cmd_get_sg*().
+ */
+ struct scatterlist *tgt_sg;
+ int tgt_sg_cnt;
+ struct scatterlist *tgt_out_sg; /* bidirectional */
+ int tgt_out_sg_cnt; /* bidirectional */
+
+ /*
+ * The status fields in case of errors must be set using
+ * scst_set_cmd_error_status()!
+ */
+ uint8_t status; /* status byte from target device */
+ uint8_t msg_status; /* return status from host adapter itself */
+ uint8_t host_status; /* set by low-level driver to indicate status */
+ uint8_t driver_status; /* set by mid-level */
+
+ uint8_t *sense; /* pointer to sense buffer */
+ unsigned short sense_valid_len; /* length of valid sense data */
+ unsigned short sense_buflen; /* length of the sense buffer, if any */
+
+ /* Start time when cmd was sent to rdy_to_xfer() or xmit_response() */
+ unsigned long hw_pending_start;
+
+ /* Used for storage of target driver private stuff */
+ void *tgt_priv;
+
+ /* Used for storage of dev handler private stuff */
+ void *dh_priv;
+
+ /* Used to restore sg if it was modified by scst_adjust_sg() */
+ struct scatterlist *orig_sg;
+ int *p_orig_sg_cnt;
+ int orig_sg_cnt, orig_sg_entry, orig_entry_len;
+
+ /* Used to retry commands in case of double UA */
+ int dbl_ua_orig_resp_data_len, dbl_ua_orig_data_direction;
+
+ /*
+ * List of the corresponding mgmt cmds, if any. Protected by
+ * sess_list_lock.
+ */
+ struct list_head mgmt_cmd_list;
+
+ /* List entry for dev's blocked_cmd_list */
+ struct list_head blocked_cmd_list_entry;
+
+ /* Counter of the corresponding SCST_PR_ABORT_ALL TM commands */
+ struct scst_pr_abort_all_pending_mgmt_cmds_counter *pr_abort_counter;
+
+ struct scst_cmd *orig_cmd; /* Used to issue REQUEST SENSE */
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ /*
+ * Must be the last to allow to work with drivers who don't know
+ * about this config time option.
+ */
+ uint64_t start, curr_start, parse_time, alloc_buf_time;
+ uint64_t restart_waiting_time, rdy_to_xfer_time;
+ uint64_t pre_exec_time, exec_time, dev_done_time;
+ uint64_t xmit_time, tgt_on_free_time, dev_on_free_time;
+#endif
+};
+
+/*
+ * Parameters for SCST management commands
+ */
+struct scst_rx_mgmt_params {
+ int fn;
+ uint64_t tag;
+ const uint8_t *lun;
+ int lun_len;
+ uint32_t cmd_sn;
+ int atomic;
+ void *tgt_priv;
+ unsigned char tag_set;
+ unsigned char lun_set;
+ unsigned char cmd_sn_set;
+};
+
+/*
+ * A stub structure to link an management command and affected regular commands
+ */
+struct scst_mgmt_cmd_stub {
+ struct scst_mgmt_cmd *mcmd;
+
+ /* List entry in cmd->mgmt_cmd_list */
+ struct list_head cmd_mgmt_cmd_list_entry;
+
+ /* Set if the cmd was counted in mcmd->cmd_done_wait_count */
+ unsigned int done_counted:1;
+
+ /* Set if the cmd was counted in mcmd->cmd_finish_wait_count */
+ unsigned int finish_counted:1;
+};
+
+/*
+ * SCST task management structure
+ */
+struct scst_mgmt_cmd {
+ /* List entry for *_mgmt_cmd_list */
+ struct list_head mgmt_cmd_list_entry;
+
+ struct scst_session *sess;
+
+ /* Mgmt cmd state, one of SCST_MCMD_STATE_* constants */
+ int state;
+
+ int fn; /* task management function */
+
+ /* Set if device(s) should be unblocked after mcmd's finish */
+ unsigned int needs_unblocking:1;
+ unsigned int lun_set:1; /* set, if lun field is valid */
+ unsigned int cmd_sn_set:1; /* set, if cmd_sn field is valid */
+
+ /*
+ * Number of commands to finish before sending response,
+ * protected by scst_mcmd_lock
+ */
+ int cmd_finish_wait_count;
+
+ /*
+ * Number of commands to complete (done) before resetting reservation,
+ * protected by scst_mcmd_lock
+ */
+ int cmd_done_wait_count;
+
+ /* Number of completed commands, protected by scst_mcmd_lock */
+ int completed_cmd_count;
+
+ uint64_t lun; /* LUN for this mgmt cmd */
+ /* or (and for iSCSI) */
+ uint64_t tag; /* tag of the corresponding cmd */
+
+ uint32_t cmd_sn; /* affected command's highest SN */
+
+ /* corresponding cmd (to be aborted, found by tag) */
+ struct scst_cmd *cmd_to_abort;
+
+ /* corresponding device for this mgmt cmd (found by lun) */
+ struct scst_tgt_dev *mcmd_tgt_dev;
+
+ /* completition status, one of the SCST_MGMT_STATUS_* constants */
+ int status;
+
+ /* Used for storage of target driver private stuff or origin PR cmd */
+ union {
+ void *tgt_priv;
+ struct scst_cmd *origin_pr_cmd;
+ };
+};
+
+/*
+ * Persistent reservations registrant
+ */
+struct scst_dev_registrant {
+ uint8_t *transport_id;
+ uint16_t rel_tgt_id;
+ __be64 key;
+
+ /* tgt_dev (I_T nexus) for this registrant, if any */
+ struct scst_tgt_dev *tgt_dev;
+
+ /* List entry for dev_registrants_list */
+ struct list_head dev_registrants_list_entry;
+
+ /* 2 auxiliary fields used to rollback changes for errors, etc. */
+ struct list_head aux_list_entry;
+ __be64 rollback_key;
+};
+
+/*
+ * SCST device
+ */
+struct scst_device {
+ unsigned short type; /* SCSI type of the device */
+
+ /*************************************************************
+ ** Dev's flags. Updates serialized by dev_lock or suspended
+ ** activity
+ *************************************************************/
+
+ /* Set if dev is RESERVED */
+ unsigned short dev_reserved:1;
+
+ /* Set if double reset UA is possible */
+ unsigned short dev_double_ua_possible:1;
+
+ /* If set, dev is read only */
+ unsigned short rd_only:1;
+
+ /**************************************************************/
+
+ /*************************************************************
+ ** Dev's control mode page related values. Updates serialized
+ ** by scst_block_dev(). Modified independently to the above and
+ ** below fields, hence the alignment.
+ *************************************************************/
+
+ unsigned int queue_alg:4 __attribute__((aligned(sizeof(long))));
+ unsigned int tst:3;
+ unsigned int tas:1;
+ unsigned int swp:1;
+ unsigned int d_sense:1;
+
+ /*
+ * Set if device implements own ordered commands management. If not set
+ * and queue_alg is SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER,
+ * expected_sn will be incremented only after commands finished.
+ */
+ unsigned int has_own_order_mgmt:1;
+
+ /**************************************************************/
+
+ /*
+ * Set if dev is persistently reserved. Protected by dev_pr_mutex.
+ * Modified independently to the above field, hence the alignment.
+ */
+ unsigned int pr_is_set:1 __attribute__((aligned(sizeof(long))));
+
+ /*
+ * Set if there is a thread changing or going to change PR state(s).
+ * Protected by dev_pr_mutex.
+ */
+ unsigned int pr_writer_active:1;
+
+ /*
+ * How many threads are checking commands for PR allowance. Used to
+ * implement lockless read-only fast path.
+ */
+ atomic_t pr_readers_count;
+
+ struct scst_dev_type *handler; /* corresponding dev handler */
+
+ /* Used for storage of dev handler private stuff */
+ void *dh_priv;
+
+ /* Corresponding real SCSI device, could be NULL for virtual devices */
+ struct scsi_device *scsi_dev;
+
+ /* List of commands with lock, if dedicated threads are used */
+ struct scst_cmd_threads dev_cmd_threads;
+
+ /* Memory limits for this device */
+ struct scst_mem_lim dev_mem_lim;
+
+ /* How many cmds alive on this dev */
+ atomic_t dev_cmd_count;
+
+ /*
+ * How many there are "on_dev" commands, i.e. ones those are being
+ * executed by the underlying SCSI/virtual device.
+ */
+ atomic_t on_dev_count;
+
+ /*
+ * How many times device was blocked for new cmds execution.
+ * Protected by dev_lock
+ */
+ int block_count;
+
+ /* How many write cmds alive on this dev. Temporary, ToDo */
+ atomic_t write_cmd_count;
+
+ /*************************************************************
+ ** Persistent reservation fields. Protected by dev_pr_mutex.
+ *************************************************************/
+
+ /*
+ * True if persist through power loss is activated. Modified
+ * independently to the above field, hence the alignment.
+ */
+ unsigned short pr_aptpl:1 __attribute__((aligned(sizeof(long))));
+
+ /* Persistent reservation type */
+ uint8_t pr_type;
+
+ /* Persistent reservation scope */
+ uint8_t pr_scope;
+
+ /* Mutex to protect PR operations */
+ struct mutex dev_pr_mutex;
+
+ /* Persistent reservation generation value */
+ uint32_t pr_generation;
+
+ /* Reference to registrant - persistent reservation holder */
+ struct scst_dev_registrant *pr_holder;
+
+ /* List of dev's registrants */
+ struct list_head dev_registrants_list;
+
+ /*
+ * Count of connected tgt_devs from transports, which don't support
+ * PRs, i.e. don't have get_initiator_port_transport_id(). Protected
+ * by scst_mutex.
+ */
+ int not_pr_supporting_tgt_devs_num;
+
+ /* Persist through power loss files */
+ char *pr_file_name;
+ char *pr_file_name1;
+
+ /**************************************************************/
+
+ spinlock_t dev_lock; /* device lock */
+
+ struct list_head blocked_cmd_list; /* protected by dev_lock */
+
+ /* Used to wait for requested amount of "on_dev" commands */
+ wait_queue_head_t on_dev_waitQ;
+
+ /* A list entry used during TM, protected by scst_mutex */
+ struct list_head tm_dev_list_entry;
+
+ /* Virtual device internal ID */
+ int virt_id;
+
+ /* Pointer to virtual device name, for convenience only */
+ char *virt_name;
+
+ /* List entry in global devices list */
+ struct list_head dev_list_entry;
+
+ /*
+ * List of tgt_dev's, one per session, protected by scst_mutex or
+ * dev_lock for reads and both for writes
+ */
+ struct list_head dev_tgt_dev_list;
+
+ /* List of acg_dev's, one per acg, protected by scst_mutex */
+ struct list_head dev_acg_dev_list;
+
+ /* Number of threads in the device's threads pools */
+ int threads_num;
+
+ /* Threads pool type of the device. Valid only if threads_num > 0. */
+ enum scst_dev_type_threads_pool_type threads_pool_type;
+
+ /* sysfs release completion */
+ struct completion dev_kobj_release_cmpl;
+
+ struct kobject dev_kobj; /* kobject for this struct */
+ struct kobject *dev_exp_kobj; /* exported groups */
+
+ /* Export number in the dev's sysfs list. Protected by scst_mutex */
+ int dev_exported_lun_num;
+};
+
+/*
+ * Used to store threads local tgt_dev specific data
+ */
+struct scst_thr_data_hdr {
+ /* List entry in tgt_dev->thr_data_list */
+ struct list_head thr_data_list_entry;
+ struct task_struct *owner_thr; /* the owner thread */
+ atomic_t ref;
+ /* Function that will be called on the tgt_dev destruction */
+ void (*free_fn) (struct scst_thr_data_hdr *data);
+};
+
+/*
+ * Used to clearly dispose async io_context
+ */
+struct scst_async_io_context_keeper {
+ struct kref aic_keeper_kref;
+ bool aic_ready;
+ struct io_context *aic;
+ struct task_struct *aic_keeper_thr;
+ wait_queue_head_t aic_keeper_waitQ;
+};
+
+/*
+ * Used to store per-session specific device information, analog of
+ * SCSI I_T_L nexus.
+ */
+struct scst_tgt_dev {
+ /* List entry in sess->sess_tgt_dev_list_hash */
+ struct list_head sess_tgt_dev_list_entry;
+
+ struct scst_device *dev; /* to save extra dereferences */
+ uint64_t lun; /* to save extra dereferences */
+
+ gfp_t gfp_mask;
+ struct sgv_pool *pool;
+ int max_sg_cnt;
+
+ /*
+ * Tgt_dev's async flags. Modified independently to the neighbour
+ * fields.
+ */
+ unsigned long tgt_dev_flags;
+
+ /* Used for storage of dev handler private stuff */
+ void *dh_priv;
+
+ /* How many cmds alive on this dev in this session */
+ atomic_t tgt_dev_cmd_count;
+
+ /*
+ * Used to execute cmd's in order of arrival, honoring SCSI task
+ * attributes.
+ *
+ * Protected by sn_lock, except expected_sn, which is protected by
+ * itself. Curr_sn must have the same size as expected_sn to
+ * overflow simultaneously.
+ */
+ int def_cmd_count;
+ spinlock_t sn_lock;
+ unsigned int expected_sn;
+ unsigned int curr_sn;
+ int hq_cmd_count;
+ struct list_head deferred_cmd_list;
+ struct list_head skipped_sn_list;
+
+ /*
+ * Set if the prev cmd was ORDERED. Size and, hence, alignment must
+ * allow unprotected modifications independently to the neighbour fields.
+ */
+ unsigned long prev_cmd_ordered;
+
+ int num_free_sn_slots; /* if it's <0, then all slots are busy */
+ atomic_t *cur_sn_slot;
+ atomic_t sn_slots[15];
+
+ /* List of scst_thr_data_hdr and lock */
+ spinlock_t thr_data_lock;
+ struct list_head thr_data_list;
+
+ /* Pointer to lists of commands with the lock */
+ struct scst_cmd_threads *active_cmd_threads;
+
+ /* Union to save some CPU cache footprint */
+ union {
+ struct {
+ /* Copy to save fast path dereference */
+ struct io_context *async_io_context;
+
+ struct scst_async_io_context_keeper *aic_keeper;
+ };
+
+ /* Lists of commands with lock, if dedicated threads are used */
+ struct scst_cmd_threads tgt_dev_cmd_threads;
+ };
+
+ spinlock_t tgt_dev_lock; /* per-session device lock */
+
+ /* List of UA's for this device, protected by tgt_dev_lock */
+ struct list_head UA_list;
+
+ struct scst_session *sess; /* corresponding session */
+ struct scst_acg_dev *acg_dev; /* corresponding acg_dev */
+
+ /* Reference to registrant to find quicker */
+ struct scst_dev_registrant *registrant;
+
+ /* List entry in dev->dev_tgt_dev_list */
+ struct list_head dev_tgt_dev_list_entry;
+
+ /* Internal tmp list entry */
+ struct list_head extra_tgt_dev_list_entry;
+
+ /* Set if INQUIRY DATA HAS CHANGED UA is needed */
+ unsigned int inq_changed_ua_needed:1;
+
+ /*
+ * Stored Unit Attention sense and its length for possible
+ * subsequent REQUEST SENSE. Both protected by tgt_dev_lock.
+ */
+ unsigned short tgt_dev_valid_sense_len;
+ uint8_t tgt_dev_sense[SCST_SENSE_BUFFERSIZE];
+
+ /* sysfs release completion */
+ struct completion tgt_dev_kobj_release_cmpl;
+
+ struct kobject tgt_dev_kobj; /* kobject for this struct */
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ /*
+ * Must be the last to allow to work with drivers who don't know
+ * about this config time option.
+ *
+ * Protected by sess->lat_lock.
+ */
+ uint64_t scst_time, tgt_time, dev_time;
+ unsigned int processed_cmds;
+ struct scst_ext_latency_stat dev_latency_stat[SCST_LATENCY_STATS_NUM];
+#endif
+};
+
+/*
+ * Used to store ACG-specific device information, like LUN
+ */
+struct scst_acg_dev {
+ struct scst_device *dev; /* corresponding device */
+
+ uint64_t lun; /* device's LUN in this acg */
+
+ /* If set, the corresponding LU is read only */
+ unsigned int rd_only:1;
+
+ struct scst_acg *acg; /* parent acg */
+
+ /* List entry in dev->dev_acg_dev_list */
+ struct list_head dev_acg_dev_list_entry;
+
+ /* List entry in acg->acg_dev_list */
+ struct list_head acg_dev_list_entry;
+
+ /* kobject for this structure */
+ struct kobject acg_dev_kobj;
+
+ /* sysfs release completion */
+ struct completion acg_dev_kobj_release_cmpl;
+
+ /* Name of the link to the corresponding LUN */
+ char acg_dev_link_name[20];
+};
+
+/*
+ * ACG - access control group. Used to store group related
+ * control information.
+ */
+struct scst_acg {
+ /* Owner target */
+ struct scst_tgt *tgt;
+
+ /* List of acg_dev's in this acg, protected by scst_mutex */
+ struct list_head acg_dev_list;
+
+ /* List of attached sessions, protected by scst_mutex */
+ struct list_head acg_sess_list;
+
+ /* List of attached acn's, protected by scst_mutex */
+ struct list_head acn_list;
+
+ /* List entry in acg_lists */
+ struct list_head acg_list_entry;
+
+ /* Name of this acg */
+ const char *acg_name;
+
+ /* Type of I/O initiators groupping */
+ int acg_io_grouping_type;
+
+ unsigned int tgt_acg:1;
+
+ /* sysfs release completion */
+ struct completion acg_kobj_release_cmpl;
+
+ /* kobject for this structure */
+ struct kobject acg_kobj;
+
+ struct kobject *luns_kobj;
+ struct kobject *initiators_kobj;
+
+ unsigned int addr_method;
+};
+
+/*
+ * ACN - access control name. Used to store names, by which
+ * incoming sessions will be assigned to appropriate ACG.
+ */
+struct scst_acn {
+ struct scst_acg *acg; /* owner ACG */
+
+ const char *name; /* initiator's name */
+
+ /* List entry in acg->acn_list */
+ struct list_head acn_list_entry;
+
+ /* sysfs file attributes */
+ struct kobj_attribute *acn_attr;
+};
+
+/*
+ * Used to store per-session UNIT ATTENTIONs
+ */
+struct scst_tgt_dev_UA {
+ /* List entry in tgt_dev->UA_list */
+ struct list_head UA_list_entry;
+
+ /* Set if UA is global for session */
+ unsigned short global_UA:1;
+
+ /* Unit Attention valid sense len */
+ unsigned short UA_valid_sense_len;
+ /* Unit Attention sense buf */
+ uint8_t UA_sense_buffer[SCST_SENSE_BUFFERSIZE];
+};
+
+/* Used to deliver AENs */
+struct scst_aen {
+ int event_fn; /* AEN fn */
+
+ struct scst_session *sess; /* corresponding session */
+ __be64 lun; /* corresponding LUN in SCSI form */
+
+ union {
+ /* SCSI AEN data */
+ struct {
+ int aen_sense_len;
+ uint8_t aen_sense[SCST_STANDARD_SENSE_LEN];
+ };
+ };
+
+ /* Keeps status of AEN's delivery to remote initiator */
+ int delivery_status;
+};
+
+#ifndef smp_mb__after_set_bit
+/* There is no smp_mb__after_set_bit() in the kernel */
+#define smp_mb__after_set_bit() smp_mb()
+#endif
+
+/*
+ * Registers target template.
+ * Returns 0 on success or appropriate error code otherwise.
+ */
+int __scst_register_target_template(struct scst_tgt_template *vtt,
+ const char *version);
+static inline int scst_register_target_template(struct scst_tgt_template *vtt)
+{
+ return __scst_register_target_template(vtt, SCST_INTERFACE_VERSION);
+}
+
+/*
+ * Registers target template, non-GPL version.
+ * Returns 0 on success or appropriate error code otherwise.
+ *
+ * Note: *vtt must be static!
+ */
+int __scst_register_target_template_non_gpl(struct scst_tgt_template *vtt,
+ const char *version);
+static inline int scst_register_target_template_non_gpl(
+ struct scst_tgt_template *vtt)
+{
+ return __scst_register_target_template_non_gpl(vtt,
+ SCST_INTERFACE_VERSION);
+}
+
+void scst_unregister_target_template(struct scst_tgt_template *vtt);
+
+struct scst_tgt *scst_register_target(struct scst_tgt_template *vtt,
+ const char *target_name);
+void scst_unregister_target(struct scst_tgt *tgt);
+
+struct scst_session *scst_register_session(struct scst_tgt *tgt, int atomic,
+ const char *initiator_name, void *tgt_priv, void *result_fn_data,
+ void (*result_fn) (struct scst_session *sess, void *data, int result));
+struct scst_session *scst_register_session_non_gpl(struct scst_tgt *tgt,
+ const char *initiator_name, void *tgt_priv);
+void scst_unregister_session(struct scst_session *sess, int wait,
+ void (*unreg_done_fn) (struct scst_session *sess));
+void scst_unregister_session_non_gpl(struct scst_session *sess);
+
+int __scst_register_dev_driver(struct scst_dev_type *dev_type,
+ const char *version);
+static inline int scst_register_dev_driver(struct scst_dev_type *dev_type)
+{
+ return __scst_register_dev_driver(dev_type, SCST_INTERFACE_VERSION);
+}
+void scst_unregister_dev_driver(struct scst_dev_type *dev_type);
+
+int __scst_register_virtual_dev_driver(struct scst_dev_type *dev_type,
+ const char *version);
+/*
+ * Registers dev handler driver for virtual devices (eg VDISK).
+ * Returns 0 on success or appropriate error code otherwise.
+ */
+static inline int scst_register_virtual_dev_driver(
+ struct scst_dev_type *dev_type)
+{
+ return __scst_register_virtual_dev_driver(dev_type,
+ SCST_INTERFACE_VERSION);
+}
+
+void scst_unregister_virtual_dev_driver(struct scst_dev_type *dev_type);
+
+bool scst_initiator_has_luns(struct scst_tgt *tgt, const char *initiator_name);
+
+struct scst_cmd *scst_rx_cmd(struct scst_session *sess,
+ const uint8_t *lun, int lun_len, const uint8_t *cdb,
+ unsigned int cdb_len, int atomic);
+void scst_cmd_init_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context);
+
+/*
+ * Notifies SCST that the driver finished the first stage of the command
+ * initialization, and the command is ready for execution, but after
+ * SCST done the command's preprocessing preprocessing_done() function
+ * should be called. The second argument sets preferred command execition
+ * context. See SCST_CONTEXT_* constants for details.
+ *
+ * See comment for scst_cmd_init_done() for the serialization requirements.
+ */
+static inline void scst_cmd_init_stage1_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context, int set_sn)
+{
+ cmd->preprocessing_only = 1;
+ cmd->set_sn_on_restart_cmd = !set_sn;
+ scst_cmd_init_done(cmd, pref_context);
+}
+
+void scst_restart_cmd(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context);
+
+void scst_rx_data(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context);
+
+void scst_tgt_cmd_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context);
+
+int scst_rx_mgmt_fn(struct scst_session *sess,
+ const struct scst_rx_mgmt_params *params);
+
+/*
+ * Creates new management command using tag and sends it for execution.
+ * Can be used for SCST_ABORT_TASK only.
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same sess. Returns 0 for success, error code otherwise.
+ *
+ * Obsolete in favor of scst_rx_mgmt_fn()
+ */
+static inline int scst_rx_mgmt_fn_tag(struct scst_session *sess, int fn,
+ uint64_t tag, int atomic, void *tgt_priv)
+{
+ struct scst_rx_mgmt_params params;
+
+ BUG_ON(fn != SCST_ABORT_TASK);
+
+ memset(¶ms, 0, sizeof(params));
+ params.fn = fn;
+ params.tag = tag;
+ params.tag_set = 1;
+ params.atomic = atomic;
+ params.tgt_priv = tgt_priv;
+ return scst_rx_mgmt_fn(sess, ¶ms);
+}
+
+/*
+ * Creates new management command using LUN and sends it for execution.
+ * Currently can be used for any fn, except SCST_ABORT_TASK.
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same sess. Returns 0 for success, error code otherwise.
+ *
+ * Obsolete in favor of scst_rx_mgmt_fn()
+ */
+static inline int scst_rx_mgmt_fn_lun(struct scst_session *sess, int fn,
+ const uint8_t *lun, int lun_len, int atomic, void *tgt_priv)
+{
+ struct scst_rx_mgmt_params params;
+
+ BUG_ON(fn == SCST_ABORT_TASK);
+
+ memset(¶ms, 0, sizeof(params));
+ params.fn = fn;
+ params.lun = lun;
+ params.lun_len = lun_len;
+ params.lun_set = 1;
+ params.atomic = atomic;
+ params.tgt_priv = tgt_priv;
+ return scst_rx_mgmt_fn(sess, ¶ms);
+}
+
+int scst_get_cdb_info(struct scst_cmd *cmd);
+
+int scst_set_cmd_error_status(struct scst_cmd *cmd, int status);
+int scst_set_cmd_error(struct scst_cmd *cmd, int key, int asc, int ascq);
+void scst_set_busy(struct scst_cmd *cmd);
+
+void scst_check_convert_sense(struct scst_cmd *cmd);
+
+void scst_set_initial_UA(struct scst_session *sess, int key, int asc, int ascq);
+
+void scst_capacity_data_changed(struct scst_device *dev);
+
+struct scst_cmd *scst_find_cmd_by_tag(struct scst_session *sess, uint64_t tag);
+struct scst_cmd *scst_find_cmd(struct scst_session *sess, void *data,
+ int (*cmp_fn) (struct scst_cmd *cmd,
+ void *data));
+
+enum dma_data_direction scst_to_dma_dir(int scst_dir);
+enum dma_data_direction scst_to_tgt_dma_dir(int scst_dir);
+
+/*
+ * Returns true, if cmd's CDB is fully locally handled by SCST and false
+ * otherwise. Dev handlers parse() and dev_done() not called for such commands.
+ */
+static inline bool scst_is_cmd_fully_local(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_FULLY_LOCAL_CMD) != 0;
+}
+
+/*
+ * Returns true, if cmd's CDB is locally handled by SCST and
+ * false otherwise.
+ */
+static inline bool scst_is_cmd_local(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_LOCAL_CMD) != 0;
+}
+
+/* Returns true, if cmd can deliver UA */
+static inline bool scst_is_ua_command(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_SKIP_UA) == 0;
+}
+
+int scst_register_virtual_device(struct scst_dev_type *dev_handler,
+ const char *dev_name);
+void scst_unregister_virtual_device(int id);
+
+/*
+ * Get/Set functions for tgt's sg_tablesize
+ */
+static inline int scst_tgt_get_sg_tablesize(struct scst_tgt *tgt)
+{
+ return tgt->sg_tablesize;
+}
+
+static inline void scst_tgt_set_sg_tablesize(struct scst_tgt *tgt, int val)
+{
+ tgt->sg_tablesize = val;
+}
+
+/*
+ * Get/Set functions for tgt's target private data
+ */
+static inline void *scst_tgt_get_tgt_priv(struct scst_tgt *tgt)
+{
+ return tgt->tgt_priv;
+}
+
+static inline void scst_tgt_set_tgt_priv(struct scst_tgt *tgt, void *val)
+{
+ tgt->tgt_priv = val;
+}
+
+void scst_update_hw_pending_start(struct scst_cmd *cmd);
+
+/*
+ * Get/Set functions for session's target private data
+ */
+static inline void *scst_sess_get_tgt_priv(struct scst_session *sess)
+{
+ return sess->tgt_priv;
+}
+
+static inline void scst_sess_set_tgt_priv(struct scst_session *sess,
+ void *val)
+{
+ sess->tgt_priv = val;
+}
+
+/**
+ * Returns TRUE if cmd is being executed in atomic context.
+ *
+ * Note: checkpatch will complain on the use of in_atomic() below. You can
+ * safely ignore this warning since in_atomic() is used here only for debugging
+ * purposes.
+ */
+static inline bool scst_cmd_atomic(struct scst_cmd *cmd)
+{
+ int res = cmd->atomic;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely((in_atomic() || in_interrupt() || irqs_disabled()) &&
+ !res)) {
+ printk(KERN_ERR "ERROR: atomic context and non-atomic cmd\n");
+ dump_stack();
+ cmd->atomic = 1;
+ res = 1;
+ }
+#endif
+ return res;
+}
+
+/*
+ * Returns TRUE if cmd has been preliminary completed, i.e. completed or
+ * aborted.
+ */
+static inline bool scst_cmd_prelim_completed(struct scst_cmd *cmd)
+{
+ return cmd->completed || test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags);
+}
+
+static inline enum scst_exec_context __scst_estimate_context(bool direct)
+{
+ if (in_irq())
+ return SCST_CONTEXT_TASKLET;
+ else if (irqs_disabled())
+ return SCST_CONTEXT_THREAD;
+ else
+ return direct ? SCST_CONTEXT_DIRECT :
+ SCST_CONTEXT_DIRECT_ATOMIC;
+}
+
+static inline enum scst_exec_context scst_estimate_context(void)
+{
+ return __scst_estimate_context(0);
+}
+
+static inline enum scst_exec_context scst_estimate_context_direct(void)
+{
+ return __scst_estimate_context(1);
+}
+
+/* Returns cmd's CDB */
+static inline const uint8_t *scst_cmd_get_cdb(struct scst_cmd *cmd)
+{
+ return cmd->cdb;
+}
+
+/* Returns cmd's CDB length */
+static inline unsigned int scst_cmd_get_cdb_len(struct scst_cmd *cmd)
+{
+ return cmd->cdb_len;
+}
+
+/* Returns cmd's extended CDB */
+static inline const uint8_t *scst_cmd_get_ext_cdb(struct scst_cmd *cmd)
+{
+ return cmd->ext_cdb;
+}
+
+/* Returns cmd's extended CDB length */
+static inline unsigned int scst_cmd_get_ext_cdb_len(struct scst_cmd *cmd)
+{
+ return cmd->ext_cdb_len;
+}
+
+/* Sets cmd's extended CDB and its length */
+static inline void scst_cmd_set_ext_cdb(struct scst_cmd *cmd,
+ uint8_t *ext_cdb, unsigned int ext_cdb_len)
+{
+ cmd->ext_cdb = ext_cdb;
+ cmd->ext_cdb_len = ext_cdb_len;
+}
+
+/* Returns cmd's session */
+static inline struct scst_session *scst_cmd_get_session(struct scst_cmd *cmd)
+{
+ return cmd->sess;
+}
+
+/* Returns cmd's response data length */
+static inline int scst_cmd_get_resp_data_len(struct scst_cmd *cmd)
+{
+ return cmd->resp_data_len;
+}
+
+/* Returns cmd's adjusted response data length */
+static inline int scst_cmd_get_adjusted_resp_data_len(struct scst_cmd *cmd)
+{
+ return cmd->adjusted_resp_data_len;
+}
+
+/* Returns if status should be sent for cmd */
+static inline int scst_cmd_get_is_send_status(struct scst_cmd *cmd)
+{
+ return cmd->is_send_status;
+}
+
+/*
+ * Returns pointer to cmd's SG data buffer.
+ *
+ * Usage of this function is not recommended, use scst_get_buf_*()
+ * family of functions instead.
+ */
+static inline struct scatterlist *scst_cmd_get_sg(struct scst_cmd *cmd)
+{
+ return cmd->sg;
+}
+
+/*
+ * Returns cmd's sg_cnt.
+ *
+ * Usage of this function is not recommended, use scst_get_buf_*()
+ * family of functions instead.
+ */
+static inline int scst_cmd_get_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->sg_cnt;
+}
+
+/*
+ * Returns cmd's data buffer length.
+ *
+ * In case if you need to iterate over data in the buffer, usage of
+ * this function is not recommended, use scst_get_buf_*()
+ * family of functions instead.
+ */
+static inline unsigned int scst_cmd_get_bufflen(struct scst_cmd *cmd)
+{
+ return cmd->bufflen;
+}
+
+/*
+ * Returns pointer to cmd's bidirectional in (WRITE) SG data buffer.
+ *
+ * Usage of this function is not recommended, use scst_get_out_buf_*()
+ * family of functions instead.
+ */
+static inline struct scatterlist *scst_cmd_get_out_sg(struct scst_cmd *cmd)
+{
+ return cmd->out_sg;
+}
+
+/*
+ * Returns cmd's bidirectional in (WRITE) sg_cnt.
+ *
+ * Usage of this function is not recommended, use scst_get_out_buf_*()
+ * family of functions instead.
+ */
+static inline int scst_cmd_get_out_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->out_sg_cnt;
+}
+
+void scst_restore_sg_buff(struct scst_cmd *cmd);
+
+/* Restores modified sg buffer in the original state, if necessary */
+static inline void scst_check_restore_sg_buff(struct scst_cmd *cmd)
+{
+ if (unlikely(cmd->sg_buff_modified))
+ scst_restore_sg_buff(cmd);
+}
+
+/*
+ * Returns cmd's bidirectional in (WRITE) data buffer length.
+ *
+ * In case if you need to iterate over data in the buffer, usage of
+ * this function is not recommended, use scst_get_out_buf_*()
+ * family of functions instead.
+ */
+static inline unsigned int scst_cmd_get_out_bufflen(struct scst_cmd *cmd)
+{
+ return cmd->out_bufflen;
+}
+
+/* Returns pointer to cmd's target's SG data buffer */
+static inline struct scatterlist *scst_cmd_get_tgt_sg(struct scst_cmd *cmd)
+{
+ return cmd->tgt_sg;
+}
+
+/* Returns cmd's target's sg_cnt */
+static inline int scst_cmd_get_tgt_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->tgt_sg_cnt;
+}
+
+/* Sets cmd's target's SG data buffer */
+static inline void scst_cmd_set_tgt_sg(struct scst_cmd *cmd,
+ struct scatterlist *sg, int sg_cnt)
+{
+ cmd->tgt_sg = sg;
+ cmd->tgt_sg_cnt = sg_cnt;
+ cmd->tgt_data_buf_alloced = 1;
+}
+
+/* Returns pointer to cmd's target's OUT SG data buffer */
+static inline struct scatterlist *scst_cmd_get_out_tgt_sg(struct scst_cmd *cmd)
+{
+ return cmd->tgt_out_sg;
+}
+
+/* Returns cmd's target's OUT sg_cnt */
+static inline int scst_cmd_get_tgt_out_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->tgt_out_sg_cnt;
+}
+
+/* Sets cmd's target's OUT SG data buffer */
+static inline void scst_cmd_set_tgt_out_sg(struct scst_cmd *cmd,
+ struct scatterlist *sg, int sg_cnt)
+{
+ WARN_ON(!cmd->tgt_data_buf_alloced);
+
+ cmd->tgt_out_sg = sg;
+ cmd->tgt_out_sg_cnt = sg_cnt;
+}
+
+/* Returns cmd's data direction */
+static inline scst_data_direction scst_cmd_get_data_direction(
+ struct scst_cmd *cmd)
+{
+ return cmd->data_direction;
+}
+
+/* Returns cmd's write len as well as write SG and sg_cnt */
+static inline int scst_cmd_get_write_fields(struct scst_cmd *cmd,
+ struct scatterlist **sg, int *sg_cnt)
+{
+ *sg = *cmd->write_sg;
+ *sg_cnt = *cmd->write_sg_cnt;
+ return cmd->write_len;
+}
+
+void scst_cmd_set_write_not_received_data_len(struct scst_cmd *cmd,
+ int not_received);
+
+bool __scst_get_resid(struct scst_cmd *cmd, int *resid, int *bidi_out_resid);
+
+/*
+ * Returns true if cmd has residual(s) and returns them in the corresponding
+ * parameters(s).
+ */
+static inline bool scst_get_resid(struct scst_cmd *cmd,
+ int *resid, int *bidi_out_resid)
+{
+ if (likely(!cmd->resid_possible))
+ return false;
+ return __scst_get_resid(cmd, resid, bidi_out_resid);
+}
+
+/* Returns cmd's status byte from host device */
+static inline uint8_t scst_cmd_get_status(struct scst_cmd *cmd)
+{
+ return cmd->status;
+}
+
+/* Returns cmd's status from host adapter itself */
+static inline uint8_t scst_cmd_get_msg_status(struct scst_cmd *cmd)
+{
+ return cmd->msg_status;
+}
+
+/* Returns cmd's status set by low-level driver to indicate its status */
+static inline uint8_t scst_cmd_get_host_status(struct scst_cmd *cmd)
+{
+ return cmd->host_status;
+}
+
+/* Returns cmd's status set by SCSI mid-level */
+static inline uint8_t scst_cmd_get_driver_status(struct scst_cmd *cmd)
+{
+ return cmd->driver_status;
+}
+
+/* Returns pointer to cmd's sense buffer */
+static inline uint8_t *scst_cmd_get_sense_buffer(struct scst_cmd *cmd)
+{
+ return cmd->sense;
+}
+
+/* Returns cmd's valid sense length */
+static inline int scst_cmd_get_sense_buffer_len(struct scst_cmd *cmd)
+{
+ return cmd->sense_valid_len;
+}
+
+/*
+ * Get/Set functions for cmd's queue_type
+ */
+static inline enum scst_cmd_queue_type scst_cmd_get_queue_type(
+ struct scst_cmd *cmd)
+{
+ return cmd->queue_type;
+}
+
+static inline void scst_cmd_set_queue_type(struct scst_cmd *cmd,
+ enum scst_cmd_queue_type queue_type)
+{
+ cmd->queue_type = queue_type;
+}
+
+/*
+ * Get/Set functions for cmd's target SN
+ */
+static inline uint64_t scst_cmd_get_tag(struct scst_cmd *cmd)
+{
+ return cmd->tag;
+}
+
+static inline void scst_cmd_set_tag(struct scst_cmd *cmd, uint64_t tag)
+{
+ cmd->tag = tag;
+}
+
+/*
+ * Get/Set functions for cmd's target private data.
+ * Variant with *_lock must be used if target driver uses
+ * scst_find_cmd() to avoid race with it, except inside scst_find_cmd()'s
+ * callback, where lock is already taken.
+ */
+static inline void *scst_cmd_get_tgt_priv(struct scst_cmd *cmd)
+{
+ return cmd->tgt_priv;
+}
+
+static inline void scst_cmd_set_tgt_priv(struct scst_cmd *cmd, void *val)
+{
+ cmd->tgt_priv = val;
+}
+
+/*
+ * Get/Set functions for tgt_need_alloc_data_buf flag
+ */
+static inline int scst_cmd_get_tgt_need_alloc_data_buf(struct scst_cmd *cmd)
+{
+ return cmd->tgt_need_alloc_data_buf;
+}
+
+static inline void scst_cmd_set_tgt_need_alloc_data_buf(struct scst_cmd *cmd)
+{
+ cmd->tgt_need_alloc_data_buf = 1;
+}
+
+/*
+ * Get/Set functions for tgt_data_buf_alloced flag
+ */
+static inline int scst_cmd_get_tgt_data_buff_alloced(struct scst_cmd *cmd)
+{
+ return cmd->tgt_data_buf_alloced;
+}
+
+static inline void scst_cmd_set_tgt_data_buff_alloced(struct scst_cmd *cmd)
+{
+ cmd->tgt_data_buf_alloced = 1;
+}
+
+/*
+ * Get/Set functions for dh_data_buf_alloced flag
+ */
+static inline int scst_cmd_get_dh_data_buff_alloced(struct scst_cmd *cmd)
+{
+ return cmd->dh_data_buf_alloced;
+}
+
+static inline void scst_cmd_set_dh_data_buff_alloced(struct scst_cmd *cmd)
+{
+ cmd->dh_data_buf_alloced = 1;
+}
+
+/*
+ * Get/Set functions for no_sgv flag
+ */
+static inline int scst_cmd_get_no_sgv(struct scst_cmd *cmd)
+{
+ return cmd->no_sgv;
+}
+
+static inline void scst_cmd_set_no_sgv(struct scst_cmd *cmd)
+{
+ cmd->no_sgv = 1;
+}
+
+/*
+ * Get/Set functions for tgt_sn
+ */
+static inline int scst_cmd_get_tgt_sn(struct scst_cmd *cmd)
+{
+ BUG_ON(!cmd->tgt_sn_set);
+ return cmd->tgt_sn;
+}
+
+static inline void scst_cmd_set_tgt_sn(struct scst_cmd *cmd, uint32_t tgt_sn)
+{
+ cmd->tgt_sn_set = 1;
+ cmd->tgt_sn = tgt_sn;
+}
+
+/*
+ * Returns 1 if the cmd was aborted, so its status is invalid and no
+ * reply shall be sent to the remote initiator. A target driver should
+ * only clear internal resources, associated with cmd.
+ */
+static inline int scst_cmd_aborted(struct scst_cmd *cmd)
+{
+ return test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags) &&
+ !test_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+}
+
+/* Returns sense data format for cmd's dev */
+static inline bool scst_get_cmd_dev_d_sense(struct scst_cmd *cmd)
+{
+ return (cmd->dev != NULL) ? cmd->dev->d_sense : 0;
+}
+
+/*
+ * Get/Set functions for expected data direction, transfer length
+ * and its validity flag
+ */
+static inline int scst_cmd_is_expected_set(struct scst_cmd *cmd)
+{
+ return cmd->expected_values_set;
+}
+
+static inline scst_data_direction scst_cmd_get_expected_data_direction(
+ struct scst_cmd *cmd)
+{
+ return cmd->expected_data_direction;
+}
+
+static inline int scst_cmd_get_expected_transfer_len(
+ struct scst_cmd *cmd)
+{
+ return cmd->expected_transfer_len;
+}
+
+static inline int scst_cmd_get_expected_out_transfer_len(
+ struct scst_cmd *cmd)
+{
+ return cmd->expected_out_transfer_len;
+}
+
+static inline void scst_cmd_set_expected(struct scst_cmd *cmd,
+ scst_data_direction expected_data_direction,
+ int expected_transfer_len)
+{
+ cmd->expected_data_direction = expected_data_direction;
+ cmd->expected_transfer_len = expected_transfer_len;
+ cmd->expected_values_set = 1;
+}
+
+static inline void scst_cmd_set_expected_out_transfer_len(struct scst_cmd *cmd,
+ int expected_out_transfer_len)
+{
+ WARN_ON(!cmd->expected_values_set);
+ cmd->expected_out_transfer_len = expected_out_transfer_len;
+}
+
+/*
+ * Get/clear functions for cmd's may_need_dma_sync
+ */
+static inline int scst_get_may_need_dma_sync(struct scst_cmd *cmd)
+{
+ return cmd->may_need_dma_sync;
+}
+
+static inline void scst_clear_may_need_dma_sync(struct scst_cmd *cmd)
+{
+ cmd->may_need_dma_sync = 0;
+}
+
+/*
+ * Get/set functions for cmd's delivery_status. It is one of
+ * SCST_CMD_DELIVERY_* constants. It specifies the status of the
+ * command's delivery to initiator.
+ */
+static inline int scst_get_delivery_status(struct scst_cmd *cmd)
+{
+ return cmd->delivery_status;
+}
+
+static inline void scst_set_delivery_status(struct scst_cmd *cmd,
+ int delivery_status)
+{
+ cmd->delivery_status = delivery_status;
+}
+
+static inline unsigned int scst_get_active_cmd_count(struct scst_cmd *cmd)
+{
+ if (likely(cmd->tgt_dev != NULL))
+ return atomic_read(&cmd->tgt_dev->tgt_dev_cmd_count);
+ else
+ return (unsigned int)-1;
+}
+
+/*
+ * Get/Set function for mgmt cmd's target private data
+ */
+static inline void *scst_mgmt_cmd_get_tgt_priv(struct scst_mgmt_cmd *mcmd)
+{
+ return mcmd->tgt_priv;
+}
+
+static inline void scst_mgmt_cmd_set_tgt_priv(struct scst_mgmt_cmd *mcmd,
+ void *val)
+{
+ mcmd->tgt_priv = val;
+}
+
+/* Returns mgmt cmd's completition status (SCST_MGMT_STATUS_* constants) */
+static inline int scst_mgmt_cmd_get_status(struct scst_mgmt_cmd *mcmd)
+{
+ return mcmd->status;
+}
+
+/* Returns mgmt cmd's TM fn */
+static inline int scst_mgmt_cmd_get_fn(struct scst_mgmt_cmd *mcmd)
+{
+ return mcmd->fn;
+}
+
+/*
+ * Called by dev handler's task_mgmt_fn() to notify SCST core that mcmd
+ * is going to complete asynchronously.
+ */
+void scst_prepare_async_mcmd(struct scst_mgmt_cmd *mcmd);
+
+/*
+ * Called by dev handler to notify SCST core that async. mcmd is completed
+ * with status "status".
+ */
+void scst_async_mcmd_completed(struct scst_mgmt_cmd *mcmd, int status);
+
+/* Returns AEN's fn */
+static inline int scst_aen_get_event_fn(struct scst_aen *aen)
+{
+ return aen->event_fn;
+}
+
+/* Returns AEN's session */
+static inline struct scst_session *scst_aen_get_sess(struct scst_aen *aen)
+{
+ return aen->sess;
+}
+
+/* Returns AEN's LUN */
+static inline __be64 scst_aen_get_lun(struct scst_aen *aen)
+{
+ return aen->lun;
+}
+
+/* Returns SCSI AEN's sense */
+static inline const uint8_t *scst_aen_get_sense(struct scst_aen *aen)
+{
+ return aen->aen_sense;
+}
+
+/* Returns SCSI AEN's sense length */
+static inline int scst_aen_get_sense_len(struct scst_aen *aen)
+{
+ return aen->aen_sense_len;
+}
+
+/*
+ * Get/set functions for AEN's delivery_status. It is one of
+ * SCST_AEN_RES_* constants. It specifies the status of the
+ * command's delivery to initiator.
+ */
+static inline int scst_get_aen_delivery_status(struct scst_aen *aen)
+{
+ return aen->delivery_status;
+}
+
+static inline void scst_set_aen_delivery_status(struct scst_aen *aen,
+ int status)
+{
+ aen->delivery_status = status;
+}
+
+void scst_aen_done(struct scst_aen *aen);
+
+static inline void sg_clear(struct scatterlist *sg)
+{
+ memset(sg, 0, sizeof(*sg));
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+}
+
+enum scst_sg_copy_dir {
+ SCST_SG_COPY_FROM_TARGET,
+ SCST_SG_COPY_TO_TARGET
+};
+
+void scst_copy_sg(struct scst_cmd *cmd, enum scst_sg_copy_dir copy_dir);
+
+/*
+ * Functions for access to the commands data (SG) buffer. Should be used
+ * instead of direct access. Returns the buffer length for success, 0 for EOD,
+ * negative error code otherwise.
+ *
+ * "Buf" argument returns the mapped buffer
+ *
+ * The "put" function unmaps the buffer.
+ */
+static inline int __scst_get_buf(struct scst_cmd *cmd, int sg_cnt,
+ uint8_t **buf)
+{
+ int res = 0;
+ struct scatterlist *sg = cmd->get_sg_buf_cur_sg_entry;
+
+ *buf = NULL;
+
+ if (cmd->get_sg_buf_entry_num >= sg_cnt)
+ goto out;
+
+ if (unlikely(sg_is_chain(sg)))
+ sg = sg_chain_ptr(sg);
+
+ *buf = page_address(sg_page(sg));
+ *buf += sg->offset;
+
+ res = sg->length;
+
+ cmd->get_sg_buf_entry_num++;
+ cmd->get_sg_buf_cur_sg_entry = ++sg;
+
+out:
+ return res;
+}
+
+static inline int scst_get_buf_first(struct scst_cmd *cmd, uint8_t **buf)
+{
+ if (unlikely(cmd->sg == NULL))
+ return 0;
+ cmd->get_sg_buf_entry_num = 0;
+ cmd->get_sg_buf_cur_sg_entry = cmd->sg;
+ cmd->may_need_dma_sync = 1;
+ return __scst_get_buf(cmd, cmd->sg_cnt, buf);
+}
+
+static inline int scst_get_buf_next(struct scst_cmd *cmd, uint8_t **buf)
+{
+ return __scst_get_buf(cmd, cmd->sg_cnt, buf);
+}
+
+static inline void scst_put_buf(struct scst_cmd *cmd, void *buf)
+{
+ /* Nothing to do */
+}
+
+static inline int scst_get_out_buf_first(struct scst_cmd *cmd, uint8_t **buf)
+{
+ if (unlikely(cmd->out_sg == NULL))
+ return 0;
+ cmd->get_sg_buf_entry_num = 0;
+ cmd->get_sg_buf_cur_sg_entry = cmd->out_sg;
+ cmd->may_need_dma_sync = 1;
+ return __scst_get_buf(cmd, cmd->out_sg_cnt, buf);
+}
+
+static inline int scst_get_out_buf_next(struct scst_cmd *cmd, uint8_t **buf)
+{
+ return __scst_get_buf(cmd, cmd->out_sg_cnt, buf);
+}
+
+static inline void scst_put_out_buf(struct scst_cmd *cmd, void *buf)
+{
+ /* Nothing to do */
+}
+
+static inline int scst_get_sg_buf_first(struct scst_cmd *cmd, uint8_t **buf,
+ struct scatterlist *sg, int sg_cnt)
+{
+ if (unlikely(sg == NULL))
+ return 0;
+ cmd->get_sg_buf_entry_num = 0;
+ cmd->get_sg_buf_cur_sg_entry = cmd->sg;
+ cmd->may_need_dma_sync = 1;
+ return __scst_get_buf(cmd, sg_cnt, buf);
+}
+
+static inline int scst_get_sg_buf_next(struct scst_cmd *cmd, uint8_t **buf,
+ struct scatterlist *sg, int sg_cnt)
+{
+ return __scst_get_buf(cmd, sg_cnt, buf);
+}
+
+static inline void scst_put_sg_buf(struct scst_cmd *cmd, void *buf,
+ struct scatterlist *sg, int sg_cnt)
+{
+ /* Nothing to do */
+}
+
+/*
+ * Returns approximate higher rounded buffers count that
+ * scst_get_buf_[first|next]() return.
+ */
+static inline int scst_get_buf_count(struct scst_cmd *cmd)
+{
+ return (cmd->sg_cnt == 0) ? 1 : cmd->sg_cnt;
+}
+
+/*
+ * Returns approximate higher rounded buffers count that
+ * scst_get_out_buf_[first|next]() return.
+ */
+static inline int scst_get_out_buf_count(struct scst_cmd *cmd)
+{
+ return (cmd->out_sg_cnt == 0) ? 1 : cmd->out_sg_cnt;
+}
+
+int scst_suspend_activity(bool interruptible);
+void scst_resume_activity(void);
+
+void scst_process_active_cmd(struct scst_cmd *cmd, bool atomic);
+
+void scst_post_parse(struct scst_cmd *cmd);
+void scst_post_alloc_data_buf(struct scst_cmd *cmd);
+
+int scst_check_local_events(struct scst_cmd *cmd);
+
+int scst_set_cmd_abnormal_done_state(struct scst_cmd *cmd);
+
+struct scst_trace_log {
+ unsigned int val;
+ const char *token;
+};
+
+extern struct mutex scst_mutex;
+
+const struct sysfs_ops *scst_sysfs_get_sysfs_ops(void);
+
+/*
+ * Returns target driver's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_tgtt_kobj(
+ struct scst_tgt_template *tgtt)
+{
+ return &tgtt->tgtt_kobj;
+}
+
+/*
+ * Returns target's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_tgt_kobj(
+ struct scst_tgt *tgt)
+{
+ return &tgt->tgt_kobj;
+}
+
+/*
+ * Returns device handler's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_devt_kobj(
+ struct scst_dev_type *devt)
+{
+ return &devt->devt_kobj;
+}
+
+/*
+ * Returns device's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_dev_kobj(
+ struct scst_device *dev)
+{
+ return &dev->dev_kobj;
+}
+
+/*
+ * Returns session's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_sess_kobj(
+ struct scst_session *sess)
+{
+ return &sess->sess_kobj;
+}
+
+/* Returns target name */
+static inline const char *scst_get_tgt_name(const struct scst_tgt *tgt)
+{
+ return tgt->tgt_name;
+}
+
+int scst_alloc_sense(struct scst_cmd *cmd, int atomic);
+int scst_alloc_set_sense(struct scst_cmd *cmd, int atomic,
+ const uint8_t *sense, unsigned int len);
+
+int scst_set_sense(uint8_t *buffer, int len, bool d_sense,
+ int key, int asc, int ascq);
+
+bool scst_is_ua_sense(const uint8_t *sense, int len);
+
+bool scst_analyze_sense(const uint8_t *sense, int len,
+ unsigned int valid_mask, int key, int asc, int ascq);
+
+unsigned long scst_random(void);
+
+void scst_set_resp_data_len(struct scst_cmd *cmd, int resp_data_len);
+
+void scst_get(void);
+void scst_put(void);
+
+void scst_cmd_get(struct scst_cmd *cmd);
+void scst_cmd_put(struct scst_cmd *cmd);
+
+struct scatterlist *scst_alloc(int size, gfp_t gfp_mask, int *count);
+void scst_free(struct scatterlist *sg, int count);
+
+void scst_add_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct scst_thr_data_hdr *data,
+ void (*free_fn) (struct scst_thr_data_hdr *data));
+void scst_del_all_thr_data(struct scst_tgt_dev *tgt_dev);
+void scst_dev_del_all_thr_data(struct scst_device *dev);
+struct scst_thr_data_hdr *__scst_find_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct task_struct *tsk);
+
+/* Finds local to the current thread data. Returns NULL, if they not found. */
+static inline struct scst_thr_data_hdr *scst_find_thr_data(
+ struct scst_tgt_dev *tgt_dev)
+{
+ return __scst_find_thr_data(tgt_dev, current);
+}
+
+/* Increase ref counter for the thread data */
+static inline void scst_thr_data_get(struct scst_thr_data_hdr *data)
+{
+ atomic_inc(&data->ref);
+}
+
+/* Decrease ref counter for the thread data */
+static inline void scst_thr_data_put(struct scst_thr_data_hdr *data)
+{
+ if (atomic_dec_and_test(&data->ref))
+ data->free_fn(data);
+}
+
+int scst_calc_block_shift(int sector_size);
+int scst_sbc_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd));
+int scst_cdrom_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd));
+int scst_modisk_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd));
+int scst_tape_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_size)(struct scst_cmd *cmd));
+int scst_changer_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd));
+int scst_processor_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd));
+int scst_raid_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd));
+
+int scst_block_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_shift)(struct scst_cmd *cmd, int block_shift));
+int scst_tape_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_size)(struct scst_cmd *cmd, int block_size));
+
+int scst_obtain_device_parameters(struct scst_device *dev);
+
+void scst_reassign_persistent_sess_states(struct scst_session *new_sess,
+ struct scst_session *old_sess);
+
+int scst_get_max_lun_commands(struct scst_session *sess, uint64_t lun);
+
+/*
+ * Has to be put here open coded, because Linux doesn't have equivalent, which
+ * allows exclusive wake ups of threads in LIFO order. We need it to let (yet)
+ * unneeded threads sleep and not pollute CPU cache by their stacks.
+ */
+static inline void add_wait_queue_exclusive_head(wait_queue_head_t *q,
+ wait_queue_t *wait)
+{
+ unsigned long flags;
+
+ wait->flags |= WQ_FLAG_EXCLUSIVE;
+ spin_lock_irqsave(&q->lock, flags);
+ __add_wait_queue(q, wait);
+ spin_unlock_irqrestore(&q->lock, flags);
+}
+
+/*
+ * Structure to match events to user space and replies on them
+ */
+struct scst_sysfs_user_info {
+ /* Unique cookie to identify request */
+ uint32_t info_cookie;
+
+ /* Entry in the global list */
+ struct list_head info_list_entry;
+
+ /* Set if reply from the user space is being executed */
+ unsigned int info_being_executed:1;
+
+ /* Set if this info is in the info_list */
+ unsigned int info_in_list:1;
+
+ /* Completion to wait on for the request completion */
+ struct completion info_completion;
+
+ /* Request completion status and optional data */
+ int info_status;
+ void *data;
+};
+
+int scst_sysfs_user_add_info(struct scst_sysfs_user_info **out_info);
+void scst_sysfs_user_del_info(struct scst_sysfs_user_info *info);
+struct scst_sysfs_user_info *scst_sysfs_user_get_info(uint32_t cookie);
+int scst_wait_info_completion(struct scst_sysfs_user_info *info,
+ unsigned long timeout);
+
+unsigned int scst_get_setup_id(void);
+
+/*
+ * Needed to avoid potential circular locking dependency between scst_mutex
+ * and internal sysfs locking (s_active). It could be since most sysfs entries
+ * are created and deleted under scst_mutex AND scst_mutex is taken inside
+ * sysfs functions. So, we push from the sysfs functions all the processing
+ * taking scst_mutex. To avoid deadlock, we return from them with EAGAIN
+ * if processing is taking too long. User space then should poll
+ * last_sysfs_mgmt_res until it returns the result of the processing
+ * (something other than EAGAIN).
+ */
+struct scst_sysfs_work_item {
+ struct list_head sysfs_work_list_entry;
+ struct kref sysfs_work_kref;
+ int (*sysfs_work_fn)(struct scst_sysfs_work_item *work);
+ struct completion sysfs_work_done;
+ char *buf;
+ union {
+ struct scst_dev_type *devt;
+ struct scst_tgt_template *tgtt;
+ struct {
+ struct scst_tgt *tgt;
+ struct scst_acg *acg;
+ union {
+ bool is_tgt_kobj;
+ int io_grouping_type;
+ bool enable;
+ };
+ };
+ struct {
+ struct scst_device *dev;
+ int new_threads_num;
+ enum scst_dev_type_threads_pool_type new_threads_pool_type;
+ };
+ struct scst_session *sess;
+ struct {
+ struct scst_tgt *tgt;
+ unsigned long l;
+ };
+ };
+ int work_res;
+ char *res_buf;
+};
+
+int scst_alloc_sysfs_work(int (*sysfs_work_fn)(struct scst_sysfs_work_item *),
+ struct scst_sysfs_work_item **res_work);
+int scst_sysfs_queue_wait_work(struct scst_sysfs_work_item *work);
+void scst_sysfs_work_get(struct scst_sysfs_work_item *work);
+void scst_sysfs_work_put(struct scst_sysfs_work_item *work);
+
+char *scst_get_next_lexem(char **token_str);
+void scst_restore_token_str(char *prev_lexem, char *token_str);
+char *scst_get_next_token_str(char **input_str);
+
+void scst_init_threads(struct scst_cmd_threads *cmd_threads);
+void scst_deinit_threads(struct scst_cmd_threads *cmd_threads);
+
+void scst_pass_through_cmd_done(void *data, char *sense, int result, int resid);
+int scst_scsi_exec_async(struct scst_cmd *cmd, void *data,
+ void (*done)(void *data, char *sense, int result, int resid));
+
+#endif /* __SCST_H */
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 4/17]: SCST main management files and private headers
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (2 preceding siblings ...)
2010-09-14 14:39 ` [PATCH 3/17]: SCST public headers Vladislav Bolkhovitin
@ 2010-09-14 14:40 ` Vladislav Bolkhovitin
2010-09-14 14:41 ` [PATCH 5/17]: SCST implementation of the SCSI target state machine Vladislav Bolkhovitin
` (13 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:40 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains main management files and private headers.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_main.c | 2096 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
scst_module.c | 63 +
scst_priv.h | 609 ++++++++++++++++
3 files changed, 2768 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/scst_main.c linux-2.6.35/drivers/scst/scst_main.c
--- orig/linux-2.6.35/drivers/scst/scst_main.c
+++ linux-2.6.35/drivers/scst/scst_main.c
@@ -0,0 +1,2096 @@
+/*
+ * scst_main.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+#include <linux/kthread.h>
+#include <linux/delay.h>
+
+#include <scst/scst.h>
+#include "scst_priv.h"
+#include "scst_mem.h"
+#include "scst_pres.h"
+
+#if defined(CONFIG_HIGHMEM4G) || defined(CONFIG_HIGHMEM64G)
+#warning "HIGHMEM kernel configurations are fully supported, but not\
+ recommended for performance reasons. Consider changing VMSPLIT\
+ option or use a 64-bit configuration instead. See README file for\
+ details."
+#endif
+
+/**
+ ** SCST global variables. They are all uninitialized to have their layout in
+ ** memory be exactly as specified. Otherwise compiler puts zero-initialized
+ ** variable separately from nonzero-initialized ones.
+ **/
+
+/*
+ * Main SCST mutex. All targets, devices and dev_types management is done
+ * under this mutex.
+ *
+ * It must NOT be used in any works (schedule_work(), etc.), because
+ * otherwise a deadlock (double lock, actually) is possible, e.g., with
+ * scst_user detach_tgt(), which is called under scst_mutex and calls
+ * flush_scheduled_work().
+ */
+struct mutex scst_mutex;
+EXPORT_SYMBOL_GPL(scst_mutex);
+
+/*
+ * Secondary level main mutex, inner for scst_mutex. Needed for
+ * __scst_pr_register_all_tg_pt(), since we can't use scst_mutex there,
+ * because of the circular locking dependency with dev_pr_mutex.
+ */
+struct mutex scst_mutex2;
+
+/* Both protected by scst_mutex or scst_mutex2 on read and both on write */
+struct list_head scst_template_list;
+struct list_head scst_dev_list;
+
+/* Protected by scst_mutex */
+struct list_head scst_dev_type_list;
+struct list_head scst_virtual_dev_type_list;
+
+spinlock_t scst_main_lock;
+
+static struct kmem_cache *scst_mgmt_cachep;
+mempool_t *scst_mgmt_mempool;
+static struct kmem_cache *scst_mgmt_stub_cachep;
+mempool_t *scst_mgmt_stub_mempool;
+static struct kmem_cache *scst_ua_cachep;
+mempool_t *scst_ua_mempool;
+static struct kmem_cache *scst_sense_cachep;
+mempool_t *scst_sense_mempool;
+static struct kmem_cache *scst_aen_cachep;
+mempool_t *scst_aen_mempool;
+struct kmem_cache *scst_tgtd_cachep;
+struct kmem_cache *scst_sess_cachep;
+struct kmem_cache *scst_acgd_cachep;
+
+unsigned int scst_setup_id;
+
+spinlock_t scst_init_lock;
+wait_queue_head_t scst_init_cmd_list_waitQ;
+struct list_head scst_init_cmd_list;
+unsigned int scst_init_poll_cnt;
+
+struct kmem_cache *scst_cmd_cachep;
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+unsigned long scst_trace_flag;
+#endif
+
+unsigned long scst_flags;
+atomic_t scst_cmd_count;
+
+struct scst_cmd_threads scst_main_cmd_threads;
+
+struct scst_tasklet scst_tasklets[NR_CPUS];
+
+spinlock_t scst_mcmd_lock;
+struct list_head scst_active_mgmt_cmd_list;
+struct list_head scst_delayed_mgmt_cmd_list;
+wait_queue_head_t scst_mgmt_cmd_list_waitQ;
+
+wait_queue_head_t scst_mgmt_waitQ;
+spinlock_t scst_mgmt_lock;
+struct list_head scst_sess_init_list;
+struct list_head scst_sess_shut_list;
+
+wait_queue_head_t scst_dev_cmd_waitQ;
+
+static struct mutex scst_suspend_mutex;
+/* protected by scst_suspend_mutex */
+static struct list_head scst_cmd_threads_list;
+
+int scst_threads;
+static struct task_struct *scst_init_cmd_thread;
+static struct task_struct *scst_mgmt_thread;
+static struct task_struct *scst_mgmt_cmd_thread;
+
+static int suspend_count;
+
+static int scst_virt_dev_last_id; /* protected by scst_mutex */
+
+static unsigned int scst_max_cmd_mem;
+unsigned int scst_max_dev_cmd_mem;
+
+module_param_named(scst_threads, scst_threads, int, 0);
+MODULE_PARM_DESC(scst_threads, "SCSI target threads count");
+
+module_param_named(scst_max_cmd_mem, scst_max_cmd_mem, int, S_IRUGO);
+MODULE_PARM_DESC(scst_max_cmd_mem, "Maximum memory allowed to be consumed by "
+ "all SCSI commands of all devices at any given time in MB");
+
+module_param_named(scst_max_dev_cmd_mem, scst_max_dev_cmd_mem, int, S_IRUGO);
+MODULE_PARM_DESC(scst_max_dev_cmd_mem, "Maximum memory allowed to be consumed "
+ "by all SCSI commands of a device at any given time in MB");
+
+struct scst_dev_type scst_null_devtype = {
+ .name = "none",
+ .threads_num = -1,
+};
+
+static void __scst_resume_activity(void);
+
+/**
+ * __scst_register_target_template() - register target template.
+ * @vtt: target template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the target driver use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a target template and returns 0 on success or appropriate
+ * error code otherwise.
+ *
+ * Target drivers supposed to behave sanely and not call register()
+ * and unregister() randomly sinultaneously.
+ */
+int __scst_register_target_template(struct scst_tgt_template *vtt,
+ const char *version)
+{
+ int res = 0;
+ struct scst_tgt_template *t;
+
+ INIT_LIST_HEAD(&vtt->tgt_list);
+
+ if (strcmp(version, SCST_INTERFACE_VERSION) != 0) {
+ PRINT_ERROR("Incorrect version of target %s", vtt->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (!vtt->detect) {
+ PRINT_ERROR("Target driver %s must have "
+ "detect() method.", vtt->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (!vtt->release) {
+ PRINT_ERROR("Target driver %s must have "
+ "release() method.", vtt->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (!vtt->xmit_response) {
+ PRINT_ERROR("Target driver %s must have "
+ "xmit_response() method.", vtt->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (vtt->get_initiator_port_transport_id == NULL)
+ PRINT_WARNING("Target driver %s doesn't support Persistent "
+ "Reservations", vtt->name);
+
+ if (vtt->threads_num < 0) {
+ PRINT_ERROR("Wrong threads_num value %d for "
+ "target \"%s\"", vtt->threads_num,
+ vtt->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if ((!vtt->enable_target || !vtt->is_target_enabled) &&
+ !vtt->enabled_attr_not_needed)
+ PRINT_WARNING("Target driver %s doesn't have enable_target() "
+ "and/or is_target_enabled() method(s). This is unsafe "
+ "and can lead that initiators connected on the "
+ "initialization time can see an unexpected set of "
+ "devices or no devices at all!", vtt->name);
+
+ if (((vtt->add_target != NULL) && (vtt->del_target == NULL)) ||
+ ((vtt->add_target == NULL) && (vtt->del_target != NULL))) {
+ PRINT_ERROR("Target driver %s must either define both "
+ "add_target() and del_target(), or none.", vtt->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (vtt->rdy_to_xfer == NULL)
+ vtt->rdy_to_xfer_atomic = 1;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0)
+ goto out;
+ list_for_each_entry(t, &scst_template_list, scst_template_list_entry) {
+ if (strcmp(t->name, vtt->name) == 0) {
+ PRINT_ERROR("Target driver %s already registered",
+ vtt->name);
+ mutex_unlock(&scst_mutex);
+ goto out_unlock;
+ }
+ }
+ mutex_unlock(&scst_mutex);
+
+ res = scst_tgtt_sysfs_create(vtt);
+ if (res)
+ goto out;
+
+ mutex_lock(&scst_mutex);
+ mutex_lock(&scst_mutex2);
+ list_add_tail(&vtt->scst_template_list_entry, &scst_template_list);
+ mutex_unlock(&scst_mutex2);
+ mutex_unlock(&scst_mutex);
+
+ TRACE_DBG("%s", "Calling target driver's detect()");
+ res = vtt->detect(vtt);
+ TRACE_DBG("Target driver's detect() returned %d", res);
+ if (res < 0) {
+ PRINT_ERROR("%s", "The detect() routine failed");
+ res = -EINVAL;
+ goto out_del;
+ }
+
+ PRINT_INFO("Target template %s registered successfully", vtt->name);
+
+out:
+ return res;
+
+out_del:
+ scst_tgtt_sysfs_del(vtt);
+
+ mutex_lock(&scst_mutex);
+
+ mutex_lock(&scst_mutex2);
+ list_del(&vtt->scst_template_list_entry);
+ mutex_unlock(&scst_mutex2);
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(__scst_register_target_template);
+
+static int scst_check_non_gpl_target_template(struct scst_tgt_template *vtt)
+{
+ int res;
+
+ if (vtt->task_mgmt_affected_cmds_done || vtt->threads_num ||
+ vtt->on_hw_pending_cmd_timeout) {
+ PRINT_ERROR("Not allowed functionality in non-GPL version for "
+ "target template %s", vtt->name);
+ res = -EPERM;
+ goto out;
+ }
+
+ res = 0;
+
+out:
+ return res;
+}
+
+/**
+ * __scst_register_target_template_non_gpl() - register target template,
+ * non-GPL version
+ * @vtt: target template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the target driver use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a target template and returns 0 on success or appropriate
+ * error code otherwise.
+ *
+ * Note: *vtt must be static!
+ */
+int __scst_register_target_template_non_gpl(struct scst_tgt_template *vtt,
+ const char *version)
+{
+ int res;
+
+ res = scst_check_non_gpl_target_template(vtt);
+ if (res != 0)
+ goto out;
+
+ res = __scst_register_target_template(vtt, version);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(__scst_register_target_template_non_gpl);
+
+/**
+ * scst_unregister_target_template() - unregister target template
+ *
+ * Target drivers supposed to behave sanely and not call register()
+ * and unregister() randomly sinultaneously. Also it is supposed that
+ * no attepts to create new targets for this vtt will be done in a race
+ * with this function.
+ */
+void scst_unregister_target_template(struct scst_tgt_template *vtt)
+{
+ struct scst_tgt *tgt;
+ struct scst_tgt_template *t;
+ int found = 0;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(t, &scst_template_list, scst_template_list_entry) {
+ if (strcmp(t->name, vtt->name) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found) {
+ PRINT_ERROR("Target driver %s isn't registered", vtt->name);
+ goto out_err_up;
+ }
+
+ mutex_lock(&scst_mutex2);
+ list_del(&vtt->scst_template_list_entry);
+ mutex_unlock(&scst_mutex2);
+
+ /* Wait for outstanding sysfs mgmt calls completed */
+ while (vtt->tgtt_active_sysfs_works_count > 0) {
+ mutex_unlock(&scst_mutex);
+ msleep(100);
+ mutex_lock(&scst_mutex);
+ }
+
+restart:
+ list_for_each_entry(tgt, &vtt->tgt_list, tgt_list_entry) {
+ mutex_unlock(&scst_mutex);
+ scst_unregister_target(tgt);
+ mutex_lock(&scst_mutex);
+ goto restart;
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ scst_tgtt_sysfs_del(vtt);
+
+ PRINT_INFO("Target template %s unregistered successfully", vtt->name);
+
+out:
+ return;
+
+out_err_up:
+ mutex_unlock(&scst_mutex);
+ goto out;
+}
+EXPORT_SYMBOL(scst_unregister_target_template);
+
+/**
+ * scst_register_target() - register target
+ *
+ * Registers a target for template vtt and returns new target structure on
+ * success or NULL otherwise.
+ */
+struct scst_tgt *scst_register_target(struct scst_tgt_template *vtt,
+ const char *target_name)
+{
+ struct scst_tgt *tgt;
+ int rc = 0;
+
+ rc = scst_alloc_tgt(vtt, &tgt);
+ if (rc != 0)
+ goto out;
+
+ if (target_name != NULL) {
+
+ tgt->tgt_name = kmalloc(strlen(target_name) + 1, GFP_KERNEL);
+ if (tgt->tgt_name == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of tgt name %s failed",
+ target_name);
+ rc = -ENOMEM;
+ goto out_free_tgt;
+ }
+ strcpy(tgt->tgt_name, target_name);
+ } else {
+ static int tgt_num; /* protected by scst_mutex */
+ int len = strlen(vtt->name) +
+ strlen(SCST_DEFAULT_TGT_NAME_SUFFIX) + 11 + 1;
+
+ tgt->tgt_name = kmalloc(len, GFP_KERNEL);
+ if (tgt->tgt_name == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of tgt name failed "
+ "(template name %s)", vtt->name);
+ rc = -ENOMEM;
+ goto out_free_tgt;
+ }
+ sprintf(tgt->tgt_name, "%s%s%d", vtt->name,
+ SCST_DEFAULT_TGT_NAME_SUFFIX, tgt_num++);
+ }
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ rc = -EINTR;
+ goto out_free_tgt;
+ }
+
+ rc = scst_tgt_sysfs_create(tgt);
+ if (rc < 0)
+ goto out_unlock;
+
+ tgt->default_acg = scst_alloc_add_acg(tgt, tgt->tgt_name, false);
+ if (tgt->default_acg == NULL)
+ goto out_sysfs_del;
+
+ mutex_lock(&scst_mutex2);
+ list_add_tail(&tgt->tgt_list_entry, &vtt->tgt_list);
+ mutex_unlock(&scst_mutex2);
+
+ mutex_unlock(&scst_mutex);
+
+ PRINT_INFO("Target %s for template %s registered successfully",
+ tgt->tgt_name, vtt->name);
+
+ TRACE_DBG("tgt %p", tgt);
+
+out:
+ return tgt;
+
+out_sysfs_del:
+ mutex_unlock(&scst_mutex);
+ scst_tgt_sysfs_del(tgt);
+ goto out_free_tgt;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_free_tgt:
+ /* In case of error tgt_name will be freed in scst_free_tgt() */
+ scst_free_tgt(tgt);
+ tgt = NULL;
+ goto out;
+}
+EXPORT_SYMBOL(scst_register_target);
+
+static inline int test_sess_list(struct scst_tgt *tgt)
+{
+ int res;
+ mutex_lock(&scst_mutex);
+ res = list_empty(&tgt->sess_list);
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+/**
+ * scst_unregister_target() - unregister target.
+ *
+ * It is supposed that no attepts to create new sessions for this
+ * target will be done in a race with this function.
+ */
+void scst_unregister_target(struct scst_tgt *tgt)
+{
+ struct scst_session *sess;
+ struct scst_tgt_template *vtt = tgt->tgtt;
+ struct scst_acg *acg, *acg_tmp;
+
+ TRACE_DBG("%s", "Calling target driver's release()");
+ tgt->tgtt->release(tgt);
+ TRACE_DBG("%s", "Target driver's release() returned");
+
+ mutex_lock(&scst_mutex);
+again:
+ list_for_each_entry(sess, &tgt->sess_list, sess_list_entry) {
+ if (sess->shut_phase == SCST_SESS_SPH_READY) {
+ /*
+ * Sometimes it's hard for target driver to track all
+ * its sessions (see scst_local, eg), so let's help it.
+ */
+ mutex_unlock(&scst_mutex);
+ scst_unregister_session(sess, 0, NULL);
+ mutex_lock(&scst_mutex);
+ goto again;
+ }
+ }
+ mutex_unlock(&scst_mutex);
+
+ TRACE_DBG("%s", "Waiting for sessions shutdown");
+ wait_event(tgt->unreg_waitQ, test_sess_list(tgt));
+ TRACE_DBG("%s", "wait_event() returned");
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ mutex_lock(&scst_mutex2);
+ list_del(&tgt->tgt_list_entry);
+ mutex_unlock(&scst_mutex2);
+
+ del_timer_sync(&tgt->retry_timer);
+
+ scst_del_free_acg(tgt->default_acg);
+
+ list_for_each_entry_safe(acg, acg_tmp, &tgt->tgt_acg_list,
+ acg_list_entry) {
+ scst_del_free_acg(acg);
+ }
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ scst_tgt_sysfs_del(tgt);
+
+ PRINT_INFO("Target %s for template %s unregistered successfully",
+ tgt->tgt_name, vtt->name);
+
+ scst_free_tgt(tgt);
+
+ TRACE_DBG("Unregistering tgt %p finished", tgt);
+ return;
+}
+EXPORT_SYMBOL(scst_unregister_target);
+
+static int scst_susp_wait(bool interruptible)
+{
+ int res = 0;
+
+ if (interruptible) {
+ res = wait_event_interruptible_timeout(scst_dev_cmd_waitQ,
+ (atomic_read(&scst_cmd_count) == 0),
+ SCST_SUSPENDING_TIMEOUT);
+ if (res <= 0) {
+ __scst_resume_activity();
+ if (res == 0)
+ res = -EBUSY;
+ } else
+ res = 0;
+ } else
+ wait_event(scst_dev_cmd_waitQ,
+ atomic_read(&scst_cmd_count) == 0);
+
+ TRACE_MGMT_DBG("wait_event() returned %d", res);
+ return res;
+}
+
+/**
+ * scst_suspend_activity() - globally suspend any activity
+ *
+ * Description:
+ * Globally suspends any activity and doesn't return, until there are any
+ * active commands (state after SCST_CMD_STATE_INIT). If "interruptible"
+ * is true, it returns after SCST_SUSPENDING_TIMEOUT or if it was interrupted
+ * by a signal with the corresponding error status < 0. If "interruptible"
+ * is false, it will wait virtually forever. On success returns 0.
+ *
+ * New arriving commands stay in the suspended state until
+ * scst_resume_activity() is called.
+ */
+int scst_suspend_activity(bool interruptible)
+{
+ int res = 0;
+ bool rep = false;
+
+ if (interruptible) {
+ if (mutex_lock_interruptible(&scst_suspend_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+ } else
+ mutex_lock(&scst_suspend_mutex);
+
+ TRACE_MGMT_DBG("suspend_count %d", suspend_count);
+ suspend_count++;
+ if (suspend_count > 1)
+ goto out_up;
+
+ set_bit(SCST_FLAG_SUSPENDING, &scst_flags);
+ set_bit(SCST_FLAG_SUSPENDED, &scst_flags);
+ /*
+ * Assignment of SCST_FLAG_SUSPENDING and SCST_FLAG_SUSPENDED must be
+ * ordered with scst_cmd_count. Otherwise lockless logic in
+ * scst_translate_lun() and scst_mgmt_translate_lun() won't work.
+ */
+ smp_mb__after_set_bit();
+
+ /*
+ * See comment in scst_user.c::dev_user_task_mgmt_fn() for more
+ * information about scst_user behavior.
+ *
+ * ToDo: make the global suspending unneeded (switch to per-device
+ * reference counting? That would mean to switch off from lockless
+ * implementation of scst_translate_lun().. )
+ */
+
+ if (atomic_read(&scst_cmd_count) != 0) {
+ PRINT_INFO("Waiting for %d active commands to complete... This "
+ "might take few minutes for disks or few hours for "
+ "tapes, if you use long executed commands, like "
+ "REWIND or FORMAT. In case, if you have a hung user "
+ "space device (i.e. made using scst_user module) not "
+ "responding to any commands, if might take virtually "
+ "forever until the corresponding user space "
+ "program recovers and starts responding or gets "
+ "killed.", atomic_read(&scst_cmd_count));
+ rep = true;
+ }
+
+ res = scst_susp_wait(interruptible);
+ if (res != 0)
+ goto out_clear;
+
+ clear_bit(SCST_FLAG_SUSPENDING, &scst_flags);
+ /* See comment about smp_mb() above */
+ smp_mb__after_clear_bit();
+
+ TRACE_MGMT_DBG("Waiting for %d active commands finally to complete",
+ atomic_read(&scst_cmd_count));
+
+ res = scst_susp_wait(interruptible);
+ if (res != 0)
+ goto out_clear;
+
+ if (rep)
+ PRINT_INFO("%s", "All active commands completed");
+
+out_up:
+ mutex_unlock(&scst_suspend_mutex);
+
+out:
+ return res;
+
+out_clear:
+ clear_bit(SCST_FLAG_SUSPENDING, &scst_flags);
+ /* See comment about smp_mb() above */
+ smp_mb__after_clear_bit();
+ goto out_up;
+}
+EXPORT_SYMBOL_GPL(scst_suspend_activity);
+
+static void __scst_resume_activity(void)
+{
+ struct scst_cmd_threads *l;
+
+ suspend_count--;
+ TRACE_MGMT_DBG("suspend_count %d left", suspend_count);
+ if (suspend_count > 0)
+ goto out;
+
+ clear_bit(SCST_FLAG_SUSPENDED, &scst_flags);
+ /*
+ * The barrier is needed to make sure all woken up threads see the
+ * cleared flag. Not sure if it's really needed, but let's be safe.
+ */
+ smp_mb__after_clear_bit();
+
+ list_for_each_entry(l, &scst_cmd_threads_list, lists_list_entry) {
+ wake_up_all(&l->cmd_list_waitQ);
+ }
+ wake_up_all(&scst_init_cmd_list_waitQ);
+
+ spin_lock_irq(&scst_mcmd_lock);
+ if (!list_empty(&scst_delayed_mgmt_cmd_list)) {
+ struct scst_mgmt_cmd *m;
+ m = list_entry(scst_delayed_mgmt_cmd_list.next, typeof(*m),
+ mgmt_cmd_list_entry);
+ TRACE_MGMT_DBG("Moving delayed mgmt cmd %p to head of active "
+ "mgmt cmd list", m);
+ list_move(&m->mgmt_cmd_list_entry, &scst_active_mgmt_cmd_list);
+ }
+ spin_unlock_irq(&scst_mcmd_lock);
+ wake_up_all(&scst_mgmt_cmd_list_waitQ);
+
+out:
+ return;
+}
+
+/**
+ * scst_resume_activity() - globally resume all activities
+ *
+ * Resumes suspended by scst_suspend_activity() activities.
+ */
+void scst_resume_activity(void)
+{
+
+ mutex_lock(&scst_suspend_mutex);
+ __scst_resume_activity();
+ mutex_unlock(&scst_suspend_mutex);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_resume_activity);
+
+static int scst_register_device(struct scsi_device *scsidp)
+{
+ int res = 0;
+ struct scst_device *dev, *d;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = scst_alloc_device(GFP_KERNEL, &dev);
+ if (res != 0)
+ goto out_unlock;
+
+ dev->type = scsidp->type;
+
+ dev->virt_name = kmalloc(50, GFP_KERNEL);
+ if (dev->virt_name == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc device name");
+ res = -ENOMEM;
+ goto out_free_dev;
+ }
+ snprintf(dev->virt_name, 50, "%d:%d:%d:%d", scsidp->host->host_no,
+ scsidp->channel, scsidp->id, scsidp->lun);
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (strcmp(d->virt_name, dev->virt_name) == 0) {
+ PRINT_ERROR("Device %s already exists", dev->virt_name);
+ res = -EEXIST;
+ goto out_free_dev;
+ }
+ }
+
+ dev->scsi_dev = scsidp;
+
+ list_add_tail(&dev->dev_list_entry, &scst_dev_list);
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_dev_sysfs_create(dev);
+ if (res != 0)
+ goto out_del;
+
+ PRINT_INFO("Attached to scsi%d, channel %d, id %d, lun %d, "
+ "type %d", scsidp->host->host_no, scsidp->channel,
+ scsidp->id, scsidp->lun, scsidp->type);
+
+out:
+ return res;
+
+out_del:
+ list_del(&dev->dev_list_entry);
+
+out_free_dev:
+ scst_free_device(dev);
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ goto out;
+}
+
+static void scst_unregister_device(struct scsi_device *scsidp)
+{
+ struct scst_device *d, *dev = NULL;
+ struct scst_acg_dev *acg_dev, *aa;
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (d->scsi_dev == scsidp) {
+ dev = d;
+ TRACE_DBG("Device %p found", dev);
+ break;
+ }
+ }
+ if (dev == NULL) {
+ PRINT_ERROR("SCST device for SCSI device %d:%d:%d:%d not found",
+ scsidp->host->host_no, scsidp->channel, scsidp->id,
+ scsidp->lun);
+ goto out_unlock;
+ }
+
+ list_del(&dev->dev_list_entry);
+
+ scst_assign_dev_handler(dev, &scst_null_devtype);
+
+ list_for_each_entry_safe(acg_dev, aa, &dev->dev_acg_dev_list,
+ dev_acg_dev_list_entry) {
+ scst_acg_del_lun(acg_dev->acg, acg_dev->lun, true);
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ scst_resume_activity();
+
+ scst_dev_sysfs_del(dev);
+
+ PRINT_INFO("Detached from scsi%d, channel %d, id %d, lun %d, type %d",
+ scsidp->host->host_no, scsidp->channel, scsidp->id,
+ scsidp->lun, scsidp->type);
+
+ scst_free_device(dev);
+
+out:
+ return;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+ goto out;
+}
+
+static int scst_dev_handler_check(struct scst_dev_type *dev_handler)
+{
+ int res = 0;
+
+ if (dev_handler->parse == NULL) {
+ PRINT_ERROR("scst dev handler %s must have "
+ "parse() method.", dev_handler->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (((dev_handler->add_device != NULL) &&
+ (dev_handler->del_device == NULL)) ||
+ ((dev_handler->add_device == NULL) &&
+ (dev_handler->del_device != NULL))) {
+ PRINT_ERROR("Dev handler %s must either define both "
+ "add_device() and del_device(), or none.",
+ dev_handler->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (dev_handler->alloc_data_buf == NULL)
+ dev_handler->alloc_data_buf_atomic = 1;
+
+ if (dev_handler->dev_done == NULL)
+ dev_handler->dev_done_atomic = 1;
+
+out:
+ return res;
+}
+
+static int scst_check_device_name(const char *dev_name)
+{
+ int res = 0;
+
+ if (strchr(dev_name, '/') != NULL) {
+ PRINT_ERROR("Dev name %s contains illegal character '/'",
+ dev_name);
+ res = -EINVAL;
+ }
+ return res;
+}
+
+/**
+ * scst_register_virtual_device() - register a virtual device.
+ * @dev_handler: the device's device handler
+ * @dev_name: the new device name, NULL-terminated string. Must be uniq
+ * among all virtual devices in the system.
+ *
+ * Registers a virtual device and returns assinged to the device ID on
+ * success, or negative value otherwise
+ */
+int scst_register_virtual_device(struct scst_dev_type *dev_handler,
+ const char *dev_name)
+{
+ int res, rc;
+ struct scst_device *dev, *d;
+ bool sysfs_del = false;
+
+ if (dev_handler == NULL) {
+ PRINT_ERROR("%s: valid device handler must be supplied",
+ __func__);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (dev_name == NULL) {
+ PRINT_ERROR("%s: device name must be non-NULL", __func__);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_check_device_name(dev_name);
+ if (res != 0)
+ goto out;
+
+ res = scst_dev_handler_check(dev_handler);
+ if (res != 0)
+ goto out;
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ res = scst_alloc_device(GFP_KERNEL, &dev);
+ if (res != 0)
+ goto out_unlock;
+
+ dev->type = dev_handler->type;
+ dev->scsi_dev = NULL;
+ dev->virt_name = kstrdup(dev_name, GFP_KERNEL);
+ if (dev->virt_name == NULL) {
+ PRINT_ERROR("Unable to allocate virt_name for dev %s",
+ dev_name);
+ res = -ENOMEM;
+ goto out_free_dev;
+ }
+
+ while (1) {
+ dev->virt_id = scst_virt_dev_last_id++;
+ if (dev->virt_id > 0)
+ break;
+ scst_virt_dev_last_id = 1;
+ }
+
+ res = dev->virt_id;
+
+ rc = scst_pr_init_dev(dev);
+ if (rc != 0) {
+ res = rc;
+ goto out_free_dev;
+ }
+
+ /*
+ * We can drop scst_mutex, because we have not yet added the dev in
+ * scst_dev_list, so it "doesn't exist" yet.
+ */
+ mutex_unlock(&scst_mutex);
+
+ res = scst_dev_sysfs_create(dev);
+ if (res != 0)
+ goto out_lock_pr_clear_dev;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (strcmp(d->virt_name, dev_name) == 0) {
+ PRINT_ERROR("Device %s already exists", dev_name);
+ res = -EEXIST;
+ sysfs_del = true;
+ goto out_pr_clear_dev;
+ }
+ }
+
+ rc = scst_assign_dev_handler(dev, dev_handler);
+ if (rc != 0) {
+ res = rc;
+ sysfs_del = true;
+ goto out_pr_clear_dev;
+ }
+
+ list_add_tail(&dev->dev_list_entry, &scst_dev_list);
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ res = dev->virt_id;
+
+ PRINT_INFO("Attached to virtual device %s (id %d)",
+ dev_name, res);
+
+out:
+ return res;
+
+out_lock_pr_clear_dev:
+ mutex_lock(&scst_mutex);
+
+out_pr_clear_dev:
+ scst_pr_clear_dev(dev);
+
+out_free_dev:
+ mutex_unlock(&scst_mutex);
+ if (sysfs_del)
+ scst_dev_sysfs_del(dev);
+ scst_free_device(dev);
+ goto out_resume;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_register_virtual_device);
+
+/**
+ * scst_unregister_virtual_device() - unegister a virtual device.
+ * @id: the device's ID, returned by the registration function
+ */
+void scst_unregister_virtual_device(int id)
+{
+ struct scst_device *d, *dev = NULL;
+ struct scst_acg_dev *acg_dev, *aa;
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (d->virt_id == id) {
+ dev = d;
+ TRACE_DBG("Virtual device %p (id %d) found", dev, id);
+ break;
+ }
+ }
+ if (dev == NULL) {
+ PRINT_ERROR("Virtual device (id %d) not found", id);
+ goto out_unlock;
+ }
+
+ list_del(&dev->dev_list_entry);
+
+ scst_pr_clear_dev(dev);
+
+ scst_assign_dev_handler(dev, &scst_null_devtype);
+
+ list_for_each_entry_safe(acg_dev, aa, &dev->dev_acg_dev_list,
+ dev_acg_dev_list_entry) {
+ scst_acg_del_lun(acg_dev->acg, acg_dev->lun, true);
+ }
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ scst_dev_sysfs_del(dev);
+
+ PRINT_INFO("Detached from virtual device %s (id %d)",
+ dev->virt_name, dev->virt_id);
+
+ scst_free_device(dev);
+
+out:
+ return;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_virtual_device);
+
+/**
+ * __scst_register_dev_driver() - register pass-through dev handler driver
+ * @dev_type: dev handler template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the dev handler use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a pass-through dev handler driver. Returns 0 on success
+ * or appropriate error code otherwise.
+ */
+int __scst_register_dev_driver(struct scst_dev_type *dev_type,
+ const char *version)
+{
+ int res, exist;
+ struct scst_dev_type *dt;
+
+ if (strcmp(version, SCST_INTERFACE_VERSION) != 0) {
+ PRINT_ERROR("Incorrect version of dev handler %s",
+ dev_type->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_dev_handler_check(dev_type);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ exist = 0;
+ list_for_each_entry(dt, &scst_dev_type_list, dev_type_list_entry) {
+ if (strcmp(dt->name, dev_type->name) == 0) {
+ PRINT_ERROR("Device type handler \"%s\" already "
+ "exist", dt->name);
+ exist = 1;
+ break;
+ }
+ }
+ if (exist)
+ goto out_unlock;
+
+ list_add_tail(&dev_type->dev_type_list_entry, &scst_dev_type_list);
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_devt_sysfs_create(dev_type);
+ if (res < 0)
+ goto out;
+
+ PRINT_INFO("Device handler \"%s\" for type %d registered "
+ "successfully", dev_type->name, dev_type->type);
+
+out:
+ return res;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(__scst_register_dev_driver);
+
+/**
+ * scst_unregister_dev_driver() - unregister pass-through dev handler driver
+ */
+void scst_unregister_dev_driver(struct scst_dev_type *dev_type)
+{
+ struct scst_device *dev;
+ struct scst_dev_type *dt;
+ int found = 0;
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(dt, &scst_dev_type_list, dev_type_list_entry) {
+ if (strcmp(dt->name, dev_type->name) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found) {
+ PRINT_ERROR("Dev handler \"%s\" isn't registered",
+ dev_type->name);
+ goto out_up;
+ }
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ if (dev->handler == dev_type) {
+ scst_assign_dev_handler(dev, &scst_null_devtype);
+ TRACE_DBG("Dev handler removed from device %p", dev);
+ }
+ }
+
+ list_del(&dev_type->dev_type_list_entry);
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ scst_devt_sysfs_del(dev_type);
+
+ PRINT_INFO("Device handler \"%s\" for type %d unloaded",
+ dev_type->name, dev_type->type);
+
+out:
+ return;
+
+out_up:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_dev_driver);
+
+/**
+ * __scst_register_virtual_dev_driver() - register virtual dev handler driver
+ * @dev_type: dev handler template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the dev handler use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a virtual dev handler driver. Returns 0 on success or
+ * appropriate error code otherwise.
+ */
+int __scst_register_virtual_dev_driver(struct scst_dev_type *dev_type,
+ const char *version)
+{
+ int res;
+
+ if (strcmp(version, SCST_INTERFACE_VERSION) != 0) {
+ PRINT_ERROR("Incorrect version of virtual dev handler %s",
+ dev_type->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_dev_handler_check(dev_type);
+ if (res != 0)
+ goto out;
+
+ mutex_lock(&scst_mutex);
+ list_add_tail(&dev_type->dev_type_list_entry, &scst_virtual_dev_type_list);
+ mutex_unlock(&scst_mutex);
+
+ res = scst_devt_sysfs_create(dev_type);
+ if (res < 0)
+ goto out;
+
+ if (dev_type->type != -1) {
+ PRINT_INFO("Virtual device handler %s for type %d "
+ "registered successfully", dev_type->name,
+ dev_type->type);
+ } else {
+ PRINT_INFO("Virtual device handler \"%s\" registered "
+ "successfully", dev_type->name);
+ }
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(__scst_register_virtual_dev_driver);
+
+/**
+ * scst_unregister_virtual_dev_driver() - unregister virtual dev driver
+ */
+void scst_unregister_virtual_dev_driver(struct scst_dev_type *dev_type)
+{
+
+ mutex_lock(&scst_mutex);
+
+ /* Disable sysfs mgmt calls (e.g. addition of new devices) */
+ list_del(&dev_type->dev_type_list_entry);
+
+ /* Wait for outstanding sysfs mgmt calls completed */
+ while (dev_type->devt_active_sysfs_works_count > 0) {
+ mutex_unlock(&scst_mutex);
+ msleep(100);
+ mutex_lock(&scst_mutex);
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ scst_devt_sysfs_del(dev_type);
+
+ PRINT_INFO("Device handler \"%s\" unloaded", dev_type->name);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_virtual_dev_driver);
+
+/* scst_mutex supposed to be held */
+int scst_add_threads(struct scst_cmd_threads *cmd_threads,
+ struct scst_device *dev, struct scst_tgt_dev *tgt_dev, int num)
+{
+ int res = 0, i;
+ struct scst_cmd_thread_t *thr;
+ int n = 0, tgt_dev_num = 0;
+
+ if (num == 0) {
+ res = 0;
+ goto out;
+ }
+
+ list_for_each_entry(thr, &cmd_threads->threads_list, thread_list_entry) {
+ n++;
+ }
+
+ TRACE_DBG("cmd_threads %p, dev %p, tgt_dev %p, num %d, n %d",
+ cmd_threads, dev, tgt_dev, num, n);
+
+ if (tgt_dev != NULL) {
+ struct scst_tgt_dev *t;
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (t == tgt_dev)
+ break;
+ tgt_dev_num++;
+ }
+ }
+
+ for (i = 0; i < num; i++) {
+ thr = kmalloc(sizeof(*thr), GFP_KERNEL);
+ if (!thr) {
+ res = -ENOMEM;
+ PRINT_ERROR("Fail to allocate thr %d", res);
+ goto out_wait;
+ }
+
+ if (dev != NULL) {
+ char nm[14]; /* to limit the name's len */
+ strlcpy(nm, dev->virt_name, ARRAY_SIZE(nm));
+ thr->cmd_thread = kthread_create(scst_cmd_thread,
+ cmd_threads, "%s%d", nm, n++);
+ } else if (tgt_dev != NULL) {
+ char nm[11]; /* to limit the name's len */
+ strlcpy(nm, tgt_dev->dev->virt_name, ARRAY_SIZE(nm));
+ thr->cmd_thread = kthread_create(scst_cmd_thread,
+ cmd_threads, "%s%d_%d", nm, tgt_dev_num, n++);
+ } else
+ thr->cmd_thread = kthread_create(scst_cmd_thread,
+ cmd_threads, "scstd%d", n++);
+
+ if (IS_ERR(thr->cmd_thread)) {
+ res = PTR_ERR(thr->cmd_thread);
+ PRINT_ERROR("kthread_create() failed: %d", res);
+ kfree(thr);
+ goto out_wait;
+ }
+
+ list_add(&thr->thread_list_entry, &cmd_threads->threads_list);
+ cmd_threads->nr_threads++;
+
+ TRACE_DBG("Added thr %p to threads list (nr_threads %d, n %d)",
+ thr, cmd_threads->nr_threads, n);
+
+ wake_up_process(thr->cmd_thread);
+ }
+
+out_wait:
+ if (cmd_threads != &scst_main_cmd_threads) {
+ /*
+ * Wait for io_context gets initialized to avoid possible races
+ * for it from the sharing it tgt_devs.
+ */
+ while (!*(volatile bool*)&cmd_threads->io_context_ready) {
+ TRACE_DBG("Waiting for io_context for cmd_threads %p "
+ "initialized", cmd_threads);
+ msleep(50);
+ }
+ }
+
+ if (res != 0)
+ scst_del_threads(cmd_threads, i);
+
+out:
+ return res;
+}
+
+/* scst_mutex supposed to be held */
+void scst_del_threads(struct scst_cmd_threads *cmd_threads, int num)
+{
+ struct scst_cmd_thread_t *ct, *tmp;
+
+ if (num == 0)
+ goto out;
+
+ list_for_each_entry_safe_reverse(ct, tmp, &cmd_threads->threads_list,
+ thread_list_entry) {
+ int rc;
+ struct scst_device *dev;
+
+ rc = kthread_stop(ct->cmd_thread);
+ if (rc < 0)
+ TRACE_MGMT_DBG("kthread_stop() failed: %d", rc);
+
+ list_del(&ct->thread_list_entry);
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_del_thr_data(tgt_dev, ct->cmd_thread);
+ }
+ }
+
+ kfree(ct);
+
+ cmd_threads->nr_threads--;
+
+ --num;
+ if (num == 0)
+ break;
+ }
+
+ EXTRACHECKS_BUG_ON((cmd_threads->nr_threads == 0) &&
+ (cmd_threads->io_context != NULL));
+
+out:
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_stop_dev_threads(struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_tgt_dev_stop_threads(tgt_dev);
+ }
+
+ if ((dev->threads_num > 0) &&
+ (dev->threads_pool_type == SCST_THREADS_POOL_SHARED))
+ scst_del_threads(&dev->dev_cmd_threads, -1);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_create_dev_threads(struct scst_device *dev)
+{
+ int res = 0;
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ res = scst_tgt_dev_setup_threads(tgt_dev);
+ if (res != 0)
+ goto out_err;
+ }
+
+ if ((dev->threads_num > 0) &&
+ (dev->threads_pool_type == SCST_THREADS_POOL_SHARED)) {
+ res = scst_add_threads(&dev->dev_cmd_threads, dev, NULL,
+ dev->threads_num);
+ if (res != 0)
+ goto out_err;
+ }
+
+out:
+ return res;
+
+out_err:
+ scst_stop_dev_threads(dev);
+ goto out;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_assign_dev_handler(struct scst_device *dev,
+ struct scst_dev_type *handler)
+{
+ int res = 0;
+ struct scst_tgt_dev *tgt_dev;
+ LIST_HEAD(attached_tgt_devs);
+
+ BUG_ON(handler == NULL);
+
+ if (dev->handler == handler)
+ goto out;
+
+ if (dev->handler == NULL)
+ goto assign;
+
+ if (dev->handler->detach_tgt) {
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ TRACE_DBG("Calling dev handler's detach_tgt(%p)",
+ tgt_dev);
+ dev->handler->detach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's detach_tgt() returned");
+ }
+ }
+
+ /*
+ * devt_dev sysfs must be created AFTER attach() and deleted BEFORE
+ * detach() to avoid calls from sysfs for not yet ready or already dead
+ * objects.
+ */
+ scst_devt_dev_sysfs_del(dev);
+
+ if (dev->handler->detach) {
+ TRACE_DBG("%s", "Calling dev handler's detach()");
+ dev->handler->detach(dev);
+ TRACE_DBG("%s", "Old handler's detach() returned");
+ }
+
+ scst_stop_dev_threads(dev);
+
+assign:
+ dev->handler = handler;
+
+ if (handler == NULL)
+ goto out;
+
+ dev->threads_num = handler->threads_num;
+ dev->threads_pool_type = handler->threads_pool_type;
+
+ if (handler->attach) {
+ TRACE_DBG("Calling new dev handler's attach(%p)", dev);
+ res = handler->attach(dev);
+ TRACE_DBG("New dev handler's attach() returned %d", res);
+ if (res != 0) {
+ PRINT_ERROR("New device handler's %s attach() "
+ "failed: %d", handler->name, res);
+ goto out;
+ }
+ }
+
+ res = scst_devt_dev_sysfs_create(dev);
+ if (res != 0)
+ goto out_detach;
+
+ if (handler->attach_tgt) {
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ TRACE_DBG("Calling dev handler's attach_tgt(%p)",
+ tgt_dev);
+ res = handler->attach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's attach_tgt() returned");
+ if (res != 0) {
+ PRINT_ERROR("Device handler's %s attach_tgt() "
+ "failed: %d", handler->name, res);
+ goto out_err_remove_sysfs;
+ }
+ list_add_tail(&tgt_dev->extra_tgt_dev_list_entry,
+ &attached_tgt_devs);
+ }
+ }
+
+ res = scst_create_dev_threads(dev);
+ if (res != 0)
+ goto out_err_detach_tgt;
+
+out:
+ return res;
+
+out_err_detach_tgt:
+ if (handler && handler->detach_tgt) {
+ list_for_each_entry(tgt_dev, &attached_tgt_devs,
+ extra_tgt_dev_list_entry) {
+ TRACE_DBG("Calling handler's detach_tgt(%p)",
+ tgt_dev);
+ handler->detach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Handler's detach_tgt() returned");
+ }
+ }
+
+out_err_remove_sysfs:
+ scst_devt_dev_sysfs_del(dev);
+
+out_detach:
+ if (handler && handler->detach) {
+ TRACE_DBG("%s", "Calling handler's detach()");
+ handler->detach(dev);
+ TRACE_DBG("%s", "Handler's detach() returned");
+ }
+
+ dev->handler = &scst_null_devtype;
+ dev->threads_num = scst_null_devtype.threads_num;
+ dev->threads_pool_type = scst_null_devtype.threads_pool_type;
+ goto out;
+}
+
+/**
+ * scst_init_threads() - initialize SCST processing threads pool
+ *
+ * Initializes scst_cmd_threads structure
+ */
+void scst_init_threads(struct scst_cmd_threads *cmd_threads)
+{
+
+ spin_lock_init(&cmd_threads->cmd_list_lock);
+ INIT_LIST_HEAD(&cmd_threads->active_cmd_list);
+ init_waitqueue_head(&cmd_threads->cmd_list_waitQ);
+ INIT_LIST_HEAD(&cmd_threads->threads_list);
+
+ mutex_lock(&scst_suspend_mutex);
+ list_add_tail(&cmd_threads->lists_list_entry,
+ &scst_cmd_threads_list);
+ mutex_unlock(&scst_suspend_mutex);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_init_threads);
+
+/**
+ * scst_deinit_threads() - deinitialize SCST processing threads pool
+ *
+ * Deinitializes scst_cmd_threads structure
+ */
+void scst_deinit_threads(struct scst_cmd_threads *cmd_threads)
+{
+
+ mutex_lock(&scst_suspend_mutex);
+ list_del(&cmd_threads->lists_list_entry);
+ mutex_unlock(&scst_suspend_mutex);
+
+ BUG_ON(cmd_threads->io_context);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_deinit_threads);
+
+static void scst_stop_global_threads(void)
+{
+
+ mutex_lock(&scst_mutex);
+
+ scst_del_threads(&scst_main_cmd_threads, -1);
+
+ if (scst_mgmt_cmd_thread)
+ kthread_stop(scst_mgmt_cmd_thread);
+ if (scst_mgmt_thread)
+ kthread_stop(scst_mgmt_thread);
+ if (scst_init_cmd_thread)
+ kthread_stop(scst_init_cmd_thread);
+
+ mutex_unlock(&scst_mutex);
+ return;
+}
+
+/* It does NOT stop ran threads on error! */
+static int scst_start_global_threads(int num)
+{
+ int res;
+
+ mutex_lock(&scst_mutex);
+
+ res = scst_add_threads(&scst_main_cmd_threads, NULL, NULL, num);
+ if (res < 0)
+ goto out_unlock;
+
+ scst_init_cmd_thread = kthread_run(scst_init_thread,
+ NULL, "scst_initd");
+ if (IS_ERR(scst_init_cmd_thread)) {
+ res = PTR_ERR(scst_init_cmd_thread);
+ PRINT_ERROR("kthread_create() for init cmd failed: %d", res);
+ scst_init_cmd_thread = NULL;
+ goto out_unlock;
+ }
+
+ scst_mgmt_cmd_thread = kthread_run(scst_tm_thread,
+ NULL, "scsi_tm");
+ if (IS_ERR(scst_mgmt_cmd_thread)) {
+ res = PTR_ERR(scst_mgmt_cmd_thread);
+ PRINT_ERROR("kthread_create() for TM failed: %d", res);
+ scst_mgmt_cmd_thread = NULL;
+ goto out_unlock;
+ }
+
+ scst_mgmt_thread = kthread_run(scst_global_mgmt_thread,
+ NULL, "scst_mgmtd");
+ if (IS_ERR(scst_mgmt_thread)) {
+ res = PTR_ERR(scst_mgmt_thread);
+ PRINT_ERROR("kthread_create() for mgmt failed: %d", res);
+ scst_mgmt_thread = NULL;
+ goto out_unlock;
+ }
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+/**
+ * scst_get() - increase global SCST ref counter
+ *
+ * Increases global SCST ref counter that prevents from entering into suspended
+ * activities stage, so protects from any global management operations.
+ */
+void scst_get(void)
+{
+ __scst_get(0);
+}
+EXPORT_SYMBOL(scst_get);
+
+/**
+ * scst_put() - decrease global SCST ref counter
+ *
+ * Decreses global SCST ref counter that prevents from entering into suspended
+ * activities stage, so protects from any global management operations. On
+ * zero, if suspending activities is waiting, they will be suspended.
+ */
+void scst_put(void)
+{
+ __scst_put();
+}
+EXPORT_SYMBOL(scst_put);
+
+/**
+ * scst_get_setup_id() - return SCST setup ID
+ *
+ * Returns SCST setup ID. This ID can be used for multiple
+ * setups with the same configuration.
+ */
+unsigned int scst_get_setup_id(void)
+{
+ return scst_setup_id;
+}
+EXPORT_SYMBOL_GPL(scst_get_setup_id);
+
+static int scst_add(struct device *cdev, struct class_interface *intf)
+{
+ struct scsi_device *scsidp;
+ int res = 0;
+
+ scsidp = to_scsi_device(cdev->parent);
+
+ if ((scsidp->host->hostt->name == NULL) ||
+ (strcmp(scsidp->host->hostt->name, SCST_LOCAL_NAME) != 0))
+ res = scst_register_device(scsidp);
+ return res;
+}
+
+static void scst_remove(struct device *cdev, struct class_interface *intf)
+{
+ struct scsi_device *scsidp;
+
+ scsidp = to_scsi_device(cdev->parent);
+
+ if ((scsidp->host->hostt->name == NULL) ||
+ (strcmp(scsidp->host->hostt->name, SCST_LOCAL_NAME) != 0))
+ scst_unregister_device(scsidp);
+ return;
+}
+
+static struct class_interface scst_interface = {
+ .add_dev = scst_add,
+ .remove_dev = scst_remove,
+};
+
+static void __init scst_print_config(void)
+{
+ char buf[128];
+ int i, j;
+
+ i = snprintf(buf, sizeof(buf), "Enabled features: ");
+ j = i;
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ i += snprintf(&buf[i], sizeof(buf) - i, "STRICT_SERIALIZING");
+#endif
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sEXTRACHECKS",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_TRACING
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sTRACING",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_TM
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_TM",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_RETRY",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_OOM
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_OOM",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_SN
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_SN",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sUSE_EXPECTED_VALUES",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ i += snprintf(&buf[i], sizeof(buf) - i,
+ "%sTEST_IO_IN_SIRQ",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_STRICT_SECURITY
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sSTRICT_SECURITY",
+ (j == i) ? "" : ", ");
+#endif
+
+ if (j != i)
+ PRINT_INFO("%s", buf);
+}
+
+static int __init init_scst(void)
+{
+ int res, i;
+ int scst_num_cpus;
+
+ {
+ struct scsi_sense_hdr *shdr;
+ BUILD_BUG_ON(SCST_SENSE_BUFFERSIZE < sizeof(*shdr));
+ }
+ {
+ struct scst_tgt_dev *t;
+ struct scst_cmd *c;
+ BUILD_BUG_ON(sizeof(t->curr_sn) != sizeof(t->expected_sn));
+ BUILD_BUG_ON(sizeof(c->sn) != sizeof(t->expected_sn));
+ }
+
+ mutex_init(&scst_mutex);
+ mutex_init(&scst_mutex2);
+ INIT_LIST_HEAD(&scst_template_list);
+ INIT_LIST_HEAD(&scst_dev_list);
+ INIT_LIST_HEAD(&scst_dev_type_list);
+ INIT_LIST_HEAD(&scst_virtual_dev_type_list);
+ spin_lock_init(&scst_main_lock);
+ spin_lock_init(&scst_init_lock);
+ init_waitqueue_head(&scst_init_cmd_list_waitQ);
+ INIT_LIST_HEAD(&scst_init_cmd_list);
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ scst_trace_flag = SCST_DEFAULT_LOG_FLAGS;
+#endif
+ atomic_set(&scst_cmd_count, 0);
+ spin_lock_init(&scst_mcmd_lock);
+ INIT_LIST_HEAD(&scst_active_mgmt_cmd_list);
+ INIT_LIST_HEAD(&scst_delayed_mgmt_cmd_list);
+ init_waitqueue_head(&scst_mgmt_cmd_list_waitQ);
+ init_waitqueue_head(&scst_mgmt_waitQ);
+ spin_lock_init(&scst_mgmt_lock);
+ INIT_LIST_HEAD(&scst_sess_init_list);
+ INIT_LIST_HEAD(&scst_sess_shut_list);
+ init_waitqueue_head(&scst_dev_cmd_waitQ);
+ mutex_init(&scst_suspend_mutex);
+ INIT_LIST_HEAD(&scst_cmd_threads_list);
+
+ scst_init_threads(&scst_main_cmd_threads);
+
+ res = scst_lib_init();
+ if (res != 0)
+ goto out_deinit_threads;
+
+ scst_num_cpus = num_online_cpus();
+
+ /* ToDo: register_cpu_notifier() */
+
+ if (scst_threads == 0)
+ scst_threads = scst_num_cpus;
+
+ if (scst_threads < 1) {
+ PRINT_ERROR("%s", "scst_threads can not be less than 1");
+ scst_threads = scst_num_cpus;
+ }
+
+#define INIT_CACHEP(p, s, o) do { \
+ p = KMEM_CACHE(s, SCST_SLAB_FLAGS); \
+ TRACE_MEM("Slab create: %s at %p size %zd", #s, p, \
+ sizeof(struct s)); \
+ if (p == NULL) { \
+ res = -ENOMEM; \
+ goto o; \
+ } \
+ } while (0)
+
+ INIT_CACHEP(scst_mgmt_cachep, scst_mgmt_cmd, out_lib_exit);
+ INIT_CACHEP(scst_mgmt_stub_cachep, scst_mgmt_cmd_stub,
+ out_destroy_mgmt_cache);
+ INIT_CACHEP(scst_ua_cachep, scst_tgt_dev_UA,
+ out_destroy_mgmt_stub_cache);
+ {
+ struct scst_sense { uint8_t s[SCST_SENSE_BUFFERSIZE]; };
+ INIT_CACHEP(scst_sense_cachep, scst_sense,
+ out_destroy_ua_cache);
+ }
+ INIT_CACHEP(scst_aen_cachep, scst_aen, out_destroy_sense_cache);
+ INIT_CACHEP(scst_cmd_cachep, scst_cmd, out_destroy_aen_cache);
+ INIT_CACHEP(scst_sess_cachep, scst_session, out_destroy_cmd_cache);
+ INIT_CACHEP(scst_tgtd_cachep, scst_tgt_dev, out_destroy_sess_cache);
+ INIT_CACHEP(scst_acgd_cachep, scst_acg_dev, out_destroy_tgt_cache);
+
+ scst_mgmt_mempool = mempool_create(64, mempool_alloc_slab,
+ mempool_free_slab, scst_mgmt_cachep);
+ if (scst_mgmt_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_acg_cache;
+ }
+
+ /*
+ * All mgmt stubs, UAs and sense buffers are bursty and loosing them
+ * may have fatal consequences, so let's have big pools for them.
+ */
+
+ scst_mgmt_stub_mempool = mempool_create(1024, mempool_alloc_slab,
+ mempool_free_slab, scst_mgmt_stub_cachep);
+ if (scst_mgmt_stub_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_mgmt_mempool;
+ }
+
+ scst_ua_mempool = mempool_create(512, mempool_alloc_slab,
+ mempool_free_slab, scst_ua_cachep);
+ if (scst_ua_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_mgmt_stub_mempool;
+ }
+
+ scst_sense_mempool = mempool_create(1024, mempool_alloc_slab,
+ mempool_free_slab, scst_sense_cachep);
+ if (scst_sense_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_ua_mempool;
+ }
+
+ scst_aen_mempool = mempool_create(100, mempool_alloc_slab,
+ mempool_free_slab, scst_aen_cachep);
+ if (scst_aen_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_sense_mempool;
+ }
+
+ res = scst_sysfs_init();
+ if (res != 0)
+ goto out_destroy_aen_mempool;
+
+ if (scst_max_cmd_mem == 0) {
+ struct sysinfo si;
+ si_meminfo(&si);
+#if BITS_PER_LONG == 32
+ scst_max_cmd_mem = min(
+ (((uint64_t)(si.totalram - si.totalhigh) << PAGE_SHIFT)
+ >> 20) >> 2, (uint64_t)1 << 30);
+#else
+ scst_max_cmd_mem = (((si.totalram - si.totalhigh) << PAGE_SHIFT)
+ >> 20) >> 2;
+#endif
+ }
+
+ if (scst_max_dev_cmd_mem != 0) {
+ if (scst_max_dev_cmd_mem > scst_max_cmd_mem) {
+ PRINT_ERROR("scst_max_dev_cmd_mem (%d) > "
+ "scst_max_cmd_mem (%d)",
+ scst_max_dev_cmd_mem,
+ scst_max_cmd_mem);
+ scst_max_dev_cmd_mem = scst_max_cmd_mem;
+ }
+ } else
+ scst_max_dev_cmd_mem = scst_max_cmd_mem * 2 / 5;
+
+ res = scst_sgv_pools_init(
+ ((uint64_t)scst_max_cmd_mem << 10) >> (PAGE_SHIFT - 10), 0);
+ if (res != 0)
+ goto out_sysfs_cleanup;
+
+ res = scsi_register_interface(&scst_interface);
+ if (res != 0)
+ goto out_destroy_sgv_pool;
+
+ for (i = 0; i < (int)ARRAY_SIZE(scst_tasklets); i++) {
+ spin_lock_init(&scst_tasklets[i].tasklet_lock);
+ INIT_LIST_HEAD(&scst_tasklets[i].tasklet_cmd_list);
+ tasklet_init(&scst_tasklets[i].tasklet,
+ (void *)scst_cmd_tasklet,
+ (unsigned long)&scst_tasklets[i]);
+ }
+
+ TRACE_DBG("%d CPUs found, starting %d threads", scst_num_cpus,
+ scst_threads);
+
+ res = scst_start_global_threads(scst_threads);
+ if (res < 0)
+ goto out_thread_free;
+
+ res = scst_pr_check_pr_path();
+ if (res != 0)
+ goto out_thread_free;
+
+ PRINT_INFO("SCST version %s loaded successfully (max mem for "
+ "commands %dMB, per device %dMB)", SCST_VERSION_STRING,
+ scst_max_cmd_mem, scst_max_dev_cmd_mem);
+
+ scst_print_config();
+
+out:
+ return res;
+
+out_thread_free:
+ scst_stop_global_threads();
+
+ scsi_unregister_interface(&scst_interface);
+
+out_destroy_sgv_pool:
+ scst_sgv_pools_deinit();
+
+out_sysfs_cleanup:
+ scst_sysfs_cleanup();
+
+out_destroy_aen_mempool:
+ mempool_destroy(scst_aen_mempool);
+
+out_destroy_sense_mempool:
+ mempool_destroy(scst_sense_mempool);
+
+out_destroy_ua_mempool:
+ mempool_destroy(scst_ua_mempool);
+
+out_destroy_mgmt_stub_mempool:
+ mempool_destroy(scst_mgmt_stub_mempool);
+
+out_destroy_mgmt_mempool:
+ mempool_destroy(scst_mgmt_mempool);
+
+out_destroy_acg_cache:
+ kmem_cache_destroy(scst_acgd_cachep);
+
+out_destroy_tgt_cache:
+ kmem_cache_destroy(scst_tgtd_cachep);
+
+out_destroy_sess_cache:
+ kmem_cache_destroy(scst_sess_cachep);
+
+out_destroy_cmd_cache:
+ kmem_cache_destroy(scst_cmd_cachep);
+
+out_destroy_aen_cache:
+ kmem_cache_destroy(scst_aen_cachep);
+
+out_destroy_sense_cache:
+ kmem_cache_destroy(scst_sense_cachep);
+
+out_destroy_ua_cache:
+ kmem_cache_destroy(scst_ua_cachep);
+
+out_destroy_mgmt_stub_cache:
+ kmem_cache_destroy(scst_mgmt_stub_cachep);
+
+out_destroy_mgmt_cache:
+ kmem_cache_destroy(scst_mgmt_cachep);
+
+out_lib_exit:
+ scst_lib_exit();
+
+out_deinit_threads:
+ scst_deinit_threads(&scst_main_cmd_threads);
+ goto out;
+}
+
+static void __exit exit_scst(void)
+{
+
+ /* ToDo: unregister_cpu_notifier() */
+
+ scst_stop_global_threads();
+
+ scst_deinit_threads(&scst_main_cmd_threads);
+
+ scsi_unregister_interface(&scst_interface);
+
+ scst_sgv_pools_deinit();
+
+ scst_sysfs_cleanup();
+
+#define DEINIT_CACHEP(p) do { \
+ kmem_cache_destroy(p); \
+ p = NULL; \
+ } while (0)
+
+ mempool_destroy(scst_mgmt_mempool);
+ mempool_destroy(scst_mgmt_stub_mempool);
+ mempool_destroy(scst_ua_mempool);
+ mempool_destroy(scst_sense_mempool);
+ mempool_destroy(scst_aen_mempool);
+
+ DEINIT_CACHEP(scst_mgmt_cachep);
+ DEINIT_CACHEP(scst_mgmt_stub_cachep);
+ DEINIT_CACHEP(scst_ua_cachep);
+ DEINIT_CACHEP(scst_sense_cachep);
+ DEINIT_CACHEP(scst_aen_cachep);
+ DEINIT_CACHEP(scst_cmd_cachep);
+ DEINIT_CACHEP(scst_sess_cachep);
+ DEINIT_CACHEP(scst_tgtd_cachep);
+ DEINIT_CACHEP(scst_acgd_cachep);
+
+ scst_lib_exit();
+
+ PRINT_INFO("%s", "SCST unloaded");
+ return;
+}
+
+module_init(init_scst);
+module_exit(exit_scst);
+
+MODULE_AUTHOR("Vladislav Bolkhovitin");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI target core");
+MODULE_VERSION(SCST_VERSION_STRING);
diff -uprN orig/linux-2.6.35/drivers/scst/scst_module.c linux-2.6.35/drivers/scst/scst_module.c
--- orig/linux-2.6.35/drivers/scst/scst_module.c
+++ linux-2.6.35/drivers/scst/scst_module.c
@@ -0,0 +1,63 @@
+/*
+ * scst_module.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Support for loading target modules. The usage is similar to scsi_module.c
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+
+#include <scst.h>
+
+static int __init init_this_scst_driver(void)
+{
+ int res;
+
+ res = scst_register_target_template(&driver_target_template);
+ TRACE_DBG("scst_register_target_template() returned %d", res);
+ if (res < 0)
+ goto out;
+
+#ifdef SCST_REGISTER_INITIATOR_DRIVER
+ driver_template.module = THIS_MODULE;
+ scsi_register_module(MODULE_SCSI_HA, &driver_template);
+ TRACE_DBG("driver_template.present=%d",
+ driver_template.present);
+ if (driver_template.present == 0) {
+ res = -ENODEV;
+ MOD_DEC_USE_COUNT;
+ goto out;
+ }
+#endif
+
+out:
+ return res;
+}
+
+static void __exit exit_this_scst_driver(void)
+{
+
+#ifdef SCST_REGISTER_INITIATOR_DRIVER
+ scsi_unregister_module(MODULE_SCSI_HA, &driver_template);
+#endif
+
+ scst_unregister_target_template(&driver_target_template);
+ return;
+}
+
+module_init(init_this_scst_driver);
+module_exit(exit_this_scst_driver);
diff -uprN orig/linux-2.6.35/drivers/scst/scst_priv.h linux-2.6.35/drivers/scst/scst_priv.h
--- orig/linux-2.6.35/drivers/scst/scst_priv.h
+++ linux-2.6.35/drivers/scst/scst_priv.h
@@ -0,0 +1,609 @@
+/*
+ * scst_priv.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_PRIV_H
+#define __SCST_PRIV_H
+
+#include <linux/types.h>
+
+#include <scsi/scsi.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_driver.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_host.h>
+
+#define LOG_PREFIX "scst"
+
+#include <scst/scst_debug.h>
+
+#define TRACE_RTRY 0x80000000
+#define TRACE_SCSI_SERIALIZING 0x40000000
+/** top being the edge away from the interupt */
+#define TRACE_SND_TOP 0x20000000
+#define TRACE_RCV_TOP 0x01000000
+/** bottom being the edge toward the interupt */
+#define TRACE_SND_BOT 0x08000000
+#define TRACE_RCV_BOT 0x04000000
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+#define trace_flag scst_trace_flag
+extern unsigned long scst_trace_flag;
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+
+#define SCST_DEFAULT_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_MINOR | TRACE_PID | \
+ TRACE_LINE | TRACE_FUNCTION | TRACE_SPECIAL | TRACE_MGMT | \
+ TRACE_MGMT_DEBUG | TRACE_RTRY)
+
+#define TRACE_RETRY(args...) TRACE_DBG_FLAG(TRACE_RTRY, args)
+#define TRACE_SN(args...) TRACE_DBG_FLAG(TRACE_SCSI_SERIALIZING, args)
+#define TRACE_SEND_TOP(args...) TRACE_DBG_FLAG(TRACE_SND_TOP, args)
+#define TRACE_RECV_TOP(args...) TRACE_DBG_FLAG(TRACE_RCV_TOP, args)
+#define TRACE_SEND_BOT(args...) TRACE_DBG_FLAG(TRACE_SND_BOT, args)
+#define TRACE_RECV_BOT(args...) TRACE_DBG_FLAG(TRACE_RCV_BOT, args)
+
+#else /* CONFIG_SCST_DEBUG */
+
+# ifdef CONFIG_SCST_TRACING
+#define SCST_DEFAULT_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_MGMT | \
+ TRACE_SPECIAL)
+# else
+#define SCST_DEFAULT_LOG_FLAGS 0
+# endif
+
+#define TRACE_RETRY(args...)
+#define TRACE_SN(args...)
+#define TRACE_SEND_TOP(args...)
+#define TRACE_RECV_TOP(args...)
+#define TRACE_SEND_BOT(args...)
+#define TRACE_RECV_BOT(args...)
+
+#endif
+
+/**
+ ** Bits for scst_flags
+ **/
+
+/*
+ * Set if new commands initialization is being suspended for a while.
+ * Used to let TM commands execute while preparing the suspend, since
+ * RESET or ABORT could be necessary to free SCSI commands.
+ */
+#define SCST_FLAG_SUSPENDING 0
+
+/* Set if new commands initialization is suspended for a while */
+#define SCST_FLAG_SUSPENDED 1
+
+/**
+ ** Return codes for cmd state process functions. Codes are the same as
+ ** for SCST_EXEC_* to avoid translation to them and, hence, have better code.
+ **/
+#define SCST_CMD_STATE_RES_CONT_NEXT SCST_EXEC_COMPLETED
+#define SCST_CMD_STATE_RES_CONT_SAME SCST_EXEC_NOT_COMPLETED
+#define SCST_CMD_STATE_RES_NEED_THREAD (SCST_EXEC_NOT_COMPLETED+1)
+
+/**
+ ** Maximum count of uncompleted commands that an initiator could
+ ** queue on any device. Then it will start getting TASK QUEUE FULL status.
+ **/
+#define SCST_MAX_TGT_DEV_COMMANDS 48
+
+/**
+ ** Maximum count of uncompleted commands that could be queued on any device.
+ ** Then initiators sending commands to this device will start getting
+ ** TASK QUEUE FULL status.
+ **/
+#define SCST_MAX_DEV_COMMANDS 256
+
+#define SCST_TGT_RETRY_TIMEOUT (3/2*HZ)
+
+/* Definitions of symbolic constants for LUN addressing method */
+#define SCST_LUN_ADDR_METHOD_PERIPHERAL 0
+#define SCST_LUN_ADDR_METHOD_FLAT 1
+
+/* Activities suspending timeout */
+#define SCST_SUSPENDING_TIMEOUT (90 * HZ)
+
+extern struct mutex scst_mutex2;
+
+extern int scst_threads;
+
+extern unsigned int scst_max_dev_cmd_mem;
+
+extern mempool_t *scst_mgmt_mempool;
+extern mempool_t *scst_mgmt_stub_mempool;
+extern mempool_t *scst_ua_mempool;
+extern mempool_t *scst_sense_mempool;
+extern mempool_t *scst_aen_mempool;
+
+extern struct kmem_cache *scst_cmd_cachep;
+extern struct kmem_cache *scst_sess_cachep;
+extern struct kmem_cache *scst_tgtd_cachep;
+extern struct kmem_cache *scst_acgd_cachep;
+
+extern spinlock_t scst_main_lock;
+
+extern struct scst_sgv_pools scst_sgv;
+
+extern unsigned long scst_flags;
+extern atomic_t scst_cmd_count;
+extern struct list_head scst_template_list;
+extern struct list_head scst_dev_list;
+extern struct list_head scst_dev_type_list;
+extern struct list_head scst_virtual_dev_type_list;
+extern wait_queue_head_t scst_dev_cmd_waitQ;
+
+extern unsigned int scst_setup_id;
+
+extern spinlock_t scst_init_lock;
+extern struct list_head scst_init_cmd_list;
+extern wait_queue_head_t scst_init_cmd_list_waitQ;
+extern unsigned int scst_init_poll_cnt;
+
+extern struct scst_cmd_threads scst_main_cmd_threads;
+
+extern spinlock_t scst_mcmd_lock;
+/* The following lists protected by scst_mcmd_lock */
+extern struct list_head scst_active_mgmt_cmd_list;
+extern struct list_head scst_delayed_mgmt_cmd_list;
+extern wait_queue_head_t scst_mgmt_cmd_list_waitQ;
+
+struct scst_tasklet {
+ spinlock_t tasklet_lock;
+ struct list_head tasklet_cmd_list;
+ struct tasklet_struct tasklet;
+};
+extern struct scst_tasklet scst_tasklets[NR_CPUS];
+
+extern wait_queue_head_t scst_mgmt_waitQ;
+extern spinlock_t scst_mgmt_lock;
+extern struct list_head scst_sess_init_list;
+extern struct list_head scst_sess_shut_list;
+
+struct scst_cmd_thread_t {
+ struct task_struct *cmd_thread;
+ struct list_head thread_list_entry;
+};
+
+static inline bool scst_set_io_context(struct scst_cmd *cmd,
+ struct io_context **old)
+{
+ bool res;
+
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ return false;
+#endif
+
+ if (cmd->cmd_threads == &scst_main_cmd_threads) {
+ EXTRACHECKS_BUG_ON(in_interrupt());
+ /*
+ * No need for any ref counting action, because io_context
+ * supposed to be cleared in the end of the caller function.
+ */
+ current->io_context = cmd->tgt_dev->async_io_context;
+ res = true;
+ TRACE_DBG("io_context %p (tgt_dev %p)", current->io_context,
+ cmd->tgt_dev);
+ EXTRACHECKS_BUG_ON(current->io_context == NULL);
+ } else
+ res = false;
+
+ return res;
+}
+
+static inline void scst_reset_io_context(struct scst_tgt_dev *tgt_dev,
+ struct io_context *old)
+{
+ current->io_context = old;
+ TRACE_DBG("io_context %p reset", current->io_context);
+ return;
+}
+
+/*
+ * Converts string presentation of threads pool type to enum.
+ * Returns SCST_THREADS_POOL_TYPE_INVALID if the string is invalid.
+ */
+extern enum scst_dev_type_threads_pool_type scst_parse_threads_pool_type(
+ const char *p, int len);
+
+extern int scst_add_threads(struct scst_cmd_threads *cmd_threads,
+ struct scst_device *dev, struct scst_tgt_dev *tgt_dev, int num);
+extern void scst_del_threads(struct scst_cmd_threads *cmd_threads, int num);
+
+extern int scst_create_dev_threads(struct scst_device *dev);
+extern void scst_stop_dev_threads(struct scst_device *dev);
+
+extern int scst_tgt_dev_setup_threads(struct scst_tgt_dev *tgt_dev);
+extern void scst_tgt_dev_stop_threads(struct scst_tgt_dev *tgt_dev);
+
+extern bool scst_del_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct task_struct *tsk);
+
+extern struct scst_dev_type scst_null_devtype;
+
+extern struct scst_cmd *__scst_check_deferred_commands(
+ struct scst_tgt_dev *tgt_dev);
+
+/* Used to save the function call on the fast path */
+static inline struct scst_cmd *scst_check_deferred_commands(
+ struct scst_tgt_dev *tgt_dev)
+{
+ if (tgt_dev->def_cmd_count == 0)
+ return NULL;
+ else
+ return __scst_check_deferred_commands(tgt_dev);
+}
+
+static inline void scst_make_deferred_commands_active(
+ struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_cmd *c;
+
+ c = __scst_check_deferred_commands(tgt_dev);
+ if (c != NULL) {
+ TRACE_SN("Adding cmd %p to active cmd list", c);
+ spin_lock_irq(&c->cmd_threads->cmd_list_lock);
+ list_add_tail(&c->cmd_list_entry,
+ &c->cmd_threads->active_cmd_list);
+ wake_up(&c->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&c->cmd_threads->cmd_list_lock);
+ }
+
+ return;
+}
+
+void scst_inc_expected_sn(struct scst_tgt_dev *tgt_dev, atomic_t *slot);
+int scst_check_hq_cmd(struct scst_cmd *cmd);
+
+void scst_unblock_deferred(struct scst_tgt_dev *tgt_dev,
+ struct scst_cmd *cmd_sn);
+
+void scst_on_hq_cmd_response(struct scst_cmd *cmd);
+void scst_xmit_process_aborted_cmd(struct scst_cmd *cmd);
+
+int scst_cmd_thread(void *arg);
+void scst_cmd_tasklet(long p);
+int scst_init_thread(void *arg);
+int scst_tm_thread(void *arg);
+int scst_global_mgmt_thread(void *arg);
+
+void scst_zero_write_rest(struct scst_cmd *cmd);
+void scst_limit_sg_write_len(struct scst_cmd *cmd);
+void scst_adjust_resp_data_len(struct scst_cmd *cmd);
+
+int scst_queue_retry_cmd(struct scst_cmd *cmd, int finished_cmds);
+
+int scst_alloc_tgt(struct scst_tgt_template *tgtt, struct scst_tgt **tgt);
+void scst_free_tgt(struct scst_tgt *tgt);
+
+int scst_alloc_device(gfp_t gfp_mask, struct scst_device **out_dev);
+void scst_free_device(struct scst_device *dev);
+
+struct scst_acg *scst_alloc_add_acg(struct scst_tgt *tgt,
+ const char *acg_name, bool tgt_acg);
+void scst_del_free_acg(struct scst_acg *acg);
+
+struct scst_acg *scst_tgt_find_acg(struct scst_tgt *tgt, const char *name);
+struct scst_acg *scst_find_acg(const struct scst_session *sess);
+
+void scst_check_reassign_sessions(void);
+
+int scst_sess_alloc_tgt_devs(struct scst_session *sess);
+void scst_sess_free_tgt_devs(struct scst_session *sess);
+void scst_nexus_loss(struct scst_tgt_dev *tgt_dev, bool queue_UA);
+
+int scst_acg_add_lun(struct scst_acg *acg, struct kobject *parent,
+ struct scst_device *dev, uint64_t lun, int read_only,
+ bool gen_scst_report_luns_changed, struct scst_acg_dev **out_acg_dev);
+int scst_acg_del_lun(struct scst_acg *acg, uint64_t lun,
+ bool gen_scst_report_luns_changed);
+
+int scst_acg_add_acn(struct scst_acg *acg, const char *name);
+void scst_del_free_acn(struct scst_acn *acn, bool reassign);
+struct scst_acn *scst_find_acn(struct scst_acg *acg, const char *name);
+
+/* The activity supposed to be suspended and scst_mutex held */
+static inline bool scst_acg_sess_is_empty(struct scst_acg *acg)
+{
+ return list_empty(&acg->acg_sess_list);
+}
+
+int scst_prepare_request_sense(struct scst_cmd *orig_cmd);
+int scst_finish_internal_cmd(struct scst_cmd *cmd);
+
+void scst_store_sense(struct scst_cmd *cmd);
+
+int scst_assign_dev_handler(struct scst_device *dev,
+ struct scst_dev_type *handler);
+
+struct scst_session *scst_alloc_session(struct scst_tgt *tgt, gfp_t gfp_mask,
+ const char *initiator_name);
+void scst_free_session(struct scst_session *sess);
+void scst_free_session_callback(struct scst_session *sess);
+
+struct scst_cmd *scst_alloc_cmd(gfp_t gfp_mask);
+void scst_free_cmd(struct scst_cmd *cmd);
+static inline void scst_destroy_cmd(struct scst_cmd *cmd)
+{
+ kmem_cache_free(scst_cmd_cachep, cmd);
+ return;
+}
+
+void scst_check_retries(struct scst_tgt *tgt);
+
+int scst_alloc_space(struct scst_cmd *cmd);
+
+int scst_lib_init(void);
+void scst_lib_exit(void);
+
+int scst_get_full_buf(struct scst_cmd *cmd, uint8_t **buf);
+void scst_put_full_buf(struct scst_cmd *cmd, uint8_t *buf);
+
+__be64 scst_pack_lun(const uint64_t lun, unsigned int addr_method);
+uint64_t scst_unpack_lun(const uint8_t *lun, int len);
+
+struct scst_mgmt_cmd *scst_alloc_mgmt_cmd(gfp_t gfp_mask);
+void scst_free_mgmt_cmd(struct scst_mgmt_cmd *mcmd);
+void scst_done_cmd_mgmt(struct scst_cmd *cmd);
+
+static inline void scst_devt_cleanup(struct scst_dev_type *devt) { }
+
+int scst_sysfs_init(void);
+void scst_sysfs_cleanup(void);
+int scst_tgtt_sysfs_create(struct scst_tgt_template *tgtt);
+void scst_tgtt_sysfs_del(struct scst_tgt_template *tgtt);
+int scst_tgt_sysfs_create(struct scst_tgt *tgt);
+void scst_tgt_sysfs_prepare_put(struct scst_tgt *tgt);
+void scst_tgt_sysfs_del(struct scst_tgt *tgt);
+int scst_sess_sysfs_create(struct scst_session *sess);
+void scst_sess_sysfs_del(struct scst_session *sess);
+int scst_recreate_sess_luns_link(struct scst_session *sess);
+int scst_sgv_sysfs_create(struct sgv_pool *pool);
+void scst_sgv_sysfs_del(struct sgv_pool *pool);
+int scst_devt_sysfs_create(struct scst_dev_type *devt);
+void scst_devt_sysfs_del(struct scst_dev_type *devt);
+int scst_dev_sysfs_create(struct scst_device *dev);
+void scst_dev_sysfs_del(struct scst_device *dev);
+int scst_tgt_dev_sysfs_create(struct scst_tgt_dev *tgt_dev);
+void scst_tgt_dev_sysfs_del(struct scst_tgt_dev *tgt_dev);
+int scst_devt_dev_sysfs_create(struct scst_device *dev);
+void scst_devt_dev_sysfs_del(struct scst_device *dev);
+int scst_acg_sysfs_create(struct scst_tgt *tgt,
+ struct scst_acg *acg);
+void scst_acg_sysfs_del(struct scst_acg *acg);
+int scst_acg_dev_sysfs_create(struct scst_acg_dev *acg_dev,
+ struct kobject *parent);
+void scst_acg_dev_sysfs_del(struct scst_acg_dev *acg_dev);
+int scst_acn_sysfs_create(struct scst_acn *acn);
+void scst_acn_sysfs_del(struct scst_acn *acn);
+
+void __scst_dev_check_set_UA(struct scst_device *dev, struct scst_cmd *exclude,
+ const uint8_t *sense, int sense_len);
+static inline void scst_dev_check_set_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len)
+{
+ spin_lock_bh(&dev->dev_lock);
+ __scst_dev_check_set_UA(dev, exclude, sense, sense_len);
+ spin_unlock_bh(&dev->dev_lock);
+ return;
+}
+void scst_dev_check_set_local_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len);
+
+#define SCST_SET_UA_FLAG_AT_HEAD 1
+#define SCST_SET_UA_FLAG_GLOBAL 2
+
+void scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags);
+int scst_set_pending_UA(struct scst_cmd *cmd);
+
+void scst_report_luns_changed(struct scst_acg *acg);
+
+void scst_abort_cmd(struct scst_cmd *cmd, struct scst_mgmt_cmd *mcmd,
+ bool other_ini, bool call_dev_task_mgmt_fn);
+void scst_process_reset(struct scst_device *dev,
+ struct scst_session *originator, struct scst_cmd *exclude_cmd,
+ struct scst_mgmt_cmd *mcmd, bool setUA);
+
+bool scst_is_ua_global(const uint8_t *sense, int len);
+void scst_requeue_ua(struct scst_cmd *cmd);
+
+void scst_gen_aen_or_ua(struct scst_tgt_dev *tgt_dev,
+ int key, int asc, int ascq);
+
+static inline bool scst_is_implicit_hq(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_IMPLICIT_HQ) != 0;
+}
+
+/*
+ * Some notes on devices "blocking". Blocking means that no
+ * commands will go from SCST to underlying SCSI device until it
+ * is unblocked. But we don't care about all commands that
+ * already on the device.
+ */
+
+extern int scst_inc_on_dev_cmd(struct scst_cmd *cmd);
+
+extern void __scst_block_dev(struct scst_device *dev);
+extern void scst_block_dev_cmd(struct scst_cmd *cmd, int outstanding);
+extern void scst_unblock_dev(struct scst_device *dev);
+extern void scst_unblock_dev_cmd(struct scst_cmd *cmd);
+
+/* No locks */
+static inline void scst_dec_on_dev_cmd(struct scst_cmd *cmd)
+{
+ struct scst_device *dev = cmd->dev;
+ bool unblock_dev = cmd->inc_blocking;
+
+ if (cmd->inc_blocking) {
+ TRACE_MGMT_DBG("cmd %p (tag %llu): unblocking dev %p", cmd,
+ (long long unsigned int)cmd->tag, cmd->dev);
+ cmd->inc_blocking = 0;
+ }
+ cmd->dec_on_dev_needed = 0;
+
+ if (unblock_dev)
+ scst_unblock_dev(dev);
+
+ atomic_dec(&dev->on_dev_count);
+ /* See comment in scst_block_dev() */
+ smp_mb__after_atomic_dec();
+
+ TRACE_DBG("New on_dev_count %d", atomic_read(&dev->on_dev_count));
+
+ BUG_ON(atomic_read(&dev->on_dev_count) < 0);
+
+ if (unlikely(dev->block_count != 0))
+ wake_up_all(&dev->on_dev_waitQ);
+
+ return;
+}
+
+static inline void __scst_get(int barrier)
+{
+ atomic_inc(&scst_cmd_count);
+ TRACE_DBG("Incrementing scst_cmd_count(new value %d)",
+ atomic_read(&scst_cmd_count));
+
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ if (barrier)
+ smp_mb__after_atomic_inc();
+}
+
+static inline void __scst_put(void)
+{
+ int f;
+ f = atomic_dec_and_test(&scst_cmd_count);
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ if (f && unlikely(test_bit(SCST_FLAG_SUSPENDED, &scst_flags))) {
+ TRACE_MGMT_DBG("%s", "Waking up scst_dev_cmd_waitQ");
+ wake_up_all(&scst_dev_cmd_waitQ);
+ }
+ TRACE_DBG("Decrementing scst_cmd_count(new value %d)",
+ atomic_read(&scst_cmd_count));
+}
+
+void scst_sched_session_free(struct scst_session *sess);
+
+static inline void scst_sess_get(struct scst_session *sess)
+{
+ atomic_inc(&sess->refcnt);
+ TRACE_DBG("Incrementing sess %p refcnt (new value %d)",
+ sess, atomic_read(&sess->refcnt));
+}
+
+static inline void scst_sess_put(struct scst_session *sess)
+{
+ TRACE_DBG("Decrementing sess %p refcnt (new value %d)",
+ sess, atomic_read(&sess->refcnt)-1);
+ if (atomic_dec_and_test(&sess->refcnt))
+ scst_sched_session_free(sess);
+}
+
+static inline void __scst_cmd_get(struct scst_cmd *cmd)
+{
+ atomic_inc(&cmd->cmd_ref);
+ TRACE_DBG("Incrementing cmd %p ref (new value %d)",
+ cmd, atomic_read(&cmd->cmd_ref));
+}
+
+static inline void __scst_cmd_put(struct scst_cmd *cmd)
+{
+ TRACE_DBG("Decrementing cmd %p ref (new value %d)",
+ cmd, atomic_read(&cmd->cmd_ref)-1);
+ if (atomic_dec_and_test(&cmd->cmd_ref))
+ scst_free_cmd(cmd);
+}
+
+extern void scst_throttle_cmd(struct scst_cmd *cmd);
+extern void scst_unthrottle_cmd(struct scst_cmd *cmd);
+
+#ifdef CONFIG_SCST_DEBUG_TM
+extern void tm_dbg_check_released_cmds(void);
+extern int tm_dbg_check_cmd(struct scst_cmd *cmd);
+extern void tm_dbg_release_cmd(struct scst_cmd *cmd);
+extern void tm_dbg_task_mgmt(struct scst_device *dev, const char *fn,
+ int force);
+extern int tm_dbg_is_release(void);
+#else
+static inline void tm_dbg_check_released_cmds(void) {}
+static inline int tm_dbg_check_cmd(struct scst_cmd *cmd)
+{
+ return 0;
+}
+static inline void tm_dbg_release_cmd(struct scst_cmd *cmd) {}
+static inline void tm_dbg_task_mgmt(struct scst_device *dev, const char *fn,
+ int force) {}
+static inline int tm_dbg_is_release(void)
+{
+ return 0;
+}
+#endif /* CONFIG_SCST_DEBUG_TM */
+
+#ifdef CONFIG_SCST_DEBUG_SN
+void scst_check_debug_sn(struct scst_cmd *cmd);
+#else
+static inline void scst_check_debug_sn(struct scst_cmd *cmd) {}
+#endif
+
+static inline int scst_sn_before(uint32_t seq1, uint32_t seq2)
+{
+ return (int32_t)(seq1-seq2) < 0;
+}
+
+int gen_relative_target_port_id(uint16_t *id);
+bool scst_is_relative_target_port_id_unique(uint16_t id,
+ const struct scst_tgt *t);
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+void scst_set_start_time(struct scst_cmd *cmd);
+void scst_set_cur_start(struct scst_cmd *cmd);
+void scst_set_parse_time(struct scst_cmd *cmd);
+void scst_set_alloc_buf_time(struct scst_cmd *cmd);
+void scst_set_restart_waiting_time(struct scst_cmd *cmd);
+void scst_set_rdy_to_xfer_time(struct scst_cmd *cmd);
+void scst_set_pre_exec_time(struct scst_cmd *cmd);
+void scst_set_exec_time(struct scst_cmd *cmd);
+void scst_set_dev_done_time(struct scst_cmd *cmd);
+void scst_set_xmit_time(struct scst_cmd *cmd);
+void scst_set_tgt_on_free_time(struct scst_cmd *cmd);
+void scst_set_dev_on_free_time(struct scst_cmd *cmd);
+void scst_update_lat_stats(struct scst_cmd *cmd);
+
+#else
+
+static inline void scst_set_start_time(struct scst_cmd *cmd) {}
+static inline void scst_set_cur_start(struct scst_cmd *cmd) {}
+static inline void scst_set_parse_time(struct scst_cmd *cmd) {}
+static inline void scst_set_alloc_buf_time(struct scst_cmd *cmd) {}
+static inline void scst_set_restart_waiting_time(struct scst_cmd *cmd) {}
+static inline void scst_set_rdy_to_xfer_time(struct scst_cmd *cmd) {}
+static inline void scst_set_pre_exec_time(struct scst_cmd *cmd) {}
+static inline void scst_set_exec_time(struct scst_cmd *cmd) {}
+static inline void scst_set_dev_done_time(struct scst_cmd *cmd) {}
+static inline void scst_set_xmit_time(struct scst_cmd *cmd) {}
+static inline void scst_set_tgt_on_free_time(struct scst_cmd *cmd) {}
+static inline void scst_set_dev_on_free_time(struct scst_cmd *cmd) {}
+static inline void scst_update_lat_stats(struct scst_cmd *cmd) {}
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
+
+#endif /* __SCST_PRIV_H */
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 5/17]: SCST implementation of the SCSI target state machine
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (3 preceding siblings ...)
2010-09-14 14:40 ` [PATCH 4/17]: SCST main management files and private headers Vladislav Bolkhovitin
@ 2010-09-14 14:41 ` Vladislav Bolkhovitin
2010-09-14 14:42 ` [PATCH 6/17]: SCST internal library functions Vladislav Bolkhovitin
` (12 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:41 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains implementation of the SCSI target state machine
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_targ.c | 6318 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 6318 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/scst_targ.c linux-2.6.35/drivers/scst/scst_targ.c
--- orig/linux-2.6.35/drivers/scst/scst_targ.c
+++ linux-2.6.35/drivers/scst/scst_targ.c
@@ -0,0 +1,6318 @@
+/*
+ * scst_targ.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/smp_lock.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+#include <linux/kthread.h>
+#include <linux/delay.h>
+#include <linux/ktime.h>
+
+#include <scst/scst.h>
+#include "scst_priv.h"
+#include "scst_pres.h"
+
+#if 0 /* Temporary left for future performance investigations */
+/* Deleting it don't forget to delete write_cmd_count */
+#define CONFIG_SCST_ORDERED_READS
+#endif
+
+#if 0 /* Let's disable it for now to see if users will complain about it */
+/* Deleting it don't forget to delete write_cmd_count */
+#define CONFIG_SCST_PER_DEVICE_CMD_COUNT_LIMIT
+#endif
+
+static void scst_cmd_set_sn(struct scst_cmd *cmd);
+static int __scst_init_cmd(struct scst_cmd *cmd);
+static void scst_finish_cmd_mgmt(struct scst_cmd *cmd);
+static struct scst_cmd *__scst_find_cmd_by_tag(struct scst_session *sess,
+ uint64_t tag, bool to_abort);
+static void scst_process_redirect_cmd(struct scst_cmd *cmd,
+ enum scst_exec_context context, int check_retries);
+
+/**
+ * scst_post_parse() - do post parse actions
+ *
+ * This function must be called by dev handler after its parse() callback
+ * returned SCST_CMD_STATE_STOP before calling scst_process_active_cmd().
+ */
+void scst_post_parse(struct scst_cmd *cmd)
+{
+ scst_set_parse_time(cmd);
+}
+EXPORT_SYMBOL_GPL(scst_post_parse);
+
+/**
+ * scst_post_alloc_data_buf() - do post alloc_data_buf actions
+ *
+ * This function must be called by dev handler after its alloc_data_buf()
+ * callback returned SCST_CMD_STATE_STOP before calling
+ * scst_process_active_cmd().
+ */
+void scst_post_alloc_data_buf(struct scst_cmd *cmd)
+{
+ scst_set_alloc_buf_time(cmd);
+}
+EXPORT_SYMBOL_GPL(scst_post_alloc_data_buf);
+
+static inline void scst_schedule_tasklet(struct scst_cmd *cmd)
+{
+ struct scst_tasklet *t = &scst_tasklets[smp_processor_id()];
+ unsigned long flags;
+
+ spin_lock_irqsave(&t->tasklet_lock, flags);
+ TRACE_DBG("Adding cmd %p to tasklet %d cmd list", cmd,
+ smp_processor_id());
+ list_add_tail(&cmd->cmd_list_entry, &t->tasklet_cmd_list);
+ spin_unlock_irqrestore(&t->tasklet_lock, flags);
+
+ tasklet_schedule(&t->tasklet);
+}
+
+/**
+ * scst_rx_cmd() - create new command
+ * @sess: SCST session
+ * @lun: LUN for the command
+ * @lun_len: length of the LUN in bytes
+ * @cdb: CDB of the command
+ * @cdb_len: length of the CDB in bytes
+ * @atomic: true, if current context is atomic
+ *
+ * Description:
+ * Creates new SCST command. Returns new command on success or
+ * NULL otherwise.
+ *
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same session.
+ */
+struct scst_cmd *scst_rx_cmd(struct scst_session *sess,
+ const uint8_t *lun, int lun_len, const uint8_t *cdb,
+ unsigned int cdb_len, int atomic)
+{
+ struct scst_cmd *cmd;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(sess->shut_phase != SCST_SESS_SPH_READY)) {
+ PRINT_CRIT_ERROR("%s",
+ "New cmd while shutting down the session");
+ BUG();
+ }
+#endif
+
+ cmd = scst_alloc_cmd(atomic ? GFP_ATOMIC : GFP_KERNEL);
+ if (cmd == NULL)
+ goto out;
+
+ cmd->sess = sess;
+ cmd->tgt = sess->tgt;
+ cmd->tgtt = sess->tgt->tgtt;
+
+ cmd->lun = scst_unpack_lun(lun, lun_len);
+ if (unlikely(cmd->lun == NO_SUCH_LUN)) {
+ PRINT_ERROR("Wrong LUN %d, finishing cmd", -1);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_lun_not_supported));
+ }
+
+ /*
+ * For cdb_len 0 defer the error reporting until scst_cmd_init_done(),
+ * scst_set_cmd_error() supports nested calls.
+ */
+ if (unlikely(cdb_len > SCST_MAX_CDB_SIZE)) {
+ PRINT_ERROR("Too big CDB len %d, finishing cmd", cdb_len);
+ cdb_len = SCST_MAX_CDB_SIZE;
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ }
+
+ memcpy(cmd->cdb, cdb, cdb_len);
+ cmd->cdb_len = cdb_len;
+
+ TRACE_DBG("cmd %p, sess %p", cmd, sess);
+ scst_sess_get(sess);
+
+out:
+ return cmd;
+}
+EXPORT_SYMBOL(scst_rx_cmd);
+
+/*
+ * No locks, but might be on IRQ. Returns 0 on success, <0 if processing of
+ * this command should be stopped.
+ */
+static int scst_init_cmd(struct scst_cmd *cmd, enum scst_exec_context *context)
+{
+ int rc, res = 0;
+
+ /* See the comment in scst_do_job_init() */
+ if (unlikely(!list_empty(&scst_init_cmd_list))) {
+ TRACE_MGMT_DBG("%s", "init cmd list busy");
+ goto out_redirect;
+ }
+ /*
+ * Memory barrier isn't necessary here, because CPU appears to
+ * be self-consistent and we don't care about the race, described
+ * in comment in scst_do_job_init().
+ */
+
+ rc = __scst_init_cmd(cmd);
+ if (unlikely(rc > 0))
+ goto out_redirect;
+ else if (unlikely(rc != 0)) {
+ res = 1;
+ goto out;
+ }
+
+ EXTRACHECKS_BUG_ON(*context == SCST_CONTEXT_SAME);
+
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ scst_get_cdb_info(cmd);
+ if (cmd->op_flags & SCST_TEST_IO_IN_SIRQ_ALLOWED)
+ goto out;
+#endif
+
+ /* Small context optimization */
+ if (((*context == SCST_CONTEXT_TASKLET) ||
+ (*context == SCST_CONTEXT_DIRECT_ATOMIC)) &&
+ scst_cmd_is_expected_set(cmd)) {
+ if (cmd->expected_data_direction & SCST_DATA_WRITE) {
+ if (!test_bit(SCST_TGT_DEV_AFTER_INIT_WR_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ *context = SCST_CONTEXT_THREAD;
+ } else
+ *context = SCST_CONTEXT_THREAD;
+ }
+
+out:
+ return res;
+
+out_redirect:
+ if (cmd->preprocessing_only) {
+ /*
+ * Poor man solution for single threaded targets, where
+ * blocking receiver at least sometimes means blocking all.
+ * For instance, iSCSI target won't be able to receive
+ * Data-Out PDUs.
+ */
+ BUG_ON(*context != SCST_CONTEXT_DIRECT);
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = 1;
+ /* Keep initiator away from too many BUSY commands */
+ msleep(50);
+ } else {
+ unsigned long flags;
+ spin_lock_irqsave(&scst_init_lock, flags);
+ TRACE_MGMT_DBG("Adding cmd %p to init cmd list (scst_cmd_count "
+ "%d)", cmd, atomic_read(&scst_cmd_count));
+ list_add_tail(&cmd->cmd_list_entry, &scst_init_cmd_list);
+ if (test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))
+ scst_init_poll_cnt++;
+ spin_unlock_irqrestore(&scst_init_lock, flags);
+ wake_up(&scst_init_cmd_list_waitQ);
+ res = -1;
+ }
+ goto out;
+}
+
+/**
+ * scst_cmd_init_done() - the command's initialization done
+ * @cmd: SCST command
+ * @pref_context: preferred command execution context
+ *
+ * Description:
+ * Notifies SCST that the driver finished its part of the command
+ * initialization, and the command is ready for execution.
+ * The second argument sets preferred command execition context.
+ * See SCST_CONTEXT_* constants for details.
+ *
+ * !!IMPORTANT!!
+ *
+ * If cmd->set_sn_on_restart_cmd not set, this function, as well as
+ * scst_cmd_init_stage1_done() and scst_restart_cmd(), must not be
+ * called simultaneously for the same session (more precisely,
+ * for the same session/LUN, i.e. tgt_dev), i.e. they must be
+ * somehow externally serialized. This is needed to have lock free fast
+ * path in scst_cmd_set_sn(). For majority of targets those functions are
+ * naturally serialized by the single source of commands. Only iSCSI
+ * immediate commands with multiple connections per session seems to be an
+ * exception. For it, some mutex/lock shall be used for the serialization.
+ */
+void scst_cmd_init_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context)
+{
+ unsigned long flags;
+ struct scst_session *sess = cmd->sess;
+ int rc;
+
+ scst_set_start_time(cmd);
+
+ TRACE_DBG("Preferred context: %d (cmd %p)", pref_context, cmd);
+ TRACE(TRACE_SCSI, "tag=%llu, lun=%lld, CDB len=%d, queue_type=%x "
+ "(cmd %p)", (long long unsigned int)cmd->tag,
+ (long long unsigned int)cmd->lun, cmd->cdb_len,
+ cmd->queue_type, cmd);
+ PRINT_BUFF_FLAG(TRACE_SCSI|TRACE_RCV_BOT, "Recieving CDB",
+ cmd->cdb, cmd->cdb_len);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely((in_irq() || irqs_disabled())) &&
+ ((pref_context == SCST_CONTEXT_DIRECT) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC))) {
+ PRINT_ERROR("Wrong context %d in IRQ from target %s, use "
+ "SCST_CONTEXT_THREAD instead", pref_context,
+ cmd->tgtt->name);
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+#endif
+
+ atomic_inc(&sess->sess_cmd_count);
+
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+
+ if (unlikely(sess->init_phase != SCST_SESS_IPH_READY)) {
+ /*
+ * We must always keep commands in the sess list from the
+ * very beginning, because otherwise they can be missed during
+ * TM processing. This check is needed because there might be
+ * old, i.e. deferred, commands and new, i.e. just coming, ones.
+ */
+ if (cmd->sess_cmd_list_entry.next == NULL)
+ list_add_tail(&cmd->sess_cmd_list_entry,
+ &sess->sess_cmd_list);
+ switch (sess->init_phase) {
+ case SCST_SESS_IPH_SUCCESS:
+ break;
+ case SCST_SESS_IPH_INITING:
+ TRACE_DBG("Adding cmd %p to init deferred cmd list",
+ cmd);
+ list_add_tail(&cmd->cmd_list_entry,
+ &sess->init_deferred_cmd_list);
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ goto out;
+ case SCST_SESS_IPH_FAILED:
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ goto active;
+ default:
+ BUG();
+ }
+ } else
+ list_add_tail(&cmd->sess_cmd_list_entry,
+ &sess->sess_cmd_list);
+
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+
+ if (unlikely(cmd->cdb_len == 0)) {
+ PRINT_ERROR("%s", "Wrong CDB len 0, finishing cmd");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ scst_set_cmd_abnormal_done_state(cmd);
+ goto active;
+ }
+
+ if (unlikely(cmd->queue_type >= SCST_CMD_QUEUE_ACA)) {
+ PRINT_ERROR("Unsupported queue type %d", cmd->queue_type);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ goto active;
+ }
+
+ /*
+ * Cmd must be inited here to preserve the order. In case if cmd
+ * already preliminary completed by target driver we need to init
+ * cmd anyway to find out in which format we should return sense.
+ */
+ cmd->state = SCST_CMD_STATE_INIT;
+ rc = scst_init_cmd(cmd, &pref_context);
+ if (unlikely(rc < 0))
+ goto out;
+
+active:
+ /* Here cmd must not be in any cmd list, no locks */
+ switch (pref_context) {
+ case SCST_CONTEXT_TASKLET:
+ scst_schedule_tasklet(cmd);
+ break;
+
+ case SCST_CONTEXT_DIRECT:
+ scst_process_active_cmd(cmd, false);
+ break;
+
+ case SCST_CONTEXT_DIRECT_ATOMIC:
+ scst_process_active_cmd(cmd, true);
+ break;
+
+ default:
+ PRINT_ERROR("Context %x is undefined, using the thread one",
+ pref_context);
+ /* go through */
+ case SCST_CONTEXT_THREAD:
+ spin_lock_irqsave(&cmd->cmd_threads->cmd_list_lock, flags);
+ TRACE_DBG("Adding cmd %p to active cmd list", cmd);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irqrestore(&cmd->cmd_threads->cmd_list_lock, flags);
+ break;
+ }
+
+out:
+ return;
+}
+EXPORT_SYMBOL(scst_cmd_init_done);
+
+static int scst_pre_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+ struct scst_device *dev = cmd->dev;
+ int rc;
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ cmd->inc_expected_sn_on_done = 1;
+#else
+ cmd->inc_expected_sn_on_done = dev->handler->exec_sync ||
+ (!dev->has_own_order_mgmt &&
+ (dev->queue_alg == SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER ||
+ cmd->queue_type == SCST_CMD_QUEUE_ORDERED));
+#endif
+
+ /*
+ * Expected transfer data supplied by the SCSI transport via the
+ * target driver are untrusted, so we prefer to fetch them from CDB.
+ * Additionally, not all transports support supplying the expected
+ * transfer data.
+ */
+
+ rc = scst_get_cdb_info(cmd);
+ if (unlikely(rc != 0)) {
+ if (rc > 0) {
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_err;
+ }
+
+ EXTRACHECKS_BUG_ON(cmd->op_flags & SCST_INFO_VALID);
+
+ TRACE(TRACE_MINOR, "Unknown opcode 0x%02x for %s. "
+ "Should you update scst_scsi_op_table?",
+ cmd->cdb[0], dev->handler->name);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Failed CDB", cmd->cdb,
+ cmd->cdb_len);
+ } else {
+ EXTRACHECKS_BUG_ON(!(cmd->op_flags & SCST_INFO_VALID));
+ }
+
+ cmd->state = SCST_CMD_STATE_DEV_PARSE;
+
+ TRACE_DBG("op_name <%s> (cmd %p), direction=%d "
+ "(expected %d, set %s), bufflen=%d, out_bufflen=%d (expected "
+ "len %d, out expected len %d), flags=%d", cmd->op_name, cmd,
+ cmd->data_direction, cmd->expected_data_direction,
+ scst_cmd_is_expected_set(cmd) ? "yes" : "no",
+ cmd->bufflen, cmd->out_bufflen, cmd->expected_transfer_len,
+ cmd->expected_out_transfer_len, cmd->op_flags);
+
+out:
+ return res;
+
+out_err:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+#ifndef CONFIG_SCST_USE_EXPECTED_VALUES
+static bool scst_is_allowed_to_mismatch_cmd(struct scst_cmd *cmd)
+{
+ bool res = false;
+
+ /* VERIFY commands with BYTCHK unset shouldn't fail here */
+ if ((cmd->op_flags & SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED) &&
+ (cmd->cdb[1] & BYTCHK) == 0) {
+ res = true;
+ goto out;
+ }
+
+ switch (cmd->cdb[0]) {
+ case TEST_UNIT_READY:
+ /* Crazy VMware people sometimes do TUR with READ direction */
+ if ((cmd->expected_data_direction == SCST_DATA_READ) ||
+ (cmd->expected_data_direction == SCST_DATA_NONE))
+ res = true;
+ break;
+ }
+
+out:
+ return res;
+}
+#endif
+
+static int scst_parse_cmd(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+ int state;
+ struct scst_device *dev = cmd->dev;
+ int orig_bufflen = cmd->bufflen;
+
+ if (likely(!scst_is_cmd_fully_local(cmd))) {
+ if (unlikely(!dev->handler->parse_atomic &&
+ scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Dev handler %s parse() needs thread "
+ "context, rescheduling", dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE_DBG("Calling dev handler %s parse(%p)",
+ dev->handler->name, cmd);
+ TRACE_BUFF_FLAG(TRACE_SND_BOT, "Parsing: ",
+ cmd->cdb, cmd->cdb_len);
+ scst_set_cur_start(cmd);
+ state = dev->handler->parse(cmd);
+ /* Caution: cmd can be already dead here */
+ TRACE_DBG("Dev handler %s parse() returned %d",
+ dev->handler->name, state);
+
+ switch (state) {
+ case SCST_CMD_STATE_NEED_THREAD_CTX:
+ scst_set_parse_time(cmd);
+ TRACE_DBG("Dev handler %s parse() requested thread "
+ "context, rescheduling", dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+
+ case SCST_CMD_STATE_STOP:
+ TRACE_DBG("Dev handler %s parse() requested stop "
+ "processing", dev->handler->name);
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ goto out;
+ }
+
+ scst_set_parse_time(cmd);
+
+ if (state == SCST_CMD_STATE_DEFAULT)
+ state = SCST_CMD_STATE_PREPARE_SPACE;
+ } else
+ state = SCST_CMD_STATE_PREPARE_SPACE;
+
+ if (unlikely(state == SCST_CMD_STATE_PRE_XMIT_RESP))
+ goto set_res;
+
+ if (unlikely(!(cmd->op_flags & SCST_INFO_VALID))) {
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+ if (scst_cmd_is_expected_set(cmd)) {
+ TRACE(TRACE_MINOR, "Using initiator supplied values: "
+ "direction %d, transfer_len %d/%d",
+ cmd->expected_data_direction,
+ cmd->expected_transfer_len,
+ cmd->expected_out_transfer_len);
+ cmd->data_direction = cmd->expected_data_direction;
+ cmd->bufflen = cmd->expected_transfer_len;
+ cmd->out_bufflen = cmd->expected_out_transfer_len;
+ } else {
+ PRINT_ERROR("Unknown opcode 0x%02x for %s and "
+ "target %s not supplied expected values",
+ cmd->cdb[0], dev->handler->name, cmd->tgtt->name);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+ }
+#else
+ /*
+ * Let's ignore reporting T10/04-262r7 16-byte and 12-byte ATA
+ * pass-thru commands to not pollute logs (udev(?) checks them
+ * for some reason). If somebody has their description, please,
+ * update scst_scsi_op_table.
+ */
+ if ((cmd->cdb[0] != 0x85) && (cmd->cdb[0] != 0xa1))
+ PRINT_ERROR("Refusing unknown opcode %x", cmd->cdb[0]);
+ else
+ TRACE(TRACE_MINOR, "Refusing unknown opcode %x",
+ cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+#endif
+ }
+
+ if (unlikely(cmd->cdb_len == 0)) {
+ PRINT_ERROR("Unable to get CDB length for "
+ "opcode 0x%02x. Returning INVALID "
+ "OPCODE", cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+ }
+
+ EXTRACHECKS_BUG_ON(cmd->cdb_len == 0);
+
+ TRACE(TRACE_SCSI, "op_name <%s> (cmd %p), direction=%d "
+ "(expected %d, set %s), bufflen=%d, out_bufflen=%d, (expected "
+ "len %d, out expected len %d), flags=%x", cmd->op_name, cmd,
+ cmd->data_direction, cmd->expected_data_direction,
+ scst_cmd_is_expected_set(cmd) ? "yes" : "no",
+ cmd->bufflen, cmd->out_bufflen, cmd->expected_transfer_len,
+ cmd->expected_out_transfer_len, cmd->op_flags);
+
+ if (unlikely((cmd->op_flags & SCST_UNKNOWN_LENGTH) != 0)) {
+ if (scst_cmd_is_expected_set(cmd)) {
+ /*
+ * Command data length can't be easily
+ * determined from the CDB. ToDo, all such
+ * commands processing should be fixed. Until
+ * it's done, get the length from the supplied
+ * expected value, but limit it to some
+ * reasonable value (15MB).
+ */
+ cmd->bufflen = min(cmd->expected_transfer_len,
+ 15*1024*1024);
+ if (cmd->data_direction == SCST_DATA_BIDI)
+ cmd->out_bufflen = min(cmd->expected_out_transfer_len,
+ 15*1024*1024);
+ cmd->op_flags &= ~SCST_UNKNOWN_LENGTH;
+ } else {
+ PRINT_ERROR("Unknown data transfer length for opcode "
+ "0x%x (handler %s, target %s)", cmd->cdb[0],
+ dev->handler->name, cmd->tgtt->name);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ goto out_done;
+ }
+ }
+
+ if (unlikely(cmd->cdb[cmd->cdb_len - 1] & CONTROL_BYTE_NACA_BIT)) {
+ PRINT_ERROR("NACA bit in control byte CDB is not supported "
+ "(opcode 0x%02x)", cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_done;
+ }
+
+ if (unlikely(cmd->cdb[cmd->cdb_len - 1] & CONTROL_BYTE_LINK_BIT)) {
+ PRINT_ERROR("Linked commands are not supported "
+ "(opcode 0x%02x)", cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_done;
+ }
+
+ if (cmd->dh_data_buf_alloced &&
+ unlikely((orig_bufflen > cmd->bufflen))) {
+ PRINT_ERROR("Dev handler supplied data buffer (size %d), "
+ "is less, than required (size %d)", cmd->bufflen,
+ orig_bufflen);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_hw_error;
+ }
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((cmd->bufflen != 0) &&
+ ((cmd->data_direction == SCST_DATA_NONE) ||
+ ((cmd->sg == NULL) && (state > SCST_CMD_STATE_PREPARE_SPACE)))) {
+ PRINT_ERROR("Dev handler %s parse() returned "
+ "invalid cmd data_direction %d, bufflen %d, state %d "
+ "or sg %p (opcode 0x%x)", dev->handler->name,
+ cmd->data_direction, cmd->bufflen, state, cmd->sg,
+ cmd->cdb[0]);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_hw_error;
+ }
+#endif
+
+ if (scst_cmd_is_expected_set(cmd)) {
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+ if (unlikely((cmd->data_direction != cmd->expected_data_direction) ||
+ (cmd->bufflen != cmd->expected_transfer_len) ||
+ (cmd->out_bufflen != cmd->expected_out_transfer_len))) {
+ TRACE(TRACE_MINOR, "Expected values don't match "
+ "decoded ones: data_direction %d, "
+ "expected_data_direction %d, "
+ "bufflen %d, expected_transfer_len %d, "
+ "out_bufflen %d, expected_out_transfer_len %d",
+ cmd->data_direction,
+ cmd->expected_data_direction,
+ cmd->bufflen, cmd->expected_transfer_len,
+ cmd->out_bufflen, cmd->expected_out_transfer_len);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Suspicious CDB",
+ cmd->cdb, cmd->cdb_len);
+ cmd->data_direction = cmd->expected_data_direction;
+ cmd->bufflen = cmd->expected_transfer_len;
+ cmd->out_bufflen = cmd->expected_out_transfer_len;
+ cmd->resid_possible = 1;
+ }
+#else
+ if (unlikely(cmd->data_direction !=
+ cmd->expected_data_direction)) {
+ if (((cmd->expected_data_direction != SCST_DATA_NONE) ||
+ (cmd->bufflen != 0)) &&
+ !scst_is_allowed_to_mismatch_cmd(cmd)) {
+ PRINT_ERROR("Expected data direction %d for "
+ "opcode 0x%02x (handler %s, target %s) "
+ "doesn't match decoded value %d",
+ cmd->expected_data_direction,
+ cmd->cdb[0], dev->handler->name,
+ cmd->tgtt->name, cmd->data_direction);
+ PRINT_BUFFER("Failed CDB", cmd->cdb,
+ cmd->cdb_len);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ goto out_done;
+ }
+ }
+ if (unlikely(cmd->bufflen != cmd->expected_transfer_len)) {
+ TRACE(TRACE_MINOR, "Warning: expected "
+ "transfer length %d for opcode 0x%02x "
+ "(handler %s, target %s) doesn't match "
+ "decoded value %d. Faulty initiator "
+ "(e.g. VMware is known to be such) or "
+ "scst_scsi_op_table should be updated?",
+ cmd->expected_transfer_len, cmd->cdb[0],
+ dev->handler->name, cmd->tgtt->name,
+ cmd->bufflen);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Suspicious CDB",
+ cmd->cdb, cmd->cdb_len);
+ if ((cmd->data_direction & SCST_DATA_READ) ||
+ (cmd->data_direction & SCST_DATA_WRITE))
+ cmd->resid_possible = 1;
+ }
+ if (unlikely(cmd->out_bufflen != cmd->expected_out_transfer_len)) {
+ TRACE(TRACE_MINOR, "Warning: expected bidirectional OUT "
+ "transfer length %d for opcode 0x%02x "
+ "(handler %s, target %s) doesn't match "
+ "decoded value %d. Faulty initiator "
+ "(e.g. VMware is known to be such) or "
+ "scst_scsi_op_table should be updated?",
+ cmd->expected_out_transfer_len, cmd->cdb[0],
+ dev->handler->name, cmd->tgtt->name,
+ cmd->out_bufflen);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Suspicious CDB",
+ cmd->cdb, cmd->cdb_len);
+ cmd->resid_possible = 1;
+ }
+#endif
+ }
+
+ if (unlikely(cmd->data_direction == SCST_DATA_UNKNOWN)) {
+ PRINT_ERROR("Unknown data direction. Opcode 0x%x, handler %s, "
+ "target %s", cmd->cdb[0], dev->handler->name,
+ cmd->tgtt->name);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_hw_error;
+ }
+
+set_res:
+ if (cmd->data_len == -1)
+ cmd->data_len = cmd->bufflen;
+
+ if (cmd->bufflen == 0) {
+ /*
+ * According to SPC bufflen 0 for data transfer commands isn't
+ * an error, so we need to fix the transfer direction.
+ */
+ cmd->data_direction = SCST_DATA_NONE;
+ }
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ switch (state) {
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ case SCST_CMD_STATE_PRE_PARSE:
+ case SCST_CMD_STATE_DEV_PARSE:
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ case SCST_CMD_STATE_DEV_DONE:
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ case SCST_CMD_STATE_XMIT_RESP:
+ case SCST_CMD_STATE_FINISHED:
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+#endif
+ cmd->state = state;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ break;
+
+ default:
+ if (state >= 0) {
+ PRINT_ERROR("Dev handler %s parse() returned "
+ "invalid cmd state %d (opcode %d)",
+ dev->handler->name, state, cmd->cdb[0]);
+ } else {
+ PRINT_ERROR("Dev handler %s parse() returned "
+ "error %d (opcode %d)", dev->handler->name,
+ state, cmd->cdb[0]);
+ }
+ goto out_hw_error;
+ }
+#endif
+
+ if (cmd->resp_data_len == -1) {
+ if (cmd->data_direction & SCST_DATA_READ)
+ cmd->resp_data_len = cmd->bufflen;
+ else
+ cmd->resp_data_len = 0;
+ }
+
+ /* We already completed (with an error) */
+ if (unlikely(cmd->completed))
+ goto out_done;
+
+out:
+ return res;
+
+out_hw_error:
+ /* dev_done() will be called as part of the regular cmd's finish */
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+
+out_done:
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+static void scst_set_write_len(struct scst_cmd *cmd)
+{
+
+ EXTRACHECKS_BUG_ON(!(cmd->data_direction & SCST_DATA_WRITE));
+
+ if (cmd->data_direction & SCST_DATA_READ) {
+ cmd->write_len = cmd->out_bufflen;
+ cmd->write_sg = &cmd->out_sg;
+ cmd->write_sg_cnt = &cmd->out_sg_cnt;
+ } else {
+ cmd->write_len = cmd->bufflen;
+ /* write_sg and write_sg_cnt already initialized correctly */
+ }
+
+ TRACE_MEM("cmd %p, write_len %d, write_sg %p, write_sg_cnt %d, "
+ "resid_possible %d", cmd, cmd->write_len, *cmd->write_sg,
+ *cmd->write_sg_cnt, cmd->resid_possible);
+
+ if (unlikely(cmd->resid_possible)) {
+ if (cmd->data_direction & SCST_DATA_READ) {
+ cmd->write_len = min(cmd->out_bufflen,
+ cmd->expected_out_transfer_len);
+ if (cmd->write_len == cmd->out_bufflen)
+ goto out;
+ } else {
+ cmd->write_len = min(cmd->bufflen,
+ cmd->expected_transfer_len);
+ if (cmd->write_len == cmd->bufflen)
+ goto out;
+ }
+ scst_limit_sg_write_len(cmd);
+ }
+
+out:
+ return;
+}
+
+static int scst_prepare_space(struct scst_cmd *cmd)
+{
+ int r = 0, res = SCST_CMD_STATE_RES_CONT_SAME;
+ struct scst_device *dev = cmd->dev;
+
+ if (cmd->data_direction == SCST_DATA_NONE)
+ goto done;
+
+ if (likely(!scst_is_cmd_fully_local(cmd)) &&
+ (dev->handler->alloc_data_buf != NULL)) {
+ int state;
+
+ if (unlikely(!dev->handler->alloc_data_buf_atomic &&
+ scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Dev handler %s alloc_data_buf() needs "
+ "thread context, rescheduling",
+ dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE_DBG("Calling dev handler %s alloc_data_buf(%p)",
+ dev->handler->name, cmd);
+ scst_set_cur_start(cmd);
+ state = dev->handler->alloc_data_buf(cmd);
+ /* Caution: cmd can be already dead here */
+ TRACE_DBG("Dev handler %s alloc_data_buf() returned %d",
+ dev->handler->name, state);
+
+ switch (state) {
+ case SCST_CMD_STATE_NEED_THREAD_CTX:
+ scst_set_alloc_buf_time(cmd);
+ TRACE_DBG("Dev handler %s alloc_data_buf() requested "
+ "thread context, rescheduling",
+ dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+
+ case SCST_CMD_STATE_STOP:
+ TRACE_DBG("Dev handler %s alloc_data_buf() requested "
+ "stop processing", dev->handler->name);
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ goto out;
+ }
+
+ scst_set_alloc_buf_time(cmd);
+
+ if (unlikely(state != SCST_CMD_STATE_DEFAULT)) {
+ cmd->state = state;
+ goto out;
+ }
+ }
+
+ if (cmd->tgt_need_alloc_data_buf) {
+ int orig_bufflen = cmd->bufflen;
+
+ TRACE_MEM("Custom tgt data buf allocation requested (cmd %p)",
+ cmd);
+
+ scst_set_cur_start(cmd);
+ r = cmd->tgtt->alloc_data_buf(cmd);
+ scst_set_alloc_buf_time(cmd);
+
+ if (r > 0)
+ goto alloc;
+ else if (r == 0) {
+ if (unlikely(cmd->bufflen == 0)) {
+ /* See comment in scst_alloc_space() */
+ if (cmd->sg == NULL)
+ goto alloc;
+ }
+
+ cmd->tgt_data_buf_alloced = 1;
+
+ if (unlikely(orig_bufflen < cmd->bufflen)) {
+ PRINT_ERROR("Target driver allocated data "
+ "buffer (size %d), is less, than "
+ "required (size %d)", orig_bufflen,
+ cmd->bufflen);
+ goto out_error;
+ }
+ TRACE_MEM("tgt_data_buf_alloced (cmd %p)", cmd);
+ } else
+ goto check;
+ }
+
+alloc:
+ if (!cmd->tgt_data_buf_alloced && !cmd->dh_data_buf_alloced) {
+ r = scst_alloc_space(cmd);
+ } else if (cmd->dh_data_buf_alloced && !cmd->tgt_data_buf_alloced) {
+ TRACE_MEM("dh_data_buf_alloced set (cmd %p)", cmd);
+ r = 0;
+ } else if (cmd->tgt_data_buf_alloced && !cmd->dh_data_buf_alloced) {
+ TRACE_MEM("tgt_data_buf_alloced set (cmd %p)", cmd);
+ cmd->sg = cmd->tgt_sg;
+ cmd->sg_cnt = cmd->tgt_sg_cnt;
+ cmd->out_sg = cmd->tgt_out_sg;
+ cmd->out_sg_cnt = cmd->tgt_out_sg_cnt;
+ r = 0;
+ } else {
+ TRACE_MEM("Both *_data_buf_alloced set (cmd %p, sg %p, "
+ "sg_cnt %d, tgt_sg %p, tgt_sg_cnt %d)", cmd, cmd->sg,
+ cmd->sg_cnt, cmd->tgt_sg, cmd->tgt_sg_cnt);
+ r = 0;
+ }
+
+check:
+ if (r != 0) {
+ if (scst_cmd_atomic(cmd)) {
+ TRACE_MEM("%s", "Atomic memory allocation failed, "
+ "rescheduling to the thread");
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ } else
+ goto out_no_space;
+ }
+
+done:
+ if (cmd->preprocessing_only) {
+ cmd->state = SCST_CMD_STATE_PREPROCESSING_DONE;
+ if (cmd->data_direction & SCST_DATA_WRITE)
+ scst_set_write_len(cmd);
+ } else if (cmd->data_direction & SCST_DATA_WRITE) {
+ cmd->state = SCST_CMD_STATE_RDY_TO_XFER;
+ scst_set_write_len(cmd);
+ } else
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+
+out:
+ return res;
+
+out_no_space:
+ TRACE(TRACE_OUT_OF_MEM, "Unable to allocate or build requested buffer "
+ "(size %d), sending BUSY or QUEUE FULL status", cmd->bufflen);
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+
+out_error:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+static int scst_preprocessing_done(struct scst_cmd *cmd)
+{
+ int res;
+
+ EXTRACHECKS_BUG_ON(!cmd->preprocessing_only);
+
+ cmd->preprocessing_only = 0;
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ cmd->state = SCST_CMD_STATE_PREPROCESSING_DONE_CALLED;
+
+ TRACE_DBG("Calling preprocessing_done(cmd %p)", cmd);
+ scst_set_cur_start(cmd);
+ cmd->tgtt->preprocessing_done(cmd);
+ TRACE_DBG("%s", "preprocessing_done() returned");
+ return res;
+}
+
+/**
+ * scst_restart_cmd() - restart execution of the command
+ * @cmd: SCST commands
+ * @status: completion status
+ * @pref_context: preferred command execition context
+ *
+ * Description:
+ * Notifies SCST that the driver finished its part of the command's
+ * preprocessing and it is ready for further processing.
+ *
+ * The second argument sets completion status
+ * (see SCST_PREPROCESS_STATUS_* constants for details)
+ *
+ * See also comment for scst_cmd_init_done() for the serialization
+ * requirements.
+ */
+void scst_restart_cmd(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context)
+{
+
+ scst_set_restart_waiting_time(cmd);
+
+ TRACE_DBG("Preferred context: %d", pref_context);
+ TRACE_DBG("tag=%llu, status=%#x",
+ (long long unsigned int)scst_cmd_get_tag(cmd),
+ status);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((in_irq() || irqs_disabled()) &&
+ ((pref_context == SCST_CONTEXT_DIRECT) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC))) {
+ PRINT_ERROR("Wrong context %d in IRQ from target %s, use "
+ "SCST_CONTEXT_THREAD instead", pref_context,
+ cmd->tgtt->name);
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+#endif
+
+ switch (status) {
+ case SCST_PREPROCESS_STATUS_SUCCESS:
+ if (cmd->data_direction & SCST_DATA_WRITE)
+ cmd->state = SCST_CMD_STATE_RDY_TO_XFER;
+ else
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+ if (cmd->set_sn_on_restart_cmd)
+ scst_cmd_set_sn(cmd);
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ if (cmd->op_flags & SCST_TEST_IO_IN_SIRQ_ALLOWED)
+ break;
+#endif
+ /* Small context optimization */
+ if ((pref_context == SCST_CONTEXT_TASKLET) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC) ||
+ ((pref_context == SCST_CONTEXT_SAME) &&
+ scst_cmd_atomic(cmd)))
+ pref_context = SCST_CONTEXT_THREAD;
+ break;
+
+ case SCST_PREPROCESS_STATUS_ERROR_SENSE_SET:
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ case SCST_PREPROCESS_STATUS_ERROR_FATAL:
+ set_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags);
+ /* go through */
+ case SCST_PREPROCESS_STATUS_ERROR:
+ if (cmd->sense != NULL)
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ default:
+ PRINT_ERROR("%s() received unknown status %x", __func__,
+ status);
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ }
+
+ scst_process_redirect_cmd(cmd, pref_context, 1);
+ return;
+}
+EXPORT_SYMBOL(scst_restart_cmd);
+
+static int scst_rdy_to_xfer(struct scst_cmd *cmd)
+{
+ int res, rc;
+ struct scst_tgt_template *tgtt = cmd->tgtt;
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("ABORTED set, aborting cmd %p", cmd);
+ goto out_dev_done;
+ }
+
+ if ((tgtt->rdy_to_xfer == NULL) || unlikely(cmd->internal)) {
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+ }
+
+ if (unlikely(!tgtt->rdy_to_xfer_atomic && scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Target driver %s rdy_to_xfer() needs thread "
+ "context, rescheduling", tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ while (1) {
+ int finished_cmds = atomic_read(&cmd->tgt->finished_cmds);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ cmd->state = SCST_CMD_STATE_DATA_WAIT;
+
+ if (tgtt->on_hw_pending_cmd_timeout != NULL) {
+ struct scst_session *sess = cmd->sess;
+ cmd->hw_pending_start = jiffies;
+ cmd->cmd_hw_pending = 1;
+ if (!test_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags)) {
+ TRACE_DBG("Sched HW pending work for sess %p "
+ "(max time %d)", sess,
+ tgtt->max_hw_pending_time);
+ set_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED,
+ &sess->sess_aflags);
+ schedule_delayed_work(&sess->hw_pending_work,
+ tgtt->max_hw_pending_time * HZ);
+ }
+ }
+
+ scst_set_cur_start(cmd);
+
+ TRACE_DBG("Calling rdy_to_xfer(%p)", cmd);
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ if (((scst_random() % 100) == 75))
+ rc = SCST_TGT_RES_QUEUE_FULL;
+ else
+#endif
+ rc = tgtt->rdy_to_xfer(cmd);
+ TRACE_DBG("rdy_to_xfer() returned %d", rc);
+
+ if (likely(rc == SCST_TGT_RES_SUCCESS))
+ goto out;
+
+ scst_set_rdy_to_xfer_time(cmd);
+
+ cmd->cmd_hw_pending = 0;
+
+ /* Restore the previous state */
+ cmd->state = SCST_CMD_STATE_RDY_TO_XFER;
+
+ switch (rc) {
+ case SCST_TGT_RES_QUEUE_FULL:
+ if (scst_queue_retry_cmd(cmd, finished_cmds) == 0)
+ break;
+ else
+ continue;
+
+ case SCST_TGT_RES_NEED_THREAD_CTX:
+ TRACE_DBG("Target driver %s "
+ "rdy_to_xfer() requested thread "
+ "context, rescheduling", tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+
+ default:
+ goto out_error_rc;
+ }
+ break;
+ }
+
+out:
+ return res;
+
+out_error_rc:
+ if (rc == SCST_TGT_RES_FATAL_ERROR) {
+ PRINT_ERROR("Target driver %s rdy_to_xfer() returned "
+ "fatal error", tgtt->name);
+ } else {
+ PRINT_ERROR("Target driver %s rdy_to_xfer() returned invalid "
+ "value %d", tgtt->name, rc);
+ }
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+
+out_dev_done:
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+/* No locks, but might be in IRQ */
+static void scst_process_redirect_cmd(struct scst_cmd *cmd,
+ enum scst_exec_context context, int check_retries)
+{
+ struct scst_tgt *tgt = cmd->tgt;
+ unsigned long flags;
+
+ TRACE_DBG("Context: %x", context);
+
+ if (context == SCST_CONTEXT_SAME)
+ context = scst_cmd_atomic(cmd) ? SCST_CONTEXT_DIRECT_ATOMIC :
+ SCST_CONTEXT_DIRECT;
+
+ switch (context) {
+ case SCST_CONTEXT_DIRECT_ATOMIC:
+ scst_process_active_cmd(cmd, true);
+ break;
+
+ case SCST_CONTEXT_DIRECT:
+ if (check_retries)
+ scst_check_retries(tgt);
+ scst_process_active_cmd(cmd, false);
+ break;
+
+ default:
+ PRINT_ERROR("Context %x is unknown, using the thread one",
+ context);
+ /* go through */
+ case SCST_CONTEXT_THREAD:
+ if (check_retries)
+ scst_check_retries(tgt);
+ spin_lock_irqsave(&cmd->cmd_threads->cmd_list_lock, flags);
+ TRACE_DBG("Adding cmd %p to active cmd list", cmd);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irqrestore(&cmd->cmd_threads->cmd_list_lock, flags);
+ break;
+
+ case SCST_CONTEXT_TASKLET:
+ if (check_retries)
+ scst_check_retries(tgt);
+ scst_schedule_tasklet(cmd);
+ break;
+ }
+ return;
+}
+
+/**
+ * scst_rx_data() - the command's data received
+ * @cmd: SCST commands
+ * @status: data receiving completion status
+ * @pref_context: preferred command execution context
+ *
+ * Description:
+ * Notifies SCST that the driver received all the necessary data
+ * and the command is ready for further processing.
+ *
+ * The second argument sets data receiving completion status
+ * (see SCST_RX_STATUS_* constants for details)
+ */
+void scst_rx_data(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context)
+{
+
+ scst_set_rdy_to_xfer_time(cmd);
+
+ TRACE_DBG("Preferred context: %d", pref_context);
+ TRACE(TRACE_SCSI, "cmd %p, status %#x", cmd, status);
+
+ cmd->cmd_hw_pending = 0;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((in_irq() || irqs_disabled()) &&
+ ((pref_context == SCST_CONTEXT_DIRECT) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC))) {
+ PRINT_ERROR("Wrong context %d in IRQ from target %s, use "
+ "SCST_CONTEXT_THREAD instead", pref_context,
+ cmd->tgtt->name);
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+#endif
+
+ switch (status) {
+ case SCST_RX_STATUS_SUCCESS:
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (trace_flag & TRACE_RCV_BOT) {
+ int i, j;
+ struct scatterlist *sg;
+ if (cmd->out_sg != NULL)
+ sg = cmd->out_sg;
+ else if (cmd->tgt_out_sg != NULL)
+ sg = cmd->tgt_out_sg;
+ else if (cmd->tgt_sg != NULL)
+ sg = cmd->tgt_sg;
+ else
+ sg = cmd->sg;
+ if (sg != NULL) {
+ TRACE_RECV_BOT("RX data for cmd %p "
+ "(sg_cnt %d, sg %p, sg[0].page %p)",
+ cmd, cmd->tgt_sg_cnt, sg,
+ (void *)sg_page(&sg[0]));
+ for (i = 0, j = 0; i < cmd->tgt_sg_cnt; ++i, ++j) {
+ if (unlikely(sg_is_chain(&sg[j]))) {
+ sg = sg_chain_ptr(&sg[j]);
+ j = 0;
+ }
+ PRINT_BUFF_FLAG(TRACE_RCV_BOT, "RX sg",
+ sg_virt(&sg[j]), sg[j].length);
+ }
+ }
+ }
+#endif
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ if (cmd->op_flags & SCST_TEST_IO_IN_SIRQ_ALLOWED)
+ break;
+#endif
+
+ /* Small context optimization */
+ if ((pref_context == SCST_CONTEXT_TASKLET) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC) ||
+ ((pref_context == SCST_CONTEXT_SAME) &&
+ scst_cmd_atomic(cmd)))
+ pref_context = SCST_CONTEXT_THREAD;
+ break;
+
+ case SCST_RX_STATUS_ERROR_SENSE_SET:
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ case SCST_RX_STATUS_ERROR_FATAL:
+ set_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags);
+ /* go through */
+ case SCST_RX_STATUS_ERROR:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ default:
+ PRINT_ERROR("scst_rx_data() received unknown status %x",
+ status);
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ }
+
+ scst_process_redirect_cmd(cmd, pref_context, 1);
+ return;
+}
+EXPORT_SYMBOL(scst_rx_data);
+
+static int scst_tgt_pre_exec(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME, rc;
+
+ if (unlikely(cmd->resid_possible)) {
+ if (cmd->data_direction & SCST_DATA_WRITE) {
+ bool do_zero = false;
+ if (cmd->data_direction & SCST_DATA_READ) {
+ if (cmd->write_len != cmd->out_bufflen)
+ do_zero = true;
+ } else {
+ if (cmd->write_len != cmd->bufflen)
+ do_zero = true;
+ }
+ if (do_zero) {
+ scst_check_restore_sg_buff(cmd);
+ scst_zero_write_rest(cmd);
+ }
+ }
+ }
+
+ cmd->state = SCST_CMD_STATE_SEND_FOR_EXEC;
+
+ if ((cmd->tgtt->pre_exec == NULL) || unlikely(cmd->internal))
+ goto out;
+
+ TRACE_DBG("Calling pre_exec(%p)", cmd);
+ scst_set_cur_start(cmd);
+ rc = cmd->tgtt->pre_exec(cmd);
+ scst_set_pre_exec_time(cmd);
+ TRACE_DBG("pre_exec() returned %d", rc);
+
+ if (unlikely(rc != SCST_PREPROCESS_STATUS_SUCCESS)) {
+ switch (rc) {
+ case SCST_PREPROCESS_STATUS_ERROR_SENSE_SET:
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ case SCST_PREPROCESS_STATUS_ERROR_FATAL:
+ set_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags);
+ /* go through */
+ case SCST_PREPROCESS_STATUS_ERROR:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ default:
+ BUG();
+ break;
+ }
+ }
+
+out:
+ return res;
+}
+
+static void scst_do_cmd_done(struct scst_cmd *cmd, int result,
+ const uint8_t *rq_sense, int rq_sense_len, int resid)
+{
+
+ scst_set_exec_time(cmd);
+
+ cmd->status = result & 0xff;
+ cmd->msg_status = msg_byte(result);
+ cmd->host_status = host_byte(result);
+ cmd->driver_status = driver_byte(result);
+ if (unlikely(resid != 0)) {
+ if ((cmd->data_direction & SCST_DATA_READ) &&
+ (resid > 0) && (resid < cmd->resp_data_len))
+ scst_set_resp_data_len(cmd, cmd->resp_data_len - resid);
+ /*
+ * We ignore write direction residue, because from the
+ * initiator's POV we already transferred all the data.
+ */
+ }
+
+ if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION)) {
+ /* We might have double reset UA here */
+ cmd->dbl_ua_orig_resp_data_len = cmd->resp_data_len;
+ cmd->dbl_ua_orig_data_direction = cmd->data_direction;
+
+ scst_alloc_set_sense(cmd, 1, rq_sense, rq_sense_len);
+ }
+
+ TRACE(TRACE_SCSI, "cmd %p, result %x, cmd->status %x, resid %d, "
+ "cmd->msg_status %x, cmd->host_status %x, "
+ "cmd->driver_status %x", cmd, result, cmd->status, resid,
+ cmd->msg_status, cmd->host_status, cmd->driver_status);
+
+ cmd->completed = 1;
+ return;
+}
+
+/* For small context optimization */
+static inline enum scst_exec_context scst_optimize_post_exec_context(
+ struct scst_cmd *cmd, enum scst_exec_context context)
+{
+ if (((context == SCST_CONTEXT_SAME) && scst_cmd_atomic(cmd)) ||
+ (context == SCST_CONTEXT_TASKLET) ||
+ (context == SCST_CONTEXT_DIRECT_ATOMIC)) {
+ if (!test_bit(SCST_TGT_DEV_AFTER_EXEC_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ context = SCST_CONTEXT_THREAD;
+ }
+ return context;
+}
+
+/**
+ * scst_pass_through_cmd_done - done callback for pass-through commands
+ * @data: private opaque data
+ * @sense: pointer to the sense data, if any
+ * @result: command's execution result
+ * @resid: residual, if any
+ */
+void scst_pass_through_cmd_done(void *data, char *sense, int result, int resid)
+{
+ struct scst_cmd *cmd;
+
+ cmd = (struct scst_cmd *)data;
+ if (cmd == NULL)
+ goto out;
+
+ scst_do_cmd_done(cmd, result, sense, SCSI_SENSE_BUFFERSIZE, resid);
+
+ cmd->state = SCST_CMD_STATE_PRE_DEV_DONE;
+
+ scst_process_redirect_cmd(cmd,
+ scst_optimize_post_exec_context(cmd, scst_estimate_context()), 0);
+
+out:
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_pass_through_cmd_done);
+
+static void scst_cmd_done_local(struct scst_cmd *cmd, int next_state,
+ enum scst_exec_context pref_context)
+{
+
+ EXTRACHECKS_BUG_ON(cmd->pr_abort_counter != NULL);
+
+ scst_set_exec_time(cmd);
+
+ TRACE(TRACE_SCSI, "cmd %p, status %x, msg_status %x, host_status %x, "
+ "driver_status %x, resp_data_len %d", cmd, cmd->status,
+ cmd->msg_status, cmd->host_status, cmd->driver_status,
+ cmd->resp_data_len);
+
+ if (next_state == SCST_CMD_STATE_DEFAULT)
+ next_state = SCST_CMD_STATE_PRE_DEV_DONE;
+
+#if defined(CONFIG_SCST_DEBUG)
+ if (next_state == SCST_CMD_STATE_PRE_DEV_DONE) {
+ if ((trace_flag & TRACE_RCV_TOP) && (cmd->sg != NULL)) {
+ int i, j;
+ struct scatterlist *sg = cmd->sg;
+ TRACE_RECV_TOP("Exec'd %d S/G(s) at %p sg[0].page at "
+ "%p", cmd->sg_cnt, sg, (void *)sg_page(&sg[0]));
+ for (i = 0, j = 0; i < cmd->sg_cnt; ++i, ++j) {
+ if (unlikely(sg_is_chain(&sg[j]))) {
+ sg = sg_chain_ptr(&sg[j]);
+ j = 0;
+ }
+ TRACE_BUFF_FLAG(TRACE_RCV_TOP,
+ "Exec'd sg", sg_virt(&sg[j]),
+ sg[j].length);
+ }
+ }
+ }
+#endif
+
+ cmd->state = next_state;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((next_state != SCST_CMD_STATE_PRE_DEV_DONE) &&
+ (next_state != SCST_CMD_STATE_PRE_XMIT_RESP) &&
+ (next_state != SCST_CMD_STATE_FINISHED) &&
+ (next_state != SCST_CMD_STATE_FINISHED_INTERNAL)) {
+ PRINT_ERROR("%s() received invalid cmd state %d (opcode %d)",
+ __func__, next_state, cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ }
+#endif
+ pref_context = scst_optimize_post_exec_context(cmd, pref_context);
+ scst_process_redirect_cmd(cmd, pref_context, 0);
+ return;
+}
+
+static int scst_report_luns_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_COMPLETED, rc;
+ int dev_cnt = 0;
+ int buffer_size;
+ int i;
+ struct scst_tgt_dev *tgt_dev = NULL;
+ uint8_t *buffer;
+ int offs, overflow = 0;
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ if ((cmd->cdb[2] != 0) && (cmd->cdb[2] != 2)) {
+ PRINT_ERROR("Unsupported SELECT REPORT value %x in REPORT "
+ "LUNS command", cmd->cdb[2]);
+ goto out_err;
+ }
+
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size == 0))
+ goto out_compl;
+ else if (unlikely(buffer_size < 0))
+ goto out_hw_err;
+
+ if (buffer_size < 16)
+ goto out_put_err;
+
+ memset(buffer, 0, buffer_size);
+ offs = 8;
+
+ /*
+ * cmd won't allow to suspend activities, so we can access
+ * sess->sess_tgt_dev_list_hash without any additional protection.
+ */
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (!overflow) {
+ if (offs >= buffer_size) {
+ scst_put_buf(cmd, buffer);
+ buffer_size = scst_get_buf_next(cmd,
+ &buffer);
+ if (buffer_size > 0) {
+ memset(buffer, 0, buffer_size);
+ offs = 0;
+ } else {
+ overflow = 1;
+ goto inc_dev_cnt;
+ }
+ }
+ if ((buffer_size - offs) < 8) {
+ PRINT_ERROR("Buffer allocated for "
+ "REPORT LUNS command doesn't "
+ "allow to fit 8 byte entry "
+ "(buffer_size=%d)",
+ buffer_size);
+ goto out_put_hw_err;
+ }
+ if ((cmd->sess->acg->addr_method == SCST_LUN_ADDR_METHOD_FLAT) &&
+ (tgt_dev->lun != 0)) {
+ buffer[offs] = (tgt_dev->lun >> 8) & 0x3f;
+ buffer[offs] = buffer[offs] | 0x40;
+ buffer[offs+1] = tgt_dev->lun & 0xff;
+ } else {
+ buffer[offs] = (tgt_dev->lun >> 8) & 0xff;
+ buffer[offs+1] = tgt_dev->lun & 0xff;
+ }
+ offs += 8;
+ }
+inc_dev_cnt:
+ dev_cnt++;
+ }
+ }
+ if (!overflow)
+ scst_put_buf(cmd, buffer);
+
+ /* Set the response header */
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size == 0))
+ goto out_compl;
+ else if (unlikely(buffer_size < 0))
+ goto out_hw_err;
+
+ dev_cnt *= 8;
+ buffer[0] = (dev_cnt >> 24) & 0xff;
+ buffer[1] = (dev_cnt >> 16) & 0xff;
+ buffer[2] = (dev_cnt >> 8) & 0xff;
+ buffer[3] = dev_cnt & 0xff;
+
+ scst_put_buf(cmd, buffer);
+
+ dev_cnt += 8;
+ if (dev_cnt < cmd->resp_data_len)
+ scst_set_resp_data_len(cmd, dev_cnt);
+
+out_compl:
+ cmd->completed = 1;
+
+ /* Clear left sense_reported_luns_data_changed UA, if any. */
+
+ /*
+ * cmd won't allow to suspend activities, so we can access
+ * sess->sess_tgt_dev_list_hash without any additional protection.
+ */
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ struct scst_tgt_dev_UA *ua;
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ list_for_each_entry(ua, &tgt_dev->UA_list,
+ UA_list_entry) {
+ if (scst_analyze_sense(ua->UA_sense_buffer,
+ ua->UA_valid_sense_len,
+ SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed))) {
+ TRACE_MGMT_DBG("Freeing not needed "
+ "REPORTED LUNS DATA CHANGED UA "
+ "%p", ua);
+ list_del(&ua->UA_list_entry);
+ mempool_free(ua, scst_ua_mempool);
+ break;
+ }
+ }
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ return res;
+
+out_put_err:
+ scst_put_buf(cmd, buffer);
+
+out_err:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_compl;
+
+out_put_hw_err:
+ scst_put_buf(cmd, buffer);
+
+out_hw_err:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_compl;
+}
+
+static int scst_request_sense_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_COMPLETED, rc;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ uint8_t *buffer;
+ int buffer_size = 0, sl = 0;
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+
+ if (tgt_dev->tgt_dev_valid_sense_len == 0)
+ goto out_unlock_not_completed;
+
+ TRACE(TRACE_SCSI, "%s: Returning stored sense", cmd->op_name);
+
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size == 0))
+ goto out_unlock_compl;
+ else if (unlikely(buffer_size < 0))
+ goto out_unlock_hw_err;
+
+ memset(buffer, 0, buffer_size);
+
+ if (((tgt_dev->tgt_dev_sense[0] == 0x70) ||
+ (tgt_dev->tgt_dev_sense[0] == 0x71)) && (cmd->cdb[1] & 1)) {
+ PRINT_WARNING("%s: Fixed format of the saved sense, but "
+ "descriptor format requested. Convertion will "
+ "truncated data", cmd->op_name);
+ PRINT_BUFFER("Original sense", tgt_dev->tgt_dev_sense,
+ tgt_dev->tgt_dev_valid_sense_len);
+
+ buffer_size = min(SCST_STANDARD_SENSE_LEN, buffer_size);
+ sl = scst_set_sense(buffer, buffer_size, true,
+ tgt_dev->tgt_dev_sense[2], tgt_dev->tgt_dev_sense[12],
+ tgt_dev->tgt_dev_sense[13]);
+ } else if (((tgt_dev->tgt_dev_sense[0] == 0x72) ||
+ (tgt_dev->tgt_dev_sense[0] == 0x73)) && !(cmd->cdb[1] & 1)) {
+ PRINT_WARNING("%s: Descriptor format of the "
+ "saved sense, but fixed format requested. Convertion "
+ "will truncated data", cmd->op_name);
+ PRINT_BUFFER("Original sense", tgt_dev->tgt_dev_sense,
+ tgt_dev->tgt_dev_valid_sense_len);
+
+ buffer_size = min(SCST_STANDARD_SENSE_LEN, buffer_size);
+ sl = scst_set_sense(buffer, buffer_size, false,
+ tgt_dev->tgt_dev_sense[1], tgt_dev->tgt_dev_sense[2],
+ tgt_dev->tgt_dev_sense[3]);
+ } else {
+ if (buffer_size >= tgt_dev->tgt_dev_valid_sense_len)
+ sl = tgt_dev->tgt_dev_valid_sense_len;
+ else {
+ sl = buffer_size;
+ TRACE(TRACE_MINOR, "%s: Being returned sense truncated "
+ "to size %d (needed %d)", cmd->op_name,
+ buffer_size, tgt_dev->tgt_dev_valid_sense_len);
+ }
+ memcpy(buffer, tgt_dev->tgt_dev_sense, sl);
+ }
+
+ scst_put_buf(cmd, buffer);
+
+ tgt_dev->tgt_dev_valid_sense_len = 0;
+
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+
+ scst_set_resp_data_len(cmd, sl);
+
+out_compl:
+ cmd->completed = 1;
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+
+out:
+ return res;
+
+out_unlock_hw_err:
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_compl;
+
+out_unlock_not_completed:
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ res = SCST_EXEC_NOT_COMPLETED;
+ goto out;
+
+out_unlock_compl:
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ goto out_compl;
+}
+
+static int scst_pre_select(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED;
+
+ scst_block_dev_cmd(cmd, 1);
+
+ /* Check for local events will be done when cmd will be executed */
+ return res;
+}
+
+static int scst_reserve_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED, rc;
+ struct scst_device *dev;
+ struct scst_tgt_dev *tgt_dev_tmp;
+
+ if ((cmd->cdb[0] == RESERVE_10) && (cmd->cdb[2] & SCST_RES_3RDPTY)) {
+ PRINT_ERROR("RESERVE_10: 3rdPty RESERVE not implemented "
+ "(lun=%lld)", (long long unsigned int)cmd->lun);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_done;
+ }
+
+ dev = cmd->dev;
+
+ /*
+ * There's no need to block this device, even for
+ * SCST_CONTR_MODE_ONE_TASK_SET, or anyhow else protect reservations
+ * changes, because:
+ *
+ * 1. The reservation changes are (rather) atomic, i.e., in contrast
+ * to persistent reservations, don't have any invalid intermediate
+ * states during being changed.
+ *
+ * 2. It's a duty of initiators to ensure order of regular commands
+ * around the reservation command either by ORDERED attribute, or by
+ * queue draining, or etc. For case of SCST_CONTR_MODE_ONE_TASK_SET
+ * there are no target drivers which can ensure even for ORDERED
+ * comamnds order of their delivery, so, because initiators know
+ * it, also there's no point to do any extra protection actions.
+ */
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ if (!list_empty(&dev->dev_registrants_list)) {
+ if (scst_pr_crh_case(cmd))
+ goto out_completed;
+ else {
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out_done;
+ }
+ }
+
+ spin_lock_bh(&dev->dev_lock);
+
+ if (test_bit(SCST_TGT_DEV_RESERVED, &cmd->tgt_dev->tgt_dev_flags)) {
+ spin_unlock_bh(&dev->dev_lock);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out_done;
+ }
+
+ list_for_each_entry(tgt_dev_tmp, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (cmd->tgt_dev != tgt_dev_tmp)
+ set_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 1;
+
+ spin_unlock_bh(&dev->dev_lock);
+
+out:
+ return res;
+
+out_completed:
+ cmd->completed = 1;
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ res = SCST_EXEC_COMPLETED;
+ goto out;
+}
+
+static int scst_release_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED, rc;
+ struct scst_tgt_dev *tgt_dev_tmp;
+ struct scst_device *dev;
+
+ dev = cmd->dev;
+
+ /*
+ * See comment in scst_reserve_local() why no dev blocking or any
+ * other protection is needed here.
+ */
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ if (!list_empty(&dev->dev_registrants_list)) {
+ if (scst_pr_crh_case(cmd))
+ goto out_completed;
+ else {
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out_done;
+ }
+ }
+
+ spin_lock_bh(&dev->dev_lock);
+
+ /*
+ * The device could be RELEASED behind us, if RESERVING session
+ * is closed (see scst_free_tgt_dev()), but this actually doesn't
+ * matter, so use lock and no retest for DEV_RESERVED bits again
+ */
+ if (test_bit(SCST_TGT_DEV_RESERVED, &cmd->tgt_dev->tgt_dev_flags)) {
+ res = SCST_EXEC_COMPLETED;
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+ cmd->completed = 1;
+ } else {
+ list_for_each_entry(tgt_dev_tmp,
+ &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ }
+
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (res == SCST_EXEC_COMPLETED)
+ goto out_done;
+
+out:
+ return res;
+
+out_completed:
+ cmd->completed = 1;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out;
+}
+
+/**
+ * scst_check_local_events() - check if there are any local SCSI events
+ *
+ * Description:
+ * Checks if the command can be executed or there are local events,
+ * like reservatons, pending UAs, etc. Returns < 0 if command must be
+ * aborted, > 0 if there is an event and command should be immediately
+ * completed, or 0 otherwise.
+ *
+ * !! Dev handlers implementing exec() callback must call this function there
+ * !! just before the actual command's execution!
+ *
+ * On call no locks, no IRQ or IRQ-disabled context allowed.
+ */
+static int scst_persistent_reserve_in_local(struct scst_cmd *cmd)
+{
+ int rc;
+ struct scst_device *dev;
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_session *session;
+ int action;
+ uint8_t *buffer;
+ int buffer_size;
+
+ EXTRACHECKS_BUG_ON(scst_cmd_atomic(cmd));
+
+ dev = cmd->dev;
+ tgt_dev = cmd->tgt_dev;
+ session = cmd->sess;
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ if (unlikely(dev->not_pr_supporting_tgt_devs_num != 0)) {
+ PRINT_WARNING("Persistent Reservation command %x refused for "
+ "device %s, because the device has not supporting PR "
+ "transports connected", cmd->cdb[0], dev->virt_name);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+ }
+
+ if (dev->dev_reserved) {
+ TRACE_PR("PR command rejected, because device %s holds regular "
+ "reservation", dev->virt_name);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out_done;
+ }
+
+ if (dev->scsi_dev != NULL) {
+ PRINT_WARNING("PR commands for pass-through devices not "
+ "supported (device %s)", dev->virt_name);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+ }
+
+ buffer_size = scst_get_full_buf(cmd, &buffer);
+ if (unlikely(buffer_size <= 0)) {
+ if (buffer_size < 0)
+ scst_set_busy(cmd);
+ goto out_done;
+ }
+
+ scst_pr_write_lock(dev);
+
+ /* We can be aborted by another PR command while waiting for the lock */
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("ABORTED set, aborting cmd %p", cmd);
+ goto out_unlock;
+ }
+
+ action = cmd->cdb[1] & 0x1f;
+
+ TRACE(TRACE_SCSI, "PR action %x for '%s' (LUN %llx) from '%s'", action,
+ dev->virt_name, tgt_dev->lun, session->initiator_name);
+
+ switch (action) {
+ case PR_READ_KEYS:
+ scst_pr_read_keys(cmd, buffer, buffer_size);
+ break;
+ case PR_READ_RESERVATION:
+ scst_pr_read_reservation(cmd, buffer, buffer_size);
+ break;
+ case PR_REPORT_CAPS:
+ scst_pr_report_caps(cmd, buffer, buffer_size);
+ break;
+ case PR_READ_FULL_STATUS:
+ scst_pr_read_full_status(cmd, buffer, buffer_size);
+ break;
+ default:
+ PRINT_ERROR("Unsupported action %x", action);
+ scst_pr_write_unlock(dev);
+ goto out_err;
+ }
+
+out_complete:
+ cmd->completed = 1;
+
+out_unlock:
+ scst_pr_write_unlock(dev);
+
+ scst_put_full_buf(cmd, buffer);
+
+out_done:
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ return SCST_EXEC_COMPLETED;
+
+out_err:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_complete;
+}
+
+/* No locks, no IRQ or IRQ-disabled context allowed */
+static int scst_persistent_reserve_out_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_COMPLETED;
+ int rc;
+ struct scst_device *dev;
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_session *session;
+ int action;
+ uint8_t *buffer;
+ int buffer_size;
+ bool aborted = false;
+
+ EXTRACHECKS_BUG_ON(scst_cmd_atomic(cmd));
+
+ dev = cmd->dev;
+ tgt_dev = cmd->tgt_dev;
+ session = cmd->sess;
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ if (unlikely(dev->not_pr_supporting_tgt_devs_num != 0)) {
+ PRINT_WARNING("Persistent Reservation command %x refused for "
+ "device %s, because the device has not supporting PR "
+ "transports connected", cmd->cdb[0], dev->virt_name);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+ }
+
+ action = cmd->cdb[1] & 0x1f;
+
+ TRACE(TRACE_SCSI, "PR action %x for '%s' (LUN %llx) from '%s'", action,
+ dev->virt_name, tgt_dev->lun, session->initiator_name);
+
+ if (dev->dev_reserved) {
+ TRACE_PR("PR command rejected, because device %s holds regular "
+ "reservation", dev->virt_name);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out_done;
+ }
+
+ /*
+ * Check if tgt_dev already registered. Also by this check we make
+ * sure that table "PERSISTENT RESERVE OUT service actions that are
+ * allowed in the presence of various reservations" is honored.
+ * REGISTER AND MOVE and RESERVE will be additionally checked for
+ * conflicts later.
+ */
+ if ((action != PR_REGISTER) && (action != PR_REGISTER_AND_IGNORE) &&
+ (tgt_dev->registrant == NULL)) {
+ TRACE_PR("'%s' not registered", cmd->sess->initiator_name);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out_done;
+ }
+
+ buffer_size = scst_get_full_buf(cmd, &buffer);
+ if (unlikely(buffer_size <= 0)) {
+ if (buffer_size < 0)
+ scst_set_busy(cmd);
+ goto out_done;
+ }
+
+ /* Check scope */
+ if ((action != PR_REGISTER) && (action != PR_REGISTER_AND_IGNORE) &&
+ (action != PR_CLEAR) && ((cmd->cdb[2] & 0x0f) >> 4) != SCOPE_LU) {
+ TRACE_PR("Scope must be SCOPE_LU for action %x", action);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_put_full_buf;
+ }
+
+ /* Check SPEC_I_PT (PR_REGISTER_AND_MOVE has another format) */
+ if ((action != PR_REGISTER) && (action != PR_REGISTER_AND_MOVE) &&
+ ((buffer[20] >> 3) & 0x01)) {
+ TRACE_PR("SPEC_I_PT must be zero for action %x", action);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_cdb));
+ goto out_put_full_buf;
+ }
+
+ /* Check ALL_TG_PT (PR_REGISTER_AND_MOVE has another format) */
+ if ((action != PR_REGISTER) && (action != PR_REGISTER_AND_IGNORE) &&
+ (action != PR_REGISTER_AND_MOVE) && ((buffer[20] >> 2) & 0x01)) {
+ TRACE_PR("ALL_TG_PT must be zero for action %x", action);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_cdb));
+ goto out_put_full_buf;
+ }
+
+ scst_pr_write_lock(dev);
+
+ /* We can be aborted by another PR command while waiting for the lock */
+ aborted = test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags);
+ if (unlikely(aborted)) {
+ TRACE_MGMT_DBG("ABORTED set, aborting cmd %p", cmd);
+ goto out_unlock;
+ }
+
+ switch (action) {
+ case PR_REGISTER:
+ scst_pr_register(cmd, buffer, buffer_size);
+ break;
+ case PR_RESERVE:
+ scst_pr_reserve(cmd, buffer, buffer_size);
+ break;
+ case PR_RELEASE:
+ scst_pr_release(cmd, buffer, buffer_size);
+ break;
+ case PR_CLEAR:
+ scst_pr_clear(cmd, buffer, buffer_size);
+ break;
+ case PR_PREEMPT:
+ scst_pr_preempt(cmd, buffer, buffer_size);
+ break;
+ case PR_PREEMPT_AND_ABORT:
+ scst_pr_preempt_and_abort(cmd, buffer, buffer_size);
+ break;
+ case PR_REGISTER_AND_IGNORE:
+ scst_pr_register_and_ignore(cmd, buffer, buffer_size);
+ break;
+ case PR_REGISTER_AND_MOVE:
+ scst_pr_register_and_move(cmd, buffer, buffer_size);
+ break;
+ default:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_unlock;
+ }
+
+ if (cmd->status == SAM_STAT_GOOD)
+ scst_pr_sync_device_file(tgt_dev, cmd);
+
+ if ((dev->handler->pr_cmds_notifications) &&
+ (cmd->status == SAM_STAT_GOOD)) /* sync file may change status */
+ res = SCST_EXEC_NOT_COMPLETED;
+
+out_unlock:
+ scst_pr_write_unlock(dev);
+
+out_put_full_buf:
+ scst_put_full_buf(cmd, buffer);
+
+out_done:
+ if (SCST_EXEC_COMPLETED == res) {
+ if (!aborted)
+ cmd->completed = 1;
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT,
+ SCST_CONTEXT_SAME);
+ }
+ return res;
+}
+
+/* No locks, no IRQ or IRQ-disabled context allowed */
+int scst_check_local_events(struct scst_cmd *cmd)
+{
+ int res, rc;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_device *dev = cmd->dev;
+
+ /*
+ * There's no race here, because we need to trace commands sent
+ * *after* dev_double_ua_possible flag was set.
+ */
+ if (unlikely(dev->dev_double_ua_possible))
+ cmd->double_ua_possible = 1;
+
+ /* Reserve check before Unit Attention */
+ if (unlikely(test_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev->tgt_dev_flags))) {
+ if ((cmd->op_flags & SCST_REG_RESERVE_ALLOWED) == 0) {
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out_complete;
+ }
+ }
+
+ if (dev->pr_is_set) {
+ if (unlikely(!scst_pr_is_cmd_allowed(cmd))) {
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out_complete;
+ }
+ }
+
+ /*
+ * Let's check for ABORTED after scst_pr_is_cmd_allowed(), because
+ * we might sleep for a while there.
+ */
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("ABORTED set, aborting cmd %p", cmd);
+ goto out_uncomplete;
+ }
+
+ /* If we had internal bus reset, set the command error unit attention */
+ if ((dev->scsi_dev != NULL) &&
+ unlikely(dev->scsi_dev->was_reset)) {
+ if (scst_is_ua_command(cmd)) {
+ int done = 0;
+ /*
+ * Prevent more than 1 cmd to be triggered by
+ * was_reset.
+ */
+ spin_lock_bh(&dev->dev_lock);
+ if (dev->scsi_dev->was_reset) {
+ TRACE(TRACE_MGMT, "was_reset is %d", 1);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_reset_UA));
+ /*
+ * It looks like it is safe to clear was_reset
+ * here.
+ */
+ dev->scsi_dev->was_reset = 0;
+ done = 1;
+ }
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (done)
+ goto out_complete;
+ }
+ }
+
+ if (unlikely(test_bit(SCST_TGT_DEV_UA_PENDING,
+ &cmd->tgt_dev->tgt_dev_flags))) {
+ if (scst_is_ua_command(cmd)) {
+ rc = scst_set_pending_UA(cmd);
+ if (rc == 0)
+ goto out_complete;
+ }
+ }
+
+ res = 0;
+
+out:
+ return res;
+
+out_complete:
+ res = 1;
+ BUG_ON(!cmd->completed);
+ goto out;
+
+out_uncomplete:
+ res = -1;
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_check_local_events);
+
+/* No locks */
+void scst_inc_expected_sn(struct scst_tgt_dev *tgt_dev, atomic_t *slot)
+{
+ if (slot == NULL)
+ goto inc;
+
+ /* Optimized for lockless fast path */
+
+ TRACE_SN("Slot %zd, *cur_sn_slot %d", slot - tgt_dev->sn_slots,
+ atomic_read(slot));
+
+ if (!atomic_dec_and_test(slot))
+ goto out;
+
+ TRACE_SN("Slot is 0 (num_free_sn_slots=%d)",
+ tgt_dev->num_free_sn_slots);
+ if (tgt_dev->num_free_sn_slots < (int)ARRAY_SIZE(tgt_dev->sn_slots)-1) {
+ spin_lock_irq(&tgt_dev->sn_lock);
+ if (likely(tgt_dev->num_free_sn_slots < (int)ARRAY_SIZE(tgt_dev->sn_slots)-1)) {
+ if (tgt_dev->num_free_sn_slots < 0)
+ tgt_dev->cur_sn_slot = slot;
+ /*
+ * To be in-sync with SIMPLE case in scst_cmd_set_sn()
+ */
+ smp_mb();
+ tgt_dev->num_free_sn_slots++;
+ TRACE_SN("Incremented num_free_sn_slots (%d)",
+ tgt_dev->num_free_sn_slots);
+
+ }
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ }
+
+inc:
+ /*
+ * No protection of expected_sn is needed, because only one thread
+ * at time can be here (serialized by sn). Also it is supposed that
+ * there could not be half-incremented halves.
+ */
+ tgt_dev->expected_sn++;
+ /*
+ * Write must be before def_cmd_count read to be in sync. with
+ * scst_post_exec_sn(). See comment in scst_send_for_exec().
+ */
+ smp_mb();
+ TRACE_SN("Next expected_sn: %d", tgt_dev->expected_sn);
+
+out:
+ return;
+}
+
+/* No locks */
+static struct scst_cmd *scst_post_exec_sn(struct scst_cmd *cmd,
+ bool make_active)
+{
+ /* For HQ commands SN is not set */
+ bool inc_expected_sn = !cmd->inc_expected_sn_on_done &&
+ cmd->sn_set && !cmd->retry;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_cmd *res;
+
+ if (inc_expected_sn)
+ scst_inc_expected_sn(tgt_dev, cmd->sn_slot);
+
+ if (make_active) {
+ scst_make_deferred_commands_active(tgt_dev);
+ res = NULL;
+ } else
+ res = scst_check_deferred_commands(tgt_dev);
+ return res;
+}
+
+/* cmd must be additionally referenced to not die inside */
+static int scst_do_real_exec(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED;
+ int rc;
+ struct scst_device *dev = cmd->dev;
+ struct scst_dev_type *handler = dev->handler;
+ struct io_context *old_ctx = NULL;
+ bool ctx_changed = false;
+
+ ctx_changed = scst_set_io_context(cmd, &old_ctx);
+
+ cmd->state = SCST_CMD_STATE_REAL_EXECUTING;
+
+ if (handler->exec) {
+ TRACE_DBG("Calling dev handler %s exec(%p)",
+ handler->name, cmd);
+ TRACE_BUFF_FLAG(TRACE_SND_TOP, "Execing: ", cmd->cdb,
+ cmd->cdb_len);
+ scst_set_cur_start(cmd);
+ res = handler->exec(cmd);
+ TRACE_DBG("Dev handler %s exec() returned %d",
+ handler->name, res);
+
+ if (res == SCST_EXEC_COMPLETED)
+ goto out_complete;
+
+ scst_set_exec_time(cmd);
+
+ BUG_ON(res != SCST_EXEC_NOT_COMPLETED);
+ }
+
+ TRACE_DBG("Sending cmd %p to SCSI mid-level", cmd);
+
+ if (unlikely(dev->scsi_dev == NULL)) {
+ PRINT_ERROR("Command for virtual device must be "
+ "processed by device handler (LUN %lld)!",
+ (long long unsigned int)cmd->lun);
+ goto out_error;
+ }
+
+ res = scst_check_local_events(cmd);
+ if (unlikely(res != 0))
+ goto out_done;
+
+ scst_set_cur_start(cmd);
+
+ rc = scst_scsi_exec_async(cmd, cmd, scst_pass_through_cmd_done);
+ if (unlikely(rc != 0)) {
+ PRINT_ERROR("scst pass-through exec failed: %x", rc);
+ goto out_error;
+ }
+
+out_complete:
+ res = SCST_EXEC_COMPLETED;
+
+ if (ctx_changed)
+ scst_reset_io_context(cmd->tgt_dev, old_ctx);
+ return res;
+
+out_error:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_done;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out_complete;
+}
+
+static inline int scst_real_exec(struct scst_cmd *cmd)
+{
+ int res;
+
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_SAME != SCST_EXEC_NOT_COMPLETED);
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_NEXT != SCST_EXEC_COMPLETED);
+
+ __scst_cmd_get(cmd);
+
+ res = scst_do_real_exec(cmd);
+
+ if (likely(res == SCST_EXEC_COMPLETED)) {
+ scst_post_exec_sn(cmd, true);
+ if (cmd->dev->scsi_dev != NULL)
+ generic_unplug_device(
+ cmd->dev->scsi_dev->request_queue);
+ } else
+ BUG();
+
+ __scst_cmd_put(cmd);
+
+ /* SCST_EXEC_* match SCST_CMD_STATE_RES_* */
+ return res;
+}
+
+static int scst_do_local_exec(struct scst_cmd *cmd)
+{
+ int res;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ /* Check READ_ONLY device status */
+ if ((cmd->op_flags & SCST_WRITE_MEDIUM) &&
+ (tgt_dev->acg_dev->rd_only || cmd->dev->swp ||
+ cmd->dev->rd_only)) {
+ PRINT_WARNING("Attempt of write access to read-only device: "
+ "initiator %s, LUN %lld, op %x",
+ cmd->sess->initiator_name, cmd->lun, cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_data_protect));
+ goto out_done;
+ }
+
+ if (!scst_is_cmd_local(cmd)) {
+ res = SCST_EXEC_NOT_COMPLETED;
+ goto out;
+ }
+
+ switch (cmd->cdb[0]) {
+ case MODE_SELECT:
+ case MODE_SELECT_10:
+ case LOG_SELECT:
+ res = scst_pre_select(cmd);
+ break;
+ case RESERVE:
+ case RESERVE_10:
+ res = scst_reserve_local(cmd);
+ break;
+ case RELEASE:
+ case RELEASE_10:
+ res = scst_release_local(cmd);
+ break;
+ case PERSISTENT_RESERVE_IN:
+ res = scst_persistent_reserve_in_local(cmd);
+ break;
+ case PERSISTENT_RESERVE_OUT:
+ res = scst_persistent_reserve_out_local(cmd);
+ break;
+ case REPORT_LUNS:
+ res = scst_report_luns_local(cmd);
+ break;
+ case REQUEST_SENSE:
+ res = scst_request_sense_local(cmd);
+ break;
+ default:
+ res = SCST_EXEC_NOT_COMPLETED;
+ break;
+ }
+
+out:
+ return res;
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ res = SCST_EXEC_COMPLETED;
+ goto out;
+}
+
+static int scst_local_exec(struct scst_cmd *cmd)
+{
+ int res;
+
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_SAME != SCST_EXEC_NOT_COMPLETED);
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_NEXT != SCST_EXEC_COMPLETED);
+
+ __scst_cmd_get(cmd);
+
+ res = scst_do_local_exec(cmd);
+ if (likely(res == SCST_EXEC_NOT_COMPLETED))
+ cmd->state = SCST_CMD_STATE_REAL_EXEC;
+ else if (res == SCST_EXEC_COMPLETED)
+ scst_post_exec_sn(cmd, true);
+ else
+ BUG();
+
+ __scst_cmd_put(cmd);
+
+ /* SCST_EXEC_* match SCST_CMD_STATE_RES_* */
+ return res;
+}
+
+static int scst_exec(struct scst_cmd **active_cmd)
+{
+ struct scst_cmd *cmd = *active_cmd;
+ struct scst_cmd *ref_cmd;
+ struct scst_device *dev = cmd->dev;
+ int res = SCST_CMD_STATE_RES_CONT_NEXT, count;
+
+ if (unlikely(scst_inc_on_dev_cmd(cmd) != 0))
+ goto out;
+
+ /* To protect tgt_dev */
+ ref_cmd = cmd;
+ __scst_cmd_get(ref_cmd);
+
+ count = 0;
+ while (1) {
+ int rc;
+
+ cmd->sent_for_exec = 1;
+ /*
+ * To sync with scst_abort_cmd(). The above assignment must
+ * be before SCST_CMD_ABORTED test, done later in
+ * scst_check_local_events(). It's far from here, so the order
+ * is virtually guaranteed, but let's have it just in case.
+ */
+ smp_mb();
+
+ cmd->scst_cmd_done = scst_cmd_done_local;
+ cmd->state = SCST_CMD_STATE_LOCAL_EXEC;
+
+ rc = scst_do_local_exec(cmd);
+ if (likely(rc == SCST_EXEC_NOT_COMPLETED))
+ /* Nothing to do */;
+ else {
+ BUG_ON(rc != SCST_EXEC_COMPLETED);
+ goto done;
+ }
+
+ cmd->state = SCST_CMD_STATE_REAL_EXEC;
+
+ rc = scst_do_real_exec(cmd);
+ BUG_ON(rc != SCST_EXEC_COMPLETED);
+
+done:
+ count++;
+
+ cmd = scst_post_exec_sn(cmd, false);
+ if (cmd == NULL)
+ break;
+
+ if (unlikely(scst_inc_on_dev_cmd(cmd) != 0))
+ break;
+
+ __scst_cmd_put(ref_cmd);
+ ref_cmd = cmd;
+ __scst_cmd_get(ref_cmd);
+ }
+
+ *active_cmd = cmd;
+
+ if (count == 0)
+ goto out_put;
+
+ if (dev->scsi_dev != NULL)
+ generic_unplug_device(dev->scsi_dev->request_queue);
+
+out_put:
+ __scst_cmd_put(ref_cmd);
+ /* !! At this point sess, dev and tgt_dev can be already freed !! */
+
+out:
+ return res;
+}
+
+static int scst_send_for_exec(struct scst_cmd **active_cmd)
+{
+ int res;
+ struct scst_cmd *cmd = *active_cmd;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ typeof(tgt_dev->expected_sn) expected_sn;
+
+ if (unlikely(cmd->internal))
+ goto exec;
+
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ goto exec;
+
+ BUG_ON(!cmd->sn_set);
+
+ expected_sn = tgt_dev->expected_sn;
+ /* Optimized for lockless fast path */
+ if ((cmd->sn != expected_sn) || (tgt_dev->hq_cmd_count > 0)) {
+ spin_lock_irq(&tgt_dev->sn_lock);
+
+ tgt_dev->def_cmd_count++;
+ /*
+ * Memory barrier is needed here to implement lockless fast
+ * path. We need the exact order of read and write between
+ * def_cmd_count and expected_sn. Otherwise, we can miss case,
+ * when expected_sn was changed to be equal to cmd->sn while
+ * we are queuing cmd the deferred list after the expected_sn
+ * below. It will lead to a forever stuck command. But with
+ * the barrier in such case __scst_check_deferred_commands()
+ * will be called and it will take sn_lock, so we will be
+ * synchronized.
+ */
+ smp_mb();
+
+ expected_sn = tgt_dev->expected_sn;
+ if ((cmd->sn != expected_sn) || (tgt_dev->hq_cmd_count > 0)) {
+ if (unlikely(test_bit(SCST_CMD_ABORTED,
+ &cmd->cmd_flags))) {
+ /* Necessary to allow aborting out of sn cmds */
+ TRACE_MGMT_DBG("Aborting out of sn cmd %p "
+ "(tag %llu, sn %u)", cmd,
+ (long long unsigned)cmd->tag, cmd->sn);
+ tgt_dev->def_cmd_count--;
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ } else {
+ TRACE_SN("Deferring cmd %p (sn=%d, set %d, "
+ "expected_sn=%d)", cmd, cmd->sn,
+ cmd->sn_set, expected_sn);
+ list_add_tail(&cmd->sn_cmd_list_entry,
+ &tgt_dev->deferred_cmd_list);
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ }
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ goto out;
+ } else {
+ TRACE_SN("Somebody incremented expected_sn %d, "
+ "continuing", expected_sn);
+ tgt_dev->def_cmd_count--;
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ }
+ }
+
+exec:
+ res = scst_exec(active_cmd);
+
+out:
+ return res;
+}
+
+/* No locks supposed to be held */
+static int scst_check_sense(struct scst_cmd *cmd)
+{
+ int res = 0;
+ struct scst_device *dev = cmd->dev;
+
+ if (unlikely(cmd->ua_ignore))
+ goto out;
+
+ /* If we had internal bus reset behind us, set the command error UA */
+ if ((dev->scsi_dev != NULL) &&
+ unlikely(cmd->host_status == DID_RESET) &&
+ scst_is_ua_command(cmd)) {
+ TRACE(TRACE_MGMT, "DID_RESET: was_reset=%d host_status=%x",
+ dev->scsi_dev->was_reset, cmd->host_status);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_reset_UA));
+ /* It looks like it is safe to clear was_reset here */
+ dev->scsi_dev->was_reset = 0;
+ }
+
+ if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ SCST_SENSE_VALID(cmd->sense)) {
+ PRINT_BUFF_FLAG(TRACE_SCSI, "Sense", cmd->sense,
+ cmd->sense_valid_len);
+
+ /* Check Unit Attention Sense Key */
+ if (scst_is_ua_sense(cmd->sense, cmd->sense_valid_len)) {
+ if (scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, SCST_SENSE_ASC_UA_RESET, 0)) {
+ if (cmd->double_ua_possible) {
+ TRACE_MGMT_DBG("Double UA "
+ "detected for device %p", dev);
+ TRACE_MGMT_DBG("Retrying cmd"
+ " %p (tag %llu)", cmd,
+ (long long unsigned)cmd->tag);
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+ cmd->completed = 0;
+
+ mempool_free(cmd->sense,
+ scst_sense_mempool);
+ cmd->sense = NULL;
+
+ scst_check_restore_sg_buff(cmd);
+
+ BUG_ON(cmd->dbl_ua_orig_resp_data_len < 0);
+ cmd->data_direction =
+ cmd->dbl_ua_orig_data_direction;
+ cmd->resp_data_len =
+ cmd->dbl_ua_orig_resp_data_len;
+
+ cmd->state = SCST_CMD_STATE_REAL_EXEC;
+ cmd->retry = 1;
+ res = 1;
+ goto out;
+ }
+ }
+ scst_dev_check_set_UA(dev, cmd, cmd->sense,
+ cmd->sense_valid_len);
+ }
+ }
+
+ if (unlikely(cmd->double_ua_possible)) {
+ if (scst_is_ua_command(cmd)) {
+ TRACE_MGMT_DBG("Clearing dbl_ua_possible flag (dev %p, "
+ "cmd %p)", dev, cmd);
+ /*
+ * Lock used to protect other flags in the bitfield
+ * (just in case, actually). Those flags can't be
+ * changed in parallel, because the device is
+ * serialized.
+ */
+ spin_lock_bh(&dev->dev_lock);
+ dev->dev_double_ua_possible = 0;
+ spin_unlock_bh(&dev->dev_lock);
+ }
+ }
+
+out:
+ return res;
+}
+
+static int scst_check_auto_sense(struct scst_cmd *cmd)
+{
+ int res = 0;
+
+ if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ (!SCST_SENSE_VALID(cmd->sense) ||
+ SCST_NO_SENSE(cmd->sense))) {
+ TRACE(TRACE_SCSI|TRACE_MINOR_AND_MGMT_DBG, "CHECK_CONDITION, "
+ "but no sense: cmd->status=%x, cmd->msg_status=%x, "
+ "cmd->host_status=%x, cmd->driver_status=%x (cmd %p)",
+ cmd->status, cmd->msg_status, cmd->host_status,
+ cmd->driver_status, cmd);
+ res = 1;
+ } else if (unlikely(cmd->host_status)) {
+ if ((cmd->host_status == DID_REQUEUE) ||
+ (cmd->host_status == DID_IMM_RETRY) ||
+ (cmd->host_status == DID_SOFT_ERROR) ||
+ (cmd->host_status == DID_ABORT)) {
+ scst_set_busy(cmd);
+ } else {
+ TRACE(TRACE_SCSI|TRACE_MINOR_AND_MGMT_DBG, "Host "
+ "status %x received, returning HARDWARE ERROR "
+ "instead (cmd %p)", cmd->host_status, cmd);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ }
+ return res;
+}
+
+static int scst_pre_dev_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME, rc;
+
+ if (unlikely(scst_check_auto_sense(cmd))) {
+ PRINT_INFO("Command finished with CHECK CONDITION, but "
+ "without sense data (opcode 0x%x), issuing "
+ "REQUEST SENSE", cmd->cdb[0]);
+ rc = scst_prepare_request_sense(cmd);
+ if (rc == 0)
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ else {
+ PRINT_ERROR("%s", "Unable to issue REQUEST SENSE, "
+ "returning HARDWARE ERROR");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out;
+ } else if (unlikely(scst_check_sense(cmd)))
+ goto out;
+
+ if (likely(scsi_status_is_good(cmd->status))) {
+ unsigned char type = cmd->dev->type;
+ if (unlikely((cmd->cdb[0] == MODE_SENSE ||
+ cmd->cdb[0] == MODE_SENSE_10)) &&
+ (cmd->tgt_dev->acg_dev->rd_only || cmd->dev->swp ||
+ cmd->dev->rd_only) &&
+ (type == TYPE_DISK ||
+ type == TYPE_WORM ||
+ type == TYPE_MOD ||
+ type == TYPE_TAPE)) {
+ int32_t length;
+ uint8_t *address;
+ bool err = false;
+
+ length = scst_get_buf_first(cmd, &address);
+ if (length < 0) {
+ PRINT_ERROR("%s", "Unable to get "
+ "MODE_SENSE buffer");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(
+ scst_sense_hardw_error));
+ err = true;
+ } else if (length > 2 && cmd->cdb[0] == MODE_SENSE)
+ address[2] |= 0x80; /* Write Protect*/
+ else if (length > 3 && cmd->cdb[0] == MODE_SENSE_10)
+ address[3] |= 0x80; /* Write Protect*/
+ scst_put_buf(cmd, address);
+
+ if (err)
+ goto out;
+ }
+
+ /*
+ * Check and clear NormACA option for the device, if necessary,
+ * since we don't support ACA
+ */
+ if (unlikely((cmd->cdb[0] == INQUIRY)) &&
+ /* Std INQUIRY data (no EVPD) */
+ !(cmd->cdb[1] & SCST_INQ_EVPD) &&
+ (cmd->resp_data_len > SCST_INQ_BYTE3)) {
+ uint8_t *buffer;
+ int buflen;
+ bool err = false;
+
+ /* ToDo: all pages ?? */
+ buflen = scst_get_buf_first(cmd, &buffer);
+ if (buflen > SCST_INQ_BYTE3) {
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (buffer[SCST_INQ_BYTE3] & SCST_INQ_NORMACA_BIT) {
+ PRINT_INFO("NormACA set for device: "
+ "lun=%lld, type 0x%02x. Clear it, "
+ "since it's unsupported.",
+ (long long unsigned int)cmd->lun,
+ buffer[0]);
+ }
+#endif
+ buffer[SCST_INQ_BYTE3] &= ~SCST_INQ_NORMACA_BIT;
+ } else if (buflen != 0) {
+ PRINT_ERROR("%s", "Unable to get INQUIRY "
+ "buffer");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ err = true;
+ }
+ if (buflen > 0)
+ scst_put_buf(cmd, buffer);
+
+ if (err)
+ goto out;
+ }
+
+ if (unlikely((cmd->cdb[0] == MODE_SELECT) ||
+ (cmd->cdb[0] == MODE_SELECT_10) ||
+ (cmd->cdb[0] == LOG_SELECT))) {
+ TRACE(TRACE_SCSI,
+ "MODE/LOG SELECT succeeded (LUN %lld)",
+ (long long unsigned int)cmd->lun);
+ cmd->state = SCST_CMD_STATE_MODE_SELECT_CHECKS;
+ goto out;
+ }
+ } else {
+ TRACE(TRACE_SCSI, "cmd %p not succeeded with status %x",
+ cmd, cmd->status);
+
+ if ((cmd->cdb[0] == RESERVE) || (cmd->cdb[0] == RESERVE_10)) {
+ if (!test_bit(SCST_TGT_DEV_RESERVED,
+ &cmd->tgt_dev->tgt_dev_flags)) {
+ struct scst_tgt_dev *tgt_dev_tmp;
+ struct scst_device *dev = cmd->dev;
+
+ TRACE(TRACE_SCSI, "RESERVE failed lun=%lld, "
+ "status=%x",
+ (long long unsigned int)cmd->lun,
+ cmd->status);
+ PRINT_BUFF_FLAG(TRACE_SCSI, "Sense", cmd->sense,
+ cmd->sense_valid_len);
+
+ /* Clearing the reservation */
+ spin_lock_bh(&dev->dev_lock);
+ list_for_each_entry(tgt_dev_tmp,
+ &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ spin_unlock_bh(&dev->dev_lock);
+ }
+ }
+
+ /* Check for MODE PARAMETERS CHANGED UA */
+ if ((cmd->dev->scsi_dev != NULL) &&
+ (cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ scst_is_ua_sense(cmd->sense, cmd->sense_valid_len) &&
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASCx_VALID,
+ 0, 0x2a, 0x01)) {
+ TRACE(TRACE_SCSI, "MODE PARAMETERS CHANGED UA (lun "
+ "%lld)", (long long unsigned int)cmd->lun);
+ cmd->state = SCST_CMD_STATE_MODE_SELECT_CHECKS;
+ goto out;
+ }
+ }
+
+ cmd->state = SCST_CMD_STATE_DEV_DONE;
+
+out:
+ return res;
+}
+
+static int scst_mode_select_checks(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+
+ if (likely(scsi_status_is_good(cmd->status))) {
+ int atomic = scst_cmd_atomic(cmd);
+ if (unlikely((cmd->cdb[0] == MODE_SELECT) ||
+ (cmd->cdb[0] == MODE_SELECT_10) ||
+ (cmd->cdb[0] == LOG_SELECT))) {
+ struct scst_device *dev = cmd->dev;
+ int sl;
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+
+ if (atomic && (dev->scsi_dev != NULL)) {
+ TRACE_DBG("%s", "MODE/LOG SELECT: thread "
+ "context required");
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE(TRACE_SCSI, "MODE/LOG SELECT succeeded, "
+ "setting the SELECT UA (lun=%lld)",
+ (long long unsigned int)cmd->lun);
+
+ spin_lock_bh(&dev->dev_lock);
+ if (cmd->cdb[0] == LOG_SELECT) {
+ sl = scst_set_sense(sense_buffer,
+ sizeof(sense_buffer),
+ dev->d_sense,
+ UNIT_ATTENTION, 0x2a, 0x02);
+ } else {
+ sl = scst_set_sense(sense_buffer,
+ sizeof(sense_buffer),
+ dev->d_sense,
+ UNIT_ATTENTION, 0x2a, 0x01);
+ }
+ scst_dev_check_set_local_UA(dev, cmd, sense_buffer, sl);
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (dev->scsi_dev != NULL)
+ scst_obtain_device_parameters(dev);
+ }
+ } else if ((cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ scst_is_ua_sense(cmd->sense, cmd->sense_valid_len) &&
+ /* mode parameters changed */
+ (scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASCx_VALID,
+ 0, 0x2a, 0x01) ||
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, 0x29, 0) /* reset */ ||
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, 0x28, 0) /* medium changed */ ||
+ /* cleared by another ini (just in case) */
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, 0x2F, 0))) {
+ int atomic = scst_cmd_atomic(cmd);
+ if (atomic) {
+ TRACE_DBG("Possible parameters changed UA %x: "
+ "thread context required", cmd->sense[12]);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE(TRACE_SCSI, "Possible parameters changed UA %x "
+ "(LUN %lld): getting new parameters", cmd->sense[12],
+ (long long unsigned int)cmd->lun);
+
+ scst_obtain_device_parameters(cmd->dev);
+ } else
+ BUG();
+
+ cmd->state = SCST_CMD_STATE_DEV_DONE;
+
+out:
+ return res;
+}
+
+static void scst_inc_check_expected_sn(struct scst_cmd *cmd)
+{
+ if (likely(cmd->sn_set))
+ scst_inc_expected_sn(cmd->tgt_dev, cmd->sn_slot);
+
+ scst_make_deferred_commands_active(cmd->tgt_dev);
+}
+
+static int scst_dev_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+ int state;
+ struct scst_device *dev = cmd->dev;
+
+ state = SCST_CMD_STATE_PRE_XMIT_RESP;
+
+ if (likely(!scst_is_cmd_fully_local(cmd)) &&
+ likely(dev->handler->dev_done != NULL)) {
+ int rc;
+
+ if (unlikely(!dev->handler->dev_done_atomic &&
+ scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Dev handler %s dev_done() needs thread "
+ "context, rescheduling", dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE_DBG("Calling dev handler %s dev_done(%p)",
+ dev->handler->name, cmd);
+ scst_set_cur_start(cmd);
+ rc = dev->handler->dev_done(cmd);
+ scst_set_dev_done_time(cmd);
+ TRACE_DBG("Dev handler %s dev_done() returned %d",
+ dev->handler->name, rc);
+ if (rc != SCST_CMD_STATE_DEFAULT)
+ state = rc;
+ }
+
+ switch (state) {
+#ifdef CONFIG_SCST_EXTRACHECKS
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ case SCST_CMD_STATE_DEV_PARSE:
+ case SCST_CMD_STATE_PRE_PARSE:
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ case SCST_CMD_STATE_MODE_SELECT_CHECKS:
+ case SCST_CMD_STATE_DEV_DONE:
+ case SCST_CMD_STATE_XMIT_RESP:
+ case SCST_CMD_STATE_FINISHED:
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+#else
+ default:
+#endif
+ cmd->state = state;
+ break;
+ case SCST_CMD_STATE_NEED_THREAD_CTX:
+ TRACE_DBG("Dev handler %s dev_done() requested "
+ "thread context, rescheduling",
+ dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ default:
+ if (state >= 0) {
+ PRINT_ERROR("Dev handler %s dev_done() returned "
+ "invalid cmd state %d",
+ dev->handler->name, state);
+ } else {
+ PRINT_ERROR("Dev handler %s dev_done() returned "
+ "error %d", dev->handler->name,
+ state);
+ }
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+#endif
+ }
+
+ if (cmd->needs_unblocking)
+ scst_unblock_dev_cmd(cmd);
+
+ if (likely(cmd->dec_on_dev_needed))
+ scst_dec_on_dev_cmd(cmd);
+
+ if (cmd->inc_expected_sn_on_done && cmd->sent_for_exec)
+ scst_inc_check_expected_sn(cmd);
+
+ if (unlikely(cmd->internal))
+ cmd->state = SCST_CMD_STATE_FINISHED_INTERNAL;
+
+out:
+ return res;
+}
+
+static int scst_pre_xmit_response(struct scst_cmd *cmd)
+{
+ int res;
+
+ EXTRACHECKS_BUG_ON(cmd->internal);
+
+#ifdef CONFIG_SCST_DEBUG_TM
+ if (cmd->tm_dbg_delayed &&
+ !test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ if (scst_cmd_atomic(cmd)) {
+ TRACE_MGMT_DBG("%s",
+ "DEBUG_TM delayed cmd needs a thread");
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ return res;
+ }
+ TRACE_MGMT_DBG("Delaying cmd %p (tag %llu) for 1 second",
+ cmd, cmd->tag);
+ schedule_timeout_uninterruptible(HZ);
+ }
+#endif
+
+ if (likely(cmd->tgt_dev != NULL)) {
+ /*
+ * Those counters protect from not getting too long processing
+ * latency, so we should decrement them after cmd completed.
+ */
+ atomic_dec(&cmd->tgt_dev->tgt_dev_cmd_count);
+#ifdef CONFIG_SCST_PER_DEVICE_CMD_COUNT_LIMIT
+ atomic_dec(&cmd->dev->dev_cmd_count);
+#endif
+#ifdef CONFIG_SCST_ORDERED_READS
+ /* If expected values not set, expected direction is UNKNOWN */
+ if (cmd->expected_data_direction & SCST_DATA_WRITE)
+ atomic_dec(&cmd->dev->write_cmd_count);
+#endif
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ scst_on_hq_cmd_response(cmd);
+
+ if (unlikely(!cmd->sent_for_exec)) {
+ TRACE_SN("cmd %p was not sent to mid-lev"
+ " (sn %d, set %d)",
+ cmd, cmd->sn, cmd->sn_set);
+ scst_unblock_deferred(cmd->tgt_dev, cmd);
+ cmd->sent_for_exec = 1;
+ }
+ }
+
+ cmd->done = 1;
+ smp_mb(); /* to sync with scst_abort_cmd() */
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)))
+ scst_xmit_process_aborted_cmd(cmd);
+ else if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION))
+ scst_store_sense(cmd);
+
+ if (unlikely(test_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("Flag NO_RESP set for cmd %p (tag %llu), "
+ "skipping", cmd, (long long unsigned int)cmd->tag);
+ cmd->state = SCST_CMD_STATE_FINISHED;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+ }
+
+ if (unlikely(cmd->resid_possible))
+ scst_adjust_resp_data_len(cmd);
+ else
+ cmd->adjusted_resp_data_len = cmd->resp_data_len;
+
+ cmd->state = SCST_CMD_STATE_XMIT_RESP;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+
+out:
+ return res;
+}
+
+static int scst_xmit_response(struct scst_cmd *cmd)
+{
+ struct scst_tgt_template *tgtt = cmd->tgtt;
+ int res, rc;
+
+ EXTRACHECKS_BUG_ON(cmd->internal);
+
+ if (unlikely(!tgtt->xmit_response_atomic &&
+ scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Target driver %s xmit_response() needs thread "
+ "context, rescheduling", tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ while (1) {
+ int finished_cmds = atomic_read(&cmd->tgt->finished_cmds);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ cmd->state = SCST_CMD_STATE_XMIT_WAIT;
+
+ TRACE_DBG("Calling xmit_response(%p)", cmd);
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (trace_flag & TRACE_SND_BOT) {
+ int i, j;
+ struct scatterlist *sg;
+ if (cmd->tgt_sg != NULL)
+ sg = cmd->tgt_sg;
+ else
+ sg = cmd->sg;
+ if (sg != NULL) {
+ TRACE(TRACE_SND_BOT, "Xmitting data for cmd %p "
+ "(sg_cnt %d, sg %p, sg[0].page %p)",
+ cmd, cmd->tgt_sg_cnt, sg,
+ (void *)sg_page(&sg[0]));
+ for (i = 0, j = 0; i < cmd->tgt_sg_cnt; ++i, ++j) {
+ if (unlikely(sg_is_chain(&sg[j]))) {
+ sg = sg_chain_ptr(&sg[j]);
+ j = 0;
+ }
+ PRINT_BUFF_FLAG(TRACE_SND_BOT,
+ "Xmitting sg", sg_virt(&sg[j]),
+ sg[j].length);
+ }
+ }
+ }
+#endif
+
+ if (tgtt->on_hw_pending_cmd_timeout != NULL) {
+ struct scst_session *sess = cmd->sess;
+ cmd->hw_pending_start = jiffies;
+ cmd->cmd_hw_pending = 1;
+ if (!test_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags)) {
+ TRACE_DBG("Sched HW pending work for sess %p "
+ "(max time %d)", sess,
+ tgtt->max_hw_pending_time);
+ set_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED,
+ &sess->sess_aflags);
+ schedule_delayed_work(&sess->hw_pending_work,
+ tgtt->max_hw_pending_time * HZ);
+ }
+ }
+
+ scst_set_cur_start(cmd);
+
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ if (((scst_random() % 100) == 77))
+ rc = SCST_TGT_RES_QUEUE_FULL;
+ else
+#endif
+ rc = tgtt->xmit_response(cmd);
+ TRACE_DBG("xmit_response() returned %d", rc);
+
+ if (likely(rc == SCST_TGT_RES_SUCCESS))
+ goto out;
+
+ scst_set_xmit_time(cmd);
+
+ cmd->cmd_hw_pending = 0;
+
+ /* Restore the previous state */
+ cmd->state = SCST_CMD_STATE_XMIT_RESP;
+
+ switch (rc) {
+ case SCST_TGT_RES_QUEUE_FULL:
+ if (scst_queue_retry_cmd(cmd, finished_cmds) == 0)
+ break;
+ else
+ continue;
+
+ case SCST_TGT_RES_NEED_THREAD_CTX:
+ TRACE_DBG("Target driver %s xmit_response() "
+ "requested thread context, rescheduling",
+ tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+
+ default:
+ goto out_error;
+ }
+ break;
+ }
+
+out:
+ /* Caution: cmd can be already dead here */
+ return res;
+
+out_error:
+ if (rc == SCST_TGT_RES_FATAL_ERROR) {
+ PRINT_ERROR("Target driver %s xmit_response() returned "
+ "fatal error", tgtt->name);
+ } else {
+ PRINT_ERROR("Target driver %s xmit_response() returned "
+ "invalid value %d", tgtt->name, rc);
+ }
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ cmd->state = SCST_CMD_STATE_FINISHED;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+/**
+ * scst_tgt_cmd_done() - the command's processing done
+ * @cmd: SCST command
+ * @pref_context: preferred command execution context
+ *
+ * Description:
+ * Notifies SCST that the driver sent the response and the command
+ * can be freed now. Don't forget to set the delivery status, if it
+ * isn't success, using scst_set_delivery_status() before calling
+ * this function. The third argument sets preferred command execition
+ * context (see SCST_CONTEXT_* constants for details)
+ */
+void scst_tgt_cmd_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context)
+{
+
+ BUG_ON(cmd->state != SCST_CMD_STATE_XMIT_WAIT);
+
+ scst_set_xmit_time(cmd);
+
+ cmd->cmd_hw_pending = 0;
+
+ cmd->state = SCST_CMD_STATE_FINISHED;
+ scst_process_redirect_cmd(cmd, pref_context, 1);
+ return;
+}
+EXPORT_SYMBOL(scst_tgt_cmd_done);
+
+static int scst_finish_cmd(struct scst_cmd *cmd)
+{
+ int res;
+ struct scst_session *sess = cmd->sess;
+
+ scst_update_lat_stats(cmd);
+
+ if (unlikely(cmd->delivery_status != SCST_CMD_DELIVERY_SUCCESS)) {
+ if ((cmd->tgt_dev != NULL) &&
+ scst_is_ua_sense(cmd->sense, cmd->sense_valid_len)) {
+ /* This UA delivery failed, so we need to requeue it */
+ if (scst_cmd_atomic(cmd) &&
+ scst_is_ua_global(cmd->sense, cmd->sense_valid_len)) {
+ TRACE_MGMT_DBG("Requeuing of global UA for "
+ "failed cmd %p needs a thread", cmd);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+ scst_requeue_ua(cmd);
+ }
+ }
+
+ atomic_dec(&sess->sess_cmd_count);
+
+ spin_lock_irq(&sess->sess_list_lock);
+ list_del(&cmd->sess_cmd_list_entry);
+ spin_unlock_irq(&sess->sess_list_lock);
+
+ cmd->finished = 1;
+ smp_mb(); /* to sync with scst_abort_cmd() */
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("Aborted cmd %p finished (cmd_ref %d, "
+ "scst_cmd_count %d)", cmd, atomic_read(&cmd->cmd_ref),
+ atomic_read(&scst_cmd_count));
+
+ scst_finish_cmd_mgmt(cmd);
+ }
+
+ __scst_cmd_put(cmd);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+
+out:
+ return res;
+}
+
+/*
+ * No locks, but it must be externally serialized (see comment for
+ * scst_cmd_init_done() in scst.h)
+ */
+static void scst_cmd_set_sn(struct scst_cmd *cmd)
+{
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ unsigned long flags;
+
+ if (scst_is_implicit_hq(cmd) &&
+ likely(cmd->queue_type == SCST_CMD_QUEUE_SIMPLE)) {
+ TRACE_SN("Implicit HQ cmd %p", cmd);
+ cmd->queue_type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ }
+
+ EXTRACHECKS_BUG_ON(cmd->sn_set || cmd->hq_cmd_inced);
+
+ /* Optimized for lockless fast path */
+
+ scst_check_debug_sn(cmd);
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ cmd->queue_type = SCST_CMD_QUEUE_ORDERED;
+#endif
+
+ if (cmd->dev->queue_alg == SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER) {
+ /*
+ * Not the best way, but good enough until there is a
+ * possibility to specify queue type during pass-through
+ * commands submission.
+ */
+ cmd->queue_type = SCST_CMD_QUEUE_ORDERED;
+ }
+
+ switch (cmd->queue_type) {
+ case SCST_CMD_QUEUE_SIMPLE:
+ case SCST_CMD_QUEUE_UNTAGGED:
+#ifdef CONFIG_SCST_ORDERED_READS
+ if (scst_cmd_is_expected_set(cmd)) {
+ if ((cmd->expected_data_direction == SCST_DATA_READ) &&
+ (atomic_read(&cmd->dev->write_cmd_count) == 0))
+ goto ordered;
+ } else
+ goto ordered;
+#endif
+ if (likely(tgt_dev->num_free_sn_slots >= 0)) {
+ /*
+ * atomic_inc_return() implies memory barrier to sync
+ * with scst_inc_expected_sn()
+ */
+ if (atomic_inc_return(tgt_dev->cur_sn_slot) == 1) {
+ tgt_dev->curr_sn++;
+ TRACE_SN("Incremented curr_sn %d",
+ tgt_dev->curr_sn);
+ }
+ cmd->sn_slot = tgt_dev->cur_sn_slot;
+ cmd->sn = tgt_dev->curr_sn;
+
+ tgt_dev->prev_cmd_ordered = 0;
+ } else {
+ TRACE(TRACE_MINOR, "***WARNING*** Not enough SN slots "
+ "%zd", ARRAY_SIZE(tgt_dev->sn_slots));
+ goto ordered;
+ }
+ break;
+
+ case SCST_CMD_QUEUE_ORDERED:
+ TRACE_SN("ORDERED cmd %p (op %x)", cmd, cmd->cdb[0]);
+ordered:
+ if (!tgt_dev->prev_cmd_ordered) {
+ spin_lock_irqsave(&tgt_dev->sn_lock, flags);
+ if (tgt_dev->num_free_sn_slots >= 0) {
+ tgt_dev->num_free_sn_slots--;
+ if (tgt_dev->num_free_sn_slots >= 0) {
+ int i = 0;
+ /* Commands can finish in any order, so
+ * we don't know which slot is empty.
+ */
+ while (1) {
+ tgt_dev->cur_sn_slot++;
+ if (tgt_dev->cur_sn_slot ==
+ tgt_dev->sn_slots + ARRAY_SIZE(tgt_dev->sn_slots))
+ tgt_dev->cur_sn_slot = tgt_dev->sn_slots;
+
+ if (atomic_read(tgt_dev->cur_sn_slot) == 0)
+ break;
+
+ i++;
+ BUG_ON(i == ARRAY_SIZE(tgt_dev->sn_slots));
+ }
+ TRACE_SN("New cur SN slot %zd",
+ tgt_dev->cur_sn_slot -
+ tgt_dev->sn_slots);
+ }
+ }
+ spin_unlock_irqrestore(&tgt_dev->sn_lock, flags);
+ }
+ tgt_dev->prev_cmd_ordered = 1;
+ tgt_dev->curr_sn++;
+ cmd->sn = tgt_dev->curr_sn;
+ break;
+
+ case SCST_CMD_QUEUE_HEAD_OF_QUEUE:
+ TRACE_SN("HQ cmd %p (op %x)", cmd, cmd->cdb[0]);
+ spin_lock_irqsave(&tgt_dev->sn_lock, flags);
+ tgt_dev->hq_cmd_count++;
+ spin_unlock_irqrestore(&tgt_dev->sn_lock, flags);
+ cmd->hq_cmd_inced = 1;
+ goto out;
+
+ default:
+ BUG();
+ }
+
+ TRACE_SN("cmd(%p)->sn: %d (tgt_dev %p, *cur_sn_slot %d, "
+ "num_free_sn_slots %d, prev_cmd_ordered %ld, "
+ "cur_sn_slot %zd)", cmd, cmd->sn, tgt_dev,
+ atomic_read(tgt_dev->cur_sn_slot),
+ tgt_dev->num_free_sn_slots, tgt_dev->prev_cmd_ordered,
+ tgt_dev->cur_sn_slot-tgt_dev->sn_slots);
+
+ cmd->sn_set = 1;
+
+out:
+ return;
+}
+
+/*
+ * Returns 0 on success, > 0 when we need to wait for unblock,
+ * < 0 if there is no device (lun) or device type handler.
+ *
+ * No locks, but might be on IRQ, protection is done by the
+ * suspended activity.
+ */
+static int scst_translate_lun(struct scst_cmd *cmd)
+{
+ struct scst_tgt_dev *tgt_dev = NULL;
+ int res;
+
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ __scst_get(1);
+
+ if (likely(!test_bit(SCST_FLAG_SUSPENDED, &scst_flags))) {
+ struct list_head *sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[HASH_VAL(cmd->lun)];
+ TRACE_DBG("Finding tgt_dev for cmd %p (lun %lld)", cmd,
+ (long long unsigned int)cmd->lun);
+ res = -1;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == cmd->lun) {
+ TRACE_DBG("tgt_dev %p found", tgt_dev);
+
+ if (unlikely(tgt_dev->dev->handler ==
+ &scst_null_devtype)) {
+ PRINT_INFO("Dev handler for device "
+ "%lld is NULL, the device will not "
+ "be visible remotely",
+ (long long unsigned int)cmd->lun);
+ break;
+ }
+
+ cmd->cmd_threads = tgt_dev->active_cmd_threads;
+ cmd->tgt_dev = tgt_dev;
+ cmd->dev = tgt_dev->dev;
+
+ res = 0;
+ break;
+ }
+ }
+ if (res != 0) {
+ TRACE(TRACE_MINOR,
+ "tgt_dev for LUN %lld not found, command to "
+ "unexisting LU?",
+ (long long unsigned int)cmd->lun);
+ __scst_put();
+ }
+ } else {
+ TRACE_MGMT_DBG("%s", "FLAG SUSPENDED set, skipping");
+ __scst_put();
+ res = 1;
+ }
+ return res;
+}
+
+/*
+ * No locks, but might be on IRQ.
+ *
+ * Returns 0 on success, > 0 when we need to wait for unblock,
+ * < 0 if there is no device (lun) or device type handler.
+ */
+static int __scst_init_cmd(struct scst_cmd *cmd)
+{
+ int res = 0;
+
+ res = scst_translate_lun(cmd);
+ if (likely(res == 0)) {
+ int cnt;
+ bool failure = false;
+
+ cmd->state = SCST_CMD_STATE_PRE_PARSE;
+
+ cnt = atomic_inc_return(&cmd->tgt_dev->tgt_dev_cmd_count);
+ if (unlikely(cnt > SCST_MAX_TGT_DEV_COMMANDS)) {
+ TRACE(TRACE_FLOW_CONTROL,
+ "Too many pending commands (%d) in "
+ "session, returning BUSY to initiator \"%s\"",
+ cnt, (cmd->sess->initiator_name[0] == '\0') ?
+ "Anonymous" : cmd->sess->initiator_name);
+ failure = true;
+ }
+
+#ifdef CONFIG_SCST_PER_DEVICE_CMD_COUNT_LIMIT
+ cnt = atomic_inc_return(&cmd->dev->dev_cmd_count);
+ if (unlikely(cnt > SCST_MAX_DEV_COMMANDS)) {
+ if (!failure) {
+ TRACE(TRACE_FLOW_CONTROL,
+ "Too many pending device "
+ "commands (%d), returning BUSY to "
+ "initiator \"%s\"", cnt,
+ (cmd->sess->initiator_name[0] == '\0') ?
+ "Anonymous" :
+ cmd->sess->initiator_name);
+ failure = true;
+ }
+ }
+#endif
+
+#ifdef CONFIG_SCST_ORDERED_READS
+ /* If expected values not set, expected direction is UNKNOWN */
+ if (cmd->expected_data_direction & SCST_DATA_WRITE)
+ atomic_inc(&cmd->dev->write_cmd_count);
+#endif
+
+ if (unlikely(failure))
+ goto out_busy;
+
+ if (!cmd->set_sn_on_restart_cmd)
+ scst_cmd_set_sn(cmd);
+ } else if (res < 0) {
+ TRACE_DBG("Finishing cmd %p", cmd);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_lun_not_supported));
+ scst_set_cmd_abnormal_done_state(cmd);
+ } else
+ goto out;
+
+out:
+ return res;
+
+out_busy:
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ goto out;
+}
+
+/* Called under scst_init_lock and IRQs disabled */
+static void scst_do_job_init(void)
+ __releases(&scst_init_lock)
+ __acquires(&scst_init_lock)
+{
+ struct scst_cmd *cmd;
+ int susp;
+
+restart:
+ /*
+ * There is no need for read barrier here, because we don't care where
+ * this check will be done.
+ */
+ susp = test_bit(SCST_FLAG_SUSPENDED, &scst_flags);
+ if (scst_init_poll_cnt > 0)
+ scst_init_poll_cnt--;
+
+ list_for_each_entry(cmd, &scst_init_cmd_list, cmd_list_entry) {
+ int rc;
+ if (susp && !test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))
+ continue;
+ if (!test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ spin_unlock_irq(&scst_init_lock);
+ rc = __scst_init_cmd(cmd);
+ spin_lock_irq(&scst_init_lock);
+ if (rc > 0) {
+ TRACE_MGMT_DBG("%s",
+ "FLAG SUSPENDED set, restarting");
+ goto restart;
+ }
+ } else {
+ TRACE_MGMT_DBG("Aborting not inited cmd %p (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+ scst_set_cmd_abnormal_done_state(cmd);
+ }
+
+ /*
+ * Deleting cmd from init cmd list after __scst_init_cmd()
+ * is necessary to keep the check in scst_init_cmd() correct
+ * to preserve the commands order.
+ *
+ * We don't care about the race, when init cmd list is empty
+ * and one command detected that it just was not empty, so
+ * it's inserting to it, but another command at the same time
+ * seeing init cmd list empty and goes directly, because it
+ * could affect only commands from the same initiator to the
+ * same tgt_dev, but scst_cmd_init_done*() doesn't guarantee
+ * the order in case of simultaneous such calls anyway.
+ */
+ TRACE_MGMT_DBG("Deleting cmd %p from init cmd list", cmd);
+ smp_wmb(); /* enforce the required order */
+ list_del(&cmd->cmd_list_entry);
+ spin_unlock(&scst_init_lock);
+
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ TRACE_MGMT_DBG("Adding cmd %p to active cmd list", cmd);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+
+ spin_lock(&scst_init_lock);
+ goto restart;
+ }
+
+ /* It isn't really needed, but let's keep it */
+ if (susp != test_bit(SCST_FLAG_SUSPENDED, &scst_flags))
+ goto restart;
+ return;
+}
+
+static inline int test_init_cmd_list(void)
+{
+ int res = (!list_empty(&scst_init_cmd_list) &&
+ !test_bit(SCST_FLAG_SUSPENDED, &scst_flags)) ||
+ unlikely(kthread_should_stop()) ||
+ (scst_init_poll_cnt > 0);
+ return res;
+}
+
+int scst_init_thread(void *arg)
+{
+
+ PRINT_INFO("Init thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ set_user_nice(current, -10);
+
+ spin_lock_irq(&scst_init_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_init_cmd_list()) {
+ add_wait_queue_exclusive(&scst_init_cmd_list_waitQ,
+ &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_init_cmd_list())
+ break;
+ spin_unlock_irq(&scst_init_lock);
+ schedule();
+ spin_lock_irq(&scst_init_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&scst_init_cmd_list_waitQ, &wait);
+ }
+ scst_do_job_init();
+ }
+ spin_unlock_irq(&scst_init_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so scst_init_cmd_list must be empty.
+ */
+ BUG_ON(!list_empty(&scst_init_cmd_list));
+
+ PRINT_INFO("Init thread PID %d finished", current->pid);
+ return 0;
+}
+
+/**
+ * scst_process_active_cmd() - process active command
+ *
+ * Description:
+ * Main SCST commands processing routing. Must be used only by dev handlers.
+ *
+ * Argument atomic is true, if function called in atomic context.
+ *
+ * Must be called with no locks held.
+ */
+void scst_process_active_cmd(struct scst_cmd *cmd, bool atomic)
+{
+ int res;
+
+ /*
+ * Checkpatch will complain on the use of in_atomic() below. You
+ * can safely ignore this warning since in_atomic() is used here only
+ * for debugging purposes.
+ */
+ EXTRACHECKS_BUG_ON(in_irq() || irqs_disabled());
+ EXTRACHECKS_WARN_ON((in_atomic() || in_interrupt() || irqs_disabled()) &&
+ !atomic);
+
+ cmd->atomic = atomic;
+
+ TRACE_DBG("cmd %p, atomic %d", cmd, atomic);
+
+ do {
+ switch (cmd->state) {
+ case SCST_CMD_STATE_PRE_PARSE:
+ res = scst_pre_parse(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ case SCST_CMD_STATE_DEV_PARSE:
+ res = scst_parse_cmd(cmd);
+ break;
+
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ res = scst_prepare_space(cmd);
+ break;
+
+ case SCST_CMD_STATE_PREPROCESSING_DONE:
+ res = scst_preprocessing_done(cmd);
+ break;
+
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ res = scst_rdy_to_xfer(cmd);
+ break;
+
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ res = scst_tgt_pre_exec(cmd);
+ break;
+
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ if (tm_dbg_check_cmd(cmd) != 0) {
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ TRACE_MGMT_DBG("Skipping cmd %p (tag %llu), "
+ "because of TM DBG delay", cmd,
+ (long long unsigned int)cmd->tag);
+ break;
+ }
+ res = scst_send_for_exec(&cmd);
+ /*
+ * !! At this point cmd, sess & tgt_dev can already be
+ * freed !!
+ */
+ break;
+
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ res = scst_local_exec(cmd);
+ /*
+ * !! At this point cmd, sess & tgt_dev can already be
+ * freed !!
+ */
+ break;
+
+ case SCST_CMD_STATE_REAL_EXEC:
+ res = scst_real_exec(cmd);
+ /*
+ * !! At this point cmd, sess & tgt_dev can already be
+ * freed !!
+ */
+ break;
+
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ res = scst_pre_dev_done(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ case SCST_CMD_STATE_MODE_SELECT_CHECKS:
+ res = scst_mode_select_checks(cmd);
+ break;
+
+ case SCST_CMD_STATE_DEV_DONE:
+ res = scst_dev_done(cmd);
+ break;
+
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ res = scst_pre_xmit_response(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ case SCST_CMD_STATE_XMIT_RESP:
+ res = scst_xmit_response(cmd);
+ break;
+
+ case SCST_CMD_STATE_FINISHED:
+ res = scst_finish_cmd(cmd);
+ break;
+
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+ res = scst_finish_internal_cmd(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ default:
+ PRINT_CRIT_ERROR("cmd (%p) in state %d, but shouldn't "
+ "be", cmd, cmd->state);
+ BUG();
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ break;
+ }
+ } while (res == SCST_CMD_STATE_RES_CONT_SAME);
+
+ if (res == SCST_CMD_STATE_RES_CONT_NEXT) {
+ /* None */
+ } else if (res == SCST_CMD_STATE_RES_NEED_THREAD) {
+ spin_lock_irq(&cmd->cmd_threads->cmd_list_lock);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ switch (cmd->state) {
+ case SCST_CMD_STATE_DEV_PARSE:
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_DEV_DONE:
+ case SCST_CMD_STATE_XMIT_RESP:
+#endif
+ TRACE_DBG("Adding cmd %p to head of active cmd list",
+ cmd);
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ break;
+ default:
+ PRINT_CRIT_ERROR("cmd %p is in invalid state %d)", cmd,
+ cmd->state);
+ spin_unlock_irq(&cmd->cmd_threads->cmd_list_lock);
+ BUG();
+ spin_lock_irq(&cmd->cmd_threads->cmd_list_lock);
+ break;
+ }
+#endif
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&cmd->cmd_threads->cmd_list_lock);
+ } else
+ BUG();
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_process_active_cmd);
+
+/* Called under cmd_list_lock and IRQs disabled */
+static void scst_do_job_active(struct list_head *cmd_list,
+ spinlock_t *cmd_list_lock, bool atomic)
+ __releases(cmd_list_lock)
+ __acquires(cmd_list_lock)
+{
+
+ while (!list_empty(cmd_list)) {
+ struct scst_cmd *cmd = list_entry(cmd_list->next, typeof(*cmd),
+ cmd_list_entry);
+ TRACE_DBG("Deleting cmd %p from active cmd list", cmd);
+ list_del(&cmd->cmd_list_entry);
+ spin_unlock_irq(cmd_list_lock);
+ scst_process_active_cmd(cmd, atomic);
+ spin_lock_irq(cmd_list_lock);
+ }
+ return;
+}
+
+static inline int test_cmd_threads(struct scst_cmd_threads *p_cmd_threads)
+{
+ int res = !list_empty(&p_cmd_threads->active_cmd_list) ||
+ unlikely(kthread_should_stop()) ||
+ tm_dbg_is_release();
+ return res;
+}
+
+int scst_cmd_thread(void *arg)
+{
+ struct scst_cmd_threads *p_cmd_threads = (struct scst_cmd_threads *)arg;
+ static DEFINE_MUTEX(io_context_mutex);
+
+ PRINT_INFO("Processing thread %s (PID %d) started", current->comm,
+ current->pid);
+
+#if 0
+ set_user_nice(current, 10);
+#endif
+ current->flags |= PF_NOFREEZE;
+
+ mutex_lock(&io_context_mutex);
+
+ WARN_ON(current->io_context);
+
+ if (p_cmd_threads != &scst_main_cmd_threads) {
+ if (p_cmd_threads->io_context == NULL) {
+ p_cmd_threads->io_context = get_io_context(GFP_KERNEL, -1);
+ TRACE_MGMT_DBG("Alloced new IO context %p "
+ "(p_cmd_threads %p)",
+ p_cmd_threads->io_context,
+ p_cmd_threads);
+ /*
+ * Put the extra reference. It isn't needed, because we
+ * ref counted via nr_threads below.
+ */
+ put_io_context(p_cmd_threads->io_context);
+ } else {
+ put_io_context(current->io_context);
+ current->io_context = ioc_task_link(p_cmd_threads->io_context);
+ TRACE_MGMT_DBG("Linked IO context %p "
+ "(p_cmd_threads %p)", p_cmd_threads->io_context,
+ p_cmd_threads);
+ }
+ }
+
+ mutex_unlock(&io_context_mutex);
+
+ p_cmd_threads->io_context_ready = true;
+
+ spin_lock_irq(&p_cmd_threads->cmd_list_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_cmd_threads(p_cmd_threads)) {
+ add_wait_queue_exclusive_head(
+ &p_cmd_threads->cmd_list_waitQ,
+ &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_cmd_threads(p_cmd_threads))
+ break;
+ spin_unlock_irq(&p_cmd_threads->cmd_list_lock);
+ schedule();
+ spin_lock_irq(&p_cmd_threads->cmd_list_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&p_cmd_threads->cmd_list_waitQ, &wait);
+ }
+
+ if (tm_dbg_is_release()) {
+ spin_unlock_irq(&p_cmd_threads->cmd_list_lock);
+ tm_dbg_check_released_cmds();
+ spin_lock_irq(&p_cmd_threads->cmd_list_lock);
+ }
+
+ scst_do_job_active(&p_cmd_threads->active_cmd_list,
+ &p_cmd_threads->cmd_list_lock, false);
+ }
+ spin_unlock_irq(&p_cmd_threads->cmd_list_lock);
+
+ EXTRACHECKS_BUG_ON((p_cmd_threads->nr_threads == 1) &&
+ !list_empty(&p_cmd_threads->active_cmd_list));
+
+ if (p_cmd_threads != &scst_main_cmd_threads) {
+ if (p_cmd_threads->nr_threads == 1)
+ p_cmd_threads->io_context = NULL;
+ }
+
+ PRINT_INFO("Processing thread %s (PID %d) finished", current->comm,
+ current->pid);
+ return 0;
+}
+
+void scst_cmd_tasklet(long p)
+{
+ struct scst_tasklet *t = (struct scst_tasklet *)p;
+
+ spin_lock_irq(&t->tasklet_lock);
+ scst_do_job_active(&t->tasklet_cmd_list, &t->tasklet_lock, true);
+ spin_unlock_irq(&t->tasklet_lock);
+ return;
+}
+
+/*
+ * Returns 0 on success, < 0 if there is no device handler or
+ * > 0 if SCST_FLAG_SUSPENDED set and SCST_FLAG_SUSPENDING - not.
+ * No locks, protection is done by the suspended activity.
+ */
+static int scst_mgmt_translate_lun(struct scst_mgmt_cmd *mcmd)
+{
+ struct scst_tgt_dev *tgt_dev = NULL;
+ struct list_head *sess_tgt_dev_list_head;
+ int res = -1;
+
+ TRACE_DBG("Finding tgt_dev for mgmt cmd %p (lun %lld)", mcmd,
+ (long long unsigned int)mcmd->lun);
+
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ __scst_get(1);
+
+ if (unlikely(test_bit(SCST_FLAG_SUSPENDED, &scst_flags) &&
+ !test_bit(SCST_FLAG_SUSPENDING, &scst_flags))) {
+ TRACE_MGMT_DBG("%s", "FLAG SUSPENDED set, skipping");
+ __scst_put();
+ res = 1;
+ goto out;
+ }
+
+ sess_tgt_dev_list_head =
+ &mcmd->sess->sess_tgt_dev_list_hash[HASH_VAL(mcmd->lun)];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == mcmd->lun) {
+ TRACE_DBG("tgt_dev %p found", tgt_dev);
+ mcmd->mcmd_tgt_dev = tgt_dev;
+ res = 0;
+ break;
+ }
+ }
+ if (mcmd->mcmd_tgt_dev == NULL)
+ __scst_put();
+
+out:
+ return res;
+}
+
+/* No locks */
+void scst_done_cmd_mgmt(struct scst_cmd *cmd)
+{
+ struct scst_mgmt_cmd_stub *mstb, *t;
+ bool wake = 0;
+ unsigned long flags;
+
+ TRACE_MGMT_DBG("cmd %p done (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+
+ list_for_each_entry_safe(mstb, t, &cmd->mgmt_cmd_list,
+ cmd_mgmt_cmd_list_entry) {
+ struct scst_mgmt_cmd *mcmd;
+
+ if (!mstb->done_counted)
+ continue;
+
+ mcmd = mstb->mcmd;
+ TRACE_MGMT_DBG("mcmd %p, mcmd->cmd_done_wait_count %d",
+ mcmd, mcmd->cmd_done_wait_count);
+
+ mcmd->cmd_done_wait_count--;
+
+ BUG_ON(mcmd->cmd_done_wait_count < 0);
+
+ if (mcmd->cmd_done_wait_count > 0) {
+ TRACE_MGMT_DBG("cmd_done_wait_count(%d) not 0, "
+ "skipping", mcmd->cmd_done_wait_count);
+ goto check_free;
+ }
+
+ if (mcmd->state == SCST_MCMD_STATE_WAITING_AFFECTED_CMDS_DONE) {
+ mcmd->state = SCST_MCMD_STATE_AFFECTED_CMDS_DONE;
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to active mgmt cmd "
+ "list", mcmd);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ wake = 1;
+ }
+
+check_free:
+ if (!mstb->finish_counted) {
+ TRACE_DBG("Releasing mstb %p", mstb);
+ list_del(&mstb->cmd_mgmt_cmd_list_entry);
+ mempool_free(mstb, scst_mgmt_stub_mempool);
+ }
+ }
+
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ if (wake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+ return;
+}
+
+/* Called under scst_mcmd_lock and IRQs disabled */
+static void __scst_dec_finish_wait_count(struct scst_mgmt_cmd *mcmd, bool *wake)
+{
+
+ mcmd->cmd_finish_wait_count--;
+
+ BUG_ON(mcmd->cmd_finish_wait_count < 0);
+
+ if (mcmd->cmd_finish_wait_count > 0) {
+ TRACE_MGMT_DBG("cmd_finish_wait_count(%d) not 0, "
+ "skipping", mcmd->cmd_finish_wait_count);
+ goto out;
+ }
+
+ if (mcmd->cmd_done_wait_count > 0) {
+ TRACE_MGMT_DBG("cmd_done_wait_count(%d) not 0, "
+ "skipping", mcmd->cmd_done_wait_count);
+ goto out;
+ }
+
+ if (mcmd->state == SCST_MCMD_STATE_WAITING_AFFECTED_CMDS_FINISHED) {
+ mcmd->state = SCST_MCMD_STATE_DONE;
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to active mgmt cmd "
+ "list", mcmd);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ *wake = true;
+ }
+
+out:
+ return;
+}
+
+/**
+ * scst_prepare_async_mcmd() - prepare async management command
+ *
+ * Notifies SCST that management command is going to be async, i.e.
+ * will be completed in another context.
+ *
+ * No SCST locks supposed to be held on entrance.
+ */
+void scst_prepare_async_mcmd(struct scst_mgmt_cmd *mcmd)
+{
+ unsigned long flags;
+
+ TRACE_MGMT_DBG("Preparing mcmd %p for async execution "
+ "(cmd_finish_wait_count %d)", mcmd,
+ mcmd->cmd_finish_wait_count);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+ mcmd->cmd_finish_wait_count++;
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_prepare_async_mcmd);
+
+/**
+ * scst_async_mcmd_completed() - async management command completed
+ *
+ * Notifies SCST that async management command, prepared by
+ * scst_prepare_async_mcmd(), completed.
+ *
+ * No SCST locks supposed to be held on entrance.
+ */
+void scst_async_mcmd_completed(struct scst_mgmt_cmd *mcmd, int status)
+{
+ unsigned long flags;
+ bool wake = false;
+
+ TRACE_MGMT_DBG("Async mcmd %p completed (status %d)", mcmd, status);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+
+ if (status != SCST_MGMT_STATUS_SUCCESS)
+ mcmd->status = status;
+
+ __scst_dec_finish_wait_count(mcmd, &wake);
+
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ if (wake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_async_mcmd_completed);
+
+/* No locks */
+static void scst_finish_cmd_mgmt(struct scst_cmd *cmd)
+{
+ struct scst_mgmt_cmd_stub *mstb, *t;
+ bool wake = false;
+ unsigned long flags;
+
+ TRACE_MGMT_DBG("cmd %p finished (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+
+ list_for_each_entry_safe(mstb, t, &cmd->mgmt_cmd_list,
+ cmd_mgmt_cmd_list_entry) {
+ struct scst_mgmt_cmd *mcmd = mstb->mcmd;
+
+ TRACE_MGMT_DBG("mcmd %p, mcmd->cmd_finish_wait_count %d", mcmd,
+ mcmd->cmd_finish_wait_count);
+
+ BUG_ON(!mstb->finish_counted);
+
+ if (cmd->completed)
+ mcmd->completed_cmd_count++;
+
+ __scst_dec_finish_wait_count(mcmd, &wake);
+
+ TRACE_DBG("Releasing mstb %p", mstb);
+ list_del(&mstb->cmd_mgmt_cmd_list_entry);
+ mempool_free(mstb, scst_mgmt_stub_mempool);
+ }
+
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ if (wake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+ return;
+}
+
+static int scst_call_dev_task_mgmt_fn(struct scst_mgmt_cmd *mcmd,
+ struct scst_tgt_dev *tgt_dev, int set_status)
+{
+ int res = SCST_DEV_TM_NOT_COMPLETED;
+ struct scst_dev_type *h = tgt_dev->dev->handler;
+
+ if (h->task_mgmt_fn) {
+ TRACE_MGMT_DBG("Calling dev handler %s task_mgmt_fn(fn=%d)",
+ h->name, mcmd->fn);
+ EXTRACHECKS_BUG_ON(in_irq() || irqs_disabled());
+ res = h->task_mgmt_fn(mcmd, tgt_dev);
+ TRACE_MGMT_DBG("Dev handler %s task_mgmt_fn() returned %d",
+ h->name, res);
+ if (set_status && (res != SCST_DEV_TM_NOT_COMPLETED))
+ mcmd->status = res;
+ }
+ return res;
+}
+
+static inline int scst_is_strict_mgmt_fn(int mgmt_fn)
+{
+ switch (mgmt_fn) {
+#ifdef CONFIG_SCST_ABORT_CONSIDER_FINISHED_TASKS_AS_NOT_EXISTING
+ case SCST_ABORT_TASK:
+#endif
+#if 0
+ case SCST_ABORT_TASK_SET:
+ case SCST_CLEAR_TASK_SET:
+#endif
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+/* Might be called under sess_list_lock and IRQ off + BHs also off */
+void scst_abort_cmd(struct scst_cmd *cmd, struct scst_mgmt_cmd *mcmd,
+ bool other_ini, bool call_dev_task_mgmt_fn)
+{
+ unsigned long flags;
+ static DEFINE_SPINLOCK(other_ini_lock);
+
+ TRACE(TRACE_MGMT, "Aborting cmd %p (tag %llu, op %x)",
+ cmd, (long long unsigned int)cmd->tag, cmd->cdb[0]);
+
+ /* To protect from concurrent aborts */
+ spin_lock_irqsave(&other_ini_lock, flags);
+
+ if (other_ini) {
+ struct scst_device *dev = NULL;
+
+ /* Might be necessary if command aborted several times */
+ if (!test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))
+ set_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+
+ /* Necessary for scst_xmit_process_aborted_cmd */
+ if (cmd->dev != NULL)
+ dev = cmd->dev;
+ else if ((mcmd != NULL) && (mcmd->mcmd_tgt_dev != NULL))
+ dev = mcmd->mcmd_tgt_dev->dev;
+
+ if (dev != NULL) {
+ if (dev->tas)
+ set_bit(SCST_CMD_DEVICE_TAS, &cmd->cmd_flags);
+ } else
+ PRINT_WARNING("Abort cmd %p from other initiator, but "
+ "neither cmd, nor mcmd %p have tgt_dev set, so "
+ "TAS information can be lost", cmd, mcmd);
+ } else {
+ /* Might be necessary if command aborted several times */
+ clear_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+ }
+
+ set_bit(SCST_CMD_ABORTED, &cmd->cmd_flags);
+
+ spin_unlock_irqrestore(&other_ini_lock, flags);
+
+ /*
+ * To sync with cmd->finished/done set in
+ * scst_finish_cmd()/scst_pre_xmit_response()
+ */
+ smp_mb__after_set_bit();
+
+ if (cmd->tgt_dev == NULL) {
+ spin_lock_irqsave(&scst_init_lock, flags);
+ scst_init_poll_cnt++;
+ spin_unlock_irqrestore(&scst_init_lock, flags);
+ wake_up(&scst_init_cmd_list_waitQ);
+ }
+
+ if (call_dev_task_mgmt_fn && (cmd->tgt_dev != NULL)) {
+ EXTRACHECKS_BUG_ON(irqs_disabled());
+ scst_call_dev_task_mgmt_fn(mcmd, cmd->tgt_dev, 1);
+ }
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+ if ((mcmd != NULL) && !cmd->finished) {
+ struct scst_mgmt_cmd_stub *mstb;
+
+ mstb = mempool_alloc(scst_mgmt_stub_mempool, GFP_ATOMIC);
+ if (mstb == NULL) {
+ PRINT_CRIT_ERROR("Allocation of management command "
+ "stub failed (mcmd %p, cmd %p)", mcmd, cmd);
+ goto unlock;
+ }
+ memset(mstb, 0, sizeof(*mstb));
+
+ TRACE_DBG("mstb %p, mcmd %p", mstb, mcmd);
+
+ mstb->mcmd = mcmd;
+
+ /*
+ * Delay the response until the command's finish in order to
+ * guarantee that "no further responses from the task are sent
+ * to the SCSI initiator port" after response from the TM
+ * function is sent (SAM). Plus, we must wait here to be sure
+ * that we won't receive double commands with the same tag.
+ * Moreover, if we don't wait here, we might have a possibility
+ * for data corruption, when aborted and reported as completed
+ * command actually gets executed *after* new commands sent
+ * after this TM command completed.
+ */
+
+ if (cmd->sent_for_exec && !cmd->done) {
+ TRACE_MGMT_DBG("cmd %p (tag %llu) is being executed",
+ cmd, (long long unsigned int)cmd->tag);
+ mstb->done_counted = 1;
+ mcmd->cmd_done_wait_count++;
+ }
+
+ /*
+ * We don't have to wait the command's status delivery finish
+ * to other initiators + it can affect MPIO failover.
+ */
+ if (!other_ini) {
+ mstb->finish_counted = 1;
+ mcmd->cmd_finish_wait_count++;
+ }
+
+ if (mstb->done_counted || mstb->finish_counted) {
+ TRACE_MGMT_DBG("cmd %p (tag %llu, sn %u) being "
+ "executed/xmitted (state %d, op %x, proc time "
+ "%ld sec., timeout %d sec.), deferring ABORT "
+ "(cmd_done_wait_count %d, cmd_finish_wait_count "
+ "%d)", cmd, (long long unsigned int)cmd->tag,
+ cmd->sn, cmd->state, cmd->cdb[0],
+ (long)(jiffies - cmd->start_time) / HZ,
+ cmd->timeout / HZ, mcmd->cmd_done_wait_count,
+ mcmd->cmd_finish_wait_count);
+ /*
+ * cmd can't die here or sess_list_lock already taken
+ * and cmd is in the sess list
+ */
+ list_add_tail(&mstb->cmd_mgmt_cmd_list_entry,
+ &cmd->mgmt_cmd_list);
+ } else {
+ /* We don't need to wait for this cmd */
+ mempool_free(mstb, scst_mgmt_stub_mempool);
+ }
+ }
+
+unlock:
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ tm_dbg_release_cmd(cmd);
+ return;
+}
+
+/* No locks. Returns 0, if mcmd should be processed further. */
+static int scst_set_mcmd_next_state(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+
+ spin_lock_irq(&scst_mcmd_lock);
+
+ switch (mcmd->state) {
+ case SCST_MCMD_STATE_INIT:
+ case SCST_MCMD_STATE_EXEC:
+ if (mcmd->cmd_done_wait_count == 0) {
+ mcmd->state = SCST_MCMD_STATE_AFFECTED_CMDS_DONE;
+ res = 0;
+ } else {
+ TRACE_MGMT_DBG("cmd_done_wait_count(%d) not 0, "
+ "preparing to wait", mcmd->cmd_done_wait_count);
+ mcmd->state = SCST_MCMD_STATE_WAITING_AFFECTED_CMDS_DONE;
+ res = -1;
+ }
+ break;
+
+ case SCST_MCMD_STATE_AFFECTED_CMDS_DONE:
+ if (mcmd->cmd_finish_wait_count == 0) {
+ mcmd->state = SCST_MCMD_STATE_DONE;
+ res = 0;
+ } else {
+ TRACE_MGMT_DBG("cmd_finish_wait_count(%d) not 0, "
+ "preparing to wait",
+ mcmd->cmd_finish_wait_count);
+ mcmd->state = SCST_MCMD_STATE_WAITING_AFFECTED_CMDS_FINISHED;
+ res = -1;
+ }
+ break;
+
+ case SCST_MCMD_STATE_DONE:
+ mcmd->state = SCST_MCMD_STATE_FINISHED;
+ res = 0;
+ break;
+
+ default:
+ PRINT_CRIT_ERROR("Wrong mcmd %p state %d (fn %d, "
+ "cmd_finish_wait_count %d, cmd_done_wait_count %d)",
+ mcmd, mcmd->state, mcmd->fn,
+ mcmd->cmd_finish_wait_count, mcmd->cmd_done_wait_count);
+ spin_unlock_irq(&scst_mcmd_lock);
+ res = -1;
+ BUG();
+ goto out;
+ }
+
+ spin_unlock_irq(&scst_mcmd_lock);
+
+out:
+ return res;
+}
+
+/* IRQs supposed to be disabled */
+static bool __scst_check_unblock_aborted_cmd(struct scst_cmd *cmd,
+ struct list_head *list_entry)
+{
+ bool res;
+ if (test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ list_del(list_entry);
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ res = 1;
+ } else
+ res = 0;
+ return res;
+}
+
+static void scst_unblock_aborted_cmds(int scst_mutex_held)
+{
+ struct scst_device *dev;
+
+ if (!scst_mutex_held)
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ struct scst_cmd *cmd, *tcmd;
+ struct scst_tgt_dev *tgt_dev;
+ spin_lock_bh(&dev->dev_lock);
+ local_irq_disable();
+ list_for_each_entry_safe(cmd, tcmd, &dev->blocked_cmd_list,
+ blocked_cmd_list_entry) {
+ if (__scst_check_unblock_aborted_cmd(cmd,
+ &cmd->blocked_cmd_list_entry)) {
+ TRACE_MGMT_DBG("Unblock aborted blocked cmd %p",
+ cmd);
+ }
+ }
+ local_irq_enable();
+ spin_unlock_bh(&dev->dev_lock);
+
+ local_irq_disable();
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ spin_lock(&tgt_dev->sn_lock);
+ list_for_each_entry_safe(cmd, tcmd,
+ &tgt_dev->deferred_cmd_list,
+ sn_cmd_list_entry) {
+ if (__scst_check_unblock_aborted_cmd(cmd,
+ &cmd->sn_cmd_list_entry)) {
+ TRACE_MGMT_DBG("Unblocked aborted SN "
+ "cmd %p (sn %u)",
+ cmd, cmd->sn);
+ tgt_dev->def_cmd_count--;
+ }
+ }
+ spin_unlock(&tgt_dev->sn_lock);
+ }
+ local_irq_enable();
+ }
+
+ if (!scst_mutex_held)
+ mutex_unlock(&scst_mutex);
+ return;
+}
+
+static void __scst_abort_task_set(struct scst_mgmt_cmd *mcmd,
+ struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_cmd *cmd;
+ struct scst_session *sess = tgt_dev->sess;
+ bool other_ini;
+
+ if ((mcmd->fn == SCST_PR_ABORT_ALL) &&
+ (mcmd->origin_pr_cmd->sess != sess))
+ other_ini = true;
+ else
+ other_ini = false;
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if ((mcmd->fn == SCST_PR_ABORT_ALL) &&
+ (mcmd->origin_pr_cmd == cmd))
+ continue;
+ if ((cmd->tgt_dev == tgt_dev) ||
+ ((cmd->tgt_dev == NULL) &&
+ (cmd->lun == tgt_dev->lun))) {
+ if (mcmd->cmd_sn_set) {
+ BUG_ON(!cmd->tgt_sn_set);
+ if (scst_sn_before(mcmd->cmd_sn, cmd->tgt_sn) ||
+ (mcmd->cmd_sn == cmd->tgt_sn))
+ continue;
+ }
+ scst_abort_cmd(cmd, mcmd, other_ini, 0);
+ }
+ }
+ spin_unlock_irq(&sess->sess_list_lock);
+ return;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_abort_task_set(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+ struct scst_tgt_dev *tgt_dev = mcmd->mcmd_tgt_dev;
+
+ TRACE(TRACE_MGMT, "Aborting task set (lun=%lld, mcmd=%p)",
+ (long long unsigned int)tgt_dev->lun, mcmd);
+
+ __scst_abort_task_set(mcmd, tgt_dev);
+
+ if (atomic_dec_and_test(&mcmd->origin_pr_cmd->pr_abort_counter->pr_aborting_cnt))
+ complete_all(&mcmd->origin_pr_cmd->pr_abort_counter->pr_aborting_cmpl);
+
+ tm_dbg_task_mgmt(mcmd->mcmd_tgt_dev->dev, "ABORT TASK SET/PR ABORT", 0);
+
+ scst_unblock_aborted_cmds(0);
+
+ scst_call_dev_task_mgmt_fn(mcmd, tgt_dev, 0);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+static int scst_is_cmd_belongs_to_dev(struct scst_cmd *cmd,
+ struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev = NULL;
+ struct list_head *sess_tgt_dev_list_head;
+ int res = 0;
+
+ TRACE_DBG("Finding match for dev %p and cmd %p (lun %lld)", dev, cmd,
+ (long long unsigned int)cmd->lun);
+
+ sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[HASH_VAL(cmd->lun)];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == cmd->lun) {
+ TRACE_DBG("dev %p found", tgt_dev->dev);
+ res = (tgt_dev->dev == dev);
+ goto out;
+ }
+ }
+
+out:
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_clear_task_set(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+ struct scst_device *dev = mcmd->mcmd_tgt_dev->dev;
+ struct scst_tgt_dev *tgt_dev;
+ LIST_HEAD(UA_tgt_devs);
+
+ TRACE(TRACE_MGMT, "Clearing task set (lun=%lld, mcmd=%p)",
+ (long long unsigned int)mcmd->lun, mcmd);
+
+#if 0 /* we are SAM-3 */
+ /*
+ * When a logical unit is aborting one or more tasks from a SCSI
+ * initiator port with the TASK ABORTED status it should complete all
+ * of those tasks before entering additional tasks from that SCSI
+ * initiator port into the task set - SAM2
+ */
+ mcmd->needs_unblocking = 1;
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ spin_unlock_bh(&dev->dev_lock);
+#endif
+
+ __scst_abort_task_set(mcmd, mcmd->mcmd_tgt_dev);
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ struct scst_session *sess = tgt_dev->sess;
+ struct scst_cmd *cmd;
+ int aborted = 0;
+
+ if (tgt_dev == mcmd->mcmd_tgt_dev)
+ continue;
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if ((cmd->dev == dev) ||
+ ((cmd->dev == NULL) &&
+ scst_is_cmd_belongs_to_dev(cmd, dev))) {
+ scst_abort_cmd(cmd, mcmd, 1, 0);
+ aborted = 1;
+ }
+ }
+ spin_unlock_irq(&sess->sess_list_lock);
+
+ if (aborted)
+ list_add_tail(&tgt_dev->extra_tgt_dev_list_entry,
+ &UA_tgt_devs);
+ }
+
+ tm_dbg_task_mgmt(mcmd->mcmd_tgt_dev->dev, "CLEAR TASK SET", 0);
+
+ scst_unblock_aborted_cmds(1);
+
+ mutex_unlock(&scst_mutex);
+
+ if (!dev->tas) {
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl;
+
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_cleared_by_another_ini_UA));
+
+ list_for_each_entry(tgt_dev, &UA_tgt_devs,
+ extra_tgt_dev_list_entry) {
+ scst_check_set_UA(tgt_dev, sense_buffer, sl, 0);
+ }
+ }
+
+ scst_call_dev_task_mgmt_fn(mcmd, mcmd->mcmd_tgt_dev, 0);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued,
+ * >0, if it should be requeued, <0 otherwise */
+static int scst_mgmt_cmd_init(struct scst_mgmt_cmd *mcmd)
+{
+ int res = 0, rc;
+
+ switch (mcmd->fn) {
+ case SCST_ABORT_TASK:
+ {
+ struct scst_session *sess = mcmd->sess;
+ struct scst_cmd *cmd;
+
+ spin_lock_irq(&sess->sess_list_lock);
+ cmd = __scst_find_cmd_by_tag(sess, mcmd->tag, true);
+ if (cmd == NULL) {
+ TRACE_MGMT_DBG("ABORT TASK: command "
+ "for tag %llu not found",
+ (long long unsigned int)mcmd->tag);
+ mcmd->status = SCST_MGMT_STATUS_TASK_NOT_EXIST;
+ spin_unlock_irq(&sess->sess_list_lock);
+ res = scst_set_mcmd_next_state(mcmd);
+ goto out;
+ }
+ __scst_cmd_get(cmd);
+ spin_unlock_irq(&sess->sess_list_lock);
+ TRACE_DBG("Cmd to abort %p for tag %llu found",
+ cmd, (long long unsigned int)mcmd->tag);
+ mcmd->cmd_to_abort = cmd;
+ mcmd->state = SCST_MCMD_STATE_EXEC;
+ break;
+ }
+
+ case SCST_TARGET_RESET:
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_ABORT_ALL_TASKS_SESS:
+ case SCST_NEXUS_LOSS:
+ case SCST_ABORT_ALL_TASKS:
+ case SCST_UNREG_SESS_TM:
+ mcmd->state = SCST_MCMD_STATE_EXEC;
+ break;
+
+ case SCST_ABORT_TASK_SET:
+ case SCST_CLEAR_ACA:
+ case SCST_CLEAR_TASK_SET:
+ case SCST_LUN_RESET:
+ case SCST_PR_ABORT_ALL:
+ rc = scst_mgmt_translate_lun(mcmd);
+ if (rc == 0)
+ mcmd->state = SCST_MCMD_STATE_EXEC;
+ else if (rc < 0) {
+ PRINT_ERROR("Corresponding device for LUN %lld not "
+ "found", (long long unsigned int)mcmd->lun);
+ mcmd->status = SCST_MGMT_STATUS_LUN_NOT_EXIST;
+ res = scst_set_mcmd_next_state(mcmd);
+ } else
+ res = rc;
+ break;
+
+ default:
+ BUG();
+ }
+
+out:
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_target_reset(struct scst_mgmt_cmd *mcmd)
+{
+ int res, rc;
+ struct scst_device *dev;
+ struct scst_acg *acg = mcmd->sess->acg;
+ struct scst_acg_dev *acg_dev;
+ int cont, c;
+ LIST_HEAD(host_devs);
+
+ TRACE(TRACE_MGMT, "Target reset (mcmd %p, cmd count %d)",
+ mcmd, atomic_read(&mcmd->sess->sess_cmd_count));
+
+ mcmd->needs_unblocking = 1;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ struct scst_device *d;
+ struct scst_tgt_dev *tgt_dev;
+ int found = 0;
+
+ dev = acg_dev->dev;
+
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ scst_process_reset(dev, mcmd->sess, NULL, mcmd, true);
+ spin_unlock_bh(&dev->dev_lock);
+
+ cont = 0;
+ c = 0;
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ cont = 1;
+ if (mcmd->sess == tgt_dev->sess) {
+ rc = scst_call_dev_task_mgmt_fn(mcmd,
+ tgt_dev, 0);
+ if (rc == SCST_DEV_TM_NOT_COMPLETED)
+ c = 1;
+ else if ((rc < 0) &&
+ (mcmd->status == SCST_MGMT_STATUS_SUCCESS))
+ mcmd->status = rc;
+ break;
+ }
+ }
+ if (cont && !c)
+ continue;
+
+ if (dev->scsi_dev == NULL)
+ continue;
+
+ list_for_each_entry(d, &host_devs, tm_dev_list_entry) {
+ if (dev->scsi_dev->host->host_no ==
+ d->scsi_dev->host->host_no) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found)
+ list_add_tail(&dev->tm_dev_list_entry, &host_devs);
+
+ tm_dbg_task_mgmt(dev, "TARGET RESET", 0);
+ }
+
+ scst_unblock_aborted_cmds(1);
+
+ /*
+ * We suppose here that for all commands that already on devices
+ * on/after scsi_reset_provider() completion callbacks will be called.
+ */
+
+ list_for_each_entry(dev, &host_devs, tm_dev_list_entry) {
+ /* dev->scsi_dev must be non-NULL here */
+ TRACE(TRACE_MGMT, "Resetting host %d bus ",
+ dev->scsi_dev->host->host_no);
+ rc = scsi_reset_provider(dev->scsi_dev, SCSI_TRY_RESET_TARGET);
+ TRACE(TRACE_MGMT, "Result of host %d target reset: %s",
+ dev->scsi_dev->host->host_no,
+ (rc == SUCCESS) ? "SUCCESS" : "FAILED");
+#if 0
+ if ((rc != SUCCESS) &&
+ (mcmd->status == SCST_MGMT_STATUS_SUCCESS)) {
+ /*
+ * SCSI_TRY_RESET_BUS is also done by
+ * scsi_reset_provider()
+ */
+ mcmd->status = SCST_MGMT_STATUS_FAILED;
+ }
+#else
+ /*
+ * scsi_reset_provider() returns very weird status, so let's
+ * always succeed
+ */
+#endif
+ }
+
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ dev = acg_dev->dev;
+ if (dev->scsi_dev != NULL)
+ dev->scsi_dev->was_reset = 0;
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_lun_reset(struct scst_mgmt_cmd *mcmd)
+{
+ int res, rc;
+ struct scst_tgt_dev *tgt_dev = mcmd->mcmd_tgt_dev;
+ struct scst_device *dev = tgt_dev->dev;
+
+ TRACE(TRACE_MGMT, "Resetting LUN %lld (mcmd %p)",
+ (long long unsigned int)tgt_dev->lun, mcmd);
+
+ mcmd->needs_unblocking = 1;
+
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ scst_process_reset(dev, mcmd->sess, NULL, mcmd, true);
+ spin_unlock_bh(&dev->dev_lock);
+
+ rc = scst_call_dev_task_mgmt_fn(mcmd, tgt_dev, 1);
+ if (rc != SCST_DEV_TM_NOT_COMPLETED)
+ goto out_tm_dbg;
+
+ if (dev->scsi_dev != NULL) {
+ TRACE(TRACE_MGMT, "Resetting host %d bus ",
+ dev->scsi_dev->host->host_no);
+ rc = scsi_reset_provider(dev->scsi_dev, SCSI_TRY_RESET_DEVICE);
+#if 0
+ if (rc != SUCCESS && mcmd->status == SCST_MGMT_STATUS_SUCCESS)
+ mcmd->status = SCST_MGMT_STATUS_FAILED;
+#else
+ /*
+ * scsi_reset_provider() returns very weird status, so let's
+ * always succeed
+ */
+#endif
+ dev->scsi_dev->was_reset = 0;
+ }
+
+ scst_unblock_aborted_cmds(0);
+
+out_tm_dbg:
+ tm_dbg_task_mgmt(mcmd->mcmd_tgt_dev->dev, "LUN RESET", 0);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* scst_mutex supposed to be held */
+static void scst_do_nexus_loss_sess(struct scst_mgmt_cmd *mcmd)
+{
+ int i;
+ struct scst_session *sess = mcmd->sess;
+ struct scst_tgt_dev *tgt_dev;
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ scst_nexus_loss(tgt_dev,
+ (mcmd->fn != SCST_UNREG_SESS_TM));
+ }
+ }
+ return;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_abort_all_nexus_loss_sess(struct scst_mgmt_cmd *mcmd,
+ int nexus_loss)
+{
+ int res;
+ int i;
+ struct scst_session *sess = mcmd->sess;
+ struct scst_tgt_dev *tgt_dev;
+
+ if (nexus_loss) {
+ TRACE_MGMT_DBG("Nexus loss for sess %p (mcmd %p)",
+ sess, mcmd);
+ } else {
+ TRACE_MGMT_DBG("Aborting all from sess %p (mcmd %p)",
+ sess, mcmd);
+ }
+
+ mutex_lock(&scst_mutex);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ int rc;
+
+ __scst_abort_task_set(mcmd, tgt_dev);
+
+ rc = scst_call_dev_task_mgmt_fn(mcmd, tgt_dev, 0);
+ if (rc < 0 && mcmd->status == SCST_MGMT_STATUS_SUCCESS)
+ mcmd->status = rc;
+
+ tm_dbg_task_mgmt(tgt_dev->dev, "NEXUS LOSS SESS or "
+ "ABORT ALL SESS or UNREG SESS",
+ (mcmd->fn == SCST_UNREG_SESS_TM));
+ }
+ }
+
+ scst_unblock_aborted_cmds(1);
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* scst_mutex supposed to be held */
+static void scst_do_nexus_loss_tgt(struct scst_mgmt_cmd *mcmd)
+{
+ int i;
+ struct scst_tgt *tgt = mcmd->sess->tgt;
+ struct scst_session *sess;
+
+ list_for_each_entry(sess, &tgt->sess_list, sess_list_entry) {
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ scst_nexus_loss(tgt_dev, true);
+ }
+ }
+ }
+ return;
+}
+
+static int scst_abort_all_nexus_loss_tgt(struct scst_mgmt_cmd *mcmd,
+ int nexus_loss)
+{
+ int res;
+ int i;
+ struct scst_tgt *tgt = mcmd->sess->tgt;
+ struct scst_session *sess;
+
+ if (nexus_loss) {
+ TRACE_MGMT_DBG("I_T Nexus loss (tgt %p, mcmd %p)",
+ tgt, mcmd);
+ } else {
+ TRACE_MGMT_DBG("Aborting all from tgt %p (mcmd %p)",
+ tgt, mcmd);
+ }
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(sess, &tgt->sess_list, sess_list_entry) {
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ int rc;
+
+ __scst_abort_task_set(mcmd, tgt_dev);
+
+ if (nexus_loss)
+ scst_nexus_loss(tgt_dev, true);
+
+ if (mcmd->sess == tgt_dev->sess) {
+ rc = scst_call_dev_task_mgmt_fn(
+ mcmd, tgt_dev, 0);
+ if ((rc < 0) &&
+ (mcmd->status == SCST_MGMT_STATUS_SUCCESS))
+ mcmd->status = rc;
+ }
+
+ tm_dbg_task_mgmt(tgt_dev->dev, "NEXUS LOSS or "
+ "ABORT ALL", 0);
+ }
+ }
+ }
+
+ scst_unblock_aborted_cmds(1);
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+static int scst_abort_task(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+ struct scst_cmd *cmd = mcmd->cmd_to_abort;
+
+ TRACE_MGMT_DBG("Abortind task (cmd %p, sn %d, set %d, tag %llu, "
+ "queue_type %x)", cmd, cmd->sn, cmd->sn_set,
+ (long long unsigned int)mcmd->tag, cmd->queue_type);
+
+ if (mcmd->lun_set && (mcmd->lun != cmd->lun)) {
+ PRINT_ERROR("ABORT TASK: LUN mismatch: mcmd LUN %llx, "
+ "cmd LUN %llx, cmd tag %llu",
+ (long long unsigned int)mcmd->lun,
+ (long long unsigned int)cmd->lun,
+ (long long unsigned int)mcmd->tag);
+ mcmd->status = SCST_MGMT_STATUS_REJECTED;
+ } else if (mcmd->cmd_sn_set &&
+ (scst_sn_before(mcmd->cmd_sn, cmd->tgt_sn) ||
+ (mcmd->cmd_sn == cmd->tgt_sn))) {
+ PRINT_ERROR("ABORT TASK: SN mismatch: mcmd SN %x, "
+ "cmd SN %x, cmd tag %llu", mcmd->cmd_sn,
+ cmd->tgt_sn, (long long unsigned int)mcmd->tag);
+ mcmd->status = SCST_MGMT_STATUS_REJECTED;
+ } else {
+ scst_abort_cmd(cmd, mcmd, 0, 1);
+ scst_unblock_aborted_cmds(0);
+ }
+
+ res = scst_set_mcmd_next_state(mcmd);
+
+ mcmd->cmd_to_abort = NULL; /* just in case */
+
+ __scst_cmd_put(cmd);
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_mgmt_cmd_exec(struct scst_mgmt_cmd *mcmd)
+{
+ int res = 0;
+
+ mcmd->status = SCST_MGMT_STATUS_SUCCESS;
+
+ switch (mcmd->fn) {
+ case SCST_ABORT_TASK:
+ res = scst_abort_task(mcmd);
+ break;
+
+ case SCST_ABORT_TASK_SET:
+ case SCST_PR_ABORT_ALL:
+ res = scst_abort_task_set(mcmd);
+ break;
+
+ case SCST_CLEAR_TASK_SET:
+ if (mcmd->mcmd_tgt_dev->dev->tst ==
+ SCST_CONTR_MODE_SEP_TASK_SETS)
+ res = scst_abort_task_set(mcmd);
+ else
+ res = scst_clear_task_set(mcmd);
+ break;
+
+ case SCST_LUN_RESET:
+ res = scst_lun_reset(mcmd);
+ break;
+
+ case SCST_TARGET_RESET:
+ res = scst_target_reset(mcmd);
+ break;
+
+ case SCST_ABORT_ALL_TASKS_SESS:
+ res = scst_abort_all_nexus_loss_sess(mcmd, 0);
+ break;
+
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_UNREG_SESS_TM:
+ res = scst_abort_all_nexus_loss_sess(mcmd, 1);
+ break;
+
+ case SCST_ABORT_ALL_TASKS:
+ res = scst_abort_all_nexus_loss_tgt(mcmd, 0);
+ break;
+
+ case SCST_NEXUS_LOSS:
+ res = scst_abort_all_nexus_loss_tgt(mcmd, 1);
+ break;
+
+ case SCST_CLEAR_ACA:
+ if (scst_call_dev_task_mgmt_fn(mcmd, mcmd->mcmd_tgt_dev, 1) ==
+ SCST_DEV_TM_NOT_COMPLETED) {
+ mcmd->status = SCST_MGMT_STATUS_FN_NOT_SUPPORTED;
+ /* Nothing to do (yet) */
+ }
+ goto out_done;
+
+ default:
+ PRINT_ERROR("Unknown task management function %d", mcmd->fn);
+ mcmd->status = SCST_MGMT_STATUS_REJECTED;
+ goto out_done;
+ }
+
+out:
+ return res;
+
+out_done:
+ res = scst_set_mcmd_next_state(mcmd);
+ goto out;
+}
+
+static void scst_call_task_mgmt_affected_cmds_done(struct scst_mgmt_cmd *mcmd)
+{
+ struct scst_session *sess = mcmd->sess;
+
+ if ((sess->tgt->tgtt->task_mgmt_affected_cmds_done != NULL) &&
+ (mcmd->fn != SCST_UNREG_SESS_TM) &&
+ (mcmd->fn != SCST_PR_ABORT_ALL)) {
+ TRACE_DBG("Calling target %s task_mgmt_affected_cmds_done(%p)",
+ sess->tgt->tgtt->name, sess);
+ sess->tgt->tgtt->task_mgmt_affected_cmds_done(mcmd);
+ TRACE_MGMT_DBG("Target's %s task_mgmt_affected_cmds_done() "
+ "returned", sess->tgt->tgtt->name);
+ }
+ return;
+}
+
+static int scst_mgmt_affected_cmds_done(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+
+ mutex_lock(&scst_mutex);
+
+ switch (mcmd->fn) {
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_UNREG_SESS_TM:
+ scst_do_nexus_loss_sess(mcmd);
+ break;
+
+ case SCST_NEXUS_LOSS:
+ scst_do_nexus_loss_tgt(mcmd);
+ break;
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ scst_call_task_mgmt_affected_cmds_done(mcmd);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+static void scst_mgmt_cmd_send_done(struct scst_mgmt_cmd *mcmd)
+{
+ struct scst_device *dev;
+ struct scst_session *sess = mcmd->sess;
+
+ mcmd->state = SCST_MCMD_STATE_FINISHED;
+ if (scst_is_strict_mgmt_fn(mcmd->fn) && (mcmd->completed_cmd_count > 0))
+ mcmd->status = SCST_MGMT_STATUS_TASK_NOT_EXIST;
+
+ TRACE(TRACE_MINOR_AND_MGMT_DBG, "TM command fn %d finished, "
+ "status %x", mcmd->fn, mcmd->status);
+
+ if (mcmd->fn == SCST_PR_ABORT_ALL) {
+ mcmd->origin_pr_cmd->scst_cmd_done(mcmd->origin_pr_cmd,
+ SCST_CMD_STATE_DEFAULT,
+ SCST_CONTEXT_THREAD);
+ } else if ((sess->tgt->tgtt->task_mgmt_fn_done != NULL) &&
+ (mcmd->fn != SCST_UNREG_SESS_TM)) {
+ TRACE_DBG("Calling target %s task_mgmt_fn_done(%p)",
+ sess->tgt->tgtt->name, sess);
+ sess->tgt->tgtt->task_mgmt_fn_done(mcmd);
+ TRACE_MGMT_DBG("Target's %s task_mgmt_fn_done() "
+ "returned", sess->tgt->tgtt->name);
+ }
+
+ if (mcmd->needs_unblocking) {
+ switch (mcmd->fn) {
+ case SCST_LUN_RESET:
+ case SCST_CLEAR_TASK_SET:
+ scst_unblock_dev(mcmd->mcmd_tgt_dev->dev);
+ break;
+
+ case SCST_TARGET_RESET:
+ {
+ struct scst_acg *acg = mcmd->sess->acg;
+ struct scst_acg_dev *acg_dev;
+
+ mutex_lock(&scst_mutex);
+ list_for_each_entry(acg_dev, &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ dev = acg_dev->dev;
+ scst_unblock_dev(dev);
+ }
+ mutex_unlock(&scst_mutex);
+ break;
+ }
+
+ default:
+ BUG();
+ break;
+ }
+ }
+
+ mcmd->tgt_priv = NULL;
+ return;
+}
+
+/* Returns >0, if cmd should be requeued */
+static int scst_process_mgmt_cmd(struct scst_mgmt_cmd *mcmd)
+{
+ int res = 0;
+
+ /*
+ * We are in the TM thread and mcmd->state guaranteed to not be
+ * changed behind us.
+ */
+
+ TRACE_DBG("mcmd %p, state %d", mcmd, mcmd->state);
+
+ while (1) {
+ switch (mcmd->state) {
+ case SCST_MCMD_STATE_INIT:
+ res = scst_mgmt_cmd_init(mcmd);
+ if (res)
+ goto out;
+ break;
+
+ case SCST_MCMD_STATE_EXEC:
+ if (scst_mgmt_cmd_exec(mcmd))
+ goto out;
+ break;
+
+ case SCST_MCMD_STATE_AFFECTED_CMDS_DONE:
+ if (scst_mgmt_affected_cmds_done(mcmd))
+ goto out;
+ break;
+
+ case SCST_MCMD_STATE_DONE:
+ scst_mgmt_cmd_send_done(mcmd);
+ break;
+
+ case SCST_MCMD_STATE_FINISHED:
+ scst_free_mgmt_cmd(mcmd);
+ /* mcmd is dead */
+ goto out;
+
+ default:
+ PRINT_CRIT_ERROR("Wrong mcmd %p state %d (fn %d, "
+ "cmd_finish_wait_count %d, cmd_done_wait_count "
+ "%d)", mcmd, mcmd->state, mcmd->fn,
+ mcmd->cmd_finish_wait_count,
+ mcmd->cmd_done_wait_count);
+ BUG();
+ res = -1;
+ goto out;
+ }
+ }
+
+out:
+ return res;
+}
+
+static inline int test_mgmt_cmd_list(void)
+{
+ int res = !list_empty(&scst_active_mgmt_cmd_list) ||
+ unlikely(kthread_should_stop());
+ return res;
+}
+
+int scst_tm_thread(void *arg)
+{
+
+ PRINT_INFO("Task management thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ set_user_nice(current, -10);
+
+ spin_lock_irq(&scst_mcmd_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_mgmt_cmd_list()) {
+ add_wait_queue_exclusive(&scst_mgmt_cmd_list_waitQ,
+ &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_mgmt_cmd_list())
+ break;
+ spin_unlock_irq(&scst_mcmd_lock);
+ schedule();
+ spin_lock_irq(&scst_mcmd_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&scst_mgmt_cmd_list_waitQ, &wait);
+ }
+
+ while (!list_empty(&scst_active_mgmt_cmd_list)) {
+ int rc;
+ struct scst_mgmt_cmd *mcmd;
+ mcmd = list_entry(scst_active_mgmt_cmd_list.next,
+ typeof(*mcmd), mgmt_cmd_list_entry);
+ TRACE_MGMT_DBG("Deleting mgmt cmd %p from active cmd "
+ "list", mcmd);
+ list_del(&mcmd->mgmt_cmd_list_entry);
+ spin_unlock_irq(&scst_mcmd_lock);
+ rc = scst_process_mgmt_cmd(mcmd);
+ spin_lock_irq(&scst_mcmd_lock);
+ if (rc > 0) {
+ if (test_bit(SCST_FLAG_SUSPENDED, &scst_flags) &&
+ !test_bit(SCST_FLAG_SUSPENDING,
+ &scst_flags)) {
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to "
+ "head of delayed mgmt cmd list",
+ mcmd);
+ list_add(&mcmd->mgmt_cmd_list_entry,
+ &scst_delayed_mgmt_cmd_list);
+ } else {
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to "
+ "head of active mgmt cmd list",
+ mcmd);
+ list_add(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ }
+ }
+ }
+ }
+ spin_unlock_irq(&scst_mcmd_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so scst_active_mgmt_cmd_list must be empty.
+ */
+ BUG_ON(!list_empty(&scst_active_mgmt_cmd_list));
+
+ PRINT_INFO("Task management thread PID %d finished", current->pid);
+ return 0;
+}
+
+static struct scst_mgmt_cmd *scst_pre_rx_mgmt_cmd(struct scst_session
+ *sess, int fn, int atomic, void *tgt_priv)
+{
+ struct scst_mgmt_cmd *mcmd = NULL;
+
+ if (unlikely(sess->tgt->tgtt->task_mgmt_fn_done == NULL)) {
+ PRINT_ERROR("New mgmt cmd, but task_mgmt_fn_done() is NULL "
+ "(target %s)", sess->tgt->tgtt->name);
+ goto out;
+ }
+
+ mcmd = scst_alloc_mgmt_cmd(atomic ? GFP_ATOMIC : GFP_KERNEL);
+ if (mcmd == NULL) {
+ PRINT_CRIT_ERROR("Lost TM fn %d, initiator %s", fn,
+ sess->initiator_name);
+ goto out;
+ }
+
+ mcmd->sess = sess;
+ mcmd->fn = fn;
+ mcmd->state = SCST_MCMD_STATE_INIT;
+ mcmd->tgt_priv = tgt_priv;
+
+ if (fn == SCST_PR_ABORT_ALL) {
+ atomic_inc(&mcmd->origin_pr_cmd->pr_abort_counter->pr_abort_pending_cnt);
+ atomic_inc(&mcmd->origin_pr_cmd->pr_abort_counter->pr_aborting_cnt);
+ }
+
+out:
+ return mcmd;
+}
+
+static int scst_post_rx_mgmt_cmd(struct scst_session *sess,
+ struct scst_mgmt_cmd *mcmd)
+{
+ unsigned long flags;
+ int res = 0;
+
+ scst_sess_get(sess);
+
+ if (unlikely(sess->shut_phase != SCST_SESS_SPH_READY)) {
+ PRINT_CRIT_ERROR("New mgmt cmd while shutting down the "
+ "session %p shut_phase %ld", sess, sess->shut_phase);
+ BUG();
+ }
+
+ local_irq_save(flags);
+
+ spin_lock(&sess->sess_list_lock);
+ atomic_inc(&sess->sess_cmd_count);
+
+ if (unlikely(sess->init_phase != SCST_SESS_IPH_READY)) {
+ switch (sess->init_phase) {
+ case SCST_SESS_IPH_INITING:
+ TRACE_DBG("Adding mcmd %p to init deferred mcmd list",
+ mcmd);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry,
+ &sess->init_deferred_mcmd_list);
+ goto out_unlock;
+ case SCST_SESS_IPH_SUCCESS:
+ break;
+ case SCST_SESS_IPH_FAILED:
+ res = -1;
+ goto out_unlock;
+ default:
+ BUG();
+ }
+ }
+
+ spin_unlock(&sess->sess_list_lock);
+
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to active mgmt cmd list", mcmd);
+ spin_lock(&scst_mcmd_lock);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry, &scst_active_mgmt_cmd_list);
+ spin_unlock(&scst_mcmd_lock);
+
+ local_irq_restore(flags);
+
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+
+out:
+ return res;
+
+out_unlock:
+ spin_unlock(&sess->sess_list_lock);
+ local_irq_restore(flags);
+ goto out;
+}
+
+/**
+ * scst_rx_mgmt_fn() - create new management command and send it for execution
+ *
+ * Description:
+ * Creates new management command and sends it for execution.
+ *
+ * Returns 0 for success, error code otherwise.
+ *
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same sess.
+ */
+int scst_rx_mgmt_fn(struct scst_session *sess,
+ const struct scst_rx_mgmt_params *params)
+{
+ int res = -EFAULT;
+ struct scst_mgmt_cmd *mcmd = NULL;
+
+ switch (params->fn) {
+ case SCST_ABORT_TASK:
+ BUG_ON(!params->tag_set);
+ break;
+ case SCST_TARGET_RESET:
+ case SCST_ABORT_ALL_TASKS:
+ case SCST_NEXUS_LOSS:
+ break;
+ default:
+ BUG_ON(!params->lun_set);
+ }
+
+ mcmd = scst_pre_rx_mgmt_cmd(sess, params->fn, params->atomic,
+ params->tgt_priv);
+ if (mcmd == NULL)
+ goto out;
+
+ if (params->lun_set) {
+ mcmd->lun = scst_unpack_lun(params->lun, params->lun_len);
+ if (mcmd->lun == NO_SUCH_LUN)
+ goto out_free;
+ mcmd->lun_set = 1;
+ }
+
+ if (params->tag_set)
+ mcmd->tag = params->tag;
+
+ mcmd->cmd_sn_set = params->cmd_sn_set;
+ mcmd->cmd_sn = params->cmd_sn;
+
+ if (params->fn <= SCST_TARGET_RESET)
+ TRACE(TRACE_MGMT, "TM fn %d", params->fn);
+ else
+ TRACE_MGMT_DBG("TM fn %d", params->fn);
+
+ TRACE_MGMT_DBG("sess=%p, tag_set %d, tag %lld, lun_set %d, "
+ "lun=%lld, cmd_sn_set %d, cmd_sn %d, priv %p", sess,
+ params->tag_set,
+ (long long unsigned int)params->tag,
+ params->lun_set,
+ (long long unsigned int)mcmd->lun,
+ params->cmd_sn_set,
+ params->cmd_sn,
+ params->tgt_priv);
+
+ if (scst_post_rx_mgmt_cmd(sess, mcmd) != 0)
+ goto out_free;
+
+ res = 0;
+
+out:
+ return res;
+
+out_free:
+ scst_free_mgmt_cmd(mcmd);
+ mcmd = NULL;
+ goto out;
+}
+EXPORT_SYMBOL(scst_rx_mgmt_fn);
+
+/*
+ * Written by Jack Handy - jakkhandy@hotmail.com
+ * Taken by Gennadiy Nerubayev <parakie@gmail.com> from
+ * http://www.codeproject.com/KB/string/wildcmp.aspx. No license attached
+ * to it, and it's posted on a free site; assumed to be free for use.
+ *
+ * Added the negative sign support - VLNB
+ *
+ * Also see comment for wildcmp().
+ *
+ * User space part of iSCSI-SCST also has a copy of this code, so fixing a bug
+ * here, don't forget to fix the copy too!
+ */
+static bool __wildcmp(const char *wild, const char *string, int recursion_level)
+{
+ const char *cp = NULL, *mp = NULL;
+
+ while ((*string) && (*wild != '*')) {
+ if ((*wild == '!') && (recursion_level == 0))
+ return !__wildcmp(++wild, string, ++recursion_level);
+
+ if ((*wild != *string) && (*wild != '?'))
+ return false;
+
+ wild++;
+ string++;
+ }
+
+ while (*string) {
+ if ((*wild == '!') && (recursion_level == 0))
+ return !__wildcmp(++wild, string, ++recursion_level);
+
+ if (*wild == '*') {
+ if (!*++wild)
+ return true;
+
+ mp = wild;
+ cp = string+1;
+ } else if ((*wild == *string) || (*wild == '?')) {
+ wild++;
+ string++;
+ } else {
+ wild = mp;
+ string = cp++;
+ }
+ }
+
+ while (*wild == '*')
+ wild++;
+
+ return !*wild;
+}
+
+/*
+ * Returns true if string "string" matches pattern "wild", false otherwise.
+ * Pattern is a regular DOS-type pattern, containing '*' and '?' symbols.
+ * '*' means match all any symbols, '?' means match only any single symbol.
+ *
+ * For instance:
+ * if (wildcmp("bl?h.*", "blah.jpg")) {
+ * // match
+ * } else {
+ * // no match
+ * }
+ *
+ * Also it supports boolean inversion sign '!', which does boolean inversion of
+ * the value of the rest of the string. Only one '!' allowed in the pattern,
+ * other '!' are treated as regular symbols. For instance:
+ * if (wildcmp("bl!?h.*", "blah.jpg")) {
+ * // no match
+ * } else {
+ * // match
+ * }
+ *
+ * Also see comment for __wildcmp().
+ */
+static bool wildcmp(const char *wild, const char *string)
+{
+ return __wildcmp(wild, string, 0);
+}
+
+/* scst_mutex supposed to be held */
+static struct scst_acg *scst_find_tgt_acg_by_name_wild(struct scst_tgt *tgt,
+ const char *initiator_name)
+{
+ struct scst_acg *acg, *res = NULL;
+ struct scst_acn *n;
+
+ if (initiator_name == NULL)
+ goto out;
+
+ list_for_each_entry(acg, &tgt->tgt_acg_list, acg_list_entry) {
+ list_for_each_entry(n, &acg->acn_list, acn_list_entry) {
+ if (wildcmp(n->name, initiator_name)) {
+ TRACE_DBG("Access control group %s found",
+ acg->acg_name);
+ res = acg;
+ goto out;
+ }
+ }
+ }
+
+out:
+ return res;
+}
+
+/* Must be called under scst_mutex */
+static struct scst_acg *__scst_find_acg(struct scst_tgt *tgt,
+ const char *initiator_name)
+{
+ struct scst_acg *acg = NULL;
+
+ acg = scst_find_tgt_acg_by_name_wild(tgt, initiator_name);
+ if (acg == NULL)
+ acg = tgt->default_acg;
+ return acg;
+}
+
+/* Must be called under scst_mutex */
+struct scst_acg *scst_find_acg(const struct scst_session *sess)
+{
+ return __scst_find_acg(sess->tgt, sess->initiator_name);
+}
+
+/**
+ * scst_initiator_has_luns() - check if this initiator will see any LUNs
+ *
+ * Checks if this initiator will see any LUNs upon connect to this target.
+ * Returns true if yes and false otherwise.
+ */
+bool scst_initiator_has_luns(struct scst_tgt *tgt, const char *initiator_name)
+{
+ bool res;
+ struct scst_acg *acg;
+
+ mutex_lock(&scst_mutex);
+
+ acg = __scst_find_acg(tgt, initiator_name);
+
+ res = !list_empty(&acg->acg_dev_list);
+
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_initiator_has_luns);
+
+static int scst_init_session(struct scst_session *sess)
+{
+ int res = 0;
+ struct scst_cmd *cmd;
+ struct scst_mgmt_cmd *mcmd, *tm;
+ int mwake = 0;
+
+ mutex_lock(&scst_mutex);
+
+ sess->acg = scst_find_acg(sess);
+
+ PRINT_INFO("Using security group \"%s\" for initiator \"%s\"",
+ sess->acg->acg_name, sess->initiator_name);
+
+ list_add_tail(&sess->acg_sess_list_entry, &sess->acg->acg_sess_list);
+
+ TRACE_DBG("Adding sess %p to tgt->sess_list", sess);
+ list_add_tail(&sess->sess_list_entry, &sess->tgt->sess_list);
+
+ if (sess->tgt->tgtt->get_initiator_port_transport_id != NULL) {
+ res = sess->tgt->tgtt->get_initiator_port_transport_id(sess,
+ &sess->transport_id);
+ if (res != 0) {
+ PRINT_ERROR("Unable to make initiator %s port "
+ "transport id", sess->initiator_name);
+ goto failed;
+ }
+ TRACE_PR("sess %p (ini %s), transport id %s/%d", sess,
+ sess->initiator_name,
+ debug_transport_id_to_initiator_name(
+ sess->transport_id), sess->tgt->rel_tgt_id);
+ }
+
+ res = scst_sess_sysfs_create(sess);
+ if (res != 0)
+ goto failed;
+
+ res = scst_sess_alloc_tgt_devs(sess);
+
+failed:
+ mutex_unlock(&scst_mutex);
+
+ if (sess->init_result_fn) {
+ TRACE_DBG("Calling init_result_fn(%p)", sess);
+ sess->init_result_fn(sess, sess->reg_sess_data, res);
+ TRACE_DBG("%s", "init_result_fn() returned");
+ }
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ if (res == 0)
+ sess->init_phase = SCST_SESS_IPH_SUCCESS;
+ else
+ sess->init_phase = SCST_SESS_IPH_FAILED;
+
+restart:
+ list_for_each_entry(cmd, &sess->init_deferred_cmd_list,
+ cmd_list_entry) {
+ TRACE_DBG("Deleting cmd %p from init deferred cmd list", cmd);
+ list_del(&cmd->cmd_list_entry);
+ atomic_dec(&sess->sess_cmd_count);
+ spin_unlock_irq(&sess->sess_list_lock);
+ scst_cmd_init_done(cmd, SCST_CONTEXT_THREAD);
+ spin_lock_irq(&sess->sess_list_lock);
+ goto restart;
+ }
+
+ spin_lock(&scst_mcmd_lock);
+ list_for_each_entry_safe(mcmd, tm, &sess->init_deferred_mcmd_list,
+ mgmt_cmd_list_entry) {
+ TRACE_DBG("Moving mgmt command %p from init deferred mcmd list",
+ mcmd);
+ list_move_tail(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ mwake = 1;
+ }
+
+ spin_unlock(&scst_mcmd_lock);
+ /*
+ * In case of an error at this point the caller target driver supposed
+ * to already call this sess's unregistration.
+ */
+ sess->init_phase = SCST_SESS_IPH_READY;
+ spin_unlock_irq(&sess->sess_list_lock);
+
+ if (mwake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+
+ scst_sess_put(sess);
+ return res;
+}
+
+/**
+ * scst_register_session() - register session
+ * @tgt: target
+ * @atomic: true, if the function called in the atomic context. If false,
+ * this function will block until the session registration is
+ * completed.
+ * @initiator_name: remote initiator's name, any NULL-terminated string,
+ * e.g. iSCSI name, which used as the key to found appropriate
+ * access control group. Could be NULL, then the default
+ * target's LUNs are used.
+ * @tgt_priv: pointer to target driver's private data
+ * @result_fn_data: any target driver supplied data
+ * @result_fn: pointer to the function that will be asynchronously called
+ * when session initialization finishes.
+ * Can be NULL. Parameters:
+ * - sess - session
+ * - data - target driver supplied to scst_register_session()
+ * data
+ * - result - session initialization result, 0 on success or
+ * appropriate error code otherwise
+ *
+ * Description:
+ * Registers new session. Returns new session on success or NULL otherwise.
+ *
+ * Note: A session creation and initialization is a complex task,
+ * which requires sleeping state, so it can't be fully done
+ * in interrupt context. Therefore the "bottom half" of it, if
+ * scst_register_session() is called from atomic context, will be
+ * done in SCST thread context. In this case scst_register_session()
+ * will return not completely initialized session, but the target
+ * driver can supply commands to this session via scst_rx_cmd().
+ * Those commands processing will be delayed inside SCST until
+ * the session initialization is finished, then their processing
+ * will be restarted. The target driver will be notified about
+ * finish of the session initialization by function result_fn().
+ * On success the target driver could do nothing, but if the
+ * initialization fails, the target driver must ensure that
+ * no more new commands being sent or will be sent to SCST after
+ * result_fn() returns. All already sent to SCST commands for
+ * failed session will be returned in xmit_response() with BUSY status.
+ * In case of failure the driver shall call scst_unregister_session()
+ * inside result_fn(), it will NOT be called automatically.
+ */
+struct scst_session *scst_register_session(struct scst_tgt *tgt, int atomic,
+ const char *initiator_name, void *tgt_priv, void *result_fn_data,
+ void (*result_fn) (struct scst_session *sess, void *data, int result))
+{
+ struct scst_session *sess;
+ int res;
+ unsigned long flags;
+
+ sess = scst_alloc_session(tgt, atomic ? GFP_ATOMIC : GFP_KERNEL,
+ initiator_name);
+ if (sess == NULL)
+ goto out;
+
+ scst_sess_set_tgt_priv(sess, tgt_priv);
+
+ scst_sess_get(sess); /* one for registered session */
+ scst_sess_get(sess); /* one held until sess is inited */
+
+ if (atomic) {
+ sess->reg_sess_data = result_fn_data;
+ sess->init_result_fn = result_fn;
+ spin_lock_irqsave(&scst_mgmt_lock, flags);
+ TRACE_DBG("Adding sess %p to scst_sess_init_list", sess);
+ list_add_tail(&sess->sess_init_list_entry,
+ &scst_sess_init_list);
+ spin_unlock_irqrestore(&scst_mgmt_lock, flags);
+ wake_up(&scst_mgmt_waitQ);
+ } else {
+ res = scst_init_session(sess);
+ if (res != 0)
+ goto out_free;
+ }
+
+out:
+ return sess;
+
+out_free:
+ scst_free_session(sess);
+ sess = NULL;
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_register_session);
+
+/**
+ * scst_register_session_non_gpl() - register session (non-GPL version)
+ * @tgt: target
+ * @initiator_name: remote initiator's name, any NULL-terminated string,
+ * e.g. iSCSI name, which used as the key to found appropriate
+ * access control group. Could be NULL, then the default
+ * target's LUNs are used.
+ * @tgt_priv: pointer to target driver's private data
+ *
+ * Description:
+ * Registers new session. Returns new session on success or NULL otherwise.
+ */
+struct scst_session *scst_register_session_non_gpl(struct scst_tgt *tgt,
+ const char *initiator_name, void *tgt_priv)
+{
+ return scst_register_session(tgt, 0, initiator_name, tgt_priv,
+ NULL, NULL);
+}
+EXPORT_SYMBOL(scst_register_session_non_gpl);
+
+/**
+ * scst_unregister_session() - unregister session
+ * @sess: session to be unregistered
+ * @wait: if true, instructs to wait until all commands, which
+ * currently is being executed and belonged to the session,
+ * finished. Otherwise, target driver should be prepared to
+ * receive xmit_response() for the session's command after
+ * scst_unregister_session() returns.
+ * @unreg_done_fn: pointer to the function that will be asynchronously called
+ * when the last session's command finishes and
+ * the session is about to be completely freed. Can be NULL.
+ * Parameter:
+ * - sess - session
+ *
+ * Unregisters session.
+ *
+ * Notes:
+ * - All outstanding commands will be finished regularly. After
+ * scst_unregister_session() returned, no new commands must be sent to
+ * SCST via scst_rx_cmd().
+ *
+ * - The caller must ensure that no scst_rx_cmd() or scst_rx_mgmt_fn_*() is
+ * called in paralell with scst_unregister_session().
+ *
+ * - Can be called before result_fn() of scst_register_session() called,
+ * i.e. during the session registration/initialization.
+ *
+ * - It is highly recommended to call scst_unregister_session() as soon as it
+ * gets clear that session will be unregistered and not to wait until all
+ * related commands finished. This function provides the wait functionality,
+ * but it also starts recovering stuck commands, if there are any.
+ * Otherwise, your target driver could wait for those commands forever.
+ */
+void scst_unregister_session(struct scst_session *sess, int wait,
+ void (*unreg_done_fn) (struct scst_session *sess))
+{
+ unsigned long flags;
+ DECLARE_COMPLETION_ONSTACK(c);
+ int rc, lun;
+
+ TRACE_MGMT_DBG("Unregistering session %p (wait %d)", sess, wait);
+
+ sess->unreg_done_fn = unreg_done_fn;
+
+ /* Abort all outstanding commands and clear reservation, if necessary */
+ lun = 0;
+ rc = scst_rx_mgmt_fn_lun(sess, SCST_UNREG_SESS_TM,
+ (uint8_t *)&lun, sizeof(lun), SCST_ATOMIC, NULL);
+ if (rc != 0) {
+ PRINT_ERROR("SCST_UNREG_SESS_TM failed %d (sess %p)",
+ rc, sess);
+ }
+
+ sess->shut_phase = SCST_SESS_SPH_SHUTDOWN;
+
+ spin_lock_irqsave(&scst_mgmt_lock, flags);
+
+ if (wait)
+ sess->shutdown_compl = &c;
+
+ spin_unlock_irqrestore(&scst_mgmt_lock, flags);
+
+ scst_sess_put(sess);
+
+ if (wait) {
+ TRACE_DBG("Waiting for session %p to complete", sess);
+ wait_for_completion(&c);
+ }
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_session);
+
+/**
+ * scst_unregister_session_non_gpl() - unregister session, non-GPL version
+ * @sess: session to be unregistered
+ *
+ * Unregisters session.
+ *
+ * See notes for scst_unregister_session() above.
+ */
+void scst_unregister_session_non_gpl(struct scst_session *sess)
+{
+
+ scst_unregister_session(sess, 1, NULL);
+ return;
+}
+EXPORT_SYMBOL(scst_unregister_session_non_gpl);
+
+static inline int test_mgmt_list(void)
+{
+ int res = !list_empty(&scst_sess_init_list) ||
+ !list_empty(&scst_sess_shut_list) ||
+ unlikely(kthread_should_stop());
+ return res;
+}
+
+int scst_global_mgmt_thread(void *arg)
+{
+ struct scst_session *sess;
+
+ PRINT_INFO("Management thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ set_user_nice(current, -10);
+
+ spin_lock_irq(&scst_mgmt_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_mgmt_list()) {
+ add_wait_queue_exclusive(&scst_mgmt_waitQ, &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_mgmt_list())
+ break;
+ spin_unlock_irq(&scst_mgmt_lock);
+ schedule();
+ spin_lock_irq(&scst_mgmt_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&scst_mgmt_waitQ, &wait);
+ }
+
+ while (!list_empty(&scst_sess_init_list)) {
+ sess = list_entry(scst_sess_init_list.next,
+ typeof(*sess), sess_init_list_entry);
+ TRACE_DBG("Removing sess %p from scst_sess_init_list",
+ sess);
+ list_del(&sess->sess_init_list_entry);
+ spin_unlock_irq(&scst_mgmt_lock);
+
+ if (sess->init_phase == SCST_SESS_IPH_INITING)
+ scst_init_session(sess);
+ else {
+ PRINT_CRIT_ERROR("session %p is in "
+ "scst_sess_init_list, but in unknown "
+ "init phase %x", sess,
+ sess->init_phase);
+ BUG();
+ }
+
+ spin_lock_irq(&scst_mgmt_lock);
+ }
+
+ while (!list_empty(&scst_sess_shut_list)) {
+ sess = list_entry(scst_sess_shut_list.next,
+ typeof(*sess), sess_shut_list_entry);
+ TRACE_DBG("Removing sess %p from scst_sess_shut_list",
+ sess);
+ list_del(&sess->sess_shut_list_entry);
+ spin_unlock_irq(&scst_mgmt_lock);
+
+ switch (sess->shut_phase) {
+ case SCST_SESS_SPH_SHUTDOWN:
+ BUG_ON(atomic_read(&sess->refcnt) != 0);
+ scst_free_session_callback(sess);
+ break;
+ default:
+ PRINT_CRIT_ERROR("session %p is in "
+ "scst_sess_shut_list, but in unknown "
+ "shut phase %lx", sess,
+ sess->shut_phase);
+ BUG();
+ break;
+ }
+
+ spin_lock_irq(&scst_mgmt_lock);
+ }
+ }
+ spin_unlock_irq(&scst_mgmt_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so both lists must be empty.
+ */
+ BUG_ON(!list_empty(&scst_sess_init_list));
+ BUG_ON(!list_empty(&scst_sess_shut_list));
+
+ PRINT_INFO("Management thread PID %d finished", current->pid);
+ return 0;
+}
+
+/* Called under sess->sess_list_lock */
+static struct scst_cmd *__scst_find_cmd_by_tag(struct scst_session *sess,
+ uint64_t tag, bool to_abort)
+{
+ struct scst_cmd *cmd, *res = NULL;
+
+ /* ToDo: hash list */
+
+ TRACE_DBG("%s (sess=%p, tag=%llu)", "Searching in sess cmd list",
+ sess, (long long unsigned int)tag);
+
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if (cmd->tag == tag) {
+ /*
+ * We must not count done commands, because
+ * they were submitted for transmittion.
+ * Otherwise we can have a race, when for
+ * some reason cmd's release delayed
+ * after transmittion and initiator sends
+ * cmd with the same tag => it can be possible
+ * that a wrong cmd will be returned.
+ */
+ if (cmd->done) {
+ if (to_abort) {
+ /*
+ * We should return the latest not
+ * aborted cmd with this tag.
+ */
+ if (res == NULL)
+ res = cmd;
+ else {
+ if (test_bit(SCST_CMD_ABORTED,
+ &res->cmd_flags)) {
+ res = cmd;
+ } else if (!test_bit(SCST_CMD_ABORTED,
+ &cmd->cmd_flags))
+ res = cmd;
+ }
+ }
+ continue;
+ } else {
+ res = cmd;
+ break;
+ }
+ }
+ }
+ return res;
+}
+
+/**
+ * scst_find_cmd() - find command by custom comparison function
+ *
+ * Finds a command based on user supplied data and comparision
+ * callback function, that should return true, if the command is found.
+ * Returns the command on success or NULL otherwise.
+ */
+struct scst_cmd *scst_find_cmd(struct scst_session *sess, void *data,
+ int (*cmp_fn) (struct scst_cmd *cmd,
+ void *data))
+{
+ struct scst_cmd *cmd = NULL;
+ unsigned long flags = 0;
+
+ if (cmp_fn == NULL)
+ goto out;
+
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list, sess_cmd_list_entry) {
+ /*
+ * We must not count done commands, because they were
+ * submitted for transmittion. Otherwise we can have a race,
+ * when for some reason cmd's release delayed after
+ * transmittion and initiator sends cmd with the same tag =>
+ * it can be possible that a wrong cmd will be returned.
+ */
+ if (cmd->done)
+ continue;
+ if (cmp_fn(cmd, data))
+ goto out_unlock;
+ }
+
+ cmd = NULL;
+
+out_unlock:
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+
+out:
+ return cmd;
+}
+EXPORT_SYMBOL(scst_find_cmd);
+
+/**
+ * scst_find_cmd_by_tag() - find command by tag
+ *
+ * Finds a command based on the supplied tag comparing it with one
+ * that previously set by scst_cmd_set_tag(). Returns the found command on
+ * success or NULL otherwise.
+ */
+struct scst_cmd *scst_find_cmd_by_tag(struct scst_session *sess,
+ uint64_t tag)
+{
+ unsigned long flags;
+ struct scst_cmd *cmd;
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+ cmd = __scst_find_cmd_by_tag(sess, tag, false);
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ return cmd;
+}
+EXPORT_SYMBOL(scst_find_cmd_by_tag);
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 6/17]: SCST internal library functions
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (4 preceding siblings ...)
2010-09-14 14:41 ` [PATCH 5/17]: SCST implementation of the SCSI target state machine Vladislav Bolkhovitin
@ 2010-09-14 14:42 ` Vladislav Bolkhovitin
2010-09-14 14:43 ` [PATCH 7/17]: SCST Persistent Reservations implementation Vladislav Bolkhovitin
` (11 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:42 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST internal library functions
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_lib.c | 7030 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 7030 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/scst_lib.c linux-2.6.35/drivers/scst/scst_lib.c
--- orig/linux-2.6.35/drivers/scst/scst_lib.c
+++ linux-2.6.35/drivers/scst/scst_lib.c
@@ -0,0 +1,7030 @@
+/*
+ * scst_lib.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/kthread.h>
+#include <linux/cdrom.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+#include <linux/ctype.h>
+#include <linux/delay.h>
+#include <asm/kmap_types.h>
+#include <asm/unaligned.h>
+
+#include <scst/scst.h>
+#include "scst_priv.h"
+#include "scst_mem.h"
+#include "scst_pres.h"
+
+struct scsi_io_context {
+ unsigned int full_cdb_used:1;
+ void *data;
+ void (*done)(void *data, char *sense, int result, int resid);
+ char sense[SCST_SENSE_BUFFERSIZE];
+ unsigned char full_cdb[0];
+};
+static struct kmem_cache *scsi_io_context_cache;
+
+/* get_trans_len_x extract x bytes from cdb as length starting from off */
+static int get_trans_len_1(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_1_256(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_2(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_3(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_4(struct scst_cmd *cmd, uint8_t off);
+
+static int get_bidi_trans_len_2(struct scst_cmd *cmd, uint8_t off);
+
+/* for special commands */
+static int get_trans_len_block_limit(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_read_capacity(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_serv_act_in(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_single(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_none(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_read_pos(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_cdb_len_10(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_prevent_allow_medium_removal(struct scst_cmd *cmd,
+ uint8_t off);
+static int get_trans_len_3_read_elem_stat(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_start_stop(struct scst_cmd *cmd, uint8_t off);
+
+/*
++=====================================-============-======-
+| Command name | Operation | Type |
+| | code | |
+|-------------------------------------+------------+------+
+
++=========================================================+
+|Key: M = command implementation is mandatory. |
+| O = command implementation is optional. |
+| V = Vendor-specific |
+| R = Reserved |
+| ' '= DON'T use for this device |
++=========================================================+
+*/
+
+#define SCST_CDB_MANDATORY 'M' /* mandatory */
+#define SCST_CDB_OPTIONAL 'O' /* optional */
+#define SCST_CDB_VENDOR 'V' /* vendor */
+#define SCST_CDB_RESERVED 'R' /* reserved */
+#define SCST_CDB_NOTSUPP ' ' /* don't use */
+
+struct scst_sdbops {
+ uint8_t ops; /* SCSI-2 op codes */
+ uint8_t devkey[16]; /* Key for every device type M,O,V,R
+ * type_disk devkey[0]
+ * type_tape devkey[1]
+ * type_printer devkey[2]
+ * type_proseccor devkey[3]
+ * type_worm devkey[4]
+ * type_cdrom devkey[5]
+ * type_scanner devkey[6]
+ * type_mod devkey[7]
+ * type_changer devkey[8]
+ * type_commdev devkey[9]
+ * type_reserv devkey[A]
+ * type_reserv devkey[B]
+ * type_raid devkey[C]
+ * type_enclosure devkey[D]
+ * type_reserv devkey[E]
+ * type_reserv devkey[F]
+ */
+ const char *op_name; /* SCSI-2 op codes full name */
+ uint8_t direction; /* init --> target: SCST_DATA_WRITE
+ * target --> init: SCST_DATA_READ
+ */
+ uint16_t flags; /* opcode -- various flags */
+ uint8_t off; /* length offset in cdb */
+ int (*get_trans_len)(struct scst_cmd *cmd, uint8_t off);
+};
+
+static int scst_scsi_op_list[256];
+
+#define FLAG_NONE 0
+
+static const struct scst_sdbops scst_scsi_op_table[] = {
+ /*
+ * +-------------------> TYPE_IS_DISK (0)
+ * |
+ * |+------------------> TYPE_IS_TAPE (1)
+ * ||
+ * || +----------------> TYPE_IS_PROCESSOR (3)
+ * || |
+ * || | +--------------> TYPE_IS_CDROM (5)
+ * || | |
+ * || | | +------------> TYPE_IS_MOD (7)
+ * || | | |
+ * || | | |+-----------> TYPE_IS_CHANGER (8)
+ * || | | ||
+ * || | | || +-------> TYPE_IS_RAID (C)
+ * || | | || |
+ * || | | || |
+ * 0123456789ABCDEF ---> TYPE_IS_???? */
+
+ /* 6-bytes length CDB */
+ {0x00, "MMMMMMMMMMMMMMMM", "TEST UNIT READY",
+ /* let's be HQ to don't look dead under high load */
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_IMPLICIT_HQ|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_EXCL_ACCESS_ALLOWED,
+ 0, get_trans_len_none},
+ {0x01, " M ", "REWIND",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x01, "O V OO OO ", "REZERO UNIT",
+ SCST_DATA_NONE, SCST_WRITE_EXCL_ALLOWED,
+ 0, get_trans_len_none},
+ {0x02, "VVVVVV V ", "REQUEST BLOCK ADDR",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT, 0, get_trans_len_none},
+ {0x03, "MMMMMMMMMMMMMMMM", "REQUEST SENSE",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_SKIP_UA|SCST_LOCAL_CMD|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 4, get_trans_len_1},
+ {0x04, "M O O ", "FORMAT UNIT",
+ SCST_DATA_WRITE, SCST_LONG_TIMEOUT|SCST_UNKNOWN_LENGTH|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x04, " O ", "FORMAT",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x05, "VMVVVV V ", "READ BLOCK LIMITS",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 0, get_trans_len_block_limit},
+ {0x07, " O ", "INITIALIZE ELEMENT STATUS",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x07, "OVV O OV ", "REASSIGN BLOCKS",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x08, "O ", "READ(6)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_EXCL_ALLOWED,
+ 4, get_trans_len_1_256},
+ {0x08, " MV OO OV ", "READ(6)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_WRITE_EXCL_ALLOWED,
+ 2, get_trans_len_3},
+ {0x08, " M ", "GET MESSAGE(6)",
+ SCST_DATA_READ, FLAG_NONE, 2, get_trans_len_3},
+ {0x08, " O ", "RECEIVE",
+ SCST_DATA_READ, FLAG_NONE, 2, get_trans_len_3},
+ {0x0A, "O ", "WRITE(6)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_MEDIUM,
+ 4, get_trans_len_1_256},
+ {0x0A, " M O OV ", "WRITE(6)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 2, get_trans_len_3},
+ {0x0A, " M ", "PRINT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0A, " M ", "SEND MESSAGE(6)",
+ SCST_DATA_WRITE, FLAG_NONE, 2, get_trans_len_3},
+ {0x0A, " M ", "SEND(6)",
+ SCST_DATA_WRITE, FLAG_NONE, 2, get_trans_len_3},
+ {0x0B, "O OO OV ", "SEEK(6)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0B, " ", "TRACK SELECT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0B, " O ", "SLEW AND PRINT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0C, "VVVVVV V ", "SEEK BLOCK",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x0D, "VVVVVV V ", "PARTITION",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x0F, "VOVVVV V ", "READ REVERSE",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_WRITE_EXCL_ALLOWED,
+ 2, get_trans_len_3},
+ {0x10, "VM V V ", "WRITE FILEMARKS",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x10, " O O ", "SYNCHRONIZE BUFFER",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x11, "VMVVVV ", "SPACE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|
+ SCST_WRITE_EXCL_ALLOWED,
+ 0, get_trans_len_none},
+ {0x12, "MMMMMMMMMMMMMMMM", "INQUIRY",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_IMPLICIT_HQ|SCST_SKIP_UA|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|SCST_EXCL_ACCESS_ALLOWED,
+ 4, get_trans_len_1},
+ {0x13, "VOVVVV ", "VERIFY(6)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED,
+ 2, get_trans_len_3},
+ {0x14, "VOOVVV ", "RECOVER BUFFERED DATA",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_WRITE_EXCL_ALLOWED,
+ 2, get_trans_len_3},
+ {0x15, "OMOOOOOOOOOOOOOO", "MODE SELECT(6)",
+ SCST_DATA_WRITE, SCST_LOCAL_CMD, 4, get_trans_len_1},
+ {0x16, "MMMMMMMMMMMMMMMM", "RESERVE",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD|
+ SCST_WRITE_EXCL_ALLOWED|SCST_EXCL_ACCESS_ALLOWED,
+ 0, get_trans_len_none},
+ {0x17, "MMMMMMMMMMMMMMMM", "RELEASE",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|SCST_EXCL_ACCESS_ALLOWED,
+ 0, get_trans_len_none},
+ {0x18, "OOOOOOOO ", "COPY",
+ SCST_DATA_WRITE, SCST_LONG_TIMEOUT, 2, get_trans_len_3},
+ {0x19, "VMVVVV ", "ERASE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x1A, "OMOOOOOOOOOOOOOO", "MODE SENSE(6)",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT, 4, get_trans_len_1},
+ {0x1B, " O ", "SCAN",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x1B, " O ", "LOAD UNLOAD",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x1B, " O ", "STOP PRINT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x1B, "O OO O O ", "START STOP UNIT",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_start_stop},
+ {0x1C, "OOOOOOOOOOOOOOOO", "RECEIVE DIAGNOSTIC RESULTS",
+ SCST_DATA_READ, FLAG_NONE, 3, get_trans_len_2},
+ {0x1D, "MMMMMMMMMMMMMMMM", "SEND DIAGNOSTIC",
+ SCST_DATA_WRITE, FLAG_NONE, 4, get_trans_len_1},
+ {0x1E, "OOOOOOOOOOOOOOOO", "PREVENT ALLOW MEDIUM REMOVAL",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0,
+ get_trans_len_prevent_allow_medium_removal},
+ {0x1F, " O ", "PORT STATUS",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+
+ /* 10-bytes length CDB */
+ {0x23, "V VV V ", "READ FORMAT CAPACITY",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x24, "V VVM ", "SET WINDOW",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_3},
+ {0x25, "M MM M ", "READ CAPACITY",
+ SCST_DATA_READ, SCST_IMPLICIT_HQ|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 0, get_trans_len_read_capacity},
+ {0x25, " O ", "GET WINDOW",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_3},
+ {0x28, "M MMMM ", "READ(10)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_EXCL_ALLOWED,
+ 7, get_trans_len_2},
+ {0x28, " O ", "GET MESSAGE(10)",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x29, "V VV O ", "READ GENERATION",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_1},
+ {0x2A, "O MO M ", "WRITE(10)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_MEDIUM,
+ 7, get_trans_len_2},
+ {0x2A, " O ", "SEND MESSAGE(10)",
+ SCST_DATA_WRITE, FLAG_NONE, 7, get_trans_len_2},
+ {0x2A, " O ", "SEND(10)",
+ SCST_DATA_WRITE, FLAG_NONE, 7, get_trans_len_2},
+ {0x2B, " O ", "LOCATE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|
+ SCST_WRITE_EXCL_ALLOWED,
+ 0, get_trans_len_none},
+ {0x2B, " O ", "POSITION TO ELEMENT",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x2B, "O OO O ", "SEEK(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x2C, "V O O ", "ERASE(10)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x2D, "V O O ", "READ UPDATED BLOCK",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 0, get_trans_len_single},
+ {0x2E, "O OO O ", "WRITE AND VERIFY(10)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 7, get_trans_len_2},
+ {0x2F, "O OO O ", "VERIFY(10)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED,
+ 7, get_trans_len_2},
+ {0x33, "O OO O ", "SET LIMITS(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x34, " O ", "READ POSITION",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|
+ SCST_WRITE_EXCL_ALLOWED,
+ 7, get_trans_len_read_pos},
+ {0x34, " O ", "GET DATA BUFFER STATUS",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x34, "O OO O ", "PRE-FETCH",
+ SCST_DATA_NONE, SCST_WRITE_EXCL_ALLOWED,
+ 0, get_trans_len_none},
+ {0x35, "O OO O ", "SYNCHRONIZE CACHE",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x36, "O OO O ", "LOCK UNLOCK CACHE",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x37, "O O ", "READ DEFECT DATA(10)",
+ SCST_DATA_READ, SCST_WRITE_EXCL_ALLOWED,
+ 8, get_trans_len_1},
+ {0x37, " O ", "INIT ELEMENT STATUS WRANGE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x38, " O O ", "MEDIUM SCAN",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_1},
+ {0x39, "OOOOOOOO ", "COMPARE",
+ SCST_DATA_WRITE, FLAG_NONE, 3, get_trans_len_3},
+ {0x3A, "OOOOOOOO ", "COPY AND VERIFY",
+ SCST_DATA_WRITE, FLAG_NONE, 3, get_trans_len_3},
+ {0x3B, "OOOOOOOOOOOOOOOO", "WRITE BUFFER",
+ SCST_DATA_WRITE, SCST_SMALL_TIMEOUT, 6, get_trans_len_3},
+ {0x3C, "OOOOOOOOOOOOOOOO", "READ BUFFER",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT, 6, get_trans_len_3},
+ {0x3D, " O O ", "UPDATE BLOCK",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED,
+ 0, get_trans_len_single},
+ {0x3E, "O OO O ", "READ LONG",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x3F, "O O O ", "WRITE LONG",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 7, get_trans_len_2},
+ {0x40, "OOOOOOOOOO ", "CHANGE DEFINITION",
+ SCST_DATA_WRITE, SCST_SMALL_TIMEOUT, 8, get_trans_len_1},
+ {0x41, "O O ", "WRITE SAME",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_single},
+ {0x42, " O ", "READ SUB-CHANNEL",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x43, " O ", "READ TOC/PMA/ATIP",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x44, " M ", "REPORT DENSITY SUPPORT",
+ SCST_DATA_READ, SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 7, get_trans_len_2},
+ {0x44, " O ", "READ HEADER",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x45, " O ", "PLAY AUDIO(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x46, " O ", "GET CONFIGURATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x47, " O ", "PLAY AUDIO MSF",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x48, " O ", "PLAY AUDIO TRACK INDEX",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x49, " O ", "PLAY TRACK RELATIVE(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x4A, " O ", "GET EVENT STATUS NOTIFICATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x4B, " O ", "PAUSE/RESUME",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x4C, "OOOOOOOOOOOOOOOO", "LOG SELECT",
+ SCST_DATA_WRITE, SCST_SMALL_TIMEOUT, 7, get_trans_len_2},
+ {0x4D, "OOOOOOOOOOOOOOOO", "LOG SENSE",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 7, get_trans_len_2},
+ {0x4E, " O ", "STOP PLAY/SCAN",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x50, " ", "XDWRITE",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x51, " O ", "READ DISC INFORMATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x51, " ", "XPWRITE",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x52, " O ", "READ TRACK INFORMATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x53, "O ", "XDWRITEREAD(10)",
+ SCST_DATA_READ|SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_WRITE_MEDIUM,
+ 7, get_bidi_trans_len_2},
+ {0x53, " O ", "RESERVE TRACK",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x54, " O ", "SEND OPC INFORMATION",
+ SCST_DATA_WRITE, FLAG_NONE, 7, get_trans_len_2},
+ {0x55, "OOOOOOOOOOOOOOOO", "MODE SELECT(10)",
+ SCST_DATA_WRITE, SCST_LOCAL_CMD, 7, get_trans_len_2},
+ {0x56, "OOOOOOOOOOOOOOOO", "RESERVE(10)",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD,
+ 0, get_trans_len_none},
+ {0x57, "OOOOOOOOOOOOOOOO", "RELEASE(10)",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD|
+ SCST_REG_RESERVE_ALLOWED,
+ 0, get_trans_len_none},
+ {0x58, " O ", "REPAIR TRACK",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x5A, "OOOOOOOOOOOOOOOO", "MODE SENSE(10)",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT, 7, get_trans_len_2},
+ {0x5B, " O ", "CLOSE TRACK/SESSION",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x5C, " O ", "READ BUFFER CAPACITY",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x5D, " O ", "SEND CUE SHEET",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_3},
+ {0x5E, "OOOOO OOOO ", "PERSISTENT RESERV IN",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|
+ SCST_LOCAL_CMD|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 5, get_trans_len_4},
+ {0x5F, "OOOOO OOOO ", "PERSISTENT RESERV OUT",
+ SCST_DATA_WRITE, SCST_SMALL_TIMEOUT|
+ SCST_LOCAL_CMD|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 5, get_trans_len_4},
+
+ /* 16-bytes length CDB */
+ {0x80, "O OO O ", "XDWRITE EXTENDED",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x80, " M ", "WRITE FILEMARKS",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x81, "O OO O ", "REBUILD",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x82, "O OO O ", "REGENERATE",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x83, "OOOOOOOOOOOOOOOO", "EXTENDED COPY",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x84, "OOOOOOOOOOOOOOOO", "RECEIVE COPY RESULT",
+ SCST_DATA_WRITE, FLAG_NONE, 10, get_trans_len_4},
+ {0x86, "OOOOOOOOOO ", "ACCESS CONTROL IN",
+ SCST_DATA_NONE, SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 0, get_trans_len_none},
+ {0x87, "OOOOOOOOOO ", "ACCESS CONTROL OUT",
+ SCST_DATA_NONE, SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|
+ SCST_EXCL_ACCESS_ALLOWED,
+ 0, get_trans_len_none},
+ {0x88, "M MMMM ", "READ(16)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_EXCL_ALLOWED,
+ 10, get_trans_len_4},
+ {0x8A, "O OO O ", "WRITE(16)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_MEDIUM,
+ 10, get_trans_len_4},
+ {0x8C, "OOOOOOOOOO ", "READ ATTRIBUTE",
+ SCST_DATA_READ, FLAG_NONE, 10, get_trans_len_4},
+ {0x8D, "OOOOOOOOOO ", "WRITE ATTRIBUTE",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x8E, "O OO O ", "WRITE AND VERIFY(16)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 10, get_trans_len_4},
+ {0x8F, "O OO O ", "VERIFY(16)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED,
+ 10, get_trans_len_4},
+ {0x90, "O OO O ", "PRE-FETCH(16)",
+ SCST_DATA_NONE, SCST_WRITE_EXCL_ALLOWED,
+ 0, get_trans_len_none},
+ {0x91, "O OO O ", "SYNCHRONIZE CACHE(16)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x91, " M ", "SPACE(16)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|
+ SCST_WRITE_EXCL_ALLOWED,
+ 0, get_trans_len_none},
+ {0x92, "O OO O ", "LOCK UNLOCK CACHE(16)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x92, " O ", "LOCATE(16)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|
+ SCST_WRITE_EXCL_ALLOWED,
+ 0, get_trans_len_none},
+ {0x93, "O O ", "WRITE SAME(16)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 10, get_trans_len_4},
+ {0x93, " M ", "ERASE(16)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x9E, "O ", "SERVICE ACTION IN",
+ SCST_DATA_READ, FLAG_NONE, 0, get_trans_len_serv_act_in},
+
+ /* 12-bytes length CDB */
+ {0xA0, "VVVVVVVVVV M ", "REPORT LUNS",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_IMPLICIT_HQ|SCST_SKIP_UA|
+ SCST_FULLY_LOCAL_CMD|SCST_LOCAL_CMD|
+ SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|SCST_EXCL_ACCESS_ALLOWED,
+ 6, get_trans_len_4},
+ {0xA1, " O ", "BLANK",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xA3, " O ", "SEND KEY",
+ SCST_DATA_WRITE, FLAG_NONE, 8, get_trans_len_2},
+ {0xA3, "OOOOO OOOO ", "REPORT DEVICE IDENTIDIER",
+ SCST_DATA_READ, SCST_REG_RESERVE_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED|SCST_EXCL_ACCESS_ALLOWED,
+ 6, get_trans_len_4},
+ {0xA3, " M ", "MAINTENANCE(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xA4, " O ", "REPORT KEY",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_2},
+ {0xA4, " O ", "MAINTENANCE(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xA5, " M ", "MOVE MEDIUM",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xA5, " O ", "PLAY AUDIO(12)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xA6, " O O ", "EXCHANGE/LOAD/UNLOAD MEDIUM",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xA7, " O ", "SET READ AHEAD",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xA8, " O ", "GET MESSAGE(12)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xA8, "O OO O ", "READ(12)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_EXCL_ALLOWED,
+ 6, get_trans_len_4},
+ {0xA9, " O ", "PLAY TRACK RELATIVE(12)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xAA, "O OO O ", "WRITE(12)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ SCST_TEST_IO_IN_SIRQ_ALLOWED|
+#endif
+ SCST_WRITE_MEDIUM,
+ 6, get_trans_len_4},
+ {0xAA, " O ", "SEND MESSAGE(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xAC, " O ", "ERASE(12)",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0xAC, " M ", "GET PERFORMANCE",
+ SCST_DATA_READ, SCST_UNKNOWN_LENGTH, 0, get_trans_len_none},
+ {0xAD, " O ", "READ DVD STRUCTURE",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_2},
+ {0xAE, "O OO O ", "WRITE AND VERIFY(12)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 6, get_trans_len_4},
+ {0xAF, "O OO O ", "VERIFY(12)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED|
+ SCST_WRITE_EXCL_ALLOWED,
+ 6, get_trans_len_4},
+#if 0 /* No need to support at all */
+ {0xB0, " OO O ", "SEARCH DATA HIGH(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+ {0xB1, " OO O ", "SEARCH DATA EQUAL(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+ {0xB2, " OO O ", "SEARCH DATA LOW(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+#endif
+ {0xB3, " OO O ", "SET LIMITS(12)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xB5, " O ", "REQUEST VOLUME ELEMENT ADDRESS",
+ SCST_DATA_READ, FLAG_NONE, 9, get_trans_len_1},
+ {0xB6, " O ", "SEND VOLUME TAG",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+ {0xB6, " M ", "SET STREAMING",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_2},
+ {0xB7, " O ", "READ DEFECT DATA(12)",
+ SCST_DATA_READ, SCST_WRITE_EXCL_ALLOWED,
+ 9, get_trans_len_1},
+ {0xB8, " O ", "READ ELEMENT STATUS",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_3_read_elem_stat},
+ {0xB9, " O ", "READ CD MSF",
+ SCST_DATA_READ, SCST_UNKNOWN_LENGTH, 0, get_trans_len_none},
+ {0xBA, " O ", "SCAN",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xBA, " O ", "REDUNDANCY GROUP(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xBB, " O ", "SET SPEED",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xBB, " O ", "REDUNDANCY GROUP(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xBC, " O ", "SPARE(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xBD, " O ", "MECHANISM STATUS",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_2},
+ {0xBD, " O ", "SPARE(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xBE, " O ", "READ CD",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 6, get_trans_len_3},
+ {0xBE, " O ", "VOLUME SET(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xBF, " O ", "SEND DVD STRUCTUE",
+ SCST_DATA_WRITE, FLAG_NONE, 8, get_trans_len_2},
+ {0xBF, " O ", "VOLUME SET(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xE7, " V ", "INIT ELEMENT STATUS WRANGE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_cdb_len_10}
+};
+
+#define SCST_CDB_TBL_SIZE ((int)ARRAY_SIZE(scst_scsi_op_table))
+
+static void scst_free_tgt_dev(struct scst_tgt_dev *tgt_dev);
+static void scst_check_internal_sense(struct scst_device *dev, int result,
+ uint8_t *sense, int sense_len);
+static void scst_queue_report_luns_changed_UA(struct scst_session *sess,
+ int flags);
+static void __scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags);
+static void scst_alloc_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags);
+static void scst_free_all_UA(struct scst_tgt_dev *tgt_dev);
+static void scst_release_space(struct scst_cmd *cmd);
+static void scst_unblock_cmds(struct scst_device *dev);
+static void scst_clear_reservation(struct scst_tgt_dev *tgt_dev);
+static int scst_alloc_add_tgt_dev(struct scst_session *sess,
+ struct scst_acg_dev *acg_dev, struct scst_tgt_dev **out_tgt_dev);
+static void scst_tgt_retry_timer_fn(unsigned long arg);
+
+#ifdef CONFIG_SCST_DEBUG_TM
+static void tm_dbg_init_tgt_dev(struct scst_tgt_dev *tgt_dev);
+static void tm_dbg_deinit_tgt_dev(struct scst_tgt_dev *tgt_dev);
+#else
+static inline void tm_dbg_init_tgt_dev(struct scst_tgt_dev *tgt_dev) {}
+static inline void tm_dbg_deinit_tgt_dev(struct scst_tgt_dev *tgt_dev) {}
+#endif /* CONFIG_SCST_DEBUG_TM */
+
+/**
+ * scst_alloc_sense() - allocate sense buffer for command
+ *
+ * Allocates, if necessary, sense buffer for command. Returns 0 on success
+ * and error code othrwise. Parameter "atomic" should be non-0 if the
+ * function called in atomic context.
+ */
+int scst_alloc_sense(struct scst_cmd *cmd, int atomic)
+{
+ int res = 0;
+ gfp_t gfp_mask = atomic ? GFP_ATOMIC : (GFP_KERNEL|__GFP_NOFAIL);
+
+ if (cmd->sense != NULL)
+ goto memzero;
+
+ cmd->sense = mempool_alloc(scst_sense_mempool, gfp_mask);
+ if (cmd->sense == NULL) {
+ PRINT_CRIT_ERROR("Sense memory allocation failed (op %x). "
+ "The sense data will be lost!!", cmd->cdb[0]);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ cmd->sense_buflen = SCST_SENSE_BUFFERSIZE;
+
+memzero:
+ cmd->sense_valid_len = 0;
+ memset(cmd->sense, 0, cmd->sense_buflen);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_alloc_sense);
+
+/**
+ * scst_alloc_set_sense() - allocate and fill sense buffer for command
+ *
+ * Allocates, if necessary, sense buffer for command and copies in
+ * it data from the supplied sense buffer. Returns 0 on success
+ * and error code othrwise.
+ */
+int scst_alloc_set_sense(struct scst_cmd *cmd, int atomic,
+ const uint8_t *sense, unsigned int len)
+{
+ int res;
+
+ /*
+ * We don't check here if the existing sense is valid or not, because
+ * we suppose the caller did it based on cmd->status.
+ */
+
+ res = scst_alloc_sense(cmd, atomic);
+ if (res != 0) {
+ PRINT_BUFFER("Lost sense", sense, len);
+ goto out;
+ }
+
+ cmd->sense_valid_len = len;
+ if (cmd->sense_buflen < len) {
+ PRINT_WARNING("Sense truncated (needed %d), shall you increase "
+ "SCST_SENSE_BUFFERSIZE? Op: %x", len, cmd->cdb[0]);
+ cmd->sense_valid_len = cmd->sense_buflen;
+ }
+
+ memcpy(cmd->sense, sense, cmd->sense_valid_len);
+ TRACE_BUFFER("Sense set", cmd->sense, cmd->sense_valid_len);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_alloc_set_sense);
+
+/**
+ * scst_set_cmd_error_status() - set error SCSI status
+ * @cmd: SCST command
+ * @status: SCSI status to set
+ *
+ * Description:
+ * Sets error SCSI status in the command and prepares it for returning it.
+ * Returns 0 on success, error code otherwise.
+ */
+int scst_set_cmd_error_status(struct scst_cmd *cmd, int status)
+{
+ int res = 0;
+
+ if (cmd->status != 0) {
+ TRACE_MGMT_DBG("cmd %p already has status %x set", cmd,
+ cmd->status);
+ res = -EEXIST;
+ goto out;
+ }
+
+ cmd->status = status;
+ cmd->host_status = DID_OK;
+
+ cmd->dbl_ua_orig_resp_data_len = cmd->resp_data_len;
+ cmd->dbl_ua_orig_data_direction = cmd->data_direction;
+
+ cmd->data_direction = SCST_DATA_NONE;
+ cmd->resp_data_len = 0;
+ cmd->resid_possible = 1;
+ cmd->is_send_status = 1;
+
+ cmd->completed = 1;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_set_cmd_error_status);
+
+static int scst_set_lun_not_supported_request_sense(struct scst_cmd *cmd,
+ int key, int asc, int ascq)
+{
+ int res;
+ int sense_len, len;
+ struct scatterlist *sg;
+
+ if (cmd->status != 0) {
+ TRACE_MGMT_DBG("cmd %p already has status %x set", cmd,
+ cmd->status);
+ res = -EEXIST;
+ goto out;
+ }
+
+ if ((cmd->sg != NULL) && SCST_SENSE_VALID(sg_virt(cmd->sg))) {
+ TRACE_MGMT_DBG("cmd %p already has sense set", cmd);
+ res = -EEXIST;
+ goto out;
+ }
+
+ if (cmd->sg == NULL) {
+ /*
+ * If target driver preparing data buffer using alloc_data_buf()
+ * callback, it is responsible to copy the sense to its buffer
+ * in xmit_response().
+ */
+ if (cmd->tgt_data_buf_alloced && (cmd->tgt_sg != NULL)) {
+ cmd->sg = cmd->tgt_sg;
+ cmd->sg_cnt = cmd->tgt_sg_cnt;
+ TRACE_MEM("Tgt sg used for sense for cmd %p", cmd);
+ goto go;
+ }
+
+ if (cmd->bufflen == 0)
+ cmd->bufflen = cmd->cdb[4];
+
+ cmd->sg = scst_alloc(cmd->bufflen, GFP_ATOMIC, &cmd->sg_cnt);
+ if (cmd->sg == NULL) {
+ PRINT_ERROR("Unable to alloc sg for REQUEST SENSE"
+ "(sense %x/%x/%x)", key, asc, ascq);
+ res = 1;
+ goto out;
+ }
+
+ TRACE_MEM("sg %p alloced for sense for cmd %p (cnt %d, "
+ "len %d)", cmd->sg, cmd, cmd->sg_cnt, cmd->bufflen);
+ }
+
+go:
+ sg = cmd->sg;
+ len = sg->length;
+
+ TRACE_MEM("sg %p (len %d) for sense for cmd %p", sg, len, cmd);
+
+ sense_len = scst_set_sense(sg_virt(sg), len, cmd->cdb[1] & 1,
+ key, asc, ascq);
+
+ TRACE_BUFFER("Sense set", sg_virt(sg), sense_len);
+
+ cmd->data_direction = SCST_DATA_READ;
+ scst_set_resp_data_len(cmd, sense_len);
+
+ res = 0;
+ cmd->completed = 1;
+
+out:
+ return res;
+}
+
+static int scst_set_lun_not_supported_inquiry(struct scst_cmd *cmd)
+{
+ int res;
+ uint8_t *buf;
+ struct scatterlist *sg;
+ int len;
+
+ if (cmd->status != 0) {
+ TRACE_MGMT_DBG("cmd %p already has status %x set", cmd,
+ cmd->status);
+ res = -EEXIST;
+ goto out;
+ }
+
+ if (cmd->sg == NULL) {
+ /*
+ * If target driver preparing data buffer using alloc_data_buf()
+ * callback, it is responsible to copy the sense to its buffer
+ * in xmit_response().
+ */
+ if (cmd->tgt_data_buf_alloced && (cmd->tgt_sg != NULL)) {
+ cmd->sg = cmd->tgt_sg;
+ cmd->sg_cnt = cmd->tgt_sg_cnt;
+ TRACE_MEM("Tgt used for INQUIRY for not supported "
+ "LUN for cmd %p", cmd);
+ goto go;
+ }
+
+ if (cmd->bufflen == 0)
+ cmd->bufflen = min_t(int, 36, (cmd->cdb[3] << 8) | cmd->cdb[4]);
+
+ cmd->sg = scst_alloc(cmd->bufflen, GFP_ATOMIC, &cmd->sg_cnt);
+ if (cmd->sg == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc sg for INQUIRY "
+ "for not supported LUN");
+ res = 1;
+ goto out;
+ }
+
+ TRACE_MEM("sg %p alloced for INQUIRY for not supported LUN for "
+ "cmd %p (cnt %d, len %d)", cmd->sg, cmd, cmd->sg_cnt,
+ cmd->bufflen);
+ }
+
+go:
+ sg = cmd->sg;
+ len = sg->length;
+
+ TRACE_MEM("sg %p (len %d) for INQUIRY for cmd %p", sg, len, cmd);
+
+ buf = sg_virt(sg);
+ len = min_t(int, 36, len);
+
+ memset(buf, 0, len);
+ buf[0] = 0x7F; /* Peripheral qualifier 011b, Peripheral device type 1Fh */
+
+ TRACE_BUFFER("INQUIRY for not supported LUN set", buf, len);
+
+ cmd->data_direction = SCST_DATA_READ;
+ scst_set_resp_data_len(cmd, len);
+
+ res = 0;
+ cmd->completed = 1;
+
+out:
+ return res;
+}
+
+/**
+ * scst_set_cmd_error() - set error in the command and fill the sense buffer.
+ *
+ * Sets error in the command and fill the sense buffer. Returns 0 on success,
+ * error code otherwise.
+ */
+int scst_set_cmd_error(struct scst_cmd *cmd, int key, int asc, int ascq)
+{
+ int res;
+
+ /*
+ * We need for LOGICAL UNIT NOT SUPPORTED special handling for
+ * REQUEST SENSE and INQUIRY.
+ */
+ if ((key == ILLEGAL_REQUEST) && (asc == 0x25) && (ascq == 0)) {
+ if (cmd->cdb[0] == REQUEST_SENSE)
+ res = scst_set_lun_not_supported_request_sense(cmd,
+ key, asc, ascq);
+ else if (cmd->cdb[0] == INQUIRY)
+ res = scst_set_lun_not_supported_inquiry(cmd);
+ else
+ goto do_sense;
+
+ if (res > 0)
+ goto do_sense;
+ else
+ goto out;
+ }
+
+do_sense:
+ res = scst_set_cmd_error_status(cmd, SAM_STAT_CHECK_CONDITION);
+ if (res != 0)
+ goto out;
+
+ res = scst_alloc_sense(cmd, 1);
+ if (res != 0) {
+ PRINT_ERROR("Lost sense data (key %x, asc %x, ascq %x)",
+ key, asc, ascq);
+ goto out;
+ }
+
+ cmd->sense_valid_len = scst_set_sense(cmd->sense, cmd->sense_buflen,
+ scst_get_cmd_dev_d_sense(cmd), key, asc, ascq);
+ TRACE_BUFFER("Sense set", cmd->sense, cmd->sense_valid_len);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_set_cmd_error);
+
+/**
+ * scst_set_sense() - set sense from KEY/ASC/ASCQ numbers
+ *
+ * Sets the corresponding fields in the sense buffer taking sense type
+ * into account. Returns resulting sense length.
+ */
+int scst_set_sense(uint8_t *buffer, int len, bool d_sense,
+ int key, int asc, int ascq)
+{
+ int res;
+
+ BUG_ON(len == 0);
+
+ memset(buffer, 0, len);
+
+ if (d_sense) {
+ /* Descriptor format */
+ if (len < 8) {
+ PRINT_ERROR("Length %d of sense buffer too small to "
+ "fit sense %x:%x:%x", len, key, asc, ascq);
+ }
+
+ buffer[0] = 0x72; /* Response Code */
+ if (len > 1)
+ buffer[1] = key; /* Sense Key */
+ if (len > 2)
+ buffer[2] = asc; /* ASC */
+ if (len > 3)
+ buffer[3] = ascq; /* ASCQ */
+ res = 8;
+ } else {
+ /* Fixed format */
+ if (len < 18) {
+ PRINT_ERROR("Length %d of sense buffer too small to "
+ "fit sense %x:%x:%x", len, key, asc, ascq);
+ }
+
+ buffer[0] = 0x70; /* Response Code */
+ if (len > 2)
+ buffer[2] = key; /* Sense Key */
+ if (len > 7)
+ buffer[7] = 0x0a; /* Additional Sense Length */
+ if (len > 12)
+ buffer[12] = asc; /* ASC */
+ if (len > 13)
+ buffer[13] = ascq; /* ASCQ */
+ res = 18;
+ }
+
+ TRACE_BUFFER("Sense set", buffer, res);
+ return res;
+}
+EXPORT_SYMBOL(scst_set_sense);
+
+/**
+ * scst_analyze_sense() - analyze sense
+ *
+ * Returns true if sense matches to (key, asc, ascq) and false otherwise.
+ * Valid_mask is one or several SCST_SENSE_*_VALID constants setting valid
+ * (key, asc, ascq) values.
+ */
+bool scst_analyze_sense(const uint8_t *sense, int len, unsigned int valid_mask,
+ int key, int asc, int ascq)
+{
+ bool res = false;
+
+ /* Response Code */
+ if ((sense[0] == 0x70) || (sense[0] == 0x71)) {
+ /* Fixed format */
+
+ /* Sense Key */
+ if (valid_mask & SCST_SENSE_KEY_VALID) {
+ if (len < 3)
+ goto out;
+ if (sense[2] != key)
+ goto out;
+ }
+
+ /* ASC */
+ if (valid_mask & SCST_SENSE_ASC_VALID) {
+ if (len < 13)
+ goto out;
+ if (sense[12] != asc)
+ goto out;
+ }
+
+ /* ASCQ */
+ if (valid_mask & SCST_SENSE_ASCQ_VALID) {
+ if (len < 14)
+ goto out;
+ if (sense[13] != ascq)
+ goto out;
+ }
+ } else if ((sense[0] == 0x72) || (sense[0] == 0x73)) {
+ /* Descriptor format */
+
+ /* Sense Key */
+ if (valid_mask & SCST_SENSE_KEY_VALID) {
+ if (len < 2)
+ goto out;
+ if (sense[1] != key)
+ goto out;
+ }
+
+ /* ASC */
+ if (valid_mask & SCST_SENSE_ASC_VALID) {
+ if (len < 3)
+ goto out;
+ if (sense[2] != asc)
+ goto out;
+ }
+
+ /* ASCQ */
+ if (valid_mask & SCST_SENSE_ASCQ_VALID) {
+ if (len < 4)
+ goto out;
+ if (sense[3] != ascq)
+ goto out;
+ }
+ } else
+ goto out;
+
+ res = true;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_analyze_sense);
+
+/**
+ * scst_is_ua_sense() - determine if the sense is UA sense
+ *
+ * Returns true if the sense is valid and carrying a Unit
+ * Attention or false otherwise.
+ */
+bool scst_is_ua_sense(const uint8_t *sense, int len)
+{
+ if (SCST_SENSE_VALID(sense))
+ return scst_analyze_sense(sense, len,
+ SCST_SENSE_KEY_VALID, UNIT_ATTENTION, 0, 0);
+ else
+ return false;
+}
+EXPORT_SYMBOL(scst_is_ua_sense);
+
+bool scst_is_ua_global(const uint8_t *sense, int len)
+{
+ bool res;
+
+ /* Changing it don't forget to change scst_requeue_ua() as well!! */
+
+ if (scst_analyze_sense(sense, len, SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed)))
+ res = true;
+ else
+ res = false;
+
+ return res;
+}
+
+/**
+ * scst_check_convert_sense() - check sense type and convert it if needed
+ *
+ * Checks if sense in the sense buffer, if any, is in the correct format.
+ * If not, converts it in the correct format.
+ */
+void scst_check_convert_sense(struct scst_cmd *cmd)
+{
+ bool d_sense;
+
+ if ((cmd->sense == NULL) || (cmd->status != SAM_STAT_CHECK_CONDITION))
+ goto out;
+
+ d_sense = scst_get_cmd_dev_d_sense(cmd);
+ if (d_sense && ((cmd->sense[0] == 0x70) || (cmd->sense[0] == 0x71))) {
+ TRACE_MGMT_DBG("Converting fixed sense to descriptor (cmd %p)",
+ cmd);
+ if ((cmd->sense_valid_len < 18)) {
+ PRINT_ERROR("Sense too small to convert (%d, "
+ "type: fixed)", cmd->sense_buflen);
+ goto out;
+ }
+ cmd->sense_valid_len = scst_set_sense(cmd->sense, cmd->sense_buflen,
+ d_sense, cmd->sense[2], cmd->sense[12], cmd->sense[13]);
+ } else if (!d_sense && ((cmd->sense[0] == 0x72) ||
+ (cmd->sense[0] == 0x73))) {
+ TRACE_MGMT_DBG("Converting descriptor sense to fixed (cmd %p)",
+ cmd);
+ if ((cmd->sense_buflen < 18) || (cmd->sense_valid_len < 8)) {
+ PRINT_ERROR("Sense too small to convert (%d, "
+ "type: descryptor, valid %d)",
+ cmd->sense_buflen, cmd->sense_valid_len);
+ goto out;
+ }
+ cmd->sense_valid_len = scst_set_sense(cmd->sense,
+ cmd->sense_buflen, d_sense,
+ cmd->sense[1], cmd->sense[2], cmd->sense[3]);
+ }
+
+out:
+ return;
+}
+EXPORT_SYMBOL(scst_check_convert_sense);
+
+static int scst_set_cmd_error_sense(struct scst_cmd *cmd, uint8_t *sense,
+ unsigned int len)
+{
+ int res;
+
+ res = scst_set_cmd_error_status(cmd, SAM_STAT_CHECK_CONDITION);
+ if (res != 0)
+ goto out;
+
+ res = scst_alloc_set_sense(cmd, 1, sense, len);
+
+out:
+ return res;
+}
+
+/**
+ * scst_set_busy() - set BUSY or TASK QUEUE FULL status
+ *
+ * Sets BUSY or TASK QUEUE FULL status depending on if this session has other
+ * outstanding commands or not.
+ */
+void scst_set_busy(struct scst_cmd *cmd)
+{
+ int c = atomic_read(&cmd->sess->sess_cmd_count);
+
+ if ((c <= 1) || (cmd->sess->init_phase != SCST_SESS_IPH_READY)) {
+ scst_set_cmd_error_status(cmd, SAM_STAT_BUSY);
+ TRACE(TRACE_FLOW_CONTROL, "Sending BUSY status to initiator %s "
+ "(cmds count %d, queue_type %x, sess->init_phase %d)",
+ cmd->sess->initiator_name, c,
+ cmd->queue_type, cmd->sess->init_phase);
+ } else {
+ scst_set_cmd_error_status(cmd, SAM_STAT_TASK_SET_FULL);
+ TRACE(TRACE_FLOW_CONTROL, "Sending QUEUE_FULL status to "
+ "initiator %s (cmds count %d, queue_type %x, "
+ "sess->init_phase %d)", cmd->sess->initiator_name, c,
+ cmd->queue_type, cmd->sess->init_phase);
+ }
+ return;
+}
+EXPORT_SYMBOL(scst_set_busy);
+
+/**
+ * scst_set_initial_UA() - set initial Unit Attention
+ *
+ * Sets initial Unit Attention on all devices of the session,
+ * replacing default scst_sense_reset_UA
+ */
+void scst_set_initial_UA(struct scst_session *sess, int key, int asc, int ascq)
+{
+ int i;
+
+ TRACE_MGMT_DBG("Setting for sess %p initial UA %x/%x/%x", sess, key,
+ asc, ascq);
+
+ /* Protect sess_tgt_dev_list_hash */
+ mutex_lock(&scst_mutex);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ if (!list_empty(&tgt_dev->UA_list)) {
+ struct scst_tgt_dev_UA *ua;
+
+ ua = list_entry(tgt_dev->UA_list.next,
+ typeof(*ua), UA_list_entry);
+ if (scst_analyze_sense(ua->UA_sense_buffer,
+ ua->UA_valid_sense_len,
+ SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reset_UA))) {
+ ua->UA_valid_sense_len = scst_set_sense(
+ ua->UA_sense_buffer,
+ sizeof(ua->UA_sense_buffer),
+ tgt_dev->dev->d_sense,
+ key, asc, ascq);
+ } else
+ PRINT_ERROR("%s",
+ "The first UA isn't RESET UA");
+ } else
+ PRINT_ERROR("%s", "There's no RESET UA to "
+ "replace");
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ mutex_unlock(&scst_mutex);
+ return;
+}
+EXPORT_SYMBOL(scst_set_initial_UA);
+
+static struct scst_aen *scst_alloc_aen(struct scst_session *sess,
+ uint64_t unpacked_lun)
+{
+ struct scst_aen *aen;
+
+ aen = mempool_alloc(scst_aen_mempool, GFP_KERNEL);
+ if (aen == NULL) {
+ PRINT_ERROR("AEN memory allocation failed. Corresponding "
+ "event notification will not be performed (initiator "
+ "%s)", sess->initiator_name);
+ goto out;
+ }
+ memset(aen, 0, sizeof(*aen));
+
+ aen->sess = sess;
+ scst_sess_get(sess);
+
+ aen->lun = scst_pack_lun(unpacked_lun, sess->acg->addr_method);
+
+out:
+ return aen;
+};
+
+static void scst_free_aen(struct scst_aen *aen)
+{
+
+ scst_sess_put(aen->sess);
+ mempool_free(aen, scst_aen_mempool);
+ return;
+};
+
+/* Must be called under scst_mutex */
+void scst_gen_aen_or_ua(struct scst_tgt_dev *tgt_dev,
+ int key, int asc, int ascq)
+{
+ struct scst_tgt_template *tgtt = tgt_dev->sess->tgt->tgtt;
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl;
+
+ if ((tgt_dev->sess->init_phase != SCST_SESS_IPH_READY) ||
+ (tgt_dev->sess->shut_phase != SCST_SESS_SPH_READY))
+ goto out;
+
+ if (tgtt->report_aen != NULL) {
+ struct scst_aen *aen;
+ int rc;
+
+ aen = scst_alloc_aen(tgt_dev->sess, tgt_dev->lun);
+ if (aen == NULL)
+ goto queue_ua;
+
+ aen->event_fn = SCST_AEN_SCSI;
+ aen->aen_sense_len = scst_set_sense(aen->aen_sense,
+ sizeof(aen->aen_sense), tgt_dev->dev->d_sense,
+ key, asc, ascq);
+
+ TRACE_DBG("Calling target's %s report_aen(%p)",
+ tgtt->name, aen);
+ rc = tgtt->report_aen(aen);
+ TRACE_DBG("Target's %s report_aen(%p) returned %d",
+ tgtt->name, aen, rc);
+ if (rc == SCST_AEN_RES_SUCCESS)
+ goto out;
+
+ scst_free_aen(aen);
+ }
+
+queue_ua:
+ TRACE_MGMT_DBG("AEN not supported, queuing plain UA (tgt_dev %p)",
+ tgt_dev);
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ tgt_dev->dev->d_sense, key, asc, ascq);
+ scst_check_set_UA(tgt_dev, sense_buffer, sl, 0);
+
+out:
+ return;
+}
+
+/**
+ * scst_capacity_data_changed() - notify SCST about device capacity change
+ *
+ * Notifies SCST core that dev has changed its capacity. Called under no locks.
+ */
+void scst_capacity_data_changed(struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev;
+
+ if (dev->type != TYPE_DISK) {
+ TRACE_MGMT_DBG("Device type %d isn't for CAPACITY DATA "
+ "CHANGED UA", dev->type);
+ goto out;
+ }
+
+ TRACE_MGMT_DBG("CAPACITY DATA CHANGED (dev %p)", dev);
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_gen_aen_or_ua(tgt_dev,
+ SCST_LOAD_SENSE(scst_sense_capacity_data_changed));
+ }
+
+ mutex_unlock(&scst_mutex);
+
+out:
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_capacity_data_changed);
+
+static inline bool scst_is_report_luns_changed_type(int type)
+{
+ switch (type) {
+ case TYPE_DISK:
+ case TYPE_TAPE:
+ case TYPE_PRINTER:
+ case TYPE_PROCESSOR:
+ case TYPE_WORM:
+ case TYPE_ROM:
+ case TYPE_SCANNER:
+ case TYPE_MOD:
+ case TYPE_MEDIUM_CHANGER:
+ case TYPE_RAID:
+ case TYPE_ENCLOSURE:
+ return true;
+ default:
+ return false;
+ }
+}
+
+/* scst_mutex supposed to be held */
+static void scst_queue_report_luns_changed_UA(struct scst_session *sess,
+ int flags)
+{
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+ int i;
+
+ TRACE_MGMT_DBG("Queuing REPORTED LUNS DATA CHANGED UA "
+ "(sess %p)", sess);
+
+ local_bh_disable();
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ /* Lockdep triggers here a false positive.. */
+ spin_lock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ int sl;
+
+ if (!scst_is_report_luns_changed_type(
+ tgt_dev->dev->type))
+ continue;
+
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ tgt_dev->dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed));
+
+ __scst_check_set_UA(tgt_dev, sense_buffer,
+ sl, flags | SCST_SET_UA_FLAG_GLOBAL);
+ }
+ }
+
+ for (i = TGT_DEV_HASH_SIZE-1; i >= 0; i--) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry_reverse(tgt_dev,
+ shead, sess_tgt_dev_list_entry) {
+ spin_unlock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ local_bh_enable();
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+static void scst_report_luns_changed_sess(struct scst_session *sess)
+{
+ int i;
+ struct scst_tgt_template *tgtt = sess->tgt->tgtt;
+ int d_sense = 0;
+ uint64_t lun = 0;
+
+ if ((sess->init_phase != SCST_SESS_IPH_READY) ||
+ (sess->shut_phase != SCST_SESS_SPH_READY))
+ goto out;
+
+ TRACE_DBG("REPORTED LUNS DATA CHANGED (sess %p)", sess);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if (scst_is_report_luns_changed_type(
+ tgt_dev->dev->type)) {
+ lun = tgt_dev->lun;
+ d_sense = tgt_dev->dev->d_sense;
+ goto found;
+ }
+ }
+ }
+
+found:
+ if (tgtt->report_aen != NULL) {
+ struct scst_aen *aen;
+ int rc;
+
+ aen = scst_alloc_aen(sess, lun);
+ if (aen == NULL)
+ goto queue_ua;
+
+ aen->event_fn = SCST_AEN_SCSI;
+ aen->aen_sense_len = scst_set_sense(aen->aen_sense,
+ sizeof(aen->aen_sense), d_sense,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed));
+
+ TRACE_DBG("Calling target's %s report_aen(%p)",
+ tgtt->name, aen);
+ rc = tgtt->report_aen(aen);
+ TRACE_DBG("Target's %s report_aen(%p) returned %d",
+ tgtt->name, aen, rc);
+ if (rc == SCST_AEN_RES_SUCCESS)
+ goto out;
+
+ scst_free_aen(aen);
+ }
+
+queue_ua:
+ scst_queue_report_luns_changed_UA(sess, 0);
+
+out:
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_report_luns_changed(struct scst_acg *acg)
+{
+ struct scst_session *sess;
+
+ TRACE_MGMT_DBG("REPORTED LUNS DATA CHANGED (acg %s)", acg->acg_name);
+
+ list_for_each_entry(sess, &acg->acg_sess_list, acg_sess_list_entry) {
+ scst_report_luns_changed_sess(sess);
+ }
+ return;
+}
+
+/**
+ * scst_aen_done() - AEN processing done
+ *
+ * Notifies SCST that the driver has sent the AEN and it
+ * can be freed now. Don't forget to set the delivery status, if it
+ * isn't success, using scst_set_aen_delivery_status() before calling
+ * this function.
+ */
+void scst_aen_done(struct scst_aen *aen)
+{
+
+ TRACE_MGMT_DBG("AEN %p (fn %d) done (initiator %s)", aen,
+ aen->event_fn, aen->sess->initiator_name);
+
+ if (aen->delivery_status == SCST_AEN_RES_SUCCESS)
+ goto out_free;
+
+ if (aen->event_fn != SCST_AEN_SCSI)
+ goto out_free;
+
+ TRACE_MGMT_DBG("Delivery of SCSI AEN failed (initiator %s)",
+ aen->sess->initiator_name);
+
+ if (scst_analyze_sense(aen->aen_sense, aen->aen_sense_len,
+ SCST_SENSE_ALL_VALID, SCST_LOAD_SENSE(
+ scst_sense_reported_luns_data_changed))) {
+ mutex_lock(&scst_mutex);
+ scst_queue_report_luns_changed_UA(aen->sess,
+ SCST_SET_UA_FLAG_AT_HEAD);
+ mutex_unlock(&scst_mutex);
+ } else {
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+ uint64_t lun;
+
+ lun = scst_unpack_lun((uint8_t *)&aen->lun, sizeof(aen->lun));
+
+ mutex_lock(&scst_mutex);
+
+ /* tgt_dev might get dead, so we need to reseek it */
+ shead = &aen->sess->sess_tgt_dev_list_hash[HASH_VAL(lun)];
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == lun) {
+ TRACE_MGMT_DBG("Requeuing failed AEN UA for "
+ "tgt_dev %p", tgt_dev);
+ scst_check_set_UA(tgt_dev, aen->aen_sense,
+ aen->aen_sense_len,
+ SCST_SET_UA_FLAG_AT_HEAD);
+ break;
+ }
+ }
+
+ mutex_unlock(&scst_mutex);
+ }
+
+out_free:
+ scst_free_aen(aen);
+ return;
+}
+EXPORT_SYMBOL(scst_aen_done);
+
+void scst_requeue_ua(struct scst_cmd *cmd)
+{
+
+ if (scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed))) {
+ TRACE_MGMT_DBG("Requeuing REPORTED LUNS DATA CHANGED UA "
+ "for delivery failed cmd %p", cmd);
+ mutex_lock(&scst_mutex);
+ scst_queue_report_luns_changed_UA(cmd->sess,
+ SCST_SET_UA_FLAG_AT_HEAD);
+ mutex_unlock(&scst_mutex);
+ } else {
+ TRACE_MGMT_DBG("Requeuing UA for delivery failed cmd %p", cmd);
+ scst_check_set_UA(cmd->tgt_dev, cmd->sense,
+ cmd->sense_valid_len, SCST_SET_UA_FLAG_AT_HEAD);
+ }
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+static void scst_check_reassign_sess(struct scst_session *sess)
+{
+ struct scst_acg *acg, *old_acg;
+ struct scst_acg_dev *acg_dev;
+ int i, rc;
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+ bool luns_changed = false;
+ bool add_failed, something_freed, not_needed_freed = false;
+
+ TRACE_MGMT_DBG("Checking reassignment for sess %p (initiator %s)",
+ sess, sess->initiator_name);
+
+ acg = scst_find_acg(sess);
+ if (acg == sess->acg) {
+ TRACE_MGMT_DBG("No reassignment for sess %p", sess);
+ goto out;
+ }
+
+ TRACE_MGMT_DBG("sess %p will be reassigned from acg %s to acg %s",
+ sess, sess->acg->acg_name, acg->acg_name);
+
+ old_acg = sess->acg;
+ sess->acg = NULL; /* to catch implicit dependencies earlier */
+
+retry_add:
+ add_failed = false;
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ unsigned int inq_changed_ua_needed = 0;
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if ((tgt_dev->dev == acg_dev->dev) &&
+ (tgt_dev->lun == acg_dev->lun) &&
+ (tgt_dev->acg_dev->rd_only == acg_dev->rd_only)) {
+ TRACE_MGMT_DBG("sess %p: tgt_dev %p for "
+ "LUN %lld stays the same",
+ sess, tgt_dev,
+ (unsigned long long)tgt_dev->lun);
+ tgt_dev->acg_dev = acg_dev;
+ goto next;
+ } else if (tgt_dev->lun == acg_dev->lun)
+ inq_changed_ua_needed = 1;
+ }
+ }
+
+ luns_changed = true;
+
+ TRACE_MGMT_DBG("sess %p: Allocing new tgt_dev for LUN %lld",
+ sess, (unsigned long long)acg_dev->lun);
+
+ rc = scst_alloc_add_tgt_dev(sess, acg_dev, &tgt_dev);
+ if (rc == -EPERM)
+ continue;
+ else if (rc != 0) {
+ add_failed = true;
+ break;
+ }
+
+ tgt_dev->inq_changed_ua_needed = inq_changed_ua_needed ||
+ not_needed_freed;
+next:
+ continue;
+ }
+
+ something_freed = false;
+ not_needed_freed = true;
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct scst_tgt_dev *t;
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry_safe(tgt_dev, t, shead,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->acg_dev->acg != acg) {
+ TRACE_MGMT_DBG("sess %p: Deleting not used "
+ "tgt_dev %p for LUN %lld",
+ sess, tgt_dev,
+ (unsigned long long)tgt_dev->lun);
+ luns_changed = true;
+ something_freed = true;
+ scst_free_tgt_dev(tgt_dev);
+ }
+ }
+ }
+
+ if (add_failed && something_freed) {
+ TRACE_MGMT_DBG("sess %p: Retrying adding new tgt_devs", sess);
+ goto retry_add;
+ }
+
+ sess->acg = acg;
+
+ TRACE_DBG("Moving sess %p from acg %s to acg %s", sess,
+ old_acg->acg_name, acg->acg_name);
+ list_move_tail(&sess->acg_sess_list_entry, &acg->acg_sess_list);
+
+ scst_recreate_sess_luns_link(sess);
+ /* Ignore possible error, since we can't do anything on it */
+
+ if (luns_changed) {
+ scst_report_luns_changed_sess(sess);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->inq_changed_ua_needed) {
+ TRACE_MGMT_DBG("sess %p: Setting "
+ "INQUIRY DATA HAS CHANGED UA "
+ "(tgt_dev %p)", sess, tgt_dev);
+
+ tgt_dev->inq_changed_ua_needed = 0;
+
+ scst_gen_aen_or_ua(tgt_dev,
+ SCST_LOAD_SENSE(scst_sense_inquery_data_changed));
+ }
+ }
+ }
+ }
+
+out:
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_check_reassign_sessions(void)
+{
+ struct scst_tgt_template *tgtt;
+
+ list_for_each_entry(tgtt, &scst_template_list, scst_template_list_entry) {
+ struct scst_tgt *tgt;
+ list_for_each_entry(tgt, &tgtt->tgt_list, tgt_list_entry) {
+ struct scst_session *sess;
+ list_for_each_entry(sess, &tgt->sess_list,
+ sess_list_entry) {
+ scst_check_reassign_sess(sess);
+ }
+ }
+ }
+ return;
+}
+
+static int scst_get_cmd_abnormal_done_state(const struct scst_cmd *cmd)
+{
+ int res;
+
+ switch (cmd->state) {
+ case SCST_CMD_STATE_INIT_WAIT:
+ case SCST_CMD_STATE_INIT:
+ case SCST_CMD_STATE_PRE_PARSE:
+ case SCST_CMD_STATE_DEV_PARSE:
+ if (cmd->preprocessing_only) {
+ res = SCST_CMD_STATE_PREPROCESSING_DONE;
+ break;
+ } /* else go through */
+ case SCST_CMD_STATE_DEV_DONE:
+ if (cmd->internal)
+ res = SCST_CMD_STATE_FINISHED_INTERNAL;
+ else
+ res = SCST_CMD_STATE_PRE_XMIT_RESP;
+ break;
+
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ case SCST_CMD_STATE_MODE_SELECT_CHECKS:
+ res = SCST_CMD_STATE_DEV_DONE;
+ break;
+
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ res = SCST_CMD_STATE_XMIT_RESP;
+ break;
+
+ case SCST_CMD_STATE_PREPROCESSING_DONE:
+ case SCST_CMD_STATE_PREPROCESSING_DONE_CALLED:
+ if (cmd->tgt_dev == NULL)
+ res = SCST_CMD_STATE_PRE_XMIT_RESP;
+ else
+ res = SCST_CMD_STATE_PRE_DEV_DONE;
+ break;
+
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ if (cmd->preprocessing_only) {
+ res = SCST_CMD_STATE_PREPROCESSING_DONE;
+ break;
+ } /* else go through */
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_DATA_WAIT:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXECUTING:
+ res = SCST_CMD_STATE_PRE_DEV_DONE;
+ break;
+
+ default:
+ PRINT_CRIT_ERROR("Wrong cmd state %d (cmd %p, op %x)",
+ cmd->state, cmd, cmd->cdb[0]);
+ BUG();
+ /* Invalid state to supress compiler's warning */
+ res = SCST_CMD_STATE_LAST_ACTIVE;
+ }
+ return res;
+}
+
+/**
+ * scst_set_cmd_abnormal_done_state() - set command's next abnormal done state
+ *
+ * Sets state of the SCSI target state machine to abnormally complete command
+ * ASAP.
+ *
+ * Returns the new state.
+ */
+int scst_set_cmd_abnormal_done_state(struct scst_cmd *cmd)
+{
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ switch (cmd->state) {
+ case SCST_CMD_STATE_XMIT_RESP:
+ case SCST_CMD_STATE_FINISHED:
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+ case SCST_CMD_STATE_XMIT_WAIT:
+ PRINT_CRIT_ERROR("Wrong cmd state %d (cmd %p, op %x)",
+ cmd->state, cmd, cmd->cdb[0]);
+ BUG();
+ }
+#endif
+
+ cmd->state = scst_get_cmd_abnormal_done_state(cmd);
+
+ switch (cmd->state) {
+ case SCST_CMD_STATE_INIT_WAIT:
+ case SCST_CMD_STATE_INIT:
+ case SCST_CMD_STATE_PRE_PARSE:
+ case SCST_CMD_STATE_PREPROCESSING_DONE:
+ case SCST_CMD_STATE_PREPROCESSING_DONE_CALLED:
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_DATA_WAIT:
+ cmd->write_len = 0;
+ cmd->resid_possible = 1;
+ break;
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXECUTING:
+ case SCST_CMD_STATE_DEV_PARSE:
+ case SCST_CMD_STATE_DEV_DONE:
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ case SCST_CMD_STATE_MODE_SELECT_CHECKS:
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ break;
+ default:
+ PRINT_CRIT_ERROR("Wrong cmd state %d (cmd %p, op %x)",
+ cmd->state, cmd, cmd->cdb[0]);
+ BUG();
+ break;
+ }
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (((cmd->state != SCST_CMD_STATE_PRE_XMIT_RESP) &&
+ (cmd->state != SCST_CMD_STATE_PREPROCESSING_DONE)) &&
+ (cmd->tgt_dev == NULL) && !cmd->internal) {
+ PRINT_CRIT_ERROR("Wrong not inited cmd state %d (cmd %p, "
+ "op %x)", cmd->state, cmd, cmd->cdb[0]);
+ BUG();
+ }
+#endif
+ return cmd->state;
+}
+EXPORT_SYMBOL_GPL(scst_set_cmd_abnormal_done_state);
+
+void scst_zero_write_rest(struct scst_cmd *cmd)
+{
+ int len, offs = 0;
+ uint8_t *buf;
+
+ len = scst_get_sg_buf_first(cmd, &buf, *cmd->write_sg,
+ *cmd->write_sg_cnt);
+ while (len > 0) {
+ int cur_offs;
+
+ if (offs + len <= cmd->write_len)
+ goto next;
+ else if (offs >= cmd->write_len)
+ cur_offs = 0;
+ else
+ cur_offs = cmd->write_len - offs;
+
+ memset(&buf[cur_offs], 0, len - cur_offs);
+
+next:
+ offs += len;
+ scst_put_sg_buf(cmd, buf, *cmd->write_sg, *cmd->write_sg_cnt);
+ len = scst_get_sg_buf_next(cmd, &buf, *cmd->write_sg,
+ *cmd->write_sg_cnt);
+ }
+ return;
+}
+
+static void scst_adjust_sg(struct scst_cmd *cmd, struct scatterlist *sg,
+ int *sg_cnt, int adjust_len)
+{
+ int i, j, l;
+
+ l = 0;
+ for (i = 0, j = 0; i < *sg_cnt; i++, j++) {
+ TRACE_DBG("i %d, j %d, sg_cnt %d, sg %p, page_link %lx", i, j,
+ *sg_cnt, sg, sg[j].page_link);
+ if (unlikely(sg_is_chain(&sg[j]))) {
+ sg = sg_chain_ptr(&sg[j]);
+ j = 0;
+ }
+ l += sg[j].length;
+ if (l >= adjust_len) {
+ int left = adjust_len - (l - sg[j].length);
+#ifdef CONFIG_SCST_DEBUG
+ TRACE(TRACE_SG_OP|TRACE_MEMORY, "cmd %p (tag %llu), "
+ "sg %p, sg_cnt %d, adjust_len %d, i %d, j %d, "
+ "sg[j].length %d, left %d",
+ cmd, (long long unsigned int)cmd->tag,
+ sg, *sg_cnt, adjust_len, i, j,
+ sg[j].length, left);
+#endif
+ cmd->orig_sg = sg;
+ cmd->p_orig_sg_cnt = sg_cnt;
+ cmd->orig_sg_cnt = *sg_cnt;
+ cmd->orig_sg_entry = j;
+ cmd->orig_entry_len = sg[j].length;
+ *sg_cnt = (left > 0) ? j+1 : j;
+ sg[j].length = left;
+ cmd->sg_buff_modified = 1;
+ break;
+ }
+ }
+ return;
+}
+
+/**
+ * scst_restore_sg_buff() - restores modified sg buffer
+ *
+ * Restores modified sg buffer in the original state.
+ */
+void scst_restore_sg_buff(struct scst_cmd *cmd)
+{
+ TRACE_MEM("cmd %p, sg %p, orig_sg_entry %d, "
+ "orig_entry_len %d, orig_sg_cnt %d", cmd, cmd->orig_sg,
+ cmd->orig_sg_entry, cmd->orig_entry_len,
+ cmd->orig_sg_cnt);
+ cmd->orig_sg[cmd->orig_sg_entry].length = cmd->orig_entry_len;
+ *cmd->p_orig_sg_cnt = cmd->orig_sg_cnt;
+ cmd->sg_buff_modified = 0;
+}
+EXPORT_SYMBOL(scst_restore_sg_buff);
+
+/**
+ * scst_set_resp_data_len() - set response data length
+ *
+ * Sets response data length for cmd and truncates its SG vector accordingly.
+ *
+ * The cmd->resp_data_len must not be set directly, it must be set only
+ * using this function. Value of resp_data_len must be <= cmd->bufflen.
+ */
+void scst_set_resp_data_len(struct scst_cmd *cmd, int resp_data_len)
+{
+
+ scst_check_restore_sg_buff(cmd);
+ cmd->resp_data_len = resp_data_len;
+
+ if (resp_data_len == cmd->bufflen)
+ goto out;
+
+ scst_adjust_sg(cmd, cmd->sg, &cmd->sg_cnt, resp_data_len);
+
+ cmd->resid_possible = 1;
+
+out:
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_set_resp_data_len);
+
+void scst_limit_sg_write_len(struct scst_cmd *cmd)
+{
+
+ TRACE_MEM("Limiting sg write len to %d (cmd %p, sg %p, sg_cnt %d)",
+ cmd->write_len, cmd, *cmd->write_sg, *cmd->write_sg_cnt);
+
+ scst_check_restore_sg_buff(cmd);
+ scst_adjust_sg(cmd, *cmd->write_sg, cmd->write_sg_cnt, cmd->write_len);
+ return;
+}
+
+void scst_adjust_resp_data_len(struct scst_cmd *cmd)
+{
+
+ EXTRACHECKS_BUG_ON(!cmd->expected_values_set);
+
+ cmd->adjusted_resp_data_len = min(cmd->resp_data_len,
+ cmd->expected_transfer_len);
+
+ if (cmd->adjusted_resp_data_len != cmd->resp_data_len) {
+ TRACE_MEM("Abjusting resp_data_len to %d (cmd %p, sg %p, "
+ "sg_cnt %d)", cmd->adjusted_resp_data_len, cmd, cmd->sg,
+ cmd->sg_cnt);
+ scst_check_restore_sg_buff(cmd);
+ scst_adjust_sg(cmd, cmd->sg, &cmd->sg_cnt,
+ cmd->adjusted_resp_data_len);
+ }
+ return;
+}
+
+/**
+ * scst_cmd_set_write_not_received_data_len() - sets cmd's not received len
+ *
+ * Sets cmd's not received data length. Also automatically sets resid_possible.
+ */
+void scst_cmd_set_write_not_received_data_len(struct scst_cmd *cmd,
+ int not_received)
+{
+
+ BUG_ON(!cmd->expected_values_set);
+
+ cmd->resid_possible = 1;
+
+ if ((cmd->expected_data_direction & SCST_DATA_READ) &&
+ (cmd->expected_data_direction & SCST_DATA_WRITE)) {
+ cmd->write_len = cmd->expected_out_transfer_len - not_received;
+ if (cmd->write_len == cmd->out_bufflen)
+ goto out;
+ } else if (cmd->expected_data_direction & SCST_DATA_WRITE) {
+ cmd->write_len = cmd->expected_transfer_len - not_received;
+ if (cmd->write_len == cmd->bufflen)
+ goto out;
+ }
+
+ /*
+ * Write len now can be bigger cmd->(out_)bufflen, but that's OK,
+ * because it will be used to only calculate write residuals.
+ */
+
+ TRACE_DBG("cmd %p, not_received %d, write_len %d", cmd, not_received,
+ cmd->write_len);
+
+ if (cmd->data_direction & SCST_DATA_WRITE)
+ scst_limit_sg_write_len(cmd);
+
+out:
+ return;
+}
+EXPORT_SYMBOL(scst_cmd_set_write_not_received_data_len);
+
+/**
+ * __scst_get_resid() - returns residuals for cmd
+ *
+ * Returns residuals for command. Must not be called directly, use
+ * scst_get_resid() instead.
+ */
+bool __scst_get_resid(struct scst_cmd *cmd, int *resid, int *bidi_out_resid)
+{
+
+ *resid = 0;
+ if (bidi_out_resid != NULL)
+ *bidi_out_resid = 0;
+
+ BUG_ON(!cmd->expected_values_set);
+
+ if (cmd->expected_data_direction & SCST_DATA_READ) {
+ *resid = cmd->expected_transfer_len - cmd->resp_data_len;
+ if ((cmd->expected_data_direction & SCST_DATA_WRITE) && bidi_out_resid) {
+ if (cmd->write_len < cmd->expected_out_transfer_len)
+ *bidi_out_resid = cmd->expected_out_transfer_len -
+ cmd->write_len;
+ else
+ *bidi_out_resid = cmd->write_len - cmd->out_bufflen;
+ }
+ } else if (cmd->expected_data_direction & SCST_DATA_WRITE) {
+ if (cmd->write_len < cmd->expected_transfer_len)
+ *resid = cmd->expected_transfer_len - cmd->write_len;
+ else
+ *resid = cmd->write_len - cmd->bufflen;
+ }
+
+ TRACE_DBG("cmd %p, resid %d, bidi_out_resid %d (resp_data_len %d, "
+ "expected_data_direction %d, write_len %d, bufflen %d)", cmd,
+ *resid, bidi_out_resid ? *bidi_out_resid : 0, cmd->resp_data_len,
+ cmd->expected_data_direction, cmd->write_len, cmd->bufflen);
+ return true;
+}
+EXPORT_SYMBOL(__scst_get_resid);
+
+/* No locks */
+int scst_queue_retry_cmd(struct scst_cmd *cmd, int finished_cmds)
+{
+ struct scst_tgt *tgt = cmd->tgt;
+ int res = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&tgt->tgt_lock, flags);
+ tgt->retry_cmds++;
+ /*
+ * Memory barrier is needed here, because we need the exact order
+ * between the read and write between retry_cmds and finished_cmds to
+ * not miss the case when a command finished while we queuing it for
+ * retry after the finished_cmds check.
+ */
+ smp_mb();
+ TRACE_RETRY("TGT QUEUE FULL: incrementing retry_cmds %d",
+ tgt->retry_cmds);
+ if (finished_cmds != atomic_read(&tgt->finished_cmds)) {
+ /* At least one cmd finished, so try again */
+ tgt->retry_cmds--;
+ TRACE_RETRY("Some command(s) finished, direct retry "
+ "(finished_cmds=%d, tgt->finished_cmds=%d, "
+ "retry_cmds=%d)", finished_cmds,
+ atomic_read(&tgt->finished_cmds), tgt->retry_cmds);
+ res = -1;
+ goto out_unlock_tgt;
+ }
+
+ TRACE_RETRY("Adding cmd %p to retry cmd list", cmd);
+ list_add_tail(&cmd->cmd_list_entry, &tgt->retry_cmd_list);
+
+ if (!tgt->retry_timer_active) {
+ tgt->retry_timer.expires = jiffies + SCST_TGT_RETRY_TIMEOUT;
+ add_timer(&tgt->retry_timer);
+ tgt->retry_timer_active = 1;
+ }
+
+out_unlock_tgt:
+ spin_unlock_irqrestore(&tgt->tgt_lock, flags);
+ return res;
+}
+
+/**
+ * scst_update_hw_pending_start() - update commands pending start
+ *
+ * Updates the command's hw_pending_start as if it's just started hw pending.
+ * Target drivers should call it if they received reply from this pending
+ * command, but SCST core won't see it.
+ */
+void scst_update_hw_pending_start(struct scst_cmd *cmd)
+{
+ unsigned long flags;
+
+ /* To sync with scst_check_hw_pending_cmd() */
+ spin_lock_irqsave(&cmd->sess->sess_list_lock, flags);
+ cmd->hw_pending_start = jiffies;
+ TRACE_MGMT_DBG("Updated hw_pending_start to %ld (cmd %p)",
+ cmd->hw_pending_start, cmd);
+ spin_unlock_irqrestore(&cmd->sess->sess_list_lock, flags);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_update_hw_pending_start);
+
+/*
+ * Supposed to be called under sess_list_lock, but can release/reaquire it.
+ * Returns 0 to continue, >0 to restart, <0 to break.
+ */
+static int scst_check_hw_pending_cmd(struct scst_cmd *cmd,
+ unsigned long cur_time, unsigned long max_time,
+ struct scst_session *sess, unsigned long *flags,
+ struct scst_tgt_template *tgtt)
+{
+ int res = -1; /* break */
+
+ TRACE_DBG("cmd %p, hw_pending %d, proc time %ld, "
+ "pending time %ld", cmd, cmd->cmd_hw_pending,
+ (long)(cur_time - cmd->start_time) / HZ,
+ (long)(cur_time - cmd->hw_pending_start) / HZ);
+
+ if (time_before(cur_time, cmd->start_time + max_time)) {
+ /* Cmds are ordered, so no need to check more */
+ goto out;
+ }
+
+ if (!cmd->cmd_hw_pending) {
+ res = 0; /* continue */
+ goto out;
+ }
+
+ if (time_before(cur_time, cmd->hw_pending_start + max_time)) {
+ res = 0; /* continue */
+ goto out;
+ }
+
+ TRACE_MGMT_DBG("Cmd %p HW pending for too long %ld (state %x)",
+ cmd, (cur_time - cmd->hw_pending_start) / HZ,
+ cmd->state);
+
+ cmd->cmd_hw_pending = 0;
+
+ spin_unlock_irqrestore(&sess->sess_list_lock, *flags);
+ tgtt->on_hw_pending_cmd_timeout(cmd);
+ spin_lock_irqsave(&sess->sess_list_lock, *flags);
+
+ res = 1; /* restart */
+
+out:
+ return res;
+}
+
+static void scst_hw_pending_work_fn(struct delayed_work *work)
+{
+ struct scst_session *sess = container_of(work, struct scst_session,
+ hw_pending_work);
+ struct scst_tgt_template *tgtt = sess->tgt->tgtt;
+ struct scst_cmd *cmd;
+ unsigned long cur_time = jiffies;
+ unsigned long flags;
+ unsigned long max_time = tgtt->max_hw_pending_time * HZ;
+
+ TRACE_DBG("HW pending work (sess %p, max time %ld)", sess, max_time/HZ);
+
+ clear_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags);
+
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+
+restart:
+ list_for_each_entry(cmd, &sess->sess_cmd_list, sess_cmd_list_entry) {
+ int rc;
+
+ rc = scst_check_hw_pending_cmd(cmd, cur_time, max_time, sess,
+ &flags, tgtt);
+ if (rc < 0)
+ break;
+ else if (rc == 0)
+ continue;
+ else
+ goto restart;
+ }
+
+ if (!list_empty(&sess->sess_cmd_list)) {
+ /*
+ * For stuck cmds if there is no activity we might need to have
+ * one more run to release them, so reschedule once again.
+ */
+ TRACE_DBG("Sched HW pending work for sess %p (max time %d)",
+ sess, tgtt->max_hw_pending_time);
+ set_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags);
+ schedule_delayed_work(&sess->hw_pending_work,
+ tgtt->max_hw_pending_time * HZ);
+ }
+
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ return;
+}
+
+static bool __scst_is_relative_target_port_id_unique(uint16_t id,
+ const struct scst_tgt *t)
+{
+ bool res = true;
+ struct scst_tgt_template *tgtt;
+
+ list_for_each_entry(tgtt, &scst_template_list,
+ scst_template_list_entry) {
+ struct scst_tgt *tgt;
+ list_for_each_entry(tgt, &tgtt->tgt_list, tgt_list_entry) {
+ if (tgt == t)
+ continue;
+ if ((tgt->tgtt->is_target_enabled != NULL) &&
+ !tgt->tgtt->is_target_enabled(tgt))
+ continue;
+ if (id == tgt->rel_tgt_id) {
+ res = false;
+ break;
+ }
+ }
+ }
+ return res;
+}
+
+/* scst_mutex supposed to be locked */
+bool scst_is_relative_target_port_id_unique(uint16_t id,
+ const struct scst_tgt *t)
+{
+ bool res;
+
+ mutex_lock(&scst_mutex);
+ res = __scst_is_relative_target_port_id_unique(id, t);
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+int gen_relative_target_port_id(uint16_t *id)
+{
+ int res = -EOVERFLOW;
+ static unsigned long rti = SCST_MIN_REL_TGT_ID, rti_prev;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ rti_prev = rti;
+ do {
+ if (__scst_is_relative_target_port_id_unique(rti, NULL)) {
+ *id = (uint16_t)rti++;
+ res = 0;
+ goto out_unlock;
+ }
+ rti++;
+ if (rti > SCST_MAX_REL_TGT_ID)
+ rti = SCST_MIN_REL_TGT_ID;
+ } while (rti != rti_prev);
+
+ PRINT_ERROR("%s", "Unable to create unique relative target port id");
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out:
+ return res;
+}
+
+/* No locks */
+int scst_alloc_tgt(struct scst_tgt_template *tgtt, struct scst_tgt **tgt)
+{
+ struct scst_tgt *t;
+ int res = 0;
+
+ t = kzalloc(sizeof(*t), GFP_KERNEL);
+ if (t == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of tgt failed");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ INIT_LIST_HEAD(&t->sess_list);
+ init_waitqueue_head(&t->unreg_waitQ);
+ t->tgtt = tgtt;
+ t->sg_tablesize = tgtt->sg_tablesize;
+ spin_lock_init(&t->tgt_lock);
+ INIT_LIST_HEAD(&t->retry_cmd_list);
+ atomic_set(&t->finished_cmds, 0);
+ init_timer(&t->retry_timer);
+ t->retry_timer.data = (unsigned long)t;
+ t->retry_timer.function = scst_tgt_retry_timer_fn;
+
+ INIT_LIST_HEAD(&t->tgt_acg_list);
+
+ *tgt = t;
+
+out:
+ return res;
+}
+
+/* No locks */
+void scst_free_tgt(struct scst_tgt *tgt)
+{
+
+ kfree(tgt->tgt_name);
+
+ kfree(tgt);
+ return;
+}
+
+/* Called under scst_mutex and suspended activity */
+int scst_alloc_device(gfp_t gfp_mask, struct scst_device **out_dev)
+{
+ struct scst_device *dev;
+ int res = 0;
+
+ dev = kzalloc(sizeof(*dev), gfp_mask);
+ if (dev == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Allocation of scst_device failed");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ dev->handler = &scst_null_devtype;
+ atomic_set(&dev->dev_cmd_count, 0);
+ atomic_set(&dev->write_cmd_count, 0);
+ scst_init_mem_lim(&dev->dev_mem_lim);
+ spin_lock_init(&dev->dev_lock);
+ atomic_set(&dev->on_dev_count, 0);
+ INIT_LIST_HEAD(&dev->blocked_cmd_list);
+ INIT_LIST_HEAD(&dev->dev_tgt_dev_list);
+ INIT_LIST_HEAD(&dev->dev_acg_dev_list);
+ init_waitqueue_head(&dev->on_dev_waitQ);
+ dev->dev_double_ua_possible = 1;
+ dev->queue_alg = SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER;
+
+ mutex_init(&dev->dev_pr_mutex);
+ atomic_set(&dev->pr_readers_count, 0);
+ dev->pr_generation = 0;
+ dev->pr_is_set = 0;
+ dev->pr_holder = NULL;
+ dev->pr_scope = SCOPE_LU;
+ dev->pr_type = TYPE_UNSPECIFIED;
+ INIT_LIST_HEAD(&dev->dev_registrants_list);
+
+ scst_init_threads(&dev->dev_cmd_threads);
+
+ *out_dev = dev;
+
+out:
+ return res;
+}
+
+void scst_free_device(struct scst_device *dev)
+{
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (!list_empty(&dev->dev_tgt_dev_list) ||
+ !list_empty(&dev->dev_acg_dev_list)) {
+ PRINT_CRIT_ERROR("%s: dev_tgt_dev_list or dev_acg_dev_list "
+ "is not empty!", __func__);
+ BUG();
+ }
+#endif
+
+ scst_deinit_threads(&dev->dev_cmd_threads);
+
+ kfree(dev->virt_name);
+ kfree(dev);
+ return;
+}
+
+/**
+ * scst_init_mem_lim - initialize memory limits structure
+ *
+ * Initializes memory limits structure mem_lim according to
+ * the current system configuration. This structure should be latter used
+ * to track and limit allocated by one or more SGV pools memory.
+ */
+void scst_init_mem_lim(struct scst_mem_lim *mem_lim)
+{
+ atomic_set(&mem_lim->alloced_pages, 0);
+ mem_lim->max_allowed_pages =
+ ((uint64_t)scst_max_dev_cmd_mem << 10) >> (PAGE_SHIFT - 10);
+}
+EXPORT_SYMBOL_GPL(scst_init_mem_lim);
+
+static struct scst_acg_dev *scst_alloc_acg_dev(struct scst_acg *acg,
+ struct scst_device *dev, uint64_t lun)
+{
+ struct scst_acg_dev *res;
+
+ res = kmem_cache_zalloc(scst_acgd_cachep, GFP_KERNEL);
+ if (res == NULL) {
+ TRACE(TRACE_OUT_OF_MEM,
+ "%s", "Allocation of scst_acg_dev failed");
+ goto out;
+ }
+
+ res->dev = dev;
+ res->acg = acg;
+ res->lun = lun;
+
+out:
+ return res;
+}
+
+/*
+ * The activity supposed to be suspended and scst_mutex held or the
+ * corresponding target supposed to be stopped.
+ */
+static void scst_del_free_acg_dev(struct scst_acg_dev *acg_dev, bool del_sysfs)
+{
+
+ TRACE_DBG("Removing acg_dev %p from acg_dev_list and dev_acg_dev_list",
+ acg_dev);
+ list_del(&acg_dev->acg_dev_list_entry);
+ list_del(&acg_dev->dev_acg_dev_list_entry);
+
+ if (del_sysfs)
+ scst_acg_dev_sysfs_del(acg_dev);
+
+ kmem_cache_free(scst_acgd_cachep, acg_dev);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_acg_add_lun(struct scst_acg *acg, struct kobject *parent,
+ struct scst_device *dev, uint64_t lun, int read_only,
+ bool gen_scst_report_luns_changed, struct scst_acg_dev **out_acg_dev)
+{
+ int res = 0;
+ struct scst_acg_dev *acg_dev;
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_session *sess;
+ LIST_HEAD(tmp_tgt_dev_list);
+ bool del_sysfs = true;
+
+ INIT_LIST_HEAD(&tmp_tgt_dev_list);
+
+ acg_dev = scst_alloc_acg_dev(acg, dev, lun);
+ if (acg_dev == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ acg_dev->rd_only = read_only;
+
+ TRACE_DBG("Adding acg_dev %p to acg_dev_list and dev_acg_dev_list",
+ acg_dev);
+ list_add_tail(&acg_dev->acg_dev_list_entry, &acg->acg_dev_list);
+ list_add_tail(&acg_dev->dev_acg_dev_list_entry, &dev->dev_acg_dev_list);
+
+ list_for_each_entry(sess, &acg->acg_sess_list, acg_sess_list_entry) {
+ res = scst_alloc_add_tgt_dev(sess, acg_dev, &tgt_dev);
+ if (res == -EPERM)
+ continue;
+ else if (res != 0)
+ goto out_free;
+
+ list_add_tail(&tgt_dev->extra_tgt_dev_list_entry,
+ &tmp_tgt_dev_list);
+ }
+
+ res = scst_acg_dev_sysfs_create(acg_dev, parent);
+ if (res != 0) {
+ del_sysfs = false;
+ goto out_free;
+ }
+
+ if (gen_scst_report_luns_changed)
+ scst_report_luns_changed(acg);
+
+ PRINT_INFO("Added device %s to group %s (LUN %lld, "
+ "rd_only %d)", dev->virt_name, acg->acg_name,
+ (long long unsigned int)lun, read_only);
+
+ if (out_acg_dev != NULL)
+ *out_acg_dev = acg_dev;
+
+out:
+ return res;
+
+out_free:
+ list_for_each_entry(tgt_dev, &tmp_tgt_dev_list,
+ extra_tgt_dev_list_entry) {
+ scst_free_tgt_dev(tgt_dev);
+ }
+ scst_del_free_acg_dev(acg_dev, del_sysfs);
+ goto out;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_acg_del_lun(struct scst_acg *acg, uint64_t lun,
+ bool gen_scst_report_luns_changed)
+{
+ int res = 0;
+ struct scst_acg_dev *acg_dev = NULL, *a;
+ struct scst_tgt_dev *tgt_dev, *tt;
+
+ list_for_each_entry(a, &acg->acg_dev_list, acg_dev_list_entry) {
+ if (a->lun == lun) {
+ acg_dev = a;
+ break;
+ }
+ }
+ if (acg_dev == NULL) {
+ PRINT_ERROR("Device is not found in group %s", acg->acg_name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ list_for_each_entry_safe(tgt_dev, tt, &acg_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (tgt_dev->acg_dev == acg_dev)
+ scst_free_tgt_dev(tgt_dev);
+ }
+
+ scst_del_free_acg_dev(acg_dev, true);
+
+ if (gen_scst_report_luns_changed)
+ scst_report_luns_changed(acg);
+
+ PRINT_INFO("Removed LUN %lld from group %s", (unsigned long long)lun,
+ acg->acg_name);
+
+out:
+ return res;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+struct scst_acg *scst_alloc_add_acg(struct scst_tgt *tgt,
+ const char *acg_name, bool tgt_acg)
+{
+ struct scst_acg *acg;
+
+ acg = kzalloc(sizeof(*acg), GFP_KERNEL);
+ if (acg == NULL) {
+ PRINT_ERROR("%s", "Allocation of acg failed");
+ goto out;
+ }
+
+ acg->tgt = tgt;
+ INIT_LIST_HEAD(&acg->acg_dev_list);
+ INIT_LIST_HEAD(&acg->acg_sess_list);
+ INIT_LIST_HEAD(&acg->acn_list);
+ acg->acg_name = kstrdup(acg_name, GFP_KERNEL);
+ if (acg->acg_name == NULL) {
+ PRINT_ERROR("%s", "Allocation of acg_name failed");
+ goto out_free;
+ }
+
+ acg->addr_method = SCST_LUN_ADDR_METHOD_PERIPHERAL;
+
+ if (tgt_acg) {
+ int rc;
+
+ TRACE_DBG("Adding acg '%s' to device '%s' acg_list", acg_name,
+ tgt->tgt_name);
+ list_add_tail(&acg->acg_list_entry, &tgt->tgt_acg_list);
+ acg->tgt_acg = 1;
+
+ rc = scst_acg_sysfs_create(tgt, acg);
+ if (rc != 0)
+ goto out_del;
+ }
+
+out:
+ return acg;
+
+out_del:
+ list_del(&acg->acg_list_entry);
+
+out_free:
+ kfree(acg);
+ acg = NULL;
+ goto out;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_del_free_acg(struct scst_acg *acg)
+{
+ struct scst_acn *acn, *acnt;
+ struct scst_acg_dev *acg_dev, *acg_dev_tmp;
+
+ TRACE_DBG("Clearing acg %s from list", acg->acg_name);
+
+ BUG_ON(!list_empty(&acg->acg_sess_list));
+
+ /* Freeing acg_devs */
+ list_for_each_entry_safe(acg_dev, acg_dev_tmp, &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ struct scst_tgt_dev *tgt_dev, *tt;
+ list_for_each_entry_safe(tgt_dev, tt,
+ &acg_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (tgt_dev->acg_dev == acg_dev)
+ scst_free_tgt_dev(tgt_dev);
+ }
+ scst_del_free_acg_dev(acg_dev, true);
+ }
+
+ /* Freeing names */
+ list_for_each_entry_safe(acn, acnt, &acg->acn_list, acn_list_entry) {
+ scst_del_free_acn(acn,
+ list_is_last(&acn->acn_list_entry, &acg->acn_list));
+ }
+ INIT_LIST_HEAD(&acg->acn_list);
+
+ if (acg->tgt_acg) {
+ TRACE_DBG("Removing acg %s from list", acg->acg_name);
+ list_del(&acg->acg_list_entry);
+
+ scst_acg_sysfs_del(acg);
+ } else
+ acg->tgt->default_acg = NULL;
+
+ BUG_ON(!list_empty(&acg->acg_sess_list));
+ BUG_ON(!list_empty(&acg->acg_dev_list));
+ BUG_ON(!list_empty(&acg->acn_list));
+
+ kfree(acg->acg_name);
+ kfree(acg);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+struct scst_acg *scst_tgt_find_acg(struct scst_tgt *tgt, const char *name)
+{
+ struct scst_acg *acg, *acg_ret = NULL;
+
+ list_for_each_entry(acg, &tgt->tgt_acg_list, acg_list_entry) {
+ if (strcmp(acg->acg_name, name) == 0) {
+ acg_ret = acg;
+ break;
+ }
+ }
+ return acg_ret;
+}
+
+/* scst_mutex supposed to be held */
+static struct scst_tgt_dev *scst_find_shared_io_tgt_dev(
+ struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_tgt_dev *res = NULL;
+ struct scst_acg *acg = tgt_dev->acg_dev->acg;
+ struct scst_tgt_dev *t;
+
+ TRACE_DBG("tgt_dev %s (acg %p, io_grouping_type %d)",
+ tgt_dev->sess->initiator_name, acg, acg->acg_io_grouping_type);
+
+ switch (acg->acg_io_grouping_type) {
+ case SCST_IO_GROUPING_AUTO:
+ if (tgt_dev->sess->initiator_name == NULL)
+ goto out;
+
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((t == tgt_dev) ||
+ (t->sess->initiator_name == NULL) ||
+ (t->active_cmd_threads == NULL))
+ continue;
+
+ TRACE_DBG("t %s", t->sess->initiator_name);
+
+ /* We check other ACG's as well */
+
+ if (strcmp(t->sess->initiator_name,
+ tgt_dev->sess->initiator_name) == 0)
+ goto found;
+ }
+ break;
+
+ case SCST_IO_GROUPING_THIS_GROUP_ONLY:
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((t == tgt_dev) || (t->active_cmd_threads == NULL))
+ continue;
+
+ TRACE_DBG("t %s (acg %p)", t->sess->initiator_name,
+ t->acg_dev->acg);
+
+ if (t->acg_dev->acg == acg)
+ goto found;
+ }
+ break;
+
+ case SCST_IO_GROUPING_NEVER:
+ goto out;
+
+ default:
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((t == tgt_dev) || (t->active_cmd_threads == NULL))
+ continue;
+
+ TRACE_DBG("t %s (acg %p, io_grouping_type %d)",
+ t->sess->initiator_name, t->acg_dev->acg,
+ t->acg_dev->acg->acg_io_grouping_type);
+
+ if (t->acg_dev->acg->acg_io_grouping_type ==
+ acg->acg_io_grouping_type)
+ goto found;
+ }
+ break;
+ }
+
+out:
+ return res;
+
+found:
+ if (t->active_cmd_threads == &scst_main_cmd_threads) {
+ res = t;
+ TRACE_MGMT_DBG("Going to share async IO context %p (res %p, "
+ "ini %s, dev %s, grouping type %d)",
+ t->aic_keeper->aic, res, t->sess->initiator_name,
+ t->dev->virt_name,
+ t->acg_dev->acg->acg_io_grouping_type);
+ } else {
+ res = t;
+ if (!*(volatile bool*)&res->active_cmd_threads->io_context_ready) {
+ TRACE_MGMT_DBG("IO context for t %p not yet "
+ "initialized, waiting...", t);
+ msleep(100);
+ barrier();
+ goto found;
+ }
+ TRACE_MGMT_DBG("Going to share IO context %p (res %p, ini %s, "
+ "dev %s, cmd_threads %p, grouping type %d)",
+ res->active_cmd_threads->io_context, res,
+ t->sess->initiator_name, t->dev->virt_name,
+ t->active_cmd_threads,
+ t->acg_dev->acg->acg_io_grouping_type);
+ }
+ goto out;
+}
+
+enum scst_dev_type_threads_pool_type scst_parse_threads_pool_type(const char *p,
+ int len)
+{
+ enum scst_dev_type_threads_pool_type res;
+
+ if (strncasecmp(p, SCST_THREADS_POOL_PER_INITIATOR_STR,
+ min_t(int, strlen(SCST_THREADS_POOL_PER_INITIATOR_STR),
+ len)) == 0)
+ res = SCST_THREADS_POOL_PER_INITIATOR;
+ else if (strncasecmp(p, SCST_THREADS_POOL_SHARED_STR,
+ min_t(int, strlen(SCST_THREADS_POOL_SHARED_STR),
+ len)) == 0)
+ res = SCST_THREADS_POOL_SHARED;
+ else {
+ PRINT_ERROR("Unknown threads pool type %s", p);
+ res = SCST_THREADS_POOL_TYPE_INVALID;
+ }
+
+ return res;
+}
+
+static int scst_ioc_keeper_thread(void *arg)
+{
+ struct scst_async_io_context_keeper *aic_keeper =
+ (struct scst_async_io_context_keeper *)arg;
+
+ TRACE_MGMT_DBG("AIC %p keeper thread %s (PID %d) started", aic_keeper,
+ current->comm, current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ BUG_ON(aic_keeper->aic != NULL);
+
+ aic_keeper->aic = get_io_context(GFP_KERNEL, -1);
+ TRACE_MGMT_DBG("Alloced new async IO context %p (aic %p)",
+ aic_keeper->aic, aic_keeper);
+
+ /* We have our own ref counting */
+ put_io_context(aic_keeper->aic);
+
+ /* We are ready */
+ aic_keeper->aic_ready = true;
+ wake_up_all(&aic_keeper->aic_keeper_waitQ);
+
+ wait_event_interruptible(aic_keeper->aic_keeper_waitQ,
+ kthread_should_stop());
+
+ TRACE_MGMT_DBG("AIC %p keeper thread %s (PID %d) finished", aic_keeper,
+ current->comm, current->pid);
+ return 0;
+}
+
+/* scst_mutex supposed to be held */
+int scst_tgt_dev_setup_threads(struct scst_tgt_dev *tgt_dev)
+{
+ int res = 0;
+ struct scst_device *dev = tgt_dev->dev;
+ struct scst_async_io_context_keeper *aic_keeper;
+
+ if (dev->threads_num < 0)
+ goto out;
+
+ if (dev->threads_num == 0) {
+ struct scst_tgt_dev *shared_io_tgt_dev;
+ tgt_dev->active_cmd_threads = &scst_main_cmd_threads;
+
+ shared_io_tgt_dev = scst_find_shared_io_tgt_dev(tgt_dev);
+ if (shared_io_tgt_dev != NULL) {
+ aic_keeper = shared_io_tgt_dev->aic_keeper;
+ kref_get(&aic_keeper->aic_keeper_kref);
+
+ TRACE_MGMT_DBG("Linking async io context %p "
+ "for shared tgt_dev %p (dev %s)",
+ aic_keeper->aic, tgt_dev,
+ tgt_dev->dev->virt_name);
+ } else {
+ /* Create new context */
+ aic_keeper = kzalloc(sizeof(*aic_keeper), GFP_KERNEL);
+ if (aic_keeper == NULL) {
+ PRINT_ERROR("Unable to alloc aic_keeper "
+ "(size %zd)", sizeof(*aic_keeper));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ kref_init(&aic_keeper->aic_keeper_kref);
+ init_waitqueue_head(&aic_keeper->aic_keeper_waitQ);
+
+ aic_keeper->aic_keeper_thr =
+ kthread_run(scst_ioc_keeper_thread,
+ aic_keeper, "aic_keeper");
+ if (IS_ERR(aic_keeper->aic_keeper_thr)) {
+ PRINT_ERROR("Error running ioc_keeper "
+ "thread (tgt_dev %p)", tgt_dev);
+ res = PTR_ERR(aic_keeper->aic_keeper_thr);
+ goto out_free_keeper;
+ }
+
+ wait_event(aic_keeper->aic_keeper_waitQ,
+ aic_keeper->aic_ready);
+
+ TRACE_MGMT_DBG("Created async io context %p "
+ "for not shared tgt_dev %p (dev %s)",
+ aic_keeper->aic, tgt_dev,
+ tgt_dev->dev->virt_name);
+ }
+
+ tgt_dev->async_io_context = aic_keeper->aic;
+ tgt_dev->aic_keeper = aic_keeper;
+
+ res = scst_add_threads(tgt_dev->active_cmd_threads, NULL, NULL,
+ tgt_dev->sess->tgt->tgtt->threads_num);
+ goto out;
+ }
+
+ switch (dev->threads_pool_type) {
+ case SCST_THREADS_POOL_PER_INITIATOR:
+ {
+ struct scst_tgt_dev *shared_io_tgt_dev;
+
+ scst_init_threads(&tgt_dev->tgt_dev_cmd_threads);
+
+ tgt_dev->active_cmd_threads = &tgt_dev->tgt_dev_cmd_threads;
+
+ shared_io_tgt_dev = scst_find_shared_io_tgt_dev(tgt_dev);
+ if (shared_io_tgt_dev != NULL) {
+ TRACE_MGMT_DBG("Linking io context %p for "
+ "shared tgt_dev %p (cmd_threads %p)",
+ shared_io_tgt_dev->active_cmd_threads->io_context,
+ tgt_dev, tgt_dev->active_cmd_threads);
+ /* It's ref counted via threads */
+ tgt_dev->active_cmd_threads->io_context =
+ shared_io_tgt_dev->active_cmd_threads->io_context;
+ }
+
+ res = scst_add_threads(tgt_dev->active_cmd_threads, NULL,
+ tgt_dev,
+ dev->threads_num + tgt_dev->sess->tgt->tgtt->threads_num);
+ if (res != 0) {
+ /* Let's clear here, because no threads could be run */
+ tgt_dev->active_cmd_threads->io_context = NULL;
+ }
+ break;
+ }
+ case SCST_THREADS_POOL_SHARED:
+ {
+ tgt_dev->active_cmd_threads = &dev->dev_cmd_threads;
+
+ res = scst_add_threads(tgt_dev->active_cmd_threads, dev, NULL,
+ tgt_dev->sess->tgt->tgtt->threads_num);
+ break;
+ }
+ default:
+ PRINT_CRIT_ERROR("Unknown threads pool type %d (dev %s)",
+ dev->threads_pool_type, dev->virt_name);
+ BUG();
+ break;
+ }
+
+out:
+ if (res == 0)
+ tm_dbg_init_tgt_dev(tgt_dev);
+ return res;
+
+out_free_keeper:
+ kfree(aic_keeper);
+ goto out;
+}
+
+static void scst_aic_keeper_release(struct kref *kref)
+{
+ struct scst_async_io_context_keeper *aic_keeper;
+
+ aic_keeper = container_of(kref, struct scst_async_io_context_keeper,
+ aic_keeper_kref);
+
+ kthread_stop(aic_keeper->aic_keeper_thr);
+
+ kfree(aic_keeper);
+ return;
+}
+
+/* scst_mutex supposed to be held */
+void scst_tgt_dev_stop_threads(struct scst_tgt_dev *tgt_dev)
+{
+
+ if (tgt_dev->dev->threads_num < 0)
+ goto out_deinit;
+
+ if (tgt_dev->active_cmd_threads == &scst_main_cmd_threads) {
+ /* Global async threads */
+ kref_put(&tgt_dev->aic_keeper->aic_keeper_kref,
+ scst_aic_keeper_release);
+ tgt_dev->async_io_context = NULL;
+ tgt_dev->aic_keeper = NULL;
+ } else if (tgt_dev->active_cmd_threads == &tgt_dev->dev->dev_cmd_threads) {
+ /* Per device shared threads */
+ scst_del_threads(tgt_dev->active_cmd_threads,
+ tgt_dev->sess->tgt->tgtt->threads_num);
+ } else if (tgt_dev->active_cmd_threads == &tgt_dev->tgt_dev_cmd_threads) {
+ /* Per tgt_dev threads */
+ scst_del_threads(tgt_dev->active_cmd_threads, -1);
+ scst_deinit_threads(&tgt_dev->tgt_dev_cmd_threads);
+ } /* else no threads (not yet initialized, e.g.) */
+
+out_deinit:
+ tm_dbg_deinit_tgt_dev(tgt_dev);
+ tgt_dev->active_cmd_threads = NULL;
+ return;
+}
+
+/*
+ * scst_mutex supposed to be held, there must not be parallel activity in this
+ * session.
+ */
+static int scst_alloc_add_tgt_dev(struct scst_session *sess,
+ struct scst_acg_dev *acg_dev, struct scst_tgt_dev **out_tgt_dev)
+{
+ int res = 0;
+ int ini_sg, ini_unchecked_isa_dma, ini_use_clustering;
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_device *dev = acg_dev->dev;
+ struct list_head *sess_tgt_dev_list_head;
+ int i, sl;
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+
+ tgt_dev = kmem_cache_zalloc(scst_tgtd_cachep, GFP_KERNEL);
+ if (tgt_dev == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of scst_tgt_dev "
+ "failed");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ tgt_dev->dev = dev;
+ tgt_dev->lun = acg_dev->lun;
+ tgt_dev->acg_dev = acg_dev;
+ tgt_dev->sess = sess;
+ atomic_set(&tgt_dev->tgt_dev_cmd_count, 0);
+
+ scst_sgv_pool_use_norm(tgt_dev);
+
+ if (dev->scsi_dev != NULL) {
+ ini_sg = dev->scsi_dev->host->sg_tablesize;
+ ini_unchecked_isa_dma = dev->scsi_dev->host->unchecked_isa_dma;
+ ini_use_clustering = (dev->scsi_dev->host->use_clustering ==
+ ENABLE_CLUSTERING);
+ } else {
+ ini_sg = (1 << 15) /* infinite */;
+ ini_unchecked_isa_dma = 0;
+ ini_use_clustering = 0;
+ }
+ tgt_dev->max_sg_cnt = min(ini_sg, sess->tgt->sg_tablesize);
+
+ if ((sess->tgt->tgtt->use_clustering || ini_use_clustering) &&
+ !sess->tgt->tgtt->no_clustering)
+ scst_sgv_pool_use_norm_clust(tgt_dev);
+
+ if (sess->tgt->tgtt->unchecked_isa_dma || ini_unchecked_isa_dma)
+ scst_sgv_pool_use_dma(tgt_dev);
+
+ TRACE_MGMT_DBG("Device %s on SCST lun=%lld",
+ dev->virt_name, (long long unsigned int)tgt_dev->lun);
+
+ spin_lock_init(&tgt_dev->tgt_dev_lock);
+ INIT_LIST_HEAD(&tgt_dev->UA_list);
+ spin_lock_init(&tgt_dev->thr_data_lock);
+ INIT_LIST_HEAD(&tgt_dev->thr_data_list);
+ spin_lock_init(&tgt_dev->sn_lock);
+ INIT_LIST_HEAD(&tgt_dev->deferred_cmd_list);
+ INIT_LIST_HEAD(&tgt_dev->skipped_sn_list);
+ tgt_dev->curr_sn = (typeof(tgt_dev->curr_sn))(-300);
+ tgt_dev->expected_sn = tgt_dev->curr_sn + 1;
+ tgt_dev->num_free_sn_slots = ARRAY_SIZE(tgt_dev->sn_slots)-1;
+ tgt_dev->cur_sn_slot = &tgt_dev->sn_slots[0];
+ for (i = 0; i < (int)ARRAY_SIZE(tgt_dev->sn_slots); i++)
+ atomic_set(&tgt_dev->sn_slots[i], 0);
+
+ if (dev->handler->parse_atomic &&
+ dev->handler->alloc_data_buf_atomic &&
+ (sess->tgt->tgtt->preprocessing_done == NULL)) {
+ if (sess->tgt->tgtt->rdy_to_xfer_atomic)
+ __set_bit(SCST_TGT_DEV_AFTER_INIT_WR_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ }
+ if (dev->handler->dev_done_atomic &&
+ sess->tgt->tgtt->xmit_response_atomic) {
+ __set_bit(SCST_TGT_DEV_AFTER_EXEC_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ }
+
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ dev->d_sense, SCST_LOAD_SENSE(scst_sense_reset_UA));
+ scst_alloc_set_UA(tgt_dev, sense_buffer, sl, 0);
+
+ if (sess->tgt->tgtt->get_initiator_port_transport_id == NULL) {
+ if (!list_empty(&dev->dev_registrants_list)) {
+ PRINT_WARNING("Initiators from target %s can't connect "
+ "to device %s, because the device has PR "
+ "registrants and the target doesn't support "
+ "Persistent Reservations", sess->tgt->tgtt->name,
+ dev->virt_name);
+ res = -EPERM;
+ goto out_free;
+ }
+ dev->not_pr_supporting_tgt_devs_num++;
+ }
+
+ res = scst_pr_init_tgt_dev(tgt_dev);
+ if (res != 0)
+ goto out_dec_free;
+
+ res = scst_tgt_dev_setup_threads(tgt_dev);
+ if (res != 0)
+ goto out_pr_clear;
+
+ if (dev->handler && dev->handler->attach_tgt) {
+ TRACE_DBG("Calling dev handler's attach_tgt(%p)", tgt_dev);
+ res = dev->handler->attach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's attach_tgt() returned");
+ if (res != 0) {
+ PRINT_ERROR("Device handler's %s attach_tgt() "
+ "failed: %d", dev->handler->name, res);
+ goto out_stop_threads;
+ }
+ }
+
+ res = scst_tgt_dev_sysfs_create(tgt_dev);
+ if (res != 0)
+ goto out_detach;
+
+ spin_lock_bh(&dev->dev_lock);
+ list_add_tail(&tgt_dev->dev_tgt_dev_list_entry, &dev->dev_tgt_dev_list);
+ if (dev->dev_reserved)
+ __set_bit(SCST_TGT_DEV_RESERVED, &tgt_dev->tgt_dev_flags);
+ spin_unlock_bh(&dev->dev_lock);
+
+ sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[HASH_VAL(tgt_dev->lun)];
+ list_add_tail(&tgt_dev->sess_tgt_dev_list_entry,
+ sess_tgt_dev_list_head);
+
+ *out_tgt_dev = tgt_dev;
+
+out:
+ return res;
+
+out_detach:
+ if (dev->handler && dev->handler->detach_tgt) {
+ TRACE_DBG("Calling dev handler's detach_tgt(%p)",
+ tgt_dev);
+ dev->handler->detach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's detach_tgt() returned");
+ }
+
+out_stop_threads:
+ scst_tgt_dev_stop_threads(tgt_dev);
+
+out_pr_clear:
+ scst_pr_clear_tgt_dev(tgt_dev);
+
+out_dec_free:
+ if (tgt_dev->sess->tgt->tgtt->get_initiator_port_transport_id == NULL)
+ dev->not_pr_supporting_tgt_devs_num--;
+
+out_free:
+ scst_free_all_UA(tgt_dev);
+ kmem_cache_free(scst_tgtd_cachep, tgt_dev);
+ goto out;
+}
+
+/* No locks supposed to be held, scst_mutex - held */
+void scst_nexus_loss(struct scst_tgt_dev *tgt_dev, bool queue_UA)
+{
+
+ scst_clear_reservation(tgt_dev);
+
+ /* With activity suspended the lock isn't needed, but let's be safe */
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ scst_free_all_UA(tgt_dev);
+ memset(tgt_dev->tgt_dev_sense, 0, sizeof(tgt_dev->tgt_dev_sense));
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+
+ if (queue_UA) {
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ tgt_dev->dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_nexus_loss_UA));
+ scst_check_set_UA(tgt_dev, sense_buffer, sl, 0);
+ }
+ return;
+}
+
+/*
+ * scst_mutex supposed to be held, there must not be parallel activity in this
+ * session.
+ */
+static void scst_free_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_device *dev = tgt_dev->dev;
+
+ spin_lock_bh(&dev->dev_lock);
+ list_del(&tgt_dev->dev_tgt_dev_list_entry);
+ spin_unlock_bh(&dev->dev_lock);
+
+ list_del(&tgt_dev->sess_tgt_dev_list_entry);
+
+ scst_tgt_dev_sysfs_del(tgt_dev);
+
+ if (tgt_dev->sess->tgt->tgtt->get_initiator_port_transport_id == NULL)
+ dev->not_pr_supporting_tgt_devs_num--;
+
+ scst_clear_reservation(tgt_dev);
+ scst_pr_clear_tgt_dev(tgt_dev);
+ scst_free_all_UA(tgt_dev);
+
+ if (dev->handler && dev->handler->detach_tgt) {
+ TRACE_DBG("Calling dev handler's detach_tgt(%p)",
+ tgt_dev);
+ dev->handler->detach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's detach_tgt() returned");
+ }
+
+ scst_tgt_dev_stop_threads(tgt_dev);
+
+ BUG_ON(!list_empty(&tgt_dev->thr_data_list));
+
+ kmem_cache_free(scst_tgtd_cachep, tgt_dev);
+ return;
+}
+
+/* scst_mutex supposed to be held */
+int scst_sess_alloc_tgt_devs(struct scst_session *sess)
+{
+ int res = 0;
+ struct scst_acg_dev *acg_dev;
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(acg_dev, &sess->acg->acg_dev_list,
+ acg_dev_list_entry) {
+ res = scst_alloc_add_tgt_dev(sess, acg_dev, &tgt_dev);
+ if (res == -EPERM)
+ continue;
+ else if (res != 0)
+ goto out_free;
+ }
+
+out:
+ return res;
+
+out_free:
+ scst_sess_free_tgt_devs(sess);
+ goto out;
+}
+
+/*
+ * scst_mutex supposed to be held, there must not be parallel activity in this
+ * session.
+ */
+void scst_sess_free_tgt_devs(struct scst_session *sess)
+{
+ int i;
+ struct scst_tgt_dev *tgt_dev, *t;
+
+ /* The session is going down, no users, so no locks */
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry_safe(tgt_dev, t, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ scst_free_tgt_dev(tgt_dev);
+ }
+ INIT_LIST_HEAD(sess_tgt_dev_list_head);
+ }
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_acg_add_acn(struct scst_acg *acg, const char *name)
+{
+ int res = 0;
+ struct scst_acn *acn;
+ int len;
+ char *nm;
+
+ list_for_each_entry(acn, &acg->acn_list, acn_list_entry) {
+ if (strcmp(acn->name, name) == 0) {
+ PRINT_ERROR("Name %s already exists in group %s",
+ name, acg->acg_name);
+ res = -EEXIST;
+ goto out;
+ }
+ }
+
+ acn = kzalloc(sizeof(*acn), GFP_KERNEL);
+ if (acn == NULL) {
+ PRINT_ERROR("%s", "Unable to allocate scst_acn");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ acn->acg = acg;
+
+ len = strlen(name);
+ nm = kmalloc(len + 1, GFP_KERNEL);
+ if (nm == NULL) {
+ PRINT_ERROR("%s", "Unable to allocate scst_acn->name");
+ res = -ENOMEM;
+ goto out_free;
+ }
+
+ strcpy(nm, name);
+ acn->name = nm;
+
+ res = scst_acn_sysfs_create(acn);
+ if (res != 0)
+ goto out_free_nm;
+
+ list_add_tail(&acn->acn_list_entry, &acg->acn_list);
+
+out:
+ if (res == 0) {
+ PRINT_INFO("Added name %s to group %s", name, acg->acg_name);
+ scst_check_reassign_sessions();
+ }
+ return res;
+
+out_free_nm:
+ kfree(nm);
+
+out_free:
+ kfree(acn);
+ goto out;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_del_free_acn(struct scst_acn *acn, bool reassign)
+{
+
+ list_del(&acn->acn_list_entry);
+
+ scst_acn_sysfs_del(acn);
+
+ kfree(acn->name);
+ kfree(acn);
+
+ if (reassign)
+ scst_check_reassign_sessions();
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+struct scst_acn *scst_find_acn(struct scst_acg *acg, const char *name)
+{
+ struct scst_acn *acn;
+
+ TRACE_DBG("Trying to find name '%s'", name);
+
+ list_for_each_entry(acn, &acg->acn_list, acn_list_entry) {
+ if (strcmp(acn->name, name) == 0) {
+ TRACE_DBG("%s", "Found");
+ goto out;
+ }
+ }
+ acn = NULL;
+out:
+ return acn;
+}
+
+static struct scst_cmd *scst_create_prepare_internal_cmd(
+ struct scst_cmd *orig_cmd, int bufsize)
+{
+ struct scst_cmd *res;
+ gfp_t gfp_mask = scst_cmd_atomic(orig_cmd) ? GFP_ATOMIC : GFP_KERNEL;
+
+ res = scst_alloc_cmd(gfp_mask);
+ if (res == NULL)
+ goto out;
+
+ res->cmd_threads = orig_cmd->cmd_threads;
+ res->sess = orig_cmd->sess;
+ res->atomic = scst_cmd_atomic(orig_cmd);
+ res->internal = 1;
+ res->tgtt = orig_cmd->tgtt;
+ res->tgt = orig_cmd->tgt;
+ res->dev = orig_cmd->dev;
+ res->tgt_dev = orig_cmd->tgt_dev;
+ res->lun = orig_cmd->lun;
+ res->queue_type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ res->data_direction = SCST_DATA_UNKNOWN;
+ res->orig_cmd = orig_cmd;
+ res->bufflen = bufsize;
+
+ scst_sess_get(res->sess);
+ if (res->tgt_dev != NULL)
+ __scst_get(0);
+
+ res->state = SCST_CMD_STATE_PRE_PARSE;
+
+out:
+ return res;
+}
+
+int scst_prepare_request_sense(struct scst_cmd *orig_cmd)
+{
+ int res = 0;
+ static const uint8_t request_sense[6] = {
+ REQUEST_SENSE, 0, 0, 0, SCST_SENSE_BUFFERSIZE, 0
+ };
+ struct scst_cmd *rs_cmd;
+
+ if (orig_cmd->sense != NULL) {
+ TRACE_MEM("Releasing sense %p (orig_cmd %p)",
+ orig_cmd->sense, orig_cmd);
+ mempool_free(orig_cmd->sense, scst_sense_mempool);
+ orig_cmd->sense = NULL;
+ }
+
+ rs_cmd = scst_create_prepare_internal_cmd(orig_cmd,
+ SCST_SENSE_BUFFERSIZE);
+ if (rs_cmd == NULL)
+ goto out_error;
+
+ memcpy(rs_cmd->cdb, request_sense, sizeof(request_sense));
+ rs_cmd->cdb[1] |= scst_get_cmd_dev_d_sense(orig_cmd);
+ rs_cmd->cdb_len = sizeof(request_sense);
+ rs_cmd->data_direction = SCST_DATA_READ;
+ rs_cmd->expected_data_direction = rs_cmd->data_direction;
+ rs_cmd->expected_transfer_len = SCST_SENSE_BUFFERSIZE;
+ rs_cmd->expected_values_set = 1;
+
+ TRACE_MGMT_DBG("Adding REQUEST SENSE cmd %p to head of active "
+ "cmd list", rs_cmd);
+ spin_lock_irq(&rs_cmd->cmd_threads->cmd_list_lock);
+ list_add(&rs_cmd->cmd_list_entry, &rs_cmd->cmd_threads->active_cmd_list);
+ wake_up(&rs_cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&rs_cmd->cmd_threads->cmd_list_lock);
+
+out:
+ return res;
+
+out_error:
+ res = -1;
+ goto out;
+}
+
+static void scst_complete_request_sense(struct scst_cmd *req_cmd)
+{
+ struct scst_cmd *orig_cmd = req_cmd->orig_cmd;
+ uint8_t *buf;
+ int len;
+
+ BUG_ON(orig_cmd == NULL);
+
+ len = scst_get_buf_first(req_cmd, &buf);
+
+ if (scsi_status_is_good(req_cmd->status) && (len > 0) &&
+ SCST_SENSE_VALID(buf) && (!SCST_NO_SENSE(buf))) {
+ PRINT_BUFF_FLAG(TRACE_SCSI, "REQUEST SENSE returned",
+ buf, len);
+ scst_alloc_set_sense(orig_cmd, scst_cmd_atomic(req_cmd), buf,
+ len);
+ } else {
+ PRINT_ERROR("%s", "Unable to get the sense via "
+ "REQUEST SENSE, returning HARDWARE ERROR");
+ scst_set_cmd_error(orig_cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+
+ if (len > 0)
+ scst_put_buf(req_cmd, buf);
+
+ TRACE_MGMT_DBG("Adding orig cmd %p to head of active "
+ "cmd list", orig_cmd);
+ spin_lock_irq(&orig_cmd->cmd_threads->cmd_list_lock);
+ list_add(&orig_cmd->cmd_list_entry, &orig_cmd->cmd_threads->active_cmd_list);
+ wake_up(&orig_cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&orig_cmd->cmd_threads->cmd_list_lock);
+ return;
+}
+
+int scst_finish_internal_cmd(struct scst_cmd *cmd)
+{
+ int res;
+
+ BUG_ON(!cmd->internal);
+
+ if (cmd->cdb[0] == REQUEST_SENSE)
+ scst_complete_request_sense(cmd);
+
+ __scst_cmd_put(cmd);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ return res;
+}
+
+static void scst_send_release(struct scst_device *dev)
+{
+ struct scsi_device *scsi_dev;
+ unsigned char cdb[6];
+ uint8_t sense[SCSI_SENSE_BUFFERSIZE];
+ int rc, i;
+
+ if (dev->scsi_dev == NULL)
+ goto out;
+
+ scsi_dev = dev->scsi_dev;
+
+ for (i = 0; i < 5; i++) {
+ memset(cdb, 0, sizeof(cdb));
+ cdb[0] = RELEASE;
+ cdb[1] = (scsi_dev->scsi_level <= SCSI_2) ?
+ ((scsi_dev->lun << 5) & 0xe0) : 0;
+
+ memset(sense, 0, sizeof(sense));
+
+ TRACE(TRACE_DEBUG | TRACE_SCSI, "%s", "Sending RELEASE req to "
+ "SCSI mid-level");
+ rc = scsi_execute(scsi_dev, cdb, SCST_DATA_NONE, NULL, 0,
+ sense, 15, 0, 0
+ , NULL
+ );
+ TRACE_DBG("MODE_SENSE done: %x", rc);
+
+ if (scsi_status_is_good(rc)) {
+ break;
+ } else {
+ PRINT_ERROR("RELEASE failed: %d", rc);
+ PRINT_BUFFER("RELEASE sense", sense, sizeof(sense));
+ scst_check_internal_sense(dev, rc, sense,
+ sizeof(sense));
+ }
+ }
+
+out:
+ return;
+}
+
+/* scst_mutex supposed to be held */
+static void scst_clear_reservation(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_device *dev = tgt_dev->dev;
+ int release = 0;
+
+ spin_lock_bh(&dev->dev_lock);
+ if (dev->dev_reserved &&
+ !test_bit(SCST_TGT_DEV_RESERVED, &tgt_dev->tgt_dev_flags)) {
+ /* This is one who holds the reservation */
+ struct scst_tgt_dev *tgt_dev_tmp;
+ list_for_each_entry(tgt_dev_tmp, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ release = 1;
+ }
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (release)
+ scst_send_release(dev);
+ return;
+}
+
+struct scst_session *scst_alloc_session(struct scst_tgt *tgt, gfp_t gfp_mask,
+ const char *initiator_name)
+{
+ struct scst_session *sess;
+ int i;
+
+ sess = kmem_cache_zalloc(scst_sess_cachep, gfp_mask);
+ if (sess == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Allocation of scst_session failed");
+ goto out;
+ }
+
+ sess->init_phase = SCST_SESS_IPH_INITING;
+ sess->shut_phase = SCST_SESS_SPH_READY;
+ atomic_set(&sess->refcnt, 0);
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ INIT_LIST_HEAD(sess_tgt_dev_list_head);
+ }
+ spin_lock_init(&sess->sess_list_lock);
+ INIT_LIST_HEAD(&sess->sess_cmd_list);
+ sess->tgt = tgt;
+ INIT_LIST_HEAD(&sess->init_deferred_cmd_list);
+ INIT_LIST_HEAD(&sess->init_deferred_mcmd_list);
+ INIT_DELAYED_WORK(&sess->hw_pending_work,
+ (void (*)(struct work_struct *))scst_hw_pending_work_fn);
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ spin_lock_init(&sess->lat_lock);
+#endif
+
+ sess->initiator_name = kstrdup(initiator_name, gfp_mask);
+ if (sess->initiator_name == NULL) {
+ PRINT_ERROR("%s", "Unable to dup sess->initiator_name");
+ goto out_free;
+ }
+
+out:
+ return sess;
+
+out_free:
+ kmem_cache_free(scst_sess_cachep, sess);
+ sess = NULL;
+ goto out;
+}
+
+void scst_free_session(struct scst_session *sess)
+{
+
+ mutex_lock(&scst_mutex);
+
+ scst_sess_free_tgt_devs(sess);
+
+ /* tgt will stay alive at least until its sysfs alive */
+ kobject_get(&sess->tgt->tgt_kobj);
+
+ mutex_unlock(&scst_mutex);
+ scst_sess_sysfs_del(sess);
+ mutex_lock(&scst_mutex);
+
+ /*
+ * The lists delete must be after sysfs del. Otherwise it would break
+ * logic in scst_sess_sysfs_create() to avoid duplicate sysfs names.
+ */
+
+ TRACE_DBG("Removing sess %p from the list", sess);
+ list_del(&sess->sess_list_entry);
+ TRACE_DBG("Removing session %p from acg %s", sess, sess->acg->acg_name);
+ list_del(&sess->acg_sess_list_entry);
+
+ mutex_unlock(&scst_mutex);
+
+ wake_up_all(&sess->tgt->unreg_waitQ);
+
+ kobject_put(&sess->tgt->tgt_kobj);
+
+ kfree(sess->transport_id);
+ kfree(sess->initiator_name);
+
+ kmem_cache_free(scst_sess_cachep, sess);
+ return;
+}
+
+void scst_free_session_callback(struct scst_session *sess)
+{
+ struct completion *c;
+
+ TRACE_DBG("Freeing session %p", sess);
+
+ cancel_delayed_work_sync(&sess->hw_pending_work);
+
+ c = sess->shutdown_compl;
+
+ mutex_lock(&scst_mutex);
+ /*
+ * Necessary to sync with other threads trying to queue AEN, which
+ * the target driver will not be able to serve and crash, because after
+ * unreg_done_fn() called its internal session data will be destroyed.
+ */
+ sess->shut_phase = SCST_SESS_SPH_UNREG_DONE_CALLING;
+ mutex_unlock(&scst_mutex);
+
+ if (sess->unreg_done_fn) {
+ TRACE_DBG("Calling unreg_done_fn(%p)", sess);
+ sess->unreg_done_fn(sess);
+ TRACE_DBG("%s", "unreg_done_fn() returned");
+ }
+ scst_free_session(sess);
+
+ if (c)
+ complete_all(c);
+ return;
+}
+
+void scst_sched_session_free(struct scst_session *sess)
+{
+ unsigned long flags;
+
+ if (sess->shut_phase != SCST_SESS_SPH_SHUTDOWN) {
+ PRINT_CRIT_ERROR("session %p is going to shutdown with unknown "
+ "shut phase %lx", sess, sess->shut_phase);
+ BUG();
+ }
+
+ spin_lock_irqsave(&scst_mgmt_lock, flags);
+ TRACE_DBG("Adding sess %p to scst_sess_shut_list", sess);
+ list_add_tail(&sess->sess_shut_list_entry, &scst_sess_shut_list);
+ spin_unlock_irqrestore(&scst_mgmt_lock, flags);
+
+ wake_up(&scst_mgmt_waitQ);
+ return;
+}
+
+/**
+ * scst_cmd_get() - increase command's reference counter
+ */
+void scst_cmd_get(struct scst_cmd *cmd)
+{
+ __scst_cmd_get(cmd);
+}
+EXPORT_SYMBOL(scst_cmd_get);
+
+/**
+ * scst_cmd_put() - decrease command's reference counter
+ */
+void scst_cmd_put(struct scst_cmd *cmd)
+{
+ __scst_cmd_put(cmd);
+}
+EXPORT_SYMBOL(scst_cmd_put);
+
+struct scst_cmd *scst_alloc_cmd(gfp_t gfp_mask)
+{
+ struct scst_cmd *cmd;
+
+ cmd = kmem_cache_zalloc(scst_cmd_cachep, gfp_mask);
+ if (cmd == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of scst_cmd failed");
+ goto out;
+ }
+
+ cmd->state = SCST_CMD_STATE_INIT_WAIT;
+ cmd->start_time = jiffies;
+ atomic_set(&cmd->cmd_ref, 1);
+ cmd->cmd_threads = &scst_main_cmd_threads;
+ INIT_LIST_HEAD(&cmd->mgmt_cmd_list);
+ cmd->queue_type = SCST_CMD_QUEUE_SIMPLE;
+ cmd->timeout = SCST_DEFAULT_TIMEOUT;
+ cmd->retries = 0;
+ cmd->data_len = -1;
+ cmd->is_send_status = 1;
+ cmd->resp_data_len = -1;
+ cmd->write_sg = &cmd->sg;
+ cmd->write_sg_cnt = &cmd->sg_cnt;
+
+ cmd->dbl_ua_orig_data_direction = SCST_DATA_UNKNOWN;
+ cmd->dbl_ua_orig_resp_data_len = -1;
+
+out:
+ return cmd;
+}
+
+static void scst_destroy_put_cmd(struct scst_cmd *cmd)
+{
+ scst_sess_put(cmd->sess);
+
+ /*
+ * At this point tgt_dev can be dead, but the pointer remains non-NULL
+ */
+ if (likely(cmd->tgt_dev != NULL))
+ __scst_put();
+
+ scst_destroy_cmd(cmd);
+ return;
+}
+
+/* No locks supposed to be held */
+void scst_free_cmd(struct scst_cmd *cmd)
+{
+ int destroy = 1;
+
+ TRACE_DBG("Freeing cmd %p (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("Freeing aborted cmd %p (scst_cmd_count %d)",
+ cmd, atomic_read(&scst_cmd_count));
+ }
+
+ BUG_ON(cmd->inc_blocking || cmd->needs_unblocking ||
+ cmd->dec_on_dev_needed);
+
+ /*
+ * Target driver can already free sg buffer before calling
+ * scst_tgt_cmd_done(). E.g., scst_local has to do that.
+ */
+ if (!cmd->tgt_data_buf_alloced)
+ scst_check_restore_sg_buff(cmd);
+
+ if ((cmd->tgtt->on_free_cmd != NULL) && likely(!cmd->internal)) {
+ TRACE_DBG("Calling target's on_free_cmd(%p)", cmd);
+ scst_set_cur_start(cmd);
+ cmd->tgtt->on_free_cmd(cmd);
+ scst_set_tgt_on_free_time(cmd);
+ TRACE_DBG("%s", "Target's on_free_cmd() returned");
+ }
+
+ if (likely(cmd->dev != NULL)) {
+ struct scst_dev_type *handler = cmd->dev->handler;
+ if (handler->on_free_cmd != NULL) {
+ TRACE_DBG("Calling dev handler %s on_free_cmd(%p)",
+ handler->name, cmd);
+ scst_set_cur_start(cmd);
+ handler->on_free_cmd(cmd);
+ scst_set_dev_on_free_time(cmd);
+ TRACE_DBG("Dev handler %s on_free_cmd() returned",
+ handler->name);
+ }
+ }
+
+ scst_release_space(cmd);
+
+ if (unlikely(cmd->sense != NULL)) {
+ TRACE_MEM("Releasing sense %p (cmd %p)", cmd->sense, cmd);
+ mempool_free(cmd->sense, scst_sense_mempool);
+ cmd->sense = NULL;
+ }
+
+ if (likely(cmd->tgt_dev != NULL)) {
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(!cmd->sent_for_exec) && !cmd->internal) {
+ PRINT_ERROR("Finishing not executed cmd %p (opcode "
+ "%d, target %s, LUN %lld, sn %d, expected_sn %d)",
+ cmd, cmd->cdb[0], cmd->tgtt->name,
+ (long long unsigned int)cmd->lun,
+ cmd->sn, cmd->tgt_dev->expected_sn);
+ scst_unblock_deferred(cmd->tgt_dev, cmd);
+ }
+#endif
+
+ if (unlikely(cmd->out_of_sn)) {
+ TRACE_SN("Out of SN cmd %p (tag %llu, sn %d), "
+ "destroy=%d", cmd,
+ (long long unsigned int)cmd->tag,
+ cmd->sn, destroy);
+ destroy = test_and_set_bit(SCST_CMD_CAN_BE_DESTROYED,
+ &cmd->cmd_flags);
+ }
+ }
+
+ if (likely(destroy))
+ scst_destroy_put_cmd(cmd);
+ return;
+}
+
+/* No locks supposed to be held. */
+void scst_check_retries(struct scst_tgt *tgt)
+{
+ int need_wake_up = 0;
+
+ /*
+ * We don't worry about overflow of finished_cmds, because we check
+ * only for its change.
+ */
+ atomic_inc(&tgt->finished_cmds);
+ /* See comment in scst_queue_retry_cmd() */
+ smp_mb__after_atomic_inc();
+ if (unlikely(tgt->retry_cmds > 0)) {
+ struct scst_cmd *c, *tc;
+ unsigned long flags;
+
+ TRACE_RETRY("Checking retry cmd list (retry_cmds %d)",
+ tgt->retry_cmds);
+
+ spin_lock_irqsave(&tgt->tgt_lock, flags);
+ list_for_each_entry_safe(c, tc, &tgt->retry_cmd_list,
+ cmd_list_entry) {
+ tgt->retry_cmds--;
+
+ TRACE_RETRY("Moving retry cmd %p to head of active "
+ "cmd list (retry_cmds left %d)",
+ c, tgt->retry_cmds);
+ spin_lock(&c->cmd_threads->cmd_list_lock);
+ list_move(&c->cmd_list_entry,
+ &c->cmd_threads->active_cmd_list);
+ wake_up(&c->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&c->cmd_threads->cmd_list_lock);
+
+ need_wake_up++;
+ if (need_wake_up >= 2) /* "slow start" */
+ break;
+ }
+ spin_unlock_irqrestore(&tgt->tgt_lock, flags);
+ }
+ return;
+}
+
+static void scst_tgt_retry_timer_fn(unsigned long arg)
+{
+ struct scst_tgt *tgt = (struct scst_tgt *)arg;
+ unsigned long flags;
+
+ TRACE_RETRY("Retry timer expired (retry_cmds %d)", tgt->retry_cmds);
+
+ spin_lock_irqsave(&tgt->tgt_lock, flags);
+ tgt->retry_timer_active = 0;
+ spin_unlock_irqrestore(&tgt->tgt_lock, flags);
+
+ scst_check_retries(tgt);
+ return;
+}
+
+struct scst_mgmt_cmd *scst_alloc_mgmt_cmd(gfp_t gfp_mask)
+{
+ struct scst_mgmt_cmd *mcmd;
+
+ mcmd = mempool_alloc(scst_mgmt_mempool, gfp_mask);
+ if (mcmd == NULL) {
+ PRINT_CRIT_ERROR("%s", "Allocation of management command "
+ "failed, some commands and their data could leak");
+ goto out;
+ }
+ memset(mcmd, 0, sizeof(*mcmd));
+
+out:
+ return mcmd;
+}
+
+void scst_free_mgmt_cmd(struct scst_mgmt_cmd *mcmd)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&mcmd->sess->sess_list_lock, flags);
+ atomic_dec(&mcmd->sess->sess_cmd_count);
+ spin_unlock_irqrestore(&mcmd->sess->sess_list_lock, flags);
+
+ scst_sess_put(mcmd->sess);
+
+ if (mcmd->mcmd_tgt_dev != NULL)
+ __scst_put();
+
+ mempool_free(mcmd, scst_mgmt_mempool);
+ return;
+}
+
+static bool scst_on_sg_tablesize_low(struct scst_cmd *cmd, bool out)
+{
+ bool res;
+ int sg_cnt = out ? cmd->out_sg_cnt : cmd->sg_cnt;
+ static int ll;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ if (sg_cnt > cmd->tgt->sg_tablesize) {
+ /* It's the target's side business */
+ goto failed;
+ }
+
+ if (tgt_dev->dev->handler->on_sg_tablesize_low == NULL)
+ goto failed;
+
+ res = tgt_dev->dev->handler->on_sg_tablesize_low(cmd);
+
+ TRACE_DBG("on_sg_tablesize_low(%p) returned %d", cmd, res);
+
+out:
+ return res;
+
+failed:
+ res = false;
+ if ((ll < 10) || TRACING_MINOR()) {
+ PRINT_INFO("Unable to complete command due to SG IO count "
+ "limitation (%srequested %d, available %d, tgt lim %d)",
+ out ? "OUT buffer, " : "", cmd->sg_cnt,
+ tgt_dev->max_sg_cnt, cmd->tgt->sg_tablesize);
+ ll++;
+ }
+ goto out;
+}
+
+int scst_alloc_space(struct scst_cmd *cmd)
+{
+ gfp_t gfp_mask;
+ int res = -ENOMEM;
+ int atomic = scst_cmd_atomic(cmd);
+ int flags;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ gfp_mask = tgt_dev->gfp_mask | (atomic ? GFP_ATOMIC : GFP_KERNEL);
+
+ flags = atomic ? SGV_POOL_NO_ALLOC_ON_CACHE_MISS : 0;
+ if (cmd->no_sgv)
+ flags |= SGV_POOL_ALLOC_NO_CACHED;
+
+ cmd->sg = sgv_pool_alloc(tgt_dev->pool, cmd->bufflen, gfp_mask, flags,
+ &cmd->sg_cnt, &cmd->sgv, &cmd->dev->dev_mem_lim, NULL);
+ if (cmd->sg == NULL)
+ goto out;
+
+ if (unlikely(cmd->sg_cnt > tgt_dev->max_sg_cnt))
+ if (!scst_on_sg_tablesize_low(cmd, false))
+ goto out_sg_free;
+
+ if (cmd->data_direction != SCST_DATA_BIDI)
+ goto success;
+
+ cmd->out_sg = sgv_pool_alloc(tgt_dev->pool, cmd->out_bufflen, gfp_mask,
+ flags, &cmd->out_sg_cnt, &cmd->out_sgv,
+ &cmd->dev->dev_mem_lim, NULL);
+ if (cmd->out_sg == NULL)
+ goto out_sg_free;
+
+ if (unlikely(cmd->out_sg_cnt > tgt_dev->max_sg_cnt))
+ if (!scst_on_sg_tablesize_low(cmd, true))
+ goto out_out_sg_free;
+
+success:
+ res = 0;
+
+out:
+ return res;
+
+out_out_sg_free:
+ sgv_pool_free(cmd->out_sgv, &cmd->dev->dev_mem_lim);
+ cmd->out_sgv = NULL;
+ cmd->out_sg = NULL;
+ cmd->out_sg_cnt = 0;
+
+out_sg_free:
+ sgv_pool_free(cmd->sgv, &cmd->dev->dev_mem_lim);
+ cmd->sgv = NULL;
+ cmd->sg = NULL;
+ cmd->sg_cnt = 0;
+ goto out;
+}
+
+static void scst_release_space(struct scst_cmd *cmd)
+{
+
+ if (cmd->sgv == NULL) {
+ if ((cmd->sg != NULL) &&
+ !(cmd->tgt_data_buf_alloced || cmd->dh_data_buf_alloced)) {
+ TRACE_MEM("Freeing sg %p for cmd %p (cnt %d)", cmd->sg,
+ cmd, cmd->sg_cnt);
+ scst_free(cmd->sg, cmd->sg_cnt);
+ goto out_zero;
+ } else
+ goto out;
+ }
+
+ if (cmd->tgt_data_buf_alloced || cmd->dh_data_buf_alloced) {
+ TRACE_MEM("%s", "*data_buf_alloced set, returning");
+ goto out;
+ }
+
+ if (cmd->out_sgv != NULL) {
+ sgv_pool_free(cmd->out_sgv, &cmd->dev->dev_mem_lim);
+ cmd->out_sgv = NULL;
+ cmd->out_sg_cnt = 0;
+ cmd->out_sg = NULL;
+ cmd->out_bufflen = 0;
+ }
+
+ sgv_pool_free(cmd->sgv, &cmd->dev->dev_mem_lim);
+
+out_zero:
+ cmd->sgv = NULL;
+ cmd->sg_cnt = 0;
+ cmd->sg = NULL;
+ cmd->bufflen = 0;
+ cmd->data_len = 0;
+
+out:
+ return;
+}
+
+static void scsi_end_async(struct request *req, int error)
+{
+ struct scsi_io_context *sioc = req->end_io_data;
+
+ TRACE_DBG("sioc %p, cmd %p", sioc, sioc->data);
+
+ if (sioc->done)
+ sioc->done(sioc->data, sioc->sense, req->errors, req->resid_len);
+
+ if (!sioc->full_cdb_used)
+ kmem_cache_free(scsi_io_context_cache, sioc);
+ else
+ kfree(sioc);
+
+ __blk_put_request(req->q, req);
+ return;
+}
+
+/**
+ * scst_scsi_exec_async - executes a SCSI command in pass-through mode
+ * @cmd: scst command
+ * @data: pointer passed to done() as "data"
+ * @done: callback function when done
+ */
+int scst_scsi_exec_async(struct scst_cmd *cmd, void *data,
+ void (*done)(void *data, char *sense, int result, int resid))
+{
+ int res = 0;
+ struct request_queue *q = cmd->dev->scsi_dev->request_queue;
+ struct request *rq;
+ struct scsi_io_context *sioc;
+ int write = (cmd->data_direction & SCST_DATA_WRITE) ? WRITE : READ;
+ gfp_t gfp = GFP_KERNEL;
+ int cmd_len = cmd->cdb_len;
+
+ if (cmd->ext_cdb_len == 0) {
+ TRACE_DBG("Simple CDB (cmd_len %d)", cmd_len);
+ sioc = kmem_cache_zalloc(scsi_io_context_cache, gfp);
+ if (sioc == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ } else {
+ cmd_len += cmd->ext_cdb_len;
+
+ TRACE_DBG("Extended CDB (cmd_len %d)", cmd_len);
+
+ sioc = kzalloc(sizeof(*sioc) + cmd_len, gfp);
+ if (sioc == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ sioc->full_cdb_used = 1;
+
+ memcpy(sioc->full_cdb, cmd->cdb, cmd->cdb_len);
+ memcpy(&sioc->full_cdb[cmd->cdb_len], cmd->ext_cdb,
+ cmd->ext_cdb_len);
+ }
+
+ rq = blk_get_request(q, write, gfp);
+ if (rq == NULL) {
+ res = -ENOMEM;
+ goto out_free_sioc;
+ }
+
+ rq->cmd_type = REQ_TYPE_BLOCK_PC;
+ rq->cmd_flags |= REQ_QUIET;
+
+ if (cmd->sg == NULL)
+ goto done;
+
+ if (cmd->data_direction == SCST_DATA_BIDI) {
+ struct request *next_rq;
+
+ if (!test_bit(QUEUE_FLAG_BIDI, &q->queue_flags)) {
+ res = -EOPNOTSUPP;
+ goto out_free_rq;
+ }
+
+ res = blk_rq_map_kern_sg(rq, cmd->out_sg, cmd->out_sg_cnt, gfp);
+ if (res != 0) {
+ TRACE_DBG("blk_rq_map_kern_sg() failed: %d", res);
+ goto out_free_rq;
+ }
+
+ next_rq = blk_get_request(q, READ, gfp);
+ if (next_rq == NULL) {
+ res = -ENOMEM;
+ goto out_free_unmap;
+ }
+ rq->next_rq = next_rq;
+ next_rq->cmd_type = rq->cmd_type;
+
+ res = blk_rq_map_kern_sg(next_rq, cmd->sg, cmd->sg_cnt, gfp);
+ if (res != 0) {
+ TRACE_DBG("blk_rq_map_kern_sg() failed: %d", res);
+ goto out_free_unmap;
+ }
+ } else {
+ res = blk_rq_map_kern_sg(rq, cmd->sg, cmd->sg_cnt, gfp);
+ if (res != 0) {
+ TRACE_DBG("blk_rq_map_kern_sg() failed: %d", res);
+ goto out_free_rq;
+ }
+ }
+
+done:
+ TRACE_DBG("sioc %p, cmd %p", sioc, cmd);
+
+ sioc->data = data;
+ sioc->done = done;
+
+ rq->cmd_len = cmd_len;
+ if (cmd->ext_cdb_len == 0) {
+ memset(rq->cmd, 0, BLK_MAX_CDB); /* ATAPI hates garbage after CDB */
+ memcpy(rq->cmd, cmd->cdb, cmd->cdb_len);
+ } else
+ rq->cmd = sioc->full_cdb;
+
+ rq->sense = sioc->sense;
+ rq->sense_len = sizeof(sioc->sense);
+ rq->timeout = cmd->timeout;
+ rq->retries = cmd->retries;
+ rq->end_io_data = sioc;
+
+ blk_execute_rq_nowait(rq->q, NULL, rq,
+ (cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE), scsi_end_async);
+out:
+ return res;
+
+out_free_unmap:
+ if (rq->next_rq != NULL) {
+ blk_put_request(rq->next_rq);
+ rq->next_rq = NULL;
+ }
+ blk_rq_unmap_kern_sg(rq, res);
+
+out_free_rq:
+ blk_put_request(rq);
+
+out_free_sioc:
+ if (!sioc->full_cdb_used)
+ kmem_cache_free(scsi_io_context_cache, sioc);
+ else
+ kfree(sioc);
+ goto out;
+}
+EXPORT_SYMBOL(scst_scsi_exec_async);
+
+/**
+ * scst_copy_sg() - copy data between the command's SGs
+ *
+ * Copies data between cmd->tgt_sg and cmd->sg in direction defined by
+ * copy_dir parameter.
+ */
+void scst_copy_sg(struct scst_cmd *cmd, enum scst_sg_copy_dir copy_dir)
+{
+ struct scatterlist *src_sg, *dst_sg;
+ unsigned int to_copy;
+ int atomic = scst_cmd_atomic(cmd);
+
+ if (copy_dir == SCST_SG_COPY_FROM_TARGET) {
+ if (cmd->data_direction != SCST_DATA_BIDI) {
+ src_sg = cmd->tgt_sg;
+ dst_sg = cmd->sg;
+ to_copy = cmd->bufflen;
+ } else {
+ TRACE_MEM("BIDI cmd %p", cmd);
+ src_sg = cmd->tgt_out_sg;
+ dst_sg = cmd->out_sg;
+ to_copy = cmd->out_bufflen;
+ }
+ } else {
+ src_sg = cmd->sg;
+ dst_sg = cmd->tgt_sg;
+ to_copy = cmd->resp_data_len;
+ }
+
+ TRACE_MEM("cmd %p, copy_dir %d, src_sg %p, dst_sg %p, to_copy %lld",
+ cmd, copy_dir, src_sg, dst_sg, (long long)to_copy);
+
+ if (unlikely(src_sg == NULL) || unlikely(dst_sg == NULL)) {
+ /*
+ * It can happened, e.g., with scst_user for cmd with delay
+ * alloc, which failed with Check Condition.
+ */
+ goto out;
+ }
+
+ sg_copy(dst_sg, src_sg, 0, to_copy,
+ atomic ? KM_SOFTIRQ0 : KM_USER0,
+ atomic ? KM_SOFTIRQ1 : KM_USER1);
+
+out:
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_copy_sg);
+
+int scst_get_full_buf(struct scst_cmd *cmd, uint8_t **buf)
+{
+ int res = 0;
+
+ EXTRACHECKS_BUG_ON(cmd->sg_buff_vmallocated);
+
+ if (scst_get_buf_count(cmd) > 1) {
+ int len;
+ uint8_t *tmp_buf;
+ int full_size;
+
+ full_size = 0;
+ len = scst_get_buf_first(cmd, &tmp_buf);
+ while (len > 0) {
+ full_size += len;
+ scst_put_buf(cmd, tmp_buf);
+ len = scst_get_buf_next(cmd, &tmp_buf);
+ }
+
+ *buf = vmalloc(full_size);
+ if (*buf == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "vmalloc() failed for opcode "
+ "%x", cmd->cdb[0]);
+ res = -ENOMEM;
+ goto out;
+ }
+ cmd->sg_buff_vmallocated = 1;
+
+ if (scst_cmd_get_data_direction(cmd) == SCST_DATA_WRITE) {
+ uint8_t *buf_ptr;
+
+ buf_ptr = *buf;
+
+ len = scst_get_buf_first(cmd, &tmp_buf);
+ while (len > 0) {
+ memcpy(buf_ptr, tmp_buf, len);
+ buf_ptr += len;
+
+ scst_put_buf(cmd, tmp_buf);
+ len = scst_get_buf_next(cmd, &tmp_buf);
+ }
+ }
+ res = full_size;
+ } else
+ res = scst_get_buf_first(cmd, buf);
+
+out:
+ return res;
+}
+
+void scst_put_full_buf(struct scst_cmd *cmd, uint8_t *buf)
+{
+
+ if (buf == NULL)
+ goto out;
+
+ if (cmd->sg_buff_vmallocated) {
+ if (scst_cmd_get_data_direction(cmd) == SCST_DATA_READ) {
+ int len;
+ uint8_t *tmp_buf, *buf_p;
+
+ buf_p = buf;
+
+ len = scst_get_buf_first(cmd, &tmp_buf);
+ while (len > 0) {
+ memcpy(tmp_buf, buf_p, len);
+ buf_p += len;
+
+ scst_put_buf(cmd, tmp_buf);
+ len = scst_get_buf_next(cmd, &tmp_buf);
+ }
+
+ }
+
+ cmd->sg_buff_vmallocated = 0;
+
+ vfree(buf);
+ } else
+ scst_put_buf(cmd, buf);
+
+out:
+ return;
+}
+
+static const int SCST_CDB_LENGTH[8] = { 6, 10, 10, 0, 16, 12, 0, 0 };
+
+#define SCST_CDB_GROUP(opcode) ((opcode >> 5) & 0x7)
+#define SCST_GET_CDB_LEN(opcode) SCST_CDB_LENGTH[SCST_CDB_GROUP(opcode)]
+
+/* get_trans_len_x extract x bytes from cdb as length starting from off */
+
+static int get_trans_cdb_len_10(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->cdb_len = 10;
+ cmd->bufflen = 0;
+ return 0;
+}
+
+static int get_trans_len_block_limit(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = 6;
+ return 0;
+}
+
+static int get_trans_len_read_capacity(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = 8;
+ return 0;
+}
+
+static int get_trans_len_serv_act_in(struct scst_cmd *cmd, uint8_t off)
+{
+ int res = 0;
+
+ if ((cmd->cdb[1] & 0x1f) == SAI_READ_CAPACITY_16) {
+ cmd->op_name = "READ CAPACITY(16)";
+ cmd->bufflen = be32_to_cpu(get_unaligned((__be32 *)&cmd->cdb[10]));
+ cmd->op_flags |= SCST_IMPLICIT_HQ|SCST_REG_RESERVE_ALLOWED;
+ } else
+ cmd->op_flags |= SCST_UNKNOWN_LENGTH;
+ return res;
+}
+
+static int get_trans_len_single(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = 1;
+ return 0;
+}
+
+static int get_trans_len_read_pos(struct scst_cmd *cmd, uint8_t off)
+{
+ uint8_t *p = (uint8_t *)cmd->cdb + off;
+ int res = 0;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 8;
+ cmd->bufflen |= ((u32)p[1]);
+
+ switch (cmd->cdb[1] & 0x1f) {
+ case 0:
+ case 1:
+ case 6:
+ if (cmd->bufflen != 0) {
+ PRINT_ERROR("READ POSITION: Invalid non-zero (%d) "
+ "allocation length for service action %x",
+ cmd->bufflen, cmd->cdb[1] & 0x1f);
+ goto out_inval;
+ }
+ break;
+ }
+
+ switch (cmd->cdb[1] & 0x1f) {
+ case 0:
+ case 1:
+ cmd->bufflen = 20;
+ break;
+ case 6:
+ cmd->bufflen = 32;
+ break;
+ case 8:
+ cmd->bufflen = max(28, cmd->bufflen);
+ break;
+ default:
+ PRINT_ERROR("READ POSITION: Invalid service action %x",
+ cmd->cdb[1] & 0x1f);
+ goto out_inval;
+ }
+
+out:
+ return res;
+
+out_inval:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ res = 1;
+ goto out;
+}
+
+static int get_trans_len_prevent_allow_medium_removal(struct scst_cmd *cmd,
+ uint8_t off)
+{
+ if ((cmd->cdb[4] & 3) == 0)
+ cmd->op_flags |= SCST_REG_RESERVE_ALLOWED |
+ SCST_WRITE_EXCL_ALLOWED | SCST_EXCL_ACCESS_ALLOWED;
+ return 0;
+}
+
+static int get_trans_len_start_stop(struct scst_cmd *cmd, uint8_t off)
+{
+ if ((cmd->cdb[4] & 0xF1) == 0x1)
+ cmd->op_flags |= SCST_REG_RESERVE_ALLOWED |
+ SCST_WRITE_EXCL_ALLOWED | SCST_EXCL_ACCESS_ALLOWED;
+ return 0;
+}
+
+static int get_trans_len_3_read_elem_stat(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 16;
+ cmd->bufflen |= ((u32)p[1]) << 8;
+ cmd->bufflen |= ((u32)p[2]);
+
+ if ((cmd->cdb[6] & 0x2) == 0x2)
+ cmd->op_flags |= SCST_REG_RESERVE_ALLOWED |
+ SCST_WRITE_EXCL_ALLOWED | SCST_EXCL_ACCESS_ALLOWED;
+ return 0;
+}
+
+static int get_trans_len_1(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = (u32)cmd->cdb[off];
+ return 0;
+}
+
+static int get_trans_len_1_256(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = (u32)cmd->cdb[off];
+ if (cmd->bufflen == 0)
+ cmd->bufflen = 256;
+ return 0;
+}
+
+static int get_trans_len_2(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 8;
+ cmd->bufflen |= ((u32)p[1]);
+
+ return 0;
+}
+
+static int get_trans_len_3(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 16;
+ cmd->bufflen |= ((u32)p[1]) << 8;
+ cmd->bufflen |= ((u32)p[2]);
+
+ return 0;
+}
+
+static int get_trans_len_4(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 24;
+ cmd->bufflen |= ((u32)p[1]) << 16;
+ cmd->bufflen |= ((u32)p[2]) << 8;
+ cmd->bufflen |= ((u32)p[3]);
+
+ return 0;
+}
+
+static int get_trans_len_none(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = 0;
+ return 0;
+}
+
+static int get_bidi_trans_len_2(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 8;
+ cmd->bufflen |= ((u32)p[1]);
+
+ cmd->out_bufflen = cmd->bufflen;
+
+ return 0;
+}
+
+/**
+ * scst_get_cdb_info() - fill various info about the command's CDB
+ *
+ * Description:
+ * Fills various info about the command's CDB in the corresponding fields
+ * in the command.
+ *
+ * Returns: 0 on success, <0 if command is unknown, >0 if command
+ * is invalid.
+ */
+int scst_get_cdb_info(struct scst_cmd *cmd)
+{
+ int dev_type = cmd->dev->type;
+ int i, res = 0;
+ uint8_t op;
+ const struct scst_sdbops *ptr = NULL;
+
+ op = cmd->cdb[0]; /* get clear opcode */
+
+ TRACE_DBG("opcode=%02x, cdblen=%d bytes, tblsize=%d, "
+ "dev_type=%d", op, SCST_GET_CDB_LEN(op), SCST_CDB_TBL_SIZE,
+ dev_type);
+
+ i = scst_scsi_op_list[op];
+ while (i < SCST_CDB_TBL_SIZE && scst_scsi_op_table[i].ops == op) {
+ if (scst_scsi_op_table[i].devkey[dev_type] != SCST_CDB_NOTSUPP) {
+ ptr = &scst_scsi_op_table[i];
+ TRACE_DBG("op = 0x%02x+'%c%c%c%c%c%c%c%c%c%c'+<%s>",
+ ptr->ops, ptr->devkey[0], /* disk */
+ ptr->devkey[1], /* tape */
+ ptr->devkey[2], /* printer */
+ ptr->devkey[3], /* cpu */
+ ptr->devkey[4], /* cdr */
+ ptr->devkey[5], /* cdrom */
+ ptr->devkey[6], /* scanner */
+ ptr->devkey[7], /* worm */
+ ptr->devkey[8], /* changer */
+ ptr->devkey[9], /* commdev */
+ ptr->op_name);
+ TRACE_DBG("direction=%d flags=%d off=%d",
+ ptr->direction,
+ ptr->flags,
+ ptr->off);
+ break;
+ }
+ i++;
+ }
+
+ if (unlikely(ptr == NULL)) {
+ /* opcode not found or now not used */
+ TRACE(TRACE_MINOR, "Unknown opcode 0x%x for type %d", op,
+ dev_type);
+ res = -1;
+ goto out;
+ }
+
+ cmd->cdb_len = SCST_GET_CDB_LEN(op);
+ cmd->op_name = ptr->op_name;
+ cmd->data_direction = ptr->direction;
+ cmd->op_flags = ptr->flags | SCST_INFO_VALID;
+ res = (*ptr->get_trans_len)(cmd, ptr->off);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_get_cdb_info);
+
+/* Packs SCST LUN back to SCSI form */
+__be64 scst_pack_lun(const uint64_t lun, unsigned int addr_method)
+{
+ uint64_t res;
+ uint16_t *p = (uint16_t *)&res;
+
+ res = lun;
+
+ if ((addr_method == SCST_LUN_ADDR_METHOD_FLAT) && (lun != 0)) {
+ /*
+ * Flat space: luns other than 0 should use flat space
+ * addressing method.
+ */
+ *p = 0x7fff & *p;
+ *p = 0x4000 | *p;
+ }
+ /* Default is to use peripheral device addressing mode */
+
+ *p = (__force u16)cpu_to_be16(*p);
+ return (__force __be64)res;
+}
+
+/*
+ * Routine to extract a lun number from an 8-byte LUN structure
+ * in network byte order (BE).
+ * (see SAM-2, Section 4.12.3 page 40)
+ * Supports 2 types of lun unpacking: peripheral and logical unit.
+ */
+uint64_t scst_unpack_lun(const uint8_t *lun, int len)
+{
+ uint64_t res = NO_SUCH_LUN;
+ int address_method;
+
+ TRACE_BUFF_FLAG(TRACE_DEBUG, "Raw LUN", lun, len);
+
+ if (unlikely(len < 2)) {
+ PRINT_ERROR("Illegal lun length %d, expected 2 bytes or "
+ "more", len);
+ goto out;
+ }
+
+ if (len > 2) {
+ switch (len) {
+ case 8:
+ if ((*((__be64 *)lun) &
+ __constant_cpu_to_be64(0x0000FFFFFFFFFFFFLL)) != 0)
+ goto out_err;
+ break;
+ case 4:
+ if (*((__be16 *)&lun[2]) != 0)
+ goto out_err;
+ break;
+ case 6:
+ if (*((__be32 *)&lun[2]) != 0)
+ goto out_err;
+ break;
+ default:
+ goto out_err;
+ }
+ }
+
+ address_method = (*lun) >> 6; /* high 2 bits of byte 0 */
+ switch (address_method) {
+ case 0: /* peripheral device addressing method */
+#if 0
+ if (*lun) {
+ PRINT_ERROR("Illegal BUS INDENTIFIER in LUN "
+ "peripheral device addressing method 0x%02x, "
+ "expected 0", *lun);
+ break;
+ }
+ res = *(lun + 1);
+ break;
+#else
+ /*
+ * Looks like it's legal to use it as flat space addressing
+ * method as well
+ */
+
+ /* go through */
+#endif
+
+ case 1: /* flat space addressing method */
+ res = *(lun + 1) | (((*lun) & 0x3f) << 8);
+ break;
+
+ case 2: /* logical unit addressing method */
+ if (*lun & 0x3f) {
+ PRINT_ERROR("Illegal BUS NUMBER in LUN logical unit "
+ "addressing method 0x%02x, expected 0",
+ *lun & 0x3f);
+ break;
+ }
+ if (*(lun + 1) & 0xe0) {
+ PRINT_ERROR("Illegal TARGET in LUN logical unit "
+ "addressing method 0x%02x, expected 0",
+ (*(lun + 1) & 0xf8) >> 5);
+ break;
+ }
+ res = *(lun + 1) & 0x1f;
+ break;
+
+ case 3: /* extended logical unit addressing method */
+ default:
+ PRINT_ERROR("Unimplemented LUN addressing method %u",
+ address_method);
+ break;
+ }
+
+out:
+ return res;
+
+out_err:
+ PRINT_ERROR("%s", "Multi-level LUN unimplemented");
+ goto out;
+}
+
+/**
+ ** Generic parse() support routines.
+ ** Done via pointer on functions to avoid unneeded dereferences on
+ ** the fast path.
+ **/
+
+/**
+ * scst_calc_block_shift() - calculate block shift
+ *
+ * Calculates and returns block shift for the given sector size
+ */
+int scst_calc_block_shift(int sector_size)
+{
+ int block_shift = 0;
+ int t;
+
+ if (sector_size == 0)
+ sector_size = 512;
+
+ t = sector_size;
+ while (1) {
+ if ((t & 1) != 0)
+ break;
+ t >>= 1;
+ block_shift++;
+ }
+ if (block_shift < 9) {
+ PRINT_ERROR("Wrong sector size %d", sector_size);
+ block_shift = -1;
+ }
+ return block_shift;
+}
+EXPORT_SYMBOL_GPL(scst_calc_block_shift);
+
+/**
+ * scst_sbc_generic_parse() - generic SBC parsing
+ *
+ * Generic parse() for SBC (disk) devices
+ */
+int scst_sbc_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ switch (cmd->cdb[0]) {
+ case VERIFY_6:
+ case VERIFY:
+ case VERIFY_12:
+ case VERIFY_16:
+ if ((cmd->cdb[1] & BYTCHK) == 0) {
+ cmd->data_len = cmd->bufflen << get_block_shift(cmd);
+ cmd->bufflen = 0;
+ goto set_timeout;
+ } else
+ cmd->data_len = 0;
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED) {
+ int block_shift = get_block_shift(cmd);
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ cmd->bufflen = cmd->bufflen << block_shift;
+ cmd->out_bufflen = cmd->out_bufflen << block_shift;
+ }
+
+set_timeout:
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_DISK_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_DISK_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_DISK_LONG_TIMEOUT;
+
+ TRACE_DBG("res %d, bufflen %d, data_len %d, direct %d",
+ res, cmd->bufflen, cmd->data_len, cmd->data_direction);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_sbc_generic_parse);
+
+/**
+ * scst_cdrom_generic_parse() - generic MMC parse
+ *
+ * Generic parse() for MMC (cdrom) devices
+ */
+int scst_cdrom_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ cmd->cdb[1] &= 0x1f;
+
+ switch (cmd->cdb[0]) {
+ case VERIFY_6:
+ case VERIFY:
+ case VERIFY_12:
+ case VERIFY_16:
+ if ((cmd->cdb[1] & BYTCHK) == 0) {
+ cmd->data_len = cmd->bufflen << get_block_shift(cmd);
+ cmd->bufflen = 0;
+ goto set_timeout;
+ }
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED) {
+ int block_shift = get_block_shift(cmd);
+ cmd->bufflen = cmd->bufflen << block_shift;
+ cmd->out_bufflen = cmd->out_bufflen << block_shift;
+ }
+
+set_timeout:
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_CDROM_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_CDROM_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_CDROM_LONG_TIMEOUT;
+
+ TRACE_DBG("res=%d, bufflen=%d, direct=%d", res, cmd->bufflen,
+ cmd->data_direction);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_cdrom_generic_parse);
+
+/**
+ * scst_modisk_generic_parse() - generic MO parse
+ *
+ * Generic parse() for MO disk devices
+ */
+int scst_modisk_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ cmd->cdb[1] &= 0x1f;
+
+ switch (cmd->cdb[0]) {
+ case VERIFY_6:
+ case VERIFY:
+ case VERIFY_12:
+ case VERIFY_16:
+ if ((cmd->cdb[1] & BYTCHK) == 0) {
+ cmd->data_len = cmd->bufflen << get_block_shift(cmd);
+ cmd->bufflen = 0;
+ goto set_timeout;
+ }
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED) {
+ int block_shift = get_block_shift(cmd);
+ cmd->bufflen = cmd->bufflen << block_shift;
+ cmd->out_bufflen = cmd->out_bufflen << block_shift;
+ }
+
+set_timeout:
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_MODISK_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_MODISK_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_MODISK_LONG_TIMEOUT;
+
+ TRACE_DBG("res=%d, bufflen=%d, direct=%d", res, cmd->bufflen,
+ cmd->data_direction);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_modisk_generic_parse);
+
+/**
+ * scst_tape_generic_parse() - generic tape parse
+ *
+ * Generic parse() for tape devices
+ */
+int scst_tape_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_size)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ if (cmd->cdb[0] == READ_POSITION) {
+ int tclp = cmd->cdb[1] & 4;
+ int long_bit = cmd->cdb[1] & 2;
+ int bt = cmd->cdb[1] & 1;
+
+ if ((tclp == long_bit) && (!bt || !long_bit)) {
+ cmd->bufflen =
+ tclp ? POSITION_LEN_LONG : POSITION_LEN_SHORT;
+ cmd->data_direction = SCST_DATA_READ;
+ } else {
+ cmd->bufflen = 0;
+ cmd->data_direction = SCST_DATA_NONE;
+ }
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED & cmd->cdb[1]) {
+ int block_size = get_block_size(cmd);
+ cmd->bufflen = cmd->bufflen * block_size;
+ cmd->out_bufflen = cmd->out_bufflen * block_size;
+ }
+
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_TAPE_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_TAPE_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_TAPE_LONG_TIMEOUT;
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_tape_generic_parse);
+
+static int scst_null_parse(struct scst_cmd *cmd)
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+#if 0
+ switch (cmd->cdb[0]) {
+ default:
+ /* It's all good */
+ break;
+ }
+#endif
+ TRACE_DBG("res %d bufflen %d direct %d",
+ res, cmd->bufflen, cmd->data_direction);
+ return res;
+}
+
+/**
+ * scst_changer_generic_parse() - generic changer parse
+ *
+ * Generic parse() for changer devices
+ */
+int scst_changer_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd))
+{
+ int res = scst_null_parse(cmd);
+
+ if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_CHANGER_LONG_TIMEOUT;
+ else
+ cmd->timeout = SCST_GENERIC_CHANGER_TIMEOUT;
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_changer_generic_parse);
+
+/**
+ * scst_processor_generic_parse - generic SCSI processor parse
+ *
+ * Generic parse() for SCSI processor devices
+ */
+int scst_processor_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd))
+{
+ int res = scst_null_parse(cmd);
+
+ if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_PROCESSOR_LONG_TIMEOUT;
+ else
+ cmd->timeout = SCST_GENERIC_PROCESSOR_TIMEOUT;
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_processor_generic_parse);
+
+/**
+ * scst_raid_generic_parse() - generic RAID parse
+ *
+ * Generic parse() for RAID devices
+ */
+int scst_raid_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd))
+{
+ int res = scst_null_parse(cmd);
+
+ if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_RAID_LONG_TIMEOUT;
+ else
+ cmd->timeout = SCST_GENERIC_RAID_TIMEOUT;
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_raid_generic_parse);
+
+/**
+ ** Generic dev_done() support routines.
+ ** Done via pointer on functions to avoid unneeded dereferences on
+ ** the fast path.
+ **/
+
+/**
+ * scst_block_generic_dev_done() - generic SBC dev_done
+ *
+ * Generic dev_done() for block (SBC) devices
+ */
+int scst_block_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_shift)(struct scst_cmd *cmd, int block_shift))
+{
+ int opcode = cmd->cdb[0];
+ int status = cmd->status;
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ /*
+ * SCST sets good defaults for cmd->is_send_status and
+ * cmd->resp_data_len based on cmd->status and cmd->data_direction,
+ * therefore change them only if necessary
+ */
+
+ if ((status == SAM_STAT_GOOD) || (status == SAM_STAT_CONDITION_MET)) {
+ switch (opcode) {
+ case READ_CAPACITY:
+ {
+ /* Always keep track of disk capacity */
+ int buffer_size, sector_size, sh;
+ uint8_t *buffer;
+
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size <= 0)) {
+ if (buffer_size < 0) {
+ PRINT_ERROR("%s: Unable to get the"
+ " buffer (%d)", __func__, buffer_size);
+ }
+ goto out;
+ }
+
+ sector_size =
+ ((buffer[4] << 24) | (buffer[5] << 16) |
+ (buffer[6] << 8) | (buffer[7] << 0));
+ scst_put_buf(cmd, buffer);
+ if (sector_size != 0)
+ sh = scst_calc_block_shift(sector_size);
+ else
+ sh = 0;
+ set_block_shift(cmd, sh);
+ TRACE_DBG("block_shift %d", sh);
+ break;
+ }
+ default:
+ /* It's all good */
+ break;
+ }
+ }
+
+ TRACE_DBG("cmd->is_send_status=%x, cmd->resp_data_len=%d, "
+ "res=%d", cmd->is_send_status, cmd->resp_data_len, res);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_block_generic_dev_done);
+
+/**
+ * scst_tape_generic_dev_done() - generic tape dev done
+ *
+ * Generic dev_done() for tape devices
+ */
+int scst_tape_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_size)(struct scst_cmd *cmd, int block_shift))
+{
+ int opcode = cmd->cdb[0];
+ int res = SCST_CMD_STATE_DEFAULT;
+ int buffer_size, bs;
+ uint8_t *buffer = NULL;
+
+ /*
+ * SCST sets good defaults for cmd->is_send_status and
+ * cmd->resp_data_len based on cmd->status and cmd->data_direction,
+ * therefore change them only if necessary
+ */
+
+ if (cmd->status != SAM_STAT_GOOD)
+ goto out;
+
+ switch (opcode) {
+ case MODE_SENSE:
+ case MODE_SELECT:
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size <= 0)) {
+ if (buffer_size < 0) {
+ PRINT_ERROR("%s: Unable to get the buffer (%d)",
+ __func__, buffer_size);
+ }
+ goto out;
+ }
+ break;
+ }
+
+ switch (opcode) {
+ case MODE_SENSE:
+ TRACE_DBG("%s", "MODE_SENSE");
+ if ((cmd->cdb[2] & 0xC0) == 0) {
+ if (buffer[3] == 8) {
+ bs = (buffer[9] << 16) |
+ (buffer[10] << 8) | buffer[11];
+ set_block_size(cmd, bs);
+ }
+ }
+ break;
+ case MODE_SELECT:
+ TRACE_DBG("%s", "MODE_SELECT");
+ if (buffer[3] == 8) {
+ bs = (buffer[9] << 16) | (buffer[10] << 8) |
+ (buffer[11]);
+ set_block_size(cmd, bs);
+ }
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ switch (opcode) {
+ case MODE_SENSE:
+ case MODE_SELECT:
+ scst_put_buf(cmd, buffer);
+ break;
+ }
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_tape_generic_dev_done);
+
+static void scst_check_internal_sense(struct scst_device *dev, int result,
+ uint8_t *sense, int sense_len)
+{
+
+ if (host_byte(result) == DID_RESET) {
+ int sl;
+ TRACE(TRACE_MGMT, "DID_RESET received for device %s, "
+ "triggering reset UA", dev->virt_name);
+ sl = scst_set_sense(sense, sense_len, dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_reset_UA));
+ scst_dev_check_set_UA(dev, NULL, sense, sl);
+ } else if ((status_byte(result) == CHECK_CONDITION) &&
+ scst_is_ua_sense(sense, sense_len))
+ scst_dev_check_set_UA(dev, NULL, sense, sense_len);
+ return;
+}
+
+/**
+ * scst_to_dma_dir() - translate SCST's data direction to DMA direction
+ *
+ * Translates SCST's data direction to DMA one from backend storage
+ * perspective.
+ */
+enum dma_data_direction scst_to_dma_dir(int scst_dir)
+{
+ static const enum dma_data_direction tr_tbl[] = { DMA_NONE,
+ DMA_TO_DEVICE, DMA_FROM_DEVICE, DMA_BIDIRECTIONAL, DMA_NONE };
+
+ return tr_tbl[scst_dir];
+}
+EXPORT_SYMBOL(scst_to_dma_dir);
+
+/*
+ * scst_to_tgt_dma_dir() - translate SCST data direction to DMA direction
+ *
+ * Translates SCST data direction to DMA data direction from the perspective
+ * of a target.
+ */
+enum dma_data_direction scst_to_tgt_dma_dir(int scst_dir)
+{
+ static const enum dma_data_direction tr_tbl[] = { DMA_NONE,
+ DMA_FROM_DEVICE, DMA_TO_DEVICE, DMA_BIDIRECTIONAL, DMA_NONE };
+
+ return tr_tbl[scst_dir];
+}
+EXPORT_SYMBOL(scst_to_tgt_dma_dir);
+
+/**
+ * scst_obtain_device_parameters() - obtain device control parameters
+ *
+ * Issues a MODE SENSE for control mode page data and sets the corresponding
+ * dev's parameter from it. Returns 0 on success and not 0 otherwise.
+ */
+int scst_obtain_device_parameters(struct scst_device *dev)
+{
+ int rc, i;
+ uint8_t cmd[16];
+ uint8_t buffer[4+0x0A];
+ uint8_t sense_buffer[SCSI_SENSE_BUFFERSIZE];
+
+ EXTRACHECKS_BUG_ON(dev->scsi_dev == NULL);
+
+ for (i = 0; i < 5; i++) {
+ /* Get control mode page */
+ memset(cmd, 0, sizeof(cmd));
+#if 0
+ cmd[0] = MODE_SENSE_10;
+ cmd[1] = 0;
+ cmd[2] = 0x0A;
+ cmd[8] = sizeof(buffer); /* it's < 256 */
+#else
+ cmd[0] = MODE_SENSE;
+ cmd[1] = 8; /* DBD */
+ cmd[2] = 0x0A;
+ cmd[4] = sizeof(buffer);
+#endif
+
+ memset(buffer, 0, sizeof(buffer));
+ memset(sense_buffer, 0, sizeof(sense_buffer));
+
+ TRACE(TRACE_SCSI, "%s", "Doing internal MODE_SENSE");
+ rc = scsi_execute(dev->scsi_dev, cmd, SCST_DATA_READ, buffer,
+ sizeof(buffer), sense_buffer, 15, 0, 0
+ , NULL
+ );
+
+ TRACE_DBG("MODE_SENSE done: %x", rc);
+
+ if (scsi_status_is_good(rc)) {
+ int q;
+
+ PRINT_BUFF_FLAG(TRACE_SCSI, "Returned control mode "
+ "page data", buffer, sizeof(buffer));
+
+ dev->tst = buffer[4+2] >> 5;
+ q = buffer[4+3] >> 4;
+ if (q > SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER) {
+ PRINT_ERROR("Too big QUEUE ALG %x, dev %s",
+ dev->queue_alg, dev->virt_name);
+ }
+ dev->queue_alg = q;
+ dev->swp = (buffer[4+4] & 0x8) >> 3;
+ dev->tas = (buffer[4+5] & 0x40) >> 6;
+ dev->d_sense = (buffer[4+2] & 0x4) >> 2;
+
+ /*
+ * Unfortunately, SCSI ML doesn't provide a way to
+ * specify commands task attribute, so we can rely on
+ * device's restricted reordering only. Linux I/O
+ * subsystem doesn't reorder pass-through (PC) requests.
+ */
+ dev->has_own_order_mgmt = !dev->queue_alg;
+
+ PRINT_INFO("Device %s: TST %x, QUEUE ALG %x, SWP %x, "
+ "TAS %x, D_SENSE %d, has_own_order_mgmt %d",
+ dev->virt_name, dev->tst, dev->queue_alg,
+ dev->swp, dev->tas, dev->d_sense,
+ dev->has_own_order_mgmt);
+
+ goto out;
+ } else {
+ scst_check_internal_sense(dev, rc, sense_buffer,
+ sizeof(sense_buffer));
+#if 0
+ if ((status_byte(rc) == CHECK_CONDITION) &&
+ SCST_SENSE_VALID(sense_buffer)) {
+#else
+ /*
+ * 3ware controller is buggy and returns CONDITION_GOOD
+ * instead of CHECK_CONDITION
+ */
+ if (SCST_SENSE_VALID(sense_buffer)) {
+#endif
+ PRINT_BUFF_FLAG(TRACE_SCSI, "Returned sense "
+ "data", sense_buffer,
+ sizeof(sense_buffer));
+ if (scst_analyze_sense(sense_buffer,
+ sizeof(sense_buffer),
+ SCST_SENSE_KEY_VALID,
+ ILLEGAL_REQUEST, 0, 0)) {
+ PRINT_INFO("Device %s doesn't support "
+ "MODE SENSE", dev->virt_name);
+ break;
+ } else if (scst_analyze_sense(sense_buffer,
+ sizeof(sense_buffer),
+ SCST_SENSE_KEY_VALID,
+ NOT_READY, 0, 0)) {
+ PRINT_ERROR("Device %s not ready",
+ dev->virt_name);
+ break;
+ }
+ } else {
+ PRINT_INFO("Internal MODE SENSE to "
+ "device %s failed: %x",
+ dev->virt_name, rc);
+ PRINT_BUFF_FLAG(TRACE_SCSI, "MODE SENSE sense",
+ sense_buffer, sizeof(sense_buffer));
+ switch (host_byte(rc)) {
+ case DID_RESET:
+ case DID_ABORT:
+ case DID_SOFT_ERROR:
+ break;
+ default:
+ goto brk;
+ }
+ switch (driver_byte(rc)) {
+ case DRIVER_BUSY:
+ case DRIVER_SOFT:
+ break;
+ default:
+ goto brk;
+ }
+ }
+ }
+ }
+brk:
+ PRINT_WARNING("Unable to get device's %s control mode page, using "
+ "existing values/defaults: TST %x, QUEUE ALG %x, SWP %x, "
+ "TAS %x, D_SENSE %d, has_own_order_mgmt %d", dev->virt_name,
+ dev->tst, dev->queue_alg, dev->swp, dev->tas, dev->d_sense,
+ dev->has_own_order_mgmt);
+
+out:
+ return 0;
+}
+EXPORT_SYMBOL_GPL(scst_obtain_device_parameters);
+
+/* Called under dev_lock and BH off */
+void scst_process_reset(struct scst_device *dev,
+ struct scst_session *originator, struct scst_cmd *exclude_cmd,
+ struct scst_mgmt_cmd *mcmd, bool setUA)
+{
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_cmd *cmd, *tcmd;
+
+ /* Clear RESERVE'ation, if necessary */
+ if (dev->dev_reserved) {
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ TRACE_MGMT_DBG("Clearing RESERVE'ation for "
+ "tgt_dev LUN %lld",
+ (long long unsigned int)tgt_dev->lun);
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ /*
+ * There is no need to send RELEASE, since the device is going
+ * to be resetted. Actually, since we can be in RESET TM
+ * function, it might be dangerous.
+ */
+ }
+
+ dev->dev_double_ua_possible = 1;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ struct scst_session *sess = tgt_dev->sess;
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+
+ scst_free_all_UA(tgt_dev);
+
+ memset(tgt_dev->tgt_dev_sense, 0,
+ sizeof(tgt_dev->tgt_dev_sense));
+
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if (cmd == exclude_cmd)
+ continue;
+ if ((cmd->tgt_dev == tgt_dev) ||
+ ((cmd->tgt_dev == NULL) &&
+ (cmd->lun == tgt_dev->lun))) {
+ scst_abort_cmd(cmd, mcmd,
+ (tgt_dev->sess != originator), 0);
+ }
+ }
+ spin_unlock_irq(&sess->sess_list_lock);
+ }
+
+ list_for_each_entry_safe(cmd, tcmd, &dev->blocked_cmd_list,
+ blocked_cmd_list_entry) {
+ if (test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ list_del(&cmd->blocked_cmd_list_entry);
+ TRACE_MGMT_DBG("Adding aborted blocked cmd %p "
+ "to active cmd list", cmd);
+ spin_lock_irq(&cmd->cmd_threads->cmd_list_lock);
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&cmd->cmd_threads->cmd_list_lock);
+ }
+ }
+
+ if (setUA) {
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ dev->d_sense, SCST_LOAD_SENSE(scst_sense_reset_UA));
+ scst_dev_check_set_local_UA(dev, exclude_cmd, sense_buffer, sl);
+ }
+ return;
+}
+
+/* No locks, no IRQ or IRQ-disabled context allowed */
+int scst_set_pending_UA(struct scst_cmd *cmd)
+{
+ int res = 0, i;
+ struct scst_tgt_dev_UA *UA_entry;
+ bool first = true, global_unlock = false;
+ struct scst_session *sess = cmd->sess;
+
+ TRACE_MGMT_DBG("Setting pending UA cmd %p", cmd);
+
+ spin_lock_bh(&cmd->tgt_dev->tgt_dev_lock);
+
+again:
+ /* UA list could be cleared behind us, so retest */
+ if (list_empty(&cmd->tgt_dev->UA_list)) {
+ TRACE_DBG("%s",
+ "SCST_TGT_DEV_UA_PENDING set, but UA_list empty");
+ res = -1;
+ goto out_unlock;
+ }
+
+ UA_entry = list_entry(cmd->tgt_dev->UA_list.next, typeof(*UA_entry),
+ UA_list_entry);
+
+ TRACE_DBG("next %p UA_entry %p",
+ cmd->tgt_dev->UA_list.next, UA_entry);
+
+ if (UA_entry->global_UA && first) {
+ TRACE_MGMT_DBG("Global UA %p detected", UA_entry);
+
+ spin_unlock_bh(&cmd->tgt_dev->tgt_dev_lock);
+
+ /*
+ * cmd won't allow to suspend activities, so we can access
+ * sess->sess_tgt_dev_list_hash without any additional
+ * protection.
+ */
+
+ local_bh_disable();
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ /* Lockdep triggers here a false positive.. */
+ spin_lock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ first = false;
+ global_unlock = true;
+ goto again;
+ }
+
+ if (scst_set_cmd_error_sense(cmd, UA_entry->UA_sense_buffer,
+ UA_entry->UA_valid_sense_len) != 0)
+ goto out_unlock;
+
+ cmd->ua_ignore = 1;
+
+ list_del(&UA_entry->UA_list_entry);
+
+ if (UA_entry->global_UA) {
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ struct scst_tgt_dev_UA *ua;
+ list_for_each_entry(ua, &tgt_dev->UA_list,
+ UA_list_entry) {
+ if (ua->global_UA &&
+ memcmp(ua->UA_sense_buffer,
+ UA_entry->UA_sense_buffer,
+ sizeof(ua->UA_sense_buffer)) == 0) {
+ TRACE_MGMT_DBG("Freeing not "
+ "needed global UA %p",
+ ua);
+ list_del(&ua->UA_list_entry);
+ mempool_free(ua, scst_ua_mempool);
+ break;
+ }
+ }
+ }
+ }
+ }
+
+ mempool_free(UA_entry, scst_ua_mempool);
+
+ if (list_empty(&cmd->tgt_dev->UA_list)) {
+ clear_bit(SCST_TGT_DEV_UA_PENDING,
+ &cmd->tgt_dev->tgt_dev_flags);
+ }
+
+out_unlock:
+ if (global_unlock) {
+ for (i = TGT_DEV_HASH_SIZE-1; i >= 0; i--) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry_reverse(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ spin_unlock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ local_bh_enable();
+ spin_lock_bh(&cmd->tgt_dev->tgt_dev_lock);
+ }
+
+ spin_unlock_bh(&cmd->tgt_dev->tgt_dev_lock);
+ return res;
+}
+
+/* Called under tgt_dev_lock and BH off */
+static void scst_alloc_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags)
+{
+ struct scst_tgt_dev_UA *UA_entry = NULL;
+
+ UA_entry = mempool_alloc(scst_ua_mempool, GFP_ATOMIC);
+ if (UA_entry == NULL) {
+ PRINT_CRIT_ERROR("%s", "UNIT ATTENTION memory "
+ "allocation failed. The UNIT ATTENTION "
+ "on some sessions will be missed");
+ PRINT_BUFFER("Lost UA", sense, sense_len);
+ goto out;
+ }
+ memset(UA_entry, 0, sizeof(*UA_entry));
+
+ UA_entry->global_UA = (flags & SCST_SET_UA_FLAG_GLOBAL) != 0;
+ if (UA_entry->global_UA)
+ TRACE_MGMT_DBG("Queuing global UA %p", UA_entry);
+
+ if (sense_len > (int)sizeof(UA_entry->UA_sense_buffer)) {
+ PRINT_WARNING("Sense truncated (needed %d), shall you increase "
+ "SCST_SENSE_BUFFERSIZE?", sense_len);
+ sense_len = sizeof(UA_entry->UA_sense_buffer);
+ }
+ memcpy(UA_entry->UA_sense_buffer, sense, sense_len);
+ UA_entry->UA_valid_sense_len = sense_len;
+
+ set_bit(SCST_TGT_DEV_UA_PENDING, &tgt_dev->tgt_dev_flags);
+
+ TRACE_MGMT_DBG("Adding new UA to tgt_dev %p", tgt_dev);
+
+ if (flags & SCST_SET_UA_FLAG_AT_HEAD)
+ list_add(&UA_entry->UA_list_entry, &tgt_dev->UA_list);
+ else
+ list_add_tail(&UA_entry->UA_list_entry, &tgt_dev->UA_list);
+
+out:
+ return;
+}
+
+/* tgt_dev_lock supposed to be held and BH off */
+static void __scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags)
+{
+ int skip_UA = 0;
+ struct scst_tgt_dev_UA *UA_entry_tmp;
+ int len = min((int)sizeof(UA_entry_tmp->UA_sense_buffer), sense_len);
+
+ list_for_each_entry(UA_entry_tmp, &tgt_dev->UA_list,
+ UA_list_entry) {
+ if (memcmp(sense, UA_entry_tmp->UA_sense_buffer, len) == 0) {
+ TRACE_MGMT_DBG("%s", "UA already exists");
+ skip_UA = 1;
+ break;
+ }
+ }
+
+ if (skip_UA == 0)
+ scst_alloc_set_UA(tgt_dev, sense, len, flags);
+ return;
+}
+
+void scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags)
+{
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ __scst_check_set_UA(tgt_dev, sense, sense_len, flags);
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ return;
+}
+
+/* Called under dev_lock and BH off */
+void scst_dev_check_set_local_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len)
+{
+ struct scst_tgt_dev *tgt_dev, *exclude_tgt_dev = NULL;
+
+ if (exclude != NULL)
+ exclude_tgt_dev = exclude->tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (tgt_dev != exclude_tgt_dev)
+ scst_check_set_UA(tgt_dev, sense, sense_len, 0);
+ }
+ return;
+}
+
+/* Called under dev_lock and BH off */
+void __scst_dev_check_set_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len)
+{
+
+ TRACE_MGMT_DBG("Processing UA dev %p", dev);
+
+ /* Check for reset UA */
+ if (scst_analyze_sense(sense, sense_len, SCST_SENSE_ASC_VALID,
+ 0, SCST_SENSE_ASC_UA_RESET, 0))
+ scst_process_reset(dev,
+ (exclude != NULL) ? exclude->sess : NULL,
+ exclude, NULL, false);
+
+ scst_dev_check_set_local_UA(dev, exclude, sense, sense_len);
+ return;
+}
+
+/* Called under tgt_dev_lock or when tgt_dev is unused */
+static void scst_free_all_UA(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_tgt_dev_UA *UA_entry, *t;
+
+ list_for_each_entry_safe(UA_entry, t,
+ &tgt_dev->UA_list, UA_list_entry) {
+ TRACE_MGMT_DBG("Clearing UA for tgt_dev LUN %lld",
+ (long long unsigned int)tgt_dev->lun);
+ list_del(&UA_entry->UA_list_entry);
+ mempool_free(UA_entry, scst_ua_mempool);
+ }
+ INIT_LIST_HEAD(&tgt_dev->UA_list);
+ clear_bit(SCST_TGT_DEV_UA_PENDING, &tgt_dev->tgt_dev_flags);
+ return;
+}
+
+/* No locks */
+struct scst_cmd *__scst_check_deferred_commands(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_cmd *res = NULL, *cmd, *t;
+ typeof(tgt_dev->expected_sn) expected_sn = tgt_dev->expected_sn;
+
+ spin_lock_irq(&tgt_dev->sn_lock);
+
+ if (unlikely(tgt_dev->hq_cmd_count != 0))
+ goto out_unlock;
+
+restart:
+ list_for_each_entry_safe(cmd, t, &tgt_dev->deferred_cmd_list,
+ sn_cmd_list_entry) {
+ EXTRACHECKS_BUG_ON(cmd->queue_type ==
+ SCST_CMD_QUEUE_HEAD_OF_QUEUE);
+ if (cmd->sn == expected_sn) {
+ TRACE_SN("Deferred command %p (sn %d, set %d) found",
+ cmd, cmd->sn, cmd->sn_set);
+ tgt_dev->def_cmd_count--;
+ list_del(&cmd->sn_cmd_list_entry);
+ if (res == NULL)
+ res = cmd;
+ else {
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ TRACE_SN("Adding cmd %p to active cmd list",
+ cmd);
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ }
+ }
+ }
+ if (res != NULL)
+ goto out_unlock;
+
+ list_for_each_entry(cmd, &tgt_dev->skipped_sn_list,
+ sn_cmd_list_entry) {
+ EXTRACHECKS_BUG_ON(cmd->queue_type ==
+ SCST_CMD_QUEUE_HEAD_OF_QUEUE);
+ if (cmd->sn == expected_sn) {
+ atomic_t *slot = cmd->sn_slot;
+ /*
+ * !! At this point any pointer in cmd, except !!
+ * !! sn_slot and sn_cmd_list_entry, could be !!
+ * !! already destroyed !!
+ */
+ TRACE_SN("cmd %p (tag %llu) with skipped sn %d found",
+ cmd,
+ (long long unsigned int)cmd->tag,
+ cmd->sn);
+ tgt_dev->def_cmd_count--;
+ list_del(&cmd->sn_cmd_list_entry);
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ if (test_and_set_bit(SCST_CMD_CAN_BE_DESTROYED,
+ &cmd->cmd_flags))
+ scst_destroy_put_cmd(cmd);
+ scst_inc_expected_sn(tgt_dev, slot);
+ expected_sn = tgt_dev->expected_sn;
+ spin_lock_irq(&tgt_dev->sn_lock);
+ goto restart;
+ }
+ }
+
+out_unlock:
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ return res;
+}
+
+/*****************************************************************
+ ** The following thr_data functions are necessary, because the
+ ** kernel doesn't provide a better way to have threads local
+ ** storage
+ *****************************************************************/
+
+/**
+ * scst_add_thr_data() - add the current thread's local data
+ *
+ * Adds local to the current thread data to tgt_dev
+ * (they will be local for the tgt_dev and current thread).
+ */
+void scst_add_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct scst_thr_data_hdr *data,
+ void (*free_fn) (struct scst_thr_data_hdr *data))
+{
+ data->owner_thr = current;
+ atomic_set(&data->ref, 1);
+ EXTRACHECKS_BUG_ON(free_fn == NULL);
+ data->free_fn = free_fn;
+ spin_lock(&tgt_dev->thr_data_lock);
+ list_add_tail(&data->thr_data_list_entry, &tgt_dev->thr_data_list);
+ spin_unlock(&tgt_dev->thr_data_lock);
+}
+EXPORT_SYMBOL_GPL(scst_add_thr_data);
+
+/**
+ * scst_del_all_thr_data() - delete all thread's local data
+ *
+ * Deletes all local to threads data from tgt_dev
+ */
+void scst_del_all_thr_data(struct scst_tgt_dev *tgt_dev)
+{
+ spin_lock(&tgt_dev->thr_data_lock);
+ while (!list_empty(&tgt_dev->thr_data_list)) {
+ struct scst_thr_data_hdr *d = list_entry(
+ tgt_dev->thr_data_list.next, typeof(*d),
+ thr_data_list_entry);
+ list_del(&d->thr_data_list_entry);
+ spin_unlock(&tgt_dev->thr_data_lock);
+ scst_thr_data_put(d);
+ spin_lock(&tgt_dev->thr_data_lock);
+ }
+ spin_unlock(&tgt_dev->thr_data_lock);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_del_all_thr_data);
+
+/**
+ * scst_dev_del_all_thr_data() - delete all thread's local data from device
+ *
+ * Deletes all local to threads data from all tgt_dev's of the device
+ */
+void scst_dev_del_all_thr_data(struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_del_all_thr_data(tgt_dev);
+ }
+
+ mutex_unlock(&scst_mutex);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_dev_del_all_thr_data);
+
+/* thr_data_lock supposed to be held */
+static struct scst_thr_data_hdr *__scst_find_thr_data_locked(
+ struct scst_tgt_dev *tgt_dev, struct task_struct *tsk)
+{
+ struct scst_thr_data_hdr *res = NULL, *d;
+
+ list_for_each_entry(d, &tgt_dev->thr_data_list, thr_data_list_entry) {
+ if (d->owner_thr == tsk) {
+ res = d;
+ scst_thr_data_get(res);
+ break;
+ }
+ }
+ return res;
+}
+
+/**
+ * __scst_find_thr_data() - find local to the thread data
+ *
+ * Finds local to the thread data. Returns NULL, if they not found.
+ */
+struct scst_thr_data_hdr *__scst_find_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct task_struct *tsk)
+{
+ struct scst_thr_data_hdr *res;
+
+ spin_lock(&tgt_dev->thr_data_lock);
+ res = __scst_find_thr_data_locked(tgt_dev, tsk);
+ spin_unlock(&tgt_dev->thr_data_lock);
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(__scst_find_thr_data);
+
+bool scst_del_thr_data(struct scst_tgt_dev *tgt_dev, struct task_struct *tsk)
+{
+ bool res;
+ struct scst_thr_data_hdr *td;
+
+ spin_lock(&tgt_dev->thr_data_lock);
+
+ td = __scst_find_thr_data_locked(tgt_dev, tsk);
+ if (td != NULL) {
+ list_del(&td->thr_data_list_entry);
+ res = true;
+ } else
+ res = false;
+
+ spin_unlock(&tgt_dev->thr_data_lock);
+
+ if (td != NULL) {
+ /* the find() fn also gets it */
+ scst_thr_data_put(td);
+ scst_thr_data_put(td);
+ }
+
+ return res;
+}
+
+/* dev_lock supposed to be held and BH disabled */
+void __scst_block_dev(struct scst_device *dev)
+{
+ dev->block_count++;
+ TRACE_MGMT_DBG("Device BLOCK(new %d), dev %p", dev->block_count, dev);
+}
+
+/* No locks */
+static void scst_block_dev(struct scst_device *dev, int outstanding)
+{
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ spin_unlock_bh(&dev->dev_lock);
+
+ /*
+ * Memory barrier is necessary here, because we need to read
+ * on_dev_count in wait_event() below after we increased block_count.
+ * Otherwise, we can miss wake up in scst_dec_on_dev_cmd().
+ * We use the explicit barrier, because spin_unlock_bh() doesn't
+ * provide the necessary memory barrier functionality.
+ */
+ smp_mb();
+
+ TRACE_MGMT_DBG("Waiting during blocking outstanding %d (on_dev_count "
+ "%d)", outstanding, atomic_read(&dev->on_dev_count));
+ wait_event(dev->on_dev_waitQ,
+ atomic_read(&dev->on_dev_count) <= outstanding);
+ TRACE_MGMT_DBG("%s", "wait_event() returned");
+}
+
+/* No locks */
+void scst_block_dev_cmd(struct scst_cmd *cmd, int outstanding)
+{
+ BUG_ON(cmd->needs_unblocking);
+
+ cmd->needs_unblocking = 1;
+ TRACE_MGMT_DBG("Needs unblocking cmd %p (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ scst_block_dev(cmd->dev, outstanding);
+}
+
+/* No locks */
+void scst_unblock_dev(struct scst_device *dev)
+{
+ spin_lock_bh(&dev->dev_lock);
+ TRACE_MGMT_DBG("Device UNBLOCK(new %d), dev %p",
+ dev->block_count-1, dev);
+ if (--dev->block_count == 0)
+ scst_unblock_cmds(dev);
+ spin_unlock_bh(&dev->dev_lock);
+ BUG_ON(dev->block_count < 0);
+}
+
+/* No locks */
+void scst_unblock_dev_cmd(struct scst_cmd *cmd)
+{
+ scst_unblock_dev(cmd->dev);
+ cmd->needs_unblocking = 0;
+}
+
+/* No locks */
+int scst_inc_on_dev_cmd(struct scst_cmd *cmd)
+{
+ int res = 0;
+ struct scst_device *dev = cmd->dev;
+
+ BUG_ON(cmd->inc_blocking || cmd->dec_on_dev_needed);
+
+ atomic_inc(&dev->on_dev_count);
+ cmd->dec_on_dev_needed = 1;
+ TRACE_DBG("New on_dev_count %d", atomic_read(&dev->on_dev_count));
+
+ if (unlikely(cmd->internal) && (cmd->cdb[0] == REQUEST_SENSE)) {
+ /*
+ * The original command can already block the device, so
+ * REQUEST SENSE command should always pass.
+ */
+ goto out;
+ }
+
+repeat:
+ if (unlikely(dev->block_count > 0)) {
+ spin_lock_bh(&dev->dev_lock);
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)))
+ goto out_unlock;
+ if (dev->block_count > 0) {
+ scst_dec_on_dev_cmd(cmd);
+ TRACE_MGMT_DBG("Delaying cmd %p due to blocking "
+ "(tag %llu, dev %p)", cmd,
+ (long long unsigned int)cmd->tag, dev);
+ list_add_tail(&cmd->blocked_cmd_list_entry,
+ &dev->blocked_cmd_list);
+ res = 1;
+ spin_unlock_bh(&dev->dev_lock);
+ goto out;
+ } else {
+ TRACE_MGMT_DBG("%s", "Somebody unblocked the device, "
+ "continuing");
+ }
+ spin_unlock_bh(&dev->dev_lock);
+ }
+ if (unlikely(dev->dev_double_ua_possible)) {
+ spin_lock_bh(&dev->dev_lock);
+ if (dev->block_count == 0) {
+ TRACE_MGMT_DBG("cmd %p (tag %llu), blocking further "
+ "cmds due to possible double reset UA (dev %p)",
+ cmd, (long long unsigned int)cmd->tag, dev);
+ __scst_block_dev(dev);
+ cmd->inc_blocking = 1;
+ } else {
+ spin_unlock_bh(&dev->dev_lock);
+ TRACE_MGMT_DBG("Somebody blocked the device, "
+ "repeating (count %d)", dev->block_count);
+ goto repeat;
+ }
+ spin_unlock_bh(&dev->dev_lock);
+ }
+
+out:
+ return res;
+
+out_unlock:
+ spin_unlock_bh(&dev->dev_lock);
+ goto out;
+}
+
+/* Called under dev_lock */
+static void scst_unblock_cmds(struct scst_device *dev)
+{
+ struct scst_cmd *cmd, *tcmd;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ list_for_each_entry_safe(cmd, tcmd, &dev->blocked_cmd_list,
+ blocked_cmd_list_entry) {
+ list_del(&cmd->blocked_cmd_list_entry);
+ TRACE_MGMT_DBG("Adding blocked cmd %p to active cmd list", cmd);
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ }
+ local_irq_restore(flags);
+ return;
+}
+
+static void __scst_unblock_deferred(struct scst_tgt_dev *tgt_dev,
+ struct scst_cmd *out_of_sn_cmd)
+{
+ EXTRACHECKS_BUG_ON(!out_of_sn_cmd->sn_set);
+
+ if (out_of_sn_cmd->sn == tgt_dev->expected_sn) {
+ scst_inc_expected_sn(tgt_dev, out_of_sn_cmd->sn_slot);
+ scst_make_deferred_commands_active(tgt_dev);
+ } else {
+ out_of_sn_cmd->out_of_sn = 1;
+ spin_lock_irq(&tgt_dev->sn_lock);
+ tgt_dev->def_cmd_count++;
+ list_add_tail(&out_of_sn_cmd->sn_cmd_list_entry,
+ &tgt_dev->skipped_sn_list);
+ TRACE_SN("out_of_sn_cmd %p with sn %d added to skipped_sn_list"
+ " (expected_sn %d)", out_of_sn_cmd, out_of_sn_cmd->sn,
+ tgt_dev->expected_sn);
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ }
+
+ return;
+}
+
+void scst_unblock_deferred(struct scst_tgt_dev *tgt_dev,
+ struct scst_cmd *out_of_sn_cmd)
+{
+
+ if (!out_of_sn_cmd->sn_set) {
+ TRACE_SN("cmd %p without sn", out_of_sn_cmd);
+ goto out;
+ }
+
+ __scst_unblock_deferred(tgt_dev, out_of_sn_cmd);
+
+out:
+ return;
+}
+
+void scst_on_hq_cmd_response(struct scst_cmd *cmd)
+{
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ if (!cmd->hq_cmd_inced)
+ goto out;
+
+ spin_lock_irq(&tgt_dev->sn_lock);
+ tgt_dev->hq_cmd_count--;
+ spin_unlock_irq(&tgt_dev->sn_lock);
+
+ EXTRACHECKS_BUG_ON(tgt_dev->hq_cmd_count < 0);
+
+ /*
+ * There is no problem in checking hq_cmd_count in the
+ * non-locked state. In the worst case we will only have
+ * unneeded run of the deferred commands.
+ */
+ if (tgt_dev->hq_cmd_count == 0)
+ scst_make_deferred_commands_active(tgt_dev);
+
+out:
+ return;
+}
+
+void scst_store_sense(struct scst_cmd *cmd)
+{
+
+ if (SCST_SENSE_VALID(cmd->sense) &&
+ !test_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags) &&
+ (cmd->tgt_dev != NULL)) {
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ TRACE_DBG("Storing sense (cmd %p)", cmd);
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+
+ if (cmd->sense_valid_len <= sizeof(tgt_dev->tgt_dev_sense))
+ tgt_dev->tgt_dev_valid_sense_len = cmd->sense_valid_len;
+ else {
+ tgt_dev->tgt_dev_valid_sense_len = sizeof(tgt_dev->tgt_dev_sense);
+ PRINT_ERROR("Stored sense truncated to size %d "
+ "(needed %d)", tgt_dev->tgt_dev_valid_sense_len,
+ cmd->sense_valid_len);
+ }
+ memcpy(tgt_dev->tgt_dev_sense, cmd->sense,
+ tgt_dev->tgt_dev_valid_sense_len);
+
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ }
+ return;
+}
+
+void scst_xmit_process_aborted_cmd(struct scst_cmd *cmd)
+{
+
+ TRACE_MGMT_DBG("Aborted cmd %p done (cmd_ref %d, "
+ "scst_cmd_count %d)", cmd, atomic_read(&cmd->cmd_ref),
+ atomic_read(&scst_cmd_count));
+
+ scst_done_cmd_mgmt(cmd);
+
+ if (test_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags)) {
+ if (cmd->completed) {
+ /* It's completed and it's OK to return its result */
+ goto out;
+ }
+
+ /* For not yet inited commands cmd->dev can be NULL here */
+ if (test_bit(SCST_CMD_DEVICE_TAS, &cmd->cmd_flags)) {
+ TRACE_MGMT_DBG("Flag ABORTED OTHER set for cmd %p "
+ "(tag %llu), returning TASK ABORTED ", cmd,
+ (long long unsigned int)cmd->tag);
+ scst_set_cmd_error_status(cmd, SAM_STAT_TASK_ABORTED);
+ } else {
+ TRACE_MGMT_DBG("Flag ABORTED OTHER set for cmd %p "
+ "(tag %llu), aborting without delivery or "
+ "notification",
+ cmd, (long long unsigned int)cmd->tag);
+ /*
+ * There is no need to check/requeue possible UA,
+ * because, if it exists, it will be delivered
+ * by the "completed" branch above.
+ */
+ clear_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+ }
+ }
+
+out:
+ return;
+}
+
+/**
+ * scst_get_max_lun_commands() - return maximum supported commands count
+ *
+ * Returns maximum commands count which can be queued to this LUN in this
+ * session.
+ *
+ * If lun is NO_SUCH_LUN, returns minimum of maximum commands count which
+ * can be queued to any LUN in this session.
+ *
+ * If sess is NULL, returns minimum of maximum commands count which can be
+ * queued to any SCST device.
+ */
+int scst_get_max_lun_commands(struct scst_session *sess, uint64_t lun)
+{
+ return SCST_MAX_TGT_DEV_COMMANDS;
+}
+EXPORT_SYMBOL(scst_get_max_lun_commands);
+
+/**
+ * scst_reassign_persistent_sess_states() - reassigns persistent states
+ *
+ * Reassigns persistent states from old_sess to new_sess.
+ */
+void scst_reassign_persistent_sess_states(struct scst_session *new_sess,
+ struct scst_session *old_sess)
+{
+ struct scst_device *dev;
+
+ TRACE_PR("Reassigning persistent states from old_sess %p to "
+ "new_sess %p", old_sess, new_sess);
+
+ if ((new_sess == NULL) || (old_sess == NULL)) {
+ TRACE_DBG("%s", "new_sess or old_sess is NULL");
+ goto out;
+ }
+
+ if (new_sess == old_sess) {
+ TRACE_DBG("%s", "new_sess or old_sess are the same");
+ goto out;
+ }
+
+ if ((new_sess->transport_id == NULL) ||
+ (old_sess->transport_id == NULL)) {
+ TRACE_DBG("%s", "new_sess or old_sess doesn't support PRs");
+ goto out;
+ }
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_tgt_dev *new_tgt_dev = NULL, *old_tgt_dev = NULL;
+
+ TRACE_DBG("Processing dev %s", dev->virt_name);
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (tgt_dev->sess == new_sess) {
+ new_tgt_dev = tgt_dev;
+ if (old_tgt_dev != NULL)
+ break;
+ }
+ if (tgt_dev->sess == old_sess) {
+ old_tgt_dev = tgt_dev;
+ if (new_tgt_dev != NULL)
+ break;
+ }
+ }
+
+ if ((new_tgt_dev == NULL) || (old_tgt_dev == NULL)) {
+ TRACE_DBG("new_tgt_dev %p or old_sess %p is NULL, "
+ "skipping (dev %s)", new_tgt_dev, old_tgt_dev,
+ dev->virt_name);
+ continue;
+ }
+
+ scst_pr_write_lock(dev);
+
+ if (old_tgt_dev->registrant != NULL) {
+ TRACE_PR("Reassigning reg %p from tgt_dev %p to %p",
+ old_tgt_dev->registrant, old_tgt_dev,
+ new_tgt_dev);
+
+ if (new_tgt_dev->registrant != NULL)
+ new_tgt_dev->registrant->tgt_dev = NULL;
+
+ new_tgt_dev->registrant = old_tgt_dev->registrant;
+ new_tgt_dev->registrant->tgt_dev = new_tgt_dev;
+
+ old_tgt_dev->registrant = NULL;
+ }
+
+ scst_pr_write_unlock(dev);
+ }
+
+ mutex_unlock(&scst_mutex);
+
+out:
+ return;
+}
+EXPORT_SYMBOL(scst_reassign_persistent_sess_states);
+
+/**
+ * scst_get_next_lexem() - parse and return next lexem in the string
+ *
+ * Returns pointer to the next lexem from token_str skipping
+ * spaces and '=' character and using them then as a delimeter. Content
+ * of token_str is modified by setting '\0' at the delimeter's position.
+ */
+char *scst_get_next_lexem(char **token_str)
+{
+ char *p = *token_str;
+ char *q;
+ static const char blank = '\0';
+
+ if ((token_str == NULL) || (*token_str == NULL))
+ return (char *)␣
+
+ for (p = *token_str; (*p != '\0') && (isspace(*p) || (*p == '=')); p++)
+ ;
+
+ for (q = p; (*q != '\0') && !isspace(*q) && (*q != '='); q++)
+ ;
+
+ if (*q != '\0')
+ *q++ = '\0';
+
+ *token_str = q;
+ return p;
+}
+EXPORT_SYMBOL_GPL(scst_get_next_lexem);
+
+/**
+ * scst_restore_token_str() - restore string modified by scst_get_next_lexem()
+ *
+ * Restores token_str modified by scst_get_next_lexem() to the
+ * previous value before scst_get_next_lexem() was called. Prev_lexem is
+ * a pointer to lexem returned by scst_get_next_lexem().
+ */
+void scst_restore_token_str(char *prev_lexem, char *token_str)
+{
+ if (&prev_lexem[strlen(prev_lexem)] != token_str)
+ prev_lexem[strlen(prev_lexem)] = ' ';
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_restore_token_str);
+
+/**
+ * scst_get_next_token_str() - parse and return next token
+ *
+ * This function returns pointer to the next token strings from input_str
+ * using '\n', ';' and '\0' as a delimeter. Content of input_str is
+ * modified by setting '\0' at the delimeter's position.
+ */
+char *scst_get_next_token_str(char **input_str)
+{
+ char *p = *input_str;
+ int i = 0;
+
+ while ((p[i] != '\n') && (p[i] != ';') && (p[i] != '\0'))
+ i++;
+
+ if (i == 0)
+ return NULL;
+
+ if (p[i] == '\0')
+ *input_str = &p[i];
+ else
+ *input_str = &p[i+1];
+
+ p[i] = '\0';
+
+ return p;
+}
+EXPORT_SYMBOL_GPL(scst_get_next_token_str);
+
+static void __init scst_scsi_op_list_init(void)
+{
+ int i;
+ uint8_t op = 0xff;
+
+ for (i = 0; i < 256; i++)
+ scst_scsi_op_list[i] = SCST_CDB_TBL_SIZE;
+
+ for (i = 0; i < SCST_CDB_TBL_SIZE; i++) {
+ if (scst_scsi_op_table[i].ops != op) {
+ op = scst_scsi_op_table[i].ops;
+ scst_scsi_op_list[op] = i;
+ }
+ }
+ return;
+}
+
+int __init scst_lib_init(void)
+{
+ int res = 0;
+
+ scst_scsi_op_list_init();
+
+ scsi_io_context_cache = kmem_cache_create("scst_scsi_io_context",
+ sizeof(struct scsi_io_context),
+ 0, 0, NULL);
+ if (!scsi_io_context_cache) {
+ PRINT_ERROR("%s", "Can't init scsi io context cache");
+ res = -ENOMEM;
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+void scst_lib_exit(void)
+{
+ BUILD_BUG_ON(SCST_MAX_CDB_SIZE != BLK_MAX_CDB);
+ BUILD_BUG_ON(SCST_SENSE_BUFFERSIZE < SCSI_SENSE_BUFFERSIZE);
+
+ kmem_cache_destroy(scsi_io_context_cache);
+}
+
+#ifdef CONFIG_SCST_DEBUG
+
+/**
+ * scst_random() - return a pseudo-random number for debugging purposes.
+ *
+ * Returns a pseudo-random number for debugging purposes. Available only in
+ * the DEBUG build.
+ *
+ * Original taken from the XFS code
+ */
+unsigned long scst_random(void)
+{
+ static int Inited;
+ static unsigned long RandomValue;
+ static DEFINE_SPINLOCK(lock);
+ /* cycles pseudo-randomly through all values between 1 and 2^31 - 2 */
+ register long rv;
+ register long lo;
+ register long hi;
+ unsigned long flags;
+
+ spin_lock_irqsave(&lock, flags);
+ if (!Inited) {
+ RandomValue = jiffies;
+ Inited = 1;
+ }
+ rv = RandomValue;
+ hi = rv / 127773;
+ lo = rv % 127773;
+ rv = 16807 * lo - 2836 * hi;
+ if (rv <= 0)
+ rv += 2147483647;
+ RandomValue = rv;
+ spin_unlock_irqrestore(&lock, flags);
+ return rv;
+}
+EXPORT_SYMBOL_GPL(scst_random);
+#endif /* CONFIG_SCST_DEBUG */
+
+#ifdef CONFIG_SCST_DEBUG_TM
+
+#define TM_DBG_STATE_ABORT 0
+#define TM_DBG_STATE_RESET 1
+#define TM_DBG_STATE_OFFLINE 2
+
+#define INIT_TM_DBG_STATE TM_DBG_STATE_ABORT
+
+static void tm_dbg_timer_fn(unsigned long arg);
+
+static DEFINE_SPINLOCK(scst_tm_dbg_lock);
+/* All serialized by scst_tm_dbg_lock */
+static struct {
+ unsigned int tm_dbg_release:1;
+ unsigned int tm_dbg_blocked:1;
+} tm_dbg_flags;
+static LIST_HEAD(tm_dbg_delayed_cmd_list);
+static int tm_dbg_delayed_cmds_count;
+static int tm_dbg_passed_cmds_count;
+static int tm_dbg_state;
+static int tm_dbg_on_state_passes;
+static DEFINE_TIMER(tm_dbg_timer, tm_dbg_timer_fn, 0, 0);
+static struct scst_tgt_dev *tm_dbg_tgt_dev;
+
+static const int tm_dbg_on_state_num_passes[] = { 5, 1, 0x7ffffff };
+
+static void tm_dbg_init_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+ if (tgt_dev->lun == 6) {
+ unsigned long flags;
+
+ if (tm_dbg_tgt_dev != NULL)
+ tm_dbg_deinit_tgt_dev(tm_dbg_tgt_dev);
+
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ tm_dbg_state = INIT_TM_DBG_STATE;
+ tm_dbg_on_state_passes =
+ tm_dbg_on_state_num_passes[tm_dbg_state];
+ tm_dbg_tgt_dev = tgt_dev;
+ PRINT_INFO("LUN %lld connected from initiator %s is under "
+ "TM debugging (tgt_dev %p)",
+ (unsigned long long)tgt_dev->lun,
+ tgt_dev->sess->initiator_name, tgt_dev);
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ }
+ return;
+}
+
+static void tm_dbg_deinit_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+ if (tm_dbg_tgt_dev == tgt_dev) {
+ unsigned long flags;
+ TRACE_MGMT_DBG("Deinit TM debugging tgt_dev %p", tgt_dev);
+ del_timer_sync(&tm_dbg_timer);
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ tm_dbg_tgt_dev = NULL;
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ }
+ return;
+}
+
+static void tm_dbg_timer_fn(unsigned long arg)
+{
+ TRACE_MGMT_DBG("%s", "delayed cmd timer expired");
+ tm_dbg_flags.tm_dbg_release = 1;
+ /* Used to make sure that all woken up threads see the new value */
+ smp_wmb();
+ wake_up_all(&tm_dbg_tgt_dev->active_cmd_threads->cmd_list_waitQ);
+ return;
+}
+
+/* Called under scst_tm_dbg_lock and IRQs off */
+static void tm_dbg_delay_cmd(struct scst_cmd *cmd)
+{
+ switch (tm_dbg_state) {
+ case TM_DBG_STATE_ABORT:
+ if (tm_dbg_delayed_cmds_count == 0) {
+ unsigned long d = 58*HZ + (scst_random() % (4*HZ));
+ TRACE_MGMT_DBG("STATE ABORT: delaying cmd %p (tag %llu)"
+ " for %ld.%ld seconds (%ld HZ), "
+ "tm_dbg_on_state_passes=%d", cmd, cmd->tag,
+ d/HZ, (d%HZ)*100/HZ, d, tm_dbg_on_state_passes);
+ mod_timer(&tm_dbg_timer, jiffies + d);
+#if 0
+ tm_dbg_flags.tm_dbg_blocked = 1;
+#endif
+ } else {
+ TRACE_MGMT_DBG("Delaying another timed cmd %p "
+ "(tag %llu), delayed_cmds_count=%d, "
+ "tm_dbg_on_state_passes=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count,
+ tm_dbg_on_state_passes);
+ if (tm_dbg_delayed_cmds_count == 2)
+ tm_dbg_flags.tm_dbg_blocked = 0;
+ }
+ break;
+
+ case TM_DBG_STATE_RESET:
+ case TM_DBG_STATE_OFFLINE:
+ TRACE_MGMT_DBG("STATE RESET/OFFLINE: delaying cmd %p "
+ "(tag %llu), delayed_cmds_count=%d, "
+ "tm_dbg_on_state_passes=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count, tm_dbg_on_state_passes);
+ tm_dbg_flags.tm_dbg_blocked = 1;
+ break;
+
+ default:
+ BUG();
+ }
+ /* IRQs already off */
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_add_tail(&cmd->cmd_list_entry, &tm_dbg_delayed_cmd_list);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ cmd->tm_dbg_delayed = 1;
+ tm_dbg_delayed_cmds_count++;
+ return;
+}
+
+/* No locks */
+void tm_dbg_check_released_cmds(void)
+{
+ if (tm_dbg_flags.tm_dbg_release) {
+ struct scst_cmd *cmd, *tc;
+ spin_lock_irq(&scst_tm_dbg_lock);
+ list_for_each_entry_safe_reverse(cmd, tc,
+ &tm_dbg_delayed_cmd_list, cmd_list_entry) {
+ TRACE_MGMT_DBG("Releasing timed cmd %p (tag %llu), "
+ "delayed_cmds_count=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count);
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_move(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ }
+ tm_dbg_flags.tm_dbg_release = 0;
+ spin_unlock_irq(&scst_tm_dbg_lock);
+ }
+}
+
+/* Called under scst_tm_dbg_lock */
+static void tm_dbg_change_state(void)
+{
+ tm_dbg_flags.tm_dbg_blocked = 0;
+ if (--tm_dbg_on_state_passes == 0) {
+ switch (tm_dbg_state) {
+ case TM_DBG_STATE_ABORT:
+ TRACE_MGMT_DBG("%s", "Changing "
+ "tm_dbg_state to RESET");
+ tm_dbg_state = TM_DBG_STATE_RESET;
+ tm_dbg_flags.tm_dbg_blocked = 0;
+ break;
+ case TM_DBG_STATE_RESET:
+ case TM_DBG_STATE_OFFLINE:
+#ifdef CONFIG_SCST_TM_DBG_GO_OFFLINE
+ TRACE_MGMT_DBG("%s", "Changing "
+ "tm_dbg_state to OFFLINE");
+ tm_dbg_state = TM_DBG_STATE_OFFLINE;
+#else
+ TRACE_MGMT_DBG("%s", "Changing "
+ "tm_dbg_state to ABORT");
+ tm_dbg_state = TM_DBG_STATE_ABORT;
+#endif
+ break;
+ default:
+ BUG();
+ }
+ tm_dbg_on_state_passes =
+ tm_dbg_on_state_num_passes[tm_dbg_state];
+ }
+
+ TRACE_MGMT_DBG("%s", "Deleting timer");
+ del_timer_sync(&tm_dbg_timer);
+ return;
+}
+
+/* No locks */
+int tm_dbg_check_cmd(struct scst_cmd *cmd)
+{
+ int res = 0;
+ unsigned long flags;
+
+ if (cmd->tm_dbg_immut)
+ goto out;
+
+ if (cmd->tm_dbg_delayed) {
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ TRACE_MGMT_DBG("Processing delayed cmd %p (tag %llu), "
+ "delayed_cmds_count=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count);
+
+ cmd->tm_dbg_immut = 1;
+ tm_dbg_delayed_cmds_count--;
+ if ((tm_dbg_delayed_cmds_count == 0) &&
+ (tm_dbg_state == TM_DBG_STATE_ABORT))
+ tm_dbg_change_state();
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ } else if (cmd->tgt_dev && (tm_dbg_tgt_dev == cmd->tgt_dev)) {
+ /* Delay 50th command */
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ if (tm_dbg_flags.tm_dbg_blocked ||
+ (++tm_dbg_passed_cmds_count % 50) == 0) {
+ tm_dbg_delay_cmd(cmd);
+ res = 1;
+ } else
+ cmd->tm_dbg_immut = 1;
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ }
+
+out:
+ return res;
+}
+
+/* No locks */
+void tm_dbg_release_cmd(struct scst_cmd *cmd)
+{
+ struct scst_cmd *c;
+ unsigned long flags;
+
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ list_for_each_entry(c, &tm_dbg_delayed_cmd_list,
+ cmd_list_entry) {
+ if (c == cmd) {
+ TRACE_MGMT_DBG("Abort request for "
+ "delayed cmd %p (tag=%llu), moving it to "
+ "active cmd list (delayed_cmds_count=%d)",
+ c, c->tag, tm_dbg_delayed_cmds_count);
+
+ if (!test_bit(SCST_CMD_ABORTED_OTHER,
+ &cmd->cmd_flags)) {
+ /* Test how completed commands handled */
+ if (((scst_random() % 10) == 5)) {
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(
+ scst_sense_hardw_error));
+ /* It's completed now */
+ }
+ }
+
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_move(&c->cmd_list_entry,
+ &c->cmd_threads->active_cmd_list);
+ wake_up(&c->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ return;
+}
+
+/* Might be called under scst_mutex */
+void tm_dbg_task_mgmt(struct scst_device *dev, const char *fn, int force)
+{
+ unsigned long flags;
+
+ if (dev != NULL) {
+ if (tm_dbg_tgt_dev == NULL)
+ goto out;
+
+ if (tm_dbg_tgt_dev->dev != dev)
+ goto out;
+ }
+
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ if ((tm_dbg_state != TM_DBG_STATE_OFFLINE) || force) {
+ TRACE_MGMT_DBG("%s: freeing %d delayed cmds", fn,
+ tm_dbg_delayed_cmds_count);
+ tm_dbg_change_state();
+ tm_dbg_flags.tm_dbg_release = 1;
+ /*
+ * Used to make sure that all woken up threads see the new
+ * value.
+ */
+ smp_wmb();
+ if (tm_dbg_tgt_dev != NULL)
+ wake_up_all(&tm_dbg_tgt_dev->active_cmd_threads->cmd_list_waitQ);
+ } else {
+ TRACE_MGMT_DBG("%s: while OFFLINE state, doing nothing", fn);
+ }
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+
+out:
+ return;
+}
+
+int tm_dbg_is_release(void)
+{
+ return tm_dbg_flags.tm_dbg_release;
+}
+#endif /* CONFIG_SCST_DEBUG_TM */
+
+#ifdef CONFIG_SCST_DEBUG_SN
+void scst_check_debug_sn(struct scst_cmd *cmd)
+{
+ static DEFINE_SPINLOCK(lock);
+ static int type;
+ static int cnt;
+ unsigned long flags;
+ int old = cmd->queue_type;
+
+ spin_lock_irqsave(&lock, flags);
+
+ if (cnt == 0) {
+ if ((scst_random() % 1000) == 500) {
+ if ((scst_random() % 3) == 1)
+ type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ else
+ type = SCST_CMD_QUEUE_ORDERED;
+ do {
+ cnt = scst_random() % 10;
+ } while (cnt == 0);
+ } else
+ goto out_unlock;
+ }
+
+ cmd->queue_type = type;
+ cnt--;
+
+ if (((scst_random() % 1000) == 750))
+ cmd->queue_type = SCST_CMD_QUEUE_ORDERED;
+ else if (((scst_random() % 1000) == 751))
+ cmd->queue_type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ else if (((scst_random() % 1000) == 752))
+ cmd->queue_type = SCST_CMD_QUEUE_SIMPLE;
+
+ TRACE_SN("DbgSN changed cmd %p: %d/%d (cnt %d)", cmd, old,
+ cmd->queue_type, cnt);
+
+out_unlock:
+ spin_unlock_irqrestore(&lock, flags);
+ return;
+}
+#endif /* CONFIG_SCST_DEBUG_SN */
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+static uint64_t scst_get_nsec(void)
+{
+ struct timespec ts;
+ ktime_get_ts(&ts);
+ return (uint64_t)ts.tv_sec * 1000000000 + ts.tv_nsec;
+}
+
+void scst_set_start_time(struct scst_cmd *cmd)
+{
+ cmd->start = scst_get_nsec();
+ TRACE_DBG("cmd %p: start %lld", cmd, cmd->start);
+}
+
+void scst_set_cur_start(struct scst_cmd *cmd)
+{
+ cmd->curr_start = scst_get_nsec();
+ TRACE_DBG("cmd %p: cur_start %lld", cmd, cmd->curr_start);
+}
+
+void scst_set_parse_time(struct scst_cmd *cmd)
+{
+ cmd->parse_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: parse_time %lld", cmd, cmd->parse_time);
+}
+
+void scst_set_alloc_buf_time(struct scst_cmd *cmd)
+{
+ cmd->alloc_buf_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: alloc_buf_time %lld", cmd, cmd->alloc_buf_time);
+}
+
+void scst_set_restart_waiting_time(struct scst_cmd *cmd)
+{
+ cmd->restart_waiting_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: restart_waiting_time %lld", cmd,
+ cmd->restart_waiting_time);
+}
+
+void scst_set_rdy_to_xfer_time(struct scst_cmd *cmd)
+{
+ cmd->rdy_to_xfer_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: rdy_to_xfer_time %lld", cmd, cmd->rdy_to_xfer_time);
+}
+
+void scst_set_pre_exec_time(struct scst_cmd *cmd)
+{
+ cmd->pre_exec_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: pre_exec_time %lld", cmd, cmd->pre_exec_time);
+}
+
+void scst_set_exec_time(struct scst_cmd *cmd)
+{
+ cmd->exec_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: exec_time %lld", cmd, cmd->exec_time);
+}
+
+void scst_set_dev_done_time(struct scst_cmd *cmd)
+{
+ cmd->dev_done_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: dev_done_time %lld", cmd, cmd->dev_done_time);
+}
+
+void scst_set_xmit_time(struct scst_cmd *cmd)
+{
+ cmd->xmit_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: xmit_time %lld", cmd, cmd->xmit_time);
+}
+
+void scst_set_tgt_on_free_time(struct scst_cmd *cmd)
+{
+ cmd->tgt_on_free_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: tgt_on_free_time %lld", cmd, cmd->tgt_on_free_time);
+}
+
+void scst_set_dev_on_free_time(struct scst_cmd *cmd)
+{
+ cmd->dev_on_free_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: dev_on_free_time %lld", cmd, cmd->dev_on_free_time);
+}
+
+void scst_update_lat_stats(struct scst_cmd *cmd)
+{
+ uint64_t finish, scst_time, tgt_time, dev_time;
+ struct scst_session *sess = cmd->sess;
+ int data_len;
+ int i;
+ struct scst_ext_latency_stat *latency_stat, *dev_latency_stat;
+
+ finish = scst_get_nsec();
+
+ /* Determine the IO size for extended latency statistics */
+ data_len = cmd->bufflen;
+ i = SCST_LATENCY_STAT_INDEX_OTHER;
+ if (data_len <= SCST_IO_SIZE_THRESHOLD_SMALL)
+ i = SCST_LATENCY_STAT_INDEX_SMALL;
+ else if (data_len <= SCST_IO_SIZE_THRESHOLD_MEDIUM)
+ i = SCST_LATENCY_STAT_INDEX_MEDIUM;
+ else if (data_len <= SCST_IO_SIZE_THRESHOLD_LARGE)
+ i = SCST_LATENCY_STAT_INDEX_LARGE;
+ else if (data_len <= SCST_IO_SIZE_THRESHOLD_VERY_LARGE)
+ i = SCST_LATENCY_STAT_INDEX_VERY_LARGE;
+ latency_stat = &sess->sess_latency_stat[i];
+ dev_latency_stat = &cmd->tgt_dev->dev_latency_stat[i];
+
+ /* Calculate the latencies */
+ scst_time = finish - cmd->start - (cmd->parse_time +
+ cmd->alloc_buf_time + cmd->restart_waiting_time +
+ cmd->rdy_to_xfer_time + cmd->pre_exec_time +
+ cmd->exec_time + cmd->dev_done_time + cmd->xmit_time +
+ cmd->tgt_on_free_time + cmd->dev_on_free_time);
+ tgt_time = cmd->alloc_buf_time + cmd->restart_waiting_time +
+ cmd->rdy_to_xfer_time + cmd->pre_exec_time +
+ cmd->xmit_time + cmd->tgt_on_free_time;
+ dev_time = cmd->parse_time + cmd->exec_time + cmd->dev_done_time +
+ cmd->dev_on_free_time;
+
+ spin_lock_bh(&sess->lat_lock);
+
+ /* Save the basic latency information */
+ sess->scst_time += scst_time;
+ sess->tgt_time += tgt_time;
+ sess->dev_time += dev_time;
+ sess->processed_cmds++;
+
+ if ((sess->min_scst_time == 0) ||
+ (sess->min_scst_time > scst_time))
+ sess->min_scst_time = scst_time;
+ if ((sess->min_tgt_time == 0) ||
+ (sess->min_tgt_time > tgt_time))
+ sess->min_tgt_time = tgt_time;
+ if ((sess->min_dev_time == 0) ||
+ (sess->min_dev_time > dev_time))
+ sess->min_dev_time = dev_time;
+
+ if (sess->max_scst_time < scst_time)
+ sess->max_scst_time = scst_time;
+ if (sess->max_tgt_time < tgt_time)
+ sess->max_tgt_time = tgt_time;
+ if (sess->max_dev_time < dev_time)
+ sess->max_dev_time = dev_time;
+
+ /* Save the extended latency information */
+ if (cmd->data_direction & SCST_DATA_READ) {
+ latency_stat->scst_time_rd += scst_time;
+ latency_stat->tgt_time_rd += tgt_time;
+ latency_stat->dev_time_rd += dev_time;
+ latency_stat->processed_cmds_rd++;
+
+ if ((latency_stat->min_scst_time_rd == 0) ||
+ (latency_stat->min_scst_time_rd > scst_time))
+ latency_stat->min_scst_time_rd = scst_time;
+ if ((latency_stat->min_tgt_time_rd == 0) ||
+ (latency_stat->min_tgt_time_rd > tgt_time))
+ latency_stat->min_tgt_time_rd = tgt_time;
+ if ((latency_stat->min_dev_time_rd == 0) ||
+ (latency_stat->min_dev_time_rd > dev_time))
+ latency_stat->min_dev_time_rd = dev_time;
+
+ if (latency_stat->max_scst_time_rd < scst_time)
+ latency_stat->max_scst_time_rd = scst_time;
+ if (latency_stat->max_tgt_time_rd < tgt_time)
+ latency_stat->max_tgt_time_rd = tgt_time;
+ if (latency_stat->max_dev_time_rd < dev_time)
+ latency_stat->max_dev_time_rd = dev_time;
+
+ dev_latency_stat->scst_time_rd += scst_time;
+ dev_latency_stat->tgt_time_rd += tgt_time;
+ dev_latency_stat->dev_time_rd += dev_time;
+ dev_latency_stat->processed_cmds_rd++;
+
+ if ((dev_latency_stat->min_scst_time_rd == 0) ||
+ (dev_latency_stat->min_scst_time_rd > scst_time))
+ dev_latency_stat->min_scst_time_rd = scst_time;
+ if ((dev_latency_stat->min_tgt_time_rd == 0) ||
+ (dev_latency_stat->min_tgt_time_rd > tgt_time))
+ dev_latency_stat->min_tgt_time_rd = tgt_time;
+ if ((dev_latency_stat->min_dev_time_rd == 0) ||
+ (dev_latency_stat->min_dev_time_rd > dev_time))
+ dev_latency_stat->min_dev_time_rd = dev_time;
+
+ if (dev_latency_stat->max_scst_time_rd < scst_time)
+ dev_latency_stat->max_scst_time_rd = scst_time;
+ if (dev_latency_stat->max_tgt_time_rd < tgt_time)
+ dev_latency_stat->max_tgt_time_rd = tgt_time;
+ if (dev_latency_stat->max_dev_time_rd < dev_time)
+ dev_latency_stat->max_dev_time_rd = dev_time;
+ } else if (cmd->data_direction & SCST_DATA_WRITE) {
+ latency_stat->scst_time_wr += scst_time;
+ latency_stat->tgt_time_wr += tgt_time;
+ latency_stat->dev_time_wr += dev_time;
+ latency_stat->processed_cmds_wr++;
+
+ if ((latency_stat->min_scst_time_wr == 0) ||
+ (latency_stat->min_scst_time_wr > scst_time))
+ latency_stat->min_scst_time_wr = scst_time;
+ if ((latency_stat->min_tgt_time_wr == 0) ||
+ (latency_stat->min_tgt_time_wr > tgt_time))
+ latency_stat->min_tgt_time_wr = tgt_time;
+ if ((latency_stat->min_dev_time_wr == 0) ||
+ (latency_stat->min_dev_time_wr > dev_time))
+ latency_stat->min_dev_time_wr = dev_time;
+
+ if (latency_stat->max_scst_time_wr < scst_time)
+ latency_stat->max_scst_time_wr = scst_time;
+ if (latency_stat->max_tgt_time_wr < tgt_time)
+ latency_stat->max_tgt_time_wr = tgt_time;
+ if (latency_stat->max_dev_time_wr < dev_time)
+ latency_stat->max_dev_time_wr = dev_time;
+
+ dev_latency_stat->scst_time_wr += scst_time;
+ dev_latency_stat->tgt_time_wr += tgt_time;
+ dev_latency_stat->dev_time_wr += dev_time;
+ dev_latency_stat->processed_cmds_wr++;
+
+ if ((dev_latency_stat->min_scst_time_wr == 0) ||
+ (dev_latency_stat->min_scst_time_wr > scst_time))
+ dev_latency_stat->min_scst_time_wr = scst_time;
+ if ((dev_latency_stat->min_tgt_time_wr == 0) ||
+ (dev_latency_stat->min_tgt_time_wr > tgt_time))
+ dev_latency_stat->min_tgt_time_wr = tgt_time;
+ if ((dev_latency_stat->min_dev_time_wr == 0) ||
+ (dev_latency_stat->min_dev_time_wr > dev_time))
+ dev_latency_stat->min_dev_time_wr = dev_time;
+
+ if (dev_latency_stat->max_scst_time_wr < scst_time)
+ dev_latency_stat->max_scst_time_wr = scst_time;
+ if (dev_latency_stat->max_tgt_time_wr < tgt_time)
+ dev_latency_stat->max_tgt_time_wr = tgt_time;
+ if (dev_latency_stat->max_dev_time_wr < dev_time)
+ dev_latency_stat->max_dev_time_wr = dev_time;
+ }
+
+ spin_unlock_bh(&sess->lat_lock);
+
+ TRACE_DBG("cmd %p: finish %lld, scst_time %lld, "
+ "tgt_time %lld, dev_time %lld", cmd, finish, scst_time,
+ tgt_time, dev_time);
+ return;
+}
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 7/17]: SCST Persistent Reservations implementation
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (5 preceding siblings ...)
2010-09-14 14:42 ` [PATCH 6/17]: SCST internal library functions Vladislav Bolkhovitin
@ 2010-09-14 14:43 ` Vladislav Bolkhovitin
2010-09-14 14:44 ` [PATCH 8/17]: SCST SYSFS interface implementation Vladislav Bolkhovitin
` (10 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:43 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains Persistent Reservations implementation
Signed-off-by: Alexey Obitotskiy <alexeyo1@open-e.com>
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_pres.c | 2497 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
scst_pres.h | 159 +++
2 files changed, 2656 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/scst_pres.c linux-2.6.35/drivers/scst/scst_pres.c
--- orig/linux-2.6.35/drivers/scst/scst_pres.c
+++ linux-2.6.35/drivers/scst/scst_pres.c
@@ -0,0 +1,2497 @@
+/*
+ * scst_pres.c
+ *
+ * Copyright (C) 2009 - 2010 Alexey Obitotskiy <alexeyo1@open-e.com>
+ * Copyright (C) 2009 - 2010 Open-E, Inc.
+ * Copyright (C) 2009 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/smp_lock.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+#include <linux/kthread.h>
+#include <linux/delay.h>
+#include <linux/time.h>
+#include <linux/ctype.h>
+#include <asm/byteorder.h>
+#include <linux/syscalls.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/fcntl.h>
+#include <linux/uaccess.h>
+#include <linux/namei.h>
+#include <linux/version.h>
+#include <asm/unaligned.h>
+
+#include <scst/scst.h>
+#include <scst/scst_const.h>
+#include "scst_priv.h"
+#include "scst_pres.h"
+
+#define SCST_PR_ROOT_ENTRY "pr"
+#define SCST_PR_FILE_SIGN 0xBBEEEEAAEEBBDD77LLU
+#define SCST_PR_FILE_VERSION 1LLU
+
+#define FILE_BUFFER_SIZE 512
+
+#ifndef isblank
+#define isblank(c) ((c) == ' ' || (c) == '\t')
+#endif
+
+static inline int tid_size(const uint8_t *tid)
+{
+ BUG_ON(tid == NULL);
+
+ if ((tid[0] & 0x0f) == SCSI_TRANSPORTID_PROTOCOLID_ISCSI)
+ return be16_to_cpu(get_unaligned((__be16 *)&tid[2])) + 4;
+ else
+ return TID_COMMON_SIZE;
+}
+
+/* Secures tid by setting 0 in the last byte of NULL-terminated tid's */
+static inline void tid_secure(uint8_t *tid)
+{
+ if ((tid[0] & 0x0f) == SCSI_TRANSPORTID_PROTOCOLID_ISCSI) {
+ int size = tid_size(tid);
+ tid[size - 1] = '\0';
+ }
+
+ return;
+}
+
+/* Returns false if tid's are not equal, true otherwise */
+static bool tid_equal(const uint8_t *tid_a, const uint8_t *tid_b)
+{
+ int len;
+
+ if (tid_a == NULL || tid_b == NULL)
+ return false;
+
+ if ((tid_a[0] & 0x0f) != (tid_b[0] & 0x0f)) {
+ TRACE_DBG("%s", "Different protocol IDs");
+ return false;
+ }
+
+ if ((tid_a[0] & 0x0f) == SCSI_TRANSPORTID_PROTOCOLID_ISCSI) {
+ const uint8_t tid_a_fmt = tid_a[0] & 0xc0;
+ const uint8_t tid_b_fmt = tid_b[0] & 0xc0;
+ int tid_a_len, tid_a_max = tid_size(tid_a) - 4;
+ int tid_b_len, tid_b_max = tid_size(tid_b) - 4;
+ int i;
+
+ tid_a += 4;
+ tid_b += 4;
+
+ if (tid_a_fmt == 0x00)
+ tid_a_len = strnlen(tid_a, tid_a_max);
+ else if (tid_a_fmt == 0x40) {
+ if (tid_a_fmt != tid_b_fmt) {
+ uint8_t *p = strnchr(tid_a, tid_a_max, ',');
+ if (p == NULL)
+ goto out_error;
+ tid_a_len = p - tid_a;
+
+ BUG_ON(tid_a_len > tid_a_max);
+ BUG_ON(tid_a_len < 0);
+ } else
+ tid_a_len = strnlen(tid_a, tid_a_max);
+ } else
+ goto out_error;
+
+ if (tid_b_fmt == 0x00)
+ tid_b_len = strnlen(tid_b, tid_b_max);
+ else if (tid_b_fmt == 0x40) {
+ if (tid_a_fmt != tid_b_fmt) {
+ uint8_t *p = strnchr(tid_b, tid_b_max, ',');
+ if (p == NULL)
+ goto out_error;
+ tid_b_len = p - tid_b;
+
+ BUG_ON(tid_b_len > tid_b_max);
+ BUG_ON(tid_b_len < 0);
+ } else
+ tid_b_len = strnlen(tid_b, tid_b_max);
+ } else
+ goto out_error;
+
+ if (tid_a_len != tid_b_len)
+ return false;
+
+ len = tid_a_len;
+
+ /* ISCSI names are case insensitive */
+ for (i = 0; i < len; i++)
+ if (tolower(tid_a[i]) != tolower(tid_b[i]))
+ return false;
+ return true;
+ } else
+ len = TID_COMMON_SIZE;
+
+ return memcmp(tid_a, tid_b, len) == 0;
+
+out_error:
+ PRINT_ERROR("%s", "Invalid initiator port transport id");
+ return false;
+}
+
+/* Must be called under dev_pr_mutex */
+static inline void scst_pr_set_holder(struct scst_device *dev,
+ struct scst_dev_registrant *holder, uint8_t scope, uint8_t type)
+{
+ dev->pr_is_set = 1;
+ dev->pr_scope = scope;
+ dev->pr_type = type;
+ if (dev->pr_type != TYPE_EXCLUSIVE_ACCESS_ALL_REG &&
+ dev->pr_type != TYPE_WRITE_EXCLUSIVE_ALL_REG)
+ dev->pr_holder = holder;
+}
+
+/* Must be called under dev_pr_mutex */
+static bool scst_pr_is_holder(struct scst_device *dev,
+ struct scst_dev_registrant *reg)
+{
+ bool res = false;
+
+ if (!dev->pr_is_set)
+ goto out;
+
+ if (dev->pr_type == TYPE_EXCLUSIVE_ACCESS_ALL_REG ||
+ dev->pr_type == TYPE_WRITE_EXCLUSIVE_ALL_REG) {
+ res = (reg != NULL);
+ } else
+ res = (dev->pr_holder == reg);
+
+out:
+ return res;
+}
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+/* Must be called under dev_pr_mutex */
+void scst_pr_dump_prs(struct scst_device *dev, bool force)
+{
+ if (!force) {
+#if defined(CONFIG_SCST_DEBUG)
+ if ((trace_flag & TRACE_PRES) == 0)
+#endif
+ goto out;
+ }
+
+ PRINT_INFO("Persistent reservations for device %s:", dev->virt_name);
+
+ if (list_empty(&dev->dev_registrants_list))
+ PRINT_INFO("%s", " No registrants");
+ else {
+ struct scst_dev_registrant *reg;
+ int i = 0;
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ PRINT_INFO(" [%d] registrant %s/%d, key %016llx "
+ "(reg %p, tgt_dev %p)", i++,
+ debug_transport_id_to_initiator_name(
+ reg->transport_id),
+ reg->rel_tgt_id, reg->key, reg, reg->tgt_dev);
+ }
+ }
+
+ if (dev->pr_is_set) {
+ struct scst_dev_registrant *holder = dev->pr_holder;
+ if (holder != NULL)
+ PRINT_INFO("Reservation holder is %s/%d (key %016llx, "
+ "scope %x, type %x, reg %p, tgt_dev %p)",
+ debug_transport_id_to_initiator_name(
+ holder->transport_id),
+ holder->rel_tgt_id, holder->key, dev->pr_scope,
+ dev->pr_type, holder, holder->tgt_dev);
+ else
+ PRINT_INFO("All registrants are reservation holders "
+ "(scope %x, type %x)", dev->pr_scope,
+ dev->pr_type);
+ } else
+ PRINT_INFO("%s", "Not reserved");
+
+out:
+ return;
+}
+
+#endif /* defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+/* dev_pr_mutex must be locked */
+static void scst_pr_find_registrants_list_all(struct scst_device *dev,
+ struct scst_dev_registrant *exclude_reg, struct list_head *list)
+{
+ struct scst_dev_registrant *reg;
+
+ TRACE_PR("Finding all registered records for device '%s' "
+ "with exclude reg key %016llx",
+ dev->virt_name, exclude_reg->key);
+
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ if (reg == exclude_reg)
+ continue;
+ TRACE_PR("Adding registrant %s/%d (%p) to find list (key %016llx)",
+ debug_transport_id_to_initiator_name(reg->transport_id),
+ reg->rel_tgt_id, reg, reg->key);
+ list_add_tail(®->aux_list_entry, list);
+ }
+ return;
+}
+
+/* dev_pr_mutex must be locked */
+static void scst_pr_find_registrants_list_key(struct scst_device *dev,
+ __be64 key, struct list_head *list)
+{
+ struct scst_dev_registrant *reg;
+
+ TRACE_PR("Finding registrants for device '%s' with key %016llx",
+ dev->virt_name, key);
+
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ if (reg->key == key) {
+ TRACE_PR("Adding registrant %s/%d (%p) to the find "
+ "list (key %016llx)",
+ debug_transport_id_to_initiator_name(
+ reg->transport_id),
+ reg->rel_tgt_id, reg->tgt_dev, key);
+ list_add_tail(®->aux_list_entry, list);
+ }
+ }
+ return;
+}
+
+/* dev_pr_mutex must be locked */
+static struct scst_dev_registrant *scst_pr_find_reg(
+ struct scst_device *dev, const uint8_t *transport_id,
+ const uint16_t rel_tgt_id)
+{
+ struct scst_dev_registrant *reg, *res = NULL;
+
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ if ((reg->rel_tgt_id == rel_tgt_id) &&
+ tid_equal(reg->transport_id, transport_id)) {
+ res = reg;
+ break;
+ }
+ }
+ return res;
+}
+
+/* Must be called under dev_pr_mutex */
+static void scst_pr_clear_reservation(struct scst_device *dev)
+{
+
+ WARN_ON(!dev->pr_is_set);
+
+ dev->pr_is_set = 0;
+ dev->pr_scope = SCOPE_LU;
+ dev->pr_type = TYPE_UNSPECIFIED;
+
+ dev->pr_holder = NULL;
+ return;
+}
+
+/* Must be called under dev_pr_mutex */
+static void scst_pr_clear_holder(struct scst_device *dev)
+{
+
+ WARN_ON(!dev->pr_is_set);
+
+ if (dev->pr_type == TYPE_WRITE_EXCLUSIVE_ALL_REG ||
+ dev->pr_type == TYPE_EXCLUSIVE_ACCESS_ALL_REG) {
+ if (list_empty(&dev->dev_registrants_list))
+ scst_pr_clear_reservation(dev);
+ } else
+ scst_pr_clear_reservation(dev);
+
+ dev->pr_holder = NULL;
+ return;
+}
+
+/* Must be called under dev_pr_mutex */
+static struct scst_dev_registrant *scst_pr_add_registrant(
+ struct scst_device *dev, const uint8_t *transport_id,
+ const uint16_t rel_tgt_id, __be64 key,
+ bool dev_lock_locked)
+{
+ struct scst_dev_registrant *reg;
+ struct scst_tgt_dev *t;
+ gfp_t gfp_flags = dev_lock_locked ? GFP_ATOMIC : GFP_KERNEL;
+
+ BUG_ON(dev == NULL);
+ BUG_ON(transport_id == NULL);
+
+ TRACE_PR("Registering %s/%d (dev %s)",
+ debug_transport_id_to_initiator_name(transport_id),
+ rel_tgt_id, dev->virt_name);
+
+ reg = scst_pr_find_reg(dev, transport_id, rel_tgt_id);
+ if (reg != NULL) {
+ /*
+ * It might happen when a target driver would make >1 session
+ * from the same initiator to the same target.
+ */
+ PRINT_ERROR("Registrant %p/%d (dev %s) already exists!", reg,
+ rel_tgt_id, dev->virt_name);
+ PRINT_BUFFER("TransportID", transport_id, 24);
+ WARN_ON(1);
+ reg = NULL;
+ goto out;
+ }
+
+ reg = kzalloc(sizeof(*reg), gfp_flags);
+ if (reg == NULL) {
+ PRINT_ERROR("%s", "Unable to allocate registration record");
+ goto out;
+ }
+
+ reg->transport_id = kmalloc(tid_size(transport_id), gfp_flags);
+ if (reg->transport_id == NULL) {
+ PRINT_ERROR("%s", "Unable to allocate initiator port "
+ "transport id");
+ goto out_free;
+ }
+ memcpy(reg->transport_id, transport_id, tid_size(transport_id));
+
+ reg->rel_tgt_id = rel_tgt_id;
+ reg->key = key;
+
+ /*
+ * We can't use scst_mutex here, because of the circular
+ * locking dependency with dev_pr_mutex.
+ */
+ if (!dev_lock_locked)
+ spin_lock_bh(&dev->dev_lock);
+ list_for_each_entry(t, &dev->dev_tgt_dev_list, dev_tgt_dev_list_entry) {
+ if (tid_equal(t->sess->transport_id, transport_id) &&
+ (t->sess->tgt->rel_tgt_id == rel_tgt_id) &&
+ (t->registrant == NULL)) {
+ /*
+ * We must assign here, because t can die
+ * immediately after we release dev_lock.
+ */
+ TRACE_PR("Found tgt_dev %p", t);
+ reg->tgt_dev = t;
+ t->registrant = reg;
+ break;
+ }
+ }
+ if (!dev_lock_locked)
+ spin_unlock_bh(&dev->dev_lock);
+
+ list_add_tail(®->dev_registrants_list_entry,
+ &dev->dev_registrants_list);
+
+ TRACE_PR("Reg %p registered (dev %s, tgt_dev %p)", reg,
+ dev->virt_name, reg->tgt_dev);
+
+out:
+ return reg;
+
+out_free:
+ kfree(reg);
+ reg = NULL;
+ goto out;
+}
+
+/* Must be called under dev_pr_mutex */
+static void scst_pr_remove_registrant(struct scst_device *dev,
+ struct scst_dev_registrant *reg)
+{
+
+ TRACE_PR("Removing registrant %s/%d (reg %p, tgt_dev %p, key %016llx, "
+ "dev %s)", debug_transport_id_to_initiator_name(reg->transport_id),
+ reg->rel_tgt_id, reg, reg->tgt_dev, reg->key, dev->virt_name);
+
+ list_del(®->dev_registrants_list_entry);
+
+ if (scst_pr_is_holder(dev, reg))
+ scst_pr_clear_holder(dev);
+
+ if (reg->tgt_dev)
+ reg->tgt_dev->registrant = NULL;
+
+ kfree(reg->transport_id);
+ kfree(reg);
+ return;
+}
+
+/* Must be called under dev_pr_mutex */
+static void scst_pr_send_ua_reg(struct scst_device *dev,
+ struct scst_dev_registrant *reg,
+ int key, int asc, int ascq)
+{
+ static uint8_t ua[SCST_STANDARD_SENSE_LEN];
+
+ scst_set_sense(ua, sizeof(ua), dev->d_sense, key, asc, ascq);
+
+ TRACE_PR("Queuing UA [%x %x %x]: registrant %s/%d (%p), tgt_dev %p, "
+ "key %016llx", ua[2], ua[12], ua[13],
+ debug_transport_id_to_initiator_name(reg->transport_id),
+ reg->rel_tgt_id, reg, reg->tgt_dev, reg->key);
+
+ if (reg->tgt_dev)
+ scst_check_set_UA(reg->tgt_dev, ua, sizeof(ua), 0);
+ return;
+}
+
+/* Must be called under dev_pr_mutex */
+static void scst_pr_send_ua_all(struct scst_device *dev,
+ struct scst_dev_registrant *exclude_reg,
+ int key, int asc, int ascq)
+{
+ struct scst_dev_registrant *reg;
+
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ if (reg != exclude_reg)
+ scst_pr_send_ua_reg(dev, reg, key, asc, ascq);
+ }
+ return;
+}
+
+/* Must be called under dev_pr_mutex */
+static void scst_pr_abort_reg(struct scst_device *dev,
+ struct scst_cmd *pr_cmd, struct scst_dev_registrant *reg)
+{
+ struct scst_session *sess;
+ __be64 packed_lun;
+ int rc;
+
+ if (reg->tgt_dev == NULL) {
+ TRACE_PR("Registrant %s/%d (%p, key 0x%016llx) has no session",
+ debug_transport_id_to_initiator_name(reg->transport_id),
+ reg->rel_tgt_id, reg, reg->key);
+ goto out;
+ }
+
+ sess = reg->tgt_dev->sess;
+
+ TRACE_PR("Aborting %d commands for %s/%d (reg %p, key 0x%016llx, "
+ "tgt_dev %p, sess %p)",
+ atomic_read(®->tgt_dev->tgt_dev_cmd_count),
+ debug_transport_id_to_initiator_name(reg->transport_id),
+ reg->rel_tgt_id, reg, reg->key, reg->tgt_dev, sess);
+
+ packed_lun = scst_pack_lun(reg->tgt_dev->lun, sess->acg->addr_method);
+
+ rc = scst_rx_mgmt_fn_lun(sess, SCST_PR_ABORT_ALL,
+ (uint8_t *)&packed_lun, sizeof(packed_lun), SCST_NON_ATOMIC,
+ pr_cmd);
+ if (rc != 0) {
+ /*
+ * There's nothing more we can do here... Hopefully, it would
+ * never happen.
+ */
+ PRINT_ERROR("SCST_PR_ABORT_ALL failed %d (sess %p)",
+ rc, sess);
+ }
+
+out:
+ return;
+}
+
+/* Abstract vfs_unlink & path_put for different kernel versions */
+static inline void scst_pr_vfs_unlink_and_put(struct nameidata *nd)
+{
+ vfs_unlink(nd->path.dentry->d_parent->d_inode,
+ nd->path.dentry);
+ path_put(&nd->path);
+}
+
+static inline void scst_pr_path_put(struct nameidata *nd)
+{
+ path_put(&nd->path);
+}
+
+/* Called under scst_mutex */
+static int scst_pr_do_load_device_file(struct scst_device *dev,
+ const char *file_name)
+{
+ int res = 0, rc;
+ struct file *file = NULL;
+ struct inode *inode;
+ char *buf = NULL;
+ loff_t file_size, pos, data_size;
+ uint64_t sign, version;
+ mm_segment_t old_fs;
+ uint8_t pr_is_set, aptpl;
+ __be64 key;
+ uint16_t rel_tgt_id;
+
+ old_fs = get_fs();
+ set_fs(KERNEL_DS);
+
+ TRACE_PR("Loading persistent file '%s'", file_name);
+
+ file = filp_open(file_name, O_RDONLY, 0);
+ if (IS_ERR(file)) {
+ res = PTR_ERR(file);
+ TRACE_PR("Unable to open file '%s' - error %d", file_name, res);
+ goto out;
+ }
+
+ inode = file->f_dentry->d_inode;
+
+ if (S_ISREG(inode->i_mode))
+ /* Nothing to do */;
+ else if (S_ISBLK(inode->i_mode))
+ inode = inode->i_bdev->bd_inode;
+ else {
+ PRINT_ERROR("Invalid file mode 0x%x", inode->i_mode);
+ goto out_close;
+ }
+
+ file_size = inode->i_size;
+
+ /* Let's limit the file size by some reasonable number */
+ if ((file_size == 0) || (file_size >= 15*1024*1024)) {
+ PRINT_ERROR("Invalid PR file size %d", (int)file_size);
+ res = -EINVAL;
+ goto out_close;
+ }
+
+ buf = vmalloc(file_size);
+ if (buf == NULL) {
+ res = -ENOMEM;
+ PRINT_ERROR("%s", "Unable to allocate buffer");
+ goto out_close;
+ }
+
+ pos = 0;
+ rc = vfs_read(file, (void __force __user *)buf, file_size, &pos);
+ if (rc != file_size) {
+ PRINT_ERROR("Unable to read file '%s' - error %d", file_name,
+ rc);
+ res = rc;
+ goto out_close;
+ }
+
+ data_size = 0;
+ data_size += sizeof(sign);
+ data_size += sizeof(version);
+ data_size += sizeof(aptpl);
+ data_size += sizeof(pr_is_set);
+ data_size += sizeof(dev->pr_type);
+ data_size += sizeof(dev->pr_scope);
+
+ if (file_size < data_size) {
+ res = -EINVAL;
+ PRINT_ERROR("Invalid file '%s' - size too small", file_name);
+ goto out_close;
+ }
+
+ pos = 0;
+
+ sign = get_unaligned((uint64_t *)&buf[pos]);
+ if (sign != SCST_PR_FILE_SIGN) {
+ res = -EINVAL;
+ PRINT_ERROR("Invalid persistent file signature %016llx "
+ "(expected %016llx)", sign, SCST_PR_FILE_SIGN);
+ goto out_close;
+ }
+ pos += sizeof(sign);
+
+ version = get_unaligned((uint64_t *)&buf[pos]);
+ if (version != SCST_PR_FILE_VERSION) {
+ res = -EINVAL;
+ PRINT_ERROR("Invalid persistent file version %016llx "
+ "(expected %016llx)", version, SCST_PR_FILE_VERSION);
+ goto out_close;
+ }
+ pos += sizeof(version);
+
+ while (data_size < file_size) {
+ uint8_t *tid;
+
+ data_size++;
+ tid = &buf[data_size];
+ data_size += tid_size(tid);
+ data_size += sizeof(key);
+ data_size += sizeof(rel_tgt_id);
+
+ if (data_size > file_size) {
+ res = -EINVAL;
+ PRINT_ERROR("Invalid file '%s' - size mismatch have "
+ "%lld expected %lld", file_name, file_size,
+ data_size);
+ goto out_close;
+ }
+ }
+
+ aptpl = buf[pos];
+ dev->pr_aptpl = aptpl ? 1 : 0;
+ pos += sizeof(aptpl);
+
+ pr_is_set = buf[pos];
+ dev->pr_is_set = pr_is_set ? 1 : 0;
+ pos += sizeof(pr_is_set);
+
+ dev->pr_type = buf[pos];
+ pos += sizeof(dev->pr_type);
+
+ dev->pr_scope = buf[pos];
+ pos += sizeof(dev->pr_scope);
+
+ while (pos < file_size) {
+ uint8_t is_holder;
+ uint8_t *tid;
+ struct scst_dev_registrant *reg = NULL;
+
+ is_holder = buf[pos++];
+
+ tid = &buf[pos];
+ pos += tid_size(tid);
+
+ key = get_unaligned((__be64 *)&buf[pos]);
+ pos += sizeof(key);
+
+ rel_tgt_id = get_unaligned((uint16_t *)&buf[pos]);
+ pos += sizeof(rel_tgt_id);
+
+ reg = scst_pr_add_registrant(dev, tid, rel_tgt_id, key, false);
+ if (reg == NULL) {
+ res = -ENOMEM;
+ goto out_close;
+ }
+
+ if (is_holder)
+ dev->pr_holder = reg;
+ }
+
+out_close:
+ filp_close(file, NULL);
+
+out:
+ if (buf != NULL)
+ vfree(buf);
+
+ set_fs(old_fs);
+ return res;
+}
+
+static int scst_pr_load_device_file(struct scst_device *dev)
+{
+ int res;
+
+ if (dev->pr_file_name == NULL || dev->pr_file_name1 == NULL) {
+ PRINT_ERROR("Invalid file paths for '%s'", dev->virt_name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_pr_do_load_device_file(dev, dev->pr_file_name);
+ if (res == 0)
+ goto out;
+ else if (res == -ENOMEM)
+ goto out;
+
+ res = scst_pr_do_load_device_file(dev, dev->pr_file_name1);
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return res;
+}
+
+static int scst_pr_copy_file(const char *src, const char *dest)
+{
+ int res = 0;
+ struct inode *inode;
+ loff_t file_size, pos;
+ uint8_t *buf = NULL;
+ struct file *file_src = NULL, *file_dest = NULL;
+ mm_segment_t old_fs = get_fs();
+
+ if (src == NULL || dest == NULL) {
+ res = -EINVAL;
+ PRINT_ERROR("%s", "Invalid persistent files path - backup "
+ "skipped");
+ goto out;
+ }
+
+ TRACE_PR("Copying '%s' into '%s'", src, dest);
+
+ set_fs(KERNEL_DS);
+
+ file_src = filp_open(src, O_RDONLY, 0);
+ if (IS_ERR(file_src)) {
+ res = PTR_ERR(file_src);
+ TRACE_PR("Unable to open file '%s' - error %d", src,
+ res);
+ goto out_free;
+ }
+
+ file_dest = filp_open(dest, O_WRONLY | O_CREAT | O_TRUNC, 0644);
+ if (IS_ERR(file_dest)) {
+ res = PTR_ERR(file_dest);
+ TRACE_PR("Unable to open backup file '%s' - error %d", dest,
+ res);
+ goto out_close;
+ }
+
+ inode = file_src->f_dentry->d_inode;
+
+ if (S_ISREG(inode->i_mode))
+ /* Nothing to do */;
+ else if (S_ISBLK(inode->i_mode))
+ inode = inode->i_bdev->bd_inode;
+ else {
+ PRINT_ERROR("Invalid file mode 0x%x", inode->i_mode);
+ res = -EINVAL;
+ set_fs(old_fs);
+ goto out_skip;
+ }
+
+ file_size = inode->i_size;
+
+ buf = vmalloc(file_size);
+ if (buf == NULL) {
+ res = -ENOMEM;
+ PRINT_ERROR("%s", "Unable to allocate temporary buffer");
+ goto out_skip;
+ }
+
+ pos = 0;
+ res = vfs_read(file_src, (void __force __user *)buf, file_size, &pos);
+ if (res != file_size) {
+ PRINT_ERROR("Unable to read file '%s' - error %d", src, res);
+ goto out_skip;
+ }
+
+ pos = 0;
+ res = vfs_write(file_dest, (void __force __user *)buf, file_size, &pos);
+ if (res != file_size) {
+ PRINT_ERROR("Unable to write to '%s' - error %d", dest, res);
+ goto out_skip;
+ }
+
+ res = vfs_fsync(file_dest, 0);
+ if (res != 0) {
+ PRINT_ERROR("fsync() of the backup PR file failed: %d", res);
+ goto out_skip;
+ }
+
+out_skip:
+ filp_close(file_dest, NULL);
+
+out_close:
+ filp_close(file_src, NULL);
+
+out_free:
+ if (buf != NULL)
+ vfree(buf);
+
+ set_fs(old_fs);
+
+out:
+ return res;
+}
+
+static void scst_pr_remove_device_files(struct scst_tgt_dev *tgt_dev)
+{
+ int res = 0;
+ struct scst_device *dev = tgt_dev->dev;
+ struct nameidata nd;
+ mm_segment_t old_fs = get_fs();
+
+ set_fs(KERNEL_DS);
+
+ res = path_lookup(dev->pr_file_name, 0, &nd);
+ if (!res)
+ scst_pr_vfs_unlink_and_put(&nd);
+ else
+ TRACE_DBG("Unable to lookup file '%s' - error %d",
+ dev->pr_file_name, res);
+
+ res = path_lookup(dev->pr_file_name1, 0, &nd);
+ if (!res)
+ scst_pr_vfs_unlink_and_put(&nd);
+ else
+ TRACE_DBG("Unable to lookup file '%s' - error %d",
+ dev->pr_file_name1, res);
+
+ set_fs(old_fs);
+ return;
+}
+
+/* Must be called under dev_pr_mutex */
+void scst_pr_sync_device_file(struct scst_tgt_dev *tgt_dev, struct scst_cmd *cmd)
+{
+ int res = 0;
+ struct scst_device *dev = tgt_dev->dev;
+ struct file *file;
+ mm_segment_t old_fs = get_fs();
+ loff_t pos = 0;
+ uint64_t sign;
+ uint64_t version;
+ uint8_t pr_is_set, aptpl;
+
+ if ((dev->pr_aptpl == 0) || list_empty(&dev->dev_registrants_list)) {
+ scst_pr_remove_device_files(tgt_dev);
+ goto out;
+ }
+
+ scst_pr_copy_file(dev->pr_file_name, dev->pr_file_name1);
+
+ set_fs(KERNEL_DS);
+
+ file = filp_open(dev->pr_file_name, O_WRONLY | O_CREAT | O_TRUNC, 0644);
+ if (IS_ERR(file)) {
+ res = PTR_ERR(file);
+ PRINT_ERROR("Unable to (re)create PR file '%s' - error %d",
+ dev->pr_file_name, res);
+ goto out_set_fs;
+ }
+
+ TRACE_PR("Updating pr file '%s'", dev->pr_file_name);
+
+ /*
+ * signature
+ */
+ sign = 0;
+ pos = 0;
+ res = vfs_write(file, (void __force __user *)&sign, sizeof(sign), &pos);
+ if (res != sizeof(sign))
+ goto write_error;
+
+ /*
+ * version
+ */
+ version = SCST_PR_FILE_VERSION;
+ res = vfs_write(file, (void __force __user *)&version, sizeof(version), &pos);
+ if (res != sizeof(version))
+ goto write_error;
+
+ /*
+ * APTPL
+ */
+ aptpl = dev->pr_aptpl;
+ res = vfs_write(file, (void __force __user *)&aptpl, sizeof(aptpl), &pos);
+ if (res != sizeof(aptpl))
+ goto write_error;
+
+ /*
+ * reservation
+ */
+ pr_is_set = dev->pr_is_set;
+ res = vfs_write(file, (void __force __user *)&pr_is_set, sizeof(pr_is_set), &pos);
+ if (res != sizeof(pr_is_set))
+ goto write_error;
+
+ res = vfs_write(file, (void __force __user *)&dev->pr_type, sizeof(dev->pr_type), &pos);
+ if (res != sizeof(dev->pr_type))
+ goto write_error;
+
+ res = vfs_write(file, (void __force __user *)&dev->pr_scope, sizeof(dev->pr_scope), &pos);
+ if (res != sizeof(dev->pr_scope))
+ goto write_error;
+
+ /*
+ * registration records
+ */
+ if (!list_empty(&dev->dev_registrants_list)) {
+ struct scst_dev_registrant *reg;
+
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ uint8_t is_holder = 0;
+ int size;
+
+ is_holder = (dev->pr_holder == reg);
+
+ res = vfs_write(file, (void __force __user *)&is_holder, sizeof(is_holder),
+ &pos);
+ if (res != sizeof(is_holder))
+ goto write_error;
+
+ size = tid_size(reg->transport_id);
+ res = vfs_write(file, (void __force __user *)reg->transport_id, size, &pos);
+ if (res != size)
+ goto write_error;
+
+ res = vfs_write(file, (void __force __user *)®->key,
+ sizeof(reg->key), &pos);
+ if (res != sizeof(reg->key))
+ goto write_error;
+
+ res = vfs_write(file, (void __force __user *)®->rel_tgt_id,
+ sizeof(reg->rel_tgt_id), &pos);
+ if (res != sizeof(reg->rel_tgt_id))
+ goto write_error;
+ }
+ }
+
+ res = vfs_fsync(file, 0);
+ if (res != 0) {
+ PRINT_ERROR("fsync() of the PR file failed: %d", res);
+ goto write_error_close;
+ }
+
+ sign = SCST_PR_FILE_SIGN;
+ pos = 0;
+ res = vfs_write(file, (void __force __user *)&sign, sizeof(sign), &pos);
+ if (res != sizeof(sign))
+ goto write_error;
+
+ res = vfs_fsync(file, 0);
+ if (res != 0) {
+ PRINT_ERROR("fsync() of the PR file failed: %d", res);
+ goto write_error_close;
+ }
+
+ res = 0;
+
+ filp_close(file, NULL);
+
+out_set_fs:
+ set_fs(old_fs);
+
+out:
+ if (res != 0) {
+ PRINT_CRIT_ERROR("Unable to save persistent information "
+ "(target %s, initiator %s, device %s)",
+ tgt_dev->sess->tgt->tgt_name,
+ tgt_dev->sess->initiator_name, dev->virt_name);
+#if 0 /*
+ * Looks like it's safer to return SUCCESS and expect operator's
+ * intervention to be able to save the PR's state next time, than
+ * to return HARDWARE ERROR and screw up all the interaction with
+ * the affected initiator.
+ */
+ if (cmd != NULL)
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+#endif
+ }
+ return;
+
+write_error:
+ PRINT_ERROR("Error writing to '%s' - error %d", dev->pr_file_name, res);
+
+write_error_close:
+ filp_close(file, NULL);
+ {
+ struct nameidata nd;
+ int rc;
+
+ rc = path_lookup(dev->pr_file_name, 0, &nd);
+ if (!rc)
+ scst_pr_vfs_unlink_and_put(&nd);
+ else
+ TRACE_PR("Unable to lookup '%s' - error %d",
+ dev->pr_file_name, rc);
+ }
+ goto out_set_fs;
+}
+
+int scst_pr_check_pr_path(void)
+{
+ int res;
+ struct nameidata nd;
+ mm_segment_t old_fs = get_fs();
+
+ set_fs(KERNEL_DS);
+
+ res = path_lookup(SCST_PR_DIR, 0, &nd);
+ if (res != 0) {
+ PRINT_ERROR("Unable to find %s (err %d), you should create "
+ "this directory manually or reinstall SCST",
+ SCST_PR_DIR, res);
+ goto out_setfs;
+ }
+
+ scst_pr_path_put(&nd);
+
+out_setfs:
+ set_fs(old_fs);
+ return res;
+}
+
+/* Called under scst_mutex */
+int scst_pr_init_dev(struct scst_device *dev)
+{
+ int res = 0;
+ uint8_t q;
+ int name_len;
+
+ name_len = snprintf(&q, sizeof(q), "%s/%s", SCST_PR_DIR, dev->virt_name) + 1;
+ dev->pr_file_name = kmalloc(name_len, GFP_KERNEL);
+ if (dev->pr_file_name == NULL) {
+ PRINT_ERROR("Allocation of device '%s' file path failed",
+ dev->virt_name);
+ res = -ENOMEM;
+ goto out;
+ } else
+ snprintf(dev->pr_file_name, name_len, "%s/%s", SCST_PR_DIR,
+ dev->virt_name);
+
+ name_len = snprintf(&q, sizeof(q), "%s/%s.1", SCST_PR_DIR, dev->virt_name) + 1;
+ dev->pr_file_name1 = kmalloc(name_len, GFP_KERNEL);
+ if (dev->pr_file_name1 == NULL) {
+ PRINT_ERROR("Allocation of device '%s' backup file path failed",
+ dev->virt_name);
+ res = -ENOMEM;
+ goto out_free_name;
+ } else
+ snprintf(dev->pr_file_name1, name_len, "%s/%s.1", SCST_PR_DIR,
+ dev->virt_name);
+
+ res = scst_pr_load_device_file(dev);
+ if (res == -ENOENT)
+ res = 0;
+
+ if (res != 0)
+ goto out_free_name1;
+
+out:
+ return res;
+
+out_free_name1:
+ kfree(dev->pr_file_name1);
+ dev->pr_file_name1 = NULL;
+
+out_free_name:
+ kfree(dev->pr_file_name);
+ dev->pr_file_name = NULL;
+ goto out;
+}
+
+/* Called under scst_mutex */
+void scst_pr_clear_dev(struct scst_device *dev)
+{
+ struct scst_dev_registrant *reg, *tmp_reg;
+
+ list_for_each_entry_safe(reg, tmp_reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ scst_pr_remove_registrant(dev, reg);
+ }
+
+ kfree(dev->pr_file_name);
+ kfree(dev->pr_file_name1);
+ return;
+}
+
+/* Called under scst_mutex */
+int scst_pr_init_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+ int res = 0;
+ struct scst_dev_registrant *reg;
+ struct scst_device *dev = tgt_dev->dev;
+ const uint8_t *transport_id = tgt_dev->sess->transport_id;
+ const uint16_t rel_tgt_id = tgt_dev->sess->tgt->rel_tgt_id;
+
+ if (tgt_dev->sess->transport_id == NULL)
+ goto out;
+
+ scst_pr_write_lock(dev);
+
+ reg = scst_pr_find_reg(dev, transport_id, rel_tgt_id);
+ if ((reg != NULL) && (reg->tgt_dev == NULL)) {
+ TRACE_PR("Assigning reg %s/%d (%p) to tgt_dev %p (dev %s)",
+ debug_transport_id_to_initiator_name(transport_id),
+ rel_tgt_id, reg, tgt_dev, dev->virt_name);
+ tgt_dev->registrant = reg;
+ reg->tgt_dev = tgt_dev;
+ }
+
+ scst_pr_write_unlock(dev);
+
+out:
+ return res;
+}
+
+/* Called under scst_mutex */
+void scst_pr_clear_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+
+ if (tgt_dev->registrant != NULL) {
+ struct scst_dev_registrant *reg = tgt_dev->registrant;
+ struct scst_device *dev = tgt_dev->dev;
+ struct scst_tgt_dev *t;
+
+ scst_pr_write_lock(dev);
+
+ tgt_dev->registrant = NULL;
+ reg->tgt_dev = NULL;
+
+ /* Just in case, actually. It should never happen. */
+ list_for_each_entry(t, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (t == tgt_dev)
+ continue;
+ if ((t->sess->tgt->rel_tgt_id == reg->rel_tgt_id) &&
+ tid_equal(t->sess->transport_id, reg->transport_id)) {
+ TRACE_PR("Reassigning reg %s/%d (%p) to tgt_dev "
+ "%p (being cleared tgt_dev %p)",
+ debug_transport_id_to_initiator_name(
+ reg->transport_id),
+ reg->rel_tgt_id, reg, t, tgt_dev);
+ t->registrant = reg;
+ reg->tgt_dev = t;
+ break;
+ }
+ }
+
+ scst_pr_write_unlock(dev);
+ }
+ return;
+}
+
+/* Called with dev_pr_mutex locked. Might also be called under scst_mutex2. */
+static int scst_pr_register_with_spec_i_pt(struct scst_cmd *cmd,
+ const uint16_t rel_tgt_id, uint8_t *buffer, int buffer_size,
+ struct list_head *rollback_list)
+{
+ int res = 0;
+ int offset, ext_size;
+ __be64 action_key;
+ struct scst_device *dev = cmd->dev;
+ struct scst_dev_registrant *reg;
+ uint8_t *transport_id;
+
+ action_key = get_unaligned((__be64 *)&buffer[8]);
+
+ ext_size = be32_to_cpu(get_unaligned((__be32 *)&buffer[24]));
+ if ((ext_size + 28) > buffer_size) {
+ TRACE_PR("Invalid buffer size %d (max %d)", buffer_size,
+ ext_size + 28);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_parameter_list_length_invalid));
+ res = -EINVAL;
+ goto out;
+ }
+
+ offset = 0;
+ while (offset < ext_size) {
+ transport_id = &buffer[28 + offset];
+
+ if ((offset + tid_size(transport_id)) > ext_size) {
+ TRACE_PR("Invalid transport_id size %d (max %d)",
+ tid_size(transport_id), ext_size - offset);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_parm_list));
+ res = -EINVAL;
+ goto out;
+ }
+ tid_secure(transport_id);
+ offset += tid_size(transport_id);
+ }
+
+ offset = 0;
+ while (offset < ext_size) {
+ struct scst_tgt_dev *t;
+
+ transport_id = &buffer[28 + offset];
+
+ TRACE_PR("rel_tgt_id %d, transport_id %s", rel_tgt_id,
+ debug_transport_id_to_initiator_name(transport_id));
+
+ if ((transport_id[0] & 0x0f) == SCSI_TRANSPORTID_PROTOCOLID_ISCSI &&
+ (transport_id[0] & 0xc0) == 0) {
+ TRACE_PR("Wildcard iSCSI TransportID %s",
+ &transport_id[4]);
+ /*
+ * We can't use scst_mutex here, because of the
+ * circular locking dependency with dev_pr_mutex.
+ */
+ spin_lock_bh(&dev->dev_lock);
+ list_for_each_entry(t, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ /*
+ * We must go over all matching tgt_devs and
+ * register them on the requested rel_tgt_id
+ */
+ if (!tid_equal(t->sess->transport_id,
+ transport_id))
+ continue;
+
+ reg = scst_pr_find_reg(dev,
+ t->sess->transport_id, rel_tgt_id);
+ if (reg == NULL) {
+ reg = scst_pr_add_registrant(dev,
+ t->sess->transport_id,
+ rel_tgt_id, action_key, true);
+ if (reg == NULL) {
+ spin_unlock_bh(&dev->dev_lock);
+ scst_set_busy(cmd);
+ res = -ENOMEM;
+ goto out;
+ }
+ } else if (reg->key != action_key) {
+ TRACE_PR("Changing key of reg %p "
+ "(tgt_dev %p)", reg, t);
+ reg->rollback_key = reg->key;
+ reg->key = action_key;
+ } else
+ continue;
+
+ list_add_tail(®->aux_list_entry,
+ rollback_list);
+ }
+ spin_unlock_bh(&dev->dev_lock);
+ } else {
+ reg = scst_pr_find_reg(dev, transport_id, rel_tgt_id);
+ if (reg != NULL) {
+ if (reg->key == action_key)
+ goto next;
+ TRACE_PR("Changing key of reg %p (tgt_dev %p)",
+ reg, reg->tgt_dev);
+ reg->rollback_key = reg->key;
+ reg->key = action_key;
+ } else {
+ reg = scst_pr_add_registrant(dev, transport_id,
+ rel_tgt_id, action_key, false);
+ if (reg == NULL) {
+ scst_set_busy(cmd);
+ res = -ENOMEM;
+ goto out;
+ }
+ }
+
+ list_add_tail(®->aux_list_entry,
+ rollback_list);
+ }
+next:
+ offset += tid_size(transport_id);
+ }
+out:
+ return res;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+static void scst_pr_unregister(struct scst_device *dev,
+ struct scst_dev_registrant *reg)
+{
+ bool is_holder;
+ uint8_t pr_type;
+
+ TRACE_PR("Unregistering key %0llx", reg->key);
+
+ is_holder = scst_pr_is_holder(dev, reg);
+ pr_type = dev->pr_type;
+
+ scst_pr_remove_registrant(dev, reg);
+
+ if (is_holder && !dev->pr_is_set) {
+ /* A registration just released */
+ switch (pr_type) {
+ case TYPE_WRITE_EXCLUSIVE_REGONLY:
+ case TYPE_EXCLUSIVE_ACCESS_REGONLY:
+ scst_pr_send_ua_all(dev, NULL,
+ SCST_LOAD_SENSE(scst_sense_reservation_released));
+ break;
+ }
+ }
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+static void scst_pr_unregister_all_tg_pt(struct scst_device *dev,
+ const uint8_t *transport_id)
+{
+ struct scst_tgt_template *tgtt;
+ uint8_t proto_id = transport_id[0] & 0x0f;
+
+ /*
+ * We can't use scst_mutex here, because of the circular locking
+ * dependency with dev_pr_mutex.
+ */
+ mutex_lock(&scst_mutex2);
+
+ list_for_each_entry(tgtt, &scst_template_list, scst_template_list_entry) {
+ struct scst_tgt *tgt;
+
+ if (tgtt->get_initiator_port_transport_id == NULL)
+ continue;
+
+ if (tgtt->get_initiator_port_transport_id(NULL, NULL) != proto_id)
+ continue;
+
+ list_for_each_entry(tgt, &tgtt->tgt_list, tgt_list_entry) {
+ struct scst_dev_registrant *reg;
+
+ reg = scst_pr_find_reg(dev, transport_id,
+ tgt->rel_tgt_id);
+ if (reg == NULL)
+ continue;
+
+ scst_pr_unregister(dev, reg);
+ }
+ }
+
+ mutex_unlock(&scst_mutex2);
+ return;
+}
+
+/* Called with dev_pr_mutex locked. Might also be called under scst_mutex2. */
+static int scst_pr_register_on_tgt_id(struct scst_cmd *cmd,
+ const uint16_t rel_tgt_id, uint8_t *buffer, int buffer_size,
+ bool spec_i_pt, struct list_head *rollback_list)
+{
+ int res;
+
+ TRACE_PR("rel_tgt_id %d, spec_i_pt %d", rel_tgt_id, spec_i_pt);
+
+ if (spec_i_pt) {
+ res = scst_pr_register_with_spec_i_pt(cmd, rel_tgt_id, buffer,
+ buffer_size, rollback_list);
+ if (res != 0)
+ goto out;
+ }
+
+ /* tgt_dev can be among TIDs for scst_pr_register_with_spec_i_pt() */
+
+ if (scst_pr_find_reg(cmd->dev, cmd->sess->transport_id, rel_tgt_id) == NULL) {
+ __be64 action_key;
+ struct scst_dev_registrant *reg;
+
+ action_key = get_unaligned((__be64 *)&buffer[8]);
+
+ reg = scst_pr_add_registrant(cmd->dev, cmd->sess->transport_id,
+ rel_tgt_id, action_key, false);
+ if (reg == NULL) {
+ res = -ENOMEM;
+ scst_set_busy(cmd);
+ goto out;
+ }
+
+ list_add_tail(®->aux_list_entry, rollback_list);
+ }
+
+out:
+ return res;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+static int scst_pr_register_all_tg_pt(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size, bool spec_i_pt, struct list_head *rollback_list)
+{
+ int res = 0;
+ struct scst_tgt_template *tgtt;
+ uint8_t proto_id = cmd->sess->transport_id[0] & 0x0f;
+
+ /*
+ * We can't use scst_mutex here, because of the circular locking
+ * dependency with dev_pr_mutex.
+ */
+ mutex_lock(&scst_mutex2);
+
+ list_for_each_entry(tgtt, &scst_template_list, scst_template_list_entry) {
+ struct scst_tgt *tgt;
+
+ if (tgtt->get_initiator_port_transport_id == NULL)
+ continue;
+
+ if (tgtt->get_initiator_port_transport_id(NULL, NULL) != proto_id)
+ continue;
+
+ TRACE_PR("tgtt %s, spec_i_pt %d", tgtt->name, spec_i_pt);
+
+ list_for_each_entry(tgt, &tgtt->tgt_list, tgt_list_entry) {
+ if (tgt->rel_tgt_id == 0)
+ continue;
+ TRACE_PR("tgt %s, rel_tgt_id %d", tgt->tgt_name,
+ tgt->rel_tgt_id);
+ res = scst_pr_register_on_tgt_id(cmd, tgt->rel_tgt_id,
+ buffer, buffer_size, spec_i_pt, rollback_list);
+ if (res != 0)
+ goto out_unlock;
+ }
+ }
+
+out_unlock:
+ mutex_unlock(&scst_mutex2);
+ return res;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+static int __scst_pr_register(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size, bool spec_i_pt, bool all_tg_pt)
+{
+ int res;
+ struct scst_dev_registrant *reg, *treg;
+ LIST_HEAD(rollback_list);
+
+ if (all_tg_pt) {
+ res = scst_pr_register_all_tg_pt(cmd, buffer, buffer_size,
+ spec_i_pt, &rollback_list);
+ if (res != 0)
+ goto out_rollback;
+ } else {
+ res = scst_pr_register_on_tgt_id(cmd,
+ cmd->sess->tgt->rel_tgt_id, buffer, buffer_size,
+ spec_i_pt, &rollback_list);
+ if (res != 0)
+ goto out_rollback;
+ }
+
+ list_for_each_entry(reg, &rollback_list, aux_list_entry) {
+ reg->rollback_key = 0;
+ }
+
+out:
+ return res;
+
+out_rollback:
+ list_for_each_entry_safe(reg, treg, &rollback_list, aux_list_entry) {
+ list_del(®->aux_list_entry);
+ if (reg->rollback_key == 0)
+ scst_pr_remove_registrant(cmd->dev, reg);
+ else {
+ reg->key = reg->rollback_key;
+ reg->rollback_key = 0;
+ }
+ }
+ goto out;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_register(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size)
+{
+ int aptpl, spec_i_pt, all_tg_pt;
+ __be64 key, action_key;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_session *sess = cmd->sess;
+ struct scst_dev_registrant *reg;
+
+ aptpl = buffer[20] & 0x01;
+ spec_i_pt = (buffer[20] >> 3) & 0x01;
+ all_tg_pt = (buffer[20] >> 2) & 0x01;
+ key = get_unaligned((__be64 *)&buffer[0]);
+ action_key = get_unaligned((__be64 *)&buffer[8]);
+
+ if (spec_i_pt == 0 && buffer_size != 24) {
+ TRACE_PR("Invalid buffer size %d", buffer_size);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_parameter_list_length_invalid));
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+
+ TRACE_PR("Register: initiator %s/%d (%p), key %0llx, action_key %0llx "
+ "(tgt_dev %p)",
+ debug_transport_id_to_initiator_name(sess->transport_id),
+ sess->tgt->rel_tgt_id, reg, key, action_key, tgt_dev);
+
+ if (reg == NULL) {
+ TRACE_PR("tgt_dev %p is not registered yet - registering",
+ tgt_dev);
+ if (key) {
+ TRACE_PR("%s", "Key must be zero on new registration");
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+ if (action_key) {
+ int rc = __scst_pr_register(cmd, buffer, buffer_size,
+ spec_i_pt, all_tg_pt);
+ if (rc != 0)
+ goto out;
+ } else
+ TRACE_PR("%s", "Doing nothing - action_key is zero");
+ } else {
+ if (reg->key != key) {
+ TRACE_PR("tgt_dev %p already registered - reservation "
+ "key %0llx mismatch", tgt_dev, reg->key);
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+ if (spec_i_pt) {
+ TRACE_PR("%s", "spec_i_pt must be zero in this case");
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+ if (action_key == 0) {
+ if (all_tg_pt)
+ scst_pr_unregister_all_tg_pt(dev,
+ sess->transport_id);
+ else
+ scst_pr_unregister(dev, reg);
+ } else
+ reg->key = action_key;
+ }
+
+ dev->pr_generation++;
+
+ dev->pr_aptpl = aptpl;
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_register_and_ignore(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size)
+{
+ int aptpl, all_tg_pt;
+ __be64 action_key;
+ struct scst_dev_registrant *reg = NULL;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_session *sess = cmd->sess;
+
+ aptpl = buffer[20] & 0x01;
+ all_tg_pt = (buffer[20] >> 2) & 0x01;
+ action_key = get_unaligned((__be64 *)&buffer[8]);
+
+ if (buffer_size != 24) {
+ TRACE_PR("Invalid buffer size %d", buffer_size);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_parameter_list_length_invalid));
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+
+ TRACE_PR("Register and ignore: initiator %s/%d (%p), action_key "
+ "%016llx (tgt_dev %p)",
+ debug_transport_id_to_initiator_name(sess->transport_id),
+ sess->tgt->rel_tgt_id, reg, action_key, tgt_dev);
+
+ if (reg == NULL) {
+ TRACE_PR("Tgt_dev %p is not registered yet - trying to "
+ "register", tgt_dev);
+ if (action_key) {
+ int rc = __scst_pr_register(cmd, buffer, buffer_size,
+ false, all_tg_pt);
+ if (rc != 0)
+ goto out;
+ } else
+ TRACE_PR("%s", "Doing nothing, action_key is zero");
+ } else {
+ if (action_key == 0) {
+ if (all_tg_pt)
+ scst_pr_unregister_all_tg_pt(dev,
+ sess->transport_id);
+ else
+ scst_pr_unregister(dev, reg);
+ } else
+ reg->key = action_key;
+ }
+
+ dev->pr_generation++;
+
+ dev->pr_aptpl = aptpl;
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_register_and_move(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size)
+{
+ int aptpl;
+ int unreg;
+ int tid_buffer_size;
+ __be64 key, action_key;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_session *sess = cmd->sess;
+ struct scst_dev_registrant *reg, *reg_move;
+ const uint8_t *transport_id = NULL;
+ uint8_t *transport_id_move = NULL;
+ uint16_t rel_tgt_id_move;
+
+ aptpl = buffer[17] & 0x01;
+ key = get_unaligned((__be64 *)&buffer[0]);
+ action_key = get_unaligned((__be64 *)&buffer[8]);
+ unreg = (buffer[17] >> 1) & 0x01;
+ tid_buffer_size = be32_to_cpu(get_unaligned((__be32 *)&buffer[20]));
+
+ if ((tid_buffer_size + 24) > buffer_size) {
+ TRACE_PR("Invalid buffer size %d (%d)",
+ buffer_size, tid_buffer_size + 24);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_parm_list));
+ goto out;
+ }
+
+ if (tid_buffer_size < 24) {
+ TRACE_PR("%s", "Transport id buffer too small");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_parm_list));
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+ /* We already checked reg is not NULL */
+ if (reg->key != key) {
+ TRACE_PR("Registrant's %s/%d (%p) key %016llx mismatch with "
+ "%016llx (tgt_dev %p)",
+ debug_transport_id_to_initiator_name(reg->transport_id),
+ reg->rel_tgt_id, reg, reg->key, key, tgt_dev);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+
+ if (!dev->pr_is_set) {
+ TRACE_PR("%s", "There must be a PR");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+
+ /*
+ * This check also required by table "PERSISTENT RESERVE OUT service
+ * actions that are allowed in the presence of various reservations".
+ */
+ if (!scst_pr_is_holder(dev, reg)) {
+ TRACE_PR("Registrant %s/%d (%p) is not a holder (tgt_dev %p)",
+ debug_transport_id_to_initiator_name(
+ reg->transport_id), reg->rel_tgt_id,
+ reg, tgt_dev);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+
+ if (action_key == 0) {
+ TRACE_PR("%s", "Action key must be non-zero");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+
+ transport_id = sess->transport_id;
+ transport_id_move = (uint8_t *)&buffer[24];
+ rel_tgt_id_move = be16_to_cpu(get_unaligned((__be16 *)&buffer[18]));
+
+ if ((tid_size(transport_id_move) + 24) > buffer_size) {
+ TRACE_PR("Invalid buffer size %d (%d)",
+ buffer_size, tid_size(transport_id_move) + 24);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_parm_list));
+ goto out;
+ }
+
+ tid_secure(transport_id_move);
+
+ if (dev->pr_type == TYPE_WRITE_EXCLUSIVE_ALL_REG ||
+ dev->pr_type == TYPE_EXCLUSIVE_ACCESS_ALL_REG) {
+ TRACE_PR("Unable to finish operation due to wrong reservation "
+ "type %02x", dev->pr_type);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+
+ if (tid_equal(transport_id, transport_id_move)) {
+ TRACE_PR("%s", "Equal transport id's");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_parm_list));
+ goto out;
+ }
+
+ reg_move = scst_pr_find_reg(dev, transport_id_move, rel_tgt_id_move);
+ if (reg_move == NULL) {
+ reg_move = scst_pr_add_registrant(dev, transport_id_move,
+ rel_tgt_id_move, action_key, false);
+ if (reg_move == NULL) {
+ scst_set_busy(cmd);
+ goto out;
+ }
+ } else if (reg_move->key != action_key) {
+ TRACE_PR("Changing key for reg %p", reg);
+ reg_move->key = action_key;
+ }
+
+ TRACE_PR("Register and move: from initiator %s/%d (%p, tgt_dev %p) to "
+ "initiator %s/%d (%p, tgt_dev %p), key %016llx (unreg %d)",
+ debug_transport_id_to_initiator_name(reg->transport_id),
+ reg->rel_tgt_id, reg, reg->tgt_dev,
+ debug_transport_id_to_initiator_name(transport_id_move),
+ rel_tgt_id_move, reg_move, reg_move->tgt_dev, action_key,
+ unreg);
+
+ /* Move the holder */
+ scst_pr_set_holder(dev, reg_move, dev->pr_scope, dev->pr_type);
+
+ if (unreg)
+ scst_pr_remove_registrant(dev, reg);
+
+ dev->pr_generation++;
+
+ dev->pr_aptpl = aptpl;
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_reserve(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size)
+{
+ uint8_t scope, type;
+ __be64 key;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_dev_registrant *reg;
+
+ key = get_unaligned((__be64 *)&buffer[0]);
+ scope = (cmd->cdb[2] & 0x0f) >> 4;
+ type = cmd->cdb[2] & 0x0f;
+
+ if (buffer_size != 24) {
+ TRACE_PR("Invalid buffer size %d", buffer_size);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_parameter_list_length_invalid));
+ goto out;
+ }
+
+ if (!scst_pr_type_valid(type)) {
+ TRACE_PR("Invalid reservation type %d", type);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+
+ if (((cmd->cdb[2] & 0x0f) >> 4) != SCOPE_LU) {
+ TRACE_PR("Invalid reservation scope %d", scope);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+
+ TRACE_PR("Reserve: initiator %s/%d (%p), key %016llx, scope %d, "
+ "type %d (tgt_dev %p)",
+ debug_transport_id_to_initiator_name(cmd->sess->transport_id),
+ cmd->sess->tgt->rel_tgt_id, reg, key, scope, type, tgt_dev);
+
+ /* We already checked reg is not NULL */
+ if (reg->key != key) {
+ TRACE_PR("Registrant's %p key %016llx mismatch with %016llx",
+ reg, reg->key, key);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+
+ if (!dev->pr_is_set)
+ scst_pr_set_holder(dev, reg, scope, type);
+ else {
+ if (!scst_pr_is_holder(dev, reg)) {
+ /*
+ * This check also required by table "PERSISTENT
+ * RESERVE OUT service actions that are allowed in the
+ * presence of various reservations".
+ */
+ TRACE_PR("Only holder can override - reg %p is not a "
+ "holder", reg);
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ } else {
+ if (dev->pr_scope != scope || dev->pr_type != type) {
+ TRACE_PR("Error overriding scope or type for "
+ "reg %p", reg);
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ } else
+ TRACE_PR("Do nothing: reservation of reg %p "
+ "is the same", reg);
+ }
+ }
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_release(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size)
+{
+ int scope, type;
+ __be64 key;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_dev_registrant *reg;
+ uint8_t cur_pr_type;
+
+ key = get_unaligned((__be64 *)&buffer[0]);
+ scope = (cmd->cdb[2] & 0x0f) >> 4;
+ type = cmd->cdb[2] & 0x0f;
+
+ if (buffer_size != 24) {
+ TRACE_PR("Invalid buffer size %d", buffer_size);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_parameter_list_length_invalid));
+ goto out;
+ }
+
+ if (!dev->pr_is_set) {
+ TRACE_PR("%s", "There is no PR - do nothing");
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+
+ TRACE_PR("Release: initiator %s/%d (%p), key %016llx, scope %d, type "
+ "%d (tgt_dev %p)", debug_transport_id_to_initiator_name(
+ cmd->sess->transport_id),
+ cmd->sess->tgt->rel_tgt_id, reg, key, scope, type, tgt_dev);
+
+ /* We already checked reg is not NULL */
+ if (reg->key != key) {
+ TRACE_PR("Registrant's %p key %016llx mismatch with %016llx",
+ reg, reg->key, key);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+
+ if (!scst_pr_is_holder(dev, reg)) {
+ TRACE_PR("Registrant %p is not a holder - do nothing", reg);
+ goto out;
+ }
+
+ if (dev->pr_scope != scope || dev->pr_type != type) {
+ TRACE_PR("%s", "Released scope or type do not match with "
+ "holder");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_release));
+ goto out;
+ }
+
+ cur_pr_type = dev->pr_type; /* it will be cleared */
+
+ scst_pr_clear_reservation(dev);
+
+ switch (cur_pr_type) {
+ case TYPE_WRITE_EXCLUSIVE_REGONLY:
+ case TYPE_EXCLUSIVE_ACCESS_REGONLY:
+ case TYPE_WRITE_EXCLUSIVE_ALL_REG:
+ case TYPE_EXCLUSIVE_ACCESS_ALL_REG:
+ scst_pr_send_ua_all(dev, reg,
+ SCST_LOAD_SENSE(scst_sense_reservation_released));
+ }
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_clear(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size)
+{
+ int scope, type;
+ __be64 key;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_dev_registrant *reg, *r, *t;
+
+ key = get_unaligned((__be64 *)&buffer[0]);
+ scope = (cmd->cdb[2] & 0x0f) >> 4;
+ type = cmd->cdb[2] & 0x0f;
+
+ if (buffer_size != 24) {
+ TRACE_PR("Invalid buffer size %d", buffer_size);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_parameter_list_length_invalid));
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+
+ TRACE_PR("Clear: initiator %s/%d (%p), key %016llx (tgt_dev %p)",
+ debug_transport_id_to_initiator_name(cmd->sess->transport_id),
+ cmd->sess->tgt->rel_tgt_id, reg, key, tgt_dev);
+
+ /* We already checked reg is not NULL */
+ if (reg->key != key) {
+ TRACE_PR("Registrant's %p key %016llx mismatch with %016llx",
+ reg, reg->key, key);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+
+ scst_pr_send_ua_all(dev, reg,
+ SCST_LOAD_SENSE(scst_sense_reservation_preempted));
+
+ list_for_each_entry_safe(r, t, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ scst_pr_remove_registrant(dev, r);
+ }
+
+ dev->pr_generation++;
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return;
+}
+
+static void scst_pr_do_preempt(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size, bool abort)
+{
+ __be64 key, action_key;
+ int scope, type;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_dev_registrant *reg, *r, *rt;
+ LIST_HEAD(preempt_list);
+
+ key = get_unaligned((__be64 *)&buffer[0]);
+ action_key = get_unaligned((__be64 *)&buffer[8]);
+ scope = (cmd->cdb[2] & 0x0f) >> 4;
+ type = cmd->cdb[2] & 0x0f;
+
+ if (buffer_size != 24) {
+ TRACE_PR("Invalid buffer size %d", buffer_size);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_parameter_list_length_invalid));
+ goto out;
+ }
+
+ if (!scst_pr_type_valid(type)) {
+ TRACE_PR("Invalid reservation type %d", type);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+
+ TRACE_PR("Preempt%s: initiator %s/%d (%p), key %016llx, action_key "
+ "%016llx, scope %x type %x (tgt_dev %p)",
+ abort ? " and abort" : "",
+ debug_transport_id_to_initiator_name(cmd->sess->transport_id),
+ cmd->sess->tgt->rel_tgt_id, reg, key, action_key, scope, type,
+ tgt_dev);
+
+ /* We already checked reg is not NULL */
+ if (reg->key != key) {
+ TRACE_PR("Registrant's %p key %016llx mismatch with %016llx",
+ reg, reg->key, key);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+ }
+
+ if (!dev->pr_is_set) {
+ scst_pr_find_registrants_list_key(dev, action_key,
+ &preempt_list);
+ if (list_empty(&preempt_list))
+ goto out_error;
+ list_for_each_entry_safe(r, rt, &preempt_list, aux_list_entry) {
+ if (r != reg)
+ scst_pr_send_ua_reg(dev, r, SCST_LOAD_SENSE(
+ scst_sense_registrations_preempted));
+ scst_pr_remove_registrant(dev, r);
+ }
+ goto done;
+ }
+
+ if (dev->pr_type == TYPE_WRITE_EXCLUSIVE_ALL_REG ||
+ dev->pr_type == TYPE_EXCLUSIVE_ACCESS_ALL_REG) {
+ if (action_key == 0) {
+ scst_pr_find_registrants_list_all(dev, reg,
+ &preempt_list);
+ list_for_each_entry_safe(r, rt, &preempt_list,
+ aux_list_entry) {
+ if (r != reg)
+ scst_pr_send_ua_reg(dev, r,
+ SCST_LOAD_SENSE(
+ scst_sense_registrations_preempted));
+ else
+ reg = NULL;
+ scst_pr_remove_registrant(dev, r);
+ }
+ if (reg != NULL)
+ scst_pr_set_holder(dev, reg, scope, type);
+ } else {
+ scst_pr_find_registrants_list_key(dev, action_key,
+ &preempt_list);
+ if (list_empty(&preempt_list))
+ goto out_error;
+ list_for_each_entry_safe(r, rt, &preempt_list,
+ aux_list_entry) {
+ if (r != reg)
+ scst_pr_send_ua_reg(dev, r,
+ SCST_LOAD_SENSE(
+ scst_sense_registrations_preempted));
+ else
+ reg = NULL;
+ scst_pr_remove_registrant(dev, r);
+ }
+ }
+ goto done;
+ }
+
+ BUG_ON(dev->pr_holder == NULL);
+
+ if (dev->pr_holder->key != action_key) {
+ if (action_key == 0) {
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_parm_list));
+ goto out;
+ } else {
+ scst_pr_find_registrants_list_key(dev, action_key,
+ &preempt_list);
+ if (list_empty(&preempt_list))
+ goto out_error;
+ list_for_each_entry_safe(r, rt, &preempt_list,
+ aux_list_entry) {
+ if (r != reg)
+ scst_pr_send_ua_reg(dev, r,
+ SCST_LOAD_SENSE(
+ scst_sense_registrations_preempted));
+ else
+ reg = NULL;
+ scst_pr_remove_registrant(dev, r);
+ }
+ goto done;
+ }
+ }
+
+ scst_pr_find_registrants_list_key(dev, action_key,
+ &preempt_list);
+
+ list_for_each_entry_safe(r, rt, &preempt_list, aux_list_entry) {
+ if (abort)
+ scst_pr_abort_reg(dev, cmd, r);
+ if (r != reg)
+ scst_pr_send_ua_reg(dev, r, SCST_LOAD_SENSE(
+ scst_sense_registrations_preempted));
+ else
+ reg = NULL;
+ scst_pr_remove_registrant(dev, r);
+ }
+
+ if (dev->pr_type != type || dev->pr_scope != scope)
+ list_for_each_entry(r, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ if (r != reg)
+ scst_pr_send_ua_reg(dev, r, SCST_LOAD_SENSE(
+ scst_sense_reservation_released));
+ }
+
+ if (reg != NULL)
+ scst_pr_set_holder(dev, reg, scope, type);
+
+done:
+ dev->pr_generation++;
+
+ scst_pr_dump_prs(dev, false);
+
+out:
+ return;
+
+out_error:
+ TRACE_PR("Invalid key %016llx", action_key);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_preempt(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size)
+{
+
+ scst_pr_do_preempt(cmd, buffer, buffer_size, false);
+ return;
+}
+
+static void scst_cmd_done_pr_preempt(struct scst_cmd *cmd, int next_state,
+ enum scst_exec_context pref_context)
+{
+ void (*saved_cmd_done) (struct scst_cmd *cmd, int next_state,
+ enum scst_exec_context pref_context);
+
+ saved_cmd_done = NULL; /* to remove warning that it's used not inited */
+
+ if (cmd->pr_abort_counter != NULL) {
+ if (!atomic_dec_and_test(&cmd->pr_abort_counter->pr_abort_pending_cnt))
+ goto out;
+ saved_cmd_done = cmd->pr_abort_counter->saved_cmd_done;
+ kfree(cmd->pr_abort_counter);
+ cmd->pr_abort_counter = NULL;
+ }
+
+ saved_cmd_done(cmd, next_state, pref_context);
+
+out:
+ return;
+}
+
+/*
+ * Called with dev_pr_mutex locked, no IRQ. Expects session_list_lock
+ * not locked
+ */
+void scst_pr_preempt_and_abort(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size)
+{
+
+ cmd->pr_abort_counter = kzalloc(sizeof(*cmd->pr_abort_counter),
+ GFP_KERNEL);
+ if (cmd->pr_abort_counter == NULL) {
+ PRINT_ERROR("Unable to allocate PR abort counter (size %zd)",
+ sizeof(*cmd->pr_abort_counter));
+ scst_set_busy(cmd);
+ goto out;
+ }
+
+ /* 1 to protect cmd from be done by the TM thread too early */
+ atomic_set(&cmd->pr_abort_counter->pr_abort_pending_cnt, 1);
+ atomic_set(&cmd->pr_abort_counter->pr_aborting_cnt, 1);
+ init_completion(&cmd->pr_abort_counter->pr_aborting_cmpl);
+
+ cmd->pr_abort_counter->saved_cmd_done = cmd->scst_cmd_done;
+ cmd->scst_cmd_done = scst_cmd_done_pr_preempt;
+
+ scst_pr_do_preempt(cmd, buffer, buffer_size, true);
+
+ if (!atomic_dec_and_test(&cmd->pr_abort_counter->pr_aborting_cnt))
+ wait_for_completion(&cmd->pr_abort_counter->pr_aborting_cmpl);
+
+out:
+ return;
+}
+
+/* Checks if this is a Compatible Reservation Handling (CRH) case */
+bool scst_pr_crh_case(struct scst_cmd *cmd)
+{
+ bool allowed;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_dev_registrant *reg;
+ uint8_t type;
+
+ TRACE_DBG("Test if there is a CRH case for command %s (0x%x) from "
+ "%s", cmd->op_name, cmd->cdb[0], cmd->sess->initiator_name);
+
+ if (!dev->pr_is_set) {
+ TRACE_PR("%s", "PR not set");
+ allowed = false;
+ goto out;
+ }
+
+ reg = tgt_dev->registrant;
+ type = dev->pr_type;
+
+ switch (type) {
+ case TYPE_WRITE_EXCLUSIVE:
+ case TYPE_EXCLUSIVE_ACCESS:
+ WARN_ON(dev->pr_holder == NULL);
+ if (reg == dev->pr_holder)
+ allowed = true;
+ else
+ allowed = false;
+ break;
+
+ case TYPE_WRITE_EXCLUSIVE_REGONLY:
+ case TYPE_EXCLUSIVE_ACCESS_REGONLY:
+ case TYPE_WRITE_EXCLUSIVE_ALL_REG:
+ case TYPE_EXCLUSIVE_ACCESS_ALL_REG:
+ allowed = (reg != NULL);
+ break;
+
+ default:
+ PRINT_ERROR("Invalid PR type %x", type);
+ allowed = false;
+ break;
+ }
+
+ if (!allowed)
+ TRACE_PR("Command %s (0x%x) from %s rejected due to not CRH "
+ "reservation", cmd->op_name, cmd->cdb[0],
+ cmd->sess->initiator_name);
+ else
+ TRACE_DBG("Command %s (0x%x) from %s is allowed to execute "
+ "due to CRH", cmd->op_name, cmd->cdb[0],
+ cmd->sess->initiator_name);
+
+out:
+ return allowed;
+
+}
+
+/* Check if command allowed in presence of reservation */
+bool scst_pr_is_cmd_allowed(struct scst_cmd *cmd)
+{
+ bool allowed;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_dev_registrant *reg;
+ uint8_t type;
+ bool unlock;
+
+ unlock = scst_pr_read_lock(dev);
+
+ TRACE_DBG("Testing if command %s (0x%x) from %s allowed to execute",
+ cmd->op_name, cmd->cdb[0], cmd->sess->initiator_name);
+
+ /* Recheck, because it can change while we were waiting for the lock */
+ if (unlikely(!dev->pr_is_set)) {
+ allowed = true;
+ goto out_unlock;
+ }
+
+ reg = tgt_dev->registrant;
+ type = dev->pr_type;
+
+ switch (type) {
+ case TYPE_WRITE_EXCLUSIVE:
+ if (reg && reg == dev->pr_holder)
+ allowed = true;
+ else
+ allowed = (cmd->op_flags & SCST_WRITE_EXCL_ALLOWED) != 0;
+ break;
+
+ case TYPE_EXCLUSIVE_ACCESS:
+ if (reg && reg == dev->pr_holder)
+ allowed = true;
+ else
+ allowed = (cmd->op_flags & SCST_EXCL_ACCESS_ALLOWED) != 0;
+ break;
+
+ case TYPE_WRITE_EXCLUSIVE_REGONLY:
+ case TYPE_WRITE_EXCLUSIVE_ALL_REG:
+ if (reg)
+ allowed = true;
+ else
+ allowed = (cmd->op_flags & SCST_WRITE_EXCL_ALLOWED) != 0;
+ break;
+
+ case TYPE_EXCLUSIVE_ACCESS_REGONLY:
+ case TYPE_EXCLUSIVE_ACCESS_ALL_REG:
+ if (reg)
+ allowed = true;
+ else
+ allowed = (cmd->op_flags & SCST_EXCL_ACCESS_ALLOWED) != 0;
+ break;
+
+ default:
+ PRINT_ERROR("Invalid PR type %x", type);
+ allowed = false;
+ break;
+ }
+
+ if (!allowed)
+ TRACE_PR("Command %s (0x%x) from %s rejected due "
+ "to PR", cmd->op_name, cmd->cdb[0],
+ cmd->sess->initiator_name);
+ else
+ TRACE_DBG("Command %s (0x%x) from %s is allowed to execute",
+ cmd->op_name, cmd->cdb[0], cmd->sess->initiator_name);
+
+out_unlock:
+ scst_pr_read_unlock(dev, unlock);
+ return allowed;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_read_keys(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size)
+{
+ int i, offset = 0, size, size_max;
+ struct scst_device *dev = cmd->dev;
+ struct scst_dev_registrant *reg;
+
+ if (buffer_size < 8) {
+ TRACE_PR("buffer_size too small: %d. expected >= 8 "
+ "(buffer %p)", buffer_size, buffer);
+ goto skip;
+ }
+
+ TRACE_PR("Read Keys (dev %s): PRGen %d", dev->virt_name,
+ dev->pr_generation);
+
+ put_unaligned(cpu_to_be32(dev->pr_generation), (__be32 *)&buffer[0]);
+
+ offset = 8;
+ size = 0;
+ size_max = buffer_size - 8;
+
+ i = 0;
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ if (size_max - size >= 8) {
+ TRACE_PR("Read Keys (dev %s): key 0x%llx",
+ dev->virt_name, reg->key);
+
+ WARN_ON(reg->key == 0);
+
+ put_unaligned(reg->key,
+ (__be64 *)&buffer[offset + 8 * i]);
+
+ offset += 8;
+ }
+ size += 8;
+ }
+
+ put_unaligned(cpu_to_be32(size), (__be32 *)&buffer[4]);
+
+skip:
+ scst_set_resp_data_len(cmd, offset);
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_read_reservation(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size)
+{
+ struct scst_device *dev = cmd->dev;
+ uint8_t b[24];
+ int size = 0;
+
+ if (buffer_size < 8) {
+ TRACE_PR("buffer_size too small: %d. expected >= 8 "
+ "(buffer %p)", buffer_size, buffer);
+ goto skip;
+ }
+
+ memset(b, 0, sizeof(b));
+
+ put_unaligned(cpu_to_be32(dev->pr_generation), (__be32 *)&buffer[0]);
+
+ if (!dev->pr_is_set) {
+ TRACE_PR("Read Reservation: no reservations for dev %s",
+ dev->virt_name);
+ b[4] =
+ b[5] =
+ b[6] =
+ b[7] = 0;
+
+ size = 8;
+ } else {
+ __be64 key = dev->pr_holder ? dev->pr_holder->key : 0;
+
+ TRACE_PR("Read Reservation: dev %s, holder %p, key 0x%llx, "
+ "scope %d, type %d", dev->virt_name, dev->pr_holder,
+ key, dev->pr_scope, dev->pr_type);
+
+ b[4] =
+ b[5] =
+ b[6] = 0;
+ b[7] = 0x10;
+
+ put_unaligned(key, (__be64 *)&b[8]);
+ b[21] = dev->pr_scope << 4 | dev->pr_type;
+
+ size = 24;
+ }
+
+ memset(buffer, 0, buffer_size);
+ memcpy(buffer, b, min(size, buffer_size));
+
+skip:
+ scst_set_resp_data_len(cmd, size);
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_report_caps(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size)
+{
+ int offset = 0;
+ unsigned int crh = 1;
+ unsigned int atp_c = 1;
+ unsigned int sip_c = 1;
+ unsigned int ptpl_c = 1;
+ struct scst_device *dev = cmd->dev;
+
+ if (buffer_size < 8) {
+ TRACE_PR("buffer_size too small: %d. expected >= 8 "
+ "(buffer %p)", buffer_size, buffer);
+ goto skip;
+ }
+
+ TRACE_PR("Reporting capabilities (dev %s): crh %x, sip_c %x, "
+ "atp_c %x, ptpl_c %x, pr_aptpl %x", dev->virt_name,
+ crh, sip_c, atp_c, ptpl_c, dev->pr_aptpl);
+
+ buffer[0] = 0;
+ buffer[1] = 8;
+
+ buffer[2] = crh << 4 | sip_c << 3 | atp_c << 2 | ptpl_c;
+ buffer[3] = (1 << 7) | (dev->pr_aptpl > 0 ? 1 : 0);
+
+ /* All commands supported */
+ buffer[4] = 0xEA;
+ buffer[5] = 0x1;
+
+ offset += 8;
+
+skip:
+ scst_set_resp_data_len(cmd, offset);
+ return;
+}
+
+/* Called with dev_pr_mutex locked, no IRQ */
+void scst_pr_read_full_status(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size)
+{
+ int offset = 0, size, size_max;
+ struct scst_device *dev = cmd->dev;
+ struct scst_dev_registrant *reg;
+
+ if (buffer_size < 8)
+ goto skip;
+
+ put_unaligned(cpu_to_be32(dev->pr_generation), (__be32 *)&buffer[0]);
+ offset += 8;
+
+ size = 0;
+ size_max = buffer_size - 8;
+
+ list_for_each_entry(reg, &dev->dev_registrants_list,
+ dev_registrants_list_entry) {
+ int ts;
+ int rec_len;
+
+ ts = tid_size(reg->transport_id);
+ rec_len = 24 + ts;
+
+ if (size_max - size > rec_len) {
+ memset(&buffer[offset], 0, rec_len);
+
+ put_unaligned(reg->key, (__be64 *)(&buffer[offset]));
+
+ if (dev->pr_is_set && scst_pr_is_holder(dev, reg)) {
+ buffer[offset + 12] = 1;
+ buffer[offset + 13] = (dev->pr_scope << 8) | dev->pr_type;
+ }
+
+ put_unaligned(cpu_to_be16(reg->rel_tgt_id),
+ (__be16 *)&buffer[offset + 18]);
+ put_unaligned(cpu_to_be32(ts),
+ (__be32 *)&buffer[offset + 20]);
+
+ memcpy(&buffer[offset + 24], reg->transport_id, ts);
+
+ offset += rec_len;
+ }
+ size += rec_len;
+ }
+
+ put_unaligned(cpu_to_be32(size), (__be32 *)&buffer[4]);
+
+skip:
+ scst_set_resp_data_len(cmd, offset);
+ return;
+}
diff -uprN orig/linux-2.6.35/drivers/scst/scst_pres.h linux-2.6.35/drivers/scst/scst_pres.h
--- orig/linux-2.6.35/drivers/scst/scst_pres.h
+++ linux-2.6.35/drivers/scst/scst_pres.h
@@ -0,0 +1,159 @@
+/*
+ * scst_pres.c
+ *
+ * Copyright (C) 2009 - 2010 Alexey Obitotskiy <alexeyo1@open-e.com>
+ * Copyright (C) 2009 - 2010 Open-E, Inc.
+ * Copyright (C) 2009 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef SCST_PRES_H_
+#define SCST_PRES_H_
+
+#include <linux/delay.h>
+
+#define PR_REGISTER 0x00
+#define PR_RESERVE 0x01
+#define PR_RELEASE 0x02
+#define PR_CLEAR 0x03
+#define PR_PREEMPT 0x04
+#define PR_PREEMPT_AND_ABORT 0x05
+#define PR_REGISTER_AND_IGNORE 0x06
+#define PR_REGISTER_AND_MOVE 0x07
+
+#define PR_READ_KEYS 0x00
+#define PR_READ_RESERVATION 0x01
+#define PR_REPORT_CAPS 0x02
+#define PR_READ_FULL_STATUS 0x03
+
+#define TYPE_UNSPECIFIED (-1)
+#define TYPE_WRITE_EXCLUSIVE 0x01
+#define TYPE_EXCLUSIVE_ACCESS 0x03
+#define TYPE_WRITE_EXCLUSIVE_REGONLY 0x05
+#define TYPE_EXCLUSIVE_ACCESS_REGONLY 0x06
+#define TYPE_WRITE_EXCLUSIVE_ALL_REG 0x07
+#define TYPE_EXCLUSIVE_ACCESS_ALL_REG 0x08
+
+#define SCOPE_LU 0x00
+
+static inline bool scst_pr_type_valid(uint8_t type)
+{
+ switch (type) {
+ case TYPE_WRITE_EXCLUSIVE:
+ case TYPE_EXCLUSIVE_ACCESS:
+ case TYPE_WRITE_EXCLUSIVE_REGONLY:
+ case TYPE_EXCLUSIVE_ACCESS_REGONLY:
+ case TYPE_WRITE_EXCLUSIVE_ALL_REG:
+ case TYPE_EXCLUSIVE_ACCESS_ALL_REG:
+ return true;
+ default:
+ return false;
+ }
+}
+
+int scst_pr_check_pr_path(void);
+
+static inline bool scst_pr_read_lock(struct scst_device *dev)
+{
+ bool unlock = false;
+
+ atomic_inc(&dev->pr_readers_count);
+ smp_mb__after_atomic_inc(); /* to sync with scst_pr_write_lock() */
+
+ if (unlikely(dev->pr_writer_active)) {
+ unlock = true;
+ atomic_dec(&dev->pr_readers_count);
+ mutex_lock(&dev->dev_pr_mutex);
+ }
+ return unlock;
+}
+
+static inline void scst_pr_read_unlock(struct scst_device *dev, bool unlock)
+{
+
+ if (unlikely(unlock))
+ mutex_unlock(&dev->dev_pr_mutex);
+ else {
+ /*
+ * To sync with scst_pr_write_lock(). We need it to ensure
+ * order of our reads with the writer's writes.
+ */
+ smp_mb__before_atomic_dec();
+ atomic_dec(&dev->pr_readers_count);
+ }
+ return;
+}
+
+static inline void scst_pr_write_lock(struct scst_device *dev)
+{
+
+ mutex_lock(&dev->dev_pr_mutex);
+
+ dev->pr_writer_active = 1;
+
+ /* to sync with scst_pr_read_lock() and unlock() */
+ smp_mb();
+
+ while (atomic_read(&dev->pr_readers_count) != 0) {
+ TRACE_DBG("Waiting for %d readers (dev %p)",
+ atomic_read(&dev->pr_readers_count), dev);
+ msleep(1);
+ }
+ return;
+}
+
+static inline void scst_pr_write_unlock(struct scst_device *dev)
+{
+
+ dev->pr_writer_active = 0;
+
+ mutex_unlock(&dev->dev_pr_mutex);
+ return;
+}
+
+int scst_pr_init_dev(struct scst_device *dev);
+void scst_pr_clear_dev(struct scst_device *dev);
+
+int scst_pr_init_tgt_dev(struct scst_tgt_dev *tgt_dev);
+void scst_pr_clear_tgt_dev(struct scst_tgt_dev *tgt_dev);
+
+bool scst_pr_crh_case(struct scst_cmd *cmd);
+bool scst_pr_is_cmd_allowed(struct scst_cmd *cmd);
+
+void scst_pr_register(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size);
+void scst_pr_register_and_ignore(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size);
+void scst_pr_register_and_move(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size);
+void scst_pr_reserve(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size);
+void scst_pr_release(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size);
+void scst_pr_clear(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size);
+void scst_pr_preempt(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size);
+void scst_pr_preempt_and_abort(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size);
+
+void scst_pr_read_keys(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size);
+void scst_pr_read_reservation(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size);
+void scst_pr_report_caps(struct scst_cmd *cmd, uint8_t *buffer, int buffer_size);
+void scst_pr_read_full_status(struct scst_cmd *cmd, uint8_t *buffer,
+ int buffer_size);
+
+void scst_pr_sync_device_file(struct scst_tgt_dev *tgt_dev, struct scst_cmd *cmd);
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+void scst_pr_dump_prs(struct scst_device *dev, bool force);
+#else
+static inline void scst_pr_dump_prs(struct scst_device *dev, bool force) {}
+#endif
+
+#endif /* SCST_PRES_H_ */
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 8/17]: SCST SYSFS interface implementation
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (6 preceding siblings ...)
2010-09-14 14:43 ` [PATCH 7/17]: SCST Persistent Reservations implementation Vladislav Bolkhovitin
@ 2010-09-14 14:44 ` Vladislav Bolkhovitin
2010-09-14 14:44 ` [PATCH 9/17]: SCST debugging support routines Vladislav Bolkhovitin
` (9 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:44 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SYSFS interface implementation
Signed-off-by: Daniel Henrique Debonzi <debonzi@linux.vnet.ibm.com>
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_sysfs.c | 4967 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 4967 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/scst_sysfs.c linux-2.6.35/drivers/scst/scst_sysfs.c
--- orig/linux-2.6.35/drivers/scst/scst_sysfs.c
+++ linux-2.6.35/drivers/scst/scst_sysfs.c
@@ -0,0 +1,4967 @@
+/*
+ * scst_sysfs.c
+ *
+ * Copyright (C) 2009 Daniel Henrique Debonzi <debonzi@linux.vnet.ibm.com>
+ * Copyright (C) 2009 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2009 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kobject.h>
+#include <linux/string.h>
+#include <linux/sysfs.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/ctype.h>
+#include <linux/slab.h>
+#include <linux/kthread.h>
+
+#include <scst/scst.h>
+#include "scst_priv.h"
+#include "scst_mem.h"
+#include "scst_pres.h"
+
+static DECLARE_COMPLETION(scst_sysfs_root_release_completion);
+
+static struct kobject scst_sysfs_root_kobj;
+static struct kobject *scst_targets_kobj;
+static struct kobject *scst_devices_kobj;
+static struct kobject *scst_sgv_kobj;
+static struct kobject *scst_handlers_kobj;
+
+static const char *scst_dev_handler_types[] = {
+ "Direct-access device (e.g., magnetic disk)",
+ "Sequential-access device (e.g., magnetic tape)",
+ "Printer device",
+ "Processor device",
+ "Write-once device (e.g., some optical disks)",
+ "CD-ROM device",
+ "Scanner device (obsolete)",
+ "Optical memory device (e.g., some optical disks)",
+ "Medium changer device (e.g., jukeboxes)",
+ "Communications device (obsolete)",
+ "Defined by ASC IT8 (Graphic arts pre-press devices)",
+ "Defined by ASC IT8 (Graphic arts pre-press devices)",
+ "Storage array controller device (e.g., RAID)",
+ "Enclosure services device",
+ "Simplified direct-access device (e.g., magnetic disk)",
+ "Optical card reader/writer device"
+};
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static DEFINE_MUTEX(scst_log_mutex);
+
+static struct scst_trace_log scst_trace_tbl[] = {
+ { TRACE_OUT_OF_MEM, "out_of_mem" },
+ { TRACE_MINOR, "minor" },
+ { TRACE_SG_OP, "sg" },
+ { TRACE_MEMORY, "mem" },
+ { TRACE_BUFF, "buff" },
+ { TRACE_PID, "pid" },
+ { TRACE_LINE, "line" },
+ { TRACE_FUNCTION, "function" },
+ { TRACE_DEBUG, "debug" },
+ { TRACE_SPECIAL, "special" },
+ { TRACE_SCSI, "scsi" },
+ { TRACE_MGMT, "mgmt" },
+ { TRACE_MGMT_DEBUG, "mgmt_dbg" },
+ { TRACE_FLOW_CONTROL, "flow_control" },
+ { TRACE_PRES, "pr" },
+ { 0, NULL }
+};
+
+static struct scst_trace_log scst_local_trace_tbl[] = {
+ { TRACE_RTRY, "retry" },
+ { TRACE_SCSI_SERIALIZING, "scsi_serializing" },
+ { TRACE_RCV_BOT, "recv_bot" },
+ { TRACE_SND_BOT, "send_bot" },
+ { TRACE_RCV_TOP, "recv_top" },
+ { TRACE_SND_TOP, "send_top" },
+ { 0, NULL }
+};
+
+static ssize_t scst_trace_level_show(const struct scst_trace_log *local_tbl,
+ unsigned long log_level, char *buf, const char *help);
+static int scst_write_trace(const char *buf, size_t length,
+ unsigned long *log_level, unsigned long default_level,
+ const char *name, const struct scst_trace_log *tbl);
+
+#endif /* defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_luns_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_tgt_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_tgt_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_tgt_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_tgt_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_ini_group_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_ini_group_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_rel_tgt_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_rel_tgt_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_ini_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_acg_ini_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_acg_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_acg_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acn_file_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+
+/**
+ ** Sysfs work
+ **/
+
+static DEFINE_SPINLOCK(sysfs_work_lock);
+static LIST_HEAD(sysfs_work_list);
+static DECLARE_WAIT_QUEUE_HEAD(sysfs_work_waitQ);
+static int active_sysfs_works;
+static int last_sysfs_work_res;
+static struct task_struct *sysfs_work_thread;
+
+/**
+ * scst_alloc_sysfs_work() - allocates a sysfs work
+ */
+int scst_alloc_sysfs_work(int (*sysfs_work_fn)(struct scst_sysfs_work_item *),
+ struct scst_sysfs_work_item **res_work)
+{
+ int res = 0;
+ struct scst_sysfs_work_item *work;
+
+ if (sysfs_work_fn == NULL) {
+ PRINT_ERROR("%s", "sysfs_work_fn is NULL");
+ res = -EINVAL;
+ goto out;
+ }
+
+ *res_work = NULL;
+
+ work = kzalloc(sizeof(*work), GFP_KERNEL);
+ if (work == NULL) {
+ PRINT_ERROR("Unable to alloc sysfs work (size %zd)",
+ sizeof(*work));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ kref_init(&work->sysfs_work_kref);
+ init_completion(&work->sysfs_work_done);
+ work->sysfs_work_fn = sysfs_work_fn;
+
+ *res_work = work;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_alloc_sysfs_work);
+
+static void scst_sysfs_work_release(struct kref *kref)
+{
+ struct scst_sysfs_work_item *work;
+
+ work = container_of(kref, struct scst_sysfs_work_item,
+ sysfs_work_kref);
+
+ TRACE_DBG("Freeing sysfs work %p (buf %p)", work, work->buf);
+
+ kfree(work->buf);
+ kfree(work->res_buf);
+ kfree(work);
+ return;
+}
+
+/**
+ * scst_sysfs_work_get() - increases ref counter of the sysfs work
+ */
+void scst_sysfs_work_get(struct scst_sysfs_work_item *work)
+{
+ kref_get(&work->sysfs_work_kref);
+}
+EXPORT_SYMBOL(scst_sysfs_work_get);
+
+/**
+ * scst_sysfs_work_put() - decreases ref counter of the sysfs work
+ */
+void scst_sysfs_work_put(struct scst_sysfs_work_item *work)
+{
+ kref_put(&work->sysfs_work_kref, scst_sysfs_work_release);
+}
+EXPORT_SYMBOL(scst_sysfs_work_put);
+
+/**
+ * scst_sysfs_queue_wait_work() - waits for the work to complete
+ *
+ * Returnes status of the completed work or -EAGAIN if the work not
+ * completed before timeout. In the latter case a user should poll
+ * last_sysfs_mgmt_res until it returns the result of the processing.
+ */
+int scst_sysfs_queue_wait_work(struct scst_sysfs_work_item *work)
+{
+ int res = 0, rc;
+ unsigned long timeout = 15*HZ;
+
+ spin_lock(&sysfs_work_lock);
+
+ TRACE_DBG("Adding sysfs work %p to the list", work);
+ list_add_tail(&work->sysfs_work_list_entry, &sysfs_work_list);
+
+ active_sysfs_works++;
+
+ spin_unlock(&sysfs_work_lock);
+
+ kref_get(&work->sysfs_work_kref);
+
+ wake_up(&sysfs_work_waitQ);
+
+ while (1) {
+ rc = wait_for_completion_interruptible_timeout(
+ &work->sysfs_work_done, timeout);
+ if (rc == 0) {
+ if (!mutex_is_locked(&scst_mutex)) {
+ TRACE_DBG("scst_mutex not locked, continue "
+ "waiting (work %p)", work);
+ timeout = 5*HZ;
+ continue;
+ }
+ TRACE_MGMT_DBG("Time out waiting for work %p",
+ work);
+ res = -EAGAIN;
+ goto out_put;
+ } else if (rc < 0) {
+ res = rc;
+ goto out_put;
+ }
+ break;
+ }
+
+ res = work->work_res;
+
+out_put:
+ kref_put(&work->sysfs_work_kref, scst_sysfs_work_release);
+ return res;
+}
+EXPORT_SYMBOL(scst_sysfs_queue_wait_work);
+
+/* Called under sysfs_work_lock and drops/reaquire it inside */
+static void scst_process_sysfs_works(void)
+{
+ struct scst_sysfs_work_item *work;
+
+ while (!list_empty(&sysfs_work_list)) {
+ work = list_entry(sysfs_work_list.next,
+ struct scst_sysfs_work_item, sysfs_work_list_entry);
+ list_del(&work->sysfs_work_list_entry);
+ spin_unlock(&sysfs_work_lock);
+
+ TRACE_DBG("Sysfs work %p", work);
+
+ work->work_res = work->sysfs_work_fn(work);
+
+ spin_lock(&sysfs_work_lock);
+ last_sysfs_work_res = work->work_res;
+ active_sysfs_works--;
+ spin_unlock(&sysfs_work_lock);
+
+ complete_all(&work->sysfs_work_done);
+ kref_put(&work->sysfs_work_kref, scst_sysfs_work_release);
+
+ spin_lock(&sysfs_work_lock);
+ }
+ return;
+}
+
+static inline int test_sysfs_work_list(void)
+{
+ int res = !list_empty(&sysfs_work_list) ||
+ unlikely(kthread_should_stop());
+ return res;
+}
+
+static int sysfs_work_thread_fn(void *arg)
+{
+
+ PRINT_INFO("User interface thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ set_user_nice(current, -10);
+
+ spin_lock(&sysfs_work_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_sysfs_work_list()) {
+ add_wait_queue_exclusive(&sysfs_work_waitQ, &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_sysfs_work_list())
+ break;
+ spin_unlock(&sysfs_work_lock);
+ schedule();
+ spin_lock(&sysfs_work_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&sysfs_work_waitQ, &wait);
+ }
+
+ scst_process_sysfs_works();
+ }
+ spin_unlock(&sysfs_work_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so both lists must be empty.
+ */
+ BUG_ON(!list_empty(&sysfs_work_list));
+
+ PRINT_INFO("User interface thread PID %d finished", current->pid);
+ return 0;
+}
+
+/* No locks */
+static int scst_check_grab_tgtt_ptr(struct scst_tgt_template *tgtt)
+{
+ int res = 0;
+ struct scst_tgt_template *tt;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(tt, &scst_template_list, scst_template_list_entry) {
+ if (tt == tgtt) {
+ tgtt->tgtt_active_sysfs_works_count++;
+ goto out_unlock;
+ }
+ }
+
+ TRACE_DBG("Tgtt %p not found", tgtt);
+ res = -ENOENT;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+/* No locks */
+static void scst_ungrab_tgtt_ptr(struct scst_tgt_template *tgtt)
+{
+
+ mutex_lock(&scst_mutex);
+ tgtt->tgtt_active_sysfs_works_count--;
+ mutex_unlock(&scst_mutex);
+ return;
+}
+
+/* scst_mutex supposed to be locked */
+static int scst_check_tgt_acg_ptrs(struct scst_tgt *tgt, struct scst_acg *acg)
+{
+ int res = 0;
+ struct scst_tgt_template *tgtt;
+
+ list_for_each_entry(tgtt, &scst_template_list, scst_template_list_entry) {
+ struct scst_tgt *t;
+ list_for_each_entry(t, &tgtt->tgt_list, tgt_list_entry) {
+ if (t == tgt) {
+ struct scst_acg *a;
+ if (acg == NULL)
+ goto out;
+ if (acg == tgt->default_acg)
+ goto out;
+ list_for_each_entry(a, &tgt->tgt_acg_list,
+ acg_list_entry) {
+ if (a == acg)
+ goto out;
+ }
+ }
+ }
+ }
+
+ TRACE_DBG("Tgt %p/ACG %p not found", tgt, acg);
+ res = -ENOENT;
+
+out:
+ return res;
+}
+
+/* scst_mutex supposed to be locked */
+static int scst_check_devt_ptr(struct scst_dev_type *devt,
+ struct list_head *list)
+{
+ int res = 0;
+ struct scst_dev_type *dt;
+
+ list_for_each_entry(dt, list, dev_type_list_entry) {
+ if (dt == devt)
+ goto out;
+ }
+
+ TRACE_DBG("Devt %p not found", devt);
+ res = -ENOENT;
+
+out:
+ return res;
+}
+
+/* scst_mutex supposed to be locked */
+static int scst_check_dev_ptr(struct scst_device *dev)
+{
+ int res = 0;
+ struct scst_device *d;
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (d == dev)
+ goto out;
+ }
+
+ TRACE_DBG("Dev %p not found", dev);
+ res = -ENOENT;
+
+out:
+ return res;
+}
+
+/* No locks */
+static int scst_check_grab_devt_ptr(struct scst_dev_type *devt,
+ struct list_head *list)
+{
+ int res = 0;
+ struct scst_dev_type *dt;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(dt, list, dev_type_list_entry) {
+ if (dt == devt) {
+ devt->devt_active_sysfs_works_count++;
+ goto out_unlock;
+ }
+ }
+
+ TRACE_DBG("Devt %p not found", devt);
+ res = -ENOENT;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+/* No locks */
+static void scst_ungrab_devt_ptr(struct scst_dev_type *devt)
+{
+
+ mutex_lock(&scst_mutex);
+ devt->devt_active_sysfs_works_count--;
+ mutex_unlock(&scst_mutex);
+ return;
+}
+
+/**
+ ** Regilar SCST sysfs ops
+ **/
+static ssize_t scst_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ struct kobj_attribute *kobj_attr;
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ return kobj_attr->show(kobj, kobj_attr, buf);
+}
+
+static ssize_t scst_store(struct kobject *kobj, struct attribute *attr,
+ const char *buf, size_t count)
+{
+ struct kobj_attribute *kobj_attr;
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ if (kobj_attr->store)
+ return kobj_attr->store(kobj, kobj_attr, buf, count);
+ else
+ return -EIO;
+}
+
+static const struct sysfs_ops scst_sysfs_ops = {
+ .show = scst_show,
+ .store = scst_store,
+};
+
+const struct sysfs_ops *scst_sysfs_get_sysfs_ops(void)
+{
+ return &scst_sysfs_ops;
+}
+EXPORT_SYMBOL_GPL(scst_sysfs_get_sysfs_ops);
+
+/**
+ ** Target Template
+ **/
+
+static void scst_tgtt_release(struct kobject *kobj)
+{
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+ complete_all(&tgtt->tgtt_kobj_release_cmpl);
+ return;
+}
+
+static struct kobj_type tgtt_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_tgtt_release,
+};
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static ssize_t scst_tgtt_trace_level_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ return scst_trace_level_show(tgtt->trace_tbl,
+ tgtt->trace_flags ? *tgtt->trace_flags : 0, buf,
+ tgtt->trace_tbl_help);
+}
+
+static ssize_t scst_tgtt_trace_level_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ if (mutex_lock_interruptible(&scst_log_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = scst_write_trace(buf, count, tgtt->trace_flags,
+ tgtt->default_trace_flags, tgtt->name, tgtt->trace_tbl);
+
+ mutex_unlock(&scst_log_mutex);
+
+out:
+ return res;
+}
+
+static struct kobj_attribute tgtt_trace_attr =
+ __ATTR(trace_level, S_IRUGO | S_IWUSR,
+ scst_tgtt_trace_level_show, scst_tgtt_trace_level_store);
+
+#endif /* #if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_tgtt_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+char *help = "Usage: echo \"add_target target_name [parameters]\" "
+ ">mgmt\n"
+ " echo \"del_target target_name\" >mgmt\n"
+ "%s%s"
+ "%s"
+ "\n"
+ "where parameters are one or more "
+ "param_name=value pairs separated by ';'\n\n"
+ "%s%s%s%s%s%s%s%s\n";
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ return scnprintf(buf, SCST_SYSFS_BLOCK_SIZE, help,
+ (tgtt->tgtt_optional_attributes != NULL) ?
+ " echo \"add_attribute <attribute> <value>\" >mgmt\n"
+ " echo \"del_attribute <attribute> <value>\" >mgmt\n" : "",
+ (tgtt->tgt_optional_attributes != NULL) ?
+ " echo \"add_target_attribute target_name <attribute> <value>\" >mgmt\n"
+ " echo \"del_target_attribute target_name <attribute> <value>\" >mgmt\n" : "",
+ (tgtt->mgmt_cmd_help) ? tgtt->mgmt_cmd_help : "",
+ (tgtt->add_target_parameters != NULL) ?
+ "The following parameters available: " : "",
+ (tgtt->add_target_parameters != NULL) ?
+ tgtt->add_target_parameters : "",
+ (tgtt->tgtt_optional_attributes != NULL) ?
+ "The following target driver attributes available: " : "",
+ (tgtt->tgtt_optional_attributes != NULL) ?
+ tgtt->tgtt_optional_attributes : "",
+ (tgtt->tgtt_optional_attributes != NULL) ? "\n" : "",
+ (tgtt->tgt_optional_attributes != NULL) ?
+ "The following target attributes available: " : "",
+ (tgtt->tgt_optional_attributes != NULL) ?
+ tgtt->tgt_optional_attributes : "",
+ (tgtt->tgt_optional_attributes != NULL) ? "\n" : "");
+}
+
+static int scst_process_tgtt_mgmt_store(char *buffer,
+ struct scst_tgt_template *tgtt)
+{
+ int res = 0;
+ char *p, *pp, *target_name;
+
+ TRACE_DBG("buffer %s", buffer);
+
+ /* Check if our pointer is still alive and, if yes, grab it */
+ if (scst_check_grab_tgtt_ptr(tgtt) != 0)
+ goto out;
+
+ pp = buffer;
+ if (pp[strlen(pp) - 1] == '\n')
+ pp[strlen(pp) - 1] = '\0';
+
+ p = scst_get_next_lexem(&pp);
+
+ if (strcasecmp("add_target", p) == 0) {
+ target_name = scst_get_next_lexem(&pp);
+ if (*target_name == '\0') {
+ PRINT_ERROR("%s", "Target name required");
+ res = -EINVAL;
+ goto out_ungrab;
+ }
+ res = tgtt->add_target(target_name, pp);
+ } else if (strcasecmp("del_target", p) == 0) {
+ target_name = scst_get_next_lexem(&pp);
+ if (*target_name == '\0') {
+ PRINT_ERROR("%s", "Target name required");
+ res = -EINVAL;
+ goto out_ungrab;
+ }
+
+ p = scst_get_next_lexem(&pp);
+ if (*p != '\0')
+ goto out_syntax_err;
+
+ res = tgtt->del_target(target_name);
+ } else if (tgtt->mgmt_cmd != NULL) {
+ scst_restore_token_str(p, pp);
+ res = tgtt->mgmt_cmd(buffer);
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_ungrab;
+ }
+
+out_ungrab:
+ scst_ungrab_tgtt_ptr(tgtt);
+
+out:
+ return res;
+
+out_syntax_err:
+ PRINT_ERROR("Syntax error on \"%s\"", p);
+ res = -EINVAL;
+ goto out_ungrab;
+}
+
+static int scst_tgtt_mgmt_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return scst_process_tgtt_mgmt_store(work->buf, work->tgtt);
+}
+
+static ssize_t scst_tgtt_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ char *buffer;
+ struct scst_sysfs_work_item *work;
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ res = scst_alloc_sysfs_work(scst_tgtt_mgmt_store_work_fn, &work);
+ if (res != 0)
+ goto out_free;
+
+ work->buf = buffer;
+ work->tgtt = tgtt;
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+
+out_free:
+ kfree(buffer);
+ goto out;
+}
+
+static struct kobj_attribute scst_tgtt_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_tgtt_mgmt_show,
+ scst_tgtt_mgmt_store);
+
+int scst_tgtt_sysfs_create(struct scst_tgt_template *tgtt)
+{
+ int res = 0;
+ const struct attribute **pattr;
+
+ init_completion(&tgtt->tgtt_kobj_release_cmpl);
+
+ res = kobject_init_and_add(&tgtt->tgtt_kobj, &tgtt_ktype,
+ scst_targets_kobj, tgtt->name);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgtt %s to sysfs", tgtt->name);
+ goto out;
+ }
+
+ if (tgtt->add_target != NULL) {
+ res = sysfs_create_file(&tgtt->tgtt_kobj,
+ &scst_tgtt_mgmt.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add mgmt attr for target driver %s",
+ tgtt->name);
+ goto out_del;
+ }
+ }
+
+ pattr = tgtt->tgtt_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ TRACE_DBG("Creating attr %s for target driver %s",
+ (*pattr)->name, tgtt->name);
+ res = sysfs_create_file(&tgtt->tgtt_kobj, *pattr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add attr %s for target "
+ "driver %s", (*pattr)->name,
+ tgtt->name);
+ goto out_del;
+ }
+ pattr++;
+ }
+ }
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (tgtt->trace_flags != NULL) {
+ res = sysfs_create_file(&tgtt->tgtt_kobj,
+ &tgtt_trace_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add trace_flag for target "
+ "driver %s", tgtt->name);
+ goto out_del;
+ }
+ }
+#endif
+
+out:
+ return res;
+
+out_del:
+ scst_tgtt_sysfs_del(tgtt);
+ goto out;
+}
+
+/*
+ * Must not be called under scst_mutex, due to possible deadlock with
+ * sysfs ref counting in sysfs works (it is waiting for the last put, but
+ * the last ref counter holder is waiting for scst_mutex)
+ */
+void scst_tgtt_sysfs_del(struct scst_tgt_template *tgtt)
+{
+ int rc;
+
+ kobject_del(&tgtt->tgtt_kobj);
+ kobject_put(&tgtt->tgtt_kobj);
+
+ rc = wait_for_completion_timeout(&tgtt->tgtt_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for target template %s (%d refs)...", tgtt->name,
+ atomic_read(&tgtt->tgtt_kobj.kref.refcount));
+ wait_for_completion(&tgtt->tgtt_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for target template %s", tgtt->name);
+ }
+ return;
+}
+
+/**
+ ** Target directory implementation
+ **/
+
+static void scst_tgt_release(struct kobject *kobj)
+{
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ complete_all(&tgt->tgt_kobj_release_cmpl);
+ return;
+}
+
+static struct kobj_type tgt_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_tgt_release,
+};
+
+static void scst_acg_release(struct kobject *kobj)
+{
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+ complete_all(&acg->acg_kobj_release_cmpl);
+ return;
+}
+
+static struct kobj_type acg_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_acg_release,
+};
+
+static struct kobj_attribute scst_luns_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_luns_mgmt_show,
+ scst_luns_mgmt_store);
+
+static struct kobj_attribute scst_acg_luns_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_luns_mgmt_show,
+ scst_acg_luns_mgmt_store);
+
+static struct kobj_attribute scst_acg_ini_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_acg_ini_mgmt_show,
+ scst_acg_ini_mgmt_store);
+
+static struct kobj_attribute scst_ini_group_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_ini_group_mgmt_show,
+ scst_ini_group_mgmt_store);
+
+static struct kobj_attribute scst_tgt_addr_method =
+ __ATTR(addr_method, S_IRUGO | S_IWUSR, scst_tgt_addr_method_show,
+ scst_tgt_addr_method_store);
+
+static struct kobj_attribute scst_tgt_io_grouping_type =
+ __ATTR(io_grouping_type, S_IRUGO | S_IWUSR,
+ scst_tgt_io_grouping_type_show,
+ scst_tgt_io_grouping_type_store);
+
+static struct kobj_attribute scst_rel_tgt_id =
+ __ATTR(rel_tgt_id, S_IRUGO | S_IWUSR, scst_rel_tgt_id_show,
+ scst_rel_tgt_id_store);
+
+static struct kobj_attribute scst_acg_addr_method =
+ __ATTR(addr_method, S_IRUGO | S_IWUSR, scst_acg_addr_method_show,
+ scst_acg_addr_method_store);
+
+static struct kobj_attribute scst_acg_io_grouping_type =
+ __ATTR(io_grouping_type, S_IRUGO | S_IWUSR,
+ scst_acg_io_grouping_type_show,
+ scst_acg_io_grouping_type_store);
+
+static ssize_t scst_tgt_enable_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt *tgt;
+ int res;
+ bool enabled;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ enabled = tgt->tgtt->is_target_enabled(tgt);
+
+ res = sprintf(buf, "%d\n", enabled ? 1 : 0);
+ return res;
+}
+
+static int scst_process_tgt_enable_store(struct scst_tgt *tgt, bool enable)
+{
+ int res;
+
+ /* Tgt protected by kobject reference */
+
+ TRACE_DBG("tgt %s, enable %d", tgt->tgt_name, enable);
+
+ if (enable && (tgt->rel_tgt_id == 0)) {
+ res = gen_relative_target_port_id(&tgt->rel_tgt_id);
+ if (res != 0)
+ goto out_put;
+ PRINT_INFO("Using autogenerated rel ID %d for target "
+ "%s", tgt->rel_tgt_id, tgt->tgt_name);
+ } else {
+ if (!scst_is_relative_target_port_id_unique(
+ tgt->rel_tgt_id, tgt)) {
+ PRINT_ERROR("Relative port id %d is not unique",
+ tgt->rel_tgt_id);
+ res = -EBADSLT;
+ goto out_put;
+ }
+ }
+
+ res = tgt->tgtt->enable_target(tgt, enable);
+
+out_put:
+ kobject_put(&tgt->tgt_kobj);
+ return res;
+}
+
+static int scst_tgt_enable_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return scst_process_tgt_enable_store(work->tgt, work->enable);
+}
+
+static ssize_t scst_tgt_enable_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_tgt *tgt;
+ bool enable;
+ struct scst_sysfs_work_item *work;
+
+ if (buf == NULL) {
+ PRINT_ERROR("%s: NULL buffer?", __func__);
+ res = -EINVAL;
+ goto out;
+ }
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ switch (buf[0]) {
+ case '0':
+ enable = false;
+ break;
+ case '1':
+ enable = true;
+ break;
+ default:
+ PRINT_ERROR("%s: Requested action not understood: %s",
+ __func__, buf);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_alloc_sysfs_work(scst_tgt_enable_store_work_fn, &work);
+ if (res != 0)
+ goto out;
+
+ work->tgt = tgt;
+ work->enable = enable;
+
+ kobject_get(&tgt->tgt_kobj);
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute tgt_enable_attr =
+ __ATTR(enabled, S_IRUGO | S_IWUSR,
+ scst_tgt_enable_show, scst_tgt_enable_store);
+
+/*
+ * Supposed to be called under scst_mutex. In case of error will drop,
+ * then reacquire it.
+ */
+int scst_tgt_sysfs_create(struct scst_tgt *tgt)
+{
+ int res;
+ const struct attribute **pattr;
+
+ init_completion(&tgt->tgt_kobj_release_cmpl);
+
+ res = kobject_init_and_add(&tgt->tgt_kobj, &tgt_ktype,
+ &tgt->tgtt->tgtt_kobj, tgt->tgt_name);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgt %s to sysfs", tgt->tgt_name);
+ goto out;
+ }
+
+ if ((tgt->tgtt->enable_target != NULL) &&
+ (tgt->tgtt->is_target_enabled != NULL)) {
+ res = sysfs_create_file(&tgt->tgt_kobj,
+ &tgt_enable_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add attr %s to sysfs",
+ tgt_enable_attr.attr.name);
+ goto out_err;
+ }
+ }
+
+ tgt->tgt_sess_kobj = kobject_create_and_add("sessions", &tgt->tgt_kobj);
+ if (tgt->tgt_sess_kobj == NULL) {
+ PRINT_ERROR("Can't create sess kobj for tgt %s", tgt->tgt_name);
+ goto out_nomem;
+ }
+
+ tgt->tgt_luns_kobj = kobject_create_and_add("luns", &tgt->tgt_kobj);
+ if (tgt->tgt_luns_kobj == NULL) {
+ PRINT_ERROR("Can't create luns kobj for tgt %s", tgt->tgt_name);
+ goto out_nomem;
+ }
+
+ res = sysfs_create_file(tgt->tgt_luns_kobj, &scst_luns_mgmt.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_luns_mgmt.attr.name, tgt->tgt_name);
+ goto out_err;
+ }
+
+ tgt->tgt_ini_grp_kobj = kobject_create_and_add("ini_groups",
+ &tgt->tgt_kobj);
+ if (tgt->tgt_ini_grp_kobj == NULL) {
+ PRINT_ERROR("Can't create ini_grp kobj for tgt %s",
+ tgt->tgt_name);
+ goto out_nomem;
+ }
+
+ res = sysfs_create_file(tgt->tgt_ini_grp_kobj,
+ &scst_ini_group_mgmt.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_ini_group_mgmt.attr.name, tgt->tgt_name);
+ goto out_err;
+ }
+
+ res = sysfs_create_file(&tgt->tgt_kobj,
+ &scst_rel_tgt_id.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_rel_tgt_id.attr.name, tgt->tgt_name);
+ goto out_err;
+ }
+
+ res = sysfs_create_file(&tgt->tgt_kobj,
+ &scst_tgt_addr_method.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_tgt_addr_method.attr.name, tgt->tgt_name);
+ goto out_err;
+ }
+
+ res = sysfs_create_file(&tgt->tgt_kobj,
+ &scst_tgt_io_grouping_type.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_tgt_io_grouping_type.attr.name, tgt->tgt_name);
+ goto out_err;
+ }
+
+ pattr = tgt->tgtt->tgt_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ TRACE_DBG("Creating attr %s for tgt %s", (*pattr)->name,
+ tgt->tgt_name);
+ res = sysfs_create_file(&tgt->tgt_kobj, *pattr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ (*pattr)->name, tgt->tgt_name);
+ goto out_err;
+ }
+ pattr++;
+ }
+ }
+
+out:
+ return res;
+
+out_nomem:
+ res = -ENOMEM;
+
+out_err:
+ mutex_unlock(&scst_mutex);
+ scst_tgt_sysfs_del(tgt);
+ mutex_lock(&scst_mutex);
+ goto out;
+}
+
+/*
+ * Must not be called under scst_mutex, due to possible deadlock with
+ * sysfs ref counting in sysfs works (it is waiting for the last put, but
+ * the last ref counter holder is waiting for scst_mutex)
+ */
+void scst_tgt_sysfs_del(struct scst_tgt *tgt)
+{
+ int rc;
+
+ kobject_del(tgt->tgt_sess_kobj);
+ kobject_put(tgt->tgt_sess_kobj);
+
+ kobject_del(tgt->tgt_luns_kobj);
+ kobject_put(tgt->tgt_luns_kobj);
+
+ kobject_del(tgt->tgt_ini_grp_kobj);
+ kobject_put(tgt->tgt_ini_grp_kobj);
+
+ kobject_del(&tgt->tgt_kobj);
+ kobject_put(&tgt->tgt_kobj);
+
+ rc = wait_for_completion_timeout(&tgt->tgt_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for target %s (%d refs)...", tgt->tgt_name,
+ atomic_read(&tgt->tgt_kobj.kref.refcount));
+ wait_for_completion(&tgt->tgt_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for target %s", tgt->tgt_name);
+ }
+ return;
+}
+
+/**
+ ** Devices directory implementation
+ **/
+
+static ssize_t scst_dev_sysfs_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ pos = sprintf(buf, "%d - %s\n", dev->type,
+ (unsigned)dev->type > ARRAY_SIZE(scst_dev_handler_types) ?
+ "unknown" : scst_dev_handler_types[dev->type]);
+
+ return pos;
+}
+
+static struct kobj_attribute dev_type_attr =
+ __ATTR(type, S_IRUGO, scst_dev_sysfs_type_show, NULL);
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static ssize_t scst_dev_sysfs_dump_prs(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ scst_pr_dump_prs(dev, true);
+ return count;
+}
+
+static struct kobj_attribute dev_dump_prs_attr =
+ __ATTR(dump_prs, S_IWUSR, NULL, scst_dev_sysfs_dump_prs);
+
+#endif /* defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static int scst_process_dev_sysfs_threads_data_store(
+ struct scst_device *dev, int threads_num,
+ enum scst_dev_type_threads_pool_type threads_pool_type)
+{
+ int res = 0;
+ int oldtn = dev->threads_num;
+ enum scst_dev_type_threads_pool_type oldtt = dev->threads_pool_type;
+
+ TRACE_DBG("dev %p, threads_num %d, threads_pool_type %d", dev,
+ threads_num, threads_pool_type);
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ /* Check if our pointer is still alive */
+ if (scst_check_dev_ptr(dev) != 0)
+ goto out_unlock;
+
+ scst_stop_dev_threads(dev);
+
+ dev->threads_num = threads_num;
+ dev->threads_pool_type = threads_pool_type;
+
+ res = scst_create_dev_threads(dev);
+ if (res != 0)
+ goto out_unlock;
+
+ if (oldtn != dev->threads_num)
+ PRINT_INFO("Changed cmd threads num to %d", dev->threads_num);
+ else if (oldtt != dev->threads_pool_type)
+ PRINT_INFO("Changed cmd threads pool type to %d",
+ dev->threads_pool_type);
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+}
+
+static int scst_dev_sysfs_threads_data_store_work_fn(
+ struct scst_sysfs_work_item *work)
+{
+ return scst_process_dev_sysfs_threads_data_store(work->dev,
+ work->new_threads_num, work->new_threads_pool_type);
+}
+
+static ssize_t scst_dev_sysfs_check_threads_data(
+ struct scst_device *dev, int threads_num,
+ enum scst_dev_type_threads_pool_type threads_pool_type, bool *stop)
+{
+ int res = 0;
+
+ *stop = false;
+
+ if (dev->threads_num < 0) {
+ PRINT_ERROR("Threads pool disabled for device %s",
+ dev->virt_name);
+ res = -EPERM;
+ goto out;
+ }
+
+ if ((threads_num == dev->threads_num) &&
+ (threads_pool_type == dev->threads_pool_type)) {
+ *stop = true;
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+static ssize_t scst_dev_sysfs_threads_num_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ pos = sprintf(buf, "%d\n%s", dev->threads_num,
+ (dev->threads_num != dev->handler->threads_num) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ return pos;
+}
+
+static ssize_t scst_dev_sysfs_threads_num_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_device *dev;
+ long newtn;
+ bool stop;
+ struct scst_sysfs_work_item *work;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ res = strict_strtol(buf, 0, &newtn);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtol() for %s failed: %d ", buf, res);
+ goto out;
+ }
+ if (newtn < 0) {
+ PRINT_ERROR("Illegal threads num value %ld", newtn);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_dev_sysfs_check_threads_data(dev, newtn,
+ dev->threads_pool_type, &stop);
+ if ((res != 0) || stop)
+ goto out;
+
+ res = scst_alloc_sysfs_work(scst_dev_sysfs_threads_data_store_work_fn,
+ &work);
+ if (res != 0)
+ goto out;
+
+ work->dev = dev;
+ work->new_threads_num = newtn;
+ work->new_threads_pool_type = dev->threads_pool_type;
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute dev_threads_num_attr =
+ __ATTR(threads_num, S_IRUGO | S_IWUSR,
+ scst_dev_sysfs_threads_num_show,
+ scst_dev_sysfs_threads_num_store);
+
+static ssize_t scst_dev_sysfs_threads_pool_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ if (dev->threads_num == 0) {
+ pos = sprintf(buf, "Async\n");
+ goto out;
+ } else if (dev->threads_num < 0) {
+ pos = sprintf(buf, "Not valid\n");
+ goto out;
+ }
+
+ switch (dev->threads_pool_type) {
+ case SCST_THREADS_POOL_PER_INITIATOR:
+ pos = sprintf(buf, "%s\n%s", SCST_THREADS_POOL_PER_INITIATOR_STR,
+ (dev->threads_pool_type != dev->handler->threads_pool_type) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ break;
+ case SCST_THREADS_POOL_SHARED:
+ pos = sprintf(buf, "%s\n%s", SCST_THREADS_POOL_SHARED_STR,
+ (dev->threads_pool_type != dev->handler->threads_pool_type) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ break;
+ default:
+ pos = sprintf(buf, "Unknown\n");
+ break;
+ }
+
+out:
+ return pos;
+}
+
+static ssize_t scst_dev_sysfs_threads_pool_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_device *dev;
+ enum scst_dev_type_threads_pool_type newtpt;
+ struct scst_sysfs_work_item *work;
+ bool stop;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ newtpt = scst_parse_threads_pool_type(buf, count);
+ if (newtpt == SCST_THREADS_POOL_TYPE_INVALID) {
+ PRINT_ERROR("Illegal threads pool type %s", buf);
+ res = -EINVAL;
+ goto out;
+ }
+
+ TRACE_DBG("buf %s, count %zd, newtpt %d", buf, count, newtpt);
+
+ res = scst_dev_sysfs_check_threads_data(dev, dev->threads_num,
+ newtpt, &stop);
+ if ((res != 0) || stop)
+ goto out;
+
+ res = scst_alloc_sysfs_work(scst_dev_sysfs_threads_data_store_work_fn,
+ &work);
+ if (res != 0)
+ goto out;
+
+ work->dev = dev;
+ work->new_threads_num = dev->threads_num;
+ work->new_threads_pool_type = newtpt;
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute dev_threads_pool_type_attr =
+ __ATTR(threads_pool_type, S_IRUGO | S_IWUSR,
+ scst_dev_sysfs_threads_pool_type_show,
+ scst_dev_sysfs_threads_pool_type_store);
+
+static struct attribute *scst_dev_attrs[] = {
+ &dev_type_attr.attr,
+ NULL,
+};
+
+static void scst_sysfs_dev_release(struct kobject *kobj)
+{
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ complete_all(&dev->dev_kobj_release_cmpl);
+ return;
+}
+
+int scst_devt_dev_sysfs_create(struct scst_device *dev)
+{
+ int res = 0;
+ const struct attribute **pattr;
+
+ if (dev->handler == &scst_null_devtype)
+ goto out;
+
+ res = sysfs_create_link(&dev->dev_kobj,
+ &dev->handler->devt_kobj, "handler");
+ if (res != 0) {
+ PRINT_ERROR("Can't create handler link for dev %s",
+ dev->virt_name);
+ goto out;
+ }
+
+ res = sysfs_create_link(&dev->handler->devt_kobj,
+ &dev->dev_kobj, dev->virt_name);
+ if (res != 0) {
+ PRINT_ERROR("Can't create handler link for dev %s",
+ dev->virt_name);
+ goto out_err;
+ }
+
+ if (dev->handler->threads_num >= 0) {
+ res = sysfs_create_file(&dev->dev_kobj,
+ &dev_threads_num_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add dev attr %s for dev %s",
+ dev_threads_num_attr.attr.name,
+ dev->virt_name);
+ goto out_err;
+ }
+ res = sysfs_create_file(&dev->dev_kobj,
+ &dev_threads_pool_type_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add dev attr %s for dev %s",
+ dev_threads_pool_type_attr.attr.name,
+ dev->virt_name);
+ goto out_err;
+ }
+ }
+
+ pattr = dev->handler->dev_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ res = sysfs_create_file(&dev->dev_kobj, *pattr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add dev attr %s for dev %s",
+ (*pattr)->name, dev->virt_name);
+ goto out_err;
+ }
+ pattr++;
+ }
+ }
+
+out:
+ return res;
+
+out_err:
+ scst_devt_dev_sysfs_del(dev);
+ goto out;
+}
+
+void scst_devt_dev_sysfs_del(struct scst_device *dev)
+{
+ const struct attribute **pattr;
+
+ if (dev->handler == &scst_null_devtype)
+ goto out;
+
+ pattr = dev->handler->dev_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ sysfs_remove_file(&dev->dev_kobj, *pattr);
+ pattr++;
+ }
+ }
+
+ sysfs_remove_link(&dev->dev_kobj, "handler");
+ sysfs_remove_link(&dev->handler->devt_kobj, dev->virt_name);
+
+ if (dev->handler->threads_num >= 0) {
+ sysfs_remove_file(&dev->dev_kobj,
+ &dev_threads_num_attr.attr);
+ sysfs_remove_file(&dev->dev_kobj,
+ &dev_threads_pool_type_attr.attr);
+ }
+
+out:
+ return;
+}
+
+static struct kobj_type scst_dev_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_sysfs_dev_release,
+ .default_attrs = scst_dev_attrs,
+};
+
+/*
+ * Must not be called under scst_mutex, because it can call
+ * scst_dev_sysfs_del()
+ */
+int scst_dev_sysfs_create(struct scst_device *dev)
+{
+ int res = 0;
+
+ init_completion(&dev->dev_kobj_release_cmpl);
+
+ res = kobject_init_and_add(&dev->dev_kobj, &scst_dev_ktype,
+ scst_devices_kobj, dev->virt_name);
+ if (res != 0) {
+ PRINT_ERROR("Can't add device %s to sysfs", dev->virt_name);
+ goto out;
+ }
+
+ dev->dev_exp_kobj = kobject_create_and_add("exported",
+ &dev->dev_kobj);
+ if (dev->dev_exp_kobj == NULL) {
+ PRINT_ERROR("Can't create exported link for device %s",
+ dev->virt_name);
+ res = -ENOMEM;
+ goto out_del;
+ }
+
+ if (dev->scsi_dev != NULL) {
+ res = sysfs_create_link(&dev->dev_kobj,
+ &dev->scsi_dev->sdev_dev.kobj, "scsi_device");
+ if (res != 0) {
+ PRINT_ERROR("Can't create scsi_device link for dev %s",
+ dev->virt_name);
+ goto out_del;
+ }
+ }
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (dev->scsi_dev == NULL) {
+ res = sysfs_create_file(&dev->dev_kobj,
+ &dev_dump_prs_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't create attr %s for dev %s",
+ dev_dump_prs_attr.attr.name, dev->virt_name);
+ goto out_del;
+ }
+ }
+#endif
+
+out:
+ return res;
+
+out_del:
+ scst_dev_sysfs_del(dev);
+ goto out;
+}
+
+/*
+ * Must not be called under scst_mutex, due to possible deadlock with
+ * sysfs ref counting in sysfs works (it is waiting for the last put, but
+ * the last ref counter holder is waiting for scst_mutex)
+ */
+void scst_dev_sysfs_del(struct scst_device *dev)
+{
+ int rc;
+
+ kobject_del(dev->dev_exp_kobj);
+ kobject_put(dev->dev_exp_kobj);
+
+ kobject_del(&dev->dev_kobj);
+ kobject_put(&dev->dev_kobj);
+
+ rc = wait_for_completion_timeout(&dev->dev_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for device %s (%d refs)...", dev->virt_name,
+ atomic_read(&dev->dev_kobj.kref.refcount));
+ wait_for_completion(&dev->dev_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for device %s", dev->virt_name);
+ }
+ return;
+}
+
+/**
+ ** Tgt_dev's directory implementation
+ **/
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+static char *scst_io_size_names[] = {
+ "<=8K ",
+ "<=32K ",
+ "<=128K",
+ "<=512K",
+ ">512K "
+};
+
+static ssize_t scst_tgt_dev_latency_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buffer)
+{
+ int res = 0, i;
+ char buf[50];
+ struct scst_tgt_dev *tgt_dev;
+
+ tgt_dev = container_of(kobj, struct scst_tgt_dev, tgt_dev_kobj);
+
+ for (i = 0; i < SCST_LATENCY_STATS_NUM; i++) {
+ uint64_t scst_time_wr, tgt_time_wr, dev_time_wr;
+ unsigned int processed_cmds_wr;
+ uint64_t scst_time_rd, tgt_time_rd, dev_time_rd;
+ unsigned int processed_cmds_rd;
+ struct scst_ext_latency_stat *latency_stat;
+
+ latency_stat = &tgt_dev->dev_latency_stat[i];
+ scst_time_wr = latency_stat->scst_time_wr;
+ scst_time_rd = latency_stat->scst_time_rd;
+ tgt_time_wr = latency_stat->tgt_time_wr;
+ tgt_time_rd = latency_stat->tgt_time_rd;
+ dev_time_wr = latency_stat->dev_time_wr;
+ dev_time_rd = latency_stat->dev_time_rd;
+ processed_cmds_wr = latency_stat->processed_cmds_wr;
+ processed_cmds_rd = latency_stat->processed_cmds_rd;
+
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-5s %-9s %-15lu ", "Write", scst_io_size_names[i],
+ (unsigned long)processed_cmds_wr);
+ if (processed_cmds_wr == 0)
+ processed_cmds_wr = 1;
+
+ do_div(scst_time_wr, processed_cmds_wr);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_scst_time_wr,
+ (unsigned long)scst_time_wr,
+ (unsigned long)latency_stat->max_scst_time_wr,
+ (unsigned long)latency_stat->scst_time_wr);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(tgt_time_wr, processed_cmds_wr);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_tgt_time_wr,
+ (unsigned long)tgt_time_wr,
+ (unsigned long)latency_stat->max_tgt_time_wr,
+ (unsigned long)latency_stat->tgt_time_wr);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(dev_time_wr, processed_cmds_wr);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_dev_time_wr,
+ (unsigned long)dev_time_wr,
+ (unsigned long)latency_stat->max_dev_time_wr,
+ (unsigned long)latency_stat->dev_time_wr);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s\n", buf);
+
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-5s %-9s %-15lu ", "Read", scst_io_size_names[i],
+ (unsigned long)processed_cmds_rd);
+ if (processed_cmds_rd == 0)
+ processed_cmds_rd = 1;
+
+ do_div(scst_time_rd, processed_cmds_rd);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_scst_time_rd,
+ (unsigned long)scst_time_rd,
+ (unsigned long)latency_stat->max_scst_time_rd,
+ (unsigned long)latency_stat->scst_time_rd);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(tgt_time_rd, processed_cmds_rd);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_tgt_time_rd,
+ (unsigned long)tgt_time_rd,
+ (unsigned long)latency_stat->max_tgt_time_rd,
+ (unsigned long)latency_stat->tgt_time_rd);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(dev_time_rd, processed_cmds_rd);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_dev_time_rd,
+ (unsigned long)dev_time_rd,
+ (unsigned long)latency_stat->max_dev_time_rd,
+ (unsigned long)latency_stat->dev_time_rd);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s\n", buf);
+ }
+ return res;
+}
+
+static struct kobj_attribute tgt_dev_latency_attr =
+ __ATTR(latency, S_IRUGO,
+ scst_tgt_dev_latency_show, NULL);
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
+
+static ssize_t scst_tgt_dev_active_commands_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_tgt_dev *tgt_dev;
+
+ tgt_dev = container_of(kobj, struct scst_tgt_dev, tgt_dev_kobj);
+
+ pos = sprintf(buf, "%d\n", atomic_read(&tgt_dev->tgt_dev_cmd_count));
+
+ return pos;
+}
+
+static struct kobj_attribute tgt_dev_active_commands_attr =
+ __ATTR(active_commands, S_IRUGO,
+ scst_tgt_dev_active_commands_show, NULL);
+
+static struct attribute *scst_tgt_dev_attrs[] = {
+ &tgt_dev_active_commands_attr.attr,
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ &tgt_dev_latency_attr.attr,
+#endif
+ NULL,
+};
+
+static void scst_sysfs_tgt_dev_release(struct kobject *kobj)
+{
+ struct scst_tgt_dev *tgt_dev;
+
+ tgt_dev = container_of(kobj, struct scst_tgt_dev, tgt_dev_kobj);
+ complete_all(&tgt_dev->tgt_dev_kobj_release_cmpl);
+ return;
+}
+
+static struct kobj_type scst_tgt_dev_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_sysfs_tgt_dev_release,
+ .default_attrs = scst_tgt_dev_attrs,
+};
+
+int scst_tgt_dev_sysfs_create(struct scst_tgt_dev *tgt_dev)
+{
+ int res = 0;
+
+ init_completion(&tgt_dev->tgt_dev_kobj_release_cmpl);
+
+ res = kobject_init_and_add(&tgt_dev->tgt_dev_kobj, &scst_tgt_dev_ktype,
+ &tgt_dev->sess->sess_kobj, "lun%lld",
+ (unsigned long long)tgt_dev->lun);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgt_dev %lld to sysfs",
+ (unsigned long long)tgt_dev->lun);
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+/*
+ * Called with scst_mutex held.
+ *
+ * !! No sysfs works must use kobject_get() to protect tgt_dev, due to possible
+ * !! deadlock with scst_mutex (it is waiting for the last put, but
+ * !! the last ref counter holder is waiting for scst_mutex)
+ */
+void scst_tgt_dev_sysfs_del(struct scst_tgt_dev *tgt_dev)
+{
+ int rc;
+
+ kobject_del(&tgt_dev->tgt_dev_kobj);
+ kobject_put(&tgt_dev->tgt_dev_kobj);
+
+ rc = wait_for_completion_timeout(
+ &tgt_dev->tgt_dev_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for tgt_dev %lld (%d refs)...",
+ (unsigned long long)tgt_dev->lun,
+ atomic_read(&tgt_dev->tgt_dev_kobj.kref.refcount));
+ wait_for_completion(&tgt_dev->tgt_dev_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs entry for "
+ "tgt_dev %lld", (unsigned long long)tgt_dev->lun);
+ }
+ return;
+}
+
+/**
+ ** Sessions subdirectory implementation
+ **/
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+static ssize_t scst_sess_latency_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buffer)
+{
+ ssize_t res = 0;
+ struct scst_session *sess;
+ int i;
+ char buf[50];
+ uint64_t scst_time, tgt_time, dev_time;
+ unsigned int processed_cmds;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-15s %-15s %-46s %-46s %-46s\n",
+ "T-L names", "Total commands", "SCST latency",
+ "Target latency", "Dev latency (min/avg/max/all ns)");
+
+ spin_lock_bh(&sess->lat_lock);
+
+ for (i = 0; i < SCST_LATENCY_STATS_NUM ; i++) {
+ uint64_t scst_time_wr, tgt_time_wr, dev_time_wr;
+ unsigned int processed_cmds_wr;
+ uint64_t scst_time_rd, tgt_time_rd, dev_time_rd;
+ unsigned int processed_cmds_rd;
+ struct scst_ext_latency_stat *latency_stat;
+
+ latency_stat = &sess->sess_latency_stat[i];
+ scst_time_wr = latency_stat->scst_time_wr;
+ scst_time_rd = latency_stat->scst_time_rd;
+ tgt_time_wr = latency_stat->tgt_time_wr;
+ tgt_time_rd = latency_stat->tgt_time_rd;
+ dev_time_wr = latency_stat->dev_time_wr;
+ dev_time_rd = latency_stat->dev_time_rd;
+ processed_cmds_wr = latency_stat->processed_cmds_wr;
+ processed_cmds_rd = latency_stat->processed_cmds_rd;
+
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-5s %-9s %-15lu ",
+ "Write", scst_io_size_names[i],
+ (unsigned long)processed_cmds_wr);
+ if (processed_cmds_wr == 0)
+ processed_cmds_wr = 1;
+
+ do_div(scst_time_wr, processed_cmds_wr);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_scst_time_wr,
+ (unsigned long)scst_time_wr,
+ (unsigned long)latency_stat->max_scst_time_wr,
+ (unsigned long)latency_stat->scst_time_wr);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(tgt_time_wr, processed_cmds_wr);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_tgt_time_wr,
+ (unsigned long)tgt_time_wr,
+ (unsigned long)latency_stat->max_tgt_time_wr,
+ (unsigned long)latency_stat->tgt_time_wr);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(dev_time_wr, processed_cmds_wr);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_dev_time_wr,
+ (unsigned long)dev_time_wr,
+ (unsigned long)latency_stat->max_dev_time_wr,
+ (unsigned long)latency_stat->dev_time_wr);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s\n", buf);
+
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-5s %-9s %-15lu ",
+ "Read", scst_io_size_names[i],
+ (unsigned long)processed_cmds_rd);
+ if (processed_cmds_rd == 0)
+ processed_cmds_rd = 1;
+
+ do_div(scst_time_rd, processed_cmds_rd);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_scst_time_rd,
+ (unsigned long)scst_time_rd,
+ (unsigned long)latency_stat->max_scst_time_rd,
+ (unsigned long)latency_stat->scst_time_rd);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(tgt_time_rd, processed_cmds_rd);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_tgt_time_rd,
+ (unsigned long)tgt_time_rd,
+ (unsigned long)latency_stat->max_tgt_time_rd,
+ (unsigned long)latency_stat->tgt_time_rd);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(dev_time_rd, processed_cmds_rd);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)latency_stat->min_dev_time_rd,
+ (unsigned long)dev_time_rd,
+ (unsigned long)latency_stat->max_dev_time_rd,
+ (unsigned long)latency_stat->dev_time_rd);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s\n", buf);
+ }
+
+ scst_time = sess->scst_time;
+ tgt_time = sess->tgt_time;
+ dev_time = sess->dev_time;
+ processed_cmds = sess->processed_cmds;
+
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "\n%-15s %-16d", "Overall ", processed_cmds);
+
+ if (processed_cmds == 0)
+ processed_cmds = 1;
+
+ do_div(scst_time, processed_cmds);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)sess->min_scst_time,
+ (unsigned long)scst_time,
+ (unsigned long)sess->max_scst_time,
+ (unsigned long)sess->scst_time);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(tgt_time, processed_cmds);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)sess->min_tgt_time,
+ (unsigned long)tgt_time,
+ (unsigned long)sess->max_tgt_time,
+ (unsigned long)sess->tgt_time);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s", buf);
+
+ do_div(dev_time, processed_cmds);
+ snprintf(buf, sizeof(buf), "%lu/%lu/%lu/%lu",
+ (unsigned long)sess->min_dev_time,
+ (unsigned long)dev_time,
+ (unsigned long)sess->max_dev_time,
+ (unsigned long)sess->dev_time);
+ res += scnprintf(&buffer[res], SCST_SYSFS_BLOCK_SIZE - res,
+ "%-47s\n\n", buf);
+
+ spin_unlock_bh(&sess->lat_lock);
+ return res;
+}
+
+static int scst_sess_zero_latency(struct scst_sysfs_work_item *work)
+{
+ int res = 0, t;
+ struct scst_session *sess = work->sess;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_put;
+ }
+
+ PRINT_INFO("Zeroing latency statistics for initiator "
+ "%s", sess->initiator_name);
+
+ spin_lock_bh(&sess->lat_lock);
+
+ sess->scst_time = 0;
+ sess->tgt_time = 0;
+ sess->dev_time = 0;
+ sess->min_scst_time = 0;
+ sess->min_tgt_time = 0;
+ sess->min_dev_time = 0;
+ sess->max_scst_time = 0;
+ sess->max_tgt_time = 0;
+ sess->max_dev_time = 0;
+ sess->processed_cmds = 0;
+ memset(sess->sess_latency_stat, 0,
+ sizeof(sess->sess_latency_stat));
+
+ for (t = TGT_DEV_HASH_SIZE-1; t >= 0; t--) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[t];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ tgt_dev->scst_time = 0;
+ tgt_dev->tgt_time = 0;
+ tgt_dev->dev_time = 0;
+ tgt_dev->processed_cmds = 0;
+ memset(tgt_dev->dev_latency_stat, 0,
+ sizeof(tgt_dev->dev_latency_stat));
+ }
+ }
+
+ spin_unlock_bh(&sess->lat_lock);
+
+ mutex_unlock(&scst_mutex);
+
+out_put:
+ kobject_put(&sess->sess_kobj);
+ return res;
+}
+
+static ssize_t scst_sess_latency_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_session *sess;
+ struct scst_sysfs_work_item *work;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ res = scst_alloc_sysfs_work(scst_sess_zero_latency, &work);
+ if (res != 0)
+ goto out;
+
+ work->sess = sess;
+
+ kobject_get(&sess->sess_kobj);
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute session_latency_attr =
+ __ATTR(latency, S_IRUGO | S_IWUSR, scst_sess_latency_show,
+ scst_sess_latency_store);
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
+
+static ssize_t scst_sess_sysfs_commands_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ return sprintf(buf, "%i\n", atomic_read(&sess->sess_cmd_count));
+}
+
+static struct kobj_attribute session_commands_attr =
+ __ATTR(commands, S_IRUGO, scst_sess_sysfs_commands_show, NULL);
+
+static int scst_sysfs_sess_get_active_commands(struct scst_session *sess)
+{
+ int res;
+ int active_cmds = 0, t;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_put;
+ }
+
+ for (t = TGT_DEV_HASH_SIZE-1; t >= 0; t--) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[t];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ active_cmds += atomic_read(&tgt_dev->tgt_dev_cmd_count);
+ }
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ res = active_cmds;
+
+out_put:
+ kobject_put(&sess->sess_kobj);
+ return res;
+}
+
+static int scst_sysfs_sess_get_active_commands_work_fn(struct scst_sysfs_work_item *work)
+{
+ return scst_sysfs_sess_get_active_commands(work->sess);
+}
+
+static ssize_t scst_sess_sysfs_active_commands_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int res;
+ struct scst_session *sess;
+ struct scst_sysfs_work_item *work;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ res = scst_alloc_sysfs_work(scst_sysfs_sess_get_active_commands_work_fn,
+ &work);
+ if (res != 0)
+ goto out;
+
+ work->sess = sess;
+
+ kobject_get(&sess->sess_kobj);
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res != -EAGAIN)
+ res = sprintf(buf, "%i\n", res);
+
+out:
+ return res;
+}
+
+static struct kobj_attribute session_active_commands_attr =
+ __ATTR(active_commands, S_IRUGO, scst_sess_sysfs_active_commands_show,
+ NULL);
+
+static ssize_t scst_sess_sysfs_initiator_name_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ return scnprintf(buf, SCST_SYSFS_BLOCK_SIZE, "%s\n",
+ sess->initiator_name);
+}
+
+static struct kobj_attribute session_initiator_name_attr =
+ __ATTR(initiator_name, S_IRUGO, scst_sess_sysfs_initiator_name_show, NULL);
+
+static struct attribute *scst_session_attrs[] = {
+ &session_commands_attr.attr,
+ &session_active_commands_attr.attr,
+ &session_initiator_name_attr.attr,
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ &session_latency_attr.attr,
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
+ NULL,
+};
+
+static void scst_sysfs_session_release(struct kobject *kobj)
+{
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+ complete_all(&sess->sess_kobj_release_cmpl);
+ return;
+}
+
+static struct kobj_type scst_session_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_sysfs_session_release,
+ .default_attrs = scst_session_attrs,
+};
+
+static int scst_create_sess_luns_link(struct scst_session *sess)
+{
+ int res;
+
+ /*
+ * No locks are needed, because sess supposed to be in acg->acg_sess_list
+ * and tgt->sess_list, so blocking them from disappearing.
+ */
+
+ if (sess->acg == sess->tgt->default_acg)
+ res = sysfs_create_link(&sess->sess_kobj,
+ sess->tgt->tgt_luns_kobj, "luns");
+ else
+ res = sysfs_create_link(&sess->sess_kobj,
+ sess->acg->luns_kobj, "luns");
+
+ if (res != 0)
+ PRINT_ERROR("Can't create luns link for initiator %s",
+ sess->initiator_name);
+
+ return res;
+}
+
+int scst_recreate_sess_luns_link(struct scst_session *sess)
+{
+ sysfs_remove_link(&sess->sess_kobj, "luns");
+ return scst_create_sess_luns_link(sess);
+}
+
+/* Supposed to be called under scst_mutex */
+int scst_sess_sysfs_create(struct scst_session *sess)
+{
+ int res = 0;
+ struct scst_session *s;
+ const struct attribute **pattr;
+ char *name = (char *)sess->initiator_name;
+ int len = strlen(name) + 1, n = 1;
+
+restart:
+ list_for_each_entry(s, &sess->tgt->sess_list, sess_list_entry) {
+ if (!s->sess_kobj_ready)
+ continue;
+
+ if (strcmp(name, kobject_name(&s->sess_kobj)) == 0) {
+ if (s == sess)
+ continue;
+
+ TRACE_DBG("Dublicated session from the same initiator "
+ "%s found", name);
+
+ if (name == sess->initiator_name) {
+ len = strlen(sess->initiator_name);
+ len += 20;
+ name = kmalloc(len, GFP_KERNEL);
+ if (name == NULL) {
+ PRINT_ERROR("Unable to allocate a "
+ "replacement name (size %d)",
+ len);
+ }
+ }
+
+ snprintf(name, len, "%s_%d", sess->initiator_name, n);
+ n++;
+ goto restart;
+ }
+ }
+
+ init_completion(&sess->sess_kobj_release_cmpl);
+
+ TRACE_DBG("Adding session %s to sysfs", name);
+
+ res = kobject_init_and_add(&sess->sess_kobj, &scst_session_ktype,
+ sess->tgt->tgt_sess_kobj, name);
+ if (res != 0) {
+ PRINT_ERROR("Can't add session %s to sysfs", name);
+ goto out_free;
+ }
+
+ sess->sess_kobj_ready = 1;
+
+ pattr = sess->tgt->tgtt->sess_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ res = sysfs_create_file(&sess->sess_kobj, *pattr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add sess attr %s for sess "
+ "for initiator %s", (*pattr)->name,
+ name);
+ goto out_free;
+ }
+ pattr++;
+ }
+ }
+
+ res = scst_create_sess_luns_link(sess);
+
+out_free:
+ if (name != sess->initiator_name)
+ kfree(name);
+ return res;
+}
+
+/*
+ * Must not be called under scst_mutex, due to possible deadlock with
+ * sysfs ref counting in sysfs works (it is waiting for the last put, but
+ * the last ref counter holder is waiting for scst_mutex)
+ */
+void scst_sess_sysfs_del(struct scst_session *sess)
+{
+ int rc;
+
+ if (!sess->sess_kobj_ready)
+ goto out;
+
+ TRACE_DBG("Deleting session %s from sysfs",
+ kobject_name(&sess->sess_kobj));
+
+ kobject_del(&sess->sess_kobj);
+ kobject_put(&sess->sess_kobj);
+
+ rc = wait_for_completion_timeout(&sess->sess_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for session from %s (%d refs)...", sess->initiator_name,
+ atomic_read(&sess->sess_kobj.kref.refcount));
+ wait_for_completion(&sess->sess_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for session %s", sess->initiator_name);
+ }
+
+out:
+ return;
+}
+
+/**
+ ** Target luns directory implementation
+ **/
+
+static void scst_acg_dev_release(struct kobject *kobj)
+{
+ struct scst_acg_dev *acg_dev;
+
+ acg_dev = container_of(kobj, struct scst_acg_dev, acg_dev_kobj);
+ complete_all(&acg_dev->acg_dev_kobj_release_cmpl);
+ return;
+}
+
+static ssize_t scst_lun_rd_only_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ struct scst_acg_dev *acg_dev;
+
+ acg_dev = container_of(kobj, struct scst_acg_dev, acg_dev_kobj);
+
+ if (acg_dev->rd_only || acg_dev->dev->rd_only)
+ return sprintf(buf, "%d\n%s\n", 1, SCST_SYSFS_KEY_MARK);
+ else
+ return sprintf(buf, "%d\n", 0);
+}
+
+static struct kobj_attribute lun_options_attr =
+ __ATTR(read_only, S_IRUGO, scst_lun_rd_only_show, NULL);
+
+static struct attribute *lun_attrs[] = {
+ &lun_options_attr.attr,
+ NULL,
+};
+
+static struct kobj_type acg_dev_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_acg_dev_release,
+ .default_attrs = lun_attrs,
+};
+
+/*
+ * Called with scst_mutex held.
+ *
+ * !! No sysfs works must use kobject_get() to protect acg_dev, due to possible
+ * !! deadlock with scst_mutex (it is waiting for the last put, but
+ * !! the last ref counter holder is waiting for scst_mutex)
+ */
+void scst_acg_dev_sysfs_del(struct scst_acg_dev *acg_dev)
+{
+ int rc;
+
+ if (acg_dev->dev != NULL) {
+ sysfs_remove_link(acg_dev->dev->dev_exp_kobj,
+ acg_dev->acg_dev_link_name);
+ kobject_put(&acg_dev->dev->dev_kobj);
+ }
+
+ kobject_del(&acg_dev->acg_dev_kobj);
+ kobject_put(&acg_dev->acg_dev_kobj);
+
+ rc = wait_for_completion_timeout(&acg_dev->acg_dev_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for acg_dev %p (%d refs)...", acg_dev,
+ atomic_read(&acg_dev->acg_dev_kobj.kref.refcount));
+ wait_for_completion(&acg_dev->acg_dev_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for acg_dev %p", acg_dev);
+ }
+ return;
+}
+
+int scst_acg_dev_sysfs_create(struct scst_acg_dev *acg_dev,
+ struct kobject *parent)
+{
+ int res;
+
+ init_completion(&acg_dev->acg_dev_kobj_release_cmpl);
+
+ res = kobject_init_and_add(&acg_dev->acg_dev_kobj, &acg_dev_ktype,
+ parent, "%u", acg_dev->lun);
+ if (res != 0) {
+ PRINT_ERROR("Can't add acg_dev %p to sysfs", acg_dev);
+ goto out;
+ }
+
+ kobject_get(&acg_dev->dev->dev_kobj);
+
+ snprintf(acg_dev->acg_dev_link_name, sizeof(acg_dev->acg_dev_link_name),
+ "export%u", acg_dev->dev->dev_exported_lun_num++);
+
+ res = sysfs_create_link(acg_dev->dev->dev_exp_kobj,
+ &acg_dev->acg_dev_kobj, acg_dev->acg_dev_link_name);
+ if (res != 0) {
+ PRINT_ERROR("Can't create acg %s LUN link",
+ acg_dev->acg->acg_name);
+ goto out_del;
+ }
+
+ res = sysfs_create_link(&acg_dev->acg_dev_kobj,
+ &acg_dev->dev->dev_kobj, "device");
+ if (res != 0) {
+ PRINT_ERROR("Can't create acg %s device link",
+ acg_dev->acg->acg_name);
+ goto out_del;
+ }
+
+out:
+ return res;
+
+out_del:
+ scst_acg_dev_sysfs_del(acg_dev);
+ goto out;
+}
+
+static int __scst_process_luns_mgmt_store(char *buffer,
+ struct scst_tgt *tgt, struct scst_acg *acg, bool tgt_kobj)
+{
+ int res, read_only = 0, action;
+ char *p, *e = NULL;
+ unsigned int virt_lun;
+ struct scst_acg_dev *acg_dev = NULL, *acg_dev_tmp;
+ struct scst_device *d, *dev = NULL;
+
+#define SCST_LUN_ACTION_ADD 1
+#define SCST_LUN_ACTION_DEL 2
+#define SCST_LUN_ACTION_REPLACE 3
+#define SCST_LUN_ACTION_CLEAR 4
+
+ TRACE_DBG("buffer %s", buffer);
+
+ p = buffer;
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ if (strncasecmp("add", p, 3) == 0) {
+ p += 3;
+ action = SCST_LUN_ACTION_ADD;
+ } else if (strncasecmp("del", p, 3) == 0) {
+ p += 3;
+ action = SCST_LUN_ACTION_DEL;
+ } else if (!strncasecmp("replace", p, 7)) {
+ p += 7;
+ action = SCST_LUN_ACTION_REPLACE;
+ } else if (!strncasecmp("clear", p, 5)) {
+ p += 5;
+ action = SCST_LUN_ACTION_CLEAR;
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ /* Check if tgt and acg not already freed while we were coming here */
+ if (scst_check_tgt_acg_ptrs(tgt, acg) != 0)
+ goto out_unlock;
+
+ if ((action != SCST_LUN_ACTION_CLEAR) &&
+ (action != SCST_LUN_ACTION_DEL)) {
+ if (!isspace(*p)) {
+ PRINT_ERROR("%s", "Syntax error");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ while (isspace(*p) && *p != '\0')
+ p++;
+ e = p; /* save p */
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (!strcmp(d->virt_name, p)) {
+ dev = d;
+ TRACE_DBG("Device %p (%s) found", dev, p);
+ break;
+ }
+ }
+ if (dev == NULL) {
+ PRINT_ERROR("Device '%s' not found", p);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ }
+
+ switch (action) {
+ case SCST_LUN_ACTION_ADD:
+ case SCST_LUN_ACTION_REPLACE:
+ {
+ bool dev_replaced = false;
+
+ e++;
+ while (isspace(*e) && *e != '\0')
+ e++;
+ virt_lun = simple_strtoul(e, &e, 0);
+
+ while (isspace(*e) && *e != '\0')
+ e++;
+
+ while (1) {
+ char *pp;
+ unsigned long val;
+ char *param = scst_get_next_token_str(&e);
+ if (param == NULL)
+ break;
+
+ p = scst_get_next_lexem(¶m);
+ if (*p == '\0') {
+ PRINT_ERROR("Syntax error at %s (device %s)",
+ param, dev->virt_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ pp = scst_get_next_lexem(¶m);
+ if (*pp == '\0') {
+ PRINT_ERROR("Parameter %s value missed for device %s",
+ p, dev->virt_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (scst_get_next_lexem(¶m)[0] != '\0') {
+ PRINT_ERROR("Too many parameter's %s values (device %s)",
+ p, dev->virt_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ res = strict_strtoul(pp, 0, &val);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %d "
+ "(device %s)", pp, res, dev->virt_name);
+ goto out_unlock;
+ }
+
+ if (!strcasecmp("read_only", p)) {
+ read_only = val;
+ TRACE_DBG("READ ONLY %d", read_only);
+ } else {
+ PRINT_ERROR("Unknown parameter %s (device %s)",
+ p, dev->virt_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ }
+
+ acg_dev = NULL;
+ list_for_each_entry(acg_dev_tmp, &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ if (acg_dev_tmp->lun == virt_lun) {
+ acg_dev = acg_dev_tmp;
+ break;
+ }
+ }
+
+ if (acg_dev != NULL) {
+ if (action == SCST_LUN_ACTION_ADD) {
+ PRINT_ERROR("virt lun %d already exists in "
+ "group %s", virt_lun, acg->acg_name);
+ res = -EEXIST;
+ goto out_unlock;
+ } else {
+ /* Replace */
+ res = scst_acg_del_lun(acg, acg_dev->lun,
+ false);
+ if (res != 0)
+ goto out_unlock;
+
+ dev_replaced = true;
+ }
+ }
+
+ res = scst_acg_add_lun(acg,
+ tgt_kobj ? tgt->tgt_luns_kobj : acg->luns_kobj,
+ dev, virt_lun, read_only, !dev_replaced, NULL);
+ if (res != 0)
+ goto out_unlock;
+
+ if (dev_replaced) {
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((tgt_dev->acg_dev->acg == acg) &&
+ (tgt_dev->lun == virt_lun)) {
+ TRACE_MGMT_DBG("INQUIRY DATA HAS CHANGED"
+ " on tgt_dev %p", tgt_dev);
+ scst_gen_aen_or_ua(tgt_dev,
+ SCST_LOAD_SENSE(scst_sense_inquery_data_changed));
+ }
+ }
+ }
+
+ break;
+ }
+ case SCST_LUN_ACTION_DEL:
+ while (isspace(*p) && *p != '\0')
+ p++;
+ virt_lun = simple_strtoul(p, &p, 0);
+
+ res = scst_acg_del_lun(acg, virt_lun, true);
+ if (res != 0)
+ goto out_unlock;
+ break;
+ case SCST_LUN_ACTION_CLEAR:
+ PRINT_INFO("Removed all devices from group %s",
+ acg->acg_name);
+ list_for_each_entry_safe(acg_dev, acg_dev_tmp,
+ &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ res = scst_acg_del_lun(acg, acg_dev->lun,
+ list_is_last(&acg_dev->acg_dev_list_entry,
+ &acg->acg_dev_list));
+ if (res)
+ goto out_unlock;
+ }
+ break;
+ }
+
+ res = 0;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+
+#undef SCST_LUN_ACTION_ADD
+#undef SCST_LUN_ACTION_DEL
+#undef SCST_LUN_ACTION_REPLACE
+#undef SCST_LUN_ACTION_CLEAR
+}
+
+static int scst_luns_mgmt_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return __scst_process_luns_mgmt_store(work->buf, work->tgt, work->acg,
+ work->is_tgt_kobj);
+}
+
+static ssize_t __scst_acg_mgmt_store(struct scst_acg *acg,
+ const char *buf, size_t count, bool is_tgt_kobj,
+ int (*sysfs_work_fn)(struct scst_sysfs_work_item *))
+{
+ int res;
+ char *buffer;
+ struct scst_sysfs_work_item *work;
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ res = scst_alloc_sysfs_work(sysfs_work_fn, &work);
+ if (res != 0)
+ goto out_free;
+
+ work->buf = buffer;
+ work->tgt = acg->tgt;
+ work->acg = acg;
+ work->is_tgt_kobj = is_tgt_kobj;
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+
+out_free:
+ kfree(buffer);
+ goto out;
+}
+
+static ssize_t __scst_luns_mgmt_store(struct scst_acg *acg,
+ bool tgt_kobj, const char *buf, size_t count)
+{
+ return __scst_acg_mgmt_store(acg, buf, count, tgt_kobj,
+ scst_luns_mgmt_store_work_fn);
+}
+
+static ssize_t scst_luns_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ static char *help = "Usage: echo \"add|del H:C:I:L lun [parameters]\" >mgmt\n"
+ " echo \"add VNAME lun [parameters]\" >mgmt\n"
+ " echo \"del lun\" >mgmt\n"
+ " echo \"replace H:C:I:L lun [parameters]\" >mgmt\n"
+ " echo \"replace VNAME lun [parameters]\" >mgmt\n"
+ " echo \"clear\" >mgmt\n"
+ "\n"
+ "where parameters are one or more "
+ "param_name=value pairs separated by ';'\n"
+ "\nThe following parameters available: read_only.";
+
+ return sprintf(buf, "%s", help);
+}
+
+static ssize_t scst_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj->parent, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ res = __scst_luns_mgmt_store(acg, true, buf, count);
+ return res;
+}
+
+static ssize_t __scst_acg_addr_method_show(struct scst_acg *acg, char *buf)
+{
+ int res;
+
+ switch (acg->addr_method) {
+ case SCST_LUN_ADDR_METHOD_FLAT:
+ res = sprintf(buf, "FLAT\n%s\n", SCST_SYSFS_KEY_MARK);
+ break;
+ case SCST_LUN_ADDR_METHOD_PERIPHERAL:
+ res = sprintf(buf, "PERIPHERAL\n");
+ break;
+ default:
+ res = sprintf(buf, "UNKNOWN\n");
+ break;
+ }
+
+ return res;
+}
+
+static ssize_t __scst_acg_addr_method_store(struct scst_acg *acg,
+ const char *buf, size_t count)
+{
+ int res = count;
+
+ if (strncasecmp(buf, "FLAT", min_t(int, 4, count)) == 0)
+ acg->addr_method = SCST_LUN_ADDR_METHOD_FLAT;
+ else if (strncasecmp(buf, "PERIPHERAL", min_t(int, 10, count)) == 0)
+ acg->addr_method = SCST_LUN_ADDR_METHOD_PERIPHERAL;
+ else {
+ PRINT_ERROR("Unknown address method %s", buf);
+ res = -EINVAL;
+ }
+
+ TRACE_DBG("acg %p, addr_method %d", acg, acg->addr_method);
+
+ return res;
+}
+
+static ssize_t scst_tgt_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ return __scst_acg_addr_method_show(acg, buf);
+}
+
+static ssize_t scst_tgt_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ res = __scst_acg_addr_method_store(acg, buf, count);
+ return res;
+}
+
+static ssize_t __scst_acg_io_grouping_type_show(struct scst_acg *acg, char *buf)
+{
+ int res;
+
+ switch (acg->acg_io_grouping_type) {
+ case SCST_IO_GROUPING_AUTO:
+ res = sprintf(buf, "%s\n", SCST_IO_GROUPING_AUTO_STR);
+ break;
+ case SCST_IO_GROUPING_THIS_GROUP_ONLY:
+ res = sprintf(buf, "%s\n%s\n",
+ SCST_IO_GROUPING_THIS_GROUP_ONLY_STR,
+ SCST_SYSFS_KEY_MARK);
+ break;
+ case SCST_IO_GROUPING_NEVER:
+ res = sprintf(buf, "%s\n%s\n", SCST_IO_GROUPING_NEVER_STR,
+ SCST_SYSFS_KEY_MARK);
+ break;
+ default:
+ res = sprintf(buf, "%d\n%s\n", acg->acg_io_grouping_type,
+ SCST_SYSFS_KEY_MARK);
+ break;
+ }
+
+ return res;
+}
+
+static int __scst_acg_process_io_grouping_type_store(struct scst_tgt *tgt,
+ struct scst_acg *acg, int io_grouping_type)
+{
+ int res = 0;
+ struct scst_acg_dev *acg_dev;
+
+ TRACE_DBG("tgt %p, acg %p, io_grouping_type %d", tgt, acg,
+ io_grouping_type);
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ /* Check if tgt and acg not already freed while we were coming here */
+ if (scst_check_tgt_acg_ptrs(tgt, acg) != 0)
+ goto out_unlock;
+
+ acg->acg_io_grouping_type = io_grouping_type;
+
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ int rc;
+
+ scst_stop_dev_threads(acg_dev->dev);
+
+ rc = scst_create_dev_threads(acg_dev->dev);
+ if (rc != 0)
+ res = rc;
+ }
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+}
+
+static int __scst_acg_io_grouping_type_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return __scst_acg_process_io_grouping_type_store(work->tgt, work->acg,
+ work->io_grouping_type);
+}
+
+static ssize_t __scst_acg_io_grouping_type_store(struct scst_acg *acg,
+ const char *buf, size_t count)
+{
+ int res = 0;
+ int prev = acg->acg_io_grouping_type;
+ long io_grouping_type;
+ struct scst_sysfs_work_item *work;
+
+ if (strncasecmp(buf, SCST_IO_GROUPING_AUTO_STR,
+ min_t(int, strlen(SCST_IO_GROUPING_AUTO_STR), count)) == 0)
+ io_grouping_type = SCST_IO_GROUPING_AUTO;
+ else if (strncasecmp(buf, SCST_IO_GROUPING_THIS_GROUP_ONLY_STR,
+ min_t(int, strlen(SCST_IO_GROUPING_THIS_GROUP_ONLY_STR), count)) == 0)
+ io_grouping_type = SCST_IO_GROUPING_THIS_GROUP_ONLY;
+ else if (strncasecmp(buf, SCST_IO_GROUPING_NEVER_STR,
+ min_t(int, strlen(SCST_IO_GROUPING_NEVER_STR), count)) == 0)
+ io_grouping_type = SCST_IO_GROUPING_NEVER;
+ else {
+ res = strict_strtol(buf, 0, &io_grouping_type);
+ if ((res != 0) || (io_grouping_type <= 0)) {
+ PRINT_ERROR("Unknown or not allowed I/O grouping type "
+ "%s", buf);
+ res = -EINVAL;
+ goto out;
+ }
+ }
+
+ if (prev == io_grouping_type)
+ goto out;
+
+ res = scst_alloc_sysfs_work(__scst_acg_io_grouping_type_store_work_fn,
+ &work);
+ if (res != 0)
+ goto out;
+
+ work->tgt = acg->tgt;
+ work->acg = acg;
+ work->io_grouping_type = io_grouping_type;
+
+ res = scst_sysfs_queue_wait_work(work);
+
+out:
+ return res;
+}
+
+static ssize_t scst_tgt_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ return __scst_acg_io_grouping_type_show(acg, buf);
+}
+
+static ssize_t scst_tgt_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ res = __scst_acg_io_grouping_type_store(acg, buf, count);
+ if (res != 0)
+ goto out;
+
+ res = count;
+
+out:
+ return res;
+}
+
+/*
+ * Called with scst_mutex held.
+ *
+ * !! No sysfs works must use kobject_get() to protect acg, due to possible
+ * !! deadlock with scst_mutex (it is waiting for the last put, but
+ * !! the last ref counter holder is waiting for scst_mutex)
+ */
+void scst_acg_sysfs_del(struct scst_acg *acg)
+{
+ int rc;
+
+ kobject_del(acg->luns_kobj);
+ kobject_put(acg->luns_kobj);
+
+ kobject_del(acg->initiators_kobj);
+ kobject_put(acg->initiators_kobj);
+
+ kobject_del(&acg->acg_kobj);
+ kobject_put(&acg->acg_kobj);
+
+ rc = wait_for_completion_timeout(&acg->acg_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for acg %s (%d refs)...", acg->acg_name,
+ atomic_read(&acg->acg_kobj.kref.refcount));
+ wait_for_completion(&acg->acg_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for acg %s", acg->acg_name);
+ }
+ return;
+}
+
+int scst_acg_sysfs_create(struct scst_tgt *tgt,
+ struct scst_acg *acg)
+{
+ int res = 0;
+
+ init_completion(&acg->acg_kobj_release_cmpl);
+
+ res = kobject_init_and_add(&acg->acg_kobj, &acg_ktype,
+ tgt->tgt_ini_grp_kobj, acg->acg_name);
+ if (res != 0) {
+ PRINT_ERROR("Can't add acg '%s' to sysfs", acg->acg_name);
+ goto out;
+ }
+
+ acg->luns_kobj = kobject_create_and_add("luns", &acg->acg_kobj);
+ if (acg->luns_kobj == NULL) {
+ PRINT_ERROR("Can't create luns kobj for tgt %s",
+ tgt->tgt_name);
+ res = -ENOMEM;
+ goto out_del;
+ }
+
+ res = sysfs_create_file(acg->luns_kobj, &scst_acg_luns_mgmt.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_luns_mgmt.attr.name, tgt->tgt_name);
+ goto out_del;
+ }
+
+ acg->initiators_kobj = kobject_create_and_add("initiators",
+ &acg->acg_kobj);
+ if (acg->initiators_kobj == NULL) {
+ PRINT_ERROR("Can't create initiators kobj for tgt %s",
+ tgt->tgt_name);
+ res = -ENOMEM;
+ goto out_del;
+ }
+
+ res = sysfs_create_file(acg->initiators_kobj,
+ &scst_acg_ini_mgmt.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_ini_mgmt.attr.name, tgt->tgt_name);
+ goto out_del;
+ }
+
+ res = sysfs_create_file(&acg->acg_kobj, &scst_acg_addr_method.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_addr_method.attr.name, tgt->tgt_name);
+ goto out_del;
+ }
+
+ res = sysfs_create_file(&acg->acg_kobj, &scst_acg_io_grouping_type.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_io_grouping_type.attr.name, tgt->tgt_name);
+ goto out_del;
+ }
+
+out:
+ return res;
+
+out_del:
+ scst_acg_sysfs_del(acg);
+ goto out;
+}
+
+static ssize_t scst_acg_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ return __scst_acg_addr_method_show(acg, buf);
+}
+
+static ssize_t scst_acg_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ res = __scst_acg_addr_method_store(acg, buf, count);
+ return res;
+}
+
+static ssize_t scst_acg_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ return __scst_acg_io_grouping_type_show(acg, buf);
+}
+
+static ssize_t scst_acg_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ res = __scst_acg_io_grouping_type_store(acg, buf, count);
+ if (res != 0)
+ goto out;
+
+ res = count;
+
+out:
+ return res;
+}
+
+static ssize_t scst_ini_group_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ static char *help = "Usage: echo \"create GROUP_NAME\" >mgmt\n"
+ " echo \"del GROUP_NAME\" >mgmt\n";
+
+ return sprintf(buf, "%s", help);
+}
+
+static int scst_process_ini_group_mgmt_store(char *buffer,
+ struct scst_tgt *tgt)
+{
+ int res, action;
+ int len;
+ char *name;
+ char *p, *e = NULL;
+ struct scst_acg *a, *acg = NULL;
+
+#define SCST_INI_GROUP_ACTION_CREATE 1
+#define SCST_INI_GROUP_ACTION_DEL 2
+
+ TRACE_DBG("tgt %p, buffer %s", tgt, buffer);
+
+ p = buffer;
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ if (strncasecmp("create ", p, 7) == 0) {
+ p += 7;
+ action = SCST_INI_GROUP_ACTION_CREATE;
+ } else if (strncasecmp("del ", p, 4) == 0) {
+ p += 4;
+ action = SCST_INI_GROUP_ACTION_DEL;
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ /* Check if our pointer is still alive */
+ if (scst_check_tgt_acg_ptrs(tgt, NULL) != 0)
+ goto out_unlock;
+
+ while (isspace(*p) && *p != '\0')
+ p++;
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+
+ if (p[0] == '\0') {
+ PRINT_ERROR("%s", "Group name required");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ list_for_each_entry(a, &tgt->tgt_acg_list, acg_list_entry) {
+ if (strcmp(a->acg_name, p) == 0) {
+ TRACE_DBG("group (acg) %p %s found",
+ a, a->acg_name);
+ acg = a;
+ break;
+ }
+ }
+
+ switch (action) {
+ case SCST_INI_GROUP_ACTION_CREATE:
+ TRACE_DBG("Creating group '%s'", p);
+ if (acg != NULL) {
+ PRINT_ERROR("acg name %s exist", p);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ len = strlen(p) + 1;
+ name = kmalloc(len, GFP_KERNEL);
+ if (name == NULL) {
+ PRINT_ERROR("%s", "Allocation of name failed");
+ res = -ENOMEM;
+ goto out_unlock;
+ }
+ strlcpy(name, p, len);
+
+ acg = scst_alloc_add_acg(tgt, name, true);
+ kfree(name);
+ if (acg == NULL)
+ goto out_unlock;
+ break;
+ case SCST_INI_GROUP_ACTION_DEL:
+ TRACE_DBG("Deleting group '%s'", p);
+ if (acg == NULL) {
+ PRINT_ERROR("Group %s not found", p);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ if (!scst_acg_sess_is_empty(acg)) {
+ PRINT_ERROR("Group %s is not empty", acg->acg_name);
+ res = -EBUSY;
+ goto out_unlock;
+ }
+ scst_del_free_acg(acg);
+ break;
+ }
+
+ res = 0;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+
+#undef SCST_LUN_ACTION_CREATE
+#undef SCST_LUN_ACTION_DEL
+}
+
+static int scst_ini_group_mgmt_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return scst_process_ini_group_mgmt_store(work->buf, work->tgt);
+}
+
+static ssize_t scst_ini_group_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ char *buffer;
+ struct scst_tgt *tgt;
+ struct scst_sysfs_work_item *work;
+
+ tgt = container_of(kobj->parent, struct scst_tgt, tgt_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ res = scst_alloc_sysfs_work(scst_ini_group_mgmt_store_work_fn, &work);
+ if (res != 0)
+ goto out_free;
+
+ work->buf = buffer;
+ work->tgt = tgt;
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+
+out_free:
+ kfree(buffer);
+ goto out;
+}
+
+static ssize_t scst_rel_tgt_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt *tgt;
+ int res;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ res = sprintf(buf, "%d\n%s", tgt->rel_tgt_id,
+ (tgt->rel_tgt_id != 0) ? SCST_SYSFS_KEY_MARK "\n" : "");
+ return res;
+}
+
+static int scst_process_rel_tgt_id_store(struct scst_sysfs_work_item *work)
+{
+ int res = 0;
+ struct scst_tgt *tgt = work->tgt;
+ unsigned long rel_tgt_id = work->l;
+
+ /* tgt protected by kobject_get() */
+
+ TRACE_DBG("Trying to set relative target port id %d",
+ (uint16_t)rel_tgt_id);
+
+ if (tgt->tgtt->is_target_enabled(tgt) &&
+ rel_tgt_id != tgt->rel_tgt_id) {
+ if (!scst_is_relative_target_port_id_unique(rel_tgt_id, tgt)) {
+ PRINT_ERROR("Relative port id %d is not unique",
+ (uint16_t)rel_tgt_id);
+ res = -EBADSLT;
+ goto out_put;
+ }
+ }
+
+ if (rel_tgt_id < SCST_MIN_REL_TGT_ID ||
+ rel_tgt_id > SCST_MAX_REL_TGT_ID) {
+ if ((rel_tgt_id == 0) && !tgt->tgtt->is_target_enabled(tgt))
+ goto set;
+
+ PRINT_ERROR("Invalid relative port id %d",
+ (uint16_t)rel_tgt_id);
+ res = -EINVAL;
+ goto out_put;
+ }
+
+set:
+ tgt->rel_tgt_id = (uint16_t)rel_tgt_id;
+
+out_put:
+ kobject_put(&tgt->tgt_kobj);
+ return res;
+}
+
+static ssize_t scst_rel_tgt_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res = 0;
+ struct scst_tgt *tgt;
+ unsigned long rel_tgt_id;
+ struct scst_sysfs_work_item *work;
+
+ if (buf == NULL)
+ goto out;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ res = strict_strtoul(buf, 0, &rel_tgt_id);
+ if (res != 0) {
+ PRINT_ERROR("%s", "Wrong rel_tgt_id");
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_alloc_sysfs_work(scst_process_rel_tgt_id_store, &work);
+ if (res != 0)
+ goto out;
+
+ work->tgt = tgt;
+ work->l = rel_tgt_id;
+
+ kobject_get(&tgt->tgt_kobj);
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+int scst_acn_sysfs_create(struct scst_acn *acn)
+{
+ int res = 0;
+ int len;
+ struct scst_acg *acg = acn->acg;
+ struct kobj_attribute *attr = NULL;
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ static struct lock_class_key __key;
+#endif
+
+ acn->acn_attr = NULL;
+
+ attr = kzalloc(sizeof(struct kobj_attribute), GFP_KERNEL);
+ if (attr == NULL) {
+ PRINT_ERROR("Unable to allocate attributes for initiator '%s'",
+ acn->name);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ len = strlen(acn->name) + 1;
+ attr->attr.name = kzalloc(len, GFP_KERNEL);
+ if (attr->attr.name == NULL) {
+ PRINT_ERROR("Unable to allocate attributes for initiator '%s'",
+ acn->name);
+ res = -ENOMEM;
+ goto out_free;
+ }
+ strlcpy((char *)attr->attr.name, acn->name, len);
+
+ attr->attr.owner = THIS_MODULE;
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ attr->attr.key = &__key;
+#endif
+
+ attr->attr.mode = S_IRUGO;
+ attr->show = scst_acn_file_show;
+ attr->store = NULL;
+
+ res = sysfs_create_file(acg->initiators_kobj, &attr->attr);
+ if (res != 0) {
+ PRINT_ERROR("Unable to create acn '%s' for group '%s'",
+ acn->name, acg->acg_name);
+ kfree(attr->attr.name);
+ goto out_free;
+ }
+
+ acn->acn_attr = attr;
+
+out:
+ return res;
+
+out_free:
+ kfree(attr);
+ goto out;
+}
+
+void scst_acn_sysfs_del(struct scst_acn *acn)
+{
+ struct scst_acg *acg = acn->acg;
+
+ if (acn->acn_attr != NULL) {
+ sysfs_remove_file(acg->initiators_kobj,
+ &acn->acn_attr->attr);
+ kfree(acn->acn_attr->attr.name);
+ kfree(acn->acn_attr);
+ }
+ return;
+}
+
+static ssize_t scst_acn_file_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return scnprintf(buf, SCST_SYSFS_BLOCK_SIZE, "%s\n",
+ attr->attr.name);
+}
+
+static ssize_t scst_acg_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+
+ acg = container_of(kobj->parent, struct scst_acg, acg_kobj);
+ res = __scst_luns_mgmt_store(acg, false, buf, count);
+ return res;
+}
+
+static ssize_t scst_acg_ini_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ static char *help = "Usage: echo \"add INITIATOR_NAME\" "
+ ">mgmt\n"
+ " echo \"del INITIATOR_NAME\" "
+ ">mgmt\n"
+ " echo \"move INITIATOR_NAME DEST_GROUP_NAME\" "
+ ">mgmt\n"
+ " echo \"clear\" "
+ ">mgmt\n";
+
+ return sprintf(buf, "%s", help);
+}
+
+static int scst_process_acg_ini_mgmt_store(char *buffer,
+ struct scst_tgt *tgt, struct scst_acg *acg)
+{
+ int res, action;
+ char *p, *e = NULL;
+ char *name = NULL, *group = NULL;
+ struct scst_acg *acg_dest = NULL;
+ struct scst_acn *acn = NULL, *acn_tmp;
+
+#define SCST_ACG_ACTION_INI_ADD 1
+#define SCST_ACG_ACTION_INI_DEL 2
+#define SCST_ACG_ACTION_INI_CLEAR 3
+#define SCST_ACG_ACTION_INI_MOVE 4
+
+ TRACE_DBG("tgt %p, acg %p, buffer %s", tgt, acg, buffer);
+
+ p = buffer;
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+
+ if (strncasecmp("add", p, 3) == 0) {
+ p += 3;
+ action = SCST_ACG_ACTION_INI_ADD;
+ } else if (strncasecmp("del", p, 3) == 0) {
+ p += 3;
+ action = SCST_ACG_ACTION_INI_DEL;
+ } else if (strncasecmp("clear", p, 5) == 0) {
+ p += 5;
+ action = SCST_ACG_ACTION_INI_CLEAR;
+ } else if (strncasecmp("move", p, 4) == 0) {
+ p += 4;
+ action = SCST_ACG_ACTION_INI_MOVE;
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (action != SCST_ACG_ACTION_INI_CLEAR)
+ if (!isspace(*p)) {
+ PRINT_ERROR("%s", "Syntax error");
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ /* Check if tgt and acg not already freed while we were coming here */
+ if (scst_check_tgt_acg_ptrs(tgt, acg) != 0)
+ goto out_unlock;
+
+ if (action != SCST_ACG_ACTION_INI_CLEAR)
+ while (isspace(*p) && *p != '\0')
+ p++;
+
+ switch (action) {
+ case SCST_ACG_ACTION_INI_ADD:
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+ name = p;
+
+ if (name[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid initiator name");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ res = scst_acg_add_acn(acg, name);
+ if (res != 0)
+ goto out_unlock;
+ break;
+ case SCST_ACG_ACTION_INI_DEL:
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+ name = p;
+
+ if (name[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid initiator name");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ acn = scst_find_acn(acg, name);
+ if (acn == NULL) {
+ PRINT_ERROR("Unable to find "
+ "initiator '%s' in group '%s'",
+ name, acg->acg_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ scst_del_free_acn(acn, true);
+ break;
+ case SCST_ACG_ACTION_INI_CLEAR:
+ list_for_each_entry_safe(acn, acn_tmp, &acg->acn_list,
+ acn_list_entry) {
+ scst_del_free_acn(acn, false);
+ }
+ scst_check_reassign_sessions();
+ break;
+ case SCST_ACG_ACTION_INI_MOVE:
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ if (*e == '\0') {
+ PRINT_ERROR("%s", "Too few parameters");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ *e = '\0';
+ name = p;
+
+ if (name[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid initiator name");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ e++;
+ p = e;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+ group = p;
+
+ if (group[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid group name");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ TRACE_DBG("Move initiator '%s' to group '%s'",
+ name, group);
+
+ acn = scst_find_acn(acg, name);
+ if (acn == NULL) {
+ PRINT_ERROR("Unable to find "
+ "initiator '%s' in group '%s'",
+ name, acg->acg_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ acg_dest = scst_tgt_find_acg(tgt, group);
+ if (acg_dest == NULL) {
+ PRINT_ERROR("Unable to find group '%s' in target '%s'",
+ group, tgt->tgt_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ if (scst_find_acn(acg_dest, name) != NULL) {
+ PRINT_ERROR("Initiator '%s' already exists in group '%s'",
+ name, acg_dest->acg_name);
+ res = -EEXIST;
+ goto out_unlock;
+ }
+ scst_del_free_acn(acn, false);
+
+ res = scst_acg_add_acn(acg_dest, name);
+ if (res != 0)
+ goto out_unlock;
+ break;
+ }
+
+ res = 0;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+
+#undef SCST_ACG_ACTION_INI_ADD
+#undef SCST_ACG_ACTION_INI_DEL
+#undef SCST_ACG_ACTION_INI_CLEAR
+#undef SCST_ACG_ACTION_INI_MOVE
+}
+
+static int scst_acg_ini_mgmt_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return scst_process_acg_ini_mgmt_store(work->buf, work->tgt, work->acg);
+}
+
+static ssize_t scst_acg_ini_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ struct scst_acg *acg;
+
+ acg = container_of(kobj->parent, struct scst_acg, acg_kobj);
+
+ return __scst_acg_mgmt_store(acg, buf, count, false,
+ scst_acg_ini_mgmt_store_work_fn);
+}
+
+/**
+ ** SGV directory implementation
+ **/
+
+static struct kobj_attribute sgv_stat_attr =
+ __ATTR(stats, S_IRUGO | S_IWUSR, sgv_sysfs_stat_show,
+ sgv_sysfs_stat_reset);
+
+static struct attribute *sgv_attrs[] = {
+ &sgv_stat_attr.attr,
+ NULL,
+};
+
+static void sgv_kobj_release(struct kobject *kobj)
+{
+ struct sgv_pool *pool;
+
+ pool = container_of(kobj, struct sgv_pool, sgv_kobj);
+ complete_all(&pool->sgv_kobj_release_cmpl);
+ return;
+}
+
+static struct kobj_type sgv_pool_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = sgv_kobj_release,
+ .default_attrs = sgv_attrs,
+};
+
+int scst_sgv_sysfs_create(struct sgv_pool *pool)
+{
+ int res;
+
+ init_completion(&pool->sgv_kobj_release_cmpl);
+
+ res = kobject_init_and_add(&pool->sgv_kobj, &sgv_pool_ktype,
+ scst_sgv_kobj, pool->name);
+ if (res != 0) {
+ PRINT_ERROR("Can't add sgv pool %s to sysfs", pool->name);
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+void scst_sgv_sysfs_del(struct sgv_pool *pool)
+{
+ int rc;
+
+ kobject_del(&pool->sgv_kobj);
+ kobject_put(&pool->sgv_kobj);
+
+ rc = wait_for_completion_timeout(&pool->sgv_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for SGV pool %s (%d refs)...", pool->name,
+ atomic_read(&pool->sgv_kobj.kref.refcount));
+ wait_for_completion(&pool->sgv_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for SGV pool %s", pool->name);
+ }
+ return;
+}
+
+static struct kobj_attribute sgv_global_stat_attr =
+ __ATTR(global_stats, S_IRUGO | S_IWUSR, sgv_sysfs_global_stat_show,
+ sgv_sysfs_global_stat_reset);
+
+static struct attribute *sgv_default_attrs[] = {
+ &sgv_global_stat_attr.attr,
+ NULL,
+};
+
+static void scst_sysfs_release(struct kobject *kobj)
+{
+ kfree(kobj);
+}
+
+static struct kobj_type sgv_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_sysfs_release,
+ .default_attrs = sgv_default_attrs,
+};
+
+/**
+ ** SCST sysfs root directory implementation
+ **/
+
+static ssize_t scst_threads_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int count;
+
+ count = sprintf(buf, "%d\n%s", scst_main_cmd_threads.nr_threads,
+ (scst_main_cmd_threads.nr_threads != scst_threads) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ return count;
+}
+
+static int scst_process_threads_store(int newtn)
+{
+ int res;
+ long oldtn, delta;
+
+ TRACE_DBG("newtn %d", newtn);
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ oldtn = scst_main_cmd_threads.nr_threads;
+
+ delta = newtn - oldtn;
+ if (delta < 0)
+ scst_del_threads(&scst_main_cmd_threads, -delta);
+ else {
+ res = scst_add_threads(&scst_main_cmd_threads, NULL, NULL, delta);
+ if (res != 0)
+ goto out_up;
+ }
+
+ PRINT_INFO("Changed cmd threads num: old %ld, new %d", oldtn, newtn);
+
+out_up:
+ mutex_unlock(&scst_mutex);
+
+out:
+ return res;
+}
+
+static int scst_threads_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return scst_process_threads_store(work->new_threads_num);
+}
+
+static ssize_t scst_threads_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ long newtn;
+ struct scst_sysfs_work_item *work;
+
+ res = strict_strtol(buf, 0, &newtn);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtol() for %s failed: %d ", buf, res);
+ goto out;
+ }
+ if (newtn <= 0) {
+ PRINT_ERROR("Illegal threads num value %ld", newtn);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_alloc_sysfs_work(scst_threads_store_work_fn, &work);
+ if (res != 0)
+ goto out;
+
+ work->new_threads_num = newtn;
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+static ssize_t scst_setup_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int count;
+
+ count = sprintf(buf, "0x%x\n%s\n", scst_setup_id,
+ (scst_setup_id == 0) ? "" : SCST_SYSFS_KEY_MARK);
+ return count;
+}
+
+static ssize_t scst_setup_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ unsigned long val;
+
+ res = strict_strtoul(buf, 0, &val);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %d ", buf, res);
+ goto out;
+ }
+
+ scst_setup_id = val;
+ PRINT_INFO("Changed scst_setup_id to %x", scst_setup_id);
+
+ res = count;
+
+out:
+ return res;
+}
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static void scst_read_trace_tlb(const struct scst_trace_log *tbl, char *buf,
+ unsigned long log_level, int *pos)
+{
+ const struct scst_trace_log *t = tbl;
+
+ if (t == NULL)
+ goto out;
+
+ while (t->token) {
+ if (log_level & t->val) {
+ *pos += sprintf(&buf[*pos], "%s%s",
+ (*pos == 0) ? "" : " | ",
+ t->token);
+ }
+ t++;
+ }
+out:
+ return;
+}
+
+static ssize_t scst_trace_level_show(const struct scst_trace_log *local_tbl,
+ unsigned long log_level, char *buf, const char *help)
+{
+ int pos = 0;
+
+ scst_read_trace_tlb(scst_trace_tbl, buf, log_level, &pos);
+ scst_read_trace_tlb(local_tbl, buf, log_level, &pos);
+
+ pos += sprintf(&buf[pos], "\n\n\nUsage:\n"
+ " echo \"all|none|default\" >trace_level\n"
+ " echo \"value DEC|0xHEX|0OCT\" >trace_level\n"
+ " echo \"add|del TOKEN\" >trace_level\n"
+ "\nwhere TOKEN is one of [debug, function, line, pid,\n"
+ " buff, mem, sg, out_of_mem,\n"
+ " special, scsi, mgmt, minor,\n"
+ " mgmt_dbg, scsi_serializing,\n"
+ " retry, recv_bot, send_bot, recv_top, pr,\n"
+ " send_top%s]", help != NULL ? help : "");
+
+ return pos;
+}
+
+static ssize_t scst_main_trace_level_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return scst_trace_level_show(scst_local_trace_tbl, trace_flag,
+ buf, NULL);
+}
+
+static int scst_write_trace(const char *buf, size_t length,
+ unsigned long *log_level, unsigned long default_level,
+ const char *name, const struct scst_trace_log *tbl)
+{
+ int res = length;
+ int action;
+ unsigned long level = 0, oldlevel;
+ char *buffer, *p, *e;
+ const struct scst_trace_log *t;
+
+#define SCST_TRACE_ACTION_ALL 1
+#define SCST_TRACE_ACTION_NONE 2
+#define SCST_TRACE_ACTION_DEFAULT 3
+#define SCST_TRACE_ACTION_ADD 4
+#define SCST_TRACE_ACTION_DEL 5
+#define SCST_TRACE_ACTION_VALUE 6
+
+ if ((buf == NULL) || (length == 0)) {
+ res = -EINVAL;
+ goto out;
+ }
+
+ buffer = kmalloc(length+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ PRINT_ERROR("Unable to alloc intermediate buffer (size %zd)",
+ length+1);
+ res = -ENOMEM;
+ goto out;
+ }
+ memcpy(buffer, buf, length);
+ buffer[length] = '\0';
+
+ TRACE_DBG("buffer %s", buffer);
+
+ p = buffer;
+ if (!strncasecmp("all", p, 3)) {
+ action = SCST_TRACE_ACTION_ALL;
+ } else if (!strncasecmp("none", p, 4) || !strncasecmp("null", p, 4)) {
+ action = SCST_TRACE_ACTION_NONE;
+ } else if (!strncasecmp("default", p, 7)) {
+ action = SCST_TRACE_ACTION_DEFAULT;
+ } else if (!strncasecmp("add", p, 3)) {
+ p += 3;
+ action = SCST_TRACE_ACTION_ADD;
+ } else if (!strncasecmp("del", p, 3)) {
+ p += 3;
+ action = SCST_TRACE_ACTION_DEL;
+ } else if (!strncasecmp("value", p, 5)) {
+ p += 5;
+ action = SCST_TRACE_ACTION_VALUE;
+ } else {
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ switch (action) {
+ case SCST_TRACE_ACTION_ADD:
+ case SCST_TRACE_ACTION_DEL:
+ case SCST_TRACE_ACTION_VALUE:
+ if (!isspace(*p)) {
+ PRINT_ERROR("%s", "Syntax error");
+ res = -EINVAL;
+ goto out_free;
+ }
+ }
+
+ switch (action) {
+ case SCST_TRACE_ACTION_ALL:
+ level = TRACE_ALL;
+ break;
+ case SCST_TRACE_ACTION_DEFAULT:
+ level = default_level;
+ break;
+ case SCST_TRACE_ACTION_NONE:
+ level = TRACE_NULL;
+ break;
+ case SCST_TRACE_ACTION_ADD:
+ case SCST_TRACE_ACTION_DEL:
+ while (isspace(*p) && *p != '\0')
+ p++;
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = 0;
+ if (tbl) {
+ t = tbl;
+ while (t->token) {
+ if (!strcasecmp(p, t->token)) {
+ level = t->val;
+ break;
+ }
+ t++;
+ }
+ }
+ if (level == 0) {
+ t = scst_trace_tbl;
+ while (t->token) {
+ if (!strcasecmp(p, t->token)) {
+ level = t->val;
+ break;
+ }
+ t++;
+ }
+ }
+ if (level == 0) {
+ PRINT_ERROR("Unknown token \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+ break;
+ case SCST_TRACE_ACTION_VALUE:
+ while (isspace(*p) && *p != '\0')
+ p++;
+ res = strict_strtoul(p, 0, &level);
+ if (res != 0) {
+ PRINT_ERROR("Invalid trace value \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+ break;
+ }
+
+ oldlevel = *log_level;
+
+ switch (action) {
+ case SCST_TRACE_ACTION_ADD:
+ *log_level |= level;
+ break;
+ case SCST_TRACE_ACTION_DEL:
+ *log_level &= ~level;
+ break;
+ default:
+ *log_level = level;
+ break;
+ }
+
+ PRINT_INFO("Changed trace level for \"%s\": old 0x%08lx, new 0x%08lx",
+ name, oldlevel, *log_level);
+
+out_free:
+ kfree(buffer);
+out:
+ return res;
+
+#undef SCST_TRACE_ACTION_ALL
+#undef SCST_TRACE_ACTION_NONE
+#undef SCST_TRACE_ACTION_DEFAULT
+#undef SCST_TRACE_ACTION_ADD
+#undef SCST_TRACE_ACTION_DEL
+#undef SCST_TRACE_ACTION_VALUE
+}
+
+static ssize_t scst_main_trace_level_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+
+ if (mutex_lock_interruptible(&scst_log_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = scst_write_trace(buf, count, &trace_flag,
+ SCST_DEFAULT_LOG_FLAGS, "scst", scst_local_trace_tbl);
+
+ mutex_unlock(&scst_log_mutex);
+
+out:
+ return res;
+}
+
+#endif /* defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_version_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+
+ sprintf(buf, "%s\n", SCST_VERSION_STRING);
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ strcat(buf, "STRICT_SERIALIZING\n");
+#endif
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ strcat(buf, "EXTRACHECKS\n");
+#endif
+
+#ifdef CONFIG_SCST_TRACING
+ strcat(buf, "TRACING\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+ strcat(buf, "DEBUG\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_TM
+ strcat(buf, "DEBUG_TM\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ strcat(buf, "DEBUG_RETRY\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_OOM
+ strcat(buf, "DEBUG_OOM\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_SN
+ strcat(buf, "DEBUG_SN\n");
+#endif
+
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+ strcat(buf, "USE_EXPECTED_VALUES\n");
+#endif
+
+#ifdef CONFIG_SCST_TEST_IO_IN_SIRQ
+ strcat(buf, "TEST_IO_IN_SIRQ\n");
+#endif
+
+#ifdef CONFIG_SCST_STRICT_SECURITY
+ strcat(buf, "STRICT_SECURITY\n");
+#endif
+ return strlen(buf);
+}
+
+static ssize_t scst_last_sysfs_mgmt_res_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int res;
+
+ spin_lock(&sysfs_work_lock);
+ TRACE_DBG("active_sysfs_works %d", active_sysfs_works);
+ if (active_sysfs_works > 0)
+ res = -EAGAIN;
+ else
+ res = sprintf(buf, "%d\n", last_sysfs_work_res);
+ spin_unlock(&sysfs_work_lock);
+ return res;
+}
+
+static struct kobj_attribute scst_threads_attr =
+ __ATTR(threads, S_IRUGO | S_IWUSR, scst_threads_show,
+ scst_threads_store);
+
+static struct kobj_attribute scst_setup_id_attr =
+ __ATTR(setup_id, S_IRUGO | S_IWUSR, scst_setup_id_show,
+ scst_setup_id_store);
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+static struct kobj_attribute scst_trace_level_attr =
+ __ATTR(trace_level, S_IRUGO | S_IWUSR, scst_main_trace_level_show,
+ scst_main_trace_level_store);
+#endif
+
+static struct kobj_attribute scst_version_attr =
+ __ATTR(version, S_IRUGO, scst_version_show, NULL);
+
+static struct kobj_attribute scst_last_sysfs_mgmt_res_attr =
+ __ATTR(last_sysfs_mgmt_res, S_IRUGO,
+ scst_last_sysfs_mgmt_res_show, NULL);
+
+static struct attribute *scst_sysfs_root_default_attrs[] = {
+ &scst_threads_attr.attr,
+ &scst_setup_id_attr.attr,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ &scst_trace_level_attr.attr,
+#endif
+ &scst_version_attr.attr,
+ &scst_last_sysfs_mgmt_res_attr.attr,
+ NULL,
+};
+
+static void scst_sysfs_root_release(struct kobject *kobj)
+{
+ complete_all(&scst_sysfs_root_release_completion);
+}
+
+static struct kobj_type scst_sysfs_root_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_sysfs_root_release,
+ .default_attrs = scst_sysfs_root_default_attrs,
+};
+
+/**
+ ** Dev handlers
+ **/
+
+static void scst_devt_release(struct kobject *kobj)
+{
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+ complete_all(&devt->devt_kobj_release_compl);
+ return;
+}
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static ssize_t scst_devt_trace_level_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ return scst_trace_level_show(devt->trace_tbl,
+ devt->trace_flags ? *devt->trace_flags : 0, buf,
+ devt->trace_tbl_help);
+}
+
+static ssize_t scst_devt_trace_level_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ if (mutex_lock_interruptible(&scst_log_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = scst_write_trace(buf, count, devt->trace_flags,
+ devt->default_trace_flags, devt->name, devt->trace_tbl);
+
+ mutex_unlock(&scst_log_mutex);
+
+out:
+ return res;
+}
+
+static struct kobj_attribute devt_trace_attr =
+ __ATTR(trace_level, S_IRUGO | S_IWUSR,
+ scst_devt_trace_level_show, scst_devt_trace_level_store);
+
+#endif /* #if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_devt_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ pos = sprintf(buf, "%d - %s\n", devt->type,
+ (unsigned)devt->type > ARRAY_SIZE(scst_dev_handler_types) ?
+ "unknown" : scst_dev_handler_types[devt->type]);
+
+ return pos;
+}
+
+static struct kobj_attribute scst_devt_type_attr =
+ __ATTR(type, S_IRUGO, scst_devt_type_show, NULL);
+
+static struct attribute *scst_devt_default_attrs[] = {
+ &scst_devt_type_attr.attr,
+ NULL,
+};
+
+static struct kobj_type scst_devt_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_devt_release,
+ .default_attrs = scst_devt_default_attrs,
+};
+
+static ssize_t scst_devt_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ char *help = "Usage: echo \"add_device device_name [parameters]\" "
+ ">mgmt\n"
+ " echo \"del_device device_name\" >mgmt\n"
+ "%s%s"
+ "%s"
+ "\n"
+ "where parameters are one or more "
+ "param_name=value pairs separated by ';'\n\n"
+ "%s%s%s%s%s%s%s%s\n";
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ return scnprintf(buf, SCST_SYSFS_BLOCK_SIZE, help,
+ (devt->devt_optional_attributes != NULL) ?
+ " echo \"add_attribute <attribute> <value>\" >mgmt\n"
+ " echo \"del_attribute <attribute> <value>\" >mgmt\n" : "",
+ (devt->dev_optional_attributes != NULL) ?
+ " echo \"add_device_attribute device_name <attribute> <value>\" >mgmt"
+ " echo \"del_device_attribute device_name <attribute> <value>\" >mgmt\n" : "",
+ (devt->mgmt_cmd_help) ? devt->mgmt_cmd_help : "",
+ (devt->add_device_parameters != NULL) ?
+ "The following parameters available: " : "",
+ (devt->add_device_parameters != NULL) ?
+ devt->add_device_parameters : "",
+ (devt->devt_optional_attributes != NULL) ?
+ "The following dev handler attributes available: " : "",
+ (devt->devt_optional_attributes != NULL) ?
+ devt->devt_optional_attributes : "",
+ (devt->devt_optional_attributes != NULL) ? "\n" : "",
+ (devt->dev_optional_attributes != NULL) ?
+ "The following device attributes available: " : "",
+ (devt->dev_optional_attributes != NULL) ?
+ devt->dev_optional_attributes : "",
+ (devt->dev_optional_attributes != NULL) ? "\n" : "");
+}
+
+static int scst_process_devt_mgmt_store(char *buffer,
+ struct scst_dev_type *devt)
+{
+ int res = 0;
+ char *p, *pp, *dev_name;
+
+ /* Check if our pointer is still alive and, if yes, grab it */
+ if (scst_check_grab_devt_ptr(devt, &scst_virtual_dev_type_list) != 0)
+ goto out;
+
+ TRACE_DBG("devt %p, buffer %s", devt, buffer);
+
+ pp = buffer;
+ if (pp[strlen(pp) - 1] == '\n')
+ pp[strlen(pp) - 1] = '\0';
+
+ p = scst_get_next_lexem(&pp);
+
+ if (strcasecmp("add_device", p) == 0) {
+ dev_name = scst_get_next_lexem(&pp);
+ if (*dev_name == '\0') {
+ PRINT_ERROR("%s", "Device name required");
+ res = -EINVAL;
+ goto out_ungrab;
+ }
+ res = devt->add_device(dev_name, pp);
+ } else if (strcasecmp("del_device", p) == 0) {
+ dev_name = scst_get_next_lexem(&pp);
+ if (*dev_name == '\0') {
+ PRINT_ERROR("%s", "Device name required");
+ res = -EINVAL;
+ goto out_ungrab;
+ }
+
+ p = scst_get_next_lexem(&pp);
+ if (*p != '\0')
+ goto out_syntax_err;
+
+ res = devt->del_device(dev_name);
+ } else if (devt->mgmt_cmd != NULL) {
+ scst_restore_token_str(p, pp);
+ res = devt->mgmt_cmd(buffer);
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_ungrab;
+ }
+
+out_ungrab:
+ scst_ungrab_devt_ptr(devt);
+
+out:
+ return res;
+
+out_syntax_err:
+ PRINT_ERROR("Syntax error on \"%s\"", p);
+ res = -EINVAL;
+ goto out_ungrab;
+}
+
+static int scst_devt_mgmt_store_work_fn(struct scst_sysfs_work_item *work)
+{
+ return scst_process_devt_mgmt_store(work->buf, work->devt);
+}
+
+static ssize_t __scst_devt_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count,
+ int (*sysfs_work_fn)(struct scst_sysfs_work_item *work))
+{
+ int res;
+ char *buffer;
+ struct scst_dev_type *devt;
+ struct scst_sysfs_work_item *work;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ res = scst_alloc_sysfs_work(sysfs_work_fn, &work);
+ if (res != 0)
+ goto out_free;
+
+ work->buf = buffer;
+ work->devt = devt;
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+
+out_free:
+ kfree(buffer);
+ goto out;
+}
+
+static ssize_t scst_devt_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ return __scst_devt_mgmt_store(kobj, attr, buf, count,
+ scst_devt_mgmt_store_work_fn);
+}
+
+static struct kobj_attribute scst_devt_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_devt_mgmt_show,
+ scst_devt_mgmt_store);
+
+static ssize_t scst_devt_pass_through_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ char *help = "Usage: echo \"add_device H:C:I:L\" >mgmt\n"
+ " echo \"del_device H:C:I:L\" >mgmt\n";
+ return sprintf(buf, "%s", help);
+}
+
+static int scst_process_devt_pass_through_mgmt_store(char *buffer,
+ struct scst_dev_type *devt)
+{
+ int res = 0;
+ char *p, *pp, *action;
+ unsigned long host, channel, id, lun;
+ struct scst_device *d, *dev = NULL;
+
+ TRACE_DBG("devt %p, buffer %s", devt, buffer);
+
+ pp = buffer;
+ if (pp[strlen(pp) - 1] == '\n')
+ pp[strlen(pp) - 1] = '\0';
+
+ action = scst_get_next_lexem(&pp);
+ p = scst_get_next_lexem(&pp);
+ if (*p == '\0') {
+ PRINT_ERROR("%s", "Device required");
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (*scst_get_next_lexem(&pp) != '\0') {
+ PRINT_ERROR("%s", "Too many parameters");
+ res = -EINVAL;
+ goto out_syntax_err;
+ }
+
+ host = simple_strtoul(p, &p, 0);
+ if ((host == ULONG_MAX) || (*p != ':'))
+ goto out_syntax_err;
+ p++;
+ channel = simple_strtoul(p, &p, 0);
+ if ((channel == ULONG_MAX) || (*p != ':'))
+ goto out_syntax_err;
+ p++;
+ id = simple_strtoul(p, &p, 0);
+ if ((channel == ULONG_MAX) || (*p != ':'))
+ goto out_syntax_err;
+ p++;
+ lun = simple_strtoul(p, &p, 0);
+ if (lun == ULONG_MAX)
+ goto out_syntax_err;
+
+ TRACE_DBG("Dev %ld:%ld:%ld:%ld", host, channel, id, lun);
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ /* Check if devt not be already freed while we were coming here */
+ if (scst_check_devt_ptr(devt, &scst_dev_type_list) != 0)
+ goto out_unlock;
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if ((d->virt_id == 0) &&
+ d->scsi_dev->host->host_no == host &&
+ d->scsi_dev->channel == channel &&
+ d->scsi_dev->id == id &&
+ d->scsi_dev->lun == lun) {
+ dev = d;
+ TRACE_DBG("Dev %p (%ld:%ld:%ld:%ld) found",
+ dev, host, channel, id, lun);
+ break;
+ }
+ }
+ if (dev == NULL) {
+ PRINT_ERROR("Device %ld:%ld:%ld:%ld not found",
+ host, channel, id, lun);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (dev->scsi_dev->type != devt->type) {
+ PRINT_ERROR("Type %d of device %s differs from type "
+ "%d of dev handler %s", dev->type,
+ dev->virt_name, devt->type, devt->name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (strcasecmp("add_device", action) == 0) {
+ res = scst_assign_dev_handler(dev, devt);
+ if (res == 0)
+ PRINT_INFO("Device %s assigned to dev handler %s",
+ dev->virt_name, devt->name);
+ } else if (strcasecmp("del_device", action) == 0) {
+ if (dev->handler != devt) {
+ PRINT_ERROR("Device %s is not assigned to handler %s",
+ dev->virt_name, devt->name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ res = scst_assign_dev_handler(dev, &scst_null_devtype);
+ if (res == 0)
+ PRINT_INFO("Device %s unassigned from dev handler %s",
+ dev->virt_name, devt->name);
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", action);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out:
+ return res;
+
+out_syntax_err:
+ PRINT_ERROR("Syntax error on \"%s\"", p);
+ res = -EINVAL;
+ goto out;
+}
+
+static int scst_devt_pass_through_mgmt_store_work_fn(
+ struct scst_sysfs_work_item *work)
+{
+ return scst_process_devt_pass_through_mgmt_store(work->buf, work->devt);
+}
+
+static ssize_t scst_devt_pass_through_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ return __scst_devt_mgmt_store(kobj, attr, buf, count,
+ scst_devt_pass_through_mgmt_store_work_fn);
+}
+
+static struct kobj_attribute scst_devt_pass_through_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_devt_pass_through_mgmt_show,
+ scst_devt_pass_through_mgmt_store);
+
+int scst_devt_sysfs_create(struct scst_dev_type *devt)
+{
+ int res;
+ struct kobject *parent;
+ const struct attribute **pattr;
+
+ init_completion(&devt->devt_kobj_release_compl);
+
+ if (devt->parent != NULL)
+ parent = &devt->parent->devt_kobj;
+ else
+ parent = scst_handlers_kobj;
+
+ res = kobject_init_and_add(&devt->devt_kobj, &scst_devt_ktype,
+ parent, devt->name);
+ if (res != 0) {
+ PRINT_ERROR("Can't add devt %s to sysfs", devt->name);
+ goto out;
+ }
+
+ if (devt->add_device != NULL) {
+ res = sysfs_create_file(&devt->devt_kobj,
+ &scst_devt_mgmt.attr);
+ } else {
+ res = sysfs_create_file(&devt->devt_kobj,
+ &scst_devt_pass_through_mgmt.attr);
+ }
+ if (res != 0) {
+ PRINT_ERROR("Can't add mgmt attr for dev handler %s",
+ devt->name);
+ goto out_err;
+ }
+
+ pattr = devt->devt_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ res = sysfs_create_file(&devt->devt_kobj, *pattr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add devt attr %s for dev "
+ "handler %s", (*pattr)->name,
+ devt->name);
+ goto out_err;
+ }
+ pattr++;
+ }
+ }
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (devt->trace_flags != NULL) {
+ res = sysfs_create_file(&devt->devt_kobj,
+ &devt_trace_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Can't add devt trace_flag for dev "
+ "handler %s", devt->name);
+ goto out_err;
+ }
+ }
+#endif
+
+out:
+ return res;
+
+out_err:
+ scst_devt_sysfs_del(devt);
+ goto out;
+}
+
+void scst_devt_sysfs_del(struct scst_dev_type *devt)
+{
+ int rc;
+
+ kobject_del(&devt->devt_kobj);
+ kobject_put(&devt->devt_kobj);
+
+ rc = wait_for_completion_timeout(&devt->devt_kobj_release_compl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing of sysfs entry "
+ "for dev handler template %s (%d refs)...", devt->name,
+ atomic_read(&devt->devt_kobj.kref.refcount));
+ wait_for_completion(&devt->devt_kobj_release_compl);
+ PRINT_INFO("Done waiting for releasing sysfs entry "
+ "for dev handler template %s", devt->name);
+ }
+ return;
+}
+
+/**
+ ** Sysfs user info
+ **/
+
+static DEFINE_MUTEX(scst_sysfs_user_info_mutex);
+
+/* All protected by scst_sysfs_user_info_mutex */
+static LIST_HEAD(scst_sysfs_user_info_list);
+static uint32_t scst_sysfs_info_cur_cookie;
+
+/* scst_sysfs_user_info_mutex supposed to be held */
+static struct scst_sysfs_user_info *scst_sysfs_user_find_info(uint32_t cookie)
+{
+ struct scst_sysfs_user_info *info, *res = NULL;
+
+ list_for_each_entry(info, &scst_sysfs_user_info_list,
+ info_list_entry) {
+ if (info->info_cookie == cookie) {
+ res = info;
+ break;
+ }
+ }
+ return res;
+}
+
+/**
+ * scst_sysfs_user_get_info() - get user_info
+ *
+ * Finds the user_info based on cookie and mark it as received the reply by
+ * setting for it flag info_being_executed.
+ *
+ * Returns found entry or NULL.
+ */
+struct scst_sysfs_user_info *scst_sysfs_user_get_info(uint32_t cookie)
+{
+ struct scst_sysfs_user_info *res = NULL;
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ res = scst_sysfs_user_find_info(cookie);
+ if (res != NULL) {
+ if (!res->info_being_executed)
+ res->info_being_executed = 1;
+ }
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_sysfs_user_get_info);
+
+/**
+ ** Helper functionality to help target drivers and dev handlers support
+ ** sending events to user space and wait for their completion in a safe
+ ** manner. See samples how to use it in iscsi-scst or scst_user.
+ **/
+
+/**
+ * scst_sysfs_user_add_info() - create and add user_info in the global list
+ *
+ * Creates an info structure and adds it in the info_list.
+ * Returns 0 and out_info on success, error code otherwise.
+ */
+int scst_sysfs_user_add_info(struct scst_sysfs_user_info **out_info)
+{
+ int res = 0;
+ struct scst_sysfs_user_info *info;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (info == NULL) {
+ PRINT_ERROR("Unable to allocate sysfs user info (size %zd)",
+ sizeof(*info));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ while ((info->info_cookie == 0) ||
+ (scst_sysfs_user_find_info(info->info_cookie) != NULL))
+ info->info_cookie = scst_sysfs_info_cur_cookie++;
+
+ init_completion(&info->info_completion);
+
+ list_add_tail(&info->info_list_entry, &scst_sysfs_user_info_list);
+ info->info_in_list = 1;
+
+ *out_info = info;
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_sysfs_user_add_info);
+
+/**
+ * scst_sysfs_user_del_info - delete and frees user_info
+ */
+void scst_sysfs_user_del_info(struct scst_sysfs_user_info *info)
+{
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ if (info->info_in_list)
+ list_del(&info->info_list_entry);
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+
+ kfree(info);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_sysfs_user_del_info);
+
+/*
+ * Returns true if the reply received and being processed by another part of
+ * the kernel, false otherwise. Also removes the user_info from the list to
+ * fix for the user space that it missed the timeout.
+ */
+static bool scst_sysfs_user_info_executing(struct scst_sysfs_user_info *info)
+{
+ bool res;
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ res = info->info_being_executed;
+
+ if (info->info_in_list) {
+ list_del(&info->info_list_entry);
+ info->info_in_list = 0;
+ }
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+ return res;
+}
+
+/**
+ * scst_wait_info_completion() - wait an user space event's completion
+ *
+ * Waits for the info request been completed by user space at most timeout
+ * jiffies. If the reply received before timeout and being processed by
+ * another part of the kernel, i.e. scst_sysfs_user_info_executing()
+ * returned true, waits for it to complete indefinitely.
+ *
+ * Returns status of the request completion.
+ */
+int scst_wait_info_completion(struct scst_sysfs_user_info *info,
+ unsigned long timeout)
+{
+ int res, rc;
+
+ TRACE_DBG("Waiting for info %p completion", info);
+
+ while (1) {
+ rc = wait_for_completion_interruptible_timeout(
+ &info->info_completion, timeout);
+ if (rc > 0) {
+ TRACE_DBG("Waiting for info %p finished with %d",
+ info, rc);
+ break;
+ } else if (rc == 0) {
+ if (!scst_sysfs_user_info_executing(info)) {
+ PRINT_ERROR("Timeout waiting for user "
+ "space event %p", info);
+ res = -EBUSY;
+ goto out;
+ } else {
+ /* Req is being executed in the kernel */
+ TRACE_DBG("Keep waiting for info %p completion",
+ info);
+ wait_for_completion(&info->info_completion);
+ break;
+ }
+ } else if (rc != -ERESTARTSYS) {
+ res = rc;
+ PRINT_ERROR("wait_for_completion() failed: %d",
+ res);
+ goto out;
+ } else {
+ TRACE_DBG("Waiting for info %p finished with %d, "
+ "retrying", info, rc);
+ }
+ }
+
+ TRACE_DBG("info %p, status %d", info, info->info_status);
+ res = info->info_status;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_wait_info_completion);
+
+int __init scst_sysfs_init(void)
+{
+ int res = 0;
+
+ sysfs_work_thread = kthread_run(sysfs_work_thread_fn,
+ NULL, "scst_uid");
+ if (IS_ERR(sysfs_work_thread)) {
+ res = PTR_ERR(sysfs_work_thread);
+ PRINT_ERROR("kthread_create() for user interface thread "
+ "failed: %d", res);
+ sysfs_work_thread = NULL;
+ goto out;
+ }
+
+ res = kobject_init_and_add(&scst_sysfs_root_kobj,
+ &scst_sysfs_root_ktype, kernel_kobj, "%s", "scst_tgt");
+ if (res != 0)
+ goto sysfs_root_add_error;
+
+ scst_targets_kobj = kobject_create_and_add("targets",
+ &scst_sysfs_root_kobj);
+ if (scst_targets_kobj == NULL)
+ goto targets_kobj_error;
+
+ scst_devices_kobj = kobject_create_and_add("devices",
+ &scst_sysfs_root_kobj);
+ if (scst_devices_kobj == NULL)
+ goto devices_kobj_error;
+
+ scst_sgv_kobj = kzalloc(sizeof(*scst_sgv_kobj), GFP_KERNEL);
+ if (scst_sgv_kobj == NULL)
+ goto sgv_kobj_error;
+
+ res = kobject_init_and_add(scst_sgv_kobj, &sgv_ktype,
+ &scst_sysfs_root_kobj, "%s", "sgv");
+ if (res != 0)
+ goto sgv_kobj_add_error;
+
+ scst_handlers_kobj = kobject_create_and_add("handlers",
+ &scst_sysfs_root_kobj);
+ if (scst_handlers_kobj == NULL)
+ goto handlers_kobj_error;
+
+out:
+ return res;
+
+handlers_kobj_error:
+ kobject_del(scst_sgv_kobj);
+
+sgv_kobj_add_error:
+ kobject_put(scst_sgv_kobj);
+
+sgv_kobj_error:
+ kobject_del(scst_devices_kobj);
+ kobject_put(scst_devices_kobj);
+
+devices_kobj_error:
+ kobject_del(scst_targets_kobj);
+ kobject_put(scst_targets_kobj);
+
+targets_kobj_error:
+ kobject_del(&scst_sysfs_root_kobj);
+
+sysfs_root_add_error:
+ kobject_put(&scst_sysfs_root_kobj);
+
+ kthread_stop(sysfs_work_thread);
+
+ if (res == 0)
+ res = -EINVAL;
+
+ goto out;
+}
+
+void scst_sysfs_cleanup(void)
+{
+
+ PRINT_INFO("%s", "Exiting SCST sysfs hierarchy...");
+
+ kobject_del(scst_sgv_kobj);
+ kobject_put(scst_sgv_kobj);
+
+ kobject_del(scst_devices_kobj);
+ kobject_put(scst_devices_kobj);
+
+ kobject_del(scst_targets_kobj);
+ kobject_put(scst_targets_kobj);
+
+ kobject_del(scst_handlers_kobj);
+ kobject_put(scst_handlers_kobj);
+
+ kobject_del(&scst_sysfs_root_kobj);
+ kobject_put(&scst_sysfs_root_kobj);
+
+ wait_for_completion(&scst_sysfs_root_release_completion);
+ /*
+ * There is a race, when in the release() schedule happens just after
+ * calling complete(), so if we exit and unload scst module immediately,
+ * there will be oops there. So let's give it a chance to quit
+ * gracefully. Unfortunately, current kobjects implementation
+ * doesn't allow better ways to handle it.
+ */
+ msleep(3000);
+
+ if (sysfs_work_thread)
+ kthread_stop(sysfs_work_thread);
+
+ PRINT_INFO("%s", "Exiting SCST sysfs hierarchy done");
+ return;
+}
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 9/17]: SCST debugging support routines
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (7 preceding siblings ...)
2010-09-14 14:44 ` [PATCH 8/17]: SCST SYSFS interface implementation Vladislav Bolkhovitin
@ 2010-09-14 14:44 ` Vladislav Bolkhovitin
2010-09-14 14:45 ` [PATCH 10/17]: SCST SGV cache Vladislav Bolkhovitin
` (8 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:44 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST debugging support routines.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
drivers/scst/scst_debug.c | 223 ++++++++++++++++++++++++++++++++++++
include/scst/scst_debug.h | 284 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 507 insertions(+)
diff -uprN orig/linux-2.6.35/include/scst/scst_debug.h linux-2.6.35/include/scst/scst_debug.h
--- orig/linux-2.6.35/include/scst/scst_debug.h
+++ linux-2.6.35/include/scst/scst_debug.h
@@ -0,0 +1,284 @@
+/*
+ * include/scst_debug.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Contains macroses for execution tracing and error reporting
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_DEBUG_H
+#define __SCST_DEBUG_H
+
+#include <generated/autoconf.h> /* for CONFIG_* */
+
+#include <linux/bug.h> /* for WARN_ON_ONCE */
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+#define EXTRACHECKS_BUG_ON(a) BUG_ON(a)
+#define EXTRACHECKS_WARN_ON(a) WARN_ON(a)
+#define EXTRACHECKS_WARN_ON_ONCE(a) WARN_ON_ONCE(a)
+#else
+#define EXTRACHECKS_BUG_ON(a) do { } while (0)
+#define EXTRACHECKS_WARN_ON(a) do { } while (0)
+#define EXTRACHECKS_WARN_ON_ONCE(a) do { } while (0)
+#endif
+
+#define TRACE_NULL 0x00000000
+#define TRACE_DEBUG 0x00000001
+#define TRACE_FUNCTION 0x00000002
+#define TRACE_LINE 0x00000004
+#define TRACE_PID 0x00000008
+#define TRACE_BUFF 0x00000020
+#define TRACE_MEMORY 0x00000040
+#define TRACE_SG_OP 0x00000080
+#define TRACE_OUT_OF_MEM 0x00000100
+#define TRACE_MINOR 0x00000200 /* less important events */
+#define TRACE_MGMT 0x00000400
+#define TRACE_MGMT_DEBUG 0x00000800
+#define TRACE_SCSI 0x00001000
+#define TRACE_SPECIAL 0x00002000 /* filtering debug, etc */
+#define TRACE_FLOW_CONTROL 0x00004000 /* flow control in action */
+#define TRACE_PRES 0x00008000
+#define TRACE_ALL 0xffffffff
+/* Flags 0xXXXX0000 are local for users */
+
+#define TRACE_MINOR_AND_MGMT_DBG (TRACE_MINOR|TRACE_MGMT_DEBUG)
+
+#ifndef KERN_CONT
+#define KERN_CONT ""
+#endif
+
+/*
+ * Note: in the next two printk() statements the KERN_CONT macro is only
+ * present to suppress a checkpatch warning (KERN_CONT is defined as "").
+ */
+#define PRINT(log_flag, format, args...) \
+ printk(log_flag format "\n", ## args)
+#define PRINTN(log_flag, format, args...) \
+ printk(log_flag format, ## args)
+
+#ifdef LOG_PREFIX
+#define __LOG_PREFIX LOG_PREFIX
+#else
+#define __LOG_PREFIX NULL
+#endif
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+#ifndef CONFIG_SCST_DEBUG
+#define ___unlikely(a) (a)
+#else
+#define ___unlikely(a) unlikely(a)
+#endif
+
+/*
+ * We don't print prefix for debug traces to not put additional preasure
+ * on the logging system in case of a lot of logging.
+ */
+
+int debug_print_prefix(unsigned long trace_flag,
+ const char *prefix, const char *func, int line);
+void debug_print_buffer(const void *data, int len);
+const char *debug_transport_id_to_initiator_name(const uint8_t *transport_id);
+
+#define TRACING_MINOR() (trace_flag & TRACE_MINOR)
+
+#define TRACE(trace, format, args...) \
+do { \
+ if (___unlikely(trace_flag & (trace))) { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, \
+ __func__, __LINE__); \
+ PRINT(KERN_CONT, format, args); \
+ } \
+} while (0)
+
+#ifdef CONFIG_SCST_DEBUG
+
+#define PRINT_BUFFER(message, buff, len) \
+do { \
+ PRINT(KERN_INFO, "%s:%s:", __func__, message); \
+ debug_print_buffer(buff, len); \
+} while (0)
+
+#else
+
+#define PRINT_BUFFER(message, buff, len) \
+do { \
+ PRINT(KERN_INFO, "%s:", message); \
+ debug_print_buffer(buff, len); \
+} while (0)
+
+#endif
+
+#define PRINT_BUFF_FLAG(flag, message, buff, len) \
+do { \
+ if (___unlikely(trace_flag & (flag))) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, "%s:", message); \
+ debug_print_buffer(buff, len); \
+ } \
+} while (0)
+
+#else /* CONFIG_SCST_DEBUG || CONFIG_SCST_TRACING */
+
+#define TRACING_MINOR() (false)
+
+#define TRACE(trace, args...) do {} while (0)
+#define PRINT_BUFFER(message, buff, len) do {} while (0)
+#define PRINT_BUFF_FLAG(flag, message, buff, len) do {} while (0)
+
+#endif /* CONFIG_SCST_DEBUG || CONFIG_SCST_TRACING */
+
+#ifdef CONFIG_SCST_DEBUG
+
+#define TRACE_DBG_FLAG(trace, format, args...) \
+do { \
+ if (trace_flag & (trace)) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, format, args); \
+ } \
+} while (0)
+
+#define TRACE_MEM(args...) TRACE_DBG_FLAG(TRACE_MEMORY, args)
+#define TRACE_SG(args...) TRACE_DBG_FLAG(TRACE_SG_OP, args)
+#define TRACE_DBG(args...) TRACE_DBG_FLAG(TRACE_DEBUG, args)
+#define TRACE_DBG_SPECIAL(args...) TRACE_DBG_FLAG(TRACE_DEBUG|TRACE_SPECIAL, args)
+#define TRACE_MGMT_DBG(args...) TRACE_DBG_FLAG(TRACE_MGMT_DEBUG, args)
+#define TRACE_MGMT_DBG_SPECIAL(args...) \
+ TRACE_DBG_FLAG(TRACE_MGMT_DEBUG|TRACE_SPECIAL, args)
+#define TRACE_PR(args...) TRACE_DBG_FLAG(TRACE_PRES, args)
+
+#define TRACE_BUFFER(message, buff, len) \
+do { \
+ if (trace_flag & TRACE_BUFF) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, "%s:", message); \
+ debug_print_buffer(buff, len); \
+ } \
+} while (0)
+
+#define TRACE_BUFF_FLAG(flag, message, buff, len) \
+do { \
+ if (trace_flag & (flag)) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, "%s:", message); \
+ debug_print_buffer(buff, len); \
+ } \
+} while (0)
+
+#define PRINT_LOG_FLAG(log_flag, format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, format, args); \
+} while (0)
+
+#define PRINT_WARNING(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, "***WARNING***: " format, args); \
+} while (0)
+
+#define PRINT_ERROR(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, "***ERROR***: " format, args); \
+} while (0)
+
+#define PRINT_CRIT_ERROR(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, "***CRITICAL ERROR***: " format, args); \
+} while (0)
+
+#define PRINT_INFO(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, format, args); \
+} while (0)
+
+#else /* CONFIG_SCST_DEBUG */
+
+#define TRACE_MEM(format, args...) do {} while (0)
+#define TRACE_SG(format, args...) do {} while (0)
+#define TRACE_DBG(format, args...) do {} while (0)
+#define TRACE_DBG_FLAG(format, args...) do {} while (0)
+#define TRACE_DBG_SPECIAL(format, args...) do {} while (0)
+#define TRACE_MGMT_DBG(format, args...) do {} while (0)
+#define TRACE_MGMT_DBG_SPECIAL(format, args...) do {} while (0)
+#define TRACE_PR(format, args...) do {} while (0)
+#define TRACE_BUFFER(message, buff, len) do {} while (0)
+#define TRACE_BUFF_FLAG(flag, message, buff, len) do {} while (0)
+
+#ifdef LOG_PREFIX
+
+#define PRINT_INFO(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: " format, LOG_PREFIX, args); \
+} while (0)
+
+#define PRINT_WARNING(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: ***WARNING***: " \
+ format, LOG_PREFIX, args); \
+} while (0)
+
+#define PRINT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: ***ERROR***: " \
+ format, LOG_PREFIX, args); \
+} while (0)
+
+#define PRINT_CRIT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: ***CRITICAL ERROR***: " \
+ format, LOG_PREFIX, args); \
+} while (0)
+
+#else
+
+#define PRINT_INFO(format, args...) \
+do { \
+ PRINT(KERN_INFO, format, args); \
+} while (0)
+
+#define PRINT_WARNING(format, args...) \
+do { \
+ PRINT(KERN_INFO, "***WARNING***: " \
+ format, args); \
+} while (0)
+
+#define PRINT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_ERR, "***ERROR***: " \
+ format, args); \
+} while (0)
+
+#define PRINT_CRIT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_CRIT, "***CRITICAL ERROR***: " \
+ format, args); \
+} while (0)
+
+#endif /* LOG_PREFIX */
+
+#endif /* CONFIG_SCST_DEBUG */
+
+#if defined(CONFIG_SCST_DEBUG) && defined(CONFIG_DEBUG_SLAB)
+#define SCST_SLAB_FLAGS (SLAB_RED_ZONE | SLAB_POISON)
+#else
+#define SCST_SLAB_FLAGS 0L
+#endif
+
+#endif /* __SCST_DEBUG_H */
diff -uprN orig/linux-2.6.35/drivers/scst/scst_debug.c linux-2.6.35/drivers/scst/scst_debug.c
--- orig/linux-2.6.35/drivers/scst/scst_debug.c
+++ linux-2.6.35/drivers/scst/scst_debug.c
@@ -0,0 +1,223 @@
+/*
+ * scst_debug.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Contains helper functions for execution tracing and error reporting.
+ * Intended to be included in main .c file.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <scst/scst.h>
+#include <scst/scst_debug.h>
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+#define TRACE_BUF_SIZE 512
+
+static char trace_buf[TRACE_BUF_SIZE];
+static DEFINE_SPINLOCK(trace_buf_lock);
+
+static inline int get_current_tid(void)
+{
+ /* Code should be the same as in sys_gettid() */
+ if (in_interrupt()) {
+ /*
+ * Unfortunately, task_pid_vnr() isn't IRQ-safe, so otherwise
+ * it can oops. ToDo.
+ */
+ return 0;
+ }
+ return task_pid_vnr(current);
+}
+
+/**
+ * debug_print_prefix() - print debug prefix for a log line
+ *
+ * Prints, if requested by trace_flag, debug prefix for each log line
+ */
+int debug_print_prefix(unsigned long trace_flag,
+ const char *prefix, const char *func, int line)
+{
+ int i = 0;
+ unsigned long flags;
+ int pid = get_current_tid();
+
+ spin_lock_irqsave(&trace_buf_lock, flags);
+
+ trace_buf[0] = '\0';
+
+ if (trace_flag & TRACE_PID)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE, "[%d]: ", pid);
+ if (prefix != NULL)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%s: ",
+ prefix);
+ if (trace_flag & TRACE_FUNCTION)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%s:", func);
+ if (trace_flag & TRACE_LINE)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%i:", line);
+
+ PRINTN(KERN_INFO, "%s", trace_buf);
+
+ spin_unlock_irqrestore(&trace_buf_lock, flags);
+
+ return i;
+}
+EXPORT_SYMBOL(debug_print_prefix);
+
+/**
+ * debug_print_buffer() - print a buffer
+ *
+ * Prints in the log data from the buffer
+ */
+void debug_print_buffer(const void *data, int len)
+{
+ int z, z1, i;
+ const unsigned char *buf = (const unsigned char *) data;
+ unsigned long flags;
+
+ if (buf == NULL)
+ return;
+
+ spin_lock_irqsave(&trace_buf_lock, flags);
+
+ PRINT(KERN_INFO, " (h)___0__1__2__3__4__5__6__7__8__9__A__B__C__D__E__F");
+ for (z = 0, z1 = 0, i = 0; z < len; z++) {
+ if (z % 16 == 0) {
+ if (z != 0) {
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i,
+ " ");
+ for (; (z1 < z) && (i < TRACE_BUF_SIZE - 1);
+ z1++) {
+ if ((buf[z1] >= 0x20) &&
+ (buf[z1] < 0x80))
+ trace_buf[i++] = buf[z1];
+ else
+ trace_buf[i++] = '.';
+ }
+ trace_buf[i] = '\0';
+ PRINT(KERN_INFO, "%s", trace_buf);
+ i = 0;
+ }
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i,
+ "%4x: ", z);
+ }
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%02x ",
+ buf[z]);
+ }
+
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, " ");
+ for (; (z1 < z) && (i < TRACE_BUF_SIZE - 1); z1++) {
+ if ((buf[z1] > 0x20) && (buf[z1] < 0x80))
+ trace_buf[i++] = buf[z1];
+ else
+ trace_buf[i++] = '.';
+ }
+ trace_buf[i] = '\0';
+
+ PRINT(KERN_INFO, "%s", trace_buf);
+
+ spin_unlock_irqrestore(&trace_buf_lock, flags);
+ return;
+}
+EXPORT_SYMBOL(debug_print_buffer);
+
+/*
+ * This function converts transport_id in a string form into internal per-CPU
+ * static buffer. This buffer isn't anyhow protected, because it's acceptable
+ * if the name corrupted in the debug logs because of the race for this buffer.
+ *
+ * Note! You can't call this function 2 or more times in a single logging
+ * (printk) statement, because then each new call of this functon will override
+ * data written in this buffer by the previous call. You should instead split
+ * that logging statement on smaller statements each calling
+ * debug_transport_id_to_initiator_name() only once.
+ */
+const char *debug_transport_id_to_initiator_name(const uint8_t *transport_id)
+{
+ /*
+ * No external protection, because it's acceptable if the name
+ * corrupted in the debug logs because of the race for this
+ * buffer.
+ */
+#define SIZEOF_NAME_BUF 256
+ static char name_bufs[NR_CPUS][SIZEOF_NAME_BUF];
+ char *name_buf;
+ unsigned long flags;
+
+ BUG_ON(transport_id == NULL); /* better to catch it not under lock */
+
+ spin_lock_irqsave(&trace_buf_lock, flags);
+
+ name_buf = name_bufs[smp_processor_id()];
+
+ /*
+ * To prevent external racing with us users from accidentally
+ * missing their NULL terminator.
+ */
+ memset(name_buf, 0, SIZEOF_NAME_BUF);
+ smp_mb();
+
+ switch (transport_id[0] & 0x0f) {
+ case SCSI_TRANSPORTID_PROTOCOLID_ISCSI:
+ scnprintf(name_buf, SIZEOF_NAME_BUF, "%s",
+ &transport_id[4]);
+ break;
+ case SCSI_TRANSPORTID_PROTOCOLID_FCP2:
+ scnprintf(name_buf, SIZEOF_NAME_BUF,
+ "%02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x",
+ transport_id[8], transport_id[9],
+ transport_id[10], transport_id[11],
+ transport_id[12], transport_id[13],
+ transport_id[14], transport_id[15]);
+ break;
+ case SCSI_TRANSPORTID_PROTOCOLID_SPI5:
+ scnprintf(name_buf, SIZEOF_NAME_BUF,
+ "%x:%x", be16_to_cpu((__force __be16)transport_id[2]),
+ be16_to_cpu((__force __be16)transport_id[6]));
+ break;
+ case SCSI_TRANSPORTID_PROTOCOLID_SRP:
+ scnprintf(name_buf, SIZEOF_NAME_BUF,
+ "%02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x"
+ ":%02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x",
+ transport_id[8], transport_id[9],
+ transport_id[10], transport_id[11],
+ transport_id[12], transport_id[13],
+ transport_id[14], transport_id[15],
+ transport_id[16], transport_id[17],
+ transport_id[18], transport_id[19],
+ transport_id[20], transport_id[21],
+ transport_id[22], transport_id[23]);
+ break;
+ case SCSI_TRANSPORTID_PROTOCOLID_SAS:
+ scnprintf(name_buf, SIZEOF_NAME_BUF,
+ "%02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x",
+ transport_id[4], transport_id[5],
+ transport_id[6], transport_id[7],
+ transport_id[8], transport_id[9],
+ transport_id[10], transport_id[11]);
+ break;
+ default:
+ scnprintf(name_buf, SIZEOF_NAME_BUF,
+ "(Not known protocol ID %x)", transport_id[0] & 0x0f);
+ break;
+ }
+
+ spin_unlock_irqrestore(&trace_buf_lock, flags);
+
+ return name_buf;
+#undef SIZEOF_NAME_BUF
+}
+
+#endif /* CONFIG_SCST_DEBUG || CONFIG_SCST_TRACING */
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 10/17]: SCST SGV cache
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (8 preceding siblings ...)
2010-09-14 14:44 ` [PATCH 9/17]: SCST debugging support routines Vladislav Bolkhovitin
@ 2010-09-14 14:45 ` Vladislav Bolkhovitin
2010-09-14 14:45 ` [PATCH 11/17]: SCST core's docs Vladislav Bolkhovitin
` (7 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:45 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST SGV cache. SGV cache is a memory management
subsystem in SCST. More info about it you can find in the documentation
in this patch.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Documentation/scst/sgv_cache.txt | 224 ++++
drivers/scst/scst_mem.c | 1815 +++++++++++++++++++++++++++++++++++++++
drivers/scst/scst_mem.h | 150 +++
include/scst/scst_sgv.h | 97 ++
diff -uprN orig/linux-2.6.35/include/scst/scst_sgv.h linux-2.6.35/include/scst/scst_sgv.h
--- orig/linux-2.6.35/include/scst/scst_sgv.h
+++ linux-2.6.35/include/scst/scst_sgv.h
@@ -0,0 +1,97 @@
+/*
+ * include/scst_sgv.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Include file for SCST SGV cache.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef __SCST_SGV_H
+#define __SCST_SGV_H
+
+/** SGV pool routines and flag bits **/
+
+/* Set if the allocated object must be not from the cache */
+#define SGV_POOL_ALLOC_NO_CACHED 1
+
+/* Set if there should not be any memory allocations on a cache miss */
+#define SGV_POOL_NO_ALLOC_ON_CACHE_MISS 2
+
+/* Set an object should be returned even if it doesn't have SG vector built */
+#define SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL 4
+
+/*
+ * Set if the allocated object must be a new one, i.e. from the cache,
+ * but not cached
+ */
+#define SGV_POOL_ALLOC_GET_NEW 8
+
+struct sgv_pool_obj;
+struct sgv_pool;
+
+/*
+ * Structure to keep a memory limit for an SCST object
+ */
+struct scst_mem_lim {
+ /* How much memory allocated under this object */
+ atomic_t alloced_pages;
+
+ /*
+ * How much memory allowed to allocated under this object. Put here
+ * mostly to save a possible cache miss accessing scst_max_dev_cmd_mem.
+ */
+ int max_allowed_pages;
+};
+
+/* Types of clustering */
+enum sgv_clustering_types {
+ /* No clustering performed */
+ sgv_no_clustering = 0,
+
+ /*
+ * A page will only be merged with the latest previously allocated
+ * page, so the order of pages in the SG will be preserved.
+ */
+ sgv_tail_clustering,
+
+ /*
+ * Free merging of pages at any place in the SG is allowed. This mode
+ * usually provides the best merging rate.
+ */
+ sgv_full_clustering,
+};
+
+struct sgv_pool *sgv_pool_create(const char *name,
+ enum sgv_clustering_types clustered, int single_alloc_pages,
+ bool shared, int purge_interval);
+void sgv_pool_del(struct sgv_pool *pool);
+
+void sgv_pool_get(struct sgv_pool *pool);
+void sgv_pool_put(struct sgv_pool *pool);
+
+void sgv_pool_flush(struct sgv_pool *pool);
+
+void sgv_pool_set_allocator(struct sgv_pool *pool,
+ struct page *(*alloc_pages_fn)(struct scatterlist *, gfp_t, void *),
+ void (*free_pages_fn)(struct scatterlist *, int, void *));
+
+struct scatterlist *sgv_pool_alloc(struct sgv_pool *pool, unsigned int size,
+ gfp_t gfp_mask, int flags, int *count,
+ struct sgv_pool_obj **sgv, struct scst_mem_lim *mem_lim, void *priv);
+void sgv_pool_free(struct sgv_pool_obj *sgv, struct scst_mem_lim *mem_lim);
+
+void *sgv_get_priv(struct sgv_pool_obj *sgv);
+
+void scst_init_mem_lim(struct scst_mem_lim *mem_lim);
+
+#endif /* __SCST_SGV_H */
diff -uprN orig/linux-2.6.35/drivers/scst/scst_mem.h linux-2.6.35/drivers/scst/scst_mem.h
--- orig/linux-2.6.35/drivers/scst/scst_mem.h
+++ linux-2.6.35/drivers/scst/scst_mem.h
@@ -0,0 +1,150 @@
+/*
+ * scst_mem.h
+ *
+ * Copyright (C) 2006 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/scatterlist.h>
+#include <linux/workqueue.h>
+
+#define SGV_POOL_ELEMENTS 11
+
+/*
+ * sg_num is indexed by the page number, pg_count is indexed by the sg number.
+ * Made in one entry to simplify the code (eg all sizeof(*) parts) and save
+ * some CPU cache for non-clustered case.
+ */
+struct trans_tbl_ent {
+ unsigned short sg_num;
+ unsigned short pg_count;
+};
+
+/*
+ * SGV pool object
+ */
+struct sgv_pool_obj {
+ int cache_num;
+ int pages;
+
+ /* jiffies, protected by sgv_pool_lock */
+ unsigned long time_stamp;
+
+ struct list_head recycling_list_entry;
+ struct list_head sorted_recycling_list_entry;
+
+ struct sgv_pool *owner_pool;
+ int orig_sg;
+ int orig_length;
+ int sg_count;
+ void *allocator_priv;
+ struct trans_tbl_ent *trans_tbl;
+ struct scatterlist *sg_entries;
+ struct scatterlist sg_entries_data[0];
+};
+
+/*
+ * SGV pool statistics accounting structure
+ */
+struct sgv_pool_cache_acc {
+ atomic_t total_alloc, hit_alloc;
+ atomic_t merged;
+};
+
+/*
+ * SGV pool allocation functions
+ */
+struct sgv_pool_alloc_fns {
+ struct page *(*alloc_pages_fn)(struct scatterlist *sg, gfp_t gfp_mask,
+ void *priv);
+ void (*free_pages_fn)(struct scatterlist *sg, int sg_count,
+ void *priv);
+};
+
+/*
+ * SGV pool
+ */
+struct sgv_pool {
+ enum sgv_clustering_types clustering_type;
+ int single_alloc_pages;
+ int max_cached_pages;
+
+ struct sgv_pool_alloc_fns alloc_fns;
+
+ /* <=4K, <=8, <=16, <=32, <=64, <=128, <=256, <=512, <=1024, <=2048 */
+ struct kmem_cache *caches[SGV_POOL_ELEMENTS];
+
+ spinlock_t sgv_pool_lock; /* outer lock for sgv_pools_lock! */
+
+ int purge_interval;
+
+ /* Protected by sgv_pool_lock, if necessary */
+ unsigned int purge_work_scheduled:1;
+
+ /* Protected by sgv_pool_lock */
+ struct list_head sorted_recycling_list;
+
+ int inactive_cached_pages; /* protected by sgv_pool_lock */
+
+ /* Protected by sgv_pool_lock */
+ struct list_head recycling_lists[SGV_POOL_ELEMENTS];
+
+ int cached_pages, cached_entries; /* protected by sgv_pool_lock */
+
+ struct sgv_pool_cache_acc cache_acc[SGV_POOL_ELEMENTS];
+
+ struct delayed_work sgv_purge_work;
+
+ struct list_head sgv_active_pools_list_entry;
+
+ atomic_t big_alloc, big_pages, big_merged;
+ atomic_t other_alloc, other_pages, other_merged;
+
+ atomic_t sgv_pool_ref;
+
+ int max_caches;
+
+ /* SCST_MAX_NAME + few more bytes to match scst_user expectations */
+ char cache_names[SGV_POOL_ELEMENTS][SCST_MAX_NAME + 10];
+ char name[SCST_MAX_NAME + 10];
+
+ struct mm_struct *owner_mm;
+
+ struct list_head sgv_pools_list_entry;
+
+ struct kobject sgv_kobj;
+
+ /* sysfs release completion */
+ struct completion sgv_kobj_release_cmpl;
+};
+
+static inline struct scatterlist *sgv_pool_sg(struct sgv_pool_obj *obj)
+{
+ return obj->sg_entries;
+}
+
+int scst_sgv_pools_init(unsigned long mem_hwmark, unsigned long mem_lwmark);
+void scst_sgv_pools_deinit(void);
+
+ssize_t sgv_sysfs_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+ssize_t sgv_sysfs_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count);
+ssize_t sgv_sysfs_global_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+ssize_t sgv_sysfs_global_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count);
+
+void scst_sgv_pool_use_norm(struct scst_tgt_dev *tgt_dev);
+void scst_sgv_pool_use_norm_clust(struct scst_tgt_dev *tgt_dev);
+void scst_sgv_pool_use_dma(struct scst_tgt_dev *tgt_dev);
diff -uprN orig/linux-2.6.35/drivers/scst/scst_mem.c linux-2.6.35/drivers/scst/scst_mem.c
--- orig/linux-2.6.35/drivers/scst/scst_mem.c
+++ linux-2.6.35/drivers/scst/scst_mem.c
@@ -0,0 +1,1815 @@
+/*
+ * scst_mem.c
+ *
+ * Copyright (C) 2006 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+
+#include <scst/scst.h>
+#include "scst_priv.h"
+#include "scst_mem.h"
+
+#define SGV_DEFAULT_PURGE_INTERVAL (60 * HZ)
+#define SGV_MIN_SHRINK_INTERVAL (1 * HZ)
+
+/* Max pages freed from a pool per shrinking iteration */
+#define MAX_PAGES_PER_POOL 50
+
+static struct sgv_pool *sgv_norm_clust_pool, *sgv_norm_pool, *sgv_dma_pool;
+
+static atomic_t sgv_pages_total = ATOMIC_INIT(0);
+
+/* Both read-only */
+static int sgv_hi_wmk;
+static int sgv_lo_wmk;
+
+static int sgv_max_local_pages, sgv_max_trans_pages;
+
+static DEFINE_SPINLOCK(sgv_pools_lock); /* inner lock for sgv_pool_lock! */
+static DEFINE_MUTEX(sgv_pools_mutex);
+
+/* Both protected by sgv_pools_lock */
+static struct sgv_pool *sgv_cur_purge_pool;
+static LIST_HEAD(sgv_active_pools_list);
+
+static atomic_t sgv_releases_on_hiwmk = ATOMIC_INIT(0);
+static atomic_t sgv_releases_on_hiwmk_failed = ATOMIC_INIT(0);
+
+static atomic_t sgv_other_total_alloc = ATOMIC_INIT(0);
+
+static struct shrinker sgv_shrinker;
+
+/*
+ * Protected by sgv_pools_mutex AND sgv_pools_lock for writes,
+ * either one for reads.
+ */
+static LIST_HEAD(sgv_pools_list);
+
+static inline bool sgv_pool_clustered(const struct sgv_pool *pool)
+{
+ return pool->clustering_type != sgv_no_clustering;
+}
+
+void scst_sgv_pool_use_norm(struct scst_tgt_dev *tgt_dev)
+{
+ tgt_dev->gfp_mask = __GFP_NOWARN;
+ tgt_dev->pool = sgv_norm_pool;
+ clear_bit(SCST_TGT_DEV_CLUST_POOL, &tgt_dev->tgt_dev_flags);
+}
+
+void scst_sgv_pool_use_norm_clust(struct scst_tgt_dev *tgt_dev)
+{
+ TRACE_MEM("%s", "Use clustering");
+ tgt_dev->gfp_mask = __GFP_NOWARN;
+ tgt_dev->pool = sgv_norm_clust_pool;
+ set_bit(SCST_TGT_DEV_CLUST_POOL, &tgt_dev->tgt_dev_flags);
+}
+
+void scst_sgv_pool_use_dma(struct scst_tgt_dev *tgt_dev)
+{
+ TRACE_MEM("%s", "Use ISA DMA memory");
+ tgt_dev->gfp_mask = __GFP_NOWARN | GFP_DMA;
+ tgt_dev->pool = sgv_dma_pool;
+ clear_bit(SCST_TGT_DEV_CLUST_POOL, &tgt_dev->tgt_dev_flags);
+}
+
+/* Must be no locks */
+static void sgv_dtor_and_free(struct sgv_pool_obj *obj)
+{
+ struct sgv_pool *pool = obj->owner_pool;
+
+ TRACE_MEM("Destroying sgv obj %p", obj);
+
+ if (obj->sg_count != 0) {
+ pool->alloc_fns.free_pages_fn(obj->sg_entries,
+ obj->sg_count, obj->allocator_priv);
+ }
+ if (obj->sg_entries != obj->sg_entries_data) {
+ if (obj->trans_tbl !=
+ (struct trans_tbl_ent *)obj->sg_entries_data) {
+ /* kfree() handles NULL parameter */
+ kfree(obj->trans_tbl);
+ obj->trans_tbl = NULL;
+ }
+ kfree(obj->sg_entries);
+ }
+
+ kmem_cache_free(pool->caches[obj->cache_num], obj);
+ return;
+}
+
+/* Might be called under sgv_pool_lock */
+static inline void sgv_del_from_active(struct sgv_pool *pool)
+{
+ struct list_head *next;
+
+ TRACE_MEM("Deleting sgv pool %p from the active list", pool);
+
+ spin_lock_bh(&sgv_pools_lock);
+
+ next = pool->sgv_active_pools_list_entry.next;
+ list_del(&pool->sgv_active_pools_list_entry);
+
+ if (sgv_cur_purge_pool == pool) {
+ TRACE_MEM("Sgv pool %p is sgv cur purge pool", pool);
+
+ if (next == &sgv_active_pools_list)
+ next = next->next;
+
+ if (next == &sgv_active_pools_list) {
+ sgv_cur_purge_pool = NULL;
+ TRACE_MEM("%s", "Sgv active list now empty");
+ } else {
+ sgv_cur_purge_pool = list_entry(next, typeof(*pool),
+ sgv_active_pools_list_entry);
+ TRACE_MEM("New sgv cur purge pool %p",
+ sgv_cur_purge_pool);
+ }
+ }
+
+ spin_unlock_bh(&sgv_pools_lock);
+ return;
+}
+
+/* Must be called under sgv_pool_lock held */
+static void sgv_dec_cached_entries(struct sgv_pool *pool, int pages)
+{
+ pool->cached_entries--;
+ pool->cached_pages -= pages;
+
+ if (pool->cached_entries == 0)
+ sgv_del_from_active(pool);
+
+ return;
+}
+
+/* Must be called under sgv_pool_lock held */
+static void __sgv_purge_from_cache(struct sgv_pool_obj *obj)
+{
+ int pages = obj->pages;
+ struct sgv_pool *pool = obj->owner_pool;
+
+ TRACE_MEM("Purging sgv obj %p from pool %p (new cached_entries %d)",
+ obj, pool, pool->cached_entries-1);
+
+ list_del(&obj->sorted_recycling_list_entry);
+ list_del(&obj->recycling_list_entry);
+
+ pool->inactive_cached_pages -= pages;
+ sgv_dec_cached_entries(pool, pages);
+
+ atomic_sub(pages, &sgv_pages_total);
+
+ return;
+}
+
+/* Must be called under sgv_pool_lock held */
+static bool sgv_purge_from_cache(struct sgv_pool_obj *obj, int min_interval,
+ unsigned long cur_time)
+{
+ EXTRACHECKS_BUG_ON(min_interval < 0);
+
+ TRACE_MEM("Checking if sgv obj %p should be purged (cur time %ld, "
+ "obj time %ld, time to purge %ld)", obj, cur_time,
+ obj->time_stamp, obj->time_stamp + min_interval);
+
+ if (time_after_eq(cur_time, (obj->time_stamp + min_interval))) {
+ __sgv_purge_from_cache(obj);
+ return true;
+ }
+ return false;
+}
+
+/* No locks */
+static int sgv_shrink_pool(struct sgv_pool *pool, int nr, int min_interval,
+ unsigned long cur_time)
+{
+ int freed = 0;
+
+ TRACE_MEM("Trying to shrink pool %p (nr %d, min_interval %d)",
+ pool, nr, min_interval);
+
+ if (pool->purge_interval < 0) {
+ TRACE_MEM("Not shrinkable pool %p, skipping", pool);
+ goto out;
+ }
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ while (!list_empty(&pool->sorted_recycling_list) &&
+ (atomic_read(&sgv_pages_total) > sgv_lo_wmk)) {
+ struct sgv_pool_obj *obj = list_entry(
+ pool->sorted_recycling_list.next,
+ struct sgv_pool_obj, sorted_recycling_list_entry);
+
+ if (sgv_purge_from_cache(obj, min_interval, cur_time)) {
+ int pages = obj->pages;
+
+ freed += pages;
+ nr -= pages;
+
+ TRACE_MEM("%d pages purged from pool %p (nr left %d, "
+ "total freed %d)", pages, pool, nr, freed);
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ sgv_dtor_and_free(obj);
+ spin_lock_bh(&pool->sgv_pool_lock);
+ } else
+ break;
+
+ if ((nr <= 0) || (freed >= MAX_PAGES_PER_POOL)) {
+ if (freed >= MAX_PAGES_PER_POOL)
+ TRACE_MEM("%d pages purged from pool %p, "
+ "leaving", freed, pool);
+ break;
+ }
+ }
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+out:
+ return nr;
+}
+
+/* No locks */
+static int __sgv_shrink(int nr, int min_interval)
+{
+ struct sgv_pool *pool;
+ unsigned long cur_time = jiffies;
+ int prev_nr = nr;
+ bool circle = false;
+
+ TRACE_MEM("Trying to shrink %d pages from all sgv pools "
+ "(min_interval %d)", nr, min_interval);
+
+ while (nr > 0) {
+ struct list_head *next;
+
+ spin_lock_bh(&sgv_pools_lock);
+
+ pool = sgv_cur_purge_pool;
+ if (pool == NULL) {
+ if (list_empty(&sgv_active_pools_list)) {
+ TRACE_MEM("%s", "Active pools list is empty");
+ goto out_unlock;
+ }
+
+ pool = list_entry(sgv_active_pools_list.next,
+ typeof(*pool),
+ sgv_active_pools_list_entry);
+ }
+ sgv_pool_get(pool);
+
+ next = pool->sgv_active_pools_list_entry.next;
+ if (next == &sgv_active_pools_list) {
+ if (circle && (prev_nr == nr)) {
+ TRACE_MEM("Full circle done, but no progress, "
+ "leaving (nr %d)", nr);
+ goto out_unlock_put;
+ }
+ circle = true;
+ prev_nr = nr;
+
+ next = next->next;
+ }
+
+ sgv_cur_purge_pool = list_entry(next, typeof(*pool),
+ sgv_active_pools_list_entry);
+ TRACE_MEM("New cur purge pool %p", sgv_cur_purge_pool);
+
+ spin_unlock_bh(&sgv_pools_lock);
+
+ nr = sgv_shrink_pool(pool, nr, min_interval, cur_time);
+
+ sgv_pool_put(pool);
+ }
+
+out:
+ return nr;
+
+out_unlock:
+ spin_unlock_bh(&sgv_pools_lock);
+ goto out;
+
+out_unlock_put:
+ spin_unlock_bh(&sgv_pools_lock);
+ sgv_pool_put(pool);
+ goto out;
+}
+
+static int sgv_shrink(struct shrinker *shrinker, int nr, gfp_t gfpm)
+{
+
+ if (nr > 0) {
+ nr = __sgv_shrink(nr, SGV_MIN_SHRINK_INTERVAL);
+ TRACE_MEM("Left %d", nr);
+ } else {
+ struct sgv_pool *pool;
+ int inactive_pages = 0;
+
+ spin_lock_bh(&sgv_pools_lock);
+ list_for_each_entry(pool, &sgv_active_pools_list,
+ sgv_active_pools_list_entry) {
+ if (pool->purge_interval > 0)
+ inactive_pages += pool->inactive_cached_pages;
+ }
+ spin_unlock_bh(&sgv_pools_lock);
+
+ nr = max((int)0, inactive_pages - sgv_lo_wmk);
+ TRACE_MEM("Can free %d (total %d)", nr,
+ atomic_read(&sgv_pages_total));
+ }
+ return nr;
+}
+
+static void sgv_purge_work_fn(struct delayed_work *work)
+{
+ unsigned long cur_time = jiffies;
+ struct sgv_pool *pool = container_of(work, struct sgv_pool,
+ sgv_purge_work);
+
+ TRACE_MEM("Purge work for pool %p", pool);
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ pool->purge_work_scheduled = false;
+
+ while (!list_empty(&pool->sorted_recycling_list)) {
+ struct sgv_pool_obj *obj = list_entry(
+ pool->sorted_recycling_list.next,
+ struct sgv_pool_obj, sorted_recycling_list_entry);
+
+ if (sgv_purge_from_cache(obj, pool->purge_interval, cur_time)) {
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ sgv_dtor_and_free(obj);
+ spin_lock_bh(&pool->sgv_pool_lock);
+ } else {
+ /*
+ * Let's reschedule it for full period to not get here
+ * too often. In the worst case we have shrinker
+ * to reclaim buffers quickier.
+ */
+ TRACE_MEM("Rescheduling purge work for pool %p (delay "
+ "%d HZ/%d sec)", pool, pool->purge_interval,
+ pool->purge_interval/HZ);
+ schedule_delayed_work(&pool->sgv_purge_work,
+ pool->purge_interval);
+ pool->purge_work_scheduled = true;
+ break;
+ }
+ }
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ TRACE_MEM("Leaving purge work for pool %p", pool);
+ return;
+}
+
+static int sgv_check_full_clustering(struct scatterlist *sg, int cur, int hint)
+{
+ int res = -1;
+ int i = hint;
+ unsigned long pfn_cur = page_to_pfn(sg_page(&sg[cur]));
+ int len_cur = sg[cur].length;
+ unsigned long pfn_cur_next = pfn_cur + (len_cur >> PAGE_SHIFT);
+ int full_page_cur = (len_cur & (PAGE_SIZE - 1)) == 0;
+ unsigned long pfn, pfn_next;
+ bool full_page;
+
+#if 0
+ TRACE_MEM("pfn_cur %ld, pfn_cur_next %ld, len_cur %d, full_page_cur %d",
+ pfn_cur, pfn_cur_next, len_cur, full_page_cur);
+#endif
+
+ /* check the hint first */
+ if (i >= 0) {
+ pfn = page_to_pfn(sg_page(&sg[i]));
+ pfn_next = pfn + (sg[i].length >> PAGE_SHIFT);
+ full_page = (sg[i].length & (PAGE_SIZE - 1)) == 0;
+
+ if ((pfn == pfn_cur_next) && full_page_cur)
+ goto out_head;
+
+ if ((pfn_next == pfn_cur) && full_page)
+ goto out_tail;
+ }
+
+ /* ToDo: implement more intelligent search */
+ for (i = cur - 1; i >= 0; i--) {
+ pfn = page_to_pfn(sg_page(&sg[i]));
+ pfn_next = pfn + (sg[i].length >> PAGE_SHIFT);
+ full_page = (sg[i].length & (PAGE_SIZE - 1)) == 0;
+
+ if ((pfn == pfn_cur_next) && full_page_cur)
+ goto out_head;
+
+ if ((pfn_next == pfn_cur) && full_page)
+ goto out_tail;
+ }
+
+out:
+ return res;
+
+out_tail:
+ TRACE_MEM("SG segment %d will be tail merged with segment %d", cur, i);
+ sg[i].length += len_cur;
+ sg_clear(&sg[cur]);
+ res = i;
+ goto out;
+
+out_head:
+ TRACE_MEM("SG segment %d will be head merged with segment %d", cur, i);
+ sg_assign_page(&sg[i], sg_page(&sg[cur]));
+ sg[i].length += len_cur;
+ sg_clear(&sg[cur]);
+ res = i;
+ goto out;
+}
+
+static int sgv_check_tail_clustering(struct scatterlist *sg, int cur, int hint)
+{
+ int res = -1;
+ unsigned long pfn_cur = page_to_pfn(sg_page(&sg[cur]));
+ int len_cur = sg[cur].length;
+ int prev;
+ unsigned long pfn_prev;
+ bool full_page;
+
+#ifdef SCST_HIGHMEM
+ if (page >= highmem_start_page) {
+ TRACE_MEM("%s", "HIGHMEM page allocated, no clustering")
+ goto out;
+ }
+#endif
+
+#if 0
+ TRACE_MEM("pfn_cur %ld, pfn_cur_next %ld, len_cur %d, full_page_cur %d",
+ pfn_cur, pfn_cur_next, len_cur, full_page_cur);
+#endif
+
+ if (cur == 0)
+ goto out;
+
+ prev = cur - 1;
+ pfn_prev = page_to_pfn(sg_page(&sg[prev])) +
+ (sg[prev].length >> PAGE_SHIFT);
+ full_page = (sg[prev].length & (PAGE_SIZE - 1)) == 0;
+
+ if ((pfn_prev == pfn_cur) && full_page) {
+ TRACE_MEM("SG segment %d will be tail merged with segment %d",
+ cur, prev);
+ sg[prev].length += len_cur;
+ sg_clear(&sg[cur]);
+ res = prev;
+ }
+
+out:
+ return res;
+}
+
+static void sgv_free_sys_sg_entries(struct scatterlist *sg, int sg_count,
+ void *priv)
+{
+ int i;
+
+ TRACE_MEM("sg=%p, sg_count=%d", sg, sg_count);
+
+ for (i = 0; i < sg_count; i++) {
+ struct page *p = sg_page(&sg[i]);
+ int len = sg[i].length;
+ int pages =
+ (len >> PAGE_SHIFT) + ((len & ~PAGE_MASK) != 0);
+
+ TRACE_MEM("page %lx, len %d, pages %d",
+ (unsigned long)p, len, pages);
+
+ while (pages > 0) {
+ int order = 0;
+
+/*
+ * __free_pages() doesn't like freeing pages with not that order with
+ * which they were allocated, so disable this small optimization.
+ */
+#if 0
+ if (len > 0) {
+ while (((1 << order) << PAGE_SHIFT) < len)
+ order++;
+ len = 0;
+ }
+#endif
+ TRACE_MEM("free_pages(): order %d, page %lx",
+ order, (unsigned long)p);
+
+ __free_pages(p, order);
+
+ pages -= 1 << order;
+ p += 1 << order;
+ }
+ }
+}
+
+static struct page *sgv_alloc_sys_pages(struct scatterlist *sg,
+ gfp_t gfp_mask, void *priv)
+{
+ struct page *page = alloc_pages(gfp_mask, 0);
+
+ sg_set_page(sg, page, PAGE_SIZE, 0);
+ TRACE_MEM("page=%p, sg=%p, priv=%p", page, sg, priv);
+ if (page == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of "
+ "sg page failed");
+ }
+ return page;
+}
+
+static int sgv_alloc_sg_entries(struct scatterlist *sg, int pages,
+ gfp_t gfp_mask, enum sgv_clustering_types clustering_type,
+ struct trans_tbl_ent *trans_tbl,
+ const struct sgv_pool_alloc_fns *alloc_fns, void *priv)
+{
+ int sg_count = 0;
+ int pg, i, j;
+ int merged = -1;
+
+ TRACE_MEM("pages=%d, clustering_type=%d", pages, clustering_type);
+
+#if 0
+ gfp_mask |= __GFP_COLD;
+#endif
+#ifdef CONFIG_SCST_STRICT_SECURITY
+ gfp_mask |= __GFP_ZERO;
+#endif
+
+ for (pg = 0; pg < pages; pg++) {
+ void *rc;
+#ifdef CONFIG_SCST_DEBUG_OOM
+ if (((gfp_mask & __GFP_NOFAIL) != __GFP_NOFAIL) &&
+ ((scst_random() % 10000) == 55))
+ rc = NULL;
+ else
+#endif
+ rc = alloc_fns->alloc_pages_fn(&sg[sg_count], gfp_mask,
+ priv);
+ if (rc == NULL)
+ goto out_no_mem;
+
+ /*
+ * This code allows compiler to see full body of the clustering
+ * functions and gives it a chance to generate better code.
+ * At least, the resulting code is smaller, comparing to
+ * calling them using a function pointer.
+ */
+ if (clustering_type == sgv_full_clustering)
+ merged = sgv_check_full_clustering(sg, sg_count, merged);
+ else if (clustering_type == sgv_tail_clustering)
+ merged = sgv_check_tail_clustering(sg, sg_count, merged);
+ else
+ merged = -1;
+
+ if (merged == -1)
+ sg_count++;
+
+ TRACE_MEM("pg=%d, merged=%d, sg_count=%d", pg, merged,
+ sg_count);
+ }
+
+ if ((clustering_type != sgv_no_clustering) && (trans_tbl != NULL)) {
+ pg = 0;
+ for (i = 0; i < pages; i++) {
+ int n = (sg[i].length >> PAGE_SHIFT) +
+ ((sg[i].length & ~PAGE_MASK) != 0);
+ trans_tbl[i].pg_count = pg;
+ for (j = 0; j < n; j++)
+ trans_tbl[pg++].sg_num = i+1;
+ TRACE_MEM("i=%d, n=%d, pg_count=%d", i, n,
+ trans_tbl[i].pg_count);
+ }
+ }
+
+out:
+ TRACE_MEM("sg_count=%d", sg_count);
+ return sg_count;
+
+out_no_mem:
+ alloc_fns->free_pages_fn(sg, sg_count, priv);
+ sg_count = 0;
+ goto out;
+}
+
+static int sgv_alloc_arrays(struct sgv_pool_obj *obj,
+ int pages_to_alloc, gfp_t gfp_mask)
+{
+ int sz, tsz = 0;
+ int res = 0;
+
+ sz = pages_to_alloc * sizeof(obj->sg_entries[0]);
+
+ obj->sg_entries = kmalloc(sz, gfp_mask);
+ if (unlikely(obj->sg_entries == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of sgv_pool_obj "
+ "SG vector failed (size %d)", sz);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ sg_init_table(obj->sg_entries, pages_to_alloc);
+
+ if (sgv_pool_clustered(obj->owner_pool)) {
+ if (pages_to_alloc <= sgv_max_trans_pages) {
+ obj->trans_tbl =
+ (struct trans_tbl_ent *)obj->sg_entries_data;
+ /*
+ * No need to clear trans_tbl, if needed, it will be
+ * fully rewritten in sgv_alloc_sg_entries()
+ */
+ } else {
+ tsz = pages_to_alloc * sizeof(obj->trans_tbl[0]);
+ obj->trans_tbl = kzalloc(tsz, gfp_mask);
+ if (unlikely(obj->trans_tbl == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of "
+ "trans_tbl failed (size %d)", tsz);
+ res = -ENOMEM;
+ goto out_free;
+ }
+ }
+ }
+
+ TRACE_MEM("pages_to_alloc %d, sz %d, tsz %d, obj %p, sg_entries %p, "
+ "trans_tbl %p", pages_to_alloc, sz, tsz, obj, obj->sg_entries,
+ obj->trans_tbl);
+
+out:
+ return res;
+
+out_free:
+ kfree(obj->sg_entries);
+ obj->sg_entries = NULL;
+ goto out;
+}
+
+static struct sgv_pool_obj *sgv_get_obj(struct sgv_pool *pool, int cache_num,
+ int pages, gfp_t gfp_mask, bool get_new)
+{
+ struct sgv_pool_obj *obj;
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ if (unlikely(get_new)) {
+ /* Used only for buffers preallocation */
+ goto get_new;
+ }
+
+ if (likely(!list_empty(&pool->recycling_lists[cache_num]))) {
+ obj = list_entry(pool->recycling_lists[cache_num].next,
+ struct sgv_pool_obj, recycling_list_entry);
+
+ list_del(&obj->sorted_recycling_list_entry);
+ list_del(&obj->recycling_list_entry);
+
+ pool->inactive_cached_pages -= pages;
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ goto out;
+ }
+
+get_new:
+ if (pool->cached_entries == 0) {
+ TRACE_MEM("Adding pool %p to the active list", pool);
+ spin_lock_bh(&sgv_pools_lock);
+ list_add_tail(&pool->sgv_active_pools_list_entry,
+ &sgv_active_pools_list);
+ spin_unlock_bh(&sgv_pools_lock);
+ }
+
+ pool->cached_entries++;
+ pool->cached_pages += pages;
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ TRACE_MEM("New cached entries %d (pool %p)", pool->cached_entries,
+ pool);
+
+ obj = kmem_cache_alloc(pool->caches[cache_num],
+ gfp_mask & ~(__GFP_HIGHMEM|GFP_DMA));
+ if (likely(obj)) {
+ memset(obj, 0, sizeof(*obj));
+ obj->cache_num = cache_num;
+ obj->pages = pages;
+ obj->owner_pool = pool;
+ } else {
+ spin_lock_bh(&pool->sgv_pool_lock);
+ sgv_dec_cached_entries(pool, pages);
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ }
+
+out:
+ return obj;
+}
+
+static void sgv_put_obj(struct sgv_pool_obj *obj)
+{
+ struct sgv_pool *pool = obj->owner_pool;
+ struct list_head *entry;
+ struct list_head *list = &pool->recycling_lists[obj->cache_num];
+ int pages = obj->pages;
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ TRACE_MEM("sgv %p, cache num %d, pages %d, sg_count %d", obj,
+ obj->cache_num, pages, obj->sg_count);
+
+ if (sgv_pool_clustered(pool)) {
+ /* Make objects with less entries more preferred */
+ __list_for_each(entry, list) {
+ struct sgv_pool_obj *tmp = list_entry(entry,
+ struct sgv_pool_obj, recycling_list_entry);
+
+ TRACE_MEM("tmp %p, cache num %d, pages %d, sg_count %d",
+ tmp, tmp->cache_num, tmp->pages, tmp->sg_count);
+
+ if (obj->sg_count <= tmp->sg_count)
+ break;
+ }
+ entry = entry->prev;
+ } else
+ entry = list;
+
+ TRACE_MEM("Adding in %p (list %p)", entry, list);
+ list_add(&obj->recycling_list_entry, entry);
+
+ list_add_tail(&obj->sorted_recycling_list_entry,
+ &pool->sorted_recycling_list);
+
+ obj->time_stamp = jiffies;
+
+ pool->inactive_cached_pages += pages;
+
+ if (!pool->purge_work_scheduled) {
+ TRACE_MEM("Scheduling purge work for pool %p", pool);
+ pool->purge_work_scheduled = true;
+ schedule_delayed_work(&pool->sgv_purge_work,
+ pool->purge_interval);
+ }
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ return;
+}
+
+/* No locks */
+static int sgv_hiwmk_check(int pages_to_alloc)
+{
+ int res = 0;
+ int pages = pages_to_alloc;
+
+ pages += atomic_read(&sgv_pages_total);
+
+ if (unlikely(pages > sgv_hi_wmk)) {
+ pages -= sgv_hi_wmk;
+ atomic_inc(&sgv_releases_on_hiwmk);
+
+ pages = __sgv_shrink(pages, 0);
+ if (pages > 0) {
+ TRACE(TRACE_OUT_OF_MEM, "Requested amount of "
+ "memory (%d pages) for being executed "
+ "commands together with the already "
+ "allocated memory exceeds the allowed "
+ "maximum %d. Should you increase "
+ "scst_max_cmd_mem?", pages_to_alloc,
+ sgv_hi_wmk);
+ atomic_inc(&sgv_releases_on_hiwmk_failed);
+ res = -ENOMEM;
+ goto out_unlock;
+ }
+ }
+
+ atomic_add(pages_to_alloc, &sgv_pages_total);
+
+out_unlock:
+ TRACE_MEM("pages_to_alloc %d, new total %d", pages_to_alloc,
+ atomic_read(&sgv_pages_total));
+
+ return res;
+}
+
+/* No locks */
+static void sgv_hiwmk_uncheck(int pages)
+{
+ atomic_sub(pages, &sgv_pages_total);
+ TRACE_MEM("pages %d, new total %d", pages,
+ atomic_read(&sgv_pages_total));
+ return;
+}
+
+/* No locks */
+static bool sgv_check_allowed_mem(struct scst_mem_lim *mem_lim, int pages)
+{
+ int alloced;
+ bool res = true;
+
+ alloced = atomic_add_return(pages, &mem_lim->alloced_pages);
+ if (unlikely(alloced > mem_lim->max_allowed_pages)) {
+ TRACE(TRACE_OUT_OF_MEM, "Requested amount of memory "
+ "(%d pages) for being executed commands on a device "
+ "together with the already allocated memory exceeds "
+ "the allowed maximum %d. Should you increase "
+ "scst_max_dev_cmd_mem?", pages,
+ mem_lim->max_allowed_pages);
+ atomic_sub(pages, &mem_lim->alloced_pages);
+ res = false;
+ }
+
+ TRACE_MEM("mem_lim %p, pages %d, res %d, new alloced %d", mem_lim,
+ pages, res, atomic_read(&mem_lim->alloced_pages));
+
+ return res;
+}
+
+/* No locks */
+static void sgv_uncheck_allowed_mem(struct scst_mem_lim *mem_lim, int pages)
+{
+ atomic_sub(pages, &mem_lim->alloced_pages);
+
+ TRACE_MEM("mem_lim %p, pages %d, new alloced %d", mem_lim,
+ pages, atomic_read(&mem_lim->alloced_pages));
+ return;
+}
+
+/**
+ * sgv_pool_alloc - allocate an SG vector from the SGV pool
+ * @pool: the cache to alloc from
+ * @size: size of the resulting SG vector in bytes
+ * @gfp_mask: the allocation mask
+ * @flags: the allocation flags
+ * @count: the resulting count of SG entries in the resulting SG vector
+ * @sgv: the resulting SGV object
+ * @mem_lim: memory limits
+ * @priv: pointer to private for this allocation data
+ *
+ * Description:
+ * Allocate an SG vector from the SGV pool and returns pointer to it or
+ * NULL in case of any error. See the SGV pool documentation for more details.
+ */
+struct scatterlist *sgv_pool_alloc(struct sgv_pool *pool, unsigned int size,
+ gfp_t gfp_mask, int flags, int *count,
+ struct sgv_pool_obj **sgv, struct scst_mem_lim *mem_lim, void *priv)
+{
+ struct sgv_pool_obj *obj;
+ int cache_num, pages, cnt;
+ struct scatterlist *res = NULL;
+ int pages_to_alloc;
+ int no_cached = flags & SGV_POOL_ALLOC_NO_CACHED;
+ bool allowed_mem_checked = false, hiwmk_checked = false;
+
+ if (unlikely(size == 0))
+ goto out;
+
+ EXTRACHECKS_BUG_ON((gfp_mask & __GFP_NOFAIL) == __GFP_NOFAIL);
+
+ pages = ((size + PAGE_SIZE - 1) >> PAGE_SHIFT);
+ if (pool->single_alloc_pages == 0) {
+ int pages_order = get_order(size);
+ cache_num = pages_order;
+ pages_to_alloc = (1 << pages_order);
+ } else {
+ cache_num = 0;
+ pages_to_alloc = max(pool->single_alloc_pages, pages);
+ }
+
+ TRACE_MEM("size=%d, pages=%d, pages_to_alloc=%d, cache num=%d, "
+ "flags=%x, no_cached=%d, *sgv=%p", size, pages,
+ pages_to_alloc, cache_num, flags, no_cached, *sgv);
+
+ if (*sgv != NULL) {
+ obj = *sgv;
+
+ TRACE_MEM("Supplied obj %p, cache num %d", obj, obj->cache_num);
+
+ EXTRACHECKS_BUG_ON(obj->sg_count != 0);
+
+ if (unlikely(!sgv_check_allowed_mem(mem_lim, pages_to_alloc)))
+ goto out_fail_free_sg_entries;
+ allowed_mem_checked = true;
+
+ if (unlikely(sgv_hiwmk_check(pages_to_alloc) != 0))
+ goto out_fail_free_sg_entries;
+ hiwmk_checked = true;
+ } else if ((pages_to_alloc <= pool->max_cached_pages) && !no_cached) {
+ if (unlikely(!sgv_check_allowed_mem(mem_lim, pages_to_alloc)))
+ goto out_fail;
+ allowed_mem_checked = true;
+
+ obj = sgv_get_obj(pool, cache_num, pages_to_alloc, gfp_mask,
+ flags & SGV_POOL_ALLOC_GET_NEW);
+ if (unlikely(obj == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of "
+ "sgv_pool_obj failed (size %d)", size);
+ goto out_fail;
+ }
+
+ if (obj->sg_count != 0) {
+ TRACE_MEM("Cached obj %p", obj);
+ atomic_inc(&pool->cache_acc[cache_num].hit_alloc);
+ goto success;
+ }
+
+ if (flags & SGV_POOL_NO_ALLOC_ON_CACHE_MISS) {
+ if (!(flags & SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL))
+ goto out_fail_free;
+ }
+
+ TRACE_MEM("Brand new obj %p", obj);
+
+ if (pages_to_alloc <= sgv_max_local_pages) {
+ obj->sg_entries = obj->sg_entries_data;
+ sg_init_table(obj->sg_entries, pages_to_alloc);
+ TRACE_MEM("sg_entries %p", obj->sg_entries);
+ if (sgv_pool_clustered(pool)) {
+ obj->trans_tbl = (struct trans_tbl_ent *)
+ (obj->sg_entries + pages_to_alloc);
+ TRACE_MEM("trans_tbl %p", obj->trans_tbl);
+ /*
+ * No need to clear trans_tbl, if needed, it
+ * will be fully rewritten in
+ * sgv_alloc_sg_entries().
+ */
+ }
+ } else {
+ if (unlikely(sgv_alloc_arrays(obj, pages_to_alloc,
+ gfp_mask) != 0))
+ goto out_fail_free;
+ }
+
+ if ((flags & SGV_POOL_NO_ALLOC_ON_CACHE_MISS) &&
+ (flags & SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL))
+ goto out_return;
+
+ obj->allocator_priv = priv;
+
+ if (unlikely(sgv_hiwmk_check(pages_to_alloc) != 0))
+ goto out_fail_free_sg_entries;
+ hiwmk_checked = true;
+ } else {
+ int sz;
+
+ pages_to_alloc = pages;
+
+ if (unlikely(!sgv_check_allowed_mem(mem_lim, pages_to_alloc)))
+ goto out_fail;
+ allowed_mem_checked = true;
+
+ if (flags & SGV_POOL_NO_ALLOC_ON_CACHE_MISS)
+ goto out_return2;
+
+ sz = sizeof(*obj) + pages * sizeof(obj->sg_entries[0]);
+
+ obj = kmalloc(sz, gfp_mask);
+ if (unlikely(obj == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of "
+ "sgv_pool_obj failed (size %d)", size);
+ goto out_fail;
+ }
+ memset(obj, 0, sizeof(*obj));
+
+ obj->owner_pool = pool;
+ cache_num = -1;
+ obj->cache_num = cache_num;
+ obj->pages = pages_to_alloc;
+ obj->allocator_priv = priv;
+
+ obj->sg_entries = obj->sg_entries_data;
+ sg_init_table(obj->sg_entries, pages);
+
+ if (unlikely(sgv_hiwmk_check(pages_to_alloc) != 0))
+ goto out_fail_free_sg_entries;
+ hiwmk_checked = true;
+
+ TRACE_MEM("Big or no_cached obj %p (size %d)", obj, sz);
+ }
+
+ obj->sg_count = sgv_alloc_sg_entries(obj->sg_entries,
+ pages_to_alloc, gfp_mask, pool->clustering_type,
+ obj->trans_tbl, &pool->alloc_fns, priv);
+ if (unlikely(obj->sg_count <= 0)) {
+ obj->sg_count = 0;
+ if ((flags & SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL) &&
+ (cache_num >= 0))
+ goto out_return1;
+ else
+ goto out_fail_free_sg_entries;
+ }
+
+ if (cache_num >= 0) {
+ atomic_add(pages_to_alloc - obj->sg_count,
+ &pool->cache_acc[cache_num].merged);
+ } else {
+ if (no_cached) {
+ atomic_add(pages_to_alloc,
+ &pool->other_pages);
+ atomic_add(pages_to_alloc - obj->sg_count,
+ &pool->other_merged);
+ } else {
+ atomic_add(pages_to_alloc,
+ &pool->big_pages);
+ atomic_add(pages_to_alloc - obj->sg_count,
+ &pool->big_merged);
+ }
+ }
+
+success:
+ if (cache_num >= 0) {
+ int sg;
+ atomic_inc(&pool->cache_acc[cache_num].total_alloc);
+ if (sgv_pool_clustered(pool))
+ cnt = obj->trans_tbl[pages-1].sg_num;
+ else
+ cnt = pages;
+ sg = cnt-1;
+ obj->orig_sg = sg;
+ obj->orig_length = obj->sg_entries[sg].length;
+ if (sgv_pool_clustered(pool)) {
+ obj->sg_entries[sg].length =
+ (pages - obj->trans_tbl[sg].pg_count) << PAGE_SHIFT;
+ }
+ } else {
+ cnt = obj->sg_count;
+ if (no_cached)
+ atomic_inc(&pool->other_alloc);
+ else
+ atomic_inc(&pool->big_alloc);
+ }
+
+ *count = cnt;
+ res = obj->sg_entries;
+ *sgv = obj;
+
+ if (size & ~PAGE_MASK)
+ obj->sg_entries[cnt-1].length -=
+ PAGE_SIZE - (size & ~PAGE_MASK);
+
+ TRACE_MEM("obj=%p, sg_entries %p (size=%d, pages=%d, sg_count=%d, "
+ "count=%d, last_len=%d)", obj, obj->sg_entries, size, pages,
+ obj->sg_count, *count, obj->sg_entries[obj->orig_sg].length);
+
+out:
+ return res;
+
+out_return:
+ obj->allocator_priv = priv;
+ obj->owner_pool = pool;
+
+out_return1:
+ *sgv = obj;
+ TRACE_MEM("Returning failed obj %p (count %d)", obj, *count);
+
+out_return2:
+ *count = pages_to_alloc;
+ res = NULL;
+ goto out_uncheck;
+
+out_fail_free_sg_entries:
+ if (obj->sg_entries != obj->sg_entries_data) {
+ if (obj->trans_tbl !=
+ (struct trans_tbl_ent *)obj->sg_entries_data) {
+ /* kfree() handles NULL parameter */
+ kfree(obj->trans_tbl);
+ obj->trans_tbl = NULL;
+ }
+ kfree(obj->sg_entries);
+ obj->sg_entries = NULL;
+ }
+
+out_fail_free:
+ if (cache_num >= 0) {
+ spin_lock_bh(&pool->sgv_pool_lock);
+ sgv_dec_cached_entries(pool, pages_to_alloc);
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ kmem_cache_free(pool->caches[obj->cache_num], obj);
+ } else
+ kfree(obj);
+
+out_fail:
+ res = NULL;
+ *count = 0;
+ *sgv = NULL;
+ TRACE_MEM("%s", "Allocation failed");
+
+out_uncheck:
+ if (hiwmk_checked)
+ sgv_hiwmk_uncheck(pages_to_alloc);
+ if (allowed_mem_checked)
+ sgv_uncheck_allowed_mem(mem_lim, pages_to_alloc);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_alloc);
+
+/**
+ * sgv_get_priv - return the private allocation data
+ *
+ * Allows to get the allocation private data for this SGV
+ * cache object. The private data supposed to be set by sgv_pool_alloc().
+ */
+void *sgv_get_priv(struct sgv_pool_obj *obj)
+{
+ return obj->allocator_priv;
+}
+EXPORT_SYMBOL_GPL(sgv_get_priv);
+
+/**
+ * sgv_pool_free - free previously allocated SG vector
+ * @sgv: the SGV object to free
+ * @mem_lim: memory limits
+ *
+ * Description:
+ * Frees previously allocated SG vector and updates memory limits
+ */
+void sgv_pool_free(struct sgv_pool_obj *obj, struct scst_mem_lim *mem_lim)
+{
+ int pages = (obj->sg_count != 0) ? obj->pages : 0;
+
+ TRACE_MEM("Freeing obj %p, cache num %d, pages %d, sg_entries %p, "
+ "sg_count %d, allocator_priv %p", obj, obj->cache_num, pages,
+ obj->sg_entries, obj->sg_count, obj->allocator_priv);
+
+/*
+ * Enable it if you are investigating a data corruption and want to make
+ * sure that target or dev handler didn't leave the pages mapped somewhere and,
+ * hence, provoked a data corruption.
+ *
+ * Make sure the check value for _count is set correctly. In most cases, 1 is
+ * correct, but, e.g., iSCSI-SCST can call it with value 2, because
+ * it frees the corresponding cmd before the last put_page() call from
+ * net_put_page() for the last page in the SG. Also, user space dev handlers
+ * usually have their memory mapped in their address space.
+ */
+#if 0
+ {
+ struct scatterlist *sg = obj->sg_entries;
+ int i;
+ for (i = 0; i < obj->sg_count; i++) {
+ struct page *p = sg_page(&sg[i]);
+ int len = sg[i].length;
+ int pages = (len >> PAGE_SHIFT) + ((len & ~PAGE_MASK) != 0);
+ while (pages > 0) {
+ if (atomic_read(&p->_count) != 1) {
+ PRINT_WARNING("Freeing page %p with "
+ "additional owners (_count %d). "
+ "Data corruption possible!",
+ p, atomic_read(&p->_count));
+ WARN_ON(1);
+ }
+ pages--;
+ p++;
+ }
+ }
+ }
+#endif
+
+ if (obj->cache_num >= 0) {
+ obj->sg_entries[obj->orig_sg].length = obj->orig_length;
+ sgv_put_obj(obj);
+ } else {
+ obj->owner_pool->alloc_fns.free_pages_fn(obj->sg_entries,
+ obj->sg_count, obj->allocator_priv);
+ kfree(obj);
+ sgv_hiwmk_uncheck(pages);
+ }
+
+ sgv_uncheck_allowed_mem(mem_lim, pages);
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_free);
+
+/**
+ * scst_alloc() - allocates an SG vector
+ *
+ * Allocates and returns pointer to SG vector with data size "size".
+ * In *count returned the count of entries in the vector.
+ * Returns NULL for failure.
+ */
+struct scatterlist *scst_alloc(int size, gfp_t gfp_mask, int *count)
+{
+ struct scatterlist *res;
+ int pages = (size >> PAGE_SHIFT) + ((size & ~PAGE_MASK) != 0);
+ struct sgv_pool_alloc_fns sys_alloc_fns = {
+ sgv_alloc_sys_pages, sgv_free_sys_sg_entries };
+ int no_fail = ((gfp_mask & __GFP_NOFAIL) == __GFP_NOFAIL);
+
+ atomic_inc(&sgv_other_total_alloc);
+
+ if (unlikely(sgv_hiwmk_check(pages) != 0)) {
+ if (!no_fail) {
+ res = NULL;
+ goto out;
+ } else {
+ /*
+ * Update active_pages_total since alloc can't fail.
+ * If it wasn't updated then the counter would cross 0
+ * on free again.
+ */
+ sgv_hiwmk_uncheck(-pages);
+ }
+ }
+
+ res = kmalloc(pages*sizeof(*res), gfp_mask);
+ if (res == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Unable to allocate sg for %d pages",
+ pages);
+ goto out_uncheck;
+ }
+
+ sg_init_table(res, pages);
+
+ /*
+ * If we allow use clustering here, we will have troubles in
+ * scst_free() to figure out how many pages are in the SG vector.
+ * So, always don't use clustering.
+ */
+ *count = sgv_alloc_sg_entries(res, pages, gfp_mask, sgv_no_clustering,
+ NULL, &sys_alloc_fns, NULL);
+ if (*count <= 0)
+ goto out_free;
+
+out:
+ TRACE_MEM("Alloced sg %p (count %d) \"no fail\" %d", res, *count, no_fail);
+ return res;
+
+out_free:
+ kfree(res);
+ res = NULL;
+
+out_uncheck:
+ if (!no_fail)
+ sgv_hiwmk_uncheck(pages);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_alloc);
+
+/**
+ * scst_free() - frees SG vector
+ *
+ * Frees SG vector returned by scst_alloc().
+ */
+void scst_free(struct scatterlist *sg, int count)
+{
+ TRACE_MEM("Freeing sg=%p", sg);
+
+ sgv_hiwmk_uncheck(count);
+
+ sgv_free_sys_sg_entries(sg, count, NULL);
+ kfree(sg);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_free);
+
+/* Must be called under sgv_pools_mutex */
+static void sgv_pool_init_cache(struct sgv_pool *pool, int cache_num)
+{
+ int size;
+ int pages;
+ struct sgv_pool_obj *obj;
+
+ atomic_set(&pool->cache_acc[cache_num].total_alloc, 0);
+ atomic_set(&pool->cache_acc[cache_num].hit_alloc, 0);
+ atomic_set(&pool->cache_acc[cache_num].merged, 0);
+
+ if (pool->single_alloc_pages == 0)
+ pages = 1 << cache_num;
+ else
+ pages = pool->single_alloc_pages;
+
+ if (pages <= sgv_max_local_pages) {
+ size = sizeof(*obj) + pages *
+ (sizeof(obj->sg_entries[0]) +
+ ((pool->clustering_type != sgv_no_clustering) ?
+ sizeof(obj->trans_tbl[0]) : 0));
+ } else if (pages <= sgv_max_trans_pages) {
+ /*
+ * sg_entries is allocated outside object,
+ * but trans_tbl is still embedded.
+ */
+ size = sizeof(*obj) + pages *
+ (((pool->clustering_type != sgv_no_clustering) ?
+ sizeof(obj->trans_tbl[0]) : 0));
+ } else {
+ size = sizeof(*obj);
+ /* both sgv and trans_tbl are kmalloc'ed() */
+ }
+
+ TRACE_MEM("pages=%d, size=%d", pages, size);
+
+ scnprintf(pool->cache_names[cache_num],
+ sizeof(pool->cache_names[cache_num]),
+ "%s-%uK", pool->name, (pages << PAGE_SHIFT) >> 10);
+ pool->caches[cache_num] = kmem_cache_create(
+ pool->cache_names[cache_num], size, 0, SCST_SLAB_FLAGS, NULL
+ );
+ return;
+}
+
+/* Must be called under sgv_pools_mutex */
+static int sgv_pool_init(struct sgv_pool *pool, const char *name,
+ enum sgv_clustering_types clustering_type, int single_alloc_pages,
+ int purge_interval)
+{
+ int res = -ENOMEM;
+ int i;
+
+ if (single_alloc_pages < 0) {
+ PRINT_ERROR("Wrong single_alloc_pages value %d",
+ single_alloc_pages);
+ res = -EINVAL;
+ goto out;
+ }
+
+ memset(pool, 0, sizeof(*pool));
+
+ atomic_set(&pool->big_alloc, 0);
+ atomic_set(&pool->big_pages, 0);
+ atomic_set(&pool->big_merged, 0);
+ atomic_set(&pool->other_alloc, 0);
+ atomic_set(&pool->other_pages, 0);
+ atomic_set(&pool->other_merged, 0);
+
+ pool->clustering_type = clustering_type;
+ pool->single_alloc_pages = single_alloc_pages;
+ if (purge_interval != 0) {
+ pool->purge_interval = purge_interval;
+ if (purge_interval < 0) {
+ /* Let's pretend that it's always scheduled */
+ pool->purge_work_scheduled = 1;
+ }
+ } else
+ pool->purge_interval = SGV_DEFAULT_PURGE_INTERVAL;
+ if (single_alloc_pages == 0) {
+ pool->max_caches = SGV_POOL_ELEMENTS;
+ pool->max_cached_pages = 1 << (SGV_POOL_ELEMENTS - 1);
+ } else {
+ pool->max_caches = 1;
+ pool->max_cached_pages = single_alloc_pages;
+ }
+ pool->alloc_fns.alloc_pages_fn = sgv_alloc_sys_pages;
+ pool->alloc_fns.free_pages_fn = sgv_free_sys_sg_entries;
+
+ TRACE_MEM("name %s, sizeof(*obj)=%zd, clustering_type=%d, "
+ "single_alloc_pages=%d, max_caches=%d, max_cached_pages=%d",
+ name, sizeof(struct sgv_pool_obj), clustering_type,
+ single_alloc_pages, pool->max_caches, pool->max_cached_pages);
+
+ strlcpy(pool->name, name, sizeof(pool->name)-1);
+
+ pool->owner_mm = current->mm;
+
+ for (i = 0; i < pool->max_caches; i++) {
+ sgv_pool_init_cache(pool, i);
+ if (pool->caches[i] == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of sgv_pool "
+ "cache %s(%d) failed", name, i);
+ goto out_free;
+ }
+ }
+
+ atomic_set(&pool->sgv_pool_ref, 1);
+ spin_lock_init(&pool->sgv_pool_lock);
+ INIT_LIST_HEAD(&pool->sorted_recycling_list);
+ for (i = 0; i < pool->max_caches; i++)
+ INIT_LIST_HEAD(&pool->recycling_lists[i]);
+
+ INIT_DELAYED_WORK(&pool->sgv_purge_work,
+ (void (*)(struct work_struct *))sgv_purge_work_fn);
+
+ spin_lock_bh(&sgv_pools_lock);
+ list_add_tail(&pool->sgv_pools_list_entry, &sgv_pools_list);
+ spin_unlock_bh(&sgv_pools_lock);
+
+ res = scst_sgv_sysfs_create(pool);
+ if (res != 0)
+ goto out_del;
+
+ res = 0;
+
+out:
+ return res;
+
+out_del:
+ spin_lock_bh(&sgv_pools_lock);
+ list_del(&pool->sgv_pools_list_entry);
+ spin_unlock_bh(&sgv_pools_lock);
+
+out_free:
+ for (i = 0; i < pool->max_caches; i++) {
+ if (pool->caches[i]) {
+ kmem_cache_destroy(pool->caches[i]);
+ pool->caches[i] = NULL;
+ } else
+ break;
+ }
+ goto out;
+}
+
+static void sgv_evaluate_local_max_pages(void)
+{
+ int space4sgv_ttbl = PAGE_SIZE - sizeof(struct sgv_pool_obj);
+
+ sgv_max_local_pages = space4sgv_ttbl /
+ (sizeof(struct trans_tbl_ent) + sizeof(struct scatterlist));
+
+ sgv_max_trans_pages = space4sgv_ttbl / sizeof(struct trans_tbl_ent);
+
+ TRACE_MEM("sgv_max_local_pages %d, sgv_max_trans_pages %d",
+ sgv_max_local_pages, sgv_max_trans_pages);
+ return;
+}
+
+/**
+ * sgv_pool_flush - flushe the SGV pool
+ *
+ * Flushes, i.e. frees, all the cached entries in the SGV pool.
+ */
+void sgv_pool_flush(struct sgv_pool *pool)
+{
+ int i;
+
+ for (i = 0; i < pool->max_caches; i++) {
+ struct sgv_pool_obj *obj;
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ while (!list_empty(&pool->recycling_lists[i])) {
+ obj = list_entry(pool->recycling_lists[i].next,
+ struct sgv_pool_obj, recycling_list_entry);
+
+ __sgv_purge_from_cache(obj);
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ EXTRACHECKS_BUG_ON(obj->owner_pool != pool);
+ sgv_dtor_and_free(obj);
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+ }
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ }
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_flush);
+
+static void sgv_pool_destroy(struct sgv_pool *pool)
+{
+ int i;
+
+ cancel_delayed_work_sync(&pool->sgv_purge_work);
+
+ sgv_pool_flush(pool);
+
+ mutex_lock(&sgv_pools_mutex);
+ spin_lock_bh(&sgv_pools_lock);
+ list_del(&pool->sgv_pools_list_entry);
+ spin_unlock_bh(&sgv_pools_lock);
+ mutex_unlock(&sgv_pools_mutex);
+
+ scst_sgv_sysfs_del(pool);
+
+ for (i = 0; i < pool->max_caches; i++) {
+ if (pool->caches[i])
+ kmem_cache_destroy(pool->caches[i]);
+ pool->caches[i] = NULL;
+ }
+
+ kfree(pool);
+ return;
+}
+
+/**
+ * sgv_pool_set_allocator - set custom pages allocator
+ * @pool: the cache
+ * @alloc_pages_fn: pages allocation function
+ * @free_pages_fn: pages freeing function
+ *
+ * Description:
+ * Allows to set custom pages allocator for the SGV pool.
+ * See the SGV pool documentation for more details.
+ */
+void sgv_pool_set_allocator(struct sgv_pool *pool,
+ struct page *(*alloc_pages_fn)(struct scatterlist *, gfp_t, void *),
+ void (*free_pages_fn)(struct scatterlist *, int, void *))
+{
+ pool->alloc_fns.alloc_pages_fn = alloc_pages_fn;
+ pool->alloc_fns.free_pages_fn = free_pages_fn;
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_set_allocator);
+
+/**
+ * sgv_pool_create - creates and initializes an SGV pool
+ * @name: the name of the SGV pool
+ * @clustered: sets type of the pages clustering.
+ * @single_alloc_pages: if 0, then the SGV pool will work in the set of
+ * power 2 size buffers mode. If >0, then the SGV pool will
+ * work in the fixed size buffers mode. In this case
+ * single_alloc_pages sets the size of each buffer in pages.
+ * @shared: sets if the SGV pool can be shared between devices or not.
+ * The cache sharing allowed only between devices created inside
+ * the same address space. If an SGV pool is shared, each
+ * subsequent call of sgv_pool_create() with the same cache name
+ * will not create a new cache, but instead return a reference
+ * to it.
+ * @purge_interval: sets the cache purging interval. I.e., an SG buffer
+ * will be freed if it's unused for time t
+ * purge_interval <= t < 2*purge_interval. If purge_interval
+ * is 0, then the default interval will be used (60 seconds).
+ * If purge_interval <0, then the automatic purging will be
+ * disabled.
+ *
+ * Description:
+ * Returns the resulting SGV pool or NULL in case of any error.
+ */
+struct sgv_pool *sgv_pool_create(const char *name,
+ enum sgv_clustering_types clustering_type,
+ int single_alloc_pages, bool shared, int purge_interval)
+{
+ struct sgv_pool *pool;
+ int rc;
+
+ mutex_lock(&sgv_pools_mutex);
+
+ list_for_each_entry(pool, &sgv_pools_list, sgv_pools_list_entry) {
+ if (strcmp(pool->name, name) == 0) {
+ if (shared) {
+ if (pool->owner_mm != current->mm) {
+ PRINT_ERROR("Attempt of a shared use "
+ "of SGV pool %s with "
+ "different MM", name);
+ goto out_unlock;
+ }
+ sgv_pool_get(pool);
+ goto out_unlock;
+ } else {
+ PRINT_ERROR("SGV pool %s already exists", name);
+ pool = NULL;
+ goto out_unlock;
+ }
+ }
+ }
+
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (pool == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of sgv_pool failed");
+ goto out_unlock;
+ }
+
+ rc = sgv_pool_init(pool, name, clustering_type, single_alloc_pages,
+ purge_interval);
+ if (rc != 0)
+ goto out_free;
+
+out_unlock:
+ mutex_unlock(&sgv_pools_mutex);
+ return pool;
+
+out_free:
+ kfree(pool);
+ goto out_unlock;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_create);
+
+/**
+ * sgv_pool_get - increase ref counter for the corresponding SGV pool
+ *
+ * Increases ref counter for the corresponding SGV pool
+ */
+void sgv_pool_get(struct sgv_pool *pool)
+{
+ atomic_inc(&pool->sgv_pool_ref);
+ TRACE_MEM("Incrementing sgv pool %p ref (new value %d)",
+ pool, atomic_read(&pool->sgv_pool_ref));
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_get);
+
+/**
+ * sgv_pool_put - decrease ref counter for the corresponding SGV pool
+ *
+ * Decreases ref counter for the corresponding SGV pool. If the ref
+ * counter reaches 0, the cache will be destroyed.
+ */
+void sgv_pool_put(struct sgv_pool *pool)
+{
+ TRACE_MEM("Decrementing sgv pool %p ref (new value %d)",
+ pool, atomic_read(&pool->sgv_pool_ref)-1);
+ if (atomic_dec_and_test(&pool->sgv_pool_ref))
+ sgv_pool_destroy(pool);
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_put);
+
+/**
+ * sgv_pool_del - deletes the corresponding SGV pool
+ * @pool: the cache to delete.
+ *
+ * Description:
+ * If the cache is shared, it will decrease its reference counter.
+ * If the reference counter reaches 0, the cache will be destroyed.
+ */
+void sgv_pool_del(struct sgv_pool *pool)
+{
+
+ sgv_pool_put(pool);
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_del);
+
+/* Both parameters in pages */
+int scst_sgv_pools_init(unsigned long mem_hwmark, unsigned long mem_lwmark)
+{
+ int res = 0;
+
+ sgv_hi_wmk = mem_hwmark;
+ sgv_lo_wmk = mem_lwmark;
+
+ sgv_evaluate_local_max_pages();
+
+ sgv_norm_pool = sgv_pool_create("sgv", sgv_no_clustering, 0, false, 0);
+ if (sgv_norm_pool == NULL)
+ goto out_err;
+
+ sgv_norm_clust_pool = sgv_pool_create("sgv-clust",
+ sgv_full_clustering, 0, false, 0);
+ if (sgv_norm_clust_pool == NULL)
+ goto out_free_norm;
+
+ sgv_dma_pool = sgv_pool_create("sgv-dma", sgv_no_clustering, 0,
+ false, 0);
+ if (sgv_dma_pool == NULL)
+ goto out_free_clust;
+
+ sgv_shrinker.shrink = sgv_shrink;
+ sgv_shrinker.seeks = DEFAULT_SEEKS;
+ register_shrinker(&sgv_shrinker);
+
+out:
+ return res;
+
+out_free_clust:
+ sgv_pool_destroy(sgv_norm_clust_pool);
+
+out_free_norm:
+ sgv_pool_destroy(sgv_norm_pool);
+
+out_err:
+ res = -ENOMEM;
+ goto out;
+}
+
+void scst_sgv_pools_deinit(void)
+{
+
+ unregister_shrinker(&sgv_shrinker);
+
+ sgv_pool_destroy(sgv_dma_pool);
+ sgv_pool_destroy(sgv_norm_pool);
+ sgv_pool_destroy(sgv_norm_clust_pool);
+
+ flush_scheduled_work();
+ return;
+}
+
+ssize_t sgv_sysfs_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct sgv_pool *pool;
+ int i, total = 0, hit = 0, merged = 0, allocated = 0;
+ int oa, om, res;
+
+ pool = container_of(kobj, struct sgv_pool, sgv_kobj);
+
+ for (i = 0; i < SGV_POOL_ELEMENTS; i++) {
+ int t;
+
+ hit += atomic_read(&pool->cache_acc[i].hit_alloc);
+ total += atomic_read(&pool->cache_acc[i].total_alloc);
+
+ t = atomic_read(&pool->cache_acc[i].total_alloc) -
+ atomic_read(&pool->cache_acc[i].hit_alloc);
+ allocated += t * (1 << i);
+ merged += atomic_read(&pool->cache_acc[i].merged);
+ }
+
+ res = sprintf(buf, "%-30s %-11s %-11s %-11s %-11s", "Name", "Hit", "Total",
+ "% merged", "Cached (P/I/O)");
+
+ res += sprintf(&buf[res], "\n%-30s %-11d %-11d %-11d %d/%d/%d\n",
+ pool->name, hit, total,
+ (allocated != 0) ? merged*100/allocated : 0,
+ pool->cached_pages, pool->inactive_cached_pages,
+ pool->cached_entries);
+
+ for (i = 0; i < SGV_POOL_ELEMENTS; i++) {
+ int t = atomic_read(&pool->cache_acc[i].total_alloc) -
+ atomic_read(&pool->cache_acc[i].hit_alloc);
+ allocated = t * (1 << i);
+ merged = atomic_read(&pool->cache_acc[i].merged);
+
+ res += sprintf(&buf[res], " %-28s %-11d %-11d %d\n",
+ pool->cache_names[i],
+ atomic_read(&pool->cache_acc[i].hit_alloc),
+ atomic_read(&pool->cache_acc[i].total_alloc),
+ (allocated != 0) ? merged*100/allocated : 0);
+ }
+
+ allocated = atomic_read(&pool->big_pages);
+ merged = atomic_read(&pool->big_merged);
+ oa = atomic_read(&pool->other_pages);
+ om = atomic_read(&pool->other_merged);
+
+ res += sprintf(&buf[res], " %-40s %d/%-9d %d/%d\n", "big/other",
+ atomic_read(&pool->big_alloc), atomic_read(&pool->other_alloc),
+ (allocated != 0) ? merged*100/allocated : 0,
+ (oa != 0) ? om/oa : 0);
+
+ return res;
+}
+
+ssize_t sgv_sysfs_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ struct sgv_pool *pool;
+ int i;
+
+ pool = container_of(kobj, struct sgv_pool, sgv_kobj);
+
+ for (i = 0; i < SGV_POOL_ELEMENTS; i++) {
+ atomic_set(&pool->cache_acc[i].hit_alloc, 0);
+ atomic_set(&pool->cache_acc[i].total_alloc, 0);
+ atomic_set(&pool->cache_acc[i].merged, 0);
+ }
+
+ atomic_set(&pool->big_pages, 0);
+ atomic_set(&pool->big_merged, 0);
+ atomic_set(&pool->big_alloc, 0);
+ atomic_set(&pool->other_pages, 0);
+ atomic_set(&pool->other_merged, 0);
+ atomic_set(&pool->other_alloc, 0);
+
+ PRINT_INFO("Statistics for SGV pool %s resetted", pool->name);
+ return count;
+}
+
+ssize_t sgv_sysfs_global_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct sgv_pool *pool;
+ int inactive_pages = 0, res;
+
+ spin_lock_bh(&sgv_pools_lock);
+ list_for_each_entry(pool, &sgv_active_pools_list,
+ sgv_active_pools_list_entry) {
+ inactive_pages += pool->inactive_cached_pages;
+ }
+ spin_unlock_bh(&sgv_pools_lock);
+
+ res = sprintf(buf, "%-42s %d/%d\n%-42s %d/%d\n%-42s %d/%d\n"
+ "%-42s %-11d\n",
+ "Inactive/active pages", inactive_pages,
+ atomic_read(&sgv_pages_total) - inactive_pages,
+ "Hi/lo watermarks [pages]", sgv_hi_wmk, sgv_lo_wmk,
+ "Hi watermark releases/failures",
+ atomic_read(&sgv_releases_on_hiwmk),
+ atomic_read(&sgv_releases_on_hiwmk_failed),
+ "Other allocs", atomic_read(&sgv_other_total_alloc));
+ return res;
+}
+
+ssize_t sgv_sysfs_global_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+
+ atomic_set(&sgv_releases_on_hiwmk, 0);
+ atomic_set(&sgv_releases_on_hiwmk_failed, 0);
+ atomic_set(&sgv_other_total_alloc, 0);
+
+ PRINT_INFO("%s", "Global SGV pool statistics resetted");
+ return count;
+}
+
diff -uprN orig/linux-2.6.35/Documentation/scst/sgv_cache.txt linux-2.6.35/Documentation/scst/sgv_cache.txt
--- orig/linux-2.6.35/Documentation/scst/sgv_cache.txt
+++ linux-2.6.35/Documentation/scst/sgv_cache.txt
@@ -0,0 +1,224 @@
+ SCST SGV CACHE.
+
+ PROGRAMMING INTERFACE DESCRIPTION.
+
+ For SCST version 1.0.2
+
+SCST SGV cache is a memory management subsystem in SCST. One can call it
+a "memory pool", but Linux kernel already have a mempool interface,
+which serves different purposes. SGV cache provides to SCST core, target
+drivers and backend dev handlers facilities to allocate, build and cache
+SG vectors for data buffers. The main advantage of it is the caching
+facility, when it doesn't free to the system each vector, which is not
+used anymore, but keeps it for a while (possibly indefinitely) to let it
+be reused by the next consecutive command. This allows to:
+
+ - Reduce commands processing latencies and, hence, improve performance;
+
+ - Make commands processing latencies predictable, which is essential
+ for RT applications.
+
+The freed SG vectors are kept by the SGV cache either for some (possibly
+indefinite) time, or, optionally, until the system needs more memory and
+asks to free some using the set_shrinker() interface. Also the SGV cache
+allows to:
+
+ - Cluster pages together. "Cluster" means merging adjacent pages in a
+single SG entry. It allows to have less SG entries in the resulting SG
+vector, hence improve performance handling it as well as allow to
+work with bigger buffers on hardware with limited SG capabilities.
+
+ - Set custom page allocator functions. For instance, scst_user device
+handler uses this facility to eliminate unneeded mapping/unmapping of
+user space pages and avoid unneeded IOCTL calls for buffers allocations.
+In fileio_tgt application, which uses a regular malloc() function to
+allocate data buffers, this facility allows ~30% less CPU load and
+considerable performance increase.
+
+ - Prevent each initiator or all initiators altogether to allocate too
+much memory and DoS the target. Consider 10 initiators, which can have
+access to 10 devices each. Any of them can queue up to 64 commands, each
+can transfer up to 1MB of data. So, all of them in a peak can allocate
+up to 10*10*64 = ~6.5GB of memory for data buffers. This amount must be
+limited somehow and the SGV cache performs this function.
+
+From implementation POV the SGV cache is a simple extension of the kmem
+cache. It can work in 2 modes:
+
+1. With fixed size buffers.
+
+2. With a set of power 2 size buffers. In this mode each SGV cache
+(struct sgv_pool) has SGV_POOL_ELEMENTS (11 currently) of kmem caches.
+Each of those kmem caches keeps SGV cache objects (struct sgv_pool_obj)
+corresponding to SG vectors with size of order X pages. For instance,
+request to allocate 4 pages will be served from kmem cache[2], since the
+order of the of number of requested pages is 2. If later request to
+allocate 11KB comes, the same SG vector with 4 pages will be reused (see
+below). This mode is in average allows less memory overhead comparing
+with the fixed size buffers mode.
+
+Consider how the SGV cache works in the set of buffers mode. When a
+request to allocate new SG vector comes, sgv_pool_alloc() via
+sgv_get_obj() checks if there is already a cached vector with that
+order. If yes, then that vector will be reused and its length, if
+necessary, will be modified to match the requested size. In the above
+example request for 11KB buffer, 4 pages vector will be reused and
+modified using trans_tbl to contain 3 pages and the last entry will be
+modified to contain the requested length - 2*PAGE_SIZE. If there is no
+cached object, then a new sgv_pool_obj will be allocated from the
+corresponding kmem cache, chosen by the order of number of requested
+pages. Then that vector will be filled by pages and returned.
+
+In the fixed size buffers mode the SGV cache works similarly, except
+that it always allocate buffer with the predefined fixed size. I.e.
+even for 4K request the whole buffer with predefined size, say, 1MB,
+will be used.
+
+In both modes, if size of a request exceeds the maximum allowed for
+caching buffer size, the requested buffer will be allocated, but not
+cached.
+
+Freed cached sgv_pool_obj objects are actually freed to the system
+either by the purge work, which is scheduled once in 60 seconds, or in
+sgv_shrink() called by system, when it's asking for memory.
+
+ Interface.
+
+struct sgv_pool *sgv_pool_create(const char *name,
+ enum sgv_clustering_types clustered, int single_alloc_pages,
+ bool shared, int purge_interval)
+
+This function creates and initializes an SGV cache. It has the following
+arguments:
+
+ - name - the name of the SGV cache
+
+ - clustered - sets type of the pages clustering. The type can be:
+
+ * sgv_no_clustering - no clustering performed.
+
+ * sgv_tail_clustering - a page will only be merged with the latest
+ previously allocated page, so the order of pages in the SG will be
+ preserved
+
+ * sgv_full_clustering - free merging of pages at any place in
+ the SG is allowed. This mode usually provides the best merging
+ rate.
+
+ - single_alloc_pages - if 0, then the SGV cache will work in the set of
+ power 2 size buffers mode. If >0, then the SGV cache will work in the
+ fixed size buffers mode. In this case single_alloc_pages sets the
+ size of each buffer in pages.
+
+ - shared - sets if the SGV cache can be shared between devices or not.
+ The cache sharing allowed only between devices created inside the same
+ address space. If an SGV cache is shared, each subsequent call of
+ sgv_pool_create() with the same cache name will not create a new cache,
+ but instead return a reference to it.
+
+ - purge_interval - sets the cache purging interval. I.e. an SG buffer
+ will be freed if it's unused for time t purge_interval <= t <
+ 2*purge_interval. If purge_interval is 0, then the default interval
+ will be used (60 seconds). If purge_interval <0, then the automatic
+ purging will be disabled. Shrinking by the system's demand will also
+ be disabled.
+
+Returns the resulting SGV cache or NULL in case of any error.
+
+void sgv_pool_del(struct sgv_pool *pool)
+
+This function deletes the corresponding SGV cache. If the cache is
+shared, it will decrease its reference counter. If the reference counter
+reaches 0, the cache will be destroyed.
+
+void sgv_pool_flush(struct sgv_pool *pool)
+
+This function flushes, i.e. frees, all the cached entries in the SGV
+cache.
+
+void sgv_pool_set_allocator(struct sgv_pool *pool,
+ struct page *(*alloc_pages_fn)(struct scatterlist *sg, gfp_t gfp, void *priv),
+ void (*free_pages_fn)(struct scatterlist *sg, int sg_count, void *priv));
+
+This function allows to set for the SGV cache a custom pages allocator. For
+instance, scst_user uses such function to supply to the cache mapped from
+user space pages.
+
+alloc_pages_fn() has the following parameters:
+
+ - sg - SG entry, to which the allocated page should be added.
+
+ - gfp - the allocation GFP flags
+
+ - priv - pointer to a private data supplied to sgv_pool_alloc()
+
+This function should return the allocated page or NULL, if no page was
+allocated.
+
+free_pages_fn() has the following parameters:
+
+ - sg - SG vector to free
+
+ - sg_count - number of SG entries in the sg
+
+ - priv - pointer to a private data supplied to the corresponding sgv_pool_alloc()
+
+struct scatterlist *sgv_pool_alloc(struct sgv_pool *pool, unsigned int size,
+ gfp_t gfp_mask, int flags, int *count,
+ struct sgv_pool_obj **sgv, struct scst_mem_lim *mem_lim, void *priv)
+
+This function allocates an SG vector from the SGV cache. It has the
+following parameters:
+
+ - pool - the cache to alloc from
+
+ - size - size of the resulting SG vector in bytes
+
+ - gfp_mask - the allocation mask
+
+ - flags - the allocation flags. The following flags are possible and
+ can be set using OR operation:
+
+ * SGV_POOL_ALLOC_NO_CACHED - the SG vector must not be cached.
+
+ * SGV_POOL_NO_ALLOC_ON_CACHE_MISS - don't do an allocation on a
+ cache miss.
+
+ * SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL - return an empty SGV object,
+ i.e. without the SG vector, if the allocation can't be completed.
+ For instance, because SGV_POOL_NO_ALLOC_ON_CACHE_MISS flag set.
+
+ - count - the resulting count of SG entries in the resulting SG vector.
+
+ - sgv - the resulting SGV object. It should be used to free the
+ resulting SG vector.
+
+ - mem_lim - memory limits, see below.
+
+ - priv - pointer to private for this allocation data. This pointer will
+ be supplied to alloc_pages_fn() and free_pages_fn() and can be
+ retrieved by sgv_get_priv().
+
+This function returns pointer to the resulting SG vector or NULL in case
+of any error.
+
+void sgv_pool_free(struct sgv_pool_obj *sgv, struct scst_mem_lim *mem_lim)
+
+This function frees previously allocated SG vector, referenced by SGV
+cache object sgv.
+
+void *sgv_get_priv(struct sgv_pool_obj *sgv)
+
+This function allows to get the allocation private data for this SGV
+cache object sgv. The private data are set by sgv_pool_alloc().
+
+void scst_init_mem_lim(struct scst_mem_lim *mem_lim)
+
+This function initializes memory limits structure mem_lim according to
+the current system configuration. This structure should be latter used
+to track and limit allocated by one or more SGV caches memory.
+
+ Runtime information and statistics.
+
+Runtime information and statistics is available in /sys/kernel/scst_tgt/sgv.
+
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 11/17]: SCST core's docs
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (9 preceding siblings ...)
2010-09-14 14:45 ` [PATCH 10/17]: SCST SGV cache Vladislav Bolkhovitin
@ 2010-09-14 14:45 ` Vladislav Bolkhovitin
2010-09-14 14:46 ` [PATCH 12/17]: SCST dev handlers' Makefile Vladislav Bolkhovitin
` (6 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:45 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST core's docs.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
README.scst | 1437 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
SysfsRules | 933 ++++++++++++++++++++++++++++++++++++++
2 files changed, 2370 insertions(+)
diff -uprN orig/linux-2.6.35/Documentation/scst/README.scst linux-2.6.35/Documentation/scst/README.scst
--- orig/linux-2.6.35/Documentation/scst/README.scst
+++ linux-2.6.35/Documentation/scst/README.scst
@@ -0,0 +1,1437 @@
+Generic SCSI target mid-level for Linux (SCST)
+==============================================
+
+SCST is designed to provide unified, consistent interface between SCSI
+target drivers and Linux kernel and simplify target drivers development
+as much as possible. Detail description of SCST's features and internals
+could be found on its Internet page http://scst.sourceforge.net.
+
+SCST supports the following I/O modes:
+
+ * Pass-through mode with one to many relationship, i.e. when multiple
+ initiators can connect to the exported pass-through devices, for
+ the following SCSI devices types: disks (type 0), tapes (type 1),
+ processors (type 3), CDROMs (type 5), MO disks (type 7), medium
+ changers (type 8) and RAID controllers (type 0xC).
+
+ * FILEIO mode, which allows to use files on file systems or block
+ devices as virtual remotely available SCSI disks or CDROMs with
+ benefits of the Linux page cache.
+
+ * BLOCKIO mode, which performs direct block IO with a block device,
+ bypassing page-cache for all operations. This mode works ideally with
+ high-end storage HBAs and for applications that either do not need
+ caching between application and disk or need the large block
+ throughput.
+
+ * "Performance" device handlers, which provide in pseudo pass-through
+ mode a way for direct performance measurements without overhead of
+ actual data transferring from/to underlying SCSI device.
+
+In addition, SCST supports advanced per-initiator access and devices
+visibility management, so different initiators could see different set
+of devices with different access permissions. See below for details.
+
+Full list of SCST features and comparison with other Linux targets you
+can find on http://scst.sourceforge.net/comparison.html.
+
+Installation
+------------
+
+To see your devices remotely, you need to add a corresponding LUN for
+them (see below how). By default, no local devices are seen remotely.
+There must be LUN 0 in each LUNs set (security group), i.e. LUs
+numeration must not start from, e.g., 1. Otherwise you will see no
+devices on remote initiators and SCST core will write into the kernel
+log message: "tgt_dev for LUN 0 not found, command to unexisting LU?"
+
+It is highly recommended to use scstadmin utility for configuring
+devices and security groups.
+
+If you experience problems during modules load or running, check your
+kernel logs (or run dmesg command for the few most recent messages).
+
+IMPORTANT: Without loading appropriate device handler, corresponding devices
+========= will be invisible for remote initiators, which could lead to holes
+ in the LUN addressing, so automatic device scanning by remote SCSI
+ mid-level could not notice the devices. Therefore you will have
+ to add them manually via
+ 'echo "- - -" >/sys/class/scsi_host/hostX/scan',
+ where X - is the host number.
+
+IMPORTANT: Working of target and initiator on the same host is
+========= supported, except the following 2 cases: swap over target exported
+ device and using a writable mmap over a file from target
+ exported device. The latter means you can't mount a file
+ system over target exported device. In other words, you can
+ freely use any sg, sd, st, etc. devices imported from target
+ on the same host, but you can't mount file systems or put
+ swap on them. This is a limitation of Linux memory/cache
+ manager, because in this case an OOM deadlock like: system
+ needs some memory -> it decides to clear some cache -> cache
+ needs to write on target exported device -> initiator sends
+ request to the target -> target needs memory -> system needs
+ even more memory -> deadlock.
+
+IMPORTANT: In the current version simultaneous access to local SCSI devices
+========= via standard high-level SCSI drivers (sd, st, sg, etc.) and
+ SCST's target drivers is unsupported. Especially it is
+ important for execution via sg and st commands that change
+ the state of devices and their parameters, because that could
+ lead to data corruption. If any such command is done, at
+ least related device handler(s) must be restarted. For block
+ devices READ/WRITE commands using direct disk handler are
+ generally safe.
+
+Usage in failover mode
+----------------------
+
+It is recommended to use TEST UNIT READY ("tur") command to check if
+SCST target is alive in MPIO configurations.
+
+Device handlers
+---------------
+
+Device specific drivers (device handlers) are plugins for SCST, which
+help SCST to analyze incoming requests and determine parameters,
+specific to various types of devices. If an appropriate device handler
+for a SCSI device type isn't loaded, SCST doesn't know how to handle
+devices of this type, so they will be invisible for remote initiators
+(more precisely, "LUN not supported" sense code will be returned).
+
+In addition to device handlers for real devices, there are VDISK, user
+space and "performance" device handlers.
+
+VDISK device handler works over files on file systems and makes from
+them virtual remotely available SCSI disks or CDROM's. In addition, it
+allows to work directly over a block device, e.g. local IDE or SCSI disk
+or ever disk partition, where there is no file systems overhead. Using
+block devices comparing to sending SCSI commands directly to SCSI
+mid-level via scsi_do_req()/scsi_execute_async() has advantage that data
+are transferred via system cache, so it is possible to fully benefit from
+caching and read ahead performed by Linux's VM subsystem. The only
+disadvantage here that in the FILEIO mode there is superfluous data
+copying between the cache and SCST's buffers. This issue is going to be
+addressed in the next release. Virtual CDROM's are useful for remote
+installation. See below for details how to setup and use VDISK device
+handler.
+
+"Performance" device handlers for disks, MO disks and tapes in their
+exec() method skip (pretend to execute) all READ and WRITE operations
+and thus provide a way for direct link performance measurements without
+overhead of actual data transferring from/to underlying SCSI device.
+
+NOTE: Since "perf" device handlers on READ operations don't touch the
+==== commands' data buffer, it is returned to remote initiators as it
+ was allocated, without even being zeroed. Thus, "perf" device
+ handlers impose some security risk, so use them with caution.
+
+Compilation options
+-------------------
+
+There are the following compilation options, that could be change using
+your favorite kernel configuration Makefile target, e.g. "make xconfig":
+
+ - CONFIG_SCST_DEBUG - if defined, turns on some debugging code,
+ including some logging. Makes the driver considerably bigger and slower,
+ producing large amount of log data.
+
+ - CONFIG_SCST_TRACING - if defined, turns on ability to log events. Makes the
+ driver considerably bigger and leads to some performance loss.
+
+ - CONFIG_SCST_EXTRACHECKS - if defined, adds extra validity checks in
+ the various places.
+
+ - CONFIG_SCST_USE_EXPECTED_VALUES - if not defined (default), initiator
+ supplied expected data transfer length and direction will be used
+ only for verification purposes to return error or warn in case if one
+ of them is invalid. Instead, locally decoded from SCSI command values
+ will be used. This is necessary for security reasons, because
+ otherwise a faulty initiator can crash target by supplying invalid
+ value in one of those parameters. This is especially important in
+ case of pass-through mode. If CONFIG_SCST_USE_EXPECTED_VALUES is
+ defined, initiator supplied expected data transfer length and
+ direction will override the locally decoded values. This might be
+ necessary if internal SCST commands translation table doesn't contain
+ SCSI command, which is used in your environment. You can know that if
+ you enable "minor" trace level and have messages like "Unknown
+ opcode XX for YY. Should you update scst_scsi_op_table?" in your
+ kernel log and your initiator returns an error. Also report those
+ messages in the SCST mailing list scst-devel@lists.sourceforge.net.
+ Note, that not all SCSI transports support supplying expected values.
+
+ - CONFIG_SCST_DEBUG_TM - if defined, turns on task management functions
+ debugging, when on LUN 6 some of the commands will be delayed for
+ about 60 sec., so making the remote initiator send TM functions, eg
+ ABORT TASK and TARGET RESET. Also define
+ CONFIG_SCST_TM_DBG_GO_OFFLINE symbol in the Makefile if you want that
+ the device eventually become completely unresponsive, or otherwise to
+ circle around ABORTs and RESETs code. Needs CONFIG_SCST_DEBUG turned
+ on.
+
+ - CONFIG_SCST_STRICT_SERIALIZING - if defined, makes SCST send all commands to
+ underlying SCSI device synchronously, one after one. This makes task
+ management more reliable, with cost of some performance penalty. This
+ is mostly actual for stateful SCSI devices like tapes, where the
+ result of command's execution depends from device's settings defined
+ by previous commands. Disk and RAID devices are stateless in the most
+ cases. The current SCSI core in Linux doesn't allow to abort all
+ commands reliably if they sent asynchronously to a stateful device.
+ Turned off by default, turn it on if you use stateful device(s) and
+ need as much error recovery reliability as possible. As a side effect
+ of CONFIG_SCST_STRICT_SERIALIZING, on kernels below 2.6.30 no kernel
+ patching is necessary for pass-through device handlers (scst_disk,
+ etc.).
+
+ - CONFIG_SCST_TEST_IO_IN_SIRQ - if defined, allows SCST to submit selected
+ SCSI commands (TUR and READ/WRITE) from soft-IRQ context (tasklets).
+ Enabling it will decrease amount of context switches and slightly
+ improve performance. The goal of this option is to be able to measure
+ overhead of the context switches. If after enabling this option you
+ don't see under load in vmstat output on the target significant
+ decrease of amount of context switches, then your target driver
+ doesn't submit commands to SCST in IRQ context. For instance,
+ iSCSI-SCST doesn't do that, but qla2x00t with
+ CONFIG_QLA_TGT_DEBUG_WORK_IN_THREAD disabled - does. This option is
+ designed to be used with vdisk NULLIO backend.
+
+ WARNING! Using this option enabled with other backend than vdisk
+ NULLIO is unsafe and can lead you to a kernel crash!
+
+ - CONFIG_SCST_STRICT_SECURITY - if defined, makes SCST zero allocated data
+ buffers. Undefining it (default) considerably improves performance
+ and eases CPU load, but could create a security hole (information
+ leakage), so enable it, if you have strict security requirements.
+
+ - CONFIG_SCST_ABORT_CONSIDER_FINISHED_TASKS_AS_NOT_EXISTING - if defined,
+ in case when TASK MANAGEMENT function ABORT TASK is trying to abort a
+ command, which has already finished, remote initiator, which sent the
+ ABORT TASK request, will receive TASK NOT EXIST (or ABORT FAILED)
+ response for the ABORT TASK request. This is more logical response,
+ since, because the command finished, attempt to abort it failed, but
+ some initiators, particularly VMware iSCSI initiator, consider TASK
+ NOT EXIST response as if the target got crazy and try to RESET it.
+ Then sometimes get crazy itself. So, this option is disabled by
+ default.
+
+ - CONFIG_SCST_MEASURE_LATENCY - if defined, provides in "latency" files
+ global and per-LUN average commands processing latency statistic. You
+ can clear already measured results by writing 0 in each file. Note,
+ you need a non-preemptible kernel to have correct results.
+
+HIGHMEM kernel configurations are fully supported, but not recommended
+for performance reasons.
+
+Module parameters
+-----------------
+
+Module scst supports the following parameters:
+
+ - scst_threads - allows to set count of SCST's threads. By default it
+ is CPU count.
+
+ - scst_max_cmd_mem - sets maximum amount of memory in MB allowed to be
+ consumed by the SCST commands for data buffers at any given time. By
+ default it is approximately TotalMem/4.
+
+SCST sysfs interface
+--------------------
+
+Root of SCST sysfs interface is /sys/kernel/scst_tgt. It has the
+following entries:
+
+ - devices - this is a root subdirectory for all SCST devices
+
+ - handlers - this is a root subdirectory for all SCST dev handlers
+
+ - sgv - this is a root subdirectory for all SCST SGV caches
+
+ - targets - this is a root subdirectory for all SCST targets
+
+ - setup_id - allows to read and write SCST setup ID. This ID can be
+ used in cases, when the same SCST configuration should be installed
+ on several targets, but exported from those targets devices should
+ have different IDs and SNs. For instance, VDISK dev handler uses this
+ ID to generate T10 vendor specific identifier and SN of the devices.
+
+ - threads - allows to read and set number of global SCST I/O threads.
+ Those threads used with async. dev handlers, for instance, vdisk
+ BLOCKIO or NULLIO.
+
+ - trace_level - allows to enable and disable various tracing
+ facilities. See content of this file for help how to use it.
+
+ - version - read-only attribute, which allows to see version of
+ SCST and enabled optional features.
+
+ - last_sysfs_mgmt_res - read-only attribute returning completion status
+ of the last management command. In the sysfs implementation there are
+ some problems between internal sysfs and internal SCST locking. To
+ avoid them in some cases sysfs calls can return error with errno
+ EAGAIN. This doesn't mean the operation failed. It only means that
+ the operation queued and not yet completed. To wait for it to
+ complete, an management tool should poll this file. If the operation
+ hasn't yet completed, it will also return EAGAIN. But after it's
+ completed, it will return the result of this operation (0 for success
+ or -errno for error).
+
+Each SCST sysfs file (attribute) can contain in the last line mark
+"[key]". It is automatically added mark used to allow scstadmin to see
+which attributes it should save in the config file. You can ignore it.
+
+"Devices" subdirectory contains subdirectories for each SCST devices.
+
+Content of each device's subdirectory is dev handler specific. See
+documentation for your dev handlers for more info about it as well as
+SysfsRules file for more info about common to all dev handlers rules.
+SCST dev handlers can have the following common entries:
+
+ - exported - subdirectory containing links to all LUNs where this
+ device was exported.
+
+ - handler - if dev handler determined for this device, this link points
+ to it. The handler can be not set for pass-through devices.
+
+ - threads_num - shows and allows to set number of threads in this device's
+ threads pool. If 0 - no threads will be created, and global SCST
+ threads pool will be used. If <0 - creation of the threads pool is
+ prohibited.
+
+ - threads_pool_type - shows and allows to sets threads pool type.
+ Possible values: "per_initiator" and "shared". When the value is
+ "per_initiator" (default), each session from each initiator will use
+ separate dedicated pool of threads. When the value is "shared", all
+ sessions from all initiators will share the same per-device pool of
+ threads. Valid only if threads_num attribute >0.
+
+ - dump_prs - allows to dump persistent reservations information in the
+ kernel log.
+
+ - type - SCSI type of this device
+
+See below for more information about other entries of this subdirectory
+of the standard SCST dev handlers.
+
+"Handlers" subdirectory contains subdirectories for each SCST dev
+handler.
+
+Content of each handler's subdirectory is dev handler specific. See
+documentation for your dev handlers for more info about it as well as
+SysfsRules file for more info about common to all dev handlers rules.
+SCST dev handlers can have the following common entries:
+
+ - mgmt - this entry allows to create virtual devices and their
+ attributes (for virtual devices dev handlers) or assign/unassign real
+ SCSI devices to/from this dev handler (for pass-through dev
+ handlers).
+
+ - trace_level - allows to enable and disable various tracing
+ facilities. See content of this file for help how to use it.
+
+ - type - SCSI type of devices served by this dev handler.
+
+See below for more information about other entries of this subdirectory
+of the standard SCST dev handlers.
+
+"Sgv" subdirectory contains statistic information of SCST SGV caches. It
+has the following entries:
+
+ - None, one or more subdirectories for each existing SGV cache.
+
+ - global_stats - file containing global SGV caches statistics.
+
+Each SGV cache's subdirectory has the following item:
+
+ - stats - file containing statistics for this SGV caches.
+
+"Targets" subdirectory contains subdirectories for each SCST target.
+
+Content of each target's subdirectory is target specific. See
+documentation for your target for more info about it as well as
+SysfsRules file for more info about common to all targets rules.
+Every target should have at least the following entries:
+
+ - ini_groups - subdirectory, which contains and allows to define
+ initiator-oriented access control information, see below.
+
+ - luns - subdirectory, which contains list of available LUNs in the
+ target-oriented access control and allows to define it, see below.
+
+ - sessions - subdirectory containing connected to this target sessions.
+
+ - enabled - using this attribute you can enable or disable this target/
+ It allows to finish configuring it before it starts accepting new
+ connections. 0 by default.
+
+ - addr_method - used LUNs addressing method. Possible values:
+ "Peripheral" and "Flat". Most initiators work well with Peripheral
+ addressing method (default), but some (HP-UX, for instance) may
+ require Flat method. This attribute is also available in the
+ initiators security groups, so you can assign the addressing method
+ on per-initiator basis.
+
+ - io_grouping_type - defines how I/O from sessions to this target are
+ grouped together. This I/O grouping is very important for
+ performance. By setting this attribute in a right value, you can
+ considerably increase performance of your setup. This grouping is
+ performed only if you use CFQ I/O scheduler on the target and for
+ devices with threads_num >= 0 and, if threads_num > 0, with
+ threads_pool_type "per_initiator". Possible values:
+ "this_group_only", "never", "auto", or I/O group number >0. When the
+ value is "this_group_only" all I/O from all sessions in this target
+ will be grouped together. When the value is "never", I/O from
+ different sessions will not be grouped together, i.e. all sessions in
+ this target will have separate dedicated I/O groups. When the value
+ is "auto" (default), all I/O from initiators with the same name
+ (iSCSI initiator name, for instance) in all targets will be grouped
+ together with a separate dedicated I/O group for each initiator name.
+ For iSCSI this mode works well, but other transports usually use
+ different initiator names for different sessions, so using such
+ transports in MPIO configurations you should either use value
+ "this_group_only", or an explicit I/O group number. This attribute is
+ also available in the initiators security groups, so you can assign
+ the I/O grouping on per-initiator basis. See below for more info how
+ to use this attribute.
+
+ - rel_tgt_id - allows to read or write SCSI Relative Target Port
+ Identifier attribute. This identifier is used to identify SCSI Target
+ Ports by some SCSI commands, mainly by Persistent Reservations
+ commands. This identifier must be unique among all SCST targets, but
+ for convenience SCST allows disabled targets to have not unique
+ rel_tgt_id. In this case SCST will not allow to enable this target
+ until rel_tgt_id becomes unique. This attribute initialized unique by
+ SCST by default.
+
+A target driver may have also the following entries:
+
+ - "hw_target" - if the target driver supports both hardware and virtual
+ targets (for instance, an FC adapter supporting NPIV, which has
+ hardware targets for its physical ports as well as virtual NPIV
+ targets), this read only attribute for all hardware targets will
+ exist and contain value 1.
+
+Subdirectory "sessions" contains one subdirectory for each connected
+session with name equal to name of the connected initiator.
+
+Each session subdirectory contains the following entries:
+
+ - initiator_name - contains initiator name
+
+ - force_close - optional write-only attribute, which allows to force
+ close this session.
+
+ - active_commands - contains number of active, i.e. not yet or being
+ executed, SCSI commands in this session.
+
+ - commands - contains overall number of SCSI commands in this session.
+
+ - latency - if CONFIG_SCST_MEASURE_LATENCY enabled, contains latency
+ statistics for this session.
+
+ - luns - a link pointing out to the corresponding LUNs set (security
+ group) where this session was attached to.
+
+ - One or more "lunX" subdirectories, where 'X' is a number, for each LUN
+ this session has (see below).
+
+ - other target driver specific attributes and subdirectories.
+
+See below description of the VDISK's sysfs interface for samples.
+
+Access and devices visibility management (LUN masking)
+------------------------------------------------------
+
+Access and devices visibility management allows for an initiator or
+group of initiators to see different devices with different LUNs
+with necessary access permissions.
+
+SCST supports two modes of access control:
+
+1. Target-oriented. In this mode you define for each target a default
+set of LUNs, which are accessible to all initiators, connected to that
+target. This is a regular access control mode, which people usually mean
+thinking about access control in general. For instance, in IET this is
+the only supported mode.
+
+2. Initiator-oriented. In this mode you define which LUNs are accessible
+for each initiator. In this mode you should create for each set of one
+or more initiators, which should access to the same set of devices with
+the same LUNs, a separate security group, then add to it devices and
+names of allowed initiator(s).
+
+Both modes can be used simultaneously. In this case the
+initiator-oriented mode has higher priority, than the target-oriented,
+i.e. initiators are at first searched in all defined security groups for
+this target and, if none matches, the default target's set of LUNs is
+used. This set of LUNs might be empty, then the initiator will not see
+any LUNs from the target.
+
+You can at any time find out which set of LUNs each session is assigned
+to by looking where link
+/sys/kernel/scst_tgt/targets/target_driver/target_name/sessions/initiator_name/luns
+points to.
+
+To configure the target-oriented access control SCST provides the
+following interface. Each target's sysfs subdirectory
+(/sys/kernel/scst_tgt/targets/target_driver/target_name) has "luns"
+subdirectory. This subdirectory contains the list of already defined
+target-oriented access control LUNs for this target as well as file
+"mgmt". This file has the following commands, which you can send to it,
+for instance, using "echo" shell command. You can always get a small
+help about supported commands by looking inside this file. "Parameters"
+are one or more param_name=value pairs separated by ';'.
+
+ - "add H:C:I:L lun [parameters]" - adds a pass-through device with
+ host:channel:id:lun with LUN "lun". Optionally, the device could be
+ marked as read only by using parameter "read_only". The recommended
+ way to find out H:C:I:L numbers is use of lsscsi utility.
+
+ - "replace H:C:I:L lun [parameters]" - replaces by pass-through device
+ with host:channel:id:lun existing with LUN "lun" device with
+ generation of INQUIRY DATA HAS CHANGED Unit Attention. If the old
+ device doesn't exist, this command acts as the "add" command.
+ Optionally, the device could be marked as read only by using
+ parameter "read_only". The recommended way to find out H:C:I:L
+ numbers is use of lsscsi utility.
+
+ - "add VNAME lun [parameters]" - adds a virtual device with name VNAME
+ with LUN "lun". Optionally, the device could be marked as read only
+ by using parameter "read_only".
+
+ - "replace VNAME lun [parameters]" - replaces by virtual device
+ with name VNAME existing with LUN "lun" device with generation of
+ INQUIRY DATA HAS CHANGED Unit Attention. If the old device doesn't
+ exist, this command acts as the "add" command. Optionally, the device
+ could be marked as read only by using parameter "read_only".
+
+ - "del lun" - deletes LUN lun
+
+ - "clear" - clears the list of devices
+
+To configure the initiator-oriented access control SCST provides the
+following interface. Each target's sysfs subdirectory
+(/sys/kernel/scst_tgt/targets/target_driver/target_name) has "ini_groups"
+subdirectory. This subdirectory contains the list of already defined
+security groups for this target as well as file "mgmt". This file has
+the following commands, which you can send to it, for instance, using
+"echo" shell command. You can always get a small help about supported
+commands by looking inside this file.
+
+ - "create GROUP_NAME" - creates a new security group.
+
+ - "del GROUP_NAME" - deletes a new security group.
+
+Each security group's subdirectory contains 2 subdirectories: initiators
+and luns.
+
+Each "initiators" subdirectory contains list of added to this groups
+initiator as well as as well as file "mgmt". This file has the following
+commands, which you can send to it, for instance, using "echo" shell
+command. You can always get a small help about supported commands by
+looking inside this file.
+
+ - "add INITIATOR_NAME" - adds initiator with name INITIATOR_NAME to the
+ group.
+
+ - "del INITIATOR_NAME" - deletes initiator with name INITIATOR_NAME
+ from the group.
+
+ - "move INITIATOR_NAME DEST_GROUP_NAME" moves initiator with name
+ INITIATOR_NAME from the current group to group with name
+ DEST_GROUP_NAME.
+
+ - "clear" - deletes all initiators from this group.
+
+For "add" and "del" commands INITIATOR_NAME can be a simple DOS-type
+patterns, containing '*' and '?' symbols. '*' means match all any
+symbols, '?' means match only any single symbol. For instance,
+"blah.xxx" will match "bl?h.*". Additionally, you can use negative sign
+'!' to revert the value of the pattern. For instance, "ah.xxx" will
+match "!bl?h.*".
+
+Each "luns" subdirectory contains the list of already defined LUNs for
+this group as well as file "mgmt". Content of this file as well as list
+of available in it commands is fully identical to the "luns"
+subdirectory of the target-oriented access control.
+
+Examples:
+
+ - echo "create INI" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/mgmt -
+ creates security group INI for target iqn.2006-10.net.vlnb:tgt1.
+
+ - echo "add 2:0:1:0 11" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI/luns/mgmt -
+ adds a pass-through device sitting on host 2, channel 0, ID 1, LUN 0
+ to group with name INI as LUN 11.
+
+ - echo "add disk1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI/luns/mgmt -
+ adds a virtual disk with name disk1 to group with name INI as LUN 0.
+
+ - echo "add 21:*:e0:?b:83:*" >/sys/kernel/scst_tgt/targets/21:00:00:a0:8c:54:52:12/ini_groups/INI/initiators/mgmt -
+ adds a pattern to group with name INI to Fibre Channel target with
+ WWN 21:00:00:a0:8c:54:52:12, which matches WWNs of Fibre Channel
+ initiator ports.
+
+Consider you need to have an iSCSI target with name
+"iqn.2007-05.com.example:storage.disk1.sys1.xyz", which should export
+virtual device "dev1" with LUN 0 and virtual device "dev2" with LUN 1,
+but initiator with name
+"iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" should see only
+virtual device "dev2" read only with LUN 0. To achieve that you should
+do the following commands:
+
+# echo "iqn.2007-05.com.example:storage.disk1.sys1.xyz" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+# echo "add dev1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/luns/mgmt
+# echo "add dev2 1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/luns/mgmt
+# echo "create SPEC_INI" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/ini_groups/mgmt
+# echo "add dev2 0 read_only=1" \
+ >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/ini_groups/SPEC_INI/luns/mgmt
+# echo "iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" \
+ >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/ini_groups/SPEC_INI/initiators/mgmt
+
+For Fibre Channel or SAS in the above example you should use target's
+and initiator ports WWNs instead of iSCSI names.
+
+It is highly recommended to use scstadmin utility instead of described
+in this section low level interface.
+
+IMPORTANT
+=========
+
+There must be LUN 0 in each set of LUNs, i.e. LUs numeration must not
+start from, e.g., 1. Otherwise you will see no devices on remote
+initiators and SCST core will write into the kernel log message: "tgt_dev
+for LUN 0 not found, command to unexisting LU?"
+
+IMPORTANT
+=========
+
+All the access control must be fully configured BEFORE the corresponding
+target is enabled. When you enable a target, it will immediately start
+accepting new connections, hence creating new sessions, and those new
+sessions will be assigned to security groups according to the
+*currently* configured access control settings. For instance, to
+the default target's set of LUNs, instead of "HOST004" group as you may
+need, because "HOST004" doesn't exist yet. So, you must configure all
+the security groups before new connections from the initiators are
+created, i.e. before the target enabled.
+
+VDISK device handler
+--------------------
+
+VDISK has 4 built-in dev handlers: vdisk_fileio, vdisk_blockio,
+vdisk_nullio and vcdrom. Roots of their sysfs interface are
+/sys/kernel/scst_tgt/handlers/handler_name, e.g. for vdisk_fileio:
+/sys/kernel/scst_tgt/handlers/vdisk_fileio. Each root has the following
+entries:
+
+ - None, one or more links to devices with name equal to names
+ of the corresponding devices.
+
+ - trace_level - allows to enable and disable various tracing
+ facilities. See content of this file for help how to use it.
+
+ - mgmt - main management entry, which allows to add/delete VDISK
+ devices with the corresponding type.
+
+The "mgmt" file has the following commands, which you can send to it,
+for instance, using "echo" shell command. You can always get a small
+help about supported commands by looking inside this file. "Parameters"
+are one or more param_name=value pairs separated by ';'.
+
+ - echo "add_device device_name [parameters]" - adds a virtual device
+ with name device_name and specified parameters (see below)
+
+ - echo "del_device device_name" - deletes a virtual device with name
+ device_name.
+
+Handler vdisk_fileio provides FILEIO mode to create virtual devices.
+This mode uses as backend files and accesses to them using regular
+read()/write() file calls. This allows to use full power of Linux page
+cache. The following parameters possible for vdisk_fileio:
+
+ - filename - specifies path and file name of the backend file. The path
+ must be absolute.
+
+ - blocksize - specifies block size used by this virtual device. The
+ block size must be power of 2 and >= 512 bytes. Default is 512.
+
+ - write_through - disables write back caching. Note, this option
+ has sense only if you also *manually* disable write-back cache in
+ *all* your backstorage devices and make sure it's actually disabled,
+ since many devices are known to lie about this mode to get better
+ benchmark results. Default is 0.
+
+ - read_only - read only. Default is 0.
+
+ - o_direct - disables both read and write caching. This mode isn't
+ currently fully implemented, you should use user space fileio_tgt
+ program in O_DIRECT mode instead (see below).
+
+ - nv_cache - enables "non-volatile cache" mode. In this mode it is
+ assumed that the target has a GOOD UPS with ability to cleanly
+ shutdown target in case of power failure and it is software/hardware
+ bugs free, i.e. all data from the target's cache are guaranteed
+ sooner or later to go to the media. Hence all data synchronization
+ with media operations, like SYNCHRONIZE_CACHE, are ignored in order
+ to bring more performance. Also in this mode target reports to
+ initiators that the corresponding device has write-through cache to
+ disable all write-back cache workarounds used by initiators. Use with
+ extreme caution, since in this mode after a crash of the target
+ journaled file systems don't guarantee the consistency after journal
+ recovery, therefore manual fsck MUST be ran. Note, that since usually
+ the journal barrier protection (see "IMPORTANT" note below) turned
+ off, enabling NV_CACHE could change nothing from data protection
+ point of view, since no data synchronization with media operations
+ will go from the initiator. This option overrides "write_through"
+ option. Disabled by default.
+
+ - removable - with this flag set the device is reported to remote
+ initiators as removable.
+
+Handler vdisk_blockio provides BLOCKIO mode to create virtual devices.
+This mode performs direct block I/O with a block device, bypassing the
+page cache for all operations. This mode works ideally with high-end
+storage HBAs and for applications that either do not need caching
+between application and disk or need the large block throughput. See
+below for more info.
+
+The following parameters possible for vdisk_blockio: filename,
+blocksize, nv_cache, read_only, removable. See vdisk_fileio above for
+description of those parameters.
+
+Handler vdisk_nullio provides NULLIO mode to create virtual devices. In
+this mode no real I/O is done, but success returned to initiators.
+Intended to be used for performance measurements at the same way as
+"*_perf" handlers. The following parameters possible for vdisk_nullio:
+blocksize, read_only, removable. See vdisk_fileio above for description
+of those parameters.
+
+Handler vcdrom allows emulation of a virtual CDROM device using an ISO
+file as backend. It doesn't have any parameters.
+
+For example:
+
+echo "add_device disk1 filename=/disk1; blocksize=4096; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
+
+will create a FILEIO virtual device disk1 with backend file /disk1
+with block size 4K and NV_CACHE enabled.
+
+Each vdisk_fileio's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name:
+
+ - filename - contains path and file name of the backend file.
+
+ - blocksize - contains block size used by this virtual device.
+
+ - write_through - contains status of write back caching of this virtual
+ device.
+
+ - read_only - contains read only status of this virtual device.
+
+ - o_direct - contains O_DIRECT status of this virtual device.
+
+ - nv_cache - contains NV_CACHE status of this virtual device.
+
+ - removable - contains removable status of this virtual device.
+
+ - size_mb - contains size of this virtual device in MB.
+
+ - t10_dev_id - contains and allows to set T10 vendor specific
+ identifier for Device Identification VPD page (0x83) of INQUIRY data.
+ By default VDISK handler always generates t10_dev_id for every new
+ created device at creation time based on the device name and
+ scst_vdisk_ID scst_vdisk.ko module parameter (see below).
+
+ - usn - contains the virtual device's serial number of INQUIRY data. It
+ is created at the device creation time based on the device name and
+ scst_vdisk_ID scst_vdisk.ko module parameter (see below).
+
+ - type - contains SCSI type of this virtual device.
+
+ - resync_size - write only attribute, which makes vdisk_fileio to
+ rescan size of the backend file. It is useful if you changed it, for
+ instance, if you resized it.
+
+For example:
+
+/sys/kernel/scst_tgt/devices/disk1
+|-- blocksize
+|-- exported
+| |-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/0
+| |-- export1 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/INI/luns/0
+| |-- export2 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/0
+| |-- export3 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI1/luns/0
+| |-- export4 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI2/luns/0
+|-- filename
+|-- handler -> ../../handlers/vdisk_fileio
+|-- nv_cache
+|-- o_direct
+|-- read_only
+|-- removable
+|-- resync_size
+|-- size_mb
+|-- t10_dev_id
+|-- threads_num
+|-- threads_pool_type
+|-- type
+|-- usn
+`-- write_through
+
+Each vdisk_blockio's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name: blocksize, filename, nv_cache,
+read_only, removable, resync_size, size_mb, t10_dev_id, threads_num,
+threads_pool_type, type, usn. See above description of those parameters.
+
+Each vdisk_nullio's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name: blocksize, read_only,
+removable, size_mb, t10_dev_id, threads_num, threads_pool_type, type,
+usn. See above description of those parameters.
+
+Each vcdrom's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name: filename, size_mb,
+t10_dev_id, threads_num, threads_pool_type, type, usn. See above
+description of those parameters. Exception is filename attribute. For
+vcdrom it is writable. Writing to it allows to virtually insert or
+change virtual CD media in the virtual CDROM device. For example:
+
+ - echo "/image.iso" >/sys/kernel/scst_tgt/devices/cdrom/filename - will
+ insert file /image.iso as virtual media to the virtual CDROM cdrom.
+
+ - echo "" >/sys/kernel/scst_tgt/devices/cdrom/filename - will remove
+ "media" from the virtual CDROM cdrom.
+
+Additionally VDISK handler has module parameter "num_threads", which
+specifies count of I/O threads for each FILEIO VDISK's or VCDROM device.
+If you have a workload, which tends to produce rather random accesses
+(e.g. DB-like), you should increase this count to a bigger value, like
+32. If you have a rather sequential workload, you should decrease it to
+a lower value, like number of CPUs on the target or even 1. Due to some
+limitations of Linux I/O subsystem, increasing number of I/O threads too
+much leads to sequential performance drop, especially with deadline
+scheduler, so decreasing it can improve sequential performance. The
+default provides a good compromise between random and sequential
+accesses.
+
+You shouldn't be afraid to have too many VDISK I/O threads if you have
+many VDISK devices. Kernel threads consume very little amount of
+resources (several KBs) and only necessary threads will be used by SCST,
+so the threads will not trash your system.
+
+CAUTION: If you partitioned/formatted your device with block size X, *NEVER*
+======== ever try to export and then mount it (even accidentally) with another
+ block size. Otherwise you can *instantly* damage it pretty
+ badly as well as all your data on it. Messages on initiator
+ like: "attempt to access beyond end of device" is the sign of
+ such damage.
+
+ Moreover, if you want to compare how well different block sizes
+ work for you, you **MUST** EVERY TIME AFTER CHANGING BLOCK SIZE
+ **COMPLETELY** **WIPE OFF** ALL THE DATA FROM THE DEVICE. In
+ other words, THE **WHOLE** DEVICE **MUST** HAVE ONLY **ZEROS**
+ AS THE DATA AFTER YOU SWITCH TO NEW BLOCK SIZE. Switching block
+ sizes isn't like switching between FILEIO and BLOCKIO, after
+ changing block size all previously written with another block
+ size data MUST BE ERASED. Otherwise you will have a full set of
+ very weird behaviors, because blocks addressing will be
+ changed, but initiators in most cases will not have a
+ possibility to detect that old addresses written on the device
+ in, e.g., partition table, don't refer anymore to what they are
+ intended to refer.
+
+IMPORTANT: Some disk and partition table management utilities don't support
+========= block sizes >512 bytes, therefore make sure that your favorite one
+ supports it. Currently only cfdisk is known to work only with
+ 512 bytes blocks, other utilities like fdisk on Linux or
+ standard disk manager on Windows are proved to work well with
+ non-512 bytes blocks. Note, if you export a disk file or
+ device with some block size, different from one, with which
+ it was already partitioned, you could get various weird
+ things like utilities hang up or other unexpected behavior.
+ Hence, to be sure, zero the exported file or device before
+ the first access to it from the remote initiator with another
+ block size. On Window initiator make sure you "Set Signature"
+ in the disk manager on the imported from the target drive
+ before doing any other partitioning on it. After you
+ successfully mounted a file system over non-512 bytes block
+ size device, the block size stops matter, any program will
+ work with files on such file system.
+
+Persistent Reservations
+-----------------------
+
+SCST implements Persistent Reservations with full set of capabilities,
+including "Persistence Through Power Loss".
+
+The "Persistence Through Power Loss" data are saved in /var/lib/scst/pr
+with files with names the same as the names of the corresponding
+devices. Also this directory contains backup versions of those files
+with suffix ".1". Those backup files are used in case of power or other
+failure to prevent Persistent Reservation information from corruption
+during update.
+
+The Persistent Reservations available on all transports implementing
+get_initiator_port_transport_id() callback. Transports not implementing
+this callback will act in one of 2 possible scenarios ("all or
+nothing"):
+
+1. If a device has such transport connected and doesn't have persistent
+reservations, it will refuse Persistent Reservations commands as if it
+doesn't support them.
+
+2. If a device has persistent reservations, all initiators newly
+connecting via such transports will not see this device. After all
+persistent reservations from this device are released, upon reconnect
+the initiators will see it.
+
+Caching
+-------
+
+By default for performance reasons VDISK FILEIO devices use write back
+caching policy.
+
+Generally, write back caching is safe for use and danger of it is
+greatly overestimated, because most modern (especially, Enterprise
+level) applications are well prepared to work with write back cached
+storage. Particularly, such are all transactions-based applications.
+Those applications flush cache to completely avoid ANY data loss on a
+crash or power failure. For instance, journaled file systems flush cache
+on each meta data update, so they survive power/hardware/software
+failures pretty well.
+
+Since locally on initiators write back caching is always on, if an
+application cares about its data consistency, it does flush the cache
+when necessary or on any write, if open files with O_SYNC. If it doesn't
+care, it doesn't flush the cache. As soon as the cache flushes
+propagated to the storage, write back caching on it doesn't make any
+difference. If application doesn't flush the cache, it's doomed to loose
+data in case of a crash or power failure doesn't matter where this cache
+located, locally or on the storage.
+
+To illustrate that consider, for example, a user who wants to copy /src
+directory to /dst directory reliably, i.e. after the copy finished no
+power failure or software/hardware crash could lead to a loss of the
+data in /dst. There are 2 ways to achieve this. Let's suppose for
+simplicity cp opens files for writing with O_SYNC flag, hence bypassing
+the local cache.
+
+1. Slow. Make the device behind /dst working in write through caching
+mode and then run "cp -a /src /dst".
+
+2. Fast. Let the device behind /dst working in write back caching mode
+and then run "cp -a /src /dst; sync". The reliability of the result is
+the same, but it's much faster than (1). Nobody would care if a crash
+happens during the copy, because after recovery simply leftovers from
+the not completed attempt would be deleted and the operation would be
+restarted from the very beginning.
+
+So, you can see in (2) there is no danger of ANY data loss from the
+write back caching. Moreover, since on practice cp doesn't open files
+for writing with O_SYNC flag, to get the copy done reliably, sync
+command must be called after cp anyway, so enabling write back caching
+wouldn't make any difference for reliability.
+
+Also you can consider it from another side. Modern HDDs have at least
+16MB of cache working in write back mode by default, so for a 10 drives
+RAID it is 160MB of a write back cache. How many people are happy with
+it and how many disabled write back cache of their HDDs? Almost all and
+almost nobody correspondingly? Moreover, many HDDs lie about state of
+their cache and report write through while working in write back mode.
+They are also successfully used.
+
+Note, Linux I/O subsystem guarantees to propagated cache flushes to the
+storage only using data protection barriers, which usually turned off by
+default (see http://lwn.net/Articles/283161). Without barriers enabled
+Linux doesn't provide a guarantee that after sync()/fsync() all written
+data really hit permanent storage. They can be stored in the cache of
+your backstorage devices and, hence, lost on a power failure event.
+Thus, ever with write-through cache mode, you still either need to
+enable barriers on your backend file system on the target (for direct
+/dev/sdX devices this is, indeed, impossible), or need a good UPS to
+protect yourself from not committed data loss. Some info about barriers
+from the XFS point of view could be found at
+http://oss.sgi.com/projects/xfs/faq.html#wcache. On Linux initiators for
+Ext3 and ReiserFS file systems the barrier protection could be turned on
+using "barrier=1" and "barrier=flush" mount options correspondingly. You
+can check if the barriers turn on or off by looking in /proc/mounts.
+Windows and, AFAIK, other UNIX'es don't need any special explicit
+options and do necessary barrier actions on write-back caching devices
+by default.
+
+To limit this data loss with write back caching you can use files in
+/proc/sys/vm to limit amount of unflushed data in the system cache.
+
+If you for some reason have to use VDISK FILEIO devices in write through
+caching mode, don't forget to disable internal caching on their backend
+devices or make sure they have additional battery or supercapacitors
+power supply on board. Otherwise, you still on a power failure would
+loose all the unsaved yet data in the devices internal cache.
+
+Note, on some real-life workloads write through caching might perform
+better, than write back one with the barrier protection turned on.
+
+BLOCKIO VDISK mode
+------------------
+
+This module works best for these types of scenarios:
+
+1) Data that are not aligned to 4K sector boundaries and <4K block sizes
+are used, which is normally found in virtualization environments where
+operating systems start partitions on odd sectors (Windows and it's
+sector 63).
+
+2) Large block data transfers normally found in database loads/dumps and
+streaming media.
+
+3) Advanced relational database systems that perform their own caching
+which prefer or demand direct IO access and, because of the nature of
+their data access, can actually see worse performance with
+non-discriminate caching.
+
+4) Multiple layers of targets were the secondary and above layers need
+to have a consistent view of the primary targets in order to preserve
+data integrity which a page cache backed IO type might not provide
+reliably.
+
+Also it has an advantage over FILEIO that it doesn't copy data between
+the system cache and the commands data buffers, so it saves a
+considerable amount of CPU power and memory bandwidth.
+
+IMPORTANT: Since data in BLOCKIO and FILEIO modes are not consistent between
+========= each other, if you try to use a device in both those modes
+ simultaneously, you will almost instantly corrupt your data
+ on that device.
+
+IMPORTANT: If SCST 1.x BLOCKIO worked by default in NV_CACHE mode, when
+========= each device reported to remote initiators as having write through
+ caching. But if your backend block device has internal write
+ back caching it might create a possibility for data loss of
+ the cached in the internal cache data in case of a power
+ failure. Starting from SCST 2.0 BLOCKIO works by default in
+ non-NV_CACHE mode, when each device reported to remote
+ initiators as having write back caching, and synchronizes the
+ internal device's cache on each SYNCHRONIZE_CACHE command
+ from the initiators. It might lead to some PERFORMANCE LOSS,
+ so if you are are sure in your power supply and want to
+ restore 1.x behavior, your should recreate your BLOCKIO
+ devices in NV_CACHE mode.
+
+Pass-through mode
+-----------------
+
+In the pass-through mode (i.e. using the pass-through device handlers
+scst_disk, scst_tape, etc) SCSI commands, coming from remote initiators,
+are passed to local SCSI devices on target as is, without any
+modifications.
+
+SCST supports 1 to many pass-through, when several initiators can safely
+connect a single pass-through device (a tape, for instance). For such
+cases SCST emulates all the necessary functionality.
+
+In the sysfs interface all real SCSI devices are listed in
+/sys/kernel/scst_tgt/devices in form host:channel:id:lun numbers, for
+instance 1:0:0:0. The recommended way to match those numbers to your
+devices is use of lsscsi utility.
+
+Each pass-through dev handler has in its root subdirectory
+/sys/kernel/scst_tgt/handlers/handler_name, e.g.
+/sys/kernel/scst_tgt/handlers/dev_disk, "mgmt" file. It allows the
+following commands. They can be sent to it using, e.g., echo command.
+
+ - "add_device" - this command assigns SCSI device with
+host:channel:id:lun numbers to this dev handler.
+
+echo "add_device 1:0:0:0" >/sys/kernel/scst_tgt/handlers/dev_disk/mgmt
+
+will assign SCSI device 1:0:0:0 to this dev handler.
+
+ - "del_device" - this command unassigns SCSI device with
+host:channel:id:lun numbers from this dev handler.
+
+As usually, on read the "mgmt" file returns small help about available
+commands.
+
+You need to manually assign each your real SCSI device to the
+corresponding pass-through dev handler using the "add_device" command,
+otherwise the real SCSI devices will not be visible remotely. The
+assignment isn't done automatically, because it could lead to the
+pass-through dev handlers load and initialization problems if any of the
+local real SCSI devices are malfunctioning.
+
+As any other hardware, the local SCSI hardware can not handle commands
+with amount of data and/or segments count in scatter-gather array bigger
+some values. Therefore, when using the pass-through mode you should note
+that values for maximum number of segments and maximum amount of
+transferred data for each SCSI command on devices on initiators can not
+be bigger, than corresponding values of the corresponding SCSI devices
+on the target. Otherwise you will see symptoms like small transfers work
+well, but large ones stall and messages like: "Unable to complete
+command due to SG IO count limitation" are printed in the kernel logs.
+
+You can't control from the user space limit of the scatter-gather
+segments, but for block devices usually it is sufficient if you set on
+the initiators /sys/block/DEVICE_NAME/queue/max_sectors_kb in the same
+or lower value as in /sys/block/DEVICE_NAME/queue/max_hw_sectors_kb for
+the corresponding devices on the target.
+
+For not-block devices SCSI commands are usually generated directly by
+applications, so, if you experience large transfers stalls, you should
+check documentation for your application how to limit the transfer
+sizes.
+
+Another way to solve this issue is to build SG entries with more than 1
+page each. See the following patch as an example:
+http://scst.sourceforge.net/sgv_big_order_alloc.diff
+
+Performance
+-----------
+
+SCST from the very beginning has been designed and implemented to
+provide the best possible performance. Since there is no "one fit all"
+the best performance configuration for different setups and loads, SCST
+provides extensive set of settings to allow to tune it for the best
+performance in each particular case. You don't have to necessary use
+those settings. If you don't, SCST will do very good job to autotune for
+you, so the resulting performance will, in average, be better
+(sometimes, much better) than with other SCSI targets. But in some cases
+you can by manual tuning improve it even more.
+
+Before doing any performance measurements note that performance results
+are very much dependent from your type of load, so it is crucial that
+you choose access mode (FILEIO, BLOCKIO, O_DIRECT, pass-through), which
+suits your needs the best.
+
+In order to get the maximum performance you should:
+
+1. For SCST:
+
+ - Disable in Makefile CONFIG_SCST_STRICT_SERIALIZING, CONFIG_SCST_EXTRACHECKS,
+ CONFIG_SCST_TRACING, CONFIG_SCST_DEBUG*, CONFIG_SCST_STRICT_SECURITY,
+ CONFIG_SCST_MEASURE_LATENCY
+
+2. For target drivers:
+
+ - Disable in Makefiles CONFIG_SCST_EXTRACHECKS, CONFIG_SCST_TRACING,
+ CONFIG_SCST_DEBUG*
+
+3. For device handlers, including VDISK:
+
+ - Disable in Makefile CONFIG_SCST_TRACING and CONFIG_SCST_DEBUG.
+
+4. Make sure you have io_grouping_type option set correctly, especially
+in the following cases:
+
+ - Several initiators share your target's backstorage. It can be a
+ shared LU using some cluster FS, like VMFS, as well as can be
+ different LUs located on the same backstorage (RAID array). For
+ instance, if you have 3 initiators and each of them using its own
+ dedicated FILEIO device file from the same RAID-6 array on the
+ target.
+
+ In this case for the best performance you should have
+ io_grouping_type option set in value "never" in all the LUNs' targets
+ and security groups.
+
+ - Your initiator connected to your target in MPIO mode. In this case for
+ the best performance you should:
+
+ * Either connect all the sessions from the initiator to a single
+ target or security group and have io_grouping_type option set in
+ value "this_group_only" in the target or security group,
+
+ * Or, if it isn't possible to connect all the sessions from the
+ initiator to a single target or security group, assign the same
+ numeric io_grouping_type value for each target/security group this
+ initiator connected to. The exact value itself doesn't matter,
+ important only that all the targets/security groups use the same
+ value.
+
+Don't forget, io_grouping_type makes sense only if you use CFQ I/O
+scheduler on the target and for devices with threads_num >= 0 and, if
+threads_num > 0, with threads_pool_type "per_initiator".
+
+You can check if in your setup io_grouping_type set correctly as well as
+if the "auto" io_grouping_type value works for you by tests like the
+following:
+
+ - For not MPIO case you can run single thread sequential reading, e.g.
+ using buffered dd, from one initiator, then run the same single
+ thread sequential reading from the second initiator in parallel. If
+ io_grouping_type is set correctly the aggregate throughput measured
+ on the target should only slightly decrease as well as all initiators
+ should have nearly equal share of it. If io_grouping_type is not set
+ correctly, the aggregate throughput and/or throughput on any
+ initiator will decrease significantly, in 2 times or even more. For
+ instance, you have 80MB/s single thread sequential reading from the
+ target on any initiator. When then both initiators are reading in
+ parallel you should see on the target aggregate throughput something
+ like 70-75MB/s with correct io_grouping_type and something like
+ 35-40MB/s or 8-10MB/s on any initiator with incorrect.
+
+ - For the MPIO case it's quite easier. With incorrect io_grouping_type
+ you simply won't see performance increase from adding the second
+ session (assuming your hardware is capable to transfer data through
+ both sessions in parallel), or can even see a performance decrease.
+
+5. If you are going to use your target in an VM environment, for
+instance as a shared storage with VMware, make sure all your VMs
+connected to the target via *separate* sessions. For instance, for iSCSI
+it means that each VM has own connection to the target, not all VMs
+connected using a single connection. You can check it using SCST sysfs
+interface. For other transports you should use available facilities,
+like NPIV for Fibre Channel, to make separate sessions for each VM. If
+you miss it, you can greatly loose performance of parallel access to
+your target from different VMs. This isn't related to the case if your
+VMs are using the same shared storage, like with VMFS, for instance. In
+this case all your VM hosts will be connected to the target via separate
+sessions, which is enough.
+
+6. For other target and initiator software parts:
+
+ - Make sure you applied on your kernel all available SCST patches.
+ If for your kernel version this patch doesn't exist, it is strongly
+ recommended to upgrade your kernel to version, for which this patch
+ exists.
+
+ - Don't enable debug/hacking features in the kernel, i.e. use them as
+ they are by default.
+
+ - The default kernel read-ahead and queuing settings are optimized
+ for locally attached disks, therefore they are not optimal if they
+ attached remotely (SCSI target case), which sometimes could lead to
+ unexpectedly low throughput. You should increase read-ahead size to at
+ least 512KB or even more on all initiators and the target.
+
+ You should also limit on all initiators maximum amount of sectors per
+ SCSI command. This tuning is also recommended on targets with large
+ read-ahead values. To do it on Linux, run:
+
+ echo “64” > /sys/block/sdX/queue/max_sectors_kb
+
+ where specify instead of X your imported from target device letter,
+ like 'b', i.e. sdb.
+
+ To increase read-ahead size on Linux, run:
+
+ blockdev --setra N /dev/sdX
+
+ where N is a read-ahead number in 512-byte sectors and X is a device
+ letter like above.
+
+ Note: you need to set read-ahead setting for device sdX again after
+ you changed the maximum amount of sectors per SCSI command for that
+ device.
+
+ Note2: you need to restart SCST after you changed read-ahead settings
+ on the target.
+
+ - You may need to increase amount of requests that OS on initiator
+ sends to the target device. To do it on Linux initiators, run
+
+ echo “64” > /sys/block/sdX/queue/nr_requests
+
+ where X is a device letter like above.
+
+ You may also experiment with other parameters in /sys/block/sdX
+ directory, they also affect performance. If you find the best values,
+ please share them with us.
+
+ - On the target use CFQ IO scheduler. In most cases it has performance
+ advantage over other IO schedulers, sometimes huge (2+ times
+ aggregate throughput increase).
+
+ - It is recommended to turn the kernel preemption off, i.e. set
+ the kernel preemption model to "No Forced Preemption (Server)".
+
+ - Looks like XFS is the best filesystem on the target to store device
+ files, because it allows considerably better linear write throughput,
+ than ext3.
+
+7. For hardware on target.
+
+ - Make sure that your target hardware (e.g. target FC or network card)
+ and underlaying IO hardware (e.g. IO card, like SATA, SCSI or RAID to
+ which your disks connected) don't share the same PCI bus. You can
+ check it using lspci utility. They have to work in parallel, so it
+ will be better if they don't compete for the bus. The problem is not
+ only in the bandwidth, which they have to share, but also in the
+ interaction between cards during that competition. This is very
+ important, because in some cases if target and backend storage
+ controllers share the same PCI bus, it could lead up to 5-10 times
+ less performance, than expected. Moreover, some motherboard (by
+ Supermicro, particularly) have serious stability issues if there are
+ several high speed devices on the same bus working in parallel. If
+ you have no choice, but PCI bus sharing, set in the BIOS PCI latency
+ as low as possible.
+
+8. If you use VDISK IO module in FILEIO mode, NV_CACHE option will
+provide you the best performance. But using it make sure you use a good
+UPS with ability to shutdown the target on the power failure.
+
+Baseline performance numbers you can find in those measurements:
+http://lkml.org/lkml/2009/3/30/283.
+
+IMPORTANT: If you use on initiator some versions of Windows (at least W2K)
+========= you can't get good write performance for VDISK FILEIO devices with
+ default 512 bytes block sizes. You could get about 10% of the
+ expected one. This is because of the partition alignment, which
+ is (simplifying) incompatible with how Linux page cache
+ works, so for each write the corresponding block must be read
+ first. Use 4096 bytes block sizes for VDISK devices and you
+ will have the expected write performance. Actually, any OS on
+ initiators, not only Windows, will benefit from block size
+ max(PAGE_SIZE, BLOCK_SIZE_ON_UNDERLYING_FS), where PAGE_SIZE
+ is the page size, BLOCK_SIZE_ON_UNDERLYING_FS is block size
+ on the underlying FS, on which the device file located, or 0,
+ if a device node is used. Both values are from the target.
+ See also important notes about setting block sizes >512 bytes
+ for VDISK FILEIO devices above.
+
+9. In some cases, for instance working with SSD devices, which consume 100%
+of a single CPU load for data transfers in their internal threads, to
+maximize IOPS it can be needed to assign for those threads dedicated
+CPUs using Linux CPU affinity facilities. No IRQ processing should be
+done on those CPUs. Check that using /proc/interrupts. See taskset
+command and Documentation/IRQ-affinity.txt in your kernel's source tree
+for how to assign IRQ affinity to tasks and IRQs.
+
+The reason for that is that processing of coming commands in SIRQ
+context might be done on the same CPUs as SSD devices' threads doing data
+transfers. As the result, those threads won't receive all the processing
+power of those CPUs and perform worse.
+
+Work if target's backstorage or link is too slow
+------------------------------------------------
+
+Under high I/O load, when your target's backstorage gets overloaded, or
+working over a slow link between initiator and target, when the link
+can't serve all the queued commands on time, you can experience I/O
+stalls or see in the kernel log abort or reset messages.
+
+At first, consider the case of too slow target's backstorage. On some
+seek intensive workloads even fast disks or RAIDs, which able to serve
+continuous data stream on 500+ MB/s speed, can be as slow as 0.3 MB/s.
+Another possible cause for that can be MD/LVM/RAID on your target as in
+http://lkml.org/lkml/2008/2/27/96 (check the whole thread as well).
+
+Thus, in such situations simply processing of one or more commands takes
+too long time, hence initiator decides that they are stuck on the target
+and tries to recover. Particularly, it is known that the default amount
+of simultaneously queued commands (48) is sometimes too high if you do
+intensive writes from VMware on a target disk, which uses LVM in the
+snapshot mode. In this case value like 16 or even 8-10 depending of your
+backstorage speed could be more appropriate.
+
+Unfortunately, currently SCST lacks dynamic I/O flow control, when the
+queue depth on the target is dynamically decreased/increased based on
+how slow/fast the backstorage speed comparing to the target link. So,
+there are 6 possible actions, which you can do to workaround or fix this
+issue in this case:
+
+1. Ignore incoming task management (TM) commands. It's fine if there are
+not too many of them, so average performance isn't hurt and the
+corresponding device isn't getting put offline, i.e. if the backstorage
+isn't too slow.
+
+2. Decrease /sys/block/sdX/device/queue_depth on the initiator in case
+if it's Linux (see below how) or/and SCST_MAX_TGT_DEV_COMMANDS constant
+in scst_priv.h file until you stop seeing incoming TM commands.
+ISCSI-SCST driver also has its own iSCSI specific parameter for that,
+see its README file.
+
+To decrease device queue depth on Linux initiators you can run command:
+
+# echo Y >/sys/block/sdX/device/queue_depth
+
+where Y is the new number of simultaneously queued commands, X - your
+imported device letter, like 'a' for sda device. There are no special
+limitations for Y value, it can be any value from 1 to possible maximum
+(usually, 32), so start from dividing the current value on 2, i.e. set
+16, if /sys/block/sdX/device/queue_depth contains 32.
+
+3. Increase the corresponding timeout on the initiator. For Linux it is
+located in
+/sys/devices/platform/host*/session*/target*:0:0/*:0:0:1/timeout. It can
+be done automatically by an udev rule. For instance, the following
+rule will increase it to 300 seconds:
+
+SUBSYSTEM=="scsi", KERNEL=="[0-9]*:[0-9]*", ACTION=="add", ATTR{type}=="0|7|14", ATTR{timeout}="300"
+
+By default, this timeout is 30 or 60 seconds, depending on your distribution.
+
+4. Try to avoid such seek intensive workloads.
+
+5. Increase speed of the target's backstorage.
+
+6. Implement in SCST dynamic I/O flow control. This will be an ultimate
+solution. See "Dynamic I/O flow control" section on
+http://scst.sourceforge.net/contributing.html page for possible
+implementation idea.
+
+Next, consider the case of too slow link between initiator and target,
+when the initiator tries to simultaneously push N commands to the target
+over it. In this case time to serve those commands, i.e. send or receive
+data for them over the link, can be more, than timeout for any single
+command, hence one or more commands in the tail of the queue can not be
+served on time less than the timeout, so the initiator will decide that
+they are stuck on the target and will try to recover.
+
+To workaround/fix this issue in this case you can use ways 1, 2, 3, 6
+above or (7): increase speed of the link between target and initiator.
+But for some initiators implementations for WRITE commands there might
+be cases when target has no way to detect the issue, so dynamic I/O flow
+control will not be able to help. In those cases you could also need on
+the initiator(s) to either decrease the queue depth (way 2), or increase
+the corresponding timeout (way 3).
+
+Note, that logged messages about QUEUE_FULL status are quite different
+by nature. This is a normal work, just SCSI flow control in action.
+Simply don't enable "mgmt_minor" logging level, or, alternatively, if
+you are confident in the worst case performance of your back-end storage
+or initiator-target link, you can increase SCST_MAX_TGT_DEV_COMMANDS in
+scst_priv.h to 64. Usually initiators don't try to push more commands on
+the target.
+
+Credits
+-------
+
+Thanks to:
+
+ * Mark Buechler <mark.buechler@gmail.com> for a lot of useful
+ suggestions, bug reports and help in debugging.
+
+ * Ming Zhang <mingz@ele.uri.edu> for fixes and comments.
+
+ * Nathaniel Clark <nate@misrule.us> for fixes and comments.
+
+ * Calvin Morrow <calvin.morrow@comcast.net> for testing and useful
+ suggestions.
+
+ * Hu Gang <hugang@soulinfo.com> for the original version of the
+ LSI target driver.
+
+ * Erik Habbinga <erikhabbinga@inphase-tech.com> for fixes and support
+ of the LSI target driver.
+
+ * Ross S. W. Walker <rswwalker@hotmail.com> for the original block IO
+ code and Vu Pham <huongvp@yahoo.com> who updated it for the VDISK dev
+ handler.
+
+ * Michael G. Byrnes <michael.byrnes@hp.com> for fixes.
+
+ * Alessandro Premoli <a.premoli@andxor.it> for fixes
+
+ * Nathan Bullock <nbullock@yottayotta.com> for fixes.
+
+ * Terry Greeniaus <tgreeniaus@yottayotta.com> for fixes.
+
+ * Krzysztof Blaszkowski <kb@sysmikro.com.pl> for many fixes and bug reports.
+
+ * Jianxi Chen <pacers@users.sourceforge.net> for fixing problem with
+ devices >2TB in size
+
+ * Bart Van Assche <bart.vanassche@gmail.com> for a lot of help
+
+ * Daniel Debonzi <debonzi@linux.vnet.ibm.com> for a big part of the
+ initial SCST sysfs tree implementation
+
+Vladislav Bolkhovitin <vst@vlnb.net>, http://scst.sourceforge.net
diff -uprN orig/linux-2.6.35/Documentation/scst/SysfsRules linux-2.6.35/Documentation/scst/SysfsRules
--- orig/linux-2.6.35/Documentation/scst/SysfsRules
+++ linux-2.6.35/Documentation/scst/SysfsRules
@@ -0,0 +1,933 @@
+ SCST SYSFS interface rules
+ ==========================
+
+This file describes SYSFS interface rules, which all SCST target
+drivers, dev handlers and management utilities MUST follow. This allows
+to have a simple, self-documented, target drivers and dev handlers
+independent management interface.
+
+Words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+document are to be interpreted as described in RFC 2119.
+
+In this document "key attribute" means a configuration attribute with
+not default value, which must be configured during the target driver's
+initialization. A key attribute MUST have in the last line keyword
+"[key]". If a default value set to a key attribute, it becomes a regular
+none-key attribute. For instance, iSCSI target has attribute DataDigest.
+Default value for this attribute is "None". It value "CRC32C" is set to
+this attribute, it will become a key attribute. If value "None" is again
+set, this attribute will become back to a none-key attribute.
+
+Each user configurable attribute with a not default value MUST be marked
+as key attribute.
+
+Key attributes SHOULD NOT have sysfs names finished on digits, because
+such names SHOULD be used to store several attributes with the same name
+on the sysfs tree where duplicated names are not allowed. For instance,
+iSCSI targets can have several incoming user names, so the corresponding
+attribute should have sysfs name "IncomingUser". If there are 2 user
+names, they should have sysfs names "IncomingUser" and "IncomingUser1".
+In other words, all "IncomingUser[0-9]*" names should be considered as
+different instances of the same "IncomingUser" attribute.
+
+I. Rules for target drivers
+===========================
+
+SCST core for each target driver (struct scst_tgt_template) creates a
+root subdirectory in /sys/kernel/scst_tgt/targets with name
+scst_tgt_template.name (called "target_driver_name" further in this
+document).
+
+For each target (struct scst_tgt) SCST core creates a root subdirectory
+in /sys/kernel/scst_tgt/targets/target_driver_name with name
+scst_tgt.tgt_name (called "target_name" further in this document).
+
+There are 2 type of targets possible: hardware and virtual targets.
+Hardware targets are targets corresponding to real hardware, for
+instance, a Fibre Channel adapter's port. Virtual targets are hardware
+independent targets, which can be dynamically added or removed, for
+instance, an iSCSI target, or NPIV Fibre Channel target.
+
+A target driver supporting virtual targets MUST support "mgmt" attribute
+and "add_target"/"del_target" commands.
+
+If target driver supports both hardware and virtual targets (for
+instance, an FC adapter supporting NPIV, which has hardware targets for
+its physical ports as well as virtual NPIV targets), it MUST create each
+hardware target with hw_target mark to make SCST core create "hw_target"
+attribute (see below).
+
+Attributes for target drivers
+-----------------------------
+
+A target driver MAY support in its root subdirectory the following
+optional attributes. Target drivers MAY also support there other
+read-only or read-writable attributes.
+
+1. "enabled" - this attribute MUST allow to enable and disable target
+driver as a whole, i.e. if disabled, the target driver MUST NOT accept
+new connections. The goal of this attribute is to allow the target
+driver's initial configuration. For instance, iSCSI target may need to
+have discovery user names and passwords set before it starts serving
+discovery connections.
+
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+On read it MUST return 0, if the target driver is disabled, and 1, if it
+is enabled.
+
+On write it MUST accept '0' character as request to disable and '1' as
+request to enable, but MAY also accept other driver specific commands.
+
+During disabling the target driver MAY close already connected sessions
+in all targets, but this is OPTIONAL.
+
+MUST be 0 by default.
+
+2. "trace_level" - this attribute SHOULD allow to change log level of this
+driver.
+
+This attribute SHOULD have read and write permissions for superuser and be
+read-only for other users.
+
+On read it SHOULD return a help text about available command and log levels.
+
+On write it SHOULD accept commands to change log levels according to the
+help text.
+
+For example:
+
+out_of_mem | minor | pid | line | function | special | mgmt | mgmt_dbg | flow_control | conn
+
+Usage:
+ echo "all|none|default" >trace_level
+ echo "value DEC|0xHEX|0OCT" >trace_level
+ echo "add|del TOKEN" >trace_level
+
+where TOKEN is one of [debug, function, line, pid,
+ entryexit, buff, mem, sg, out_of_mem,
+ special, scsi, mgmt, minor,
+ mgmt_dbg, scsi_serializing,
+ retry, recv_bot, send_bot, recv_top,
+ send_top, d_read, d_write, conn, conn_dbg, iov, pdu, net_page]
+
+3. "version" - this read-only for all attribute SHOULD return version of
+the target driver and some info about its enabled compile time facilities.
+
+For example:
+
+2.0.0
+EXTRACHECKS
+DEBUG
+
+4. "mgmt" - if supported this attribute MUST allow to add and delete
+targets, if virtual targets are supported by this driver, as well as it
+MAY allow to add and delete the target driver's or its targets'
+attributes.
+
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+On read it MUST return a help string describing available commands,
+parameters and attributes.
+
+To achieve that the target driver should just set in its struct
+scst_tgt_template correctly the following fields: mgmt_cmd_help,
+add_target_parameters, tgtt_optional_attributes and
+tgt_optional_attributes.
+
+For example:
+
+Usage: echo "add_target target_name [parameters]" >mgmt
+ echo "del_target target_name" >mgmt
+ echo "add_attribute <attribute> <value>" >mgmt
+ echo "del_attribute <attribute> <value>" >mgmt
+ echo "add_target_attribute target_name <attribute> <value>" >mgmt
+ echo "del_target_attribute target_name <attribute> <value>" >mgmt
+
+where parameters are one or more param_name=value pairs separated by ';'
+
+The following target driver attributes available: IncomingUser, OutgoingUser
+The following target attributes available: IncomingUser, OutgoingUser, allowed_portal
+
+4.1. "add_target" - if supported, this command MUST add new target with
+name "target_name" and specified optional or required parameters. Each
+parameter MUST be in form "parameter=value". All parameters MUST be
+separated by ';' symbol.
+
+All target drivers supporting creation of virtual targets MUST support
+this command.
+
+All target drivers supporting "add_target" command MUST support all
+read-only targets' key attributes as parameters to "add_target" command
+with the attributes' names as parameters' names and the attributes'
+values as parameters' values.
+
+For example:
+
+echo "add_target TARGET1 parameter1=1; parameter2=2" >mgmt
+
+will add target with name "TARGET1" and parameters with names
+"parameter1" and "parameter2" with values 1 and 2 correspondingly.
+
+4.2. "del_target" - if supported, this command MUST delete target with
+name "target_name". If "add_target" command is supported "del_target"
+MUST also be supported.
+
+4.3. "add_attribute" - if supported, this command MUST add a target
+driver's attribute with the specified name and one or more values.
+
+All target drivers supporting run time creation of the target driver's
+key attributes MUST support this command.
+
+For example, for iSCSI target:
+
+echo "add_attribute IncomingUser name password" >mgmt
+
+will add for discovery sessions an incoming user (attribute
+/sys/kernel/scst_tgt/targets/iscsi/IncomingUser) with name "name" and
+password "password".
+
+4.4. "del_attribute" - if supported, this command MUST delete target
+driver's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. For instance, iSCSI target might have several
+incoming users. If not needed, target driver might ignore the values.
+
+If "add_attribute" command is supported "del_attribute" MUST
+also be supported.
+
+4.5. "add_target_attribute" - if supported, this command MUST add new
+attribute for the specified target with the specified name and one or
+more values.
+
+All target drivers supporting run time creation of targets' key
+attributes MUST support this command.
+
+For example:
+
+echo "add_target_attribute iqn.2006-10.net.vlnb:tgt IncomingUser name password" >mgmt
+
+will add for target with name "iqn.2006-10.net.vlnb:tgt" an incoming
+user (attribute
+/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/IncomingUser)
+with name "name" and password "password".
+
+4.6. "del_target_attribute" - if supported, this command MUST delete
+target's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. For instance, iSCSI target might have several
+incoming users. If not needed, target driver might ignore the values.
+
+If "add_target_attribute" command is supported "del_target_attribute"
+MUST also be supported.
+
+Attributes for targets
+----------------------
+
+Each target MAY support in its root subdirectory the following optional
+attributes. Target drivers MAY also support there other read-only or
+read-writable attributes.
+
+1. "enabled" - this attribute MUST allow to enable and disable the
+corresponding target, i.e. if disabled, the target MUST NOT accept new
+connections. The goal of this attribute is to allow the target's initial
+configuration. For instance, each target needs to have its LUNs setup
+before it starts serving initiators. Another example is iSCSI target,
+which may need to have initialized a number of iSCSI parameters before
+it starts accepting new iSCSI connections.
+
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+On read it MUST return 0, if the target is disabled, and 1, if it is
+enabled.
+
+On write it MUST accept '0' character as request to disable and '1' as
+request to enable. Other requests MUST be rejected.
+
+SCST core provides some facilities, which MUST be used to implement this
+attribute.
+
+During disabling the target driver MAY close already connected sessions
+to the target, but this is OPTIONAL.
+
+MUST be 0 by default.
+
+SCST core will automatically create for all targets the following
+attributes:
+
+1. "rel_tgt_id" - allows to read or write SCSI Relative Target Port
+Identifier attribute.
+
+2. "hw_target" - allows to distinguish hardware and virtual targets, if
+the target driver supports both.
+
+To provide OPTIONAL force close session functionality target drivers
+MUST implement it using "force_close" write only session's attribute,
+which on write to it MUST close the corresponding session.
+
+See SCST core's README for more info about those attributes.
+
+II. Rules for dev handlers
+==========================
+
+There are 2 types of dev handlers: parent dev handlers and children dev
+handlers. The children dev handlers depend from the parent dev handlers.
+
+SCST core for each parent dev handler (struct scst_dev_type with
+parent member with value NULL) creates a root subdirectory in
+/sys/kernel/scst_tgt/handlers with name scst_dev_type.name (called
+"dev_handler_name" further in this document).
+
+Parent dev handlers can have one or more subdirectories for children dev
+handlers with names scst_dev_type.name of them.
+
+Only one level of the dev handlers' parent/children hierarchy is
+allowed. Parent dev handlers, which support children dev handlers, MUST
+NOT handle devices and MUST be only placeholders for the children dev
+handlers.
+
+Further in this document children dev handlers or parent dev handlers,
+which don't support children, will be called "end level dev handlers".
+
+End level dev handlers can be recognized by existence of the "mgmt"
+attribute.
+
+For each device (struct scst_device) SCST core creates a root
+subdirectory in /sys/kernel/scst_tgt/devices/device_name with name
+scst_device.virt_name (called "device_name" further in this document).
+
+Attributes for dev handlers
+---------------------------
+
+Each dev handler MUST have it in its root subdirectory "mgmt" attribute,
+which MUST support "add_device" and "del_device" attributes as described
+below.
+
+Parent dev handlers and end level dev handlers without parents MAY
+support in its root subdirectory the following optional attributes. They
+MAY also support there other read-only or read-writable attributes.
+
+1. "trace_level" - this attribute SHOULD allow to change log level of this
+driver.
+
+This attribute SHOULD have read and write permissions for superuser and be
+read-only for other users.
+
+On read it SHOULD return a help text about available command and log levels.
+
+On write it SHOULD accept commands to change log levels according to the
+help text.
+
+For example:
+
+out_of_mem | minor | pid | line | function | special | mgmt | mgmt_dbg
+
+Usage:
+ echo "all|none|default" >trace_level
+ echo "value DEC|0xHEX|0OCT" >trace_level
+ echo "add|del TOKEN" >trace_level
+
+where TOKEN is one of [debug, function, line, pid,
+ entryexit, buff, mem, sg, out_of_mem,
+ special, scsi, mgmt, minor,
+ mgmt_dbg, scsi_serializing,
+ retry, recv_bot, send_bot, recv_top,
+ send_top]
+
+2. "version" - this read-only for all attribute SHOULD return version of
+the dev handler and some info about its enabled compile time facilities.
+
+For example:
+
+2.0.0
+EXTRACHECKS
+DEBUG
+
+End level dev handlers in their root subdirectories MUST support "mgmt"
+attribute and MAY support other read-only or read-writable attributes.
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+Attribute "mgmt" for virtual devices dev handlers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For virtual devices dev handlers "mgmt" attribute MUST allow to add and
+delete devices as well as it MAY allow to add and delete the dev
+handler's or its devices' attributes.
+
+On read it MUST return a help string describing available commands and
+parameters.
+
+To achieve that the dev handler should just set in its struct
+scst_dev_type correctly the following fields: mgmt_cmd_help,
+add_device_parameters, devt_optional_attributes and
+dev_optional_attributes.
+
+For example:
+
+Usage: echo "add_device device_name [parameters]" >mgmt
+ echo "del_device device_name" >mgmt
+ echo "add_attribute <attribute> <value>" >mgmt
+ echo "del_attribute <attribute> <value>" >mgmt
+ echo "add_device_attribute device_name <attribute> <value>" >mgmt
+ echo "del_device_attribute device_name <attribute> <value>" >mgmt
+
+where parameters are one or more param_name=value pairs separated by ';'
+
+The following parameters available: filename, blocksize, write_through, nv_cache, o_direct, read_only, removable
+The following device driver attributes available: AttributeX, AttributeY
+The following device attributes available: AttributeDX, AttributeDY
+
+1. "add_device" - this command MUST add new device with name
+"device_name" and specified optional or required parameters. Each
+parameter MUST be in form "parameter=value". All parameters MUST be
+separated by ';' symbol.
+
+All dev handlers supporting "add_device" command MUST support all
+read-only devices' key attributes as parameters to "add_device" command
+with the attributes' names as parameters' names and the attributes'
+values as parameters' values.
+
+For example:
+
+echo "add_device device1 parameter1=1; parameter2=2" >mgmt
+
+will add device with name "device1" and parameters with names
+"parameter1" and "parameter2" with values 1 and 2 correspondingly.
+
+2. "del_device" - this command MUST delete device with name
+"device_name".
+
+3. "add_attribute" - if supported, this command MUST add a device
+driver's attribute with the specified name and one or more values.
+
+All dev handlers supporting run time creation of the dev handler's
+key attributes MUST support this command.
+
+For example:
+
+echo "add_attribute AttributeX ValueX" >mgmt
+
+will add attribute
+/sys/kernel/scst_tgt/handlers/dev_handler_name/AttributeX with value ValueX.
+
+4. "del_attribute" - if supported, this command MUST delete device
+driver's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. If not needed, dev handler might ignore the
+values.
+
+If "add_attribute" command is supported "del_attribute" MUST also be
+supported.
+
+5. "add_device_attribute" - if supported, this command MUST add new
+attribute for the specified device with the specified name and one or
+more values.
+
+All dev handlers supporting run time creation of devices' key attributes
+MUST support this command.
+
+For example:
+
+echo "add_device_attribute device1 AttributeDX ValueDX" >mgmt
+
+will add for device with name "device1" attribute
+/sys/kernel/scst_tgt/devices/device_name/AttributeDX) with value
+ValueDX.
+
+6. "del_device_attribute" - if supported, this command MUST delete
+device's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. If not needed, dev handler might ignore the
+values.
+
+If "add_device_attribute" command is supported "del_device_attribute"
+MUST also be supported.
+
+Attribute "mgmt" for pass-through devices dev handlers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For pass-through devices dev handlers "mgmt" attribute MUST allow to
+assign and unassign this dev handler to existing SCSI devices via
+"add_device" and "del_device" commands correspondingly.
+
+On read it MUST return a help string describing available commands and
+parameters.
+
+For example:
+
+Usage: echo "add_device H:C:I:L" >mgmt
+ echo "del_device H:C:I:L" >mgmt
+
+1. "add_device" - this command MUST assign SCSI device with
+host:channel:id:lun numbers to this dev handler.
+
+All pass-through dev handlers MUST support this command.
+
+For example:
+
+echo "add_device 1:0:0:0" >mgmt
+
+will assign SCSI device 1:0:0:0 to this dev handler.
+
+2. "del_device" - this command MUST unassign SCSI device with
+host:channel:id:lun numbers from this dev handler.
+
+SCST core will automatically create for all dev handlers the following
+attributes:
+
+1. "type" - SCSI type of device this dev handler can handle.
+
+See SCST core's README for more info about those attributes.
+
+Attributes for devices
+----------------------
+
+Each device MAY support in its root subdirectory any read-only or
+read-writable attributes.
+
+SCST core will automatically create for all devices the following
+attributes:
+
+1. "type" - SCSI type of this device
+
+See SCST core's README for more info about those attributes.
+
+III. Rules for management utilities
+===================================
+
+Rules summary
+-------------
+
+A management utility (scstadmin) SHOULD NOT keep any knowledge specific
+to any device, dev handler, target or target driver. It SHOULD only know
+the common SCST SYSFS rules, which all dev handlers and target drivers
+MUST follow. Namely:
+
+Common rules:
+~~~~~~~~~~~~~
+
+1. All key attributes MUST be marked by mark "[key]" in the last line of
+the attribute.
+
+2. All not key attributes don't matter and SHOULD be ignored.
+
+For target drivers and targets:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. If target driver supports adding new targets, it MUST have "mgmt"
+attribute, which MUST support "add_target" and "del_target" commands as
+specified above.
+
+2. If target driver supports run time adding new key attributes, it MUST
+have "mgmt" attribute, which MUST support "add_attribute" and
+"del_attribute" commands as specified above.
+
+3. If target driver supports both hardware and virtual targets, all its
+hardware targets MUST have "hw_target" attribute with value 1.
+
+4. If target has read-only key attributes, the add_target command MUST
+support them as parameters.
+
+5. If target supports run time adding new key attributes, the target
+driver MUST have "mgmt" attribute, which MUST support
+"add_target_attribute" and "del_target_attribute" commands as specified
+above.
+
+6. Both target drivers and targets MAY support "enable" attribute. If
+supported, after configuring the corresponding target driver or target
+"1" MUST be written to this attribute in the following order: at first,
+for all targets of the target driver, then for the target driver.
+
+For devices and dev handlers:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Each dev handler in its root subdirectory MUST have "mgmt" attribute.
+
+2. Each dev handler MUST support "add_device" and "del_device" commands
+to the "mgmt" attribute as specified above.
+
+3. If dev handler driver supports run time adding new key attributes, it
+MUST support "add_attribute" and "del_attribute" commands to the "mgmt"
+attribute as specified above.
+
+4. All device handlers have links in the root subdirectory pointing to
+their devices.
+
+5. If device has read-only key attributes, the "add_device" command MUST
+support them as parameters.
+
+6. If device supports run time adding new key attributes, its dev
+handler MUST support "add_device_attribute" and "del_device_attribute"
+commands to the "mgmt" attribute as specified above.
+
+7. Each device has "handler" link to its dev handler's root
+subdirectory.
+
+How to distinguish and process different types of attributes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Since management utilities only interested in key attributes, they
+should simply ignore all non-key attributes, like
+devices/device_name/type or targets/target_driver/target_name/version
+doesn't matter if they are read-only or writable. So, the word "key"
+will be omitted later in this section.
+
+At first, any attribute can be a key attribute, doesn't matter how it's
+created.
+
+All the existing on the configuration save time attributes should be
+treated the same. Management utilities shouldn't try to separate anyhow
+them in config files.
+
+1. Always existing attributes
+-----------------------------
+
+There are 2 type of them:
+
+1.1. Writable, like devices/device_name/t10_dev_id or
+targets/qla2x00tgt/target_name/explicit_confirmation. They are the
+simplest and all the values can just be read and written from/to them.
+
+On the configuration save time they can be distinguished as existing.
+
+On the write configuration time they can be distinguished as existing
+and writable.
+
+1.2. Read-only, like devices/fileio_device_name/filename or
+devices/fileio_device_name/block_size. They are also easy to distinguish
+looking at the permissions.
+
+On the configuration save time they can be distinguished the same as for
+(1.1) as existing.
+
+On the write configuration time they can be distinguished as existing
+and read-only. They all should be passed to "add_target" or
+"add_device" commands for virtual targets and devices correspondingly.
+To apply changes to them, the whole corresponding object
+(fileio_device_name in this example) should be removed then recreated.
+
+2. Optional
+-----------
+
+For instance, targets/iscsi/IncomingUser or
+targets/iscsi/target_name/IncomingUser. There are 4 types of them:
+
+2.1. Global for target drivers and dev handlers
+-----------------------------------------------
+
+For instance, targets/iscsi/IncomingUser or handlers/vdisk_fileio/XX
+(none at the moment).
+
+On the configuration save time they can be distinguished the same as for
+(1.1).
+
+On the write configuration time they can be distinguished as one of 4
+choices:
+
+2.1.1. Existing and writable. In this case they should be treated as
+(1.1)
+
+2.1.2. Existing and read-only. In this case they should be treated as
+(1.2).
+
+2.1.3. Not existing. In this case they should be added using
+"add_attribute" command.
+
+2.1.4. Existing in the sysfs tree and not existing in the config file.
+In this case they should be deleted using "del_attribute" command.
+
+2.2. Global for targets
+-----------------------
+
+For instance, targets/iscsi/target_name/IncomingUser.
+
+On the configuration save time they can be distinguished the same as (1.1).
+
+On the write configuration time they can be distinguished as one of 4
+choices:
+
+2.2.1. Existing and writable. In this case they should be treated as
+(1.1).
+
+2.2.2. Existing and read-only. In this case they should be treated as
+(1.2).
+
+2.2.3. Not existing. In this case they should be added using
+"add_target_attribute" command.
+
+2.2.4. Existing in the sysfs tree and not existing in the config file.
+In this case they should be deleted using "del_target_attribute"
+command.
+
+2.3. Global for devices
+-----------------------
+
+For instance, devices/nullio/t10_dev_id.
+
+On the configuration save time they can be distinguished the same as (1.1).
+
+On the write configuration time they can be distinguished as one of 4
+choices:
+
+2.3.1. Existing and writable. In this case they should be treated as
+(1.1)
+
+2.3.2. Existing and read-only. In this case they should be treated as
+(1.2).
+
+2.3.3. Not existing. In this case they should be added using
+"add_device_attribute" command for the corresponding handler, e.g.
+devices/nullio/handler/.
+
+2.3.4. Existing in the sysfs tree and not existing in the config file.
+In this case they should be deleted using "del_device_attribute"
+command for the corresponding handler, e.g. devices/nullio/handler/.
+
+Thus, management utility should implement only 8 procedures: (1.1),
+(1.2), (2.1.3), (2.1.4), (2.2.3), (2.2.4), (2.3.3), (2.3.4).
+
+How to distinguish hardware and virtual targets
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A target is hardware:
+
+ * if exist both "hw_target" attribute and "mgmt" management file
+
+ * or if both don't exist
+
+A target is virtual if there is "mgmt" file and "hw_target" attribute
+doesn't exist.
+
+Algorithm to convert current SCST configuration to config file
+--------------------------------------------------------------
+
+A management utility SHOULD use the following algorithm when converting
+current SCST configuration to a config file.
+
+For all attributes with digits at the end the name, the digits part
+should be omitted from the attributes' names during the store. For
+instance, "IncomingUser1" should be stored as "IncomingUser".
+
+1. Scan all attributes in /sys/kernel/scst_tgt (not recursive) and store
+all found key attributes.
+
+2. Scan all subdirectories of /sys/kernel/scst_tgt/handlers. Each
+subdirectory with "mgmt" attribute is a root subdirectory of a dev
+handler with name the name of the subdirectory. For each found dev
+handler do the following:
+
+2.1. Store the dev handler's name. Store also its path to the root
+subdirectory, if it isn't default (/sys/kernel/scst_tgt/handlers/handler_name).
+
+2.2. Store all dev handler's key attributes.
+
+2.3. Go through all links in the root subdirectory pointing to
+/sys/kernel/scst_tgt/devices and for each device:
+
+2.3.1. For virtual devices dev handlers:
+
+2.3.1.1. Store the name of the device.
+
+2.3.1.2. Store all key attributes. Mark all read only key attributes
+during storing, they will be parameters for the device's creation.
+
+2.3.2. For pass-through devices dev handlers:
+
+2.3.2.1. Store the H:C:I:L name of the device. Optionally, instead of
+the name unique T10 vendor device ID found using command:
+
+sg_inq -p 0x83 /dev/sdX
+
+can be stored. It will allow to reliably find out this device if on the
+next reboot it will have another host:channel:id:lin numbers. The sdX
+device can be found as the last letters after ':' in
+/sys/kernel/scst_tgt/devices/H:C:I:L/scsi_device/device/block:sdX.
+
+3. Go through all subdirectories in /sys/kernel/scst_tgt/targets. For
+each target driver:
+
+3.1. Store the name of the target driver.
+
+3.2. Store all its key attributes.
+
+3.3. Go through all target's subdirectories. For each target:
+
+3.3.1. Store the name of the target.
+
+3.3.2. Mark if the target is hardware or virtual target. The target is a
+hardware target if it has "hw_target" attribute or its target driver
+doesn't have "mgmt" attribute.
+
+3.3.3. Store all key attributes. Mark all read only key attributes
+during storing, they will be parameters for the target's creation.
+
+3.3.4. Scan all "luns" subdirectory and store:
+
+ - LUN.
+
+ - LU's device name.
+
+ - Key attributes.
+
+3.3.5. Scan all "ini_groups" subdirectories. For each group store the following:
+
+ - The group's name.
+
+ - The group's LUNs (the same info as for 3.3.4).
+
+ - The group's initiators.
+
+3.3.6. Store value of "enabled" attribute, if it exists.
+
+3.4. Store value of "enabled" attribute, if it exists.
+
+Algorithm to initialize SCST from config file
+---------------------------------------------
+
+A management utility SHOULD use the following algorithm when doing
+initial SCST configuration from a config file. All necessary kernel
+modules and user space programs supposed to be already loaded, hence all
+dev handlers' entries in /sys/kernel/scst_tgt/handlers as well as all
+entries for hardware targets already created.
+
+1. Set stored values for all stored global (/sys/kernel/scst_tgt)
+attributes.
+
+2. For each dev driver:
+
+2.1. Set stored values for all already existing stored attributes.
+
+2.2. Create not existing stored attributes using "add_attribute" command.
+
+2.3. For virtual devices dev handlers for each stored device:
+
+2.3.1. Create the device using "add_device" command using marked read
+only attributes as parameters.
+
+2.3.2. Set stored values for all already existing stored attributes.
+
+2.3.3. Create not existing stored attributes using
+"add_device_attribute" command.
+
+2.4. For pass-through dev handlers for each stores device:
+
+2.4.1. Assign the corresponding pass-through device to this dev handler
+using "add_device" command.
+
+3. For each target driver:
+
+3.1. Set stored values for all already existing stored attributes.
+
+3.2. Create not existing stored attributes using "add_attribute" command.
+
+3.3. For each target:
+
+3.3.1. For virtual targets:
+
+3.3.1.1. Create the target using "add_target" command using marked read
+only attributes as parameters.
+
+3.3.1.2. Set stored values for all already existing stored attributes.
+
+3.3.1.3. Create not existing stored attributes using
+"add_target_attribute" command.
+
+3.3.2. For hardware targets for each target:
+
+3.3.2.1. Set stored values for all already existing stored attributes.
+
+3.3.2.2. Create not existing stored attributes using
+"add_target_attribute" command.
+
+3.3.3. Setup LUNs
+
+3.3.4. Setup ini_groups, their LUNs and initiators' names.
+
+3.3.5. If this target supports enabling, enable it.
+
+3.4. If this target driver supports enabling, enable it.
+
+Algorithm to apply changes in config file to currently running SCST
+-------------------------------------------------------------------
+
+A management utility SHOULD use the following algorithm when applying
+changes in config file to currently running SCST.
+
+Not all changes can be applied on enabled targets or enabled target
+drivers. From other side, for some target drivers enabling/disabling is
+a very long and disruptive operation, which should be performed as rare
+as possible. Thus, the management utility SHOULD support additional
+option, which, if set, will make it to disable all affected targets
+before doing any change with them.
+
+1. Scan all attributes in /sys/kernel/scst_tgt (not recursive) and
+compare stored and actual key attributes. Apply all changes.
+
+2. Scan all subdirectories of /sys/kernel/scst_tgt/handlers. Each
+subdirectory with "mgmt" attribute is a root subdirectory of a dev
+handler with name the name of the subdirectory. For each found dev
+handler do the following:
+
+2.1. Compare stored and actual key attributes. Apply all changes. Create
+new attributes using "add_attribute" commands and delete not needed any
+more attributes using "del_attribute" command.
+
+2.2. Compare existing devices (links in the root subdirectory pointing
+to /sys/kernel/scst_tgt/devices) and stored devices in the config file.
+Delete all not needed devices and create new devices.
+
+2.3. For all existing devices:
+
+2.3.1. Compare stored and actual key attributes. Apply all changes.
+Create new attributes using "add_device_attribute" commands and delete
+not needed any more attributes using "del_device_attribute" command.
+
+2.3.2. If any read only key attribute for virtual device should be
+changed, delete the devices and recreate it.
+
+3. Go through all subdirectories in /sys/kernel/scst_tgt/targets. For
+each target driver:
+
+3.1. If this target driver should be disabled, disable it.
+
+3.2. Compare stored and actual key attributes. Apply all changes. Create
+new attributes using "add_attribute" commands and delete not needed any
+more attributes using "del_attribute" command.
+
+3.3. Go through all target's subdirectories. Compare existing and stored
+targets. Delete all not needed targets and create new targets.
+
+3.4. For all existing targets:
+
+3.4.1. If this target should be disabled, disable it.
+
+3.4.2. Compare stored and actual key attributes. Apply all changes.
+Create new attributes using "add_target_attribute" commands and delete
+not needed any more attributes using "del_target_attribute" command.
+
+3.4.3. If any read only key attribute for virtual target should be
+changed, delete the target and recreate it.
+
+3.4.4. Scan all "luns" subdirectory and apply necessary changes, using
+"replace" commands to replace one LUN by another, if needed.
+
+3.4.5. Scan all "ini_groups" subdirectories and apply necessary changes,
+using "replace" commands to replace one LUN by another and "move"
+command to move initiator from one group to another, if needed. It MUST
+be done in the following order:
+
+ - Necessary initiators deleted, if they aren't going to be moved
+
+ - LUNs updated
+
+ - Necessary initiators added or moved
+
+3.4.6. If this target should be enabled, enable it.
+
+3.5. If this target driver should be enabled, enable it.
+
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 12/17]: SCST dev handlers' Makefile
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (10 preceding siblings ...)
2010-09-14 14:45 ` [PATCH 11/17]: SCST core's docs Vladislav Bolkhovitin
@ 2010-09-14 14:46 ` Vladislav Bolkhovitin
2010-09-14 14:47 ` [PATCH 13/17]: SCST vdisk dev handler Vladislav Bolkhovitin
` (5 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:46 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST dev handlers' Makefile.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Makefile | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/Makefile linux-2.6.35/drivers/scst/dev_handlers/Makefile
--- orig/linux-2.6.35/drivers/scst/dev_handlers/Makefile
+++ linux-2.6.35/drivers/scst/dev_handlers/Makefile
@@ -0,0 +1,13 @@
+ccflags-y += -Wno-unused-parameter
+
+obj-m := scst_cdrom.o scst_changer.o scst_disk.o scst_modisk.o scst_tape.o \
+ scst_vdisk.o scst_raid.o scst_processor.o
+
+obj-$(CONFIG_SCST_DISK) += scst_disk.o
+obj-$(CONFIG_SCST_TAPE) += scst_tape.o
+obj-$(CONFIG_SCST_CDROM) += scst_cdrom.o
+obj-$(CONFIG_SCST_MODISK) += scst_modisk.o
+obj-$(CONFIG_SCST_CHANGER) += scst_changer.o
+obj-$(CONFIG_SCST_RAID) += scst_raid.o
+obj-$(CONFIG_SCST_PROCESSOR) += scst_processor.o
+obj-$(CONFIG_SCST_VDISK) += scst_vdisk.o
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 13/17]: SCST vdisk dev handler
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (11 preceding siblings ...)
2010-09-14 14:46 ` [PATCH 12/17]: SCST dev handlers' Makefile Vladislav Bolkhovitin
@ 2010-09-14 14:47 ` Vladislav Bolkhovitin
2010-09-14 14:48 ` [PATCH 14/17]: SCST pass-through dev handlers Vladislav Bolkhovitin
` (4 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:47 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST vdisk dev handler. This dev handler allows
to create virtual disks and CDROMs from files on file system.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_vdisk.c | 3966 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 3966 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_vdisk.c linux-2.6.35/drivers/scst/dev_handlers/scst_vdisk.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_vdisk.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_vdisk.c
@@ -0,0 +1,3966 @@
+/*
+ * scst_vdisk.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 Ming Zhang <blackmagic02881 at gmail dot com>
+ * Copyright (C) 2007 Ross Walker <rswwalker at hotmail dot com>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI disk (type 0) and CDROM (type 5) dev handler using files
+ * on file systems or block devices (VDISK)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/unistd.h>
+#include <linux/smp_lock.h>
+#include <linux/spinlock.h>
+#include <linux/init.h>
+#include <linux/uio.h>
+#include <linux/list.h>
+#include <linux/ctype.h>
+#include <linux/writeback.h>
+#include <linux/vmalloc.h>
+#include <asm/atomic.h>
+#include <linux/kthread.h>
+#include <linux/sched.h>
+#include <linux/version.h>
+#include <asm/div64.h>
+#include <asm/unaligned.h>
+#include <linux/slab.h>
+#include <linux/bio.h>
+
+#define LOG_PREFIX "dev_vdisk"
+
+#include <scst/scst.h>
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+#define TRACE_ORDER 0x80000000
+
+static struct scst_trace_log vdisk_local_trace_tbl[] = {
+ { TRACE_ORDER, "order" },
+ { 0, NULL }
+};
+#define trace_log_tbl vdisk_local_trace_tbl
+
+#define VDISK_TRACE_TLB_HELP ", order"
+
+#endif
+
+#include "scst_dev_handler.h"
+
+/* 8 byte ASCII Vendor */
+#define SCST_FIO_VENDOR "SCST_FIO"
+#define SCST_BIO_VENDOR "SCST_BIO"
+/* 4 byte ASCII Product Revision Level - left aligned */
+#define SCST_FIO_REV " 210"
+
+#define MAX_USN_LEN (20+1) /* For '\0' */
+
+#define INQ_BUF_SZ 128
+#define EVPD 0x01
+#define CMDDT 0x02
+
+#define MSENSE_BUF_SZ 256
+#define DBD 0x08 /* disable block descriptor */
+#define WP 0x80 /* write protect */
+#define DPOFUA 0x10 /* DPOFUA bit */
+#define WCE 0x04 /* write cache enable */
+
+#define PF 0x10 /* page format */
+#define SP 0x01 /* save pages */
+#define PS 0x80 /* parameter saveable */
+
+#define BYTE 8
+#define DEF_DISK_BLOCKSIZE_SHIFT 9
+#define DEF_DISK_BLOCKSIZE (1 << DEF_DISK_BLOCKSIZE_SHIFT)
+#define DEF_CDROM_BLOCKSIZE_SHIFT 11
+#define DEF_CDROM_BLOCKSIZE (1 << DEF_CDROM_BLOCKSIZE_SHIFT)
+#define DEF_SECTORS 56
+#define DEF_HEADS 255
+#define LEN_MEM (32 * 1024)
+#define DEF_RD_ONLY 0
+#define DEF_WRITE_THROUGH 0
+#define DEF_NV_CACHE 0
+#define DEF_O_DIRECT 0
+#define DEF_REMOVABLE 0
+
+#define VDISK_NULLIO_SIZE (3LL*1024*1024*1024*1024/2)
+
+#define DEF_TST SCST_CONTR_MODE_SEP_TASK_SETS
+/*
+ * Since we can't control backstorage device's reordering, we have to always
+ * report unrestricted reordering.
+ */
+#define DEF_QUEUE_ALG_WT SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER
+#define DEF_QUEUE_ALG SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER
+#define DEF_SWP 0
+#define DEF_TAS 0
+
+#define DEF_DSENSE SCST_CONTR_MODE_FIXED_SENSE
+
+static unsigned int random_values[256] = {
+ 9862592UL, 3744545211UL, 2348289082UL, 4036111983UL,
+ 435574201UL, 3110343764UL, 2383055570UL, 1826499182UL,
+ 4076766377UL, 1549935812UL, 3696752161UL, 1200276050UL,
+ 3878162706UL, 1783530428UL, 2291072214UL, 125807985UL,
+ 3407668966UL, 547437109UL, 3961389597UL, 969093968UL,
+ 56006179UL, 2591023451UL, 1849465UL, 1614540336UL,
+ 3699757935UL, 479961779UL, 3768703953UL, 2529621525UL,
+ 4157893312UL, 3673555386UL, 4091110867UL, 2193909423UL,
+ 2800464448UL, 3052113233UL, 450394455UL, 3424338713UL,
+ 2113709130UL, 4082064373UL, 3708640918UL, 3841182218UL,
+ 3141803315UL, 1032476030UL, 1166423150UL, 1169646901UL,
+ 2686611738UL, 575517645UL, 2829331065UL, 1351103339UL,
+ 2856560215UL, 2402488288UL, 867847666UL, 8524618UL,
+ 704790297UL, 2228765657UL, 231508411UL, 1425523814UL,
+ 2146764591UL, 1287631730UL, 4142687914UL, 3879884598UL,
+ 729945311UL, 310596427UL, 2263511876UL, 1983091134UL,
+ 3500916580UL, 1642490324UL, 3858376049UL, 695342182UL,
+ 780528366UL, 1372613640UL, 1100993200UL, 1314818946UL,
+ 572029783UL, 3775573540UL, 776262915UL, 2684520905UL,
+ 1007252738UL, 3505856396UL, 1974886670UL, 3115856627UL,
+ 4194842288UL, 2135793908UL, 3566210707UL, 7929775UL,
+ 1321130213UL, 2627281746UL, 3587067247UL, 2025159890UL,
+ 2587032000UL, 3098513342UL, 3289360258UL, 130594898UL,
+ 2258149812UL, 2275857755UL, 3966929942UL, 1521739999UL,
+ 4191192765UL, 958953550UL, 4153558347UL, 1011030335UL,
+ 524382185UL, 4099757640UL, 498828115UL, 2396978754UL,
+ 328688935UL, 826399828UL, 3174103611UL, 3921966365UL,
+ 2187456284UL, 2631406787UL, 3930669674UL, 4282803915UL,
+ 1776755417UL, 374959755UL, 2483763076UL, 844956392UL,
+ 2209187588UL, 3647277868UL, 291047860UL, 3485867047UL,
+ 2223103546UL, 2526736133UL, 3153407604UL, 3828961796UL,
+ 3355731910UL, 2322269798UL, 2752144379UL, 519897942UL,
+ 3430536488UL, 1801511593UL, 1953975728UL, 3286944283UL,
+ 1511612621UL, 1050133852UL, 409321604UL, 1037601109UL,
+ 3352316843UL, 4198371381UL, 617863284UL, 994672213UL,
+ 1540735436UL, 2337363549UL, 1242368492UL, 665473059UL,
+ 2330728163UL, 3443103219UL, 2291025133UL, 3420108120UL,
+ 2663305280UL, 1608969839UL, 2278959931UL, 1389747794UL,
+ 2226946970UL, 2131266900UL, 3856979144UL, 1894169043UL,
+ 2692697628UL, 3797290626UL, 3248126844UL, 3922786277UL,
+ 343705271UL, 3739749888UL, 2191310783UL, 2962488787UL,
+ 4119364141UL, 1403351302UL, 2984008923UL, 3822407178UL,
+ 1932139782UL, 2323869332UL, 2793574182UL, 1852626483UL,
+ 2722460269UL, 1136097522UL, 1005121083UL, 1805201184UL,
+ 2212824936UL, 2979547931UL, 4133075915UL, 2585731003UL,
+ 2431626071UL, 134370235UL, 3763236829UL, 1171434827UL,
+ 2251806994UL, 1289341038UL, 3616320525UL, 392218563UL,
+ 1544502546UL, 2993937212UL, 1957503701UL, 3579140080UL,
+ 4270846116UL, 2030149142UL, 1792286022UL, 366604999UL,
+ 2625579499UL, 790898158UL, 770833822UL, 815540197UL,
+ 2747711781UL, 3570468835UL, 3976195842UL, 1257621341UL,
+ 1198342980UL, 1860626190UL, 3247856686UL, 351473955UL,
+ 993440563UL, 340807146UL, 1041994520UL, 3573925241UL,
+ 480246395UL, 2104806831UL, 1020782793UL, 3362132583UL,
+ 2272911358UL, 3440096248UL, 2356596804UL, 259492703UL,
+ 3899500740UL, 252071876UL, 2177024041UL, 4284810959UL,
+ 2775999888UL, 2653420445UL, 2876046047UL, 1025771859UL,
+ 1994475651UL, 3564987377UL, 4112956647UL, 1821511719UL,
+ 3113447247UL, 455315102UL, 1585273189UL, 2311494568UL,
+ 774051541UL, 1898115372UL, 2637499516UL, 247231365UL,
+ 1475014417UL, 803585727UL, 3911097303UL, 1714292230UL,
+ 476579326UL, 2496900974UL, 3397613314UL, 341202244UL,
+ 807790202UL, 4221326173UL, 499979741UL, 1301488547UL,
+ 1056807896UL, 3525009458UL, 1174811641UL, 3049738746UL,
+};
+
+struct scst_vdisk_dev {
+ uint32_t block_size;
+ uint64_t nblocks;
+ int block_shift;
+ loff_t file_size; /* in bytes */
+
+ /*
+ * This lock can be taken on both SIRQ and thread context, but in
+ * all cases for each particular instance it's taken consistenly either
+ * on SIRQ or thread context. Mix of them is forbidden.
+ */
+ spinlock_t flags_lock;
+
+ /*
+ * Below flags are protected by flags_lock or suspended activity
+ * with scst_vdisk_mutex.
+ */
+ unsigned int rd_only:1;
+ unsigned int wt_flag:1;
+ unsigned int nv_cache:1;
+ unsigned int o_direct_flag:1;
+ unsigned int media_changed:1;
+ unsigned int prevent_allow_medium_removal:1;
+ unsigned int nullio:1;
+ unsigned int blockio:1;
+ unsigned int cdrom_empty:1;
+ unsigned int removable:1;
+
+ int virt_id;
+ char name[16+1]; /* Name of the virtual device,
+ must be <= SCSI Model + 1 */
+ char *filename; /* File name, protected by
+ scst_mutex and suspended activities */
+ uint16_t command_set_version;
+ unsigned int t10_dev_id_set:1; /* true if t10_dev_id manually set */
+ char t10_dev_id[16+8+2]; /* T10 device ID */
+ char usn[MAX_USN_LEN];
+ struct scst_device *dev;
+ struct list_head vdev_list_entry;
+
+ struct scst_dev_type *vdev_devt;
+};
+
+struct scst_vdisk_thr {
+ struct scst_thr_data_hdr hdr;
+ struct file *fd;
+ struct block_device *bdev;
+ struct iovec *iv;
+ int iv_count;
+};
+
+/* Context RA patch supposed to be applied on the kernel */
+#define DEF_NUM_THREADS 8
+static int num_threads = DEF_NUM_THREADS;
+
+module_param_named(num_threads, num_threads, int, S_IRUGO);
+MODULE_PARM_DESC(num_threads, "vdisk threads count");
+
+static int vdisk_attach(struct scst_device *dev);
+static void vdisk_detach(struct scst_device *dev);
+static int vdisk_attach_tgt(struct scst_tgt_dev *tgt_dev);
+static void vdisk_detach_tgt(struct scst_tgt_dev *tgt_dev);
+static int vdisk_parse(struct scst_cmd *);
+static int vdisk_do_job(struct scst_cmd *cmd);
+static int vcdrom_parse(struct scst_cmd *);
+static int vcdrom_exec(struct scst_cmd *cmd);
+static void vdisk_exec_read(struct scst_cmd *cmd,
+ struct scst_vdisk_thr *thr, loff_t loff);
+static void vdisk_exec_write(struct scst_cmd *cmd,
+ struct scst_vdisk_thr *thr, loff_t loff);
+static void blockio_exec_rw(struct scst_cmd *cmd, struct scst_vdisk_thr *thr,
+ u64 lba_start, int write);
+static int blockio_flush(struct block_device *bdev);
+static void vdisk_exec_verify(struct scst_cmd *cmd,
+ struct scst_vdisk_thr *thr, loff_t loff);
+static void vdisk_exec_read_capacity(struct scst_cmd *cmd);
+static void vdisk_exec_read_capacity16(struct scst_cmd *cmd);
+static void vdisk_exec_inquiry(struct scst_cmd *cmd);
+static void vdisk_exec_request_sense(struct scst_cmd *cmd);
+static void vdisk_exec_mode_sense(struct scst_cmd *cmd);
+static void vdisk_exec_mode_select(struct scst_cmd *cmd);
+static void vdisk_exec_log(struct scst_cmd *cmd);
+static void vdisk_exec_read_toc(struct scst_cmd *cmd);
+static void vdisk_exec_prevent_allow_medium_removal(struct scst_cmd *cmd);
+static int vdisk_fsync(struct scst_vdisk_thr *thr, loff_t loff,
+ loff_t len, struct scst_cmd *cmd, struct scst_device *dev);
+static ssize_t vdisk_add_fileio_device(const char *device_name, char *params);
+static ssize_t vdisk_add_blockio_device(const char *device_name, char *params);
+static ssize_t vdisk_add_nullio_device(const char *device_name, char *params);
+static ssize_t vdisk_del_device(const char *device_name);
+static ssize_t vcdrom_add_device(const char *device_name, char *params);
+static ssize_t vcdrom_del_device(const char *device_name);
+static int vdisk_task_mgmt_fn(struct scst_mgmt_cmd *mcmd,
+ struct scst_tgt_dev *tgt_dev);
+static uint64_t vdisk_gen_dev_id_num(const char *virt_dev_name);
+
+/** SYSFS **/
+
+static ssize_t vdev_sysfs_size_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdisk_sysfs_blocksize_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdisk_sysfs_rd_only_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdisk_sysfs_wt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdisk_sysfs_nv_cache_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdisk_sysfs_o_direct_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdisk_sysfs_removable_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdev_sysfs_filename_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdisk_sysfs_resync_size_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count);
+static ssize_t vdev_sysfs_t10_dev_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count);
+static ssize_t vdev_sysfs_t10_dev_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+static ssize_t vdev_sysfs_usn_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+
+static ssize_t vcdrom_sysfs_filename_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count);
+
+static struct kobj_attribute vdev_size_attr =
+ __ATTR(size_mb, S_IRUGO, vdev_sysfs_size_show, NULL);
+static struct kobj_attribute vdisk_blocksize_attr =
+ __ATTR(blocksize, S_IRUGO, vdisk_sysfs_blocksize_show, NULL);
+static struct kobj_attribute vdisk_rd_only_attr =
+ __ATTR(read_only, S_IRUGO, vdisk_sysfs_rd_only_show, NULL);
+static struct kobj_attribute vdisk_wt_attr =
+ __ATTR(write_through, S_IRUGO, vdisk_sysfs_wt_show, NULL);
+static struct kobj_attribute vdisk_nv_cache_attr =
+ __ATTR(nv_cache, S_IRUGO, vdisk_sysfs_nv_cache_show, NULL);
+static struct kobj_attribute vdisk_o_direct_attr =
+ __ATTR(o_direct, S_IRUGO, vdisk_sysfs_o_direct_show, NULL);
+static struct kobj_attribute vdisk_removable_attr =
+ __ATTR(removable, S_IRUGO, vdisk_sysfs_removable_show, NULL);
+static struct kobj_attribute vdisk_filename_attr =
+ __ATTR(filename, S_IRUGO, vdev_sysfs_filename_show, NULL);
+static struct kobj_attribute vdisk_resync_size_attr =
+ __ATTR(resync_size, S_IWUSR, NULL, vdisk_sysfs_resync_size_store);
+static struct kobj_attribute vdev_t10_dev_id_attr =
+ __ATTR(t10_dev_id, S_IWUSR|S_IRUGO, vdev_sysfs_t10_dev_id_show,
+ vdev_sysfs_t10_dev_id_store);
+static struct kobj_attribute vdev_usn_attr =
+ __ATTR(usn, S_IRUGO, vdev_sysfs_usn_show, NULL);
+
+static struct kobj_attribute vcdrom_filename_attr =
+ __ATTR(filename, S_IRUGO|S_IWUSR, vdev_sysfs_filename_show,
+ vcdrom_sysfs_filename_store);
+
+static const struct attribute *vdisk_fileio_attrs[] = {
+ &vdev_size_attr.attr,
+ &vdisk_blocksize_attr.attr,
+ &vdisk_rd_only_attr.attr,
+ &vdisk_wt_attr.attr,
+ &vdisk_nv_cache_attr.attr,
+ &vdisk_o_direct_attr.attr,
+ &vdisk_removable_attr.attr,
+ &vdisk_filename_attr.attr,
+ &vdisk_resync_size_attr.attr,
+ &vdev_t10_dev_id_attr.attr,
+ &vdev_usn_attr.attr,
+ NULL,
+};
+
+static const struct attribute *vdisk_blockio_attrs[] = {
+ &vdev_size_attr.attr,
+ &vdisk_blocksize_attr.attr,
+ &vdisk_rd_only_attr.attr,
+ &vdisk_nv_cache_attr.attr,
+ &vdisk_removable_attr.attr,
+ &vdisk_filename_attr.attr,
+ &vdisk_resync_size_attr.attr,
+ &vdev_t10_dev_id_attr.attr,
+ &vdev_usn_attr.attr,
+ NULL,
+};
+
+static const struct attribute *vdisk_nullio_attrs[] = {
+ &vdev_size_attr.attr,
+ &vdisk_blocksize_attr.attr,
+ &vdisk_rd_only_attr.attr,
+ &vdisk_removable_attr.attr,
+ &vdev_t10_dev_id_attr.attr,
+ &vdev_usn_attr.attr,
+ NULL,
+};
+
+static const struct attribute *vcdrom_attrs[] = {
+ &vdev_size_attr.attr,
+ &vcdrom_filename_attr.attr,
+ &vdev_t10_dev_id_attr.attr,
+ &vdev_usn_attr.attr,
+ NULL,
+};
+
+/* Protects vdisks addition/deletion and related activities, like search */
+static DEFINE_MUTEX(scst_vdisk_mutex);
+static DEFINE_RWLOCK(vdisk_t10_dev_id_rwlock);
+
+/* Protected by scst_vdisk_mutex */
+static LIST_HEAD(vdev_list);
+
+static struct kmem_cache *vdisk_thr_cachep;
+
+/*
+ * Be careful changing "name" field, since it is the name of the corresponding
+ * /sys/kernel/scst_tgt entry, hence a part of user space ABI.
+ */
+
+static struct scst_dev_type vdisk_file_devtype = {
+ .name = "vdisk_fileio",
+ .type = TYPE_DISK,
+ .exec_sync = 1,
+ .threads_num = -1,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = vdisk_attach,
+ .detach = vdisk_detach,
+ .attach_tgt = vdisk_attach_tgt,
+ .detach_tgt = vdisk_detach_tgt,
+ .parse = vdisk_parse,
+ .exec = vdisk_do_job,
+ .task_mgmt_fn = vdisk_task_mgmt_fn,
+ .add_device = vdisk_add_fileio_device,
+ .del_device = vdisk_del_device,
+ .dev_attrs = vdisk_fileio_attrs,
+ .add_device_parameters = "filename, blocksize, write_through, "
+ "nv_cache, o_direct, read_only, removable",
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+ .trace_tbl = vdisk_local_trace_tbl,
+ .trace_tbl_help = VDISK_TRACE_TLB_HELP,
+#endif
+};
+
+static struct kmem_cache *blockio_work_cachep;
+
+static struct scst_dev_type vdisk_blk_devtype = {
+ .name = "vdisk_blockio",
+ .type = TYPE_DISK,
+ .threads_num = 1,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = vdisk_attach,
+ .detach = vdisk_detach,
+ .attach_tgt = vdisk_attach_tgt,
+ .detach_tgt = vdisk_detach_tgt,
+ .parse = vdisk_parse,
+ .exec = vdisk_do_job,
+ .task_mgmt_fn = vdisk_task_mgmt_fn,
+ .add_device = vdisk_add_blockio_device,
+ .del_device = vdisk_del_device,
+ .dev_attrs = vdisk_blockio_attrs,
+ .add_device_parameters = "filename, blocksize, nv_cache, read_only, "
+ "removable",
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+ .trace_tbl = vdisk_local_trace_tbl,
+ .trace_tbl_help = VDISK_TRACE_TLB_HELP,
+#endif
+};
+
+static struct scst_dev_type vdisk_null_devtype = {
+ .name = "vdisk_nullio",
+ .type = TYPE_DISK,
+ .threads_num = 0,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = vdisk_attach,
+ .detach = vdisk_detach,
+ .attach_tgt = vdisk_attach_tgt,
+ .detach_tgt = vdisk_detach_tgt,
+ .parse = vdisk_parse,
+ .exec = vdisk_do_job,
+ .task_mgmt_fn = vdisk_task_mgmt_fn,
+ .add_device = vdisk_add_nullio_device,
+ .del_device = vdisk_del_device,
+ .dev_attrs = vdisk_nullio_attrs,
+ .add_device_parameters = "blocksize, read_only, removable",
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+ .trace_tbl = vdisk_local_trace_tbl,
+ .trace_tbl_help = VDISK_TRACE_TLB_HELP,
+#endif
+};
+
+static struct scst_dev_type vcdrom_devtype = {
+ .name = "vcdrom",
+ .type = TYPE_ROM,
+ .exec_sync = 1,
+ .threads_num = -1,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = vdisk_attach,
+ .detach = vdisk_detach,
+ .attach_tgt = vdisk_attach_tgt,
+ .detach_tgt = vdisk_detach_tgt,
+ .parse = vcdrom_parse,
+ .exec = vcdrom_exec,
+ .task_mgmt_fn = vdisk_task_mgmt_fn,
+ .add_device = vcdrom_add_device,
+ .del_device = vcdrom_del_device,
+ .dev_attrs = vcdrom_attrs,
+ .add_device_parameters = NULL,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+ .trace_tbl = vdisk_local_trace_tbl,
+ .trace_tbl_help = VDISK_TRACE_TLB_HELP,
+#endif
+};
+
+static struct scst_vdisk_thr nullio_thr_data;
+
+static const char *vdev_get_filename(const struct scst_vdisk_dev *virt_dev)
+{
+ if (virt_dev->filename != NULL)
+ return virt_dev->filename;
+ else
+ return "none";
+}
+
+/* Returns fd, use IS_ERR(fd) to get error status */
+static struct file *vdev_open_fd(const struct scst_vdisk_dev *virt_dev)
+{
+ int open_flags = 0;
+ struct file *fd;
+
+ if (virt_dev->dev->rd_only)
+ open_flags |= O_RDONLY;
+ else
+ open_flags |= O_RDWR;
+ if (virt_dev->o_direct_flag)
+ open_flags |= O_DIRECT;
+ if (virt_dev->wt_flag && !virt_dev->nv_cache)
+ open_flags |= O_SYNC;
+ TRACE_DBG("Opening file %s, flags 0x%x",
+ virt_dev->filename, open_flags);
+ fd = filp_open(virt_dev->filename, O_LARGEFILE | open_flags, 0600);
+ return fd;
+}
+
+static void vdisk_blockio_check_flush_support(struct scst_vdisk_dev *virt_dev)
+{
+ struct inode *inode;
+ struct file *fd;
+
+ if (!virt_dev->blockio || virt_dev->rd_only || virt_dev->nv_cache)
+ goto out;
+
+ fd = filp_open(virt_dev->filename, O_LARGEFILE, 0600);
+ if (IS_ERR(fd)) {
+ PRINT_ERROR("filp_open(%s) returned error %ld",
+ virt_dev->filename, PTR_ERR(fd));
+ goto out;
+ }
+
+ inode = fd->f_dentry->d_inode;
+
+ if (!S_ISBLK(inode->i_mode)) {
+ PRINT_ERROR("%s is NOT a block device", virt_dev->filename);
+ goto out_close;
+ }
+
+ if (blockio_flush(inode->i_bdev) != 0) {
+ PRINT_WARNING("Device %s doesn't support barriers, switching "
+ "to NV_CACHE mode. Read README for more details.",
+ virt_dev->filename);
+ virt_dev->nv_cache = 1;
+ }
+
+out_close:
+ filp_close(fd, NULL);
+
+out:
+ return;
+}
+
+/* Returns 0 on success and file size in *file_size, error code otherwise */
+static int vdisk_get_file_size(const char *filename, bool blockio,
+ loff_t *file_size)
+{
+ struct inode *inode;
+ int res = 0;
+ struct file *fd;
+
+ *file_size = 0;
+
+ fd = filp_open(filename, O_LARGEFILE | O_RDONLY, 0600);
+ if (IS_ERR(fd)) {
+ res = PTR_ERR(fd);
+ PRINT_ERROR("filp_open(%s) returned error %d", filename, res);
+ goto out;
+ }
+
+ inode = fd->f_dentry->d_inode;
+
+ if (blockio && !S_ISBLK(inode->i_mode)) {
+ PRINT_ERROR("File %s is NOT a block device", filename);
+ res = -EINVAL;
+ goto out_close;
+ }
+
+ if (S_ISREG(inode->i_mode))
+ /* Nothing to do */;
+ else if (S_ISBLK(inode->i_mode))
+ inode = inode->i_bdev->bd_inode;
+ else {
+ res = -EINVAL;
+ goto out_close;
+ }
+
+ *file_size = inode->i_size;
+
+out_close:
+ filp_close(fd, NULL);
+
+out:
+ return res;
+}
+
+static int vdisk_attach(struct scst_device *dev)
+{
+ int res = 0;
+ loff_t err;
+ struct scst_vdisk_dev *virt_dev = NULL, *vv;
+
+ TRACE_DBG("virt_id %d (%s)", dev->virt_id, dev->virt_name);
+
+ if (dev->virt_id == 0) {
+ PRINT_ERROR("%s", "Not a virtual device");
+ res = -EINVAL;
+ goto out;
+ }
+
+ /*
+ * scst_vdisk_mutex must be already taken before
+ * scst_register_virtual_device()
+ */
+ list_for_each_entry(vv, &vdev_list, vdev_list_entry) {
+ if (strcmp(vv->name, dev->virt_name) == 0) {
+ virt_dev = vv;
+ break;
+ }
+ }
+
+ if (virt_dev == NULL) {
+ PRINT_ERROR("Device %s not found", dev->virt_name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ virt_dev->dev = dev;
+
+ dev->rd_only = virt_dev->rd_only;
+
+ if (!virt_dev->cdrom_empty) {
+ if (virt_dev->nullio)
+ err = VDISK_NULLIO_SIZE;
+ else {
+ res = vdisk_get_file_size(virt_dev->filename,
+ virt_dev->blockio, &err);
+ if (res != 0)
+ goto out;
+ }
+ virt_dev->file_size = err;
+
+ TRACE_DBG("size of file: %lld", (long long unsigned int)err);
+
+ vdisk_blockio_check_flush_support(virt_dev);
+ } else
+ virt_dev->file_size = 0;
+
+ virt_dev->nblocks = virt_dev->file_size >> virt_dev->block_shift;
+
+ if (!virt_dev->cdrom_empty) {
+ PRINT_INFO("Attached SCSI target virtual %s %s "
+ "(file=\"%s\", fs=%lldMB, bs=%d, nblocks=%lld,"
+ " cyln=%lld%s)",
+ (dev->type == TYPE_DISK) ? "disk" : "cdrom",
+ virt_dev->name, vdev_get_filename(virt_dev),
+ virt_dev->file_size >> 20, virt_dev->block_size,
+ (long long unsigned int)virt_dev->nblocks,
+ (long long unsigned int)virt_dev->nblocks/64/32,
+ virt_dev->nblocks < 64*32
+ ? " !WARNING! cyln less than 1" : "");
+ } else {
+ PRINT_INFO("Attached empty SCSI target virtual cdrom %s",
+ virt_dev->name);
+ }
+
+ dev->dh_priv = virt_dev;
+
+ dev->tst = DEF_TST;
+ dev->d_sense = DEF_DSENSE;
+ if (virt_dev->wt_flag && !virt_dev->nv_cache)
+ dev->queue_alg = DEF_QUEUE_ALG_WT;
+ else
+ dev->queue_alg = DEF_QUEUE_ALG;
+ dev->swp = DEF_SWP;
+ dev->tas = DEF_TAS;
+
+out:
+ return res;
+}
+
+/* scst_mutex supposed to be held */
+static void vdisk_detach(struct scst_device *dev)
+{
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)dev->dh_priv;
+
+ TRACE_DBG("virt_id %d", dev->virt_id);
+
+ PRINT_INFO("Detached virtual device %s (\"%s\")",
+ virt_dev->name, vdev_get_filename(virt_dev));
+
+ /* virt_dev will be freed by the caller */
+ dev->dh_priv = NULL;
+ return;
+}
+
+static void vdisk_free_thr_data(struct scst_thr_data_hdr *d)
+{
+ struct scst_vdisk_thr *thr =
+ container_of(d, struct scst_vdisk_thr, hdr);
+
+ if (thr->fd)
+ filp_close(thr->fd, NULL);
+
+ kfree(thr->iv);
+
+ kmem_cache_free(vdisk_thr_cachep, thr);
+ return;
+}
+
+static struct scst_vdisk_thr *vdisk_init_thr_data(
+ struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_vdisk_thr *res;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)tgt_dev->dev->dh_priv;
+
+ EXTRACHECKS_BUG_ON(virt_dev->nullio);
+
+ res = kmem_cache_zalloc(vdisk_thr_cachep, GFP_KERNEL);
+ if (res == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Unable to allocate struct "
+ "scst_vdisk_thr");
+ goto out;
+ }
+
+ if (!virt_dev->cdrom_empty) {
+ res->fd = vdev_open_fd(virt_dev);
+ if (IS_ERR(res->fd)) {
+ PRINT_ERROR("filp_open(%s) returned an error %ld",
+ virt_dev->filename, PTR_ERR(res->fd));
+ goto out_free;
+ }
+ if (virt_dev->blockio)
+ res->bdev = res->fd->f_dentry->d_inode->i_bdev;
+ else
+ res->bdev = NULL;
+ } else
+ res->fd = NULL;
+
+ scst_add_thr_data(tgt_dev, &res->hdr, vdisk_free_thr_data);
+
+out:
+ return res;
+
+out_free:
+ kmem_cache_free(vdisk_thr_cachep, res);
+ res = NULL;
+ goto out;
+}
+
+static int vdisk_attach_tgt(struct scst_tgt_dev *tgt_dev)
+{
+ int res = 0;
+
+ /* Nothing to do */
+ return res;
+}
+
+static void vdisk_detach_tgt(struct scst_tgt_dev *tgt_dev)
+{
+
+ scst_del_all_thr_data(tgt_dev);
+ return;
+}
+
+static int vdisk_do_job(struct scst_cmd *cmd)
+{
+ int rc, res;
+ uint64_t lba_start = 0;
+ loff_t data_len = 0;
+ uint8_t *cdb = cmd->cdb;
+ int opcode = cdb[0];
+ loff_t loff;
+ struct scst_device *dev = cmd->dev;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)dev->dh_priv;
+ struct scst_thr_data_hdr *d;
+ struct scst_vdisk_thr *thr = NULL;
+ int fua = 0;
+
+ switch (cmd->queue_type) {
+ case SCST_CMD_QUEUE_ORDERED:
+ TRACE(TRACE_ORDER, "ORDERED cmd %p (op %x)", cmd, cmd->cdb[0]);
+ break;
+ case SCST_CMD_QUEUE_HEAD_OF_QUEUE:
+ TRACE(TRACE_ORDER, "HQ cmd %p (op %x)", cmd, cmd->cdb[0]);
+ break;
+ default:
+ break;
+ }
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ if (!virt_dev->nullio) {
+ d = scst_find_thr_data(tgt_dev);
+ if (unlikely(d == NULL)) {
+ thr = vdisk_init_thr_data(tgt_dev);
+ if (thr == NULL) {
+ scst_set_busy(cmd);
+ goto out_compl;
+ }
+ scst_thr_data_get(&thr->hdr);
+ } else
+ thr = container_of(d, struct scst_vdisk_thr, hdr);
+ } else {
+ thr = &nullio_thr_data;
+ scst_thr_data_get(&thr->hdr);
+ }
+
+ switch (opcode) {
+ case READ_6:
+ case WRITE_6:
+ case VERIFY_6:
+ lba_start = (((cdb[1] & 0x1f) << (BYTE * 2)) +
+ (cdb[2] << (BYTE * 1)) +
+ (cdb[3] << (BYTE * 0)));
+ data_len = cmd->bufflen;
+ break;
+ case READ_10:
+ case READ_12:
+ case WRITE_10:
+ case WRITE_12:
+ case VERIFY:
+ case WRITE_VERIFY:
+ case WRITE_VERIFY_12:
+ case VERIFY_12:
+ lba_start |= ((u64)cdb[2]) << 24;
+ lba_start |= ((u64)cdb[3]) << 16;
+ lba_start |= ((u64)cdb[4]) << 8;
+ lba_start |= ((u64)cdb[5]);
+ data_len = cmd->bufflen;
+ break;
+ case READ_16:
+ case WRITE_16:
+ case WRITE_VERIFY_16:
+ case VERIFY_16:
+ lba_start |= ((u64)cdb[2]) << 56;
+ lba_start |= ((u64)cdb[3]) << 48;
+ lba_start |= ((u64)cdb[4]) << 40;
+ lba_start |= ((u64)cdb[5]) << 32;
+ lba_start |= ((u64)cdb[6]) << 24;
+ lba_start |= ((u64)cdb[7]) << 16;
+ lba_start |= ((u64)cdb[8]) << 8;
+ lba_start |= ((u64)cdb[9]);
+ data_len = cmd->bufflen;
+ break;
+ case SYNCHRONIZE_CACHE:
+ lba_start |= ((u64)cdb[2]) << 24;
+ lba_start |= ((u64)cdb[3]) << 16;
+ lba_start |= ((u64)cdb[4]) << 8;
+ lba_start |= ((u64)cdb[5]);
+ data_len = ((cdb[7] << (BYTE * 1)) + (cdb[8] << (BYTE * 0)))
+ << virt_dev->block_shift;
+ if (data_len == 0)
+ data_len = virt_dev->file_size -
+ ((loff_t)lba_start << virt_dev->block_shift);
+ break;
+ }
+
+ loff = (loff_t)lba_start << virt_dev->block_shift;
+ TRACE_DBG("cmd %p, lba_start %lld, loff %lld, data_len %lld", cmd,
+ (long long unsigned int)lba_start,
+ (long long unsigned int)loff,
+ (long long unsigned int)data_len);
+ if (unlikely(loff < 0) || unlikely(data_len < 0) ||
+ unlikely((loff + data_len) > virt_dev->file_size)) {
+ PRINT_INFO("Access beyond the end of the device "
+ "(%lld of %lld, len %lld)",
+ (long long unsigned int)loff,
+ (long long unsigned int)virt_dev->file_size,
+ (long long unsigned int)data_len);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_block_out_range_error));
+ goto out_compl;
+ }
+
+ switch (opcode) {
+ case WRITE_10:
+ case WRITE_12:
+ case WRITE_16:
+ fua = (cdb[1] & 0x8);
+ if (fua) {
+ TRACE(TRACE_ORDER, "FUA: loff=%lld, "
+ "data_len=%lld", (long long unsigned int)loff,
+ (long long unsigned int)data_len);
+ }
+ break;
+ }
+
+ switch (opcode) {
+ case READ_6:
+ case READ_10:
+ case READ_12:
+ case READ_16:
+ if (virt_dev->blockio) {
+ blockio_exec_rw(cmd, thr, lba_start, 0);
+ goto out_thr;
+ } else
+ vdisk_exec_read(cmd, thr, loff);
+ break;
+ case WRITE_6:
+ case WRITE_10:
+ case WRITE_12:
+ case WRITE_16:
+ {
+ if (virt_dev->blockio) {
+ blockio_exec_rw(cmd, thr, lba_start, 1);
+ goto out_thr;
+ } else
+ vdisk_exec_write(cmd, thr, loff);
+ /* O_SYNC flag is used for WT devices */
+ if (fua)
+ vdisk_fsync(thr, loff, data_len, cmd, dev);
+ break;
+ }
+ case WRITE_VERIFY:
+ case WRITE_VERIFY_12:
+ case WRITE_VERIFY_16:
+ {
+ /* ToDo: BLOCKIO VERIFY */
+ vdisk_exec_write(cmd, thr, loff);
+ /* O_SYNC flag is used for WT devices */
+ if (scsi_status_is_good(cmd->status))
+ vdisk_exec_verify(cmd, thr, loff);
+ break;
+ }
+ case SYNCHRONIZE_CACHE:
+ {
+ int immed = cdb[1] & 0x2;
+ TRACE(TRACE_ORDER, "SYNCHRONIZE_CACHE: "
+ "loff=%lld, data_len=%lld, immed=%d",
+ (long long unsigned int)loff,
+ (long long unsigned int)data_len, immed);
+ if (immed) {
+ scst_cmd_get(cmd); /* to protect dev */
+ cmd->completed = 1;
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT,
+ SCST_CONTEXT_SAME);
+ vdisk_fsync(thr, loff, data_len, NULL, dev);
+ /* ToDo: vdisk_fsync() error processing */
+ scst_cmd_put(cmd);
+ goto out_thr;
+ } else {
+ vdisk_fsync(thr, loff, data_len, cmd, dev);
+ break;
+ }
+ }
+ case VERIFY_6:
+ case VERIFY:
+ case VERIFY_12:
+ case VERIFY_16:
+ vdisk_exec_verify(cmd, thr, loff);
+ break;
+ case MODE_SENSE:
+ case MODE_SENSE_10:
+ vdisk_exec_mode_sense(cmd);
+ break;
+ case MODE_SELECT:
+ case MODE_SELECT_10:
+ vdisk_exec_mode_select(cmd);
+ break;
+ case LOG_SELECT:
+ case LOG_SENSE:
+ vdisk_exec_log(cmd);
+ break;
+ case ALLOW_MEDIUM_REMOVAL:
+ vdisk_exec_prevent_allow_medium_removal(cmd);
+ break;
+ case READ_TOC:
+ vdisk_exec_read_toc(cmd);
+ break;
+ case START_STOP:
+ vdisk_fsync(thr, 0, virt_dev->file_size, cmd, dev);
+ break;
+ case RESERVE:
+ case RESERVE_10:
+ case RELEASE:
+ case RELEASE_10:
+ case TEST_UNIT_READY:
+ break;
+ case INQUIRY:
+ vdisk_exec_inquiry(cmd);
+ break;
+ case REQUEST_SENSE:
+ vdisk_exec_request_sense(cmd);
+ break;
+ case READ_CAPACITY:
+ vdisk_exec_read_capacity(cmd);
+ break;
+ case SERVICE_ACTION_IN:
+ if ((cmd->cdb[1] & 0x1f) == SAI_READ_CAPACITY_16) {
+ vdisk_exec_read_capacity16(cmd);
+ break;
+ }
+ /* else go through */
+ case REPORT_LUNS:
+ default:
+ TRACE_DBG("Invalid opcode %d", opcode);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ }
+
+out_compl:
+ cmd->completed = 1;
+
+out_done:
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+
+out_thr:
+ if (likely(thr != NULL))
+ scst_thr_data_put(&thr->hdr);
+
+ res = SCST_EXEC_COMPLETED;
+ return res;
+}
+
+static int vdisk_get_block_shift(struct scst_cmd *cmd)
+{
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ return virt_dev->block_shift;
+}
+
+static int vdisk_parse(struct scst_cmd *cmd)
+{
+ scst_sbc_generic_parse(cmd, vdisk_get_block_shift);
+ return SCST_CMD_STATE_DEFAULT;
+}
+
+static int vcdrom_parse(struct scst_cmd *cmd)
+{
+ scst_cdrom_generic_parse(cmd, vdisk_get_block_shift);
+ return SCST_CMD_STATE_DEFAULT;
+}
+
+static int vcdrom_exec(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_COMPLETED;
+ int opcode = cmd->cdb[0];
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ if (virt_dev->cdrom_empty && (opcode != INQUIRY)) {
+ TRACE_DBG("%s", "CDROM empty");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_not_ready));
+ goto out_done;
+ }
+
+ if (virt_dev->media_changed && scst_is_ua_command(cmd)) {
+ spin_lock(&virt_dev->flags_lock);
+ if (virt_dev->media_changed) {
+ virt_dev->media_changed = 0;
+ TRACE_DBG("%s", "Reporting media changed");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_medium_changed_UA));
+ spin_unlock(&virt_dev->flags_lock);
+ goto out_done;
+ }
+ spin_unlock(&virt_dev->flags_lock);
+ }
+
+ res = vdisk_do_job(cmd);
+
+out:
+ return res;
+
+out_done:
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out;
+}
+
+static uint64_t vdisk_gen_dev_id_num(const char *virt_dev_name)
+{
+ unsigned int dev_id_num, i;
+
+ for (dev_id_num = 0, i = 0; i < strlen(virt_dev_name); i++) {
+ unsigned int rv = random_values[(int)(virt_dev_name[i])];
+ /* Do some rotating of the bits */
+ dev_id_num ^= ((rv << i) | (rv >> (32 - i)));
+ }
+
+ return ((uint64_t)scst_get_setup_id() << 32) | dev_id_num;
+}
+
+static void vdisk_exec_inquiry(struct scst_cmd *cmd)
+{
+ int32_t length, i, resp_len = 0;
+ uint8_t *address;
+ uint8_t *buf;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+
+ /* ToDo: Performance Boost:
+ * 1. remove kzalloc, buf
+ * 2. do all checks before touching *address
+ * 3. zero *address
+ * 4. write directly to *address
+ */
+
+ buf = kzalloc(INQ_BUF_SZ, GFP_KERNEL);
+ if (buf == NULL) {
+ scst_set_busy(cmd);
+ goto out;
+ }
+
+ length = scst_get_buf_first(cmd, &address);
+ TRACE_DBG("length %d", length);
+ if (unlikely(length <= 0)) {
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_first() failed: %d", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out_free;
+ }
+
+ if (cmd->cdb[1] & CMDDT) {
+ TRACE_DBG("%s", "INQUIRY: CMDDT is unsupported");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_put;
+ }
+
+ buf[0] = cmd->dev->type; /* type dev */
+ if (virt_dev->removable)
+ buf[1] = 0x80; /* removable */
+ /* Vital Product */
+ if (cmd->cdb[1] & EVPD) {
+ if (0 == cmd->cdb[2]) {
+ /* supported vital product data pages */
+ buf[3] = 3;
+ buf[4] = 0x0; /* this page */
+ buf[5] = 0x80; /* unit serial number */
+ buf[6] = 0x83; /* device identification */
+ if (virt_dev->dev->type == TYPE_DISK) {
+ buf[3] += 1;
+ buf[7] = 0xB0; /* block limits */
+ }
+ resp_len = buf[3] + 4;
+ } else if (0x80 == cmd->cdb[2]) {
+ /* unit serial number */
+ int usn_len = strlen(virt_dev->usn);
+ buf[1] = 0x80;
+ buf[3] = usn_len;
+ strncpy(&buf[4], virt_dev->usn, usn_len);
+ resp_len = buf[3] + 4;
+ } else if (0x83 == cmd->cdb[2]) {
+ /* device identification */
+ int num = 4;
+
+ buf[1] = 0x83;
+ /* T10 vendor identifier field format (faked) */
+ buf[num + 0] = 0x2; /* ASCII */
+ buf[num + 1] = 0x1; /* Vendor ID */
+ if (virt_dev->blockio)
+ memcpy(&buf[num + 4], SCST_BIO_VENDOR, 8);
+ else
+ memcpy(&buf[num + 4], SCST_FIO_VENDOR, 8);
+
+ read_lock_bh(&vdisk_t10_dev_id_rwlock);
+ i = strlen(virt_dev->t10_dev_id);
+ memcpy(&buf[num + 12], virt_dev->t10_dev_id, i);
+ read_unlock_bh(&vdisk_t10_dev_id_rwlock);
+
+ buf[num + 3] = 8 + i;
+ num += buf[num + 3];
+
+ num += 4;
+
+ /*
+ * Relative target port identifier
+ */
+ buf[num + 0] = 0x01; /* binary */
+ /* Relative target port id */
+ buf[num + 1] = 0x10 | 0x04;
+
+ put_unaligned(cpu_to_be16(cmd->tgt->rel_tgt_id),
+ (__be16 *)&buf[num + 4 + 2]);
+
+ buf[num + 3] = 4;
+ num += buf[num + 3];
+
+ num += 4;
+
+ /*
+ * IEEE id
+ */
+ buf[num + 0] = 0x01; /* binary */
+
+ /* EUI-64 */
+ buf[num + 1] = 0x02;
+ buf[num + 2] = 0x00;
+ buf[num + 3] = 0x08;
+
+ /* IEEE id */
+ buf[num + 4] = virt_dev->t10_dev_id[0];
+ buf[num + 5] = virt_dev->t10_dev_id[1];
+ buf[num + 6] = virt_dev->t10_dev_id[2];
+
+ /* IEEE ext id */
+ buf[num + 7] = virt_dev->t10_dev_id[3];
+ buf[num + 8] = virt_dev->t10_dev_id[4];
+ buf[num + 9] = virt_dev->t10_dev_id[5];
+ buf[num + 10] = virt_dev->t10_dev_id[6];
+ buf[num + 11] = virt_dev->t10_dev_id[7];
+ num += buf[num + 3];
+
+ resp_len = num;
+ buf[2] = (resp_len >> 8) & 0xFF;
+ buf[3] = resp_len & 0xFF;
+ resp_len += 4;
+ } else if ((0xB0 == cmd->cdb[2]) &&
+ (virt_dev->dev->type == TYPE_DISK)) {
+ /* block limits */
+ int max_transfer;
+ buf[1] = 0xB0;
+ buf[3] = 0x1C;
+ /* Optimal transfer granuality is PAGE_SIZE */
+ put_unaligned(cpu_to_be16(max_t(int,
+ PAGE_SIZE/virt_dev->block_size, 1)),
+ (uint16_t *)&buf[6]);
+ /* Max transfer len is min of sg limit and 8M */
+ max_transfer = min_t(int,
+ cmd->tgt_dev->max_sg_cnt << PAGE_SHIFT,
+ 8*1024*1024) / virt_dev->block_size;
+ put_unaligned(cpu_to_be32(max_transfer),
+ (uint32_t *)&buf[8]);
+ /*
+ * Let's have optimal transfer len 1MB. Better to not
+ * set it at all, because we don't have such limit,
+ * but some initiators may not understand that (?).
+ * From other side, too big transfers are not optimal,
+ * because SGV cache supports only <4M buffers.
+ */
+ put_unaligned(cpu_to_be32(min_t(int,
+ max_transfer,
+ 1*1024*1024 / virt_dev->block_size)),
+ (uint32_t *)&buf[12]);
+ resp_len = buf[3] + 4;
+ } else {
+ TRACE_DBG("INQUIRY: Unsupported EVPD page %x",
+ cmd->cdb[2]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_put;
+ }
+ } else {
+ int len, num;
+
+ if (cmd->cdb[2] != 0) {
+ TRACE_DBG("INQUIRY: Unsupported page %x", cmd->cdb[2]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_put;
+ }
+
+ buf[2] = 5; /* Device complies to SPC-3 */
+ buf[3] = 0x12; /* HiSup + data in format specified in SPC */
+ buf[4] = 31;/* n - 4 = 35 - 4 = 31 for full 36 byte data */
+ buf[6] = 1; /* MultiP 1 */
+ buf[7] = 2; /* CMDQUE 1, BQue 0 => commands queuing supported */
+
+ /*
+ * 8 byte ASCII Vendor Identification of the target
+ * - left aligned.
+ */
+ if (virt_dev->blockio)
+ memcpy(&buf[8], SCST_BIO_VENDOR, 8);
+ else
+ memcpy(&buf[8], SCST_FIO_VENDOR, 8);
+
+ /*
+ * 16 byte ASCII Product Identification of the target - left
+ * aligned.
+ */
+ memset(&buf[16], ' ', 16);
+ len = min(strlen(virt_dev->name), (size_t)16);
+ memcpy(&buf[16], virt_dev->name, len);
+
+ /*
+ * 4 byte ASCII Product Revision Level of the target - left
+ * aligned.
+ */
+ memcpy(&buf[32], SCST_FIO_REV, 4);
+
+ /** Version descriptors **/
+
+ buf[4] += 58 - 36;
+ num = 0;
+
+ /* SAM-3 T10/1561-D revision 14 */
+ buf[58 + num] = 0x0;
+ buf[58 + num + 1] = 0x76;
+ num += 2;
+
+ /* Physical transport */
+ if (cmd->tgtt->get_phys_transport_version != NULL) {
+ uint16_t v = cmd->tgtt->get_phys_transport_version(cmd->tgt);
+ if (v != 0) {
+ *((__be16 *)&buf[58 + num]) = cpu_to_be16(v);
+ num += 2;
+ }
+ }
+
+ /* SCSI transport */
+ if (cmd->tgtt->get_scsi_transport_version != NULL) {
+ *((__be16 *)&buf[58 + num]) =
+ cpu_to_be16(cmd->tgtt->get_scsi_transport_version(cmd->tgt));
+ num += 2;
+ }
+
+ /* SPC-3 T10/1416-D revision 23 */
+ buf[58 + num] = 0x3;
+ buf[58 + num + 1] = 0x12;
+ num += 2;
+
+ /* Device command set */
+ if (virt_dev->command_set_version != 0) {
+ *((__be16 *)&buf[58 + num]) =
+ cpu_to_be16(virt_dev->command_set_version);
+ num += 2;
+ }
+
+ buf[4] += num;
+ resp_len = buf[4] + 5;
+ }
+
+ BUG_ON(resp_len >= INQ_BUF_SZ);
+
+ if (length > resp_len)
+ length = resp_len;
+ memcpy(address, buf, length);
+
+out_put:
+ scst_put_buf(cmd, address);
+ if (length < cmd->resp_data_len)
+ scst_set_resp_data_len(cmd, length);
+
+out_free:
+ kfree(buf);
+
+out:
+ return;
+}
+
+static void vdisk_exec_request_sense(struct scst_cmd *cmd)
+{
+ int32_t length, sl;
+ uint8_t *address;
+ uint8_t b[SCST_STANDARD_SENSE_LEN];
+
+ sl = scst_set_sense(b, sizeof(b), cmd->dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_no_sense));
+
+ length = scst_get_buf_first(cmd, &address);
+ TRACE_DBG("length %d", length);
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_first() failed: %d)", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out;
+ }
+
+ length = min(sl, length);
+ memcpy(address, b, length);
+ scst_set_resp_data_len(cmd, length);
+
+ scst_put_buf(cmd, address);
+
+out:
+ return;
+}
+
+/*
+ * <<Following mode pages info copied from ST318451LW with some corrections>>
+ *
+ * ToDo: revise them
+ */
+static int vdisk_err_recov_pg(unsigned char *p, int pcontrol,
+ struct scst_vdisk_dev *virt_dev)
+{ /* Read-Write Error Recovery page for mode_sense */
+ const unsigned char err_recov_pg[] = {0x1, 0xa, 0xc0, 11, 240, 0, 0, 0,
+ 5, 0, 0xff, 0xff};
+
+ memcpy(p, err_recov_pg, sizeof(err_recov_pg));
+ if (1 == pcontrol)
+ memset(p + 2, 0, sizeof(err_recov_pg) - 2);
+ return sizeof(err_recov_pg);
+}
+
+static int vdisk_disconnect_pg(unsigned char *p, int pcontrol,
+ struct scst_vdisk_dev *virt_dev)
+{ /* Disconnect-Reconnect page for mode_sense */
+ const unsigned char disconnect_pg[] = {0x2, 0xe, 128, 128, 0, 10, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0};
+
+ memcpy(p, disconnect_pg, sizeof(disconnect_pg));
+ if (1 == pcontrol)
+ memset(p + 2, 0, sizeof(disconnect_pg) - 2);
+ return sizeof(disconnect_pg);
+}
+
+static int vdisk_rigid_geo_pg(unsigned char *p, int pcontrol,
+ struct scst_vdisk_dev *virt_dev)
+{
+ unsigned char geo_m_pg[] = {0x04, 0x16, 0, 0, 0, DEF_HEADS, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0x3a, 0x98/* 15K RPM */, 0, 0};
+ int32_t ncyl, n, rem;
+ uint64_t dividend;
+
+ memcpy(p, geo_m_pg, sizeof(geo_m_pg));
+ /*
+ * Divide virt_dev->nblocks by (DEF_HEADS * DEF_SECTORS) and store
+ * the quotient in ncyl and the remainder in rem.
+ */
+ dividend = virt_dev->nblocks;
+ rem = do_div(dividend, DEF_HEADS * DEF_SECTORS);
+ ncyl = dividend;
+ if (rem != 0)
+ ncyl++;
+ memcpy(&n, p + 2, sizeof(u32));
+ n = n | ((__force u32)cpu_to_be32(ncyl) >> 8);
+ memcpy(p + 2, &n, sizeof(u32));
+ if (1 == pcontrol)
+ memset(p + 2, 0, sizeof(geo_m_pg) - 2);
+ return sizeof(geo_m_pg);
+}
+
+static int vdisk_format_pg(unsigned char *p, int pcontrol,
+ struct scst_vdisk_dev *virt_dev)
+{ /* Format device page for mode_sense */
+ const unsigned char format_pg[] = {0x3, 0x16, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0x40, 0, 0, 0};
+
+ memcpy(p, format_pg, sizeof(format_pg));
+ p[10] = (DEF_SECTORS >> 8) & 0xff;
+ p[11] = DEF_SECTORS & 0xff;
+ p[12] = (virt_dev->block_size >> 8) & 0xff;
+ p[13] = virt_dev->block_size & 0xff;
+ if (1 == pcontrol)
+ memset(p + 2, 0, sizeof(format_pg) - 2);
+ return sizeof(format_pg);
+}
+
+static int vdisk_caching_pg(unsigned char *p, int pcontrol,
+ struct scst_vdisk_dev *virt_dev)
+{ /* Caching page for mode_sense */
+ const unsigned char caching_pg[] = {0x8, 18, 0x10, 0, 0xff, 0xff, 0, 0,
+ 0xff, 0xff, 0xff, 0xff, 0x80, 0x14, 0, 0, 0, 0, 0, 0};
+
+ memcpy(p, caching_pg, sizeof(caching_pg));
+ p[2] |= !(virt_dev->wt_flag || virt_dev->nv_cache) ? WCE : 0;
+ if (1 == pcontrol)
+ memset(p + 2, 0, sizeof(caching_pg) - 2);
+ return sizeof(caching_pg);
+}
+
+static int vdisk_ctrl_m_pg(unsigned char *p, int pcontrol,
+ struct scst_vdisk_dev *virt_dev)
+{ /* Control mode page for mode_sense */
+ const unsigned char ctrl_m_pg[] = {0xa, 0xa, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0x2, 0x4b};
+
+ memcpy(p, ctrl_m_pg, sizeof(ctrl_m_pg));
+ switch (pcontrol) {
+ case 0:
+ p[2] |= virt_dev->dev->tst << 5;
+ p[2] |= virt_dev->dev->d_sense << 2;
+ p[3] |= virt_dev->dev->queue_alg << 4;
+ p[4] |= virt_dev->dev->swp << 3;
+ p[5] |= virt_dev->dev->tas << 6;
+ break;
+ case 1:
+ memset(p + 2, 0, sizeof(ctrl_m_pg) - 2);
+#if 0 /*
+ * It's too early to implement it, since we can't control the
+ * backstorage device parameters. ToDo
+ */
+ p[2] |= 7 << 5; /* TST */
+ p[3] |= 0xF << 4; /* QUEUE ALGORITHM MODIFIER */
+#endif
+ p[2] |= 1 << 2; /* D_SENSE */
+ p[4] |= 1 << 3; /* SWP */
+ p[5] |= 1 << 6; /* TAS */
+ break;
+ case 2:
+ p[2] |= DEF_TST << 5;
+ p[2] |= DEF_DSENSE << 2;
+ if (virt_dev->wt_flag || virt_dev->nv_cache)
+ p[3] |= DEF_QUEUE_ALG_WT << 4;
+ else
+ p[3] |= DEF_QUEUE_ALG << 4;
+ p[4] |= DEF_SWP << 3;
+ p[5] |= DEF_TAS << 6;
+ break;
+ default:
+ BUG();
+ }
+ return sizeof(ctrl_m_pg);
+}
+
+static int vdisk_iec_m_pg(unsigned char *p, int pcontrol,
+ struct scst_vdisk_dev *virt_dev)
+{ /* Informational Exceptions control mode page for mode_sense */
+ const unsigned char iec_m_pg[] = {0x1c, 0xa, 0x08, 0, 0, 0, 0, 0,
+ 0, 0, 0x0, 0x0};
+ memcpy(p, iec_m_pg, sizeof(iec_m_pg));
+ if (1 == pcontrol)
+ memset(p + 2, 0, sizeof(iec_m_pg) - 2);
+ return sizeof(iec_m_pg);
+}
+
+static void vdisk_exec_mode_sense(struct scst_cmd *cmd)
+{
+ int32_t length;
+ uint8_t *address;
+ uint8_t *buf;
+ struct scst_vdisk_dev *virt_dev;
+ uint32_t blocksize;
+ uint64_t nblocks;
+ unsigned char dbd, type;
+ int pcontrol, pcode, subpcode;
+ unsigned char dev_spec;
+ int msense_6, offset = 0, len;
+ unsigned char *bp;
+
+ buf = kzalloc(MSENSE_BUF_SZ, GFP_KERNEL);
+ if (buf == NULL) {
+ scst_set_busy(cmd);
+ goto out;
+ }
+
+ virt_dev = (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ blocksize = virt_dev->block_size;
+ nblocks = virt_dev->nblocks;
+
+ type = cmd->dev->type; /* type dev */
+ dbd = cmd->cdb[1] & DBD;
+ pcontrol = (cmd->cdb[2] & 0xc0) >> 6;
+ pcode = cmd->cdb[2] & 0x3f;
+ subpcode = cmd->cdb[3];
+ msense_6 = (MODE_SENSE == cmd->cdb[0]);
+ dev_spec = (virt_dev->dev->rd_only ||
+ cmd->tgt_dev->acg_dev->rd_only) ? WP : 0;
+
+ if (!virt_dev->blockio)
+ dev_spec |= DPOFUA;
+
+ length = scst_get_buf_first(cmd, &address);
+ if (unlikely(length <= 0)) {
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_first() failed: %d", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out_free;
+ }
+
+ if (0x3 == pcontrol) {
+ TRACE_DBG("%s", "MODE SENSE: Saving values not supported");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_saving_params_unsup));
+ goto out_put;
+ }
+
+ if (msense_6) {
+ buf[1] = type;
+ buf[2] = dev_spec;
+ offset = 4;
+ } else {
+ buf[2] = type;
+ buf[3] = dev_spec;
+ offset = 8;
+ }
+
+ if (0 != subpcode) {
+ /* TODO: Control Extension page */
+ TRACE_DBG("%s", "MODE SENSE: Only subpage 0 is supported");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_put;
+ }
+
+ if (!dbd) {
+ /* Create block descriptor */
+ buf[offset - 1] = 0x08; /* block descriptor length */
+ if (nblocks >> 32) {
+ buf[offset + 0] = 0xFF;
+ buf[offset + 1] = 0xFF;
+ buf[offset + 2] = 0xFF;
+ buf[offset + 3] = 0xFF;
+ } else {
+ /* num blks */
+ buf[offset + 0] = (nblocks >> (BYTE * 3)) & 0xFF;
+ buf[offset + 1] = (nblocks >> (BYTE * 2)) & 0xFF;
+ buf[offset + 2] = (nblocks >> (BYTE * 1)) & 0xFF;
+ buf[offset + 3] = (nblocks >> (BYTE * 0)) & 0xFF;
+ }
+ buf[offset + 4] = 0; /* density code */
+ buf[offset + 5] = (blocksize >> (BYTE * 2)) & 0xFF;/* blklen */
+ buf[offset + 6] = (blocksize >> (BYTE * 1)) & 0xFF;
+ buf[offset + 7] = (blocksize >> (BYTE * 0)) & 0xFF;
+
+ offset += 8; /* increment offset */
+ }
+
+ bp = buf + offset;
+
+ switch (pcode) {
+ case 0x1: /* Read-Write error recovery page, direct access */
+ len = vdisk_err_recov_pg(bp, pcontrol, virt_dev);
+ break;
+ case 0x2: /* Disconnect-Reconnect page, all devices */
+ len = vdisk_disconnect_pg(bp, pcontrol, virt_dev);
+ break;
+ case 0x3: /* Format device page, direct access */
+ len = vdisk_format_pg(bp, pcontrol, virt_dev);
+ break;
+ case 0x4: /* Rigid disk geometry */
+ len = vdisk_rigid_geo_pg(bp, pcontrol, virt_dev);
+ break;
+ case 0x8: /* Caching page, direct access */
+ len = vdisk_caching_pg(bp, pcontrol, virt_dev);
+ break;
+ case 0xa: /* Control Mode page, all devices */
+ len = vdisk_ctrl_m_pg(bp, pcontrol, virt_dev);
+ break;
+ case 0x1c: /* Informational Exceptions Mode page, all devices */
+ len = vdisk_iec_m_pg(bp, pcontrol, virt_dev);
+ break;
+ case 0x3f: /* Read all Mode pages */
+ len = vdisk_err_recov_pg(bp, pcontrol, virt_dev);
+ len += vdisk_disconnect_pg(bp + len, pcontrol, virt_dev);
+ len += vdisk_format_pg(bp + len, pcontrol, virt_dev);
+ len += vdisk_caching_pg(bp + len, pcontrol, virt_dev);
+ len += vdisk_ctrl_m_pg(bp + len, pcontrol, virt_dev);
+ len += vdisk_iec_m_pg(bp + len, pcontrol, virt_dev);
+ len += vdisk_rigid_geo_pg(bp + len, pcontrol, virt_dev);
+ break;
+ default:
+ TRACE_DBG("MODE SENSE: Unsupported page %x", pcode);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_put;
+ }
+
+ offset += len;
+
+ if (msense_6)
+ buf[0] = offset - 1;
+ else {
+ buf[0] = ((offset - 2) >> 8) & 0xff;
+ buf[1] = (offset - 2) & 0xff;
+ }
+
+ if (offset > length)
+ offset = length;
+ memcpy(address, buf, offset);
+
+out_put:
+ scst_put_buf(cmd, address);
+ if (offset < cmd->resp_data_len)
+ scst_set_resp_data_len(cmd, offset);
+
+out_free:
+ kfree(buf);
+
+out:
+ return;
+}
+
+static int vdisk_set_wt(struct scst_vdisk_dev *virt_dev, int wt)
+{
+ int res = 0;
+
+ if ((virt_dev->wt_flag == wt) || virt_dev->nullio || virt_dev->nv_cache)
+ goto out;
+
+ spin_lock(&virt_dev->flags_lock);
+ virt_dev->wt_flag = wt;
+ spin_unlock(&virt_dev->flags_lock);
+
+ scst_dev_del_all_thr_data(virt_dev->dev);
+
+out:
+ return res;
+}
+
+static void vdisk_ctrl_m_pg_select(unsigned char *p,
+ struct scst_vdisk_dev *virt_dev)
+{
+ struct scst_device *dev = virt_dev->dev;
+ int old_swp = dev->swp, old_tas = dev->tas, old_dsense = dev->d_sense;
+
+#if 0
+ /* Not implemented yet, see comment in vdisk_ctrl_m_pg() */
+ dev->tst = p[2] >> 5;
+ dev->queue_alg = p[3] >> 4;
+#endif
+ dev->swp = (p[4] & 0x8) >> 3;
+ dev->tas = (p[5] & 0x40) >> 6;
+ dev->d_sense = (p[2] & 0x4) >> 2;
+
+ PRINT_INFO("Device %s: new control mode page parameters: SWP %x "
+ "(was %x), TAS %x (was %x), D_SENSE %d (was %d)",
+ virt_dev->name, dev->swp, old_swp, dev->tas, old_tas,
+ dev->d_sense, old_dsense);
+ return;
+}
+
+static void vdisk_exec_mode_select(struct scst_cmd *cmd)
+{
+ int32_t length;
+ uint8_t *address;
+ struct scst_vdisk_dev *virt_dev;
+ int mselect_6, offset;
+
+ virt_dev = (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ mselect_6 = (MODE_SELECT == cmd->cdb[0]);
+
+ length = scst_get_buf_first(cmd, &address);
+ if (unlikely(length <= 0)) {
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_first() failed: %d", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out;
+ }
+
+ if (!(cmd->cdb[1] & PF) || (cmd->cdb[1] & SP)) {
+ TRACE(TRACE_MINOR|TRACE_SCSI, "MODE SELECT: Unsupported "
+ "value(s) of PF and/or SP bits (cdb[1]=%x)",
+ cmd->cdb[1]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_put;
+ }
+
+ if (mselect_6)
+ offset = 4;
+ else
+ offset = 8;
+
+ if (address[offset - 1] == 8) {
+ offset += 8;
+ } else if (address[offset - 1] != 0) {
+ PRINT_ERROR("%s", "MODE SELECT: Wrong parameters list "
+ "lenght");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_parm_list));
+ goto out_put;
+ }
+
+ while (length > offset + 2) {
+ if (address[offset] & PS) {
+ PRINT_ERROR("%s", "MODE SELECT: Illegal PS bit");
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_parm_list));
+ goto out_put;
+ }
+ if ((address[offset] & 0x3f) == 0x8) {
+ /* Caching page */
+ if (address[offset + 1] != 18) {
+ PRINT_ERROR("%s", "MODE SELECT: Invalid "
+ "caching page request");
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_parm_list));
+ goto out_put;
+ }
+ if (vdisk_set_wt(virt_dev,
+ (address[offset + 2] & WCE) ? 0 : 1) != 0) {
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_put;
+ }
+ break;
+ } else if ((address[offset] & 0x3f) == 0xA) {
+ /* Control page */
+ if (address[offset + 1] != 0xA) {
+ PRINT_ERROR("%s", "MODE SELECT: Invalid "
+ "control page request");
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_parm_list));
+ goto out_put;
+ }
+ vdisk_ctrl_m_pg_select(&address[offset], virt_dev);
+ } else {
+ PRINT_ERROR("MODE SELECT: Invalid request %x",
+ address[offset] & 0x3f);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(
+ scst_sense_invalid_field_in_parm_list));
+ goto out_put;
+ }
+ offset += address[offset + 1];
+ }
+
+out_put:
+ scst_put_buf(cmd, address);
+
+out:
+ return;
+}
+
+static void vdisk_exec_log(struct scst_cmd *cmd)
+{
+
+ /* No log pages are supported */
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ return;
+}
+
+static void vdisk_exec_read_capacity(struct scst_cmd *cmd)
+{
+ int32_t length;
+ uint8_t *address;
+ struct scst_vdisk_dev *virt_dev;
+ uint32_t blocksize;
+ uint64_t nblocks;
+ uint8_t buffer[8];
+
+ virt_dev = (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ blocksize = virt_dev->block_size;
+ nblocks = virt_dev->nblocks;
+
+ if ((cmd->cdb[8] & 1) == 0) {
+ uint64_t lba = be64_to_cpu(get_unaligned((__be64 *)&cmd->cdb[2]));
+ if (lba != 0) {
+ TRACE_DBG("PMI zero and LBA not zero (cmd %p)", cmd);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+ }
+
+ /* Last block on the virt_dev is (nblocks-1) */
+ memset(buffer, 0, sizeof(buffer));
+ if (nblocks >> 32) {
+ buffer[0] = 0xFF;
+ buffer[1] = 0xFF;
+ buffer[2] = 0xFF;
+ buffer[3] = 0xFF;
+ } else {
+ buffer[0] = ((nblocks - 1) >> (BYTE * 3)) & 0xFF;
+ buffer[1] = ((nblocks - 1) >> (BYTE * 2)) & 0xFF;
+ buffer[2] = ((nblocks - 1) >> (BYTE * 1)) & 0xFF;
+ buffer[3] = ((nblocks - 1) >> (BYTE * 0)) & 0xFF;
+ }
+ buffer[4] = (blocksize >> (BYTE * 3)) & 0xFF;
+ buffer[5] = (blocksize >> (BYTE * 2)) & 0xFF;
+ buffer[6] = (blocksize >> (BYTE * 1)) & 0xFF;
+ buffer[7] = (blocksize >> (BYTE * 0)) & 0xFF;
+
+ length = scst_get_buf_first(cmd, &address);
+ if (unlikely(length <= 0)) {
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_first() failed: %d", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out;
+ }
+
+ length = min_t(int, length, sizeof(buffer));
+
+ memcpy(address, buffer, length);
+
+ scst_put_buf(cmd, address);
+
+ if (length < cmd->resp_data_len)
+ scst_set_resp_data_len(cmd, length);
+
+out:
+ return;
+}
+
+static void vdisk_exec_read_capacity16(struct scst_cmd *cmd)
+{
+ int32_t length;
+ uint8_t *address;
+ struct scst_vdisk_dev *virt_dev;
+ uint32_t blocksize;
+ uint64_t nblocks;
+ uint8_t buffer[32];
+
+ virt_dev = (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ blocksize = virt_dev->block_size;
+ nblocks = virt_dev->nblocks - 1;
+
+ if ((cmd->cdb[14] & 1) == 0) {
+ uint64_t lba = be64_to_cpu(get_unaligned((__be64 *)&cmd->cdb[2]));
+ if (lba != 0) {
+ TRACE_DBG("PMI zero and LBA not zero (cmd %p)", cmd);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+ }
+
+ memset(buffer, 0, sizeof(buffer));
+
+ buffer[0] = nblocks >> 56;
+ buffer[1] = (nblocks >> 48) & 0xFF;
+ buffer[2] = (nblocks >> 40) & 0xFF;
+ buffer[3] = (nblocks >> 32) & 0xFF;
+ buffer[4] = (nblocks >> 24) & 0xFF;
+ buffer[5] = (nblocks >> 16) & 0xFF;
+ buffer[6] = (nblocks >> 8) & 0xFF;
+ buffer[7] = nblocks & 0xFF;
+
+ buffer[8] = (blocksize >> (BYTE * 3)) & 0xFF;
+ buffer[9] = (blocksize >> (BYTE * 2)) & 0xFF;
+ buffer[10] = (blocksize >> (BYTE * 1)) & 0xFF;
+ buffer[11] = (blocksize >> (BYTE * 0)) & 0xFF;
+
+ switch (blocksize) {
+ case 512:
+ buffer[13] = 3;
+ break;
+ case 1024:
+ buffer[13] = 2;
+ break;
+ case 2048:
+ buffer[13] = 1;
+ break;
+ case 4096:
+ default:
+ buffer[13] = 0;
+ break;
+ }
+
+ length = scst_get_buf_first(cmd, &address);
+ if (unlikely(length <= 0)) {
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_first() failed: %d", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out;
+ }
+
+ length = min_t(int, length, sizeof(buffer));
+
+ memcpy(address, buffer, length);
+
+ scst_put_buf(cmd, address);
+
+ if (length < cmd->resp_data_len)
+ scst_set_resp_data_len(cmd, length);
+
+out:
+ return;
+}
+
+static void vdisk_exec_read_toc(struct scst_cmd *cmd)
+{
+ int32_t length, off = 0;
+ uint8_t *address;
+ struct scst_vdisk_dev *virt_dev;
+ uint32_t nblocks;
+ uint8_t buffer[4+8+8] = { 0x00, 0x0a, 0x01, 0x01, 0x00, 0x14,
+ 0x01, 0x00, 0x00, 0x00, 0x00, 0x00 };
+
+ if (cmd->dev->type != TYPE_ROM) {
+ PRINT_ERROR("%s", "READ TOC for non-CDROM device");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out;
+ }
+
+ if (cmd->cdb[2] & 0x0e/*Format*/) {
+ PRINT_ERROR("%s", "READ TOC: invalid requested data format");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+
+ if ((cmd->cdb[6] != 0 && (cmd->cdb[2] & 0x01)) ||
+ (cmd->cdb[6] > 1 && cmd->cdb[6] != 0xAA)) {
+ PRINT_ERROR("READ TOC: invalid requested track number %x",
+ cmd->cdb[6]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out;
+ }
+
+ length = scst_get_buf_first(cmd, &address);
+ if (unlikely(length <= 0)) {
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_first() failed: %d", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out;
+ }
+
+ virt_dev = (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ /* ToDo when you have > 8TB ROM device. */
+ nblocks = (uint32_t)virt_dev->nblocks;
+
+ /* Header */
+ memset(buffer, 0, sizeof(buffer));
+ buffer[2] = 0x01; /* First Track/Session */
+ buffer[3] = 0x01; /* Last Track/Session */
+ off = 4;
+ if (cmd->cdb[6] <= 1) {
+ /* Fistr TOC Track Descriptor */
+ /* ADDR 0x10 - Q Sub-channel encodes current position data
+ CONTROL 0x04 - Data track, recoreded uninterrupted */
+ buffer[off+1] = 0x14;
+ /* Track Number */
+ buffer[off+2] = 0x01;
+ off += 8;
+ }
+ if (!(cmd->cdb[2] & 0x01)) {
+ /* Lead-out area TOC Track Descriptor */
+ buffer[off+1] = 0x14;
+ /* Track Number */
+ buffer[off+2] = 0xAA;
+ /* Track Start Address */
+ buffer[off+4] = (nblocks >> (BYTE * 3)) & 0xFF;
+ buffer[off+5] = (nblocks >> (BYTE * 2)) & 0xFF;
+ buffer[off+6] = (nblocks >> (BYTE * 1)) & 0xFF;
+ buffer[off+7] = (nblocks >> (BYTE * 0)) & 0xFF;
+ off += 8;
+ }
+
+ buffer[1] = off - 2; /* Data Length */
+
+ if (off > length)
+ off = length;
+ memcpy(address, buffer, off);
+
+ scst_put_buf(cmd, address);
+
+ if (off < cmd->resp_data_len)
+ scst_set_resp_data_len(cmd, off);
+
+out:
+ return;
+}
+
+static void vdisk_exec_prevent_allow_medium_removal(struct scst_cmd *cmd)
+{
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+
+ TRACE_DBG("PERSIST/PREVENT 0x%02x", cmd->cdb[4]);
+
+ spin_lock(&virt_dev->flags_lock);
+ virt_dev->prevent_allow_medium_removal = cmd->cdb[4] & 0x01 ? 1 : 0;
+ spin_unlock(&virt_dev->flags_lock);
+
+ return;
+}
+
+static int vdisk_fsync(struct scst_vdisk_thr *thr, loff_t loff,
+ loff_t len, struct scst_cmd *cmd, struct scst_device *dev)
+{
+ int res = 0;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)dev->dh_priv;
+ struct file *file;
+
+ /* Hopefully, the compiler will generate the single comparison */
+ if (virt_dev->nv_cache || virt_dev->wt_flag ||
+ virt_dev->o_direct_flag || virt_dev->nullio)
+ goto out;
+
+ if (virt_dev->blockio) {
+ res = blockio_flush(thr->bdev);
+ goto out;
+ }
+
+ file = thr->fd;
+
+#if 0 /* For sparse files we might need to sync metadata as well */
+ res = generic_write_sync(file, loff, len);
+#else
+ res = filemap_write_and_wait_range(file->f_mapping, loff, len);
+#endif
+ if (unlikely(res != 0)) {
+ PRINT_ERROR("sync range failed (%d)", res);
+ if (cmd != NULL) {
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_write_error));
+ }
+ }
+
+out:
+ return res;
+}
+
+static struct iovec *vdisk_alloc_iv(struct scst_cmd *cmd,
+ struct scst_vdisk_thr *thr)
+{
+ int iv_count;
+
+ iv_count = min_t(int, scst_get_buf_count(cmd), UIO_MAXIOV);
+ if (iv_count > thr->iv_count) {
+ kfree(thr->iv);
+ /* It can't be called in atomic context */
+ thr->iv = kmalloc(sizeof(*thr->iv) * iv_count, GFP_KERNEL);
+ if (thr->iv == NULL) {
+ PRINT_ERROR("Unable to allocate iv (%d)", iv_count);
+ scst_set_busy(cmd);
+ goto out;
+ }
+ thr->iv_count = iv_count;
+ }
+
+out:
+ return thr->iv;
+}
+
+static void vdisk_exec_read(struct scst_cmd *cmd,
+ struct scst_vdisk_thr *thr, loff_t loff)
+{
+ mm_segment_t old_fs;
+ loff_t err;
+ ssize_t length, full_len;
+ uint8_t __user *address;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ struct file *fd = thr->fd;
+ struct iovec *iv;
+ int iv_count, i;
+ bool finished = false;
+
+ if (virt_dev->nullio)
+ goto out;
+
+ iv = vdisk_alloc_iv(cmd, thr);
+ if (iv == NULL)
+ goto out;
+
+ length = scst_get_buf_first(cmd, (uint8_t __force **)&address);
+ if (unlikely(length < 0)) {
+ PRINT_ERROR("scst_get_buf_first() failed: %zd", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out;
+ }
+
+ old_fs = get_fs();
+ set_fs(get_ds());
+
+ while (1) {
+ iv_count = 0;
+ full_len = 0;
+ i = -1;
+ while (length > 0) {
+ full_len += length;
+ i++;
+ iv_count++;
+ iv[i].iov_base = address;
+ iv[i].iov_len = length;
+ if (iv_count == UIO_MAXIOV)
+ break;
+ length = scst_get_buf_next(cmd,
+ (uint8_t __force **)&address);
+ }
+ if (length == 0) {
+ finished = true;
+ if (unlikely(iv_count == 0))
+ break;
+ } else if (unlikely(length < 0)) {
+ PRINT_ERROR("scst_get_buf_next() failed: %zd", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_set_fs;
+ }
+
+ TRACE_DBG("(iv_count %d, full_len %zd)", iv_count, full_len);
+ /* SEEK */
+ if (fd->f_op->llseek)
+ err = fd->f_op->llseek(fd, loff, 0/*SEEK_SET*/);
+ else
+ err = default_llseek(fd, loff, 0/*SEEK_SET*/);
+ if (err != loff) {
+ PRINT_ERROR("lseek trouble %lld != %lld",
+ (long long unsigned int)err,
+ (long long unsigned int)loff);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_set_fs;
+ }
+
+ /* READ */
+ err = vfs_readv(fd, (struct iovec __force __user *)iv, iv_count,
+ &fd->f_pos);
+
+ if ((err < 0) || (err < full_len)) {
+ PRINT_ERROR("readv() returned %lld from %zd",
+ (long long unsigned int)err,
+ full_len);
+ if (err == -EAGAIN)
+ scst_set_busy(cmd);
+ else {
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_read_error));
+ }
+ goto out_set_fs;
+ }
+
+ for (i = 0; i < iv_count; i++)
+ scst_put_buf(cmd, (void __force *)(iv[i].iov_base));
+
+ if (finished)
+ break;
+
+ loff += full_len;
+ length = scst_get_buf_next(cmd, (uint8_t __force **)&address);
+ };
+
+ set_fs(old_fs);
+
+out:
+ return;
+
+out_set_fs:
+ set_fs(old_fs);
+ for (i = 0; i < iv_count; i++)
+ scst_put_buf(cmd, (void __force *)(iv[i].iov_base));
+ goto out;
+}
+
+static void vdisk_exec_write(struct scst_cmd *cmd,
+ struct scst_vdisk_thr *thr, loff_t loff)
+{
+ mm_segment_t old_fs;
+ loff_t err;
+ ssize_t length, full_len, saved_full_len;
+ uint8_t __user *address;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ struct file *fd = thr->fd;
+ struct iovec *iv, *eiv;
+ int i, iv_count, eiv_count;
+ bool finished = false;
+
+ if (virt_dev->nullio)
+ goto out;
+
+ iv = vdisk_alloc_iv(cmd, thr);
+ if (iv == NULL)
+ goto out;
+
+ length = scst_get_buf_first(cmd, (uint8_t __force **)&address);
+ if (unlikely(length < 0)) {
+ PRINT_ERROR("scst_get_buf_first() failed: %zd", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out;
+ }
+
+ old_fs = get_fs();
+ set_fs(get_ds());
+
+ while (1) {
+ iv_count = 0;
+ full_len = 0;
+ i = -1;
+ while (length > 0) {
+ full_len += length;
+ i++;
+ iv_count++;
+ iv[i].iov_base = address;
+ iv[i].iov_len = length;
+ if (iv_count == UIO_MAXIOV)
+ break;
+ length = scst_get_buf_next(cmd,
+ (uint8_t __force **)&address);
+ }
+ if (length == 0) {
+ finished = true;
+ if (unlikely(iv_count == 0))
+ break;
+ } else if (unlikely(length < 0)) {
+ PRINT_ERROR("scst_get_buf_next() failed: %zd", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_set_fs;
+ }
+
+ saved_full_len = full_len;
+ eiv = iv;
+ eiv_count = iv_count;
+restart:
+ TRACE_DBG("writing(eiv_count %d, full_len %zd)", eiv_count, full_len);
+
+ /* SEEK */
+ if (fd->f_op->llseek)
+ err = fd->f_op->llseek(fd, loff, 0 /*SEEK_SET */);
+ else
+ err = default_llseek(fd, loff, 0 /*SEEK_SET */);
+ if (err != loff) {
+ PRINT_ERROR("lseek trouble %lld != %lld",
+ (long long unsigned int)err,
+ (long long unsigned int)loff);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_set_fs;
+ }
+
+ /* WRITE */
+ err = vfs_writev(fd, (struct iovec __force __user *)eiv, eiv_count,
+ &fd->f_pos);
+
+ if (err < 0) {
+ PRINT_ERROR("write() returned %lld from %zd",
+ (long long unsigned int)err,
+ full_len);
+ if (err == -EAGAIN)
+ scst_set_busy(cmd);
+ else {
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_write_error));
+ }
+ goto out_set_fs;
+ } else if (err < full_len) {
+ /*
+ * Probably that's wrong, but sometimes write() returns
+ * value less, than requested. Let's restart.
+ */
+ int e = eiv_count;
+ TRACE_MGMT_DBG("write() returned %d from %zd "
+ "(iv_count=%d)", (int)err, full_len,
+ eiv_count);
+ if (err == 0) {
+ PRINT_INFO("Suspicious: write() returned 0 from "
+ "%zd (iv_count=%d)", full_len, eiv_count);
+ }
+ full_len -= err;
+ for (i = 0; i < e; i++) {
+ if ((long long)eiv->iov_len < err) {
+ err -= eiv->iov_len;
+ eiv++;
+ eiv_count--;
+ } else {
+ eiv->iov_base =
+ (uint8_t __force __user *)eiv->iov_base + err;
+ eiv->iov_len -= err;
+ break;
+ }
+ }
+ goto restart;
+ }
+
+ for (i = 0; i < iv_count; i++)
+ scst_put_buf(cmd, (void __force *)(iv[i].iov_base));
+
+ if (finished)
+ break;
+
+ loff += saved_full_len;
+ length = scst_get_buf_next(cmd, (uint8_t __force **)&address);
+ }
+
+ set_fs(old_fs);
+
+out:
+ return;
+
+out_set_fs:
+ set_fs(old_fs);
+ for (i = 0; i < iv_count; i++)
+ scst_put_buf(cmd, (void __force *)(iv[i].iov_base));
+ goto out;
+}
+
+struct scst_blockio_work {
+ atomic_t bios_inflight;
+ struct scst_cmd *cmd;
+};
+
+static inline void blockio_check_finish(struct scst_blockio_work *blockio_work)
+{
+ /* Decrement the bios in processing, and if zero signal completion */
+ if (atomic_dec_and_test(&blockio_work->bios_inflight)) {
+ blockio_work->cmd->completed = 1;
+ blockio_work->cmd->scst_cmd_done(blockio_work->cmd,
+ SCST_CMD_STATE_DEFAULT, scst_estimate_context());
+ kmem_cache_free(blockio_work_cachep, blockio_work);
+ }
+ return;
+}
+
+static void blockio_endio(struct bio *bio, int error)
+{
+ struct scst_blockio_work *blockio_work = bio->bi_private;
+
+ if (unlikely(!bio_flagged(bio, BIO_UPTODATE))) {
+ if (error == 0) {
+ PRINT_ERROR("Not up to date bio with error 0 for "
+ "cmd %p, returning -EIO", blockio_work->cmd);
+ error = -EIO;
+ }
+ }
+
+ if (unlikely(error != 0)) {
+ static DEFINE_SPINLOCK(blockio_endio_lock);
+
+ PRINT_ERROR("cmd %p returned error %d", blockio_work->cmd,
+ error);
+
+ /* To protect from several bios finishing simultaneously */
+ spin_lock_bh(&blockio_endio_lock);
+
+ if (bio->bi_rw & (1 << BIO_RW))
+ scst_set_cmd_error(blockio_work->cmd,
+ SCST_LOAD_SENSE(scst_sense_write_error));
+ else
+ scst_set_cmd_error(blockio_work->cmd,
+ SCST_LOAD_SENSE(scst_sense_read_error));
+
+ spin_unlock_bh(&blockio_endio_lock);
+ }
+
+ blockio_check_finish(blockio_work);
+
+ bio_put(bio);
+ return;
+}
+
+static void blockio_exec_rw(struct scst_cmd *cmd, struct scst_vdisk_thr *thr,
+ u64 lba_start, int write)
+{
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ struct block_device *bdev = thr->bdev;
+ struct request_queue *q = bdev_get_queue(bdev);
+ int length, max_nr_vecs = 0;
+ uint8_t *address;
+ struct bio *bio = NULL, *hbio = NULL, *tbio = NULL;
+ int need_new_bio;
+ struct scst_blockio_work *blockio_work;
+ int bios = 0;
+
+ if (virt_dev->nullio)
+ goto out;
+
+ /* Allocate and initialize blockio_work struct */
+ blockio_work = kmem_cache_alloc(blockio_work_cachep, GFP_KERNEL);
+ if (blockio_work == NULL)
+ goto out_no_mem;
+
+ blockio_work->cmd = cmd;
+
+ if (q)
+ max_nr_vecs = min(bio_get_nr_vecs(bdev), BIO_MAX_PAGES);
+ else
+ max_nr_vecs = 1;
+
+ need_new_bio = 1;
+
+ length = scst_get_buf_first(cmd, &address);
+ while (length > 0) {
+ int len, bytes, off, thislen;
+ uint8_t *addr;
+ u64 lba_start0;
+
+ addr = address;
+ off = offset_in_page(addr);
+ len = length;
+ thislen = 0;
+ lba_start0 = lba_start;
+
+ while (len > 0) {
+ int rc;
+ struct page *page = virt_to_page(addr);
+
+ if (need_new_bio) {
+ bio = bio_kmalloc(GFP_KERNEL, max_nr_vecs);
+ if (!bio) {
+ PRINT_ERROR("Failed to create bio "
+ "for data segment %d (cmd %p)",
+ cmd->get_sg_buf_entry_num, cmd);
+ goto out_no_bio;
+ }
+
+ bios++;
+ need_new_bio = 0;
+ bio->bi_end_io = blockio_endio;
+ bio->bi_sector = lba_start0 <<
+ (virt_dev->block_shift - 9);
+ bio->bi_bdev = bdev;
+ bio->bi_private = blockio_work;
+ /*
+ * Better to fail fast w/o any local recovery
+ * and retries.
+ */
+#ifdef BIO_RW_FAILFAST
+ bio->bi_rw |= (1 << BIO_RW_FAILFAST);
+#else
+ bio->bi_rw |= (1 << BIO_RW_FAILFAST_DEV) |
+ (1 << BIO_RW_FAILFAST_TRANSPORT) |
+ (1 << BIO_RW_FAILFAST_DRIVER);
+#endif
+#if 0 /* It could be win, but could be not, so a performance study is needed */
+ bio->bi_rw |= 1 << BIO_RW_SYNC;
+#endif
+ if (!hbio)
+ hbio = tbio = bio;
+ else
+ tbio = tbio->bi_next = bio;
+ }
+
+ bytes = min_t(unsigned int, len, PAGE_SIZE - off);
+
+ rc = bio_add_page(bio, page, bytes, off);
+ if (rc < bytes) {
+ BUG_ON(rc != 0);
+ need_new_bio = 1;
+ lba_start0 += thislen >> virt_dev->block_shift;
+ thislen = 0;
+ continue;
+ }
+
+ addr += PAGE_SIZE;
+ thislen += bytes;
+ len -= bytes;
+ off = 0;
+ }
+
+ lba_start += length >> virt_dev->block_shift;
+
+ scst_put_buf(cmd, address);
+ length = scst_get_buf_next(cmd, &address);
+ }
+
+ /* +1 to prevent erroneous too early command completion */
+ atomic_set(&blockio_work->bios_inflight, bios+1);
+
+ while (hbio) {
+ bio = hbio;
+ hbio = hbio->bi_next;
+ bio->bi_next = NULL;
+ submit_bio((write != 0), bio);
+ }
+
+ if (q && q->unplug_fn)
+ q->unplug_fn(q);
+
+ blockio_check_finish(blockio_work);
+
+out:
+ return;
+
+out_no_bio:
+ while (hbio) {
+ bio = hbio;
+ hbio = hbio->bi_next;
+ bio_put(bio);
+ }
+ kmem_cache_free(blockio_work_cachep, blockio_work);
+
+out_no_mem:
+ scst_set_busy(cmd);
+ goto out;
+}
+
+static int blockio_flush(struct block_device *bdev)
+{
+ int res = 0;
+
+ res = blkdev_issue_flush(bdev, GFP_KERNEL, NULL, BLKDEV_IFL_WAIT);
+ if (res != 0)
+ PRINT_ERROR("blkdev_issue_flush() failed: %d", res);
+ return res;
+}
+
+static void vdisk_exec_verify(struct scst_cmd *cmd,
+ struct scst_vdisk_thr *thr, loff_t loff)
+{
+ mm_segment_t old_fs;
+ loff_t err;
+ ssize_t length, len_mem = 0;
+ uint8_t *address_sav, *address;
+ int compare;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)cmd->dev->dh_priv;
+ struct file *fd = thr->fd;
+ uint8_t *mem_verify = NULL;
+
+ if (vdisk_fsync(thr, loff, cmd->bufflen, cmd, cmd->dev) != 0)
+ goto out;
+
+ /*
+ * Until the cache is cleared prior the verifying, there is not
+ * much point in this code. ToDo.
+ *
+ * Nevertherless, this code is valuable if the data have not read
+ * from the file/disk yet.
+ */
+
+ /* SEEK */
+ old_fs = get_fs();
+ set_fs(get_ds());
+
+ if (!virt_dev->nullio) {
+ if (fd->f_op->llseek)
+ err = fd->f_op->llseek(fd, loff, 0/*SEEK_SET*/);
+ else
+ err = default_llseek(fd, loff, 0/*SEEK_SET*/);
+ if (err != loff) {
+ PRINT_ERROR("lseek trouble %lld != %lld",
+ (long long unsigned int)err,
+ (long long unsigned int)loff);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_set_fs;
+ }
+ }
+
+ mem_verify = vmalloc(LEN_MEM);
+ if (mem_verify == NULL) {
+ PRINT_ERROR("Unable to allocate memory %d for verify",
+ LEN_MEM);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_set_fs;
+ }
+
+ length = scst_get_buf_first(cmd, &address);
+ address_sav = address;
+ if (!length && cmd->data_len) {
+ length = cmd->data_len;
+ compare = 0;
+ } else
+ compare = 1;
+
+ while (length > 0) {
+ len_mem = (length > LEN_MEM) ? LEN_MEM : length;
+ TRACE_DBG("Verify: length %zd - len_mem %zd", length, len_mem);
+
+ if (!virt_dev->nullio)
+ err = vfs_read(fd, (char __force __user *)mem_verify,
+ len_mem, &fd->f_pos);
+ else
+ err = len_mem;
+ if ((err < 0) || (err < len_mem)) {
+ PRINT_ERROR("verify() returned %lld from %zd",
+ (long long unsigned int)err, len_mem);
+ if (err == -EAGAIN)
+ scst_set_busy(cmd);
+ else {
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_read_error));
+ }
+ if (compare)
+ scst_put_buf(cmd, address_sav);
+ goto out_set_fs;
+ }
+ if (compare && memcmp(address, mem_verify, len_mem) != 0) {
+ TRACE_DBG("Verify: error memcmp length %zd", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_miscompare_error));
+ scst_put_buf(cmd, address_sav);
+ goto out_set_fs;
+ }
+ length -= len_mem;
+ address += len_mem;
+ if (compare && length <= 0) {
+ scst_put_buf(cmd, address_sav);
+ length = scst_get_buf_next(cmd, &address);
+ address_sav = address;
+ }
+ }
+
+ if (length < 0) {
+ PRINT_ERROR("scst_get_buf_() failed: %zd", length);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+
+out_set_fs:
+ set_fs(old_fs);
+ if (mem_verify)
+ vfree(mem_verify);
+
+out:
+ return;
+}
+
+static int vdisk_task_mgmt_fn(struct scst_mgmt_cmd *mcmd,
+ struct scst_tgt_dev *tgt_dev)
+{
+
+ if ((mcmd->fn == SCST_LUN_RESET) || (mcmd->fn == SCST_TARGET_RESET)) {
+ /* Restore default values */
+ struct scst_device *dev = tgt_dev->dev;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)dev->dh_priv;
+
+ dev->tst = DEF_TST;
+ dev->d_sense = DEF_DSENSE;
+ if (virt_dev->wt_flag && !virt_dev->nv_cache)
+ dev->queue_alg = DEF_QUEUE_ALG_WT;
+ else
+ dev->queue_alg = DEF_QUEUE_ALG;
+ dev->swp = DEF_SWP;
+ dev->tas = DEF_TAS;
+
+ spin_lock(&virt_dev->flags_lock);
+ virt_dev->prevent_allow_medium_removal = 0;
+ spin_unlock(&virt_dev->flags_lock);
+ } else if (mcmd->fn == SCST_PR_ABORT_ALL) {
+ struct scst_device *dev = tgt_dev->dev;
+ struct scst_vdisk_dev *virt_dev =
+ (struct scst_vdisk_dev *)dev->dh_priv;
+ spin_lock(&virt_dev->flags_lock);
+ virt_dev->prevent_allow_medium_removal = 0;
+ spin_unlock(&virt_dev->flags_lock);
+ }
+ return SCST_DEV_TM_NOT_COMPLETED;
+}
+
+static void vdisk_report_registering(const struct scst_vdisk_dev *virt_dev)
+{
+ char buf[128];
+ int i, j;
+
+ i = snprintf(buf, sizeof(buf), "Registering virtual %s device %s ",
+ virt_dev->vdev_devt->name, virt_dev->name);
+ j = i;
+
+ if (virt_dev->wt_flag)
+ i += snprintf(&buf[i], sizeof(buf) - i, "(WRITE_THROUGH");
+
+ if (virt_dev->nv_cache)
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sNV_CACHE",
+ (j == i) ? "(" : ", ");
+
+ if (virt_dev->rd_only)
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sREAD_ONLY",
+ (j == i) ? "(" : ", ");
+
+ if (virt_dev->o_direct_flag)
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sO_DIRECT",
+ (j == i) ? "(" : ", ");
+
+ if (virt_dev->nullio)
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sNULLIO",
+ (j == i) ? "(" : ", ");
+
+ if (virt_dev->blockio)
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sBLOCKIO",
+ (j == i) ? "(" : ", ");
+
+ if (virt_dev->removable)
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sREMOVABLE",
+ (j == i) ? "(" : ", ");
+
+ if (j == i)
+ PRINT_INFO("%s", buf);
+ else
+ PRINT_INFO("%s)", buf);
+
+ return;
+}
+
+static int vdisk_resync_size(struct scst_vdisk_dev *virt_dev)
+{
+ loff_t file_size;
+ int res = 0;
+
+ BUG_ON(virt_dev->nullio);
+
+ res = vdisk_get_file_size(virt_dev->filename,
+ virt_dev->blockio, &file_size);
+ if (res != 0)
+ goto out;
+
+ if (file_size == virt_dev->file_size) {
+ PRINT_INFO("Size of virtual disk %s remained the same",
+ virt_dev->name);
+ goto out;
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ virt_dev->file_size = file_size;
+ virt_dev->nblocks = virt_dev->file_size >> virt_dev->block_shift;
+
+ scst_dev_del_all_thr_data(virt_dev->dev);
+
+ PRINT_INFO("New size of SCSI target virtual disk %s "
+ "(fs=%lldMB, bs=%d, nblocks=%lld, cyln=%lld%s)",
+ virt_dev->name, virt_dev->file_size >> 20,
+ virt_dev->block_size,
+ (long long unsigned int)virt_dev->nblocks,
+ (long long unsigned int)virt_dev->nblocks/64/32,
+ virt_dev->nblocks < 64*32 ? " !WARNING! cyln less "
+ "than 1" : "");
+
+ scst_capacity_data_changed(virt_dev->dev);
+
+ scst_resume_activity();
+
+out:
+ return res;
+}
+
+static int vdev_create(struct scst_dev_type *devt,
+ const char *name, struct scst_vdisk_dev **res_virt_dev)
+{
+ int res = 0;
+ struct scst_vdisk_dev *virt_dev;
+ uint64_t dev_id_num;
+ int dev_id_len;
+ char dev_id_str[17];
+ int32_t i;
+
+ virt_dev = kzalloc(sizeof(*virt_dev), GFP_KERNEL);
+ if (virt_dev == NULL) {
+ PRINT_ERROR("Allocation of virtual device %s failed",
+ devt->name);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ spin_lock_init(&virt_dev->flags_lock);
+ virt_dev->vdev_devt = devt;
+
+ virt_dev->rd_only = DEF_RD_ONLY;
+ virt_dev->removable = DEF_REMOVABLE;
+
+ virt_dev->block_size = DEF_DISK_BLOCKSIZE;
+ virt_dev->block_shift = DEF_DISK_BLOCKSIZE_SHIFT;
+
+ if (strlen(name) >= sizeof(virt_dev->name)) {
+ PRINT_ERROR("Name %s is too long (max allowed %zd)", name,
+ sizeof(virt_dev->name)-1);
+ res = -EINVAL;
+ goto out_free;
+ }
+ strcpy(virt_dev->name, name);
+
+ dev_id_num = vdisk_gen_dev_id_num(virt_dev->name);
+ dev_id_len = scnprintf(dev_id_str, sizeof(dev_id_str), "%llx",
+ dev_id_num);
+
+ i = strlen(virt_dev->name) + 1; /* for ' ' */
+ memset(virt_dev->t10_dev_id, ' ', i + dev_id_len);
+ memcpy(virt_dev->t10_dev_id, virt_dev->name, i-1);
+ memcpy(virt_dev->t10_dev_id + i, dev_id_str, dev_id_len);
+ TRACE_DBG("t10_dev_id %s", virt_dev->t10_dev_id);
+
+ scnprintf(virt_dev->usn, sizeof(virt_dev->usn), "%llx", dev_id_num);
+ TRACE_DBG("usn %s", virt_dev->usn);
+
+ *res_virt_dev = virt_dev;
+
+out:
+ return res;
+
+out_free:
+ kfree(virt_dev);
+ goto out;
+}
+
+static void vdev_destroy(struct scst_vdisk_dev *virt_dev)
+{
+ kfree(virt_dev->filename);
+ kfree(virt_dev);
+ return;
+}
+
+/* scst_vdisk_mutex supposed to be held */
+static struct scst_vdisk_dev *vdev_find(const char *name)
+{
+ struct scst_vdisk_dev *res, *vv;
+
+ res = NULL;
+ list_for_each_entry(vv, &vdev_list, vdev_list_entry) {
+ if (strcmp(vv->name, name) == 0) {
+ res = vv;
+ break;
+ }
+ }
+ return res;
+}
+
+static int vdev_parse_add_dev_params(struct scst_vdisk_dev *virt_dev,
+ char *params, const char *allowed_params[])
+{
+ int res = 0;
+ unsigned long val;
+ char *param, *p, *pp;
+
+ while (1) {
+ param = scst_get_next_token_str(¶ms);
+ if (param == NULL)
+ break;
+
+ p = scst_get_next_lexem(¶m);
+ if (*p == '\0') {
+ PRINT_ERROR("Syntax error at %s (device %s)",
+ param, virt_dev->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (allowed_params != NULL) {
+ const char **a = allowed_params;
+ bool allowed = false;
+
+ while (*a != NULL) {
+ if (!strcasecmp(*a, p)) {
+ allowed = true;
+ break;
+ }
+ a++;
+ }
+
+ if (!allowed) {
+ PRINT_ERROR("Unknown parameter %s (device %s)", p,
+ virt_dev->name);
+ res = -EINVAL;
+ goto out;
+ }
+ }
+
+ pp = scst_get_next_lexem(¶m);
+ if (*pp == '\0') {
+ PRINT_ERROR("Parameter %s value missed for device %s",
+ p, virt_dev->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (scst_get_next_lexem(¶m)[0] != '\0') {
+ PRINT_ERROR("Too many parameter's %s values (device %s)",
+ p, virt_dev->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (!strcasecmp("filename", p)) {
+ if (*pp != '/') {
+ PRINT_ERROR("Filename %s must be global "
+ "(device %s)", pp, virt_dev->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ virt_dev->filename = kstrdup(pp, GFP_KERNEL);
+ if (virt_dev->filename == NULL) {
+ PRINT_ERROR("Unable to duplicate file name %s "
+ "(device %s)", pp, virt_dev->name);
+ res = -ENOMEM;
+ goto out;
+ }
+ continue;
+ }
+
+ res = strict_strtoul(pp, 0, &val);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %d "
+ "(device %s)", pp, res, virt_dev->name);
+ goto out;
+ }
+
+ if (!strcasecmp("write_through", p)) {
+ virt_dev->wt_flag = val;
+ TRACE_DBG("WRITE THROUGH %d", virt_dev->wt_flag);
+ } else if (!strcasecmp("nv_cache", p)) {
+ virt_dev->nv_cache = val;
+ TRACE_DBG("NON-VOLATILE CACHE %d", virt_dev->nv_cache);
+ } else if (!strcasecmp("o_direct", p)) {
+#if 0
+ virt_dev->o_direct_flag = val;
+ TRACE_DBG("O_DIRECT %d", virt_dev->o_direct_flag);
+#else
+ PRINT_INFO("O_DIRECT flag doesn't currently"
+ " work, ignoring it, use fileio_tgt "
+ "in O_DIRECT mode instead (device %s)", virt_dev->name);
+#endif
+ } else if (!strcasecmp("read_only", p)) {
+ virt_dev->rd_only = val;
+ TRACE_DBG("READ ONLY %d", virt_dev->rd_only);
+ } else if (!strcasecmp("removable", p)) {
+ virt_dev->removable = val;
+ TRACE_DBG("REMOVABLE %d", virt_dev->removable);
+ } else if (!strcasecmp("blocksize", p)) {
+ virt_dev->block_size = val;
+ virt_dev->block_shift = scst_calc_block_shift(
+ virt_dev->block_size);
+ if (virt_dev->block_shift < 9) {
+ res = -EINVAL;
+ goto out;
+ }
+ TRACE_DBG("block_size %d, block_shift %d",
+ virt_dev->block_size,
+ virt_dev->block_shift);
+ } else {
+ PRINT_ERROR("Unknown parameter %s (device %s)", p,
+ virt_dev->name);
+ res = -EINVAL;
+ goto out;
+ }
+ }
+
+out:
+ return res;
+}
+
+/* scst_vdisk_mutex supposed to be held */
+static int vdev_fileio_add_device(const char *device_name, char *params)
+{
+ int res = 0;
+ struct scst_vdisk_dev *virt_dev;
+
+ res = vdev_create(&vdisk_file_devtype, device_name, &virt_dev);
+ if (res != 0)
+ goto out;
+
+ virt_dev->command_set_version = 0x04C0; /* SBC-3 */
+
+ virt_dev->wt_flag = DEF_WRITE_THROUGH;
+ virt_dev->nv_cache = DEF_NV_CACHE;
+ virt_dev->o_direct_flag = DEF_O_DIRECT;
+
+ res = vdev_parse_add_dev_params(virt_dev, params, NULL);
+ if (res != 0)
+ goto out_destroy;
+
+ if (virt_dev->rd_only && (virt_dev->wt_flag || virt_dev->nv_cache)) {
+ PRINT_ERROR("Write options on read only device %s",
+ virt_dev->name);
+ res = -EINVAL;
+ goto out_destroy;
+ }
+
+ if (virt_dev->filename == NULL) {
+ PRINT_ERROR("File name required (device %s)", virt_dev->name);
+ res = -EINVAL;
+ goto out_destroy;
+ }
+
+ list_add_tail(&virt_dev->vdev_list_entry, &vdev_list);
+
+ vdisk_report_registering(virt_dev);
+
+ virt_dev->virt_id = scst_register_virtual_device(virt_dev->vdev_devt,
+ virt_dev->name);
+ if (virt_dev->virt_id < 0) {
+ res = virt_dev->virt_id;
+ goto out_del;
+ }
+
+ TRACE_DBG("Registered virt_dev %s with id %d", virt_dev->name,
+ virt_dev->virt_id);
+
+out:
+ return res;
+
+out_del:
+ list_del(&virt_dev->vdev_list_entry);
+
+out_destroy:
+ vdev_destroy(virt_dev);
+ goto out;
+}
+
+/* scst_vdisk_mutex supposed to be held */
+static int vdev_blockio_add_device(const char *device_name, char *params)
+{
+ int res = 0;
+ const char *allowed_params[] = { "filename", "read_only", "removable",
+ "blocksize", "nv_cache", NULL };
+ struct scst_vdisk_dev *virt_dev;
+
+ res = vdev_create(&vdisk_blk_devtype, device_name, &virt_dev);
+ if (res != 0)
+ goto out;
+
+ virt_dev->command_set_version = 0x04C0; /* SBC-3 */
+
+ virt_dev->blockio = 1;
+
+ res = vdev_parse_add_dev_params(virt_dev, params, allowed_params);
+ if (res != 0)
+ goto out_destroy;
+
+ if (virt_dev->filename == NULL) {
+ PRINT_ERROR("File name required (device %s)", virt_dev->name);
+ res = -EINVAL;
+ goto out_destroy;
+ }
+
+ list_add_tail(&virt_dev->vdev_list_entry, &vdev_list);
+
+ vdisk_report_registering(virt_dev);
+
+ virt_dev->virt_id = scst_register_virtual_device(virt_dev->vdev_devt,
+ virt_dev->name);
+ if (virt_dev->virt_id < 0) {
+ res = virt_dev->virt_id;
+ goto out_del;
+ }
+
+ TRACE_DBG("Registered virt_dev %s with id %d", virt_dev->name,
+ virt_dev->virt_id);
+
+out:
+ return res;
+
+out_del:
+ list_del(&virt_dev->vdev_list_entry);
+
+out_destroy:
+ vdev_destroy(virt_dev);
+ goto out;
+}
+
+/* scst_vdisk_mutex supposed to be held */
+static int vdev_nullio_add_device(const char *device_name, char *params)
+{
+ int res = 0;
+ const char *allowed_params[] = { "read_only", "removable",
+ "blocksize", NULL };
+ struct scst_vdisk_dev *virt_dev;
+
+ res = vdev_create(&vdisk_null_devtype, device_name, &virt_dev);
+ if (res != 0)
+ goto out;
+
+ virt_dev->command_set_version = 0x04C0; /* SBC-3 */
+
+ virt_dev->nullio = 1;
+
+ res = vdev_parse_add_dev_params(virt_dev, params, allowed_params);
+ if (res != 0)
+ goto out_destroy;
+
+ list_add_tail(&virt_dev->vdev_list_entry, &vdev_list);
+
+ vdisk_report_registering(virt_dev);
+
+ virt_dev->virt_id = scst_register_virtual_device(virt_dev->vdev_devt,
+ virt_dev->name);
+ if (virt_dev->virt_id < 0) {
+ res = virt_dev->virt_id;
+ goto out_del;
+ }
+
+ TRACE_DBG("Registered virt_dev %s with id %d", virt_dev->name,
+ virt_dev->virt_id);
+
+out:
+ return res;
+
+out_del:
+ list_del(&virt_dev->vdev_list_entry);
+
+out_destroy:
+ vdev_destroy(virt_dev);
+ goto out;
+}
+
+static ssize_t vdisk_add_fileio_device(const char *device_name, char *params)
+{
+ int res;
+
+ if (mutex_lock_interruptible(&scst_vdisk_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = vdev_fileio_add_device(device_name, params);
+
+ mutex_unlock(&scst_vdisk_mutex);
+
+out:
+ return res;
+}
+
+static ssize_t vdisk_add_blockio_device(const char *device_name, char *params)
+{
+ int res;
+
+ if (mutex_lock_interruptible(&scst_vdisk_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = vdev_blockio_add_device(device_name, params);
+
+ mutex_unlock(&scst_vdisk_mutex);
+
+out:
+ return res;
+
+}
+
+static ssize_t vdisk_add_nullio_device(const char *device_name, char *params)
+{
+ int res;
+
+ if (mutex_lock_interruptible(&scst_vdisk_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = vdev_nullio_add_device(device_name, params);
+
+ mutex_unlock(&scst_vdisk_mutex);
+
+out:
+ return res;
+
+}
+
+/* scst_vdisk_mutex supposed to be held */
+static void vdev_del_device(struct scst_vdisk_dev *virt_dev)
+{
+
+ scst_unregister_virtual_device(virt_dev->virt_id);
+
+ list_del(&virt_dev->vdev_list_entry);
+
+ PRINT_INFO("Virtual device %s unregistered", virt_dev->name);
+ TRACE_DBG("virt_id %d unregistered", virt_dev->virt_id);
+
+ vdev_destroy(virt_dev);
+
+ return;
+}
+
+static ssize_t vdisk_del_device(const char *device_name)
+{
+ int res = 0;
+ struct scst_vdisk_dev *virt_dev;
+
+ if (mutex_lock_interruptible(&scst_vdisk_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ virt_dev = vdev_find(device_name);
+ if (virt_dev == NULL) {
+ PRINT_ERROR("Device %s not found", device_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ vdev_del_device(virt_dev);
+
+out_unlock:
+ mutex_unlock(&scst_vdisk_mutex);
+
+out:
+ return res;
+}
+
+/* scst_vdisk_mutex supposed to be held */
+static ssize_t __vcdrom_add_device(const char *device_name, char *params)
+{
+ int res = 0;
+ const char *allowed_params[] = { NULL }; /* no params */
+ struct scst_vdisk_dev *virt_dev;
+
+ res = vdev_create(&vcdrom_devtype, device_name, &virt_dev);
+ if (res != 0)
+ goto out;
+
+ virt_dev->command_set_version = 0x02A0; /* MMC-3 */
+
+ virt_dev->rd_only = 1;
+ virt_dev->removable = 1;
+ virt_dev->cdrom_empty = 1;
+
+ virt_dev->block_size = DEF_CDROM_BLOCKSIZE;
+ virt_dev->block_shift = DEF_CDROM_BLOCKSIZE_SHIFT;
+
+ res = vdev_parse_add_dev_params(virt_dev, params, allowed_params);
+ if (res != 0)
+ goto out_destroy;
+
+ list_add_tail(&virt_dev->vdev_list_entry, &vdev_list);
+
+ vdisk_report_registering(virt_dev);
+
+ virt_dev->virt_id = scst_register_virtual_device(virt_dev->vdev_devt,
+ virt_dev->name);
+ if (virt_dev->virt_id < 0) {
+ res = virt_dev->virt_id;
+ goto out_del;
+ }
+
+ TRACE_DBG("Registered virt_dev %s with id %d", virt_dev->name,
+ virt_dev->virt_id);
+
+out:
+ return res;
+
+out_del:
+ list_del(&virt_dev->vdev_list_entry);
+
+out_destroy:
+ vdev_destroy(virt_dev);
+ goto out;
+}
+
+static ssize_t vcdrom_add_device(const char *device_name, char *params)
+{
+ int res;
+
+ if (mutex_lock_interruptible(&scst_vdisk_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = __vcdrom_add_device(device_name, params);
+
+ mutex_unlock(&scst_vdisk_mutex);
+
+out:
+ return res;
+
+}
+
+static ssize_t vcdrom_del_device(const char *device_name)
+{
+ int res = 0;
+ struct scst_vdisk_dev *virt_dev;
+
+ if (mutex_lock_interruptible(&scst_vdisk_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ virt_dev = vdev_find(device_name);
+ if (virt_dev == NULL) {
+ PRINT_ERROR("Device %s not found", device_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ vdev_del_device(virt_dev);
+
+out_unlock:
+ mutex_unlock(&scst_vdisk_mutex);
+
+out:
+ return res;
+}
+
+static int vcdrom_change(struct scst_vdisk_dev *virt_dev,
+ char *buffer)
+{
+ loff_t err;
+ char *old_fn, *p, *pp;
+ const char *filename = NULL;
+ int length = strlen(buffer);
+ int res = 0;
+
+ p = buffer;
+
+ while (isspace(*p) && *p != '\0')
+ p++;
+ filename = p;
+ p = &buffer[length-1];
+ pp = &buffer[length];
+ while (isspace(*p) && (*p != '\0')) {
+ pp = p;
+ p--;
+ }
+ *pp = '\0';
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ /* To sync with detach*() functions */
+ mutex_lock(&scst_mutex);
+
+ if (*filename == '\0') {
+ virt_dev->cdrom_empty = 1;
+ TRACE_DBG("%s", "No media");
+ } else if (*filename != '/') {
+ PRINT_ERROR("File path \"%s\" is not absolute", filename);
+ res = -EINVAL;
+ goto out_unlock;
+ } else
+ virt_dev->cdrom_empty = 0;
+
+ old_fn = virt_dev->filename;
+
+ if (!virt_dev->cdrom_empty) {
+ int len = strlen(filename) + 1;
+ char *fn = kmalloc(len, GFP_KERNEL);
+ if (fn == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Allocation of filename failed");
+ res = -ENOMEM;
+ goto out_unlock;
+ }
+
+ strlcpy(fn, filename, len);
+ virt_dev->filename = fn;
+
+ res = vdisk_get_file_size(virt_dev->filename,
+ virt_dev->blockio, &err);
+ if (res != 0)
+ goto out_free_fn;
+ } else {
+ err = 0;
+ virt_dev->filename = NULL;
+ }
+
+ if (virt_dev->prevent_allow_medium_removal) {
+ PRINT_ERROR("Prevent medium removal for "
+ "virtual device with name %s", virt_dev->name);
+ res = -EINVAL;
+ goto out_free_fn;
+ }
+
+ virt_dev->file_size = err;
+ virt_dev->nblocks = virt_dev->file_size >> virt_dev->block_shift;
+ if (!virt_dev->cdrom_empty)
+ virt_dev->media_changed = 1;
+
+ mutex_unlock(&scst_mutex);
+
+ scst_dev_del_all_thr_data(virt_dev->dev);
+
+ if (!virt_dev->cdrom_empty) {
+ PRINT_INFO("Changed SCSI target virtual cdrom %s "
+ "(file=\"%s\", fs=%lldMB, bs=%d, nblocks=%lld,"
+ " cyln=%lld%s)", virt_dev->name,
+ vdev_get_filename(virt_dev),
+ virt_dev->file_size >> 20, virt_dev->block_size,
+ (long long unsigned int)virt_dev->nblocks,
+ (long long unsigned int)virt_dev->nblocks/64/32,
+ virt_dev->nblocks < 64*32 ? " !WARNING! cyln less "
+ "than 1" : "");
+ } else {
+ PRINT_INFO("Removed media from SCSI target virtual cdrom %s",
+ virt_dev->name);
+ }
+
+ kfree(old_fn);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+
+out_free_fn:
+ kfree(virt_dev->filename);
+ virt_dev->filename = old_fn;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ goto out_resume;
+}
+
+static int vcdrom_sysfs_process_filename_store(struct scst_sysfs_work_item *work)
+{
+ int res;
+ struct scst_device *dev = work->dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ /* It's safe, since we taken dev_kobj and dh_priv NULLed in attach() */
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ res = vcdrom_change(virt_dev, work->buf);
+
+ kobject_put(&dev->dev_kobj);
+ return res;
+}
+
+static ssize_t vcdrom_sysfs_filename_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ char *i_buf;
+ struct scst_sysfs_work_item *work;
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ i_buf = kmalloc(count+1, GFP_KERNEL);
+ if (i_buf == NULL) {
+ PRINT_ERROR("Unable to alloc intermediate buffer with size %zd",
+ count+1);
+ res = -ENOMEM;
+ goto out;
+ }
+ memcpy(i_buf, buf, count);
+ i_buf[count] = '\0';
+
+ res = scst_alloc_sysfs_work(vcdrom_sysfs_process_filename_store, &work);
+ if (res != 0)
+ goto out_free;
+
+ work->buf = i_buf;
+ work->dev = dev;
+
+ kobject_get(&dev->dev_kobj);
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+
+out_free:
+ kfree(i_buf);
+ goto out;
+}
+
+static ssize_t vdev_sysfs_size_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%lld\n", virt_dev->file_size / 1024 / 1024);
+ return pos;
+}
+
+static ssize_t vdisk_sysfs_blocksize_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%d\n%s", (int)virt_dev->block_size,
+ (virt_dev->block_size == DEF_DISK_BLOCKSIZE) ? "" :
+ SCST_SYSFS_KEY_MARK "\n");
+ return pos;
+}
+
+static ssize_t vdisk_sysfs_rd_only_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%d\n%s", virt_dev->rd_only ? 1 : 0,
+ (virt_dev->rd_only == DEF_RD_ONLY) ? "" :
+ SCST_SYSFS_KEY_MARK "");
+ return pos;
+}
+
+static ssize_t vdisk_sysfs_wt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%d\n%s", virt_dev->wt_flag ? 1 : 0,
+ (virt_dev->wt_flag == DEF_WRITE_THROUGH) ? "" :
+ SCST_SYSFS_KEY_MARK "");
+ return pos;
+}
+
+static ssize_t vdisk_sysfs_nv_cache_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%d\n%s", virt_dev->nv_cache ? 1 : 0,
+ (virt_dev->nv_cache == DEF_NV_CACHE) ? "" :
+ SCST_SYSFS_KEY_MARK "");
+ return pos;
+}
+
+static ssize_t vdisk_sysfs_o_direct_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%d\n%s", virt_dev->o_direct_flag ? 1 : 0,
+ (virt_dev->o_direct_flag == DEF_O_DIRECT) ? "" :
+ SCST_SYSFS_KEY_MARK "");
+ return pos;
+}
+
+static ssize_t vdisk_sysfs_removable_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%d\n", virt_dev->removable ? 1 : 0);
+
+ if ((virt_dev->dev->type != TYPE_ROM) &&
+ (virt_dev->removable != DEF_REMOVABLE))
+ pos += sprintf(&buf[pos], "%s\n", SCST_SYSFS_KEY_MARK);
+ return pos;
+}
+
+static int vdev_sysfs_process_get_filename(struct scst_sysfs_work_item *work)
+{
+ int res = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = work->dev;
+
+ if (mutex_lock_interruptible(&scst_vdisk_mutex) != 0) {
+ res = -EINTR;
+ goto out_put;
+ }
+
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ if (virt_dev == NULL)
+ goto out_unlock;
+
+ if (virt_dev->filename != NULL)
+ work->res_buf = kasprintf(GFP_KERNEL, "%s\n%s\n",
+ vdev_get_filename(virt_dev), SCST_SYSFS_KEY_MARK);
+ else
+ work->res_buf = kasprintf(GFP_KERNEL, "%s\n",
+ vdev_get_filename(virt_dev));
+
+out_unlock:
+ mutex_unlock(&scst_vdisk_mutex);
+
+out_put:
+ kobject_put(&dev->dev_kobj);
+ return res;
+}
+
+static ssize_t vdev_sysfs_filename_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int res = 0;
+ struct scst_device *dev;
+ struct scst_sysfs_work_item *work;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ res = scst_alloc_sysfs_work(vdev_sysfs_process_get_filename, &work);
+ if (res != 0)
+ goto out;
+
+ work->dev = dev;
+
+ kobject_get(&dev->dev_kobj);
+
+ scst_sysfs_work_get(work);
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res != 0)
+ goto out_put;
+
+ res = snprintf(buf, SCST_SYSFS_BLOCK_SIZE, "%s\n", work->res_buf);
+
+out_put:
+ scst_sysfs_work_put(work);
+
+out:
+ return res;
+}
+
+static int vdisk_sysfs_process_resync_size_store(
+ struct scst_sysfs_work_item *work)
+{
+ int res;
+ struct scst_device *dev = work->dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ /* It's safe, since we taken dev_kobj and dh_priv NULLed in attach() */
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ res = vdisk_resync_size(virt_dev);
+
+ kobject_put(&dev->dev_kobj);
+ return res;
+}
+
+static ssize_t vdisk_sysfs_resync_size_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_device *dev;
+ struct scst_sysfs_work_item *work;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ res = scst_alloc_sysfs_work(vdisk_sysfs_process_resync_size_store,
+ &work);
+ if (res != 0)
+ goto out;
+
+ work->dev = dev;
+
+ kobject_get(&dev->dev_kobj);
+
+ res = scst_sysfs_queue_wait_work(work);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+static ssize_t vdev_sysfs_t10_dev_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res, i;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ write_lock_bh(&vdisk_t10_dev_id_rwlock);
+
+ if ((count > sizeof(virt_dev->t10_dev_id)) ||
+ ((count == sizeof(virt_dev->t10_dev_id)) &&
+ (buf[count-1] != '\n'))) {
+ PRINT_ERROR("T10 device id is too long (max %zd "
+ "characters)", sizeof(virt_dev->t10_dev_id)-1);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ memset(virt_dev->t10_dev_id, 0, sizeof(virt_dev->t10_dev_id));
+ memcpy(virt_dev->t10_dev_id, buf, count);
+
+ i = 0;
+ while (i < sizeof(virt_dev->t10_dev_id)) {
+ if (virt_dev->t10_dev_id[i] == '\n') {
+ virt_dev->t10_dev_id[i] = '\0';
+ break;
+ }
+ i++;
+ }
+
+ virt_dev->t10_dev_id_set = 1;
+
+ res = count;
+
+ PRINT_INFO("T10 device id for device %s changed to %s", virt_dev->name,
+ virt_dev->t10_dev_id);
+
+out_unlock:
+ write_unlock_bh(&vdisk_t10_dev_id_rwlock);
+ return res;
+}
+
+static ssize_t vdev_sysfs_t10_dev_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ read_lock_bh(&vdisk_t10_dev_id_rwlock);
+ pos = sprintf(buf, "%s\n%s", virt_dev->t10_dev_id,
+ virt_dev->t10_dev_id_set ? SCST_SYSFS_KEY_MARK "\n" : "");
+ read_unlock_bh(&vdisk_t10_dev_id_rwlock);
+ return pos;
+}
+
+static ssize_t vdev_sysfs_usn_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+ struct scst_vdisk_dev *virt_dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+ virt_dev = (struct scst_vdisk_dev *)dev->dh_priv;
+
+ pos = sprintf(buf, "%s\n", virt_dev->usn);
+ return pos;
+}
+
+static int __init init_scst_vdisk(struct scst_dev_type *devtype)
+{
+ int res = 0;
+
+ devtype->module = THIS_MODULE;
+
+ res = scst_register_virtual_dev_driver(devtype);
+ if (res < 0)
+ goto out;
+
+out:
+ return res;
+
+}
+
+static void exit_scst_vdisk(struct scst_dev_type *devtype)
+{
+
+ mutex_lock(&scst_vdisk_mutex);
+ while (1) {
+ struct scst_vdisk_dev *virt_dev;
+
+ if (list_empty(&vdev_list))
+ break;
+
+ virt_dev = list_entry(vdev_list.next, typeof(*virt_dev),
+ vdev_list_entry);
+
+ vdev_del_device(virt_dev);
+ }
+ mutex_unlock(&scst_vdisk_mutex);
+
+ scst_unregister_virtual_dev_driver(devtype);
+ return;
+}
+
+static int __init init_scst_vdisk_driver(void)
+{
+ int res;
+
+ vdisk_thr_cachep = KMEM_CACHE(scst_vdisk_thr, SCST_SLAB_FLAGS);
+ if (vdisk_thr_cachep == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ blockio_work_cachep = KMEM_CACHE(scst_blockio_work, SCST_SLAB_FLAGS);
+ if (blockio_work_cachep == NULL) {
+ res = -ENOMEM;
+ goto out_free_vdisk_cache;
+ }
+
+ if (num_threads < 1) {
+ PRINT_ERROR("num_threads can not be less than 1, use "
+ "default %d", DEF_NUM_THREADS);
+ num_threads = DEF_NUM_THREADS;
+ }
+
+ vdisk_file_devtype.threads_num = num_threads;
+ vcdrom_devtype.threads_num = num_threads;
+
+ atomic_set(&nullio_thr_data.hdr.ref, 1); /* never destroy it */
+
+ res = init_scst_vdisk(&vdisk_file_devtype);
+ if (res != 0)
+ goto out_free_slab;
+
+ res = init_scst_vdisk(&vdisk_blk_devtype);
+ if (res != 0)
+ goto out_free_vdisk;
+
+ res = init_scst_vdisk(&vdisk_null_devtype);
+ if (res != 0)
+ goto out_free_blk;
+
+ res = init_scst_vdisk(&vcdrom_devtype);
+ if (res != 0)
+ goto out_free_null;
+
+out:
+ return res;
+
+out_free_null:
+ exit_scst_vdisk(&vdisk_null_devtype);
+
+out_free_blk:
+ exit_scst_vdisk(&vdisk_blk_devtype);
+
+out_free_vdisk:
+ exit_scst_vdisk(&vdisk_file_devtype);
+
+out_free_slab:
+ kmem_cache_destroy(blockio_work_cachep);
+
+out_free_vdisk_cache:
+ kmem_cache_destroy(vdisk_thr_cachep);
+ goto out;
+}
+
+static void __exit exit_scst_vdisk_driver(void)
+{
+ exit_scst_vdisk(&vdisk_null_devtype);
+ exit_scst_vdisk(&vdisk_blk_devtype);
+ exit_scst_vdisk(&vdisk_file_devtype);
+ exit_scst_vdisk(&vcdrom_devtype);
+
+ kmem_cache_destroy(blockio_work_cachep);
+ kmem_cache_destroy(vdisk_thr_cachep);
+}
+
+module_init(init_scst_vdisk_driver);
+module_exit(exit_scst_vdisk_driver);
+
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI disk (type 0) and CDROM (type 5) dev handler for "
+ "SCST using files on file systems or block devices");
+MODULE_VERSION(SCST_VERSION_STRING);
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 14/17]: SCST pass-through dev handlers
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (12 preceding siblings ...)
2010-09-14 14:47 ` [PATCH 13/17]: SCST vdisk dev handler Vladislav Bolkhovitin
@ 2010-09-14 14:48 ` Vladislav Bolkhovitin
2010-09-14 14:49 ` [PATCH 15/17]: Implementation of blk_rq_map_kern_sg() Vladislav Bolkhovitin
` (3 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:48 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST pass-through dev handlers.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_cdrom.c | 245 +++++++++++++++++++++
scst_changer.c | 167 ++++++++++++++
scst_dev_handler.h | 27 ++
scst_disk.c | 608 +++++++++++++++++++++++++++++++++++++++++++++++++++++
scst_modisk.c | 328 ++++++++++++++++++++++++++++
scst_processor.c | 167 ++++++++++++++
scst_raid.c | 168 ++++++++++++++
scst_tape.c | 361 +++++++++++++++++++++++++++++++
8 files changed, 2071 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_cdrom.c linux-2.6.35/drivers/scst/dev_handlers/scst_cdrom.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_cdrom.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_cdrom.c
@@ -0,0 +1,245 @@
+/*
+ * scst_cdrom.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI CDROM (type 5) dev handler
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/cdrom.h>
+#include <scsi/scsi_host.h>
+#include <linux/slab.h>
+
+#define LOG_PREFIX "dev_cdrom"
+
+#include <scst/scst.h>
+#include "scst_dev_handler.h"
+
+#define CDROM_NAME "dev_cdrom"
+
+#define CDROM_DEF_BLOCK_SHIFT 11
+
+struct cdrom_params {
+ int block_shift;
+};
+
+static int cdrom_attach(struct scst_device *);
+static void cdrom_detach(struct scst_device *);
+static int cdrom_parse(struct scst_cmd *);
+static int cdrom_done(struct scst_cmd *);
+
+static struct scst_dev_type cdrom_devtype = {
+ .name = CDROM_NAME,
+ .type = TYPE_ROM,
+ .threads_num = 1,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = cdrom_attach,
+ .detach = cdrom_detach,
+ .parse = cdrom_parse,
+ .dev_done = cdrom_done,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static int cdrom_attach(struct scst_device *dev)
+{
+ int res, rc;
+ uint8_t cmd[10];
+ const int buffer_size = 512;
+ uint8_t *buffer = NULL;
+ int retries;
+ unsigned char sense_buffer[SCSI_SENSE_BUFFERSIZE];
+ enum dma_data_direction data_dir;
+ struct cdrom_params *params;
+
+ if (dev->scsi_dev == NULL ||
+ dev->scsi_dev->type != dev->type) {
+ PRINT_ERROR("%s", "SCSI device not define or illegal type");
+ res = -ENODEV;
+ goto out;
+ }
+
+ params = kzalloc(sizeof(*params), GFP_KERNEL);
+ if (params == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Unable to allocate struct cdrom_params");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ buffer = kmalloc(buffer_size, GFP_KERNEL);
+ if (!buffer) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Memory allocation failure");
+ res = -ENOMEM;
+ goto out_free_params;
+ }
+
+ /* Clear any existing UA's and get cdrom capacity (cdrom block size) */
+ memset(cmd, 0, sizeof(cmd));
+ cmd[0] = READ_CAPACITY;
+ cmd[1] = (dev->scsi_dev->scsi_level <= SCSI_2) ?
+ ((dev->scsi_dev->lun << 5) & 0xe0) : 0;
+ retries = SCST_DEV_UA_RETRIES;
+ while (1) {
+ memset(buffer, 0, buffer_size);
+ memset(sense_buffer, 0, sizeof(sense_buffer));
+ data_dir = SCST_DATA_READ;
+
+ TRACE_DBG("%s", "Doing READ_CAPACITY");
+ rc = scsi_execute(dev->scsi_dev, cmd, data_dir, buffer,
+ buffer_size, sense_buffer,
+ SCST_GENERIC_CDROM_REG_TIMEOUT, 3, 0
+ , NULL
+ );
+
+ TRACE_DBG("READ_CAPACITY done: %x", rc);
+
+ if ((rc == 0) ||
+ !scst_analyze_sense(sense_buffer,
+ sizeof(sense_buffer), SCST_SENSE_KEY_VALID,
+ UNIT_ATTENTION, 0, 0))
+ break;
+
+ if (!--retries) {
+ PRINT_ERROR("UA not cleared after %d retries",
+ SCST_DEV_UA_RETRIES);
+ params->block_shift = CDROM_DEF_BLOCK_SHIFT;
+ res = -ENODEV;
+ goto out_free_buf;
+ }
+ }
+
+ if (rc == 0) {
+ int sector_size = ((buffer[4] << 24) | (buffer[5] << 16) |
+ (buffer[6] << 8) | (buffer[7] << 0));
+ if (sector_size == 0)
+ params->block_shift = CDROM_DEF_BLOCK_SHIFT;
+ else
+ params->block_shift =
+ scst_calc_block_shift(sector_size);
+ TRACE_DBG("Sector size is %i scsi_level %d(SCSI_2 %d)",
+ sector_size, dev->scsi_dev->scsi_level, SCSI_2);
+ } else {
+ params->block_shift = CDROM_DEF_BLOCK_SHIFT;
+ TRACE(TRACE_MINOR, "Read capacity failed: %x, using default "
+ "sector size %d", rc, params->block_shift);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Returned sense", sense_buffer,
+ sizeof(sense_buffer));
+ }
+
+ res = scst_obtain_device_parameters(dev);
+ if (res != 0) {
+ PRINT_ERROR("Failed to obtain control parameters for device "
+ "%s", dev->virt_name);
+ goto out_free_buf;
+ }
+
+out_free_buf:
+ kfree(buffer);
+
+out_free_params:
+ if (res == 0)
+ dev->dh_priv = params;
+ else
+ kfree(params);
+
+out:
+ return res;
+}
+
+static void cdrom_detach(struct scst_device *dev)
+{
+ struct cdrom_params *params =
+ (struct cdrom_params *)dev->dh_priv;
+
+ kfree(params);
+ dev->dh_priv = NULL;
+ return;
+}
+
+static int cdrom_get_block_shift(struct scst_cmd *cmd)
+{
+ struct cdrom_params *params = (struct cdrom_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ return params->block_shift;
+}
+
+static int cdrom_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ scst_cdrom_generic_parse(cmd, cdrom_get_block_shift);
+
+ cmd->retries = SCST_PASSTHROUGH_RETRIES;
+
+ return res;
+}
+
+static void cdrom_set_block_shift(struct scst_cmd *cmd, int block_shift)
+{
+ struct cdrom_params *params = (struct cdrom_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ if (block_shift != 0)
+ params->block_shift = block_shift;
+ else
+ params->block_shift = CDROM_DEF_BLOCK_SHIFT;
+ return;
+}
+
+static int cdrom_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ res = scst_block_generic_dev_done(cmd, cdrom_set_block_shift);
+ return res;
+}
+
+static int __init cdrom_init(void)
+{
+ int res = 0;
+
+ cdrom_devtype.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&cdrom_devtype);
+ if (res < 0)
+ goto out;
+
+out:
+ return res;
+
+}
+
+static void __exit cdrom_exit(void)
+{
+ scst_unregister_dev_driver(&cdrom_devtype);
+ return;
+}
+
+module_init(cdrom_init);
+module_exit(cdrom_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_DESCRIPTION("SCSI CDROM (type 5) dev handler for SCST");
+MODULE_VERSION(SCST_VERSION_STRING);
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_changer.c linux-2.6.35/drivers/scst/dev_handlers/scst_changer.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_changer.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_changer.c
@@ -0,0 +1,167 @@
+/*
+ * scst_changer.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI medium changer (type 8) dev handler
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <scsi/scsi_host.h>
+#include <linux/slab.h>
+
+#define LOG_PREFIX "dev_changer"
+
+#include <scst/scst.h>
+#include "scst_dev_handler.h"
+
+#define CHANGER_NAME "dev_changer"
+
+#define CHANGER_RETRIES 2
+
+static int changer_attach(struct scst_device *);
+/* static void changer_detach(struct scst_device *); */
+static int changer_parse(struct scst_cmd *);
+/* static int changer_done(struct scst_cmd *); */
+
+static struct scst_dev_type changer_devtype = {
+ .name = CHANGER_NAME,
+ .type = TYPE_MEDIUM_CHANGER,
+ .threads_num = 1,
+ .parse_atomic = 1,
+/* .dev_done_atomic = 1, */
+ .attach = changer_attach,
+/* .detach = changer_detach, */
+ .parse = changer_parse,
+/* .dev_done = changer_done */
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static int changer_attach(struct scst_device *dev)
+{
+ int res, rc;
+ int retries;
+
+ if (dev->scsi_dev == NULL ||
+ dev->scsi_dev->type != dev->type) {
+ PRINT_ERROR("%s", "SCSI device not define or illegal type");
+ res = -ENODEV;
+ goto out;
+ }
+
+ /*
+ * If the device is offline, don't try to read capacity or any
+ * of the other stuff
+ */
+ if (dev->scsi_dev->sdev_state == SDEV_OFFLINE) {
+ TRACE_DBG("%s", "Device is offline");
+ res = -ENODEV;
+ goto out;
+ }
+
+ retries = SCST_DEV_UA_RETRIES;
+ do {
+ TRACE_DBG("%s", "Doing TEST_UNIT_READY");
+ rc = scsi_test_unit_ready(dev->scsi_dev,
+ SCST_GENERIC_CHANGER_TIMEOUT, CHANGER_RETRIES
+ , NULL);
+ TRACE_DBG("TEST_UNIT_READY done: %x", rc);
+ } while ((--retries > 0) && rc);
+
+ if (rc) {
+ PRINT_WARNING("Unit not ready: %x", rc);
+ /* Let's try not to be too smart and continue processing */
+ }
+
+ res = scst_obtain_device_parameters(dev);
+ if (res != 0) {
+ PRINT_ERROR("Failed to obtain control parameters for device "
+ "%s", dev->virt_name);
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+#if 0
+void changer_detach(struct scst_device *dev)
+{
+ return;
+}
+#endif
+
+static int changer_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ scst_changer_generic_parse(cmd, NULL);
+
+ cmd->retries = SCST_PASSTHROUGH_RETRIES;
+
+ return res;
+}
+
+#if 0
+int changer_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ /*
+ * SCST sets good defaults for cmd->is_send_status and
+ * cmd->resp_data_len based on cmd->status and cmd->data_direction,
+ * therefore change them only if necessary
+ */
+
+#if 0
+ switch (cmd->cdb[0]) {
+ default:
+ /* It's all good */
+ break;
+ }
+#endif
+ return res;
+}
+#endif
+
+static int __init changer_init(void)
+{
+ int res = 0;
+
+ changer_devtype.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&changer_devtype);
+ if (res < 0)
+ goto out;
+
+out:
+ return res;
+}
+
+static void __exit changer_exit(void)
+{
+ scst_unregister_dev_driver(&changer_devtype);
+ return;
+}
+
+module_init(changer_init);
+module_exit(changer_exit);
+
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI medium changer (type 8) dev handler for SCST");
+MODULE_VERSION(SCST_VERSION_STRING);
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_dev_handler.h linux-2.6.35/drivers/scst/dev_handlers/scst_dev_handler.h
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_dev_handler.h
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_dev_handler.h
@@ -0,0 +1,27 @@
+#ifndef __SCST_DEV_HANDLER_H
+#define __SCST_DEV_HANDLER_H
+
+#include <linux/module.h>
+#include <scsi/scsi_eh.h>
+#include <scst/scst_debug.h>
+
+#define SCST_DEV_UA_RETRIES 5
+#define SCST_PASSTHROUGH_RETRIES 0
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+#ifdef CONFIG_SCST_DEBUG
+#define SCST_DEFAULT_DEV_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_PID | \
+ TRACE_LINE | TRACE_FUNCTION | TRACE_MGMT | TRACE_MINOR | \
+ TRACE_MGMT_DEBUG | TRACE_SPECIAL)
+#else
+#define SCST_DEFAULT_DEV_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_MGMT | \
+ TRACE_SPECIAL)
+#endif
+
+static unsigned long dh_trace_flag = SCST_DEFAULT_DEV_LOG_FLAGS;
+#define trace_flag dh_trace_flag
+
+#endif /* defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+#endif /* __SCST_DEV_HANDLER_H */
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_disk.c linux-2.6.35/drivers/scst/dev_handlers/scst_disk.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_disk.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_disk.c
@@ -0,0 +1,608 @@
+/*
+ * scst_disk.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI disk (type 0) dev handler
+ * &
+ * SCSI disk (type 0) "performance" device handler (skip all READ and WRITE
+ * operations).
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <scsi/scsi_host.h>
+#include <linux/slab.h>
+#include <asm/unaligned.h>
+
+#define LOG_PREFIX "dev_disk"
+
+#include <scst/scst.h>
+#include "scst_dev_handler.h"
+
+# define DISK_NAME "dev_disk"
+# define DISK_PERF_NAME "dev_disk_perf"
+
+#define DISK_DEF_BLOCK_SHIFT 9
+
+struct disk_params {
+ int block_shift;
+};
+
+static int disk_attach(struct scst_device *dev);
+static void disk_detach(struct scst_device *dev);
+static int disk_parse(struct scst_cmd *cmd);
+static int disk_perf_exec(struct scst_cmd *cmd);
+static int disk_done(struct scst_cmd *cmd);
+static int disk_exec(struct scst_cmd *cmd);
+static bool disk_on_sg_tablesize_low(struct scst_cmd *cmd);
+
+static struct scst_dev_type disk_devtype = {
+ .name = DISK_NAME,
+ .type = TYPE_DISK,
+ .threads_num = 1,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = disk_attach,
+ .detach = disk_detach,
+ .parse = disk_parse,
+ .exec = disk_exec,
+ .on_sg_tablesize_low = disk_on_sg_tablesize_low,
+ .dev_done = disk_done,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static struct scst_dev_type disk_devtype_perf = {
+ .name = DISK_PERF_NAME,
+ .type = TYPE_DISK,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = disk_attach,
+ .detach = disk_detach,
+ .parse = disk_parse,
+ .exec = disk_perf_exec,
+ .dev_done = disk_done,
+ .on_sg_tablesize_low = disk_on_sg_tablesize_low,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static int __init init_scst_disk_driver(void)
+{
+ int res = 0;
+
+ disk_devtype.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&disk_devtype);
+ if (res < 0)
+ goto out;
+
+ disk_devtype_perf.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&disk_devtype_perf);
+ if (res < 0)
+ goto out_unreg;
+
+out:
+ return res;
+
+out_unreg:
+ scst_unregister_dev_driver(&disk_devtype);
+ goto out;
+}
+
+static void __exit exit_scst_disk_driver(void)
+{
+
+ scst_unregister_dev_driver(&disk_devtype_perf);
+ scst_unregister_dev_driver(&disk_devtype);
+ return;
+}
+
+module_init(init_scst_disk_driver);
+module_exit(exit_scst_disk_driver);
+
+static int disk_attach(struct scst_device *dev)
+{
+ int res, rc;
+ uint8_t cmd[10];
+ const int buffer_size = 512;
+ uint8_t *buffer = NULL;
+ int retries;
+ unsigned char sense_buffer[SCSI_SENSE_BUFFERSIZE];
+ enum dma_data_direction data_dir;
+ struct disk_params *params;
+
+ if (dev->scsi_dev == NULL ||
+ dev->scsi_dev->type != dev->type) {
+ PRINT_ERROR("%s", "SCSI device not define or illegal type");
+ res = -ENODEV;
+ goto out;
+ }
+
+ params = kzalloc(sizeof(*params), GFP_KERNEL);
+ if (params == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Unable to allocate struct disk_params");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ buffer = kmalloc(buffer_size, GFP_KERNEL);
+ if (!buffer) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Memory allocation failure");
+ res = -ENOMEM;
+ goto out_free_params;
+ }
+
+ /* Clear any existing UA's and get disk capacity (disk block size) */
+ memset(cmd, 0, sizeof(cmd));
+ cmd[0] = READ_CAPACITY;
+ cmd[1] = (dev->scsi_dev->scsi_level <= SCSI_2) ?
+ ((dev->scsi_dev->lun << 5) & 0xe0) : 0;
+ retries = SCST_DEV_UA_RETRIES;
+ while (1) {
+ memset(buffer, 0, buffer_size);
+ memset(sense_buffer, 0, sizeof(sense_buffer));
+ data_dir = SCST_DATA_READ;
+
+ TRACE_DBG("%s", "Doing READ_CAPACITY");
+ rc = scsi_execute(dev->scsi_dev, cmd, data_dir, buffer,
+ buffer_size, sense_buffer,
+ SCST_GENERIC_DISK_REG_TIMEOUT, 3, 0
+ , NULL
+ );
+
+ TRACE_DBG("READ_CAPACITY done: %x", rc);
+
+ if ((rc == 0) ||
+ !scst_analyze_sense(sense_buffer,
+ sizeof(sense_buffer), SCST_SENSE_KEY_VALID,
+ UNIT_ATTENTION, 0, 0))
+ break;
+ if (!--retries) {
+ PRINT_ERROR("UA not clear after %d retries",
+ SCST_DEV_UA_RETRIES);
+ res = -ENODEV;
+ goto out_free_buf;
+ }
+ }
+ if (rc == 0) {
+ int sector_size = ((buffer[4] << 24) | (buffer[5] << 16) |
+ (buffer[6] << 8) | (buffer[7] << 0));
+ if (sector_size == 0)
+ params->block_shift = DISK_DEF_BLOCK_SHIFT;
+ else
+ params->block_shift =
+ scst_calc_block_shift(sector_size);
+ } else {
+ params->block_shift = DISK_DEF_BLOCK_SHIFT;
+ TRACE(TRACE_MINOR, "Read capacity failed: %x, using default "
+ "sector size %d", rc, params->block_shift);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Returned sense", sense_buffer,
+ sizeof(sense_buffer));
+ }
+
+ res = scst_obtain_device_parameters(dev);
+ if (res != 0) {
+ PRINT_ERROR("Failed to obtain control parameters for device "
+ "%s", dev->virt_name);
+ goto out_free_buf;
+ }
+
+out_free_buf:
+ kfree(buffer);
+
+out_free_params:
+ if (res == 0)
+ dev->dh_priv = params;
+ else
+ kfree(params);
+
+out:
+ return res;
+}
+
+static void disk_detach(struct scst_device *dev)
+{
+ struct disk_params *params =
+ (struct disk_params *)dev->dh_priv;
+
+ kfree(params);
+ dev->dh_priv = NULL;
+ return;
+}
+
+static int disk_get_block_shift(struct scst_cmd *cmd)
+{
+ struct disk_params *params = (struct disk_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ return params->block_shift;
+}
+
+static int disk_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ scst_sbc_generic_parse(cmd, disk_get_block_shift);
+
+ cmd->retries = SCST_PASSTHROUGH_RETRIES;
+
+ return res;
+}
+
+static void disk_set_block_shift(struct scst_cmd *cmd, int block_shift)
+{
+ struct disk_params *params = (struct disk_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ if (block_shift != 0)
+ params->block_shift = block_shift;
+ else
+ params->block_shift = DISK_DEF_BLOCK_SHIFT;
+ return;
+}
+
+static int disk_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ res = scst_block_generic_dev_done(cmd, disk_set_block_shift);
+ return res;
+}
+
+static bool disk_on_sg_tablesize_low(struct scst_cmd *cmd)
+{
+ bool res;
+
+ switch (cmd->cdb[0]) {
+ case WRITE_6:
+ case READ_6:
+ case WRITE_10:
+ case READ_10:
+ case WRITE_VERIFY:
+ case WRITE_12:
+ case READ_12:
+ case WRITE_VERIFY_12:
+ case WRITE_16:
+ case READ_16:
+ case WRITE_VERIFY_16:
+ res = true;
+ /* See comment in disk_exec */
+ cmd->inc_expected_sn_on_done = 1;
+ break;
+ default:
+ res = false;
+ break;
+ }
+ return res;
+}
+
+struct disk_work {
+ struct scst_cmd *cmd;
+ struct completion disk_work_cmpl;
+ volatile int result;
+ unsigned int left;
+ uint64_t save_lba;
+ unsigned int save_len;
+ struct scatterlist *save_sg;
+ int save_sg_cnt;
+};
+
+static int disk_cdb_get_transfer_data(const uint8_t *cdb,
+ uint64_t *out_lba, unsigned int *out_length)
+{
+ int res;
+ uint64_t lba;
+ unsigned int len;
+
+ switch (cdb[0]) {
+ case WRITE_6:
+ case READ_6:
+ lba = be16_to_cpu(get_unaligned((__be16 *)&cdb[2]));
+ len = cdb[4];
+ break;
+ case WRITE_10:
+ case READ_10:
+ case WRITE_VERIFY:
+ lba = be32_to_cpu(get_unaligned((__be32 *)&cdb[2]));
+ len = be16_to_cpu(get_unaligned((__be16 *)&cdb[7]));
+ break;
+ case WRITE_12:
+ case READ_12:
+ case WRITE_VERIFY_12:
+ lba = be32_to_cpu(get_unaligned((__be32 *)&cdb[2]));
+ len = be32_to_cpu(get_unaligned((__be32 *)&cdb[6]));
+ break;
+ case WRITE_16:
+ case READ_16:
+ case WRITE_VERIFY_16:
+ lba = be64_to_cpu(get_unaligned((__be64 *)&cdb[2]));
+ len = be32_to_cpu(get_unaligned((__be32 *)&cdb[10]));
+ break;
+ default:
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = 0;
+ *out_lba = lba;
+ *out_length = len;
+
+ TRACE_DBG("LBA %lld, length %d", (unsigned long long)lba, len);
+
+out:
+ return res;
+}
+
+static int disk_cdb_set_transfer_data(uint8_t *cdb,
+ uint64_t lba, unsigned int len)
+{
+ int res;
+
+ switch (cdb[0]) {
+ case WRITE_6:
+ case READ_6:
+ put_unaligned(cpu_to_be16(lba), (__be16 *)&cdb[2]);
+ cdb[4] = len;
+ break;
+ case WRITE_10:
+ case READ_10:
+ case WRITE_VERIFY:
+ put_unaligned(cpu_to_be32(lba), (__be32 *)&cdb[2]);
+ put_unaligned(cpu_to_be16(len), (__be16 *)&cdb[7]);
+ break;
+ case WRITE_12:
+ case READ_12:
+ case WRITE_VERIFY_12:
+ put_unaligned(cpu_to_be32(lba), (__be32 *)&cdb[2]);
+ put_unaligned(cpu_to_be32(len), (__be32 *)&cdb[6]);
+ break;
+ case WRITE_16:
+ case READ_16:
+ case WRITE_VERIFY_16:
+ put_unaligned(cpu_to_be64(lba), (__be64 *)&cdb[2]);
+ put_unaligned(cpu_to_be32(len), (__be32 *)&cdb[10]);
+ break;
+ default:
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = 0;
+
+ TRACE_DBG("LBA %lld, length %d", (unsigned long long)lba, len);
+ TRACE_BUFFER("New CDB", cdb, SCST_MAX_CDB_SIZE);
+
+out:
+ return res;
+}
+
+static void disk_cmd_done(void *data, char *sense, int result, int resid)
+{
+ struct disk_work *work = data;
+
+ TRACE_DBG("work %p, cmd %p, left %d, result %d, sense %p, resid %d",
+ work, work->cmd, work->left, result, sense, resid);
+
+ if (result == SAM_STAT_GOOD)
+ goto out_complete;
+
+ work->result = result;
+
+ disk_cdb_set_transfer_data(work->cmd->cdb, work->save_lba, work->save_len);
+ work->cmd->sg = work->save_sg;
+ work->cmd->sg_cnt = work->save_sg_cnt;
+
+ scst_pass_through_cmd_done(work->cmd, sense, result, resid + work->left);
+
+out_complete:
+ complete_all(&work->disk_work_cmpl);
+ return;
+}
+
+/* Executes command and split CDB, if necessary */
+static int disk_exec(struct scst_cmd *cmd)
+{
+ int res, rc;
+ struct disk_params *params = (struct disk_params *)cmd->dev->dh_priv;
+ struct disk_work work;
+ unsigned int offset, cur_len;
+ struct scatterlist *sg, *start_sg;
+ int cur_sg_cnt;
+ int sg_tablesize = cmd->dev->scsi_dev->host->sg_tablesize;
+ int num, j;
+
+ if (likely((cmd->sg_cnt <= sg_tablesize) &&
+ (cmd->out_sg_cnt <= sg_tablesize))) {
+ res = SCST_EXEC_NOT_COMPLETED;
+ goto out;
+ }
+
+ memset(&work, 0, sizeof(work));
+ work.cmd = cmd;
+ work.save_sg = cmd->sg;
+ work.save_sg_cnt = cmd->sg_cnt;
+ rc = disk_cdb_get_transfer_data(cmd->cdb, &work.save_lba,
+ &work.save_len);
+ if (rc != 0)
+ goto out_error;
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ TRACE_DBG("cmd %p, save_sg %p, save_sg_cnt %d, save_lba %lld, "
+ "save_len %d (sg_tablesize %d, sizeof(*sg) 0x%zx)", cmd,
+ work.save_sg, work.save_sg_cnt,
+ (unsigned long long)work.save_lba, work.save_len,
+ sg_tablesize, sizeof(*sg));
+
+ /*
+ * If we submit all chunks async'ly, it will be very not trivial what
+ * to do if several of them finish with sense or residual. So, let's
+ * do it synchronously.
+ */
+
+ num = 1;
+ j = 0;
+ offset = 0;
+ cur_len = 0;
+ sg = work.save_sg;
+ start_sg = sg;
+ cur_sg_cnt = 0;
+ while (1) {
+ unsigned int l;
+
+ if (unlikely(sg_is_chain(&sg[j]))) {
+ bool reset_start_sg = (start_sg == &sg[j]);
+ sg = sg_chain_ptr(&sg[j]);
+ j = 0;
+ if (reset_start_sg)
+ start_sg = sg;
+ }
+
+ l = sg[j].length >> params->block_shift;
+ cur_len += l;
+ cur_sg_cnt++;
+
+ TRACE_DBG("l %d, j %d, num %d, offset %d, cur_len %d, "
+ "cur_sg_cnt %d, start_sg %p", l, j, num, offset,
+ cur_len, cur_sg_cnt, start_sg);
+
+ if (((num % sg_tablesize) == 0) || (num == work.save_sg_cnt)) {
+ TRACE_DBG("%s", "Execing...");
+
+ disk_cdb_set_transfer_data(cmd->cdb,
+ work.save_lba + offset, cur_len);
+ cmd->sg = start_sg;
+ cmd->sg_cnt = cur_sg_cnt;
+
+ work.left = work.save_len - (offset + cur_len);
+ init_completion(&work.disk_work_cmpl);
+
+ rc = scst_scsi_exec_async(cmd, &work, disk_cmd_done);
+ if (unlikely(rc != 0)) {
+ PRINT_ERROR("scst_scsi_exec_async() failed: %d",
+ rc);
+ goto out_err_restore;
+ }
+
+ wait_for_completion(&work.disk_work_cmpl);
+
+ if (work.result != SAM_STAT_GOOD) {
+ /* cmd can be already dead */
+ res = SCST_EXEC_COMPLETED;
+ goto out;
+ }
+
+ offset += cur_len;
+ cur_len = 0;
+ cur_sg_cnt = 0;
+ start_sg = &sg[j+1];
+
+ if (num == work.save_sg_cnt)
+ break;
+ }
+ num++;
+ j++;
+ }
+
+ cmd->completed = 1;
+
+out_restore:
+ disk_cdb_set_transfer_data(cmd->cdb, work.save_lba, work.save_len);
+ cmd->sg = work.save_sg;
+ cmd->sg_cnt = work.save_sg_cnt;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+
+out:
+ return res;
+
+out_err_restore:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_restore;
+
+out_error:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_done;
+}
+
+static int disk_perf_exec(struct scst_cmd *cmd)
+{
+ int res, rc;
+ int opcode = cmd->cdb[0];
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ switch (opcode) {
+ case WRITE_6:
+ case WRITE_10:
+ case WRITE_12:
+ case WRITE_16:
+ case READ_6:
+ case READ_10:
+ case READ_12:
+ case READ_16:
+ case WRITE_VERIFY:
+ case WRITE_VERIFY_12:
+ case WRITE_VERIFY_16:
+ goto out_complete;
+ }
+
+ res = SCST_EXEC_NOT_COMPLETED;
+
+out:
+ return res;
+
+out_complete:
+ cmd->completed = 1;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out;
+}
+
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI disk (type 0) dev handler for SCST");
+MODULE_VERSION(SCST_VERSION_STRING);
+
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_modisk.c linux-2.6.35/drivers/scst/dev_handlers/scst_modisk.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_modisk.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_modisk.c
@@ -0,0 +1,328 @@
+/*
+ * scst_modisk.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI MO disk (type 7) dev handler
+ * &
+ * SCSI MO disk (type 7) "performance" device handler (skip all READ and WRITE
+ * operations).
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <scsi/scsi_host.h>
+#include <linux/slab.h>
+
+#define LOG_PREFIX "dev_modisk"
+
+#include <scst/scst.h>
+#include "scst_dev_handler.h"
+
+# define MODISK_NAME "dev_modisk"
+# define MODISK_PERF_NAME "dev_modisk_perf"
+
+#define MODISK_DEF_BLOCK_SHIFT 10
+
+struct modisk_params {
+ int block_shift;
+};
+
+static int modisk_attach(struct scst_device *);
+static void modisk_detach(struct scst_device *);
+static int modisk_parse(struct scst_cmd *);
+static int modisk_done(struct scst_cmd *);
+static int modisk_perf_exec(struct scst_cmd *);
+
+static struct scst_dev_type modisk_devtype = {
+ .name = MODISK_NAME,
+ .type = TYPE_MOD,
+ .threads_num = 1,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = modisk_attach,
+ .detach = modisk_detach,
+ .parse = modisk_parse,
+ .dev_done = modisk_done,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static struct scst_dev_type modisk_devtype_perf = {
+ .name = MODISK_PERF_NAME,
+ .type = TYPE_MOD,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = modisk_attach,
+ .detach = modisk_detach,
+ .parse = modisk_parse,
+ .dev_done = modisk_done,
+ .exec = modisk_perf_exec,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static int __init init_scst_modisk_driver(void)
+{
+ int res = 0;
+
+ modisk_devtype.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&modisk_devtype);
+ if (res < 0)
+ goto out;
+
+ modisk_devtype_perf.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&modisk_devtype_perf);
+ if (res < 0)
+ goto out_unreg;
+
+out:
+ return res;
+
+out_unreg:
+ scst_unregister_dev_driver(&modisk_devtype);
+ goto out;
+}
+
+static void __exit exit_scst_modisk_driver(void)
+{
+
+ scst_unregister_dev_driver(&modisk_devtype_perf);
+ scst_unregister_dev_driver(&modisk_devtype);
+ return;
+}
+
+module_init(init_scst_modisk_driver);
+module_exit(exit_scst_modisk_driver);
+
+static int modisk_attach(struct scst_device *dev)
+{
+ int res, rc;
+ uint8_t cmd[10];
+ const int buffer_size = 512;
+ uint8_t *buffer = NULL;
+ int retries;
+ unsigned char sense_buffer[SCSI_SENSE_BUFFERSIZE];
+ enum dma_data_direction data_dir;
+ struct modisk_params *params;
+
+ if (dev->scsi_dev == NULL ||
+ dev->scsi_dev->type != dev->type) {
+ PRINT_ERROR("%s", "SCSI device not define or illegal type");
+ res = -ENODEV;
+ goto out;
+ }
+
+ params = kzalloc(sizeof(*params), GFP_KERNEL);
+ if (params == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Unable to allocate struct modisk_params");
+ res = -ENOMEM;
+ goto out;
+ }
+ params->block_shift = MODISK_DEF_BLOCK_SHIFT;
+
+ /*
+ * If the device is offline, don't try to read capacity or any
+ * of the other stuff
+ */
+ if (dev->scsi_dev->sdev_state == SDEV_OFFLINE) {
+ TRACE_DBG("%s", "Device is offline");
+ res = -ENODEV;
+ goto out_free_params;
+ }
+
+ buffer = kmalloc(buffer_size, GFP_KERNEL);
+ if (!buffer) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Memory allocation failure");
+ res = -ENOMEM;
+ goto out_free_params;
+ }
+
+ /*
+ * Clear any existing UA's and get modisk capacity (modisk block
+ * size).
+ */
+ memset(cmd, 0, sizeof(cmd));
+ cmd[0] = READ_CAPACITY;
+ cmd[1] = (dev->scsi_dev->scsi_level <= SCSI_2) ?
+ ((dev->scsi_dev->lun << 5) & 0xe0) : 0;
+ retries = SCST_DEV_UA_RETRIES;
+ while (1) {
+ memset(buffer, 0, buffer_size);
+ memset(sense_buffer, 0, sizeof(sense_buffer));
+ data_dir = SCST_DATA_READ;
+
+ TRACE_DBG("%s", "Doing READ_CAPACITY");
+ rc = scsi_execute(dev->scsi_dev, cmd, data_dir, buffer,
+ buffer_size, sense_buffer,
+ SCST_GENERIC_MODISK_REG_TIMEOUT, 3, 0
+ , NULL
+ );
+
+ TRACE_DBG("READ_CAPACITY done: %x", rc);
+
+ if (!rc || !scst_analyze_sense(sense_buffer,
+ sizeof(sense_buffer), SCST_SENSE_KEY_VALID,
+ UNIT_ATTENTION, 0, 0))
+ break;
+
+ if (!--retries) {
+ PRINT_ERROR("UA not cleared after %d retries",
+ SCST_DEV_UA_RETRIES);
+ res = -ENODEV;
+ goto out_free_buf;
+ }
+ }
+
+ if (rc == 0) {
+ int sector_size = ((buffer[4] << 24) | (buffer[5] << 16) |
+ (buffer[6] << 8) | (buffer[7] << 0));
+ if (sector_size == 0)
+ params->block_shift = MODISK_DEF_BLOCK_SHIFT;
+ else
+ params->block_shift =
+ scst_calc_block_shift(sector_size);
+ TRACE_DBG("Sector size is %i scsi_level %d(SCSI_2 %d)",
+ sector_size, dev->scsi_dev->scsi_level, SCSI_2);
+ } else {
+ params->block_shift = MODISK_DEF_BLOCK_SHIFT;
+ TRACE(TRACE_MINOR, "Read capacity failed: %x, using default "
+ "sector size %d", rc, params->block_shift);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Returned sense", sense_buffer,
+ sizeof(sense_buffer));
+ }
+
+ res = scst_obtain_device_parameters(dev);
+ if (res != 0) {
+ PRINT_ERROR("Failed to obtain control parameters for device "
+ "%s: %x", dev->virt_name, res);
+ goto out_free_buf;
+ }
+
+out_free_buf:
+ kfree(buffer);
+
+out_free_params:
+ if (res == 0)
+ dev->dh_priv = params;
+ else
+ kfree(params);
+
+out:
+ return res;
+}
+
+static void modisk_detach(struct scst_device *dev)
+{
+ struct modisk_params *params =
+ (struct modisk_params *)dev->dh_priv;
+
+ kfree(params);
+ dev->dh_priv = NULL;
+ return;
+}
+
+static int modisk_get_block_shift(struct scst_cmd *cmd)
+{
+ struct modisk_params *params =
+ (struct modisk_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ return params->block_shift;
+}
+
+static int modisk_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ scst_modisk_generic_parse(cmd, modisk_get_block_shift);
+
+ cmd->retries = SCST_PASSTHROUGH_RETRIES;
+
+ return res;
+}
+
+static void modisk_set_block_shift(struct scst_cmd *cmd, int block_shift)
+{
+ struct modisk_params *params =
+ (struct modisk_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ if (block_shift != 0)
+ params->block_shift = block_shift;
+ else
+ params->block_shift = MODISK_DEF_BLOCK_SHIFT;
+ return;
+}
+
+static int modisk_done(struct scst_cmd *cmd)
+{
+ int res;
+
+ res = scst_block_generic_dev_done(cmd, modisk_set_block_shift);
+ return res;
+}
+
+static int modisk_perf_exec(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED, rc;
+ int opcode = cmd->cdb[0];
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ switch (opcode) {
+ case WRITE_6:
+ case WRITE_10:
+ case WRITE_12:
+ case WRITE_16:
+ case READ_6:
+ case READ_10:
+ case READ_12:
+ case READ_16:
+ cmd->completed = 1;
+ goto out_done;
+ }
+
+out:
+ return res;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out;
+}
+
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI MO disk (type 7) dev handler for SCST");
+MODULE_VERSION(SCST_VERSION_STRING);
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_processor.c linux-2.6.35/drivers/scst/dev_handlers/scst_processor.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_processor.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_processor.c
@@ -0,0 +1,167 @@
+/*
+ * scst_processor.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI medium processor (type 3) dev handler
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <scsi/scsi_host.h>
+#include <linux/slab.h>
+
+#define LOG_PREFIX "dev_processor"
+
+#include <scst/scst.h>
+#include "scst_dev_handler.h"
+
+#define PROCESSOR_NAME "dev_processor"
+
+#define PROCESSOR_RETRIES 2
+
+static int processor_attach(struct scst_device *);
+/*static void processor_detach(struct scst_device *);*/
+static int processor_parse(struct scst_cmd *);
+/*static int processor_done(struct scst_cmd *);*/
+
+static struct scst_dev_type processor_devtype = {
+ .name = PROCESSOR_NAME,
+ .type = TYPE_PROCESSOR,
+ .threads_num = 1,
+ .parse_atomic = 1,
+/* .dev_done_atomic = 1,*/
+ .attach = processor_attach,
+/* .detach = processor_detach,*/
+ .parse = processor_parse,
+/* .dev_done = processor_done*/
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static int processor_attach(struct scst_device *dev)
+{
+ int res, rc;
+ int retries;
+
+ if (dev->scsi_dev == NULL ||
+ dev->scsi_dev->type != dev->type) {
+ PRINT_ERROR("%s", "SCSI device not define or illegal type");
+ res = -ENODEV;
+ goto out;
+ }
+
+ /*
+ * If the device is offline, don't try to read capacity or any
+ * of the other stuff
+ */
+ if (dev->scsi_dev->sdev_state == SDEV_OFFLINE) {
+ TRACE_DBG("%s", "Device is offline");
+ res = -ENODEV;
+ goto out;
+ }
+
+ retries = SCST_DEV_UA_RETRIES;
+ do {
+ TRACE_DBG("%s", "Doing TEST_UNIT_READY");
+ rc = scsi_test_unit_ready(dev->scsi_dev,
+ SCST_GENERIC_PROCESSOR_TIMEOUT, PROCESSOR_RETRIES
+ , NULL);
+ TRACE_DBG("TEST_UNIT_READY done: %x", rc);
+ } while ((--retries > 0) && rc);
+
+ if (rc) {
+ PRINT_WARNING("Unit not ready: %x", rc);
+ /* Let's try not to be too smart and continue processing */
+ }
+
+ res = scst_obtain_device_parameters(dev);
+ if (res != 0) {
+ PRINT_ERROR("Failed to obtain control parameters for device "
+ "%s", dev->virt_name);
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+#if 0
+void processor_detach(struct scst_device *dev)
+{
+ return;
+}
+#endif
+
+static int processor_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ scst_processor_generic_parse(cmd, NULL);
+
+ cmd->retries = SCST_PASSTHROUGH_RETRIES;
+
+ return res;
+}
+
+#if 0
+int processor_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ /*
+ * SCST sets good defaults for cmd->is_send_status and
+ * cmd->resp_data_len based on cmd->status and cmd->data_direction,
+ * therefore change them only if necessary.
+ */
+
+#if 0
+ switch (cmd->cdb[0]) {
+ default:
+ /* It's all good */
+ break;
+ }
+#endif
+ return res;
+}
+#endif
+
+static int __init processor_init(void)
+{
+ int res = 0;
+
+ processor_devtype.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&processor_devtype);
+ if (res < 0)
+ goto out;
+
+out:
+ return res;
+}
+
+static void __exit processor_exit(void)
+{
+ scst_unregister_dev_driver(&processor_devtype);
+ return;
+}
+
+module_init(processor_init);
+module_exit(processor_exit);
+
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI medium processor (type 3) dev handler for SCST");
+MODULE_VERSION(SCST_VERSION_STRING);
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_raid.c linux-2.6.35/drivers/scst/dev_handlers/scst_raid.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_raid.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_raid.c
@@ -0,0 +1,168 @@
+/*
+ * scst_raid.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI raid(controller) (type 0xC) dev handler
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#define LOG_PREFIX "dev_raid"
+
+#include <scsi/scsi_host.h>
+#include <linux/slab.h>
+
+#include <scst/scst.h>
+#include "scst_dev_handler.h"
+
+#define RAID_NAME "dev_raid"
+
+#define RAID_RETRIES 2
+
+static int raid_attach(struct scst_device *);
+/* static void raid_detach(struct scst_device *); */
+static int raid_parse(struct scst_cmd *);
+/* static int raid_done(struct scst_cmd *); */
+
+static struct scst_dev_type raid_devtype = {
+ .name = RAID_NAME,
+ .type = TYPE_RAID,
+ .threads_num = 1,
+ .parse_atomic = 1,
+/* .dev_done_atomic = 1,*/
+ .attach = raid_attach,
+/* .detach = raid_detach,*/
+ .parse = raid_parse,
+/* .dev_done = raid_done,*/
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static int raid_attach(struct scst_device *dev)
+{
+ int res, rc;
+ int retries;
+
+ if (dev->scsi_dev == NULL ||
+ dev->scsi_dev->type != dev->type) {
+ PRINT_ERROR("%s", "SCSI device not define or illegal type");
+ res = -ENODEV;
+ goto out;
+ }
+
+ /*
+ * If the device is offline, don't try to read capacity or any
+ * of the other stuff
+ */
+ if (dev->scsi_dev->sdev_state == SDEV_OFFLINE) {
+ TRACE_DBG("%s", "Device is offline");
+ res = -ENODEV;
+ goto out;
+ }
+
+ retries = SCST_DEV_UA_RETRIES;
+ do {
+ TRACE_DBG("%s", "Doing TEST_UNIT_READY");
+ rc = scsi_test_unit_ready(dev->scsi_dev,
+ SCST_GENERIC_RAID_TIMEOUT, RAID_RETRIES
+ , NULL);
+ TRACE_DBG("TEST_UNIT_READY done: %x", rc);
+ } while ((--retries > 0) && rc);
+
+ if (rc) {
+ PRINT_WARNING("Unit not ready: %x", rc);
+ /* Let's try not to be too smart and continue processing */
+ }
+
+ res = scst_obtain_device_parameters(dev);
+ if (res != 0) {
+ PRINT_ERROR("Failed to obtain control parameters for device "
+ "%s", dev->virt_name);
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+#if 0
+void raid_detach(struct scst_device *dev)
+{
+ return;
+}
+#endif
+
+static int raid_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ scst_raid_generic_parse(cmd, NULL);
+
+ cmd->retries = SCST_PASSTHROUGH_RETRIES;
+
+ return res;
+}
+
+#if 0
+int raid_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ /*
+ * SCST sets good defaults for cmd->is_send_status and
+ * cmd->resp_data_len based on cmd->status and cmd->data_direction,
+ * therefore change them only if necessary.
+ */
+
+#if 0
+ switch (cmd->cdb[0]) {
+ default:
+ /* It's all good */
+ break;
+ }
+#endif
+ return res;
+}
+#endif
+
+static int __init raid_init(void)
+{
+ int res = 0;
+
+ raid_devtype.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&raid_devtype);
+ if (res < 0)
+ goto out;
+
+out:
+ return res;
+
+}
+
+static void __exit raid_exit(void)
+{
+ scst_unregister_dev_driver(&raid_devtype);
+ return;
+}
+
+module_init(raid_init);
+module_exit(raid_exit);
+
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI raid(controller) (type 0xC) dev handler for SCST");
+MODULE_VERSION(SCST_VERSION_STRING);
diff -uprN orig/linux-2.6.35/drivers/scst/dev_handlers/scst_tape.c linux-2.6.35/drivers/scst/dev_handlers/scst_tape.c
--- orig/linux-2.6.35/drivers/scst/dev_handlers/scst_tape.c
+++ linux-2.6.35/drivers/scst/dev_handlers/scst_tape.c
@@ -0,0 +1,361 @@
+/*
+ * scst_tape.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * SCSI tape (type 1) dev handler
+ * &
+ * SCSI tape (type 1) "performance" device handler (skip all READ and WRITE
+ * operations).
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <scsi/scsi_host.h>
+#include <linux/slab.h>
+
+#define LOG_PREFIX "dev_tape"
+
+#include <scst/scst.h>
+#include "scst_dev_handler.h"
+
+# define TAPE_NAME "dev_tape"
+# define TAPE_PERF_NAME "dev_tape_perf"
+
+#define TAPE_RETRIES 2
+
+#define TAPE_DEF_BLOCK_SIZE 512
+
+/* The fixed bit in READ/WRITE/VERIFY */
+#define SILI_BIT 2
+
+struct tape_params {
+ int block_size;
+};
+
+static int tape_attach(struct scst_device *);
+static void tape_detach(struct scst_device *);
+static int tape_parse(struct scst_cmd *);
+static int tape_done(struct scst_cmd *);
+static int tape_perf_exec(struct scst_cmd *);
+
+static struct scst_dev_type tape_devtype = {
+ .name = TAPE_NAME,
+ .type = TYPE_TAPE,
+ .threads_num = 1,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = tape_attach,
+ .detach = tape_detach,
+ .parse = tape_parse,
+ .dev_done = tape_done,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static struct scst_dev_type tape_devtype_perf = {
+ .name = TAPE_PERF_NAME,
+ .type = TYPE_TAPE,
+ .parse_atomic = 1,
+ .dev_done_atomic = 1,
+ .attach = tape_attach,
+ .detach = tape_detach,
+ .parse = tape_parse,
+ .dev_done = tape_done,
+ .exec = tape_perf_exec,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_DEFAULT_DEV_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static int __init init_scst_tape_driver(void)
+{
+ int res = 0;
+
+ tape_devtype.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&tape_devtype);
+ if (res < 0)
+ goto out;
+
+ tape_devtype_perf.module = THIS_MODULE;
+
+ res = scst_register_dev_driver(&tape_devtype_perf);
+ if (res < 0)
+ goto out_unreg;
+
+out:
+ return res;
+
+out_unreg:
+ scst_unregister_dev_driver(&tape_devtype);
+ goto out;
+}
+
+static void __exit exit_scst_tape_driver(void)
+{
+
+ scst_unregister_dev_driver(&tape_devtype_perf);
+ scst_unregister_dev_driver(&tape_devtype);
+ return;
+}
+
+module_init(init_scst_tape_driver);
+module_exit(exit_scst_tape_driver);
+
+static int tape_attach(struct scst_device *dev)
+{
+ int res, rc;
+ int retries;
+ struct scsi_mode_data data;
+ const int buffer_size = 512;
+ uint8_t *buffer = NULL;
+ struct tape_params *params;
+
+ if (dev->scsi_dev == NULL ||
+ dev->scsi_dev->type != dev->type) {
+ PRINT_ERROR("%s", "SCSI device not define or illegal type");
+ res = -ENODEV;
+ goto out;
+ }
+
+ params = kzalloc(sizeof(*params), GFP_KERNEL);
+ if (params == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Unable to allocate struct tape_params");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ params->block_size = TAPE_DEF_BLOCK_SIZE;
+
+ buffer = kmalloc(buffer_size, GFP_KERNEL);
+ if (!buffer) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Memory allocation failure");
+ res = -ENOMEM;
+ goto out_free_req;
+ }
+
+ retries = SCST_DEV_UA_RETRIES;
+ do {
+ TRACE_DBG("%s", "Doing TEST_UNIT_READY");
+ rc = scsi_test_unit_ready(dev->scsi_dev,
+ SCST_GENERIC_TAPE_SMALL_TIMEOUT, TAPE_RETRIES
+ , NULL);
+ TRACE_DBG("TEST_UNIT_READY done: %x", rc);
+ } while ((--retries > 0) && rc);
+
+ if (rc) {
+ PRINT_WARNING("Unit not ready: %x", rc);
+ /* Let's try not to be too smart and continue processing */
+ goto obtain;
+ }
+
+ TRACE_DBG("%s", "Doing MODE_SENSE");
+ rc = scsi_mode_sense(dev->scsi_dev,
+ ((dev->scsi_dev->scsi_level <= SCSI_2) ?
+ ((dev->scsi_dev->lun << 5) & 0xe0) : 0),
+ 0 /* Mode Page 0 */,
+ buffer, buffer_size,
+ SCST_GENERIC_TAPE_SMALL_TIMEOUT, TAPE_RETRIES,
+ &data, NULL);
+ TRACE_DBG("MODE_SENSE done: %x", rc);
+
+ if (rc == 0) {
+ int medium_type, mode, speed, density;
+ if (buffer[3] == 8) {
+ params->block_size = ((buffer[9] << 16) |
+ (buffer[10] << 8) |
+ (buffer[11] << 0));
+ } else
+ params->block_size = TAPE_DEF_BLOCK_SIZE;
+ medium_type = buffer[1];
+ mode = (buffer[2] & 0x70) >> 4;
+ speed = buffer[2] & 0x0f;
+ density = buffer[4];
+ TRACE_DBG("Tape: lun %d. bs %d. type 0x%02x mode 0x%02x "
+ "speed 0x%02x dens 0x%02x", dev->scsi_dev->lun,
+ params->block_size, medium_type, mode, speed, density);
+ } else {
+ PRINT_ERROR("MODE_SENSE failed: %x", rc);
+ res = -ENODEV;
+ goto out_free_buf;
+ }
+
+obtain:
+ res = scst_obtain_device_parameters(dev);
+ if (res != 0) {
+ PRINT_ERROR("Failed to obtain control parameters for device "
+ "%s", dev->virt_name);
+ goto out_free_buf;
+ }
+
+out_free_buf:
+ kfree(buffer);
+
+out_free_req:
+ if (res == 0)
+ dev->dh_priv = params;
+ else
+ kfree(params);
+
+out:
+ return res;
+}
+
+static void tape_detach(struct scst_device *dev)
+{
+ struct tape_params *params =
+ (struct tape_params *)dev->dh_priv;
+
+ kfree(params);
+ dev->dh_priv = NULL;
+ return;
+}
+
+static int tape_get_block_size(struct scst_cmd *cmd)
+{
+ struct tape_params *params = (struct tape_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be called,
+ * when there are existing commands.
+ */
+ return params->block_size;
+}
+
+static int tape_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ scst_tape_generic_parse(cmd, tape_get_block_size);
+
+ cmd->retries = SCST_PASSTHROUGH_RETRIES;
+
+ return res;
+}
+
+static void tape_set_block_size(struct scst_cmd *cmd, int block_size)
+{
+ struct tape_params *params = (struct tape_params *)cmd->dev->dh_priv;
+ /*
+ * No need for locks here, since *_detach() can not be called, when
+ * there are existing commands.
+ */
+ params->block_size = block_size;
+ return;
+}
+
+static int tape_done(struct scst_cmd *cmd)
+{
+ int opcode = cmd->cdb[0];
+ int status = cmd->status;
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ if ((status == SAM_STAT_GOOD) || (status == SAM_STAT_CONDITION_MET))
+ res = scst_tape_generic_dev_done(cmd, tape_set_block_size);
+ else if ((status == SAM_STAT_CHECK_CONDITION) &&
+ SCST_SENSE_VALID(cmd->sense)) {
+ struct tape_params *params;
+
+ TRACE_DBG("Extended sense %x", cmd->sense[0] & 0x7F);
+
+ if ((cmd->sense[0] & 0x7F) != 0x70) {
+ PRINT_ERROR("Sense format 0x%x is not supported",
+ cmd->sense[0] & 0x7F);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out;
+ }
+
+ if (opcode == READ_6 && !(cmd->cdb[1] & SILI_BIT) &&
+ (cmd->sense[2] & 0xe0)) {
+ /* EOF, EOM, or ILI */
+ int TransferLength, Residue = 0;
+ if ((cmd->sense[2] & 0x0f) == BLANK_CHECK)
+ /* No need for EOM in this case */
+ cmd->sense[2] &= 0xcf;
+ TransferLength = ((cmd->cdb[2] << 16) |
+ (cmd->cdb[3] << 8) | cmd->cdb[4]);
+ /* Compute the residual count */
+ if ((cmd->sense[0] & 0x80) != 0) {
+ Residue = ((cmd->sense[3] << 24) |
+ (cmd->sense[4] << 16) |
+ (cmd->sense[5] << 8) |
+ cmd->sense[6]);
+ }
+ TRACE_DBG("Checking the sense key "
+ "sn[2]=%x cmd->cdb[0,1]=%x,%x TransLen/Resid"
+ " %d/%d", (int)cmd->sense[2], cmd->cdb[0],
+ cmd->cdb[1], TransferLength, Residue);
+ if (TransferLength > Residue) {
+ int resp_data_len = TransferLength - Residue;
+ if (cmd->cdb[1] & SCST_TRANSFER_LEN_TYPE_FIXED) {
+ /*
+ * No need for locks here, since
+ * *_detach() can not be called, when
+ * there are existing commands.
+ */
+ params = (struct tape_params *)
+ cmd->dev->dh_priv;
+ resp_data_len *= params->block_size;
+ }
+ scst_set_resp_data_len(cmd, resp_data_len);
+ }
+ }
+ }
+
+out:
+ TRACE_DBG("cmd->is_send_status=%x, cmd->resp_data_len=%d, "
+ "res=%d", cmd->is_send_status, cmd->resp_data_len, res);
+ return res;
+}
+
+static int tape_perf_exec(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED, rc;
+ int opcode = cmd->cdb[0];
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ switch (opcode) {
+ case WRITE_6:
+ case READ_6:
+ cmd->completed = 1;
+ goto out_done;
+ }
+
+out:
+ return res;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out;
+}
+
+MODULE_AUTHOR("Vladislav Bolkhovitin & Leonid Stoljar");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI tape (type 1) dev handler for SCST");
+MODULE_VERSION(SCST_VERSION_STRING);
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 15/17]: Implementation of blk_rq_map_kern_sg()
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (13 preceding siblings ...)
2010-09-14 14:48 ` [PATCH 14/17]: SCST pass-through dev handlers Vladislav Bolkhovitin
@ 2010-09-14 14:49 ` Vladislav Bolkhovitin
2010-09-14 14:50 ` [PATCH 16/17]: scst_local target driver Vladislav Bolkhovitin
` (2 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:49 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe, Jens Axboe
This patch implements function blk_rq_map_kern_sg(), which allows to
map a kernel-originated SG vector to a block request. It is necessary
to execute SCSI commands with from the kernel going SG buffer. SCST needs
this functionality, because its target drivers, which are, basically,
SCSI drivers, can deal only with SGs, not with BIOs.
Highlight of this implementations:
- It uses BIOs chaining instead of kmalloc()'ing the whole bio.
- It uses SGs chaining instead of kmalloc()'ing one big SG in case if
direct mapping failed (e.g. because of DMA alignment or padding).
- When needed, copy_page() is used instead of memcpy() to copy the
whole pages faster.
Also this patch adds and exports function sg_copy(), which copies
one SG vector to another.
This patch has already been reviewed in LKML/linux-scsi in
http://lkml.org/lkml/2009/8/12/304. Since then it was only been updated for
2.6.35.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
block/blk-map.c | 332 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/blkdev.h | 5
include/linux/scatterlist.h | 5
lib/scatterlist.c | 129 +++++++++++++++++
4 files changed, 471 insertions(+)
diff -upkr linux-2.6.35/block/blk-map.c linux-2.6.35/block/blk-map.c
--- linux-2.6.35/block/blk-map.c 2010-05-17 01:17:36.000000000 +0400
+++ linux-2.6.35/block/blk-map.c 2010-05-24 15:19:49.000000000 +0400
@@ -5,6 +5,8 @@
#include <linux/module.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <scsi/sg.h> /* for struct sg_iovec */
#include "blk.h"
@@ -271,6 +273,336 @@ int blk_rq_unmap_user(struct bio *bio)
}
EXPORT_SYMBOL(blk_rq_unmap_user);
+struct blk_kern_sg_work {
+ atomic_t bios_inflight;
+ struct sg_table sg_table;
+ struct scatterlist *src_sgl;
+};
+
+static void blk_free_kern_sg_work(struct blk_kern_sg_work *bw)
+{
+ sg_free_table(&bw->sg_table);
+ kfree(bw);
+ return;
+}
+
+static void blk_bio_map_kern_endio(struct bio *bio, int err)
+{
+ struct blk_kern_sg_work *bw = bio->bi_private;
+
+ if (bw != NULL) {
+ /* Decrement the bios in processing and, if zero, free */
+ BUG_ON(atomic_read(&bw->bios_inflight) <= 0);
+ if (atomic_dec_and_test(&bw->bios_inflight)) {
+ if ((bio_data_dir(bio) == READ) && (err == 0)) {
+ unsigned long flags;
+
+ local_irq_save(flags); /* to protect KMs */
+ sg_copy(bw->src_sgl, bw->sg_table.sgl, 0, 0,
+ KM_BIO_DST_IRQ, KM_BIO_SRC_IRQ);
+ local_irq_restore(flags);
+ }
+ blk_free_kern_sg_work(bw);
+ }
+ }
+
+ bio_put(bio);
+ return;
+}
+
+static int blk_rq_copy_kern_sg(struct request *rq, struct scatterlist *sgl,
+ int nents, struct blk_kern_sg_work **pbw,
+ gfp_t gfp, gfp_t page_gfp)
+{
+ int res = 0, i;
+ struct scatterlist *sg;
+ struct scatterlist *new_sgl;
+ int new_sgl_nents;
+ size_t len = 0, to_copy;
+ struct blk_kern_sg_work *bw;
+
+ bw = kzalloc(sizeof(*bw), gfp);
+ if (bw == NULL)
+ goto out;
+
+ bw->src_sgl = sgl;
+
+ for_each_sg(sgl, sg, nents, i)
+ len += sg->length;
+ to_copy = len;
+
+ new_sgl_nents = PFN_UP(len);
+
+ res = sg_alloc_table(&bw->sg_table, new_sgl_nents, gfp);
+ if (res != 0)
+ goto out_free_bw;
+
+ new_sgl = bw->sg_table.sgl;
+
+ for_each_sg(new_sgl, sg, new_sgl_nents, i) {
+ struct page *pg;
+
+ pg = alloc_page(page_gfp);
+ if (pg == NULL)
+ goto err_free_new_sgl;
+
+ sg_assign_page(sg, pg);
+ sg->length = min_t(size_t, PAGE_SIZE, len);
+
+ len -= PAGE_SIZE;
+ }
+
+ if (rq_data_dir(rq) == WRITE) {
+ /*
+ * We need to limit amount of copied data to to_copy, because
+ * sgl might have the last element in sgl not marked as last in
+ * SG chaining.
+ */
+ sg_copy(new_sgl, sgl, 0, to_copy,
+ KM_USER0, KM_USER1);
+ }
+
+ *pbw = bw;
+ /*
+ * REQ_COPY_USER name is misleading. It should be something like
+ * REQ_HAS_TAIL_SPACE_FOR_PADDING.
+ */
+ rq->cmd_flags |= REQ_COPY_USER;
+
+out:
+ return res;
+
+err_free_new_sgl:
+ for_each_sg(new_sgl, sg, new_sgl_nents, i) {
+ struct page *pg = sg_page(sg);
+ if (pg == NULL)
+ break;
+ __free_page(pg);
+ }
+ sg_free_table(&bw->sg_table);
+
+out_free_bw:
+ kfree(bw);
+ res = -ENOMEM;
+ goto out;
+}
+
+static int __blk_rq_map_kern_sg(struct request *rq, struct scatterlist *sgl,
+ int nents, struct blk_kern_sg_work *bw, gfp_t gfp)
+{
+ int res;
+ struct request_queue *q = rq->q;
+ int rw = rq_data_dir(rq);
+ int max_nr_vecs, i;
+ size_t tot_len;
+ bool need_new_bio;
+ struct scatterlist *sg, *prev_sg = NULL;
+ struct bio *bio = NULL, *hbio = NULL, *tbio = NULL;
+ int bios;
+
+ if (unlikely((sgl == NULL) || (sgl->length == 0) || (nents <= 0))) {
+ WARN_ON(1);
+ res = -EINVAL;
+ goto out;
+ }
+
+ /*
+ * Let's keep each bio allocation inside a single page to decrease
+ * probability of failure.
+ */
+ max_nr_vecs = min_t(size_t,
+ ((PAGE_SIZE - sizeof(struct bio)) / sizeof(struct bio_vec)),
+ BIO_MAX_PAGES);
+
+ need_new_bio = true;
+ tot_len = 0;
+ bios = 0;
+ for_each_sg(sgl, sg, nents, i) {
+ struct page *page = sg_page(sg);
+ void *page_addr = page_address(page);
+ size_t len = sg->length, l;
+ size_t offset = sg->offset;
+
+ tot_len += len;
+ prev_sg = sg;
+
+ /*
+ * Each segment must be aligned on DMA boundary and
+ * not on stack. The last one may have unaligned
+ * length as long as the total length is aligned to
+ * DMA padding alignment.
+ */
+ if (i == nents - 1)
+ l = 0;
+ else
+ l = len;
+ if (((sg->offset | l) & queue_dma_alignment(q)) ||
+ (page_addr && object_is_on_stack(page_addr + sg->offset))) {
+ res = -EINVAL;
+ goto out_free_bios;
+ }
+
+ while (len > 0) {
+ size_t bytes;
+ int rc;
+
+ if (need_new_bio) {
+ bio = bio_kmalloc(gfp, max_nr_vecs);
+ if (bio == NULL) {
+ res = -ENOMEM;
+ goto out_free_bios;
+ }
+
+ if (rw == WRITE)
+ bio->bi_rw |= 1 << BIO_RW;
+
+ bios++;
+ bio->bi_private = bw;
+ bio->bi_end_io = blk_bio_map_kern_endio;
+
+ if (hbio == NULL)
+ hbio = tbio = bio;
+ else
+ tbio = tbio->bi_next = bio;
+ }
+
+ bytes = min_t(size_t, len, PAGE_SIZE - offset);
+
+ rc = bio_add_pc_page(q, bio, page, bytes, offset);
+ if (rc < bytes) {
+ if (unlikely(need_new_bio || (rc < 0))) {
+ if (rc < 0)
+ res = rc;
+ else
+ res = -EIO;
+ goto out_free_bios;
+ } else {
+ need_new_bio = true;
+ len -= rc;
+ offset += rc;
+ continue;
+ }
+ }
+
+ need_new_bio = false;
+ offset = 0;
+ len -= bytes;
+ page = nth_page(page, 1);
+ }
+ }
+
+ if (hbio == NULL) {
+ res = -EINVAL;
+ goto out_free_bios;
+ }
+
+ /* Total length must be aligned on DMA padding alignment */
+ if ((tot_len & q->dma_pad_mask) &&
+ !(rq->cmd_flags & REQ_COPY_USER)) {
+ res = -EINVAL;
+ goto out_free_bios;
+ }
+
+ if (bw != NULL)
+ atomic_set(&bw->bios_inflight, bios);
+
+ while (hbio != NULL) {
+ bio = hbio;
+ hbio = hbio->bi_next;
+ bio->bi_next = NULL;
+
+ blk_queue_bounce(q, &bio);
+
+ res = blk_rq_append_bio(q, rq, bio);
+ if (unlikely(res != 0)) {
+ bio->bi_next = hbio;
+ hbio = bio;
+ /* We can have one or more bios bounced */
+ goto out_unmap_bios;
+ }
+ }
+
+ rq->buffer = NULL;
+out:
+ return res;
+
+out_free_bios:
+ while (hbio != NULL) {
+ bio = hbio;
+ hbio = hbio->bi_next;
+ bio_put(bio);
+ }
+ goto out;
+
+out_unmap_bios:
+ blk_rq_unmap_kern_sg(rq, res);
+ goto out;
+}
+
+/**
+ * blk_rq_map_kern_sg - map kernel data to a request, for REQ_TYPE_BLOCK_PC
+ * @rq: request to fill
+ * @sgl: area to map
+ * @nents: number of elements in @sgl
+ * @gfp: memory allocation flags
+ *
+ * Description:
+ * Data will be mapped directly if possible. Otherwise a bounce
+ * buffer will be used.
+ */
+int blk_rq_map_kern_sg(struct request *rq, struct scatterlist *sgl,
+ int nents, gfp_t gfp)
+{
+ int res;
+
+ res = __blk_rq_map_kern_sg(rq, sgl, nents, NULL, gfp);
+ if (unlikely(res != 0)) {
+ struct blk_kern_sg_work *bw = NULL;
+
+ res = blk_rq_copy_kern_sg(rq, sgl, nents, &bw,
+ gfp, rq->q->bounce_gfp | gfp);
+ if (unlikely(res != 0))
+ goto out;
+
+ res = __blk_rq_map_kern_sg(rq, bw->sg_table.sgl,
+ bw->sg_table.nents, bw, gfp);
+ if (res != 0) {
+ blk_free_kern_sg_work(bw);
+ goto out;
+ }
+ }
+
+ rq->buffer = NULL;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(blk_rq_map_kern_sg);
+
+/**
+ * blk_rq_unmap_kern_sg - unmap a request with kernel sg
+ * @rq: request to unmap
+ * @err: non-zero error code
+ *
+ * Description:
+ * Unmap a rq previously mapped by blk_rq_map_kern_sg(). Must be called
+ * only in case of an error!
+ */
+void blk_rq_unmap_kern_sg(struct request *rq, int err)
+{
+ struct bio *bio = rq->bio;
+
+ while (bio) {
+ struct bio *b = bio;
+ bio = bio->bi_next;
+ b->bi_end_io(b, err);
+ }
+ rq->bio = NULL;
+
+ return;
+}
+EXPORT_SYMBOL(blk_rq_unmap_kern_sg);
+
/**
* blk_rq_map_kern - map kernel data to a request, for REQ_TYPE_BLOCK_PC usage
* @q: request queue where request should be inserted
diff -upkr linux-2.6.35/include/linux/blkdev.h linux-2.6.35/include/linux/blkdev.h
--- linux-2.6.35/include/linux/blkdev.h 2010-05-17 01:17:36.000000000 +0400
+++ linux-2.6.35/include/linux/blkdev.h 2010-05-24 14:51:22.000000000 +0400
@@ -717,6 +717,8 @@ extern unsigned long blk_max_low_pfn, bl
#define BLK_DEFAULT_SG_TIMEOUT (60 * HZ)
#define BLK_MIN_SG_TIMEOUT (7 * HZ)
+#define SCSI_EXEC_REQ_FIFO_DEFINED
+
#ifdef CONFIG_BOUNCE
extern int init_emergency_isa_pool(void);
extern void blk_queue_bounce(struct request_queue *q, struct bio **bio);
@@ -832,6 +834,9 @@ extern int blk_rq_map_kern(struct reques
extern int blk_rq_map_user_iov(struct request_queue *, struct request *,
struct rq_map_data *, struct sg_iovec *, int,
unsigned int, gfp_t);
+extern int blk_rq_map_kern_sg(struct request *rq, struct scatterlist *sgl,
+ int nents, gfp_t gfp);
+extern void blk_rq_unmap_kern_sg(struct request *rq, int err);
extern int blk_execute_rq(struct request_queue *, struct gendisk *,
struct request *, int);
extern void blk_execute_rq_nowait(struct request_queue *, struct gendisk *,
diff -upkr linux-2.6.35/include/linux/scatterlist.h linux-2.6.35/include/linux/scatterlist.h
--- linux-2.6.35/include/linux/scatterlist.h 2010-05-17 01:17:36.000000000 +0400
+++ linux-2.6.35/include/linux/scatterlist.h 2010-05-24 14:51:22.000000000 +0400
@@ -3,6 +3,7 @@
#include <asm/types.h>
#include <asm/scatterlist.h>
+#include <asm/kmap_types.h>
#include <linux/mm.h>
#include <linux/string.h>
#include <asm/io.h>
@@ -218,6 +219,10 @@ size_t sg_copy_from_buffer(struct scatte
size_t sg_copy_to_buffer(struct scatterlist *sgl, unsigned int nents,
void *buf, size_t buflen);
+int sg_copy(struct scatterlist *dst_sg, struct scatterlist *src_sg,
+ int nents_to_copy, size_t copy_len,
+ enum km_type d_km_type, enum km_type s_km_type);
+
/*
* Maximum number of entries that will be allocated in one piece, if
* a list larger than this is required then chaining will be utilized.
diff -upkr linux-2.6.35/lib/scatterlist.c linux-2.6.35/lib/scatterlist.c
--- linux-2.6.35/lib/scatterlist.c 2010-05-17 01:17:36.000000000 +0400
+++ linux-2.6.35/lib/scatterlist.c 2010-05-24 14:51:22.000000000 +0400
@@ -494,3 +494,132 @@ size_t sg_copy_to_buffer(struct scatterl
return sg_copy_buffer(sgl, nents, buf, buflen, 1);
}
EXPORT_SYMBOL(sg_copy_to_buffer);
+
+/*
+ * Can switch to the next dst_sg element, so, to copy to strictly only
+ * one dst_sg element, it must be either last in the chain, or
+ * copy_len == dst_sg->length.
+ */
+static int sg_copy_elem(struct scatterlist **pdst_sg, size_t *pdst_len,
+ size_t *pdst_offs, struct scatterlist *src_sg,
+ size_t copy_len,
+ enum km_type d_km_type, enum km_type s_km_type)
+{
+ int res = 0;
+ struct scatterlist *dst_sg;
+ size_t src_len, dst_len, src_offs, dst_offs;
+ struct page *src_page, *dst_page;
+
+ dst_sg = *pdst_sg;
+ dst_len = *pdst_len;
+ dst_offs = *pdst_offs;
+ dst_page = sg_page(dst_sg);
+
+ src_page = sg_page(src_sg);
+ src_len = src_sg->length;
+ src_offs = src_sg->offset;
+
+ do {
+ void *saddr, *daddr;
+ size_t n;
+
+ saddr = kmap_atomic(src_page +
+ (src_offs >> PAGE_SHIFT), s_km_type) +
+ (src_offs & ~PAGE_MASK);
+ daddr = kmap_atomic(dst_page +
+ (dst_offs >> PAGE_SHIFT), d_km_type) +
+ (dst_offs & ~PAGE_MASK);
+
+ if (((src_offs & ~PAGE_MASK) == 0) &&
+ ((dst_offs & ~PAGE_MASK) == 0) &&
+ (src_len >= PAGE_SIZE) && (dst_len >= PAGE_SIZE) &&
+ (copy_len >= PAGE_SIZE)) {
+ copy_page(daddr, saddr);
+ n = PAGE_SIZE;
+ } else {
+ n = min_t(size_t, PAGE_SIZE - (dst_offs & ~PAGE_MASK),
+ PAGE_SIZE - (src_offs & ~PAGE_MASK));
+ n = min(n, src_len);
+ n = min(n, dst_len);
+ n = min_t(size_t, n, copy_len);
+ memcpy(daddr, saddr, n);
+ }
+ dst_offs += n;
+ src_offs += n;
+
+ kunmap_atomic(saddr, s_km_type);
+ kunmap_atomic(daddr, d_km_type);
+
+ res += n;
+ copy_len -= n;
+ if (copy_len == 0)
+ goto out;
+
+ src_len -= n;
+ dst_len -= n;
+ if (dst_len == 0) {
+ dst_sg = sg_next(dst_sg);
+ if (dst_sg == NULL)
+ goto out;
+ dst_page = sg_page(dst_sg);
+ dst_len = dst_sg->length;
+ dst_offs = dst_sg->offset;
+ }
+ } while (src_len > 0);
+
+out:
+ *pdst_sg = dst_sg;
+ *pdst_len = dst_len;
+ *pdst_offs = dst_offs;
+ return res;
+}
+
+/**
+ * sg_copy - copy one SG vector to another
+ * @dst_sg: destination SG
+ * @src_sg: source SG
+ * @nents_to_copy: maximum number of entries to copy
+ * @copy_len: maximum amount of data to copy. If 0, then copy all.
+ * @d_km_type: kmap_atomic type for the destination SG
+ * @s_km_type: kmap_atomic type for the source SG
+ *
+ * Description:
+ * Data from the source SG vector will be copied to the destination SG
+ * vector. End of the vectors will be determined by sg_next() returning
+ * NULL. Returns number of bytes copied.
+ */
+int sg_copy(struct scatterlist *dst_sg, struct scatterlist *src_sg,
+ int nents_to_copy, size_t copy_len,
+ enum km_type d_km_type, enum km_type s_km_type)
+{
+ int res = 0;
+ size_t dst_len, dst_offs;
+
+ if (copy_len == 0)
+ copy_len = 0x7FFFFFFF; /* copy all */
+
+ if (nents_to_copy == 0)
+ nents_to_copy = 0x7FFFFFFF; /* copy all */
+
+ dst_len = dst_sg->length;
+ dst_offs = dst_sg->offset;
+
+ do {
+ int copied = sg_copy_elem(&dst_sg, &dst_len, &dst_offs,
+ src_sg, copy_len, d_km_type, s_km_type);
+ copy_len -= copied;
+ res += copied;
+ if ((copy_len == 0) || (dst_sg == NULL))
+ goto out;
+
+ nents_to_copy--;
+ if (nents_to_copy == 0)
+ goto out;
+
+ src_sg = sg_next(src_sg);
+ } while (src_sg != NULL);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(sg_copy);
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 16/17]: scst_local target driver
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (14 preceding siblings ...)
2010-09-14 14:49 ` [PATCH 15/17]: Implementation of blk_rq_map_kern_sg() Vladislav Bolkhovitin
@ 2010-09-14 14:50 ` Vladislav Bolkhovitin
2010-09-14 14:51 ` [PATCH 17/17]: SCST InfiniBand SRP " Vladislav Bolkhovitin
2010-09-15 5:56 ` [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Dmitry Torokhov
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:50 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patchset contains scst_local target driver with documentation.
Signed-off-by: Richard Sharpe <realrichardsharpe@gmail.com>
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Documentation/scst/README.scst_local | 195 ++++
drivers/scst/scst_local/Kconfig | 22
drivers/scst/scst_local/Makefile | 2
drivers/scst/scst_local/scst_local.c | 1453 +++++++++++++++++++++++++++++++++++
diff -uprN orig/linux-2.6.35/drivers/scst/scst_local/Kconfig linux-2.6.35/drivers/scst/scst_local/Kconfig
--- orig/linux-2.6.35/drivers/scst/scst_local/Kconfig
+++ linux-2.6.35/drivers/scst/scst_local/Kconfig
@@ -0,0 +1,22 @@
+config SCST_LOCAL
+ tristate "SCST Local driver"
+ depends on SCST && !HIGHMEM4G && !HIGHMEM64G
+ ---help---
+ This module provides a LLD SCSI driver that connects to
+ the SCST target mode subsystem in a loop-back manner.
+ It allows you to test target-mode device-handlers locally.
+ You will need the SCST subsystem as well.
+
+ If unsure whether you really want or need this, say N.
+
+config SCST_LOCAL_FORCE_DIRECT_PROCESSING
+ bool "Force local processing"
+ depends on SCST_LOCAL
+ help
+ This experimental option forces scst_local to make SCST process
+ SCSI commands in the same context, in which they was submitted.
+ Otherwise, they will be processed in SCST threads. Setting this
+ option to "Y" will give some performance increase, but might be
+ unsafe.
+
+ If unsure, say "N".
diff -uprN orig/linux-2.6.35/drivers/scst/scst_local/Makefile linux-2.6.35/drivers/scst/scst_local/Makefile
--- orig/linux-2.6.35/drivers/scst/scst_local/Makefile
+++ linux-2.6.35/drivers/scst/scst_local/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_SCST_LOCAL) += scst_local.o
+
diff -uprN orig/linux-2.6.35/drivers/scst/scst_local/scst_local.c linux-2.6.35/drivers/scst/scst_local/scst_local.c
--- orig/linux-2.6.35/drivers/scst/scst_local/scst_local.c
+++ linux-2.6.35/drivers/scst/scst_local/scst_local.c
@@ -0,0 +1,1453 @@
+/*
+ * Copyright (C) 2008 Richard Sharpe
+ * Copyright (C) 1992 Eric Youngdale
+ * Copyright (C) 2008 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ *
+ * Simulate a host adapter and an SCST target adapter back to back
+ *
+ * Based on the scsi_debug.c driver originally by Eric Youngdale and
+ * others, including D Gilbert et al
+ *
+ */
+
+#include <linux/module.h>
+
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+#include <linux/init.h>
+#include <linux/moduleparam.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
+#include <linux/completion.h>
+#include <linux/spinlock.h>
+
+#include <scsi/scsi.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_host.h>
+#include <scsi/scsi_tcq.h>
+
+#define LOG_PREFIX "scst_local"
+
+/* SCST includes ... */
+#include <scst/scst_const.h>
+#include <scst/scst.h>
+#include <scst/scst_debug.h>
+
+#ifdef CONFIG_SCST_DEBUG
+#define SCST_LOCAL_DEFAULT_LOG_FLAGS (TRACE_FUNCTION | TRACE_PID | \
+ TRACE_LINE | TRACE_OUT_OF_MEM | TRACE_MGMT | TRACE_MGMT_DEBUG | \
+ TRACE_MINOR | TRACE_SPECIAL)
+#else
+# ifdef CONFIG_SCST_TRACING
+#define SCST_LOCAL_DEFAULT_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_MGMT | \
+ TRACE_SPECIAL)
+# endif
+#endif
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+#define trace_flag scst_local_trace_flag
+static unsigned long scst_local_trace_flag = SCST_LOCAL_DEFAULT_LOG_FLAGS;
+#endif
+
+#define TRUE 1
+#define FALSE 0
+
+#define SCST_LOCAL_VERSION "1.0.0"
+static const char *scst_local_version_date = "20100910";
+
+/* Some statistics */
+static atomic_t num_aborts = ATOMIC_INIT(0);
+static atomic_t num_dev_resets = ATOMIC_INIT(0);
+static atomic_t num_target_resets = ATOMIC_INIT(0);
+
+static bool scst_local_add_default_tgt;
+module_param_named(add_default_tgt, scst_local_add_default_tgt, bool, S_IRUGO);
+MODULE_PARM_DESC(add_default_tgt, "add or not (default) on start default "
+ "target scst_local_tgt with default session scst_local_host");
+
+struct scst_aen_work_item {
+ struct list_head work_list_entry;
+ struct scst_aen *aen;
+};
+
+struct scst_local_tgt {
+ struct scst_tgt *scst_tgt;
+ struct list_head sessions_list; /* protected by scst_local_mutex */
+ struct list_head tgts_list_entry;
+
+ /* SCSI version descriptors */
+ uint16_t scsi_transport_version;
+ uint16_t phys_transport_version;
+};
+
+struct scst_local_sess {
+ struct scst_session *scst_sess;
+
+ unsigned int unregistering:1;
+
+ struct device dev;
+ struct Scsi_Host *shost;
+ struct scst_local_tgt *tgt;
+
+ int number;
+
+ struct mutex tr_id_mutex;
+ uint8_t *transport_id;
+ int transport_id_len;
+
+ struct work_struct aen_work;
+ spinlock_t aen_lock;
+ struct list_head aen_work_list; /* protected by aen_lock */
+
+ struct list_head sessions_list_entry;
+};
+
+#define to_scst_lcl_sess(d) \
+ container_of(d, struct scst_local_sess, dev)
+
+static int __scst_local_add_adapter(struct scst_local_tgt *tgt,
+ const char *initiator_name, struct scst_local_sess **out_sess,
+ bool locked);
+static int scst_local_add_adapter(struct scst_local_tgt *tgt,
+ const char *initiator_name, struct scst_local_sess **out_sess);
+static void scst_local_remove_adapter(struct scst_local_sess *sess);
+static int scst_local_add_target(const char *target_name,
+ struct scst_local_tgt **out_tgt);
+static void __scst_local_remove_target(struct scst_local_tgt *tgt);
+static void scst_local_remove_target(struct scst_local_tgt *tgt);
+
+static atomic_t scst_local_sess_num = ATOMIC_INIT(0);
+
+static LIST_HEAD(scst_local_tgts_list);
+static DEFINE_MUTEX(scst_local_mutex);
+
+static DECLARE_RWSEM(scst_local_exit_rwsem);
+
+MODULE_AUTHOR("Richard Sharpe, Vladislav Bolkhovitin + ideas from SCSI_DEBUG");
+MODULE_DESCRIPTION("SCSI+SCST local adapter driver");
+MODULE_LICENSE("GPL");
+MODULE_VERSION(SCST_LOCAL_VERSION);
+
+static int scst_local_get_sas_transport_id(struct scst_local_sess *sess,
+ uint8_t **transport_id, int *len)
+{
+ int res = 0;
+ int tr_id_size = 0;
+ uint8_t *tr_id = NULL;
+
+ tr_id_size = 24; /* A SAS TransportID */
+
+ tr_id = kzalloc(tr_id_size, GFP_KERNEL);
+ if (tr_id == NULL) {
+ PRINT_ERROR("Allocation of TransportID (size %d) failed",
+ tr_id_size);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ tr_id[0] = 0x00 | SCSI_TRANSPORTID_PROTOCOLID_SAS;
+
+ /*
+ * Assemble a valid SAS address = 0x5OOUUIIR12345678 ... Does SCST
+ * have one?
+ */
+
+ tr_id[4] = 0x5F;
+ tr_id[5] = 0xEE;
+ tr_id[6] = 0xDE;
+ tr_id[7] = 0x40 | ((sess->number >> 4) & 0x0F);
+ tr_id[8] = 0x0F | (sess->number & 0xF0);
+ tr_id[9] = 0xAD;
+ tr_id[10] = 0xE0;
+ tr_id[11] = 0x50;
+
+ *transport_id = tr_id;
+ *len = tr_id_size;
+
+ TRACE_DBG("Created tid '%02X:%02X:%02X:%02X:%02X:%02X:%02X:%02X'",
+ tr_id[4], tr_id[5], tr_id[6], tr_id[7],
+ tr_id[8], tr_id[9], tr_id[10], tr_id[11]);
+
+out:
+ return res;
+}
+
+static int scst_local_get_initiator_port_transport_id(
+ struct scst_session *scst_sess, uint8_t **transport_id)
+{
+ int res = 0;
+ int tr_id_size = 0;
+ uint8_t *tr_id = NULL;
+ struct scst_local_sess *sess;
+
+ if (scst_sess == NULL) {
+ res = SCSI_TRANSPORTID_PROTOCOLID_SAS;
+ goto out;
+ }
+
+ sess = (struct scst_local_sess *)scst_sess_get_tgt_priv(scst_sess);
+
+ mutex_lock(&sess->tr_id_mutex);
+
+ if (sess->transport_id == NULL) {
+ res = scst_local_get_sas_transport_id(sess,
+ transport_id, &tr_id_size);
+ goto out_unlock;
+ }
+
+ tr_id_size = sess->transport_id_len;
+ BUG_ON(tr_id_size == 0);
+
+ tr_id = kzalloc(tr_id_size, GFP_KERNEL);
+ if (tr_id == NULL) {
+ PRINT_ERROR("Allocation of TransportID (size %d) failed",
+ tr_id_size);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(tr_id, sess->transport_id, sess->transport_id_len);
+
+out_unlock:
+ mutex_unlock(&sess->tr_id_mutex);
+
+out:
+ return res;
+}
+
+/**
+ ** Tgtt attributes
+ **/
+
+static ssize_t scst_local_version_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ sprintf(buf, "%s/%s\n", SCST_LOCAL_VERSION, scst_local_version_date);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ strcat(buf, "EXTRACHECKS\n");
+#endif
+
+#ifdef CONFIG_SCST_TRACING
+ strcat(buf, "TRACING\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+ strcat(buf, "DEBUG\n");
+#endif
+ return strlen(buf);
+}
+
+static struct kobj_attribute scst_local_version_attr =
+ __ATTR(version, S_IRUGO, scst_local_version_show, NULL);
+
+static ssize_t scst_local_stats_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+
+{
+ return sprintf(buf, "Aborts: %d, Device Resets: %d, Target Resets: %d",
+ atomic_read(&num_aborts), atomic_read(&num_dev_resets),
+ atomic_read(&num_target_resets));
+}
+
+static struct kobj_attribute scst_local_stats_attr =
+ __ATTR(stats, S_IRUGO, scst_local_stats_show, NULL);
+
+static const struct attribute *scst_local_tgtt_attrs[] = {
+ &scst_local_version_attr.attr,
+ &scst_local_stats_attr.attr,
+ NULL,
+};
+
+/**
+ ** Tgt attributes
+ **/
+
+static ssize_t scst_local_scsi_transport_version_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt *scst_tgt;
+ struct scst_local_tgt *tgt;
+ ssize_t res;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ scst_tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ tgt = (struct scst_local_tgt *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ if (tgt->scsi_transport_version != 0)
+ res = sprintf(buf, "0x%x\n%s", tgt->scsi_transport_version,
+ SCST_SYSFS_KEY_MARK "\n");
+ else
+ res = sprintf(buf, "0x%x\n", 0x0BE0); /* SAS */
+
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static ssize_t scst_local_scsi_transport_version_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buffer, size_t size)
+{
+ ssize_t res;
+ struct scst_tgt *scst_tgt;
+ struct scst_local_tgt *tgt;
+ unsigned long val;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ scst_tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ tgt = (struct scst_local_tgt *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ res = strict_strtoul(buffer, 0, &val);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %zd", buffer, res);
+ goto out_up;
+ }
+
+ tgt->scsi_transport_version = val;
+
+ res = size;
+
+out_up:
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static struct kobj_attribute scst_local_scsi_transport_version_attr =
+ __ATTR(scsi_transport_version, S_IRUGO | S_IWUSR,
+ scst_local_scsi_transport_version_show,
+ scst_local_scsi_transport_version_store);
+
+static ssize_t scst_local_phys_transport_version_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt *scst_tgt;
+ struct scst_local_tgt *tgt;
+ ssize_t res;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ scst_tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ tgt = (struct scst_local_tgt *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ res = sprintf(buf, "0x%x\n%s", tgt->phys_transport_version,
+ (tgt->phys_transport_version != 0) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static ssize_t scst_local_phys_transport_version_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buffer, size_t size)
+{
+ ssize_t res;
+ struct scst_tgt *scst_tgt;
+ struct scst_local_tgt *tgt;
+ unsigned long val;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ scst_tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ tgt = (struct scst_local_tgt *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ res = strict_strtoul(buffer, 0, &val);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %zd", buffer, res);
+ goto out_up;
+ }
+
+ tgt->phys_transport_version = val;
+
+ res = size;
+
+out_up:
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static struct kobj_attribute scst_local_phys_transport_version_attr =
+ __ATTR(phys_transport_version, S_IRUGO | S_IWUSR,
+ scst_local_phys_transport_version_show,
+ scst_local_phys_transport_version_store);
+
+static const struct attribute *scst_local_tgt_attrs[] = {
+ &scst_local_scsi_transport_version_attr.attr,
+ &scst_local_phys_transport_version_attr.attr,
+ NULL,
+};
+
+/**
+ ** Session attributes
+ **/
+
+static ssize_t scst_local_transport_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ ssize_t res;
+ struct scst_session *scst_sess;
+ struct scst_local_sess *sess;
+ uint8_t *tr_id;
+ int tr_id_len, i;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj);
+ sess = (struct scst_local_sess *)scst_sess_get_tgt_priv(scst_sess);
+
+ mutex_lock(&sess->tr_id_mutex);
+
+ if (sess->transport_id != NULL) {
+ tr_id = sess->transport_id;
+ tr_id_len = sess->transport_id_len;
+ } else {
+ res = scst_local_get_sas_transport_id(sess, &tr_id, &tr_id_len);
+ if (res != 0)
+ goto out_unlock;
+ }
+
+ res = 0;
+ for (i = 0; i < tr_id_len; i++)
+ res += sprintf(&buf[res], "%c", tr_id[i]);
+
+ if (sess->transport_id == NULL)
+ kfree(tr_id);
+
+out_unlock:
+ mutex_unlock(&sess->tr_id_mutex);
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static ssize_t scst_local_transport_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buffer, size_t size)
+{
+ ssize_t res;
+ struct scst_session *scst_sess;
+ struct scst_local_sess *sess;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj);
+ sess = (struct scst_local_sess *)scst_sess_get_tgt_priv(scst_sess);
+
+ mutex_lock(&sess->tr_id_mutex);
+
+ if (sess->transport_id != NULL) {
+ kfree(sess->transport_id);
+ sess->transport_id = NULL;
+ sess->transport_id_len = 0;
+ }
+
+ if (size == 0)
+ goto out_res;
+
+ sess->transport_id = kzalloc(size, GFP_KERNEL);
+ if (sess->transport_id == NULL) {
+ PRINT_ERROR("Allocation of transport_id (size %zd) failed",
+ size);
+ res = -ENOMEM;
+ goto out_unlock;
+ }
+
+ sess->transport_id_len = size;
+
+ memcpy(sess->transport_id, buffer, sess->transport_id_len);
+
+out_res:
+ res = size;
+
+out_unlock:
+ mutex_unlock(&sess->tr_id_mutex);
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static struct kobj_attribute scst_local_transport_id_attr =
+ __ATTR(transport_id, S_IRUGO | S_IWUSR,
+ scst_local_transport_id_show,
+ scst_local_transport_id_store);
+
+static const struct attribute *scst_local_sess_attrs[] = {
+ &scst_local_transport_id_attr.attr,
+ NULL,
+};
+
+static ssize_t scst_local_sysfs_add_target(const char *target_name, char *params)
+{
+ int res;
+ struct scst_local_tgt *tgt;
+ char *param, *p;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ res = scst_local_add_target(target_name, &tgt);
+ if (res != 0)
+ goto out_up;
+
+ while (1) {
+ param = scst_get_next_token_str(¶ms);
+ if (param == NULL)
+ break;
+
+ p = scst_get_next_lexem(¶m);
+ if (*p == '\0')
+ break;
+
+ if (strcasecmp("session_name", p) != 0) {
+ PRINT_ERROR("Unknown parameter %s", p);
+ res = -EINVAL;
+ goto out_remove;
+ }
+
+ p = scst_get_next_lexem(¶m);
+ if (*p == '\0') {
+ PRINT_ERROR("Wrong session name %s", p);
+ res = -EINVAL;
+ goto out_remove;
+ }
+
+ res = scst_local_add_adapter(tgt, p, NULL);
+ if (res != 0)
+ goto out_remove;
+ }
+
+out_up:
+ up_read(&scst_local_exit_rwsem);
+ return res;
+
+out_remove:
+ scst_local_remove_target(tgt);
+ goto out_up;
+}
+
+static ssize_t scst_local_sysfs_del_target(const char *target_name)
+{
+ int res;
+ struct scst_local_tgt *tgt;
+ bool deleted = false;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ mutex_lock(&scst_local_mutex);
+ list_for_each_entry(tgt, &scst_local_tgts_list, tgts_list_entry) {
+ if (strcmp(target_name, tgt->scst_tgt->tgt_name) == 0) {
+ __scst_local_remove_target(tgt);
+ deleted = true;
+ break;
+ }
+ }
+ mutex_unlock(&scst_local_mutex);
+
+ if (!deleted) {
+ PRINT_ERROR("Target %s not found", target_name);
+ res = -ENOENT;
+ goto out_up;
+ }
+
+ res = 0;
+
+out_up:
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static ssize_t scst_local_sysfs_mgmt_cmd(char *buf)
+{
+ ssize_t res;
+ char *command, *target_name, *session_name;
+ struct scst_local_tgt *t, *tgt;
+
+ if (down_read_trylock(&scst_local_exit_rwsem) == 0)
+ return -ENOENT;
+
+ command = scst_get_next_lexem(&buf);
+
+ target_name = scst_get_next_lexem(&buf);
+ if (*target_name == '\0') {
+ PRINT_ERROR("%s", "Target name required");
+ res = -EINVAL;
+ goto out_up;
+ }
+
+ mutex_lock(&scst_local_mutex);
+
+ tgt = NULL;
+ list_for_each_entry(t, &scst_local_tgts_list, tgts_list_entry) {
+ if (strcmp(t->scst_tgt->tgt_name, target_name) == 0) {
+ tgt = t;
+ break;
+ }
+ }
+ if (tgt == NULL) {
+ PRINT_ERROR("Target %s not found", target_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ session_name = scst_get_next_lexem(&buf);
+ if (*session_name == '\0') {
+ PRINT_ERROR("%s", "Session name required");
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (strcasecmp("add_session", command) == 0) {
+ res = __scst_local_add_adapter(tgt, session_name, NULL, true);
+ } else if (strcasecmp("del_session", command) == 0) {
+ struct scst_local_sess *s, *sess = NULL;
+ list_for_each_entry(s, &tgt->sessions_list,
+ sessions_list_entry) {
+ if (strcmp(s->scst_sess->initiator_name, session_name) == 0) {
+ sess = s;
+ break;
+ }
+ }
+ if (sess == NULL) {
+ PRINT_ERROR("Session %s not found (target %s)",
+ session_name, target_name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ scst_local_remove_adapter(sess);
+ }
+
+ res = 0;
+
+out_unlock:
+ mutex_unlock(&scst_local_mutex);
+
+out_up:
+ up_read(&scst_local_exit_rwsem);
+ return res;
+}
+
+static int scst_local_abort(struct scsi_cmnd *SCpnt)
+{
+ struct scst_local_sess *sess;
+ int ret;
+ DECLARE_COMPLETION_ONSTACK(dev_reset_completion);
+
+ sess = to_scst_lcl_sess(scsi_get_device(SCpnt->device->host));
+
+ ret = scst_rx_mgmt_fn_tag(sess->scst_sess, SCST_ABORT_TASK, SCpnt->tag,
+ FALSE, &dev_reset_completion);
+
+ /* Now wait for the completion ... */
+ wait_for_completion_interruptible(&dev_reset_completion);
+
+ atomic_inc(&num_aborts);
+
+ if (ret == 0)
+ ret = SUCCESS;
+ return ret;
+}
+
+static int scst_local_device_reset(struct scsi_cmnd *SCpnt)
+{
+ struct scst_local_sess *sess;
+ uint16_t lun;
+ int ret;
+ DECLARE_COMPLETION_ONSTACK(dev_reset_completion);
+
+ sess = to_scst_lcl_sess(scsi_get_device(SCpnt->device->host));
+
+ lun = SCpnt->device->lun;
+ lun = cpu_to_be16(lun);
+
+ ret = scst_rx_mgmt_fn_lun(sess->scst_sess, SCST_LUN_RESET,
+ (const uint8_t *)&lun, sizeof(lun), FALSE,
+ &dev_reset_completion);
+
+ /* Now wait for the completion ... */
+ wait_for_completion_interruptible(&dev_reset_completion);
+
+ atomic_inc(&num_dev_resets);
+
+ if (ret == 0)
+ ret = SUCCESS;
+ return ret;
+}
+
+static int scst_local_target_reset(struct scsi_cmnd *SCpnt)
+{
+ struct scst_local_sess *sess;
+ uint16_t lun;
+ int ret;
+ DECLARE_COMPLETION_ONSTACK(dev_reset_completion);
+
+ sess = to_scst_lcl_sess(scsi_get_device(SCpnt->device->host));
+
+ lun = SCpnt->device->lun;
+ lun = cpu_to_be16(lun);
+
+ ret = scst_rx_mgmt_fn_lun(sess->scst_sess, SCST_TARGET_RESET,
+ (const uint8_t *)&lun, sizeof(lun), FALSE,
+ &dev_reset_completion);
+
+ /* Now wait for the completion ... */
+ wait_for_completion_interruptible(&dev_reset_completion);
+
+ atomic_inc(&num_target_resets);
+
+ if (ret == 0)
+ ret = SUCCESS;
+ return ret;
+}
+
+static void copy_sense(struct scsi_cmnd *cmnd, struct scst_cmd *scst_cmnd)
+{
+ int scst_cmnd_sense_len = scst_cmd_get_sense_buffer_len(scst_cmnd);
+
+ scst_cmnd_sense_len = (SCSI_SENSE_BUFFERSIZE < scst_cmnd_sense_len ?
+ SCSI_SENSE_BUFFERSIZE : scst_cmnd_sense_len);
+ memcpy(cmnd->sense_buffer, scst_cmd_get_sense_buffer(scst_cmnd),
+ scst_cmnd_sense_len);
+
+ TRACE_BUFFER("Sense set", cmnd->sense_buffer, scst_cmnd_sense_len);
+ return;
+}
+
+/*
+ * Utility function to handle processing of done and allow
+ * easy insertion of error injection if desired
+ */
+static int scst_local_send_resp(struct scsi_cmnd *cmnd,
+ struct scst_cmd *scst_cmnd,
+ void (*done)(struct scsi_cmnd *),
+ int scsi_result)
+{
+ int ret = 0;
+
+ if (scst_cmnd) {
+ /* The buffer isn't ours, so let's be safe and restore it */
+ scst_check_restore_sg_buff(scst_cmnd);
+
+ /* Simulate autosense by this driver */
+ if (unlikely(SCST_SENSE_VALID(scst_cmnd->sense)))
+ copy_sense(cmnd, scst_cmnd);
+ }
+
+ cmnd->result = scsi_result;
+
+ done(cmnd);
+ return ret;
+}
+
+/*
+ * This does the heavy lifting ... we pass all the commands on to the
+ * target driver and have it do its magic ...
+ */
+static int scst_local_queuecommand(struct scsi_cmnd *SCpnt,
+ void (*done)(struct scsi_cmnd *))
+ __acquires(&h->host_lock)
+ __releases(&h->host_lock)
+{
+ struct scst_local_sess *sess;
+ struct scatterlist *sgl = NULL;
+ int sgl_count = 0;
+ uint16_t lun;
+ struct scst_cmd *scst_cmd = NULL;
+ scst_data_direction dir;
+
+ TRACE_DBG("lun %d, cmd: 0x%02X", SCpnt->device->lun, SCpnt->cmnd[0]);
+
+ sess = to_scst_lcl_sess(scsi_get_device(SCpnt->device->host));
+
+ scsi_set_resid(SCpnt, 0);
+
+ /*
+ * We save a pointer to the done routine in SCpnt->scsi_done and
+ * we save that as tgt specific stuff below.
+ */
+ SCpnt->scsi_done = done;
+
+ /*
+ * Tell the target that we have a command ... but first we need
+ * to get the LUN into a format that SCST understand
+ */
+ lun = SCpnt->device->lun;
+ lun = cpu_to_be16(lun);
+ scst_cmd = scst_rx_cmd(sess->scst_sess, (const uint8_t *)&lun,
+ sizeof(lun), SCpnt->cmnd, SCpnt->cmd_len, TRUE);
+ if (!scst_cmd) {
+ PRINT_ERROR("%s", "scst_rx_cmd() failed");
+ return -ENOMEM;
+ }
+
+ scst_cmd_set_tag(scst_cmd, SCpnt->tag);
+ switch (scsi_get_tag_type(SCpnt->device)) {
+ case MSG_SIMPLE_TAG:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_SIMPLE);
+ break;
+ case MSG_HEAD_TAG:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_HEAD_OF_QUEUE);
+ break;
+ case MSG_ORDERED_TAG:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_ORDERED);
+ break;
+ case SCSI_NO_TAG:
+ default:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_UNTAGGED);
+ break;
+ }
+
+ sgl = scsi_sglist(SCpnt);
+ sgl_count = scsi_sg_count(SCpnt);
+
+ dir = SCST_DATA_NONE;
+ switch (SCpnt->sc_data_direction) {
+ case DMA_TO_DEVICE:
+ dir = SCST_DATA_WRITE;
+ scst_cmd_set_expected(scst_cmd, dir, scsi_bufflen(SCpnt));
+ scst_cmd_set_tgt_sg(scst_cmd, sgl, sgl_count);
+ break;
+ case DMA_FROM_DEVICE:
+ dir = SCST_DATA_READ;
+ scst_cmd_set_expected(scst_cmd, dir, scsi_bufflen(SCpnt));
+ scst_cmd_set_tgt_sg(scst_cmd, sgl, sgl_count);
+ break;
+ case DMA_BIDIRECTIONAL:
+ /* Some of these symbols are only defined after 2.6.24 */
+ dir = SCST_DATA_BIDI;
+ scst_cmd_set_expected(scst_cmd, dir, scsi_bufflen(SCpnt));
+ scst_cmd_set_expected_out_transfer_len(scst_cmd,
+ scsi_in(SCpnt)->length);
+ scst_cmd_set_tgt_sg(scst_cmd, scsi_in(SCpnt)->table.sgl,
+ scsi_in(SCpnt)->table.nents);
+ scst_cmd_set_tgt_out_sg(scst_cmd, sgl, sgl_count);
+ break;
+ case DMA_NONE:
+ default:
+ dir = SCST_DATA_NONE;
+ scst_cmd_set_expected(scst_cmd, dir, 0);
+ break;
+ }
+
+ /* Save the correct thing below depending on version */
+ scst_cmd_set_tgt_priv(scst_cmd, SCpnt);
+
+#ifdef CONFIG_SCST_LOCAL_FORCE_DIRECT_PROCESSING
+ {
+ struct Scsi_Host *h = SCpnt->device->host;
+ spin_unlock_irq(h->host_lock);
+ scst_cmd_init_done(scst_cmd, scst_estimate_context_direct());
+ spin_lock_irq(h->host_lock);
+ }
+#else
+ /*
+ * Unfortunately, we called with IRQs disabled, so have no choice,
+ * except to pass to the thread context.
+ */
+ scst_cmd_init_done(scst_cmd, SCST_CONTEXT_THREAD);
+#endif
+ return 0;
+}
+
+static int scst_local_targ_pre_exec(struct scst_cmd *scst_cmd)
+{
+ int res = SCST_PREPROCESS_STATUS_SUCCESS;
+
+ if (scst_cmd_get_dh_data_buff_alloced(scst_cmd) &&
+ (scst_cmd_get_data_direction(scst_cmd) & SCST_DATA_WRITE))
+ scst_copy_sg(scst_cmd, SCST_SG_COPY_FROM_TARGET);
+ return res;
+}
+
+/* Must be called under sess->aen_lock. Drops then reacquires it inside. */
+static void scst_process_aens(struct scst_local_sess *sess,
+ bool cleanup_only)
+{
+ struct scst_aen_work_item *work_item = NULL;
+
+ TRACE_DBG("Target work sess %p", sess);
+
+ while (!list_empty(&sess->aen_work_list)) {
+ work_item = list_entry(sess->aen_work_list.next,
+ struct scst_aen_work_item, work_list_entry);
+ list_del(&work_item->work_list_entry);
+
+ spin_unlock(&sess->aen_lock);
+
+ if (cleanup_only)
+ goto done;
+
+ BUG_ON(work_item->aen->event_fn != SCST_AEN_SCSI);
+
+ /* Let's always rescan */
+ scsi_scan_target(&sess->shost->shost_gendev, 0, 0,
+ SCAN_WILD_CARD, 1);
+
+done:
+ scst_aen_done(work_item->aen);
+ kfree(work_item);
+
+ spin_lock(&sess->aen_lock);
+ }
+ return;
+}
+
+static void scst_aen_work_fn(struct work_struct *work)
+{
+ struct scst_local_sess *sess =
+ container_of(work, struct scst_local_sess, aen_work);
+
+ TRACE_MGMT_DBG("Target work %p)", work);
+
+ spin_lock(&sess->aen_lock);
+ scst_process_aens(sess, false);
+ spin_unlock(&sess->aen_lock);
+ return;
+}
+
+static int scst_local_report_aen(struct scst_aen *aen)
+{
+ int res = 0;
+ int event_fn = scst_aen_get_event_fn(aen);
+ struct scst_local_sess *sess;
+ struct scst_aen_work_item *work_item = NULL;
+
+ sess = (struct scst_local_sess *)scst_sess_get_tgt_priv(
+ scst_aen_get_sess(aen));
+ switch (event_fn) {
+ case SCST_AEN_SCSI:
+ /*
+ * Allocate a work item and place it on the queue
+ */
+ work_item = kzalloc(sizeof(*work_item), GFP_KERNEL);
+ if (!work_item) {
+ PRINT_ERROR("%s", "Unable to allocate work item "
+ "to handle AEN!");
+ return -ENOMEM;
+ }
+
+ spin_lock(&sess->aen_lock);
+
+ if (unlikely(sess->unregistering)) {
+ spin_unlock(&sess->aen_lock);
+ kfree(work_item);
+ res = SCST_AEN_RES_NOT_SUPPORTED;
+ goto out;
+ }
+
+ list_add_tail(&work_item->work_list_entry, &sess->aen_work_list);
+ work_item->aen = aen;
+
+ spin_unlock(&sess->aen_lock);
+
+ schedule_work(&sess->aen_work);
+ break;
+
+ default:
+ TRACE_MGMT_DBG("Unsupported AEN %d", event_fn);
+ res = SCST_AEN_RES_NOT_SUPPORTED;
+ break;
+ }
+
+out:
+ return res;
+}
+
+static int scst_local_targ_detect(struct scst_tgt_template *tgt_template)
+{
+ return 0;
+};
+
+static int scst_local_targ_release(struct scst_tgt *tgt)
+{
+ return 0;
+}
+
+static int scst_local_targ_xmit_response(struct scst_cmd *scst_cmd)
+{
+ struct scsi_cmnd *SCpnt = NULL;
+ void (*done)(struct scsi_cmnd *);
+
+ if (unlikely(scst_cmd_aborted(scst_cmd))) {
+ scst_set_delivery_status(scst_cmd, SCST_CMD_DELIVERY_ABORTED);
+ scst_tgt_cmd_done(scst_cmd, SCST_CONTEXT_SAME);
+ return SCST_TGT_RES_SUCCESS;
+ }
+
+ if (scst_cmd_get_dh_data_buff_alloced(scst_cmd) &&
+ (scst_cmd_get_data_direction(scst_cmd) & SCST_DATA_READ))
+ scst_copy_sg(scst_cmd, SCST_SG_COPY_TO_TARGET);
+
+ SCpnt = scst_cmd_get_tgt_priv(scst_cmd);
+ done = SCpnt->scsi_done;
+
+ /*
+ * This might have to change to use the two status flags
+ */
+ if (scst_cmd_get_is_send_status(scst_cmd)) {
+ int resid = 0, out_resid = 0;
+
+ /* Calculate the residual ... */
+ if (likely(!scst_get_resid(scst_cmd, &resid, &out_resid))) {
+ TRACE_DBG("No residuals for request %p", SCpnt);
+ } else {
+ if (out_resid != 0)
+ PRINT_ERROR("Unable to return OUT residual %d "
+ "(op %02x)", out_resid, SCpnt->cmnd[0]);
+ }
+
+ scsi_set_resid(SCpnt, resid);
+
+ /*
+ * It seems like there is no way to set out_resid ...
+ */
+
+ (void)scst_local_send_resp(SCpnt, scst_cmd, done,
+ scst_cmd_get_status(scst_cmd));
+ }
+
+ /* Now tell SCST that the command is done ... */
+ scst_tgt_cmd_done(scst_cmd, SCST_CONTEXT_SAME);
+ return SCST_TGT_RES_SUCCESS;
+}
+
+static void scst_local_targ_task_mgmt_done(struct scst_mgmt_cmd *mgmt_cmd)
+{
+ struct completion *compl;
+
+ compl = (struct completion *)scst_mgmt_cmd_get_tgt_priv(mgmt_cmd);
+ if (compl)
+ complete(compl);
+ return;
+}
+
+static uint16_t scst_local_get_scsi_transport_version(struct scst_tgt *scst_tgt)
+{
+ struct scst_local_tgt *tgt;
+
+ tgt = (struct scst_local_tgt *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ if (tgt->scsi_transport_version == 0)
+ return 0x0BE0; /* SAS */
+ else
+ return tgt->scsi_transport_version;
+}
+
+static uint16_t scst_local_get_phys_transport_version(struct scst_tgt *scst_tgt)
+{
+ struct scst_local_tgt *tgt;
+
+ tgt = (struct scst_local_tgt *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ return tgt->phys_transport_version;
+}
+
+static struct scst_tgt_template scst_local_targ_tmpl = {
+ .name = "scst_local",
+ .sg_tablesize = 0xffff,
+ .xmit_response_atomic = 1,
+ .enabled_attr_not_needed = 1,
+ .tgtt_attrs = scst_local_tgtt_attrs,
+ .tgt_attrs = scst_local_tgt_attrs,
+ .sess_attrs = scst_local_sess_attrs,
+ .add_target = scst_local_sysfs_add_target,
+ .del_target = scst_local_sysfs_del_target,
+ .mgmt_cmd = scst_local_sysfs_mgmt_cmd,
+ .add_target_parameters = "session_name",
+ .mgmt_cmd_help = " echo \"add_session target_name session_name\" >mgmt\n"
+ " echo \"del_session target_name session_name\" >mgmt\n",
+ .detect = scst_local_targ_detect,
+ .release = scst_local_targ_release,
+ .pre_exec = scst_local_targ_pre_exec,
+ .xmit_response = scst_local_targ_xmit_response,
+ .task_mgmt_fn_done = scst_local_targ_task_mgmt_done,
+ .report_aen = scst_local_report_aen,
+ .get_initiator_port_transport_id = scst_local_get_initiator_port_transport_id,
+ .get_scsi_transport_version = scst_local_get_scsi_transport_version,
+ .get_phys_transport_version = scst_local_get_phys_transport_version,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = SCST_LOCAL_DEFAULT_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+};
+
+static struct scsi_host_template scst_lcl_ini_driver_template = {
+ .name = SCST_LOCAL_NAME,
+ .queuecommand = scst_local_queuecommand,
+ .eh_abort_handler = scst_local_abort,
+ .eh_device_reset_handler = scst_local_device_reset,
+ .eh_target_reset_handler = scst_local_target_reset,
+ .can_queue = 256,
+ .this_id = -1,
+ .sg_tablesize = 0xFFFF,
+ .cmd_per_lun = 32,
+ .max_sectors = 0xffff,
+ /* Possible pass-through backend device may not support clustering */
+ .use_clustering = DISABLE_CLUSTERING,
+ .skip_settle_delay = 1,
+ .module = THIS_MODULE,
+};
+
+/*
+ * LLD Bus and functions
+ */
+
+static int scst_local_driver_probe(struct device *dev)
+{
+ int ret;
+ struct scst_local_sess *sess;
+ struct Scsi_Host *hpnt;
+
+ sess = to_scst_lcl_sess(dev);
+
+ TRACE_DBG("sess %p", sess);
+
+ hpnt = scsi_host_alloc(&scst_lcl_ini_driver_template, sizeof(*sess));
+ if (NULL == hpnt) {
+ PRINT_ERROR("%s", "scsi_register() failed");
+ ret = -ENODEV;
+ goto out;
+ }
+
+ sess->shost = hpnt;
+
+ hpnt->max_id = 0; /* Don't want more than one id */
+ hpnt->max_lun = 0xFFFF;
+
+ /*
+ * Because of a change in the size of this field at 2.6.26
+ * we use this check ... it allows us to work on earlier
+ * kernels. If we don't, max_cmd_size gets set to 4 (and we get
+ * a compiler warning) so a scan never occurs.
+ */
+ hpnt->max_cmd_len = 260;
+
+ ret = scsi_add_host(hpnt, &sess->dev);
+ if (ret) {
+ PRINT_ERROR("%s", "scsi_add_host() failed");
+ ret = -ENODEV;
+ scsi_host_put(hpnt);
+ goto out;
+ }
+
+out:
+ return ret;
+}
+
+static int scst_local_driver_remove(struct device *dev)
+{
+ struct scst_local_sess *sess;
+
+ sess = to_scst_lcl_sess(dev);
+ if (!sess) {
+ PRINT_ERROR("%s", "Unable to locate sess info");
+ return -ENODEV;
+ }
+
+ scsi_remove_host(sess->shost);
+ scsi_host_put(sess->shost);
+ return 0;
+}
+
+static int scst_local_bus_match(struct device *dev,
+ struct device_driver *dev_driver)
+{
+ return 1;
+}
+
+static struct bus_type scst_local_lld_bus = {
+ .name = "scst_local_bus",
+ .match = scst_local_bus_match,
+ .probe = scst_local_driver_probe,
+ .remove = scst_local_driver_remove,
+};
+
+static struct device_driver scst_local_driver = {
+ .name = SCST_LOCAL_NAME,
+ .bus = &scst_local_lld_bus,
+};
+
+static struct device *scst_local_root;
+
+static void scst_local_release_adapter(struct device *dev)
+{
+ struct scst_local_sess *sess;
+
+ sess = to_scst_lcl_sess(dev);
+ if (sess == NULL)
+ goto out;
+
+ spin_lock(&sess->aen_lock);
+ sess->unregistering = 1;
+ scst_process_aens(sess, true);
+ spin_unlock(&sess->aen_lock);
+
+ cancel_work_sync(&sess->aen_work);
+
+ scst_unregister_session(sess->scst_sess, TRUE, NULL);
+
+ kfree(sess);
+
+out:
+ return;
+}
+
+static int __scst_local_add_adapter(struct scst_local_tgt *tgt,
+ const char *initiator_name, struct scst_local_sess **out_sess,
+ bool locked)
+{
+ int res;
+ struct scst_local_sess *sess;
+
+ sess = kzalloc(sizeof(*sess), GFP_KERNEL);
+ if (NULL == sess) {
+ PRINT_ERROR("Unable to alloc scst_lcl_host (size %zd)",
+ sizeof(*sess));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ sess->tgt = tgt;
+ sess->number = atomic_inc_return(&scst_local_sess_num);
+ mutex_init(&sess->tr_id_mutex);
+
+ /*
+ * Init this stuff we need for scheduling AEN work
+ */
+ INIT_WORK(&sess->aen_work, scst_aen_work_fn);
+ spin_lock_init(&sess->aen_lock);
+ INIT_LIST_HEAD(&sess->aen_work_list);
+
+ sess->scst_sess = scst_register_session(tgt->scst_tgt, 0,
+ initiator_name, (void *)sess, NULL, NULL);
+ if (sess->scst_sess == NULL) {
+ PRINT_ERROR("%s", "scst_register_session failed");
+ kfree(sess);
+ res = -EFAULT;
+ goto out_free;
+ }
+
+ sess->dev.bus = &scst_local_lld_bus;
+ sess->dev.parent = scst_local_root;
+ sess->dev.release = &scst_local_release_adapter;
+ sess->dev.init_name = kobject_name(&sess->scst_sess->sess_kobj);
+
+ res = device_register(&sess->dev);
+ if (res != 0)
+ goto unregister_session;
+
+ res = sysfs_create_link(scst_sysfs_get_sess_kobj(sess->scst_sess),
+ &sess->shost->shost_dev.kobj, "host");
+ if (res != 0) {
+ PRINT_ERROR("Unable to create \"host\" link for target "
+ "%s", scst_get_tgt_name(tgt->scst_tgt));
+ goto unregister_dev;
+ }
+
+ if (!locked)
+ mutex_lock(&scst_local_mutex);
+ list_add_tail(&sess->sessions_list_entry, &tgt->sessions_list);
+ if (!locked)
+ mutex_unlock(&scst_local_mutex);
+
+ if (scst_initiator_has_luns(tgt->scst_tgt, initiator_name))
+ scsi_scan_target(&sess->shost->shost_gendev, 0, 0,
+ SCAN_WILD_CARD, 1);
+
+out:
+ return res;
+
+unregister_dev:
+ device_unregister(&sess->dev);
+
+unregister_session:
+ scst_unregister_session(sess->scst_sess, TRUE, NULL);
+
+out_free:
+ kfree(sess);
+ goto out;
+}
+
+static int scst_local_add_adapter(struct scst_local_tgt *tgt,
+ const char *initiator_name, struct scst_local_sess **out_sess)
+{
+ return __scst_local_add_adapter(tgt, initiator_name, out_sess, false);
+}
+
+/* Must be called under scst_local_mutex */
+static void scst_local_remove_adapter(struct scst_local_sess *sess)
+{
+
+ list_del(&sess->sessions_list_entry);
+
+ device_unregister(&sess->dev);
+ return;
+}
+
+static int scst_local_add_target(const char *target_name,
+ struct scst_local_tgt **out_tgt)
+{
+ int res;
+ struct scst_local_tgt *tgt;
+
+ tgt = kzalloc(sizeof(*tgt), GFP_KERNEL);
+ if (NULL == tgt) {
+ PRINT_ERROR("Unable to alloc tgt (size %zd)", sizeof(*tgt));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ INIT_LIST_HEAD(&tgt->sessions_list);
+
+ tgt->scst_tgt = scst_register_target(&scst_local_targ_tmpl, target_name);
+ if (tgt->scst_tgt == NULL) {
+ PRINT_ERROR("%s", "scst_register_target() failed:");
+ res = -EFAULT;
+ goto out_free;
+ }
+
+ scst_tgt_set_tgt_priv(tgt->scst_tgt, tgt);
+
+ mutex_lock(&scst_local_mutex);
+ list_add_tail(&tgt->tgts_list_entry, &scst_local_tgts_list);
+ mutex_unlock(&scst_local_mutex);
+
+ if (out_tgt != NULL)
+ *out_tgt = tgt;
+
+ res = 0;
+
+out:
+ return res;
+
+out_free:
+ kfree(tgt);
+ goto out;
+}
+
+/* Must be called under scst_local_mutex */
+static void __scst_local_remove_target(struct scst_local_tgt *tgt)
+{
+ struct scst_local_sess *sess, *ts;
+
+ list_for_each_entry_safe(sess, ts, &tgt->sessions_list,
+ sessions_list_entry) {
+ scst_local_remove_adapter(sess);
+ }
+
+ list_del(&tgt->tgts_list_entry);
+
+ scst_unregister_target(tgt->scst_tgt);
+
+ kfree(tgt);
+ return;
+}
+
+static void scst_local_remove_target(struct scst_local_tgt *tgt)
+{
+
+ mutex_lock(&scst_local_mutex);
+ __scst_local_remove_target(tgt);
+ mutex_unlock(&scst_local_mutex);
+ return;
+}
+
+static int __init scst_local_init(void)
+{
+ int ret;
+ struct scst_local_tgt *tgt;
+
+ scst_local_root = root_device_register(SCST_LOCAL_NAME);
+ if (IS_ERR(scst_local_root)) {
+ ret = PTR_ERR(scst_local_root);
+ goto out;
+ }
+
+ ret = bus_register(&scst_local_lld_bus);
+ if (ret < 0) {
+ PRINT_ERROR("bus_register() error: %d", ret);
+ goto dev_unreg;
+ }
+
+ ret = driver_register(&scst_local_driver);
+ if (ret < 0) {
+ PRINT_ERROR("driver_register() error: %d", ret);
+ goto bus_unreg;
+ }
+
+ ret = scst_register_target_template(&scst_local_targ_tmpl);
+ if (ret != 0) {
+ PRINT_ERROR("Unable to register target template: %d", ret);
+ goto driver_unreg;
+ }
+
+ /*
+ * If we are using sysfs, then don't add a default target unless
+ * we are told to do so. When using procfs, we always add a default
+ * target because that was what the earliest versions did. Just
+ * remove the preprocessor directives when no longer needed.
+ */
+ if (!scst_local_add_default_tgt)
+ goto out;
+
+ ret = scst_local_add_target("scst_local_tgt", &tgt);
+ if (ret != 0)
+ goto tgt_templ_unreg;
+
+ ret = scst_local_add_adapter(tgt, "scst_local_host", NULL);
+ if (ret != 0)
+ goto tgt_unreg;
+
+out:
+ return ret;
+
+tgt_unreg:
+ scst_local_remove_target(tgt);
+
+tgt_templ_unreg:
+ scst_unregister_target_template(&scst_local_targ_tmpl);
+
+driver_unreg:
+ driver_unregister(&scst_local_driver);
+
+bus_unreg:
+ bus_unregister(&scst_local_lld_bus);
+
+dev_unreg:
+ root_device_unregister(scst_local_root);
+
+ goto out;
+}
+
+static void __exit scst_local_exit(void)
+{
+ struct scst_local_tgt *tgt, *tt;
+
+ down_write(&scst_local_exit_rwsem);
+
+ mutex_lock(&scst_local_mutex);
+ list_for_each_entry_safe(tgt, tt, &scst_local_tgts_list,
+ tgts_list_entry) {
+ __scst_local_remove_target(tgt);
+ }
+ mutex_unlock(&scst_local_mutex);
+
+ driver_unregister(&scst_local_driver);
+ bus_unregister(&scst_local_lld_bus);
+ root_device_unregister(scst_local_root);
+
+ /* Now unregister the target template */
+ scst_unregister_target_template(&scst_local_targ_tmpl);
+
+ /* To make lockdep happy */
+ up_write(&scst_local_exit_rwsem);
+ return;
+}
+
+device_initcall(scst_local_init);
+module_exit(scst_local_exit);
+
Signed-off-by:
diff -uprN orig/linux-2.6.35/Documentation/scst/README.scst_local linux-2.6.35/Documentation/scst/README.scst_local
--- orig/linux-2.6.35/Documentation/scst/README.scst_local
+++ linux-2.6.35/Documentation/scst/README.scst_local
@@ -0,0 +1,195 @@
+SCST Local ...
+Richard Sharpe, 30-Nov-2008
+
+This is the SCST Local driver. Its function is to allow you to access devices
+that are exported via SCST directly on the same Linux system that they are
+exported from.
+
+No assumptions are made in the code about the device types on the target, so
+any device handlers that you load in SCST should be visible, including tapes
+and so forth.
+
+You can freely use any sg, sd, st, etc. devices imported from target,
+except the following: you can't mount file systems or put swap on them.
+This is a limitation of Linux memory/cache manager. See SCST README file
+for details.
+
+To build, simply issue 'make' in the scst_local directory.
+
+Try 'modinfo scst_local' for a listing of module parameters so far.
+
+Here is how I have used it so far:
+
+1. Load up scst:
+
+ modprobe scst
+ modprobe scst_vdisk
+
+2. Create a virtual disk (or your own device handler):
+
+ dd if=/dev/zero of=/some/path/vdisk1.img bs=16384 count=1000000
+ echo "add_device vm_disk1 filename=/some/path/vdisk1.img" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
+
+3. Load the scst_local driver:
+
+ insmod scst_local add_default_tgt=1
+ echo "add vm_disk1 0" >/sys/kernel/scst_tgt/targets/scst_local/scst_local_tgt/luns/mgmt
+
+4. Check what you have
+
+ cat /proc/scsi/scsi
+ Attached devices:
+ Host: scsi0 Channel: 00 Id: 00 Lun: 00
+ Vendor: ATA Model: ST9320320AS Rev: 0303
+ Type: Direct-Access ANSI SCSI revision: 05
+ Host: scsi4 Channel: 00 Id: 00 Lun: 00
+ Vendor: TSSTcorp Model: CD/DVDW TS-L632D Rev: TO04
+ Type: CD-ROM ANSI SCSI revision: 05
+ Host: scsi7 Channel: 00 Id: 00 Lun: 00
+ Vendor: SCST_FIO Model: vm_disk1 Rev: 200
+ Type: Direct-Access ANSI SCSI revision: 04
+
+Or for (3) you can:
+
+ insmod scst_local
+ echo "add_target scst_local_tgt session_name=scst_local_host" >/sys/kernel/scst_tgt/targets/scst_local//mgmt
+ echo "add vm_disk1 0" >/sys/kernel/scst_tgt/targets/scst_local/scst_local_tgt/luns/mgmt
+
+Or instead of manually "add_device" in (2) and (3) write and scstadmin config
+(doesn't create the session at the moment due to bug in scstadmin):
+
+HANDLER vdisk_fileio {
+ DEVICE vm_disk1 {
+ filename /some/path/vdisk1.img
+ }
+}
+
+TARGET_DRIVER scst_local {
+ TARGET scst_local_tgt {
+ session_name=scst_local_host
+
+ LUN 0 vm_disk1
+ }
+}
+
+then:
+
+ insmod scst_local
+ scstadmin -config conf_file.cfg
+
+There can be any number of targets and sessions created. Each SCST
+session corresponds to SCSI host. You can change which LUNs assigned to
+each session by using SCST access control.
+
+5. Have fun.
+
+Some of this was coded while in Santa Clara, some in Bangalore, and some in
+Hyderabad. Noe doubt some will be coded on the way back to Santa Clara.
+
+The code still has bugs, so if you encounter any, email me the fixes at:
+
+ realrichardsharpe@gmail.com
+
+I am thinking of renaming this to something more interesting.
+
+Sysfs interface
+===============
+
+See SCST's README for a common SCST sysfs description.
+
+Root of this driver is /sys/kernel/scst_tgt/targets/scst_local. It has
+the following additional entry:
+
+ - stats - read-only attribute with some statistical information.
+
+Each target subdirectory contains the following additional entries:
+
+ - phys_transport_version - contains and allows to change physical
+ transport version descriptor. It determines by which phisical
+ interface this target will look like. See SPC for more details. By
+ default, it is not defined (0).
+
+ - scsi_transport_version - contains and allows to change SCSI
+ transport version descriptor. It determines by which SCSI
+ transport this target will look like. See SPC for more details. By
+ default, it is SAS.
+
+Each session subdirectory contains the following additional entries:
+
+ - transport_id - contains this host's TransportID. This TransportID
+ used to identify initiator in Persisten Reservation commands. If you
+ change scsi_transport_version for a target, make sure you set for all
+ its sessions correct TransportID. See SPC for more details.
+
+ - host - links to the corresponding SCSI host. Using it you can find
+ local sg/bsg/sd/etc. devices of this session. For instance, this
+ links points out to host12, so you can find your sg devices by:
+
+$ lsscsi -g|grep "\[12:"
+[12:0:0:0] disk SCST_FIO rd1 200 /dev/sdc /dev/sg2
+[12:0:0:1] disk SCST_FIO nullio 200 /dev/sdd /dev/sg3
+
+They are /dev/sg2 and /dev/sg3.
+
+The following management commands available via /sys/kernel/scst_tgt/targets/scst_local/mgmt:
+
+ - add_target target_name [session_name=sess_name; [session_name=sess_name1;] [...]] -
+ creates a target with optionally one or more sessions.
+
+ - del_target target_name - deletes a target.
+
+ - add_session target_name session_name - adds to target target_name
+ session (SCSI host) with name session_name.
+
+ - del_session target_name session_name - deletes session session_name
+ from target target_name.
+
+Note on performance
+===================
+
+Although this driver implemented in the most performance effective way,
+including zero-copy passing data between SCSI/block subsystems and SCST,
+in many cases it is NOT suited to measure performance as a NULL link.
+For example, it is not suited for max IOPS measurements. This is because
+for such cases not performance of the link between the target and
+initiator is the bottleneck, but CPU or memory speed on the target or
+initiator. For scst_local you have both initiator and target on the same
+system, which means each your initiator and target are much less
+CPU/memory powerful.
+
+Change log
+==========
+
+V0.1 24-Sep-2008 (Hyderabad) Initial coding, pretty chatty and messy,
+ but worked.
+
+V0.2 25-Sep-2008 (Hong Kong) Cleaned up the code a lot, reduced the log
+ chatter, fixed a bug where multiple LUNs did not
+ work. Also, added logging control. Tested with
+ five virtual disks. They all came up as /dev/sdb
+ through /dev/sdf and I could dd to them. Also
+ fixed a bug preventing multiple adapters.
+
+V0.3 26-Sep-2008 (Santa Clara) Added back a copyright plus cleaned up some
+ unused functions and structures.
+
+V0.4 5-Oct-2008 (Santa Clara) Changed name to scst_local as suggested, cleaned
+ up some unused variables (made them used) and
+ change allocation to a kmem_cache pool.
+
+V0.5 5-Oct-2008 (Santa Clara) Added mgmt commands to handle dev reset and
+ aborts. Not sure if aborts works. Also corrected
+ the version info and renamed readme to README.
+
+V0.6 7-Oct-2008 (Santa Clara) Removed some redundant code and made some
+ changes suggested by Vladislav.
+
+V0.7 11-Oct-2008 (Santa Clara) Moved into the scst tree. Cleaned up some
+ unused functions, used TRACE macros etc.
+
+V0.9 30-Nov-2008 (Mtn View) Cleaned up an additional problem with symbols not
+ being defined in older version of the kernel. Also
+ fixed some English and cleaned up this doc.
+
+V1.0 10-Sep-2010 (Moscow) Sysfs management added. Reviewed and cleaned up.
+
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 17/17]: SCST InfiniBand SRP target driver
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (15 preceding siblings ...)
2010-09-14 14:50 ` [PATCH 16/17]: scst_local target driver Vladislav Bolkhovitin
@ 2010-09-14 14:51 ` Vladislav Bolkhovitin
2010-09-15 5:56 ` [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Dmitry Torokhov
17 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-14 14:51 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Vu Pham,
Bart Van Assche, James Smart, Joe Eykholt, Andy Yan, Chetan Loke,
Dmitry Torokhov, Hannes Reinecke, Richard Sharpe
This patch contains SCST InfiniBand SRP target driver.
This driver works directly on top of InfiniBand stack and SCST.
It was originally developed by Vu Pham/Mellanox, currently
Bart Van Assche is maintaining and improving it.
Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Bart Van Assche <bart.vanassche@gmail.com>
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Documentation/scst/README.srpt | 109 +
drivers/scst/srpt/Kconfig | 12
drivers/scst/srpt/Makefile | 1
drivers/scst/srpt/ib_dm_mad.h | 139 +
drivers/scst/srpt/ib_srpt.c | 3800 +++++++++++++++++++++++++++++++++++++++++
drivers/scst/srpt/ib_srpt.h | 368 +++
6 files changed, 4429 insertions(+)
diff -uprN orig/linux-2.6.35/drivers/scst/srpt/Kconfig linux-2.6.35/drivers/scst/srpt/Kconfig
--- orig/linux-2.6.35/drivers/scst/srpt/Kconfig
+++ linux-2.6.35/drivers/scst/srpt/Kconfig
@@ -0,0 +1,12 @@
+config SCST_SRPT
+ tristate "InfiniBand SCSI RDMA Protocol target support"
+ depends on INFINIBAND && SCST
+ ---help---
+
+ Support for the SCSI RDMA Protocol (SRP) Target driver. The
+ SRP protocol is a protocol that allows an initiator to access
+ a block storage device on another host (target) over a network
+ that supports the RDMA protocol. Currently the RDMA protocol is
+ supported by InfiniBand and by iWarp network hardware. More
+ information about the SRP protocol can be found on the website
+ of the INCITS T10 technical committee (http://www.t10.org/).
diff -uprN orig/linux-2.6.35/drivers/scst/srpt/Makefile linux-2.6.35/drivers/scst/srpt/Makefile
--- orig/linux-2.6.35/drivers/scst/srpt/Makefile
+++ linux-2.6.35/drivers/scst/srpt/Makefile
@@ -0,0 +1,1 @@
+obj-$(CONFIG_SCST_SRPT) += ib_srpt.o
diff -uprN orig/linux-2.6.35/drivers/scst/srpt/ib_dm_mad.h linux-2.6.35/drivers/scst/srpt/ib_dm_mad.h
--- orig/linux-2.6.35/drivers/scst/srpt/ib_dm_mad.h
+++ linux-2.6.35/drivers/scst/srpt/ib_dm_mad.h
@@ -0,0 +1,139 @@
+/*
+ * Copyright (c) 2006 - 2009 Mellanox Technology Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#ifndef IB_DM_MAD_H
+#define IB_DM_MAD_H
+
+#include <linux/types.h>
+
+#include <rdma/ib_mad.h>
+
+enum {
+ /*
+ * See also section 13.4.7 Status Field, table 115 MAD Common Status
+ * Field Bit Values and also section 16.3.1.1 Status Field in the
+ * InfiniBand Architecture Specification.
+ */
+ DM_MAD_STATUS_UNSUP_METHOD = 0x0008,
+ DM_MAD_STATUS_UNSUP_METHOD_ATTR = 0x000c,
+ DM_MAD_STATUS_INVALID_FIELD = 0x001c,
+ DM_MAD_STATUS_NO_IOC = 0x0100,
+
+ /*
+ * See also the Device Management chapter, section 16.3.3 Attributes,
+ * table 279 Device Management Attributes in the InfiniBand
+ * Architecture Specification.
+ */
+ DM_ATTR_CLASS_PORT_INFO = 0x01,
+ DM_ATTR_IOU_INFO = 0x10,
+ DM_ATTR_IOC_PROFILE = 0x11,
+ DM_ATTR_SVC_ENTRIES = 0x12
+};
+
+struct ib_dm_hdr {
+ u8 reserved[28];
+};
+
+/*
+ * Structure of management datagram sent by the SRP target implementation.
+ * Contains a management datagram header, reliable multi-packet transaction
+ * protocol (RMPP) header and ib_dm_hdr. Notes:
+ * - The SRP target implementation does not use RMPP or ib_dm_hdr when sending
+ * management datagrams.
+ * - The header size must be exactly 64 bytes (IB_MGMT_DEVICE_HDR), since this
+ * is the header size that is passed to ib_create_send_mad() in ib_srpt.c.
+ * - The maximum supported size for a management datagram when not using RMPP
+ * is 256 bytes -- 64 bytes header and 192 (IB_MGMT_DEVICE_DATA) bytes data.
+ */
+struct ib_dm_mad {
+ struct ib_mad_hdr mad_hdr;
+ struct ib_rmpp_hdr rmpp_hdr;
+ struct ib_dm_hdr dm_hdr;
+ u8 data[IB_MGMT_DEVICE_DATA];
+};
+
+/*
+ * IOUnitInfo as defined in section 16.3.3.3 IOUnitInfo of the InfiniBand
+ * Architecture Specification.
+ */
+struct ib_dm_iou_info {
+ __be16 change_id;
+ u8 max_controllers;
+ u8 op_rom;
+ u8 controller_list[128];
+};
+
+/*
+ * IOControllerprofile as defined in section 16.3.3.4 IOControllerProfile of
+ * the InfiniBand Architecture Specification.
+ */
+struct ib_dm_ioc_profile {
+ __be64 guid;
+ __be32 vendor_id;
+ __be32 device_id;
+ __be16 device_version;
+ __be16 reserved1;
+ __be32 subsys_vendor_id;
+ __be32 subsys_device_id;
+ __be16 io_class;
+ __be16 io_subclass;
+ __be16 protocol;
+ __be16 protocol_version;
+ __be16 service_conn;
+ __be16 initiators_supported;
+ __be16 send_queue_depth;
+ u8 reserved2;
+ u8 rdma_read_depth;
+ __be32 send_size;
+ __be32 rdma_size;
+ u8 op_cap_mask;
+ u8 svc_cap_mask;
+ u8 num_svc_entries;
+ u8 reserved3[9];
+ u8 id_string[64];
+};
+
+struct ib_dm_svc_entry {
+ u8 name[40];
+ __be64 id;
+};
+
+/*
+ * See also section 16.3.3.5 ServiceEntries in the InfiniBand Architecture
+ * Specification. See also section B.7, table B.8 in the T10 SRP r16a document.
+ */
+struct ib_dm_svc_entries {
+ struct ib_dm_svc_entry service_entries[4];
+};
+
+#endif
diff -uprN orig/linux-2.6.35/drivers/scst/srpt/ib_srpt.c linux-2.6.35/drivers/scst/srpt/ib_srpt.c
--- orig/linux-2.6.35/drivers/scst/srpt/ib_srpt.c
+++ linux-2.6.35/drivers/scst/srpt/ib_srpt.c
@@ -0,0 +1,3800 @@
+/*
+ * Copyright (c) 2006 - 2009 Mellanox Technology Inc. All rights reserved.
+ * Copyright (C) 2008 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2008 - 2010 Bart Van Assche <bart.vanassche@gmail.com>
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+#include <linux/ctype.h>
+#include <linux/kthread.h>
+#include <linux/string.h>
+#include <linux/delay.h>
+#include <asm/atomic.h>
+#include "ib_srpt.h"
+#define LOG_PREFIX "ib_srpt" /* Prefix for SCST tracing macros. */
+#include <scst/scst_debug.h>
+
+/* Name of this kernel module. */
+#define DRV_NAME "ib_srpt"
+#define DRV_VERSION "1.0.1"
+#define DRV_RELDATE "July 10, 2008"
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+/* Flags to be used in SCST debug tracing statements. */
+#define DEFAULT_SRPT_TRACE_FLAGS (TRACE_OUT_OF_MEM | TRACE_MINOR \
+ | TRACE_MGMT | TRACE_SPECIAL)
+/* Name of the entry that will be created under /proc/scsi_tgt/ib_srpt. */
+#define SRPT_PROC_TRACE_LEVEL_NAME "trace_level"
+#endif
+
+#define MELLANOX_SRPT_ID_STRING "SCST SRP target"
+
+MODULE_AUTHOR("Vu Pham");
+MODULE_DESCRIPTION("InfiniBand SCSI RDMA Protocol target "
+ "v" DRV_VERSION " (" DRV_RELDATE ")");
+MODULE_LICENSE("Dual BSD/GPL");
+
+/*
+ * Local data types.
+ */
+
+enum threading_mode {
+ MODE_ALL_IN_SIRQ = 0,
+ MODE_IB_COMPLETION_IN_THREAD = 1,
+ MODE_IB_COMPLETION_IN_SIRQ = 2,
+};
+
+/*
+ * Global Variables
+ */
+
+static u64 srpt_service_guid;
+/* List of srpt_device structures. */
+static atomic_t srpt_device_count;
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+static unsigned long trace_flag = DEFAULT_SRPT_TRACE_FLAGS;
+module_param(trace_flag, long, 0644);
+MODULE_PARM_DESC(trace_flag, "SCST trace flags.");
+#endif
+#if defined(CONFIG_SCST_DEBUG)
+static unsigned long processing_delay_in_us;
+module_param(processing_delay_in_us, long, 0744);
+MODULE_PARM_DESC(processing_delay_in_us,
+ "SRP_CMD processing delay in microseconds. Useful for"
+ " testing the initiator lockup avoidance algorithm.");
+#endif
+
+static int thread = 1;
+module_param(thread, int, 0444);
+MODULE_PARM_DESC(thread,
+ "IB completion and SCSI command processing context. Defaults"
+ " to one, i.e. process IB completions and SCSI commands in"
+ " kernel thread context. 0 means soft IRQ whenever possible"
+ " and 2 means process IB completions in soft IRQ context and"
+ " SCSI commands in kernel thread context.");
+
+static unsigned srp_max_rdma_size = DEFAULT_MAX_RDMA_SIZE;
+module_param(srp_max_rdma_size, int, 0744);
+MODULE_PARM_DESC(srp_max_rdma_size,
+ "Maximum size of SRP RDMA transfers for new connections.");
+
+static unsigned srp_max_message_size = DEFAULT_MAX_MESSAGE_SIZE;
+module_param(srp_max_message_size, int, 0444);
+MODULE_PARM_DESC(srp_max_message_size,
+ "Maximum size of SRP control messages in bytes.");
+
+static int srpt_srq_size = DEFAULT_SRPT_SRQ_SIZE;
+module_param(srpt_srq_size, int, 0444);
+MODULE_PARM_DESC(srpt_srq_size,
+ "Shared receive queue (SRQ) size.");
+
+static int srpt_sq_size = DEF_SRPT_SQ_SIZE;
+module_param(srpt_sq_size, int, 0444);
+MODULE_PARM_DESC(srpt_sq_size,
+ "Per-channel send queue (SQ) size.");
+
+static bool srpt_autodetect_cred_req;
+module_param(srpt_autodetect_cred_req, bool, 0444);
+MODULE_PARM_DESC(srpt_autodetect_cred_req,
+ "Whether or not to autodetect whether the initiator supports"
+ " SRP_CRED_REQ.");
+
+static bool use_port_guid_in_session_name;
+module_param(use_port_guid_in_session_name, bool, 0444);
+MODULE_PARM_DESC(use_port_guid_in_session_name,
+ "Use target port ID in the SCST session name such that"
+ " redundant paths between multiport systems can be masked.");
+
+static int srpt_get_u64_x(char *buffer, struct kernel_param *kp)
+{
+ return sprintf(buffer, "0x%016llx", *(u64 *)kp->arg);
+}
+module_param_call(srpt_service_guid, NULL, srpt_get_u64_x, &srpt_service_guid,
+ 0444);
+MODULE_PARM_DESC(srpt_service_guid,
+ "Using this value for ioc_guid, id_ext, and cm_listen_id"
+ " instead of using the node_guid of the first HCA.");
+
+static void srpt_add_one(struct ib_device *device);
+static void srpt_remove_one(struct ib_device *device);
+static void srpt_unregister_mad_agent(struct srpt_device *sdev);
+static void srpt_unmap_sg_to_ib_sge(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx);
+static void srpt_release_channel(struct scst_session *scst_sess);
+
+static struct ib_client srpt_client = {
+ .name = DRV_NAME,
+ .add = srpt_add_one,
+ .remove = srpt_remove_one
+};
+
+/**
+ * srpt_test_and_set_channel_state() - Test and set the channel state.
+ *
+ * @ch: RDMA channel.
+ * @old: channel state to compare with.
+ * @new: state to change the channel state to if the current state matches the
+ * argument 'old'.
+ *
+ * Returns the previous channel state.
+ */
+static enum rdma_ch_state
+srpt_test_and_set_channel_state(struct srpt_rdma_ch *ch,
+ enum rdma_ch_state old,
+ enum rdma_ch_state new)
+{
+ return atomic_cmpxchg(&ch->state, old, new);
+}
+
+/**
+ * srpt_event_handler() - Asynchronous IB event callback function.
+ *
+ * Callback function called by the InfiniBand core when an asynchronous IB
+ * event occurs. This callback may occur in interrupt context. See also
+ * section 11.5.2, Set Asynchronous Event Handler in the InfiniBand
+ * Architecture Specification.
+ */
+static void srpt_event_handler(struct ib_event_handler *handler,
+ struct ib_event *event)
+{
+ struct srpt_device *sdev;
+ struct srpt_port *sport;
+
+ sdev = ib_get_client_data(event->device, &srpt_client);
+ if (!sdev || sdev->device != event->device)
+ return;
+
+ TRACE_DBG("ASYNC event= %d on device= %s",
+ event->event, sdev->device->name);
+
+ switch (event->event) {
+ case IB_EVENT_PORT_ERR:
+ if (event->element.port_num <= sdev->device->phys_port_cnt) {
+ sport = &sdev->port[event->element.port_num - 1];
+ sport->lid = 0;
+ sport->sm_lid = 0;
+ }
+ break;
+ case IB_EVENT_PORT_ACTIVE:
+ case IB_EVENT_LID_CHANGE:
+ case IB_EVENT_PKEY_CHANGE:
+ case IB_EVENT_SM_CHANGE:
+ case IB_EVENT_CLIENT_REREGISTER:
+ /*
+ * Refresh port data asynchronously. Note: it is safe to call
+ * schedule_work() even if &sport->work is already on the
+ * global workqueue because schedule_work() tests for the
+ * work_pending() condition before adding &sport->work to the
+ * global work queue.
+ */
+ if (event->element.port_num <= sdev->device->phys_port_cnt) {
+ sport = &sdev->port[event->element.port_num - 1];
+ if (!sport->lid && !sport->sm_lid)
+ schedule_work(&sport->work);
+ }
+ break;
+ default:
+ PRINT_ERROR("received unrecognized IB event %d", event->event);
+ break;
+ }
+}
+
+/**
+ * srpt_srq_event() - SRQ event callback function.
+ */
+static void srpt_srq_event(struct ib_event *event, void *ctx)
+{
+ PRINT_INFO("SRQ event %d", event->event);
+}
+
+/**
+ * srpt_qp_event() - QP event callback function.
+ */
+static void srpt_qp_event(struct ib_event *event, struct srpt_rdma_ch *ch)
+{
+ TRACE_DBG("QP event %d on cm_id=%p sess_name=%s state=%d",
+ event->event, ch->cm_id, ch->sess_name,
+ atomic_read(&ch->state));
+
+ switch (event->event) {
+ case IB_EVENT_COMM_EST:
+ ib_cm_notify(ch->cm_id, event->event);
+ break;
+ case IB_EVENT_QP_LAST_WQE_REACHED:
+ if (srpt_test_and_set_channel_state(ch, RDMA_CHANNEL_LIVE,
+ RDMA_CHANNEL_DISCONNECTING) == RDMA_CHANNEL_LIVE) {
+ PRINT_INFO("disconnected session %s.", ch->sess_name);
+ ib_send_cm_dreq(ch->cm_id, NULL, 0);
+ }
+ break;
+ default:
+ PRINT_ERROR("received unrecognized IB QP event %d",
+ event->event);
+ break;
+ }
+}
+
+/**
+ * srpt_set_ioc() - Helper function for initializing an IOUnitInfo structure.
+ *
+ * @slot: one-based slot number.
+ * @value: four-bit value.
+ *
+ * Copies the lowest four bits of value in element slot of the array of four
+ * bit elements called c_list (controller list). The index slot is one-based.
+ */
+static void srpt_set_ioc(u8 *c_list, u32 slot, u8 value)
+{
+ u16 id;
+ u8 tmp;
+
+ id = (slot - 1) / 2;
+ if (slot & 0x1) {
+ tmp = c_list[id] & 0xf;
+ c_list[id] = (value << 4) | tmp;
+ } else {
+ tmp = c_list[id] & 0xf0;
+ c_list[id] = (value & 0xf) | tmp;
+ }
+}
+
+/**
+ * srpt_get_class_port_info() - Copy ClassPortInfo to a management datagram.
+ *
+ * See also section 16.3.3.1 ClassPortInfo in the InfiniBand Architecture
+ * Specification.
+ */
+static void srpt_get_class_port_info(struct ib_dm_mad *mad)
+{
+ struct ib_class_port_info *cif;
+
+ cif = (struct ib_class_port_info *)mad->data;
+ memset(cif, 0, sizeof *cif);
+ cif->base_version = 1;
+ cif->class_version = 1;
+ cif->resp_time_value = 20;
+
+ mad->mad_hdr.status = 0;
+}
+
+/**
+ * srpt_get_iou() - Write IOUnitInfo to a management datagram.
+ *
+ * See also section 16.3.3.3 IOUnitInfo in the InfiniBand Architecture
+ * Specification. See also section B.7, table B.6 in the SRP r16a document.
+ */
+static void srpt_get_iou(struct ib_dm_mad *mad)
+{
+ struct ib_dm_iou_info *ioui;
+ u8 slot;
+ int i;
+
+ ioui = (struct ib_dm_iou_info *)mad->data;
+ ioui->change_id = cpu_to_be16(1);
+ ioui->max_controllers = 16;
+
+ /* set present for slot 1 and empty for the rest */
+ srpt_set_ioc(ioui->controller_list, 1, 1);
+ for (i = 1, slot = 2; i < 16; i++, slot++)
+ srpt_set_ioc(ioui->controller_list, slot, 0);
+
+ mad->mad_hdr.status = 0;
+}
+
+/**
+ * srpt_get_ioc() - Write IOControllerprofile to a management datagram.
+ *
+ * See also section 16.3.3.4 IOControllerProfile in the InfiniBand
+ * Architecture Specification. See also section B.7, table B.7 in the SRP
+ * r16a document.
+ */
+static void srpt_get_ioc(struct srpt_device *sdev, u32 slot,
+ struct ib_dm_mad *mad)
+{
+ struct ib_dm_ioc_profile *iocp;
+
+ iocp = (struct ib_dm_ioc_profile *)mad->data;
+
+ if (!slot || slot > 16) {
+ mad->mad_hdr.status = cpu_to_be16(DM_MAD_STATUS_INVALID_FIELD);
+ return;
+ }
+
+ if (slot > 2) {
+ mad->mad_hdr.status = cpu_to_be16(DM_MAD_STATUS_NO_IOC);
+ return;
+ }
+
+ memset(iocp, 0, sizeof *iocp);
+ strcpy(iocp->id_string, MELLANOX_SRPT_ID_STRING);
+ iocp->guid = cpu_to_be64(srpt_service_guid);
+ iocp->vendor_id = cpu_to_be32(sdev->dev_attr.vendor_id);
+ iocp->device_id = cpu_to_be32(sdev->dev_attr.vendor_part_id);
+ iocp->device_version = cpu_to_be16(sdev->dev_attr.hw_ver);
+ iocp->subsys_vendor_id = cpu_to_be32(sdev->dev_attr.vendor_id);
+ iocp->subsys_device_id = 0x0;
+ iocp->io_class = cpu_to_be16(SRP_REV16A_IB_IO_CLASS);
+ iocp->io_subclass = cpu_to_be16(SRP_IO_SUBCLASS);
+ iocp->protocol = cpu_to_be16(SRP_PROTOCOL);
+ iocp->protocol_version = cpu_to_be16(SRP_PROTOCOL_VERSION);
+ iocp->send_queue_depth = cpu_to_be16(sdev->srq_size);
+ iocp->rdma_read_depth = 4;
+ iocp->send_size = cpu_to_be32(srp_max_message_size);
+ iocp->rdma_size = cpu_to_be32(min(max(srp_max_rdma_size, 256U),
+ 1U << 24));
+ iocp->num_svc_entries = 1;
+ iocp->op_cap_mask = SRP_SEND_TO_IOC | SRP_SEND_FROM_IOC |
+ SRP_RDMA_READ_FROM_IOC | SRP_RDMA_WRITE_FROM_IOC;
+
+ mad->mad_hdr.status = 0;
+}
+
+/**
+ * srpt_get_svc_entries() - Write ServiceEntries to a management datagram.
+ *
+ * See also section 16.3.3.5 ServiceEntries in the InfiniBand Architecture
+ * Specification. See also section B.7, table B.8 in the SRP r16a document.
+ */
+static void srpt_get_svc_entries(u64 ioc_guid,
+ u16 slot, u8 hi, u8 lo, struct ib_dm_mad *mad)
+{
+ struct ib_dm_svc_entries *svc_entries;
+
+ WARN_ON(!ioc_guid);
+
+ if (!slot || slot > 16) {
+ mad->mad_hdr.status = cpu_to_be16(DM_MAD_STATUS_INVALID_FIELD);
+ return;
+ }
+
+ if (slot > 2 || lo > hi || hi > 1) {
+ mad->mad_hdr.status = cpu_to_be16(DM_MAD_STATUS_NO_IOC);
+ return;
+ }
+
+ svc_entries = (struct ib_dm_svc_entries *)mad->data;
+ memset(svc_entries, 0, sizeof *svc_entries);
+ svc_entries->service_entries[0].id = cpu_to_be64(ioc_guid);
+ snprintf(svc_entries->service_entries[0].name,
+ sizeof(svc_entries->service_entries[0].name),
+ "%s%016llx",
+ SRP_SERVICE_NAME_PREFIX,
+ ioc_guid);
+
+ mad->mad_hdr.status = 0;
+}
+
+/**
+ * srpt_mgmt_method_get() - Process a received management datagram.
+ * @sp: source port through which the MAD has been received.
+ * @rq_mad: received MAD.
+ * @rsp_mad: response MAD.
+ */
+static void srpt_mgmt_method_get(struct srpt_port *sp, struct ib_mad *rq_mad,
+ struct ib_dm_mad *rsp_mad)
+{
+ u16 attr_id;
+ u32 slot;
+ u8 hi, lo;
+
+ attr_id = be16_to_cpu(rq_mad->mad_hdr.attr_id);
+ switch (attr_id) {
+ case DM_ATTR_CLASS_PORT_INFO:
+ srpt_get_class_port_info(rsp_mad);
+ break;
+ case DM_ATTR_IOU_INFO:
+ srpt_get_iou(rsp_mad);
+ break;
+ case DM_ATTR_IOC_PROFILE:
+ slot = be32_to_cpu(rq_mad->mad_hdr.attr_mod);
+ srpt_get_ioc(sp->sdev, slot, rsp_mad);
+ break;
+ case DM_ATTR_SVC_ENTRIES:
+ slot = be32_to_cpu(rq_mad->mad_hdr.attr_mod);
+ hi = (u8) ((slot >> 8) & 0xff);
+ lo = (u8) (slot & 0xff);
+ slot = (u16) ((slot >> 16) & 0xffff);
+ srpt_get_svc_entries(srpt_service_guid,
+ slot, hi, lo, rsp_mad);
+ break;
+ default:
+ rsp_mad->mad_hdr.status =
+ cpu_to_be16(DM_MAD_STATUS_UNSUP_METHOD_ATTR);
+ break;
+ }
+}
+
+/**
+ * srpt_mad_send_handler() - Post MAD-send callback function.
+ */
+static void srpt_mad_send_handler(struct ib_mad_agent *mad_agent,
+ struct ib_mad_send_wc *mad_wc)
+{
+ ib_destroy_ah(mad_wc->send_buf->ah);
+ ib_free_send_mad(mad_wc->send_buf);
+}
+
+/**
+ * srpt_mad_recv_handler() - MAD reception callback function.
+ */
+static void srpt_mad_recv_handler(struct ib_mad_agent *mad_agent,
+ struct ib_mad_recv_wc *mad_wc)
+{
+ struct srpt_port *sport = (struct srpt_port *)mad_agent->context;
+ struct ib_ah *ah;
+ struct ib_mad_send_buf *rsp;
+ struct ib_dm_mad *dm_mad;
+
+ if (!mad_wc || !mad_wc->recv_buf.mad)
+ return;
+
+ ah = ib_create_ah_from_wc(mad_agent->qp->pd, mad_wc->wc,
+ mad_wc->recv_buf.grh, mad_agent->port_num);
+ if (IS_ERR(ah))
+ goto err;
+
+ BUILD_BUG_ON(offsetof(struct ib_dm_mad, data) != IB_MGMT_DEVICE_HDR);
+
+ rsp = ib_create_send_mad(mad_agent, mad_wc->wc->src_qp,
+ mad_wc->wc->pkey_index, 0,
+ IB_MGMT_DEVICE_HDR, IB_MGMT_DEVICE_DATA,
+ GFP_KERNEL);
+ if (IS_ERR(rsp))
+ goto err_rsp;
+
+ rsp->ah = ah;
+
+ dm_mad = rsp->mad;
+ memcpy(dm_mad, mad_wc->recv_buf.mad, sizeof *dm_mad);
+ dm_mad->mad_hdr.method = IB_MGMT_METHOD_GET_RESP;
+ dm_mad->mad_hdr.status = 0;
+
+ switch (mad_wc->recv_buf.mad->mad_hdr.method) {
+ case IB_MGMT_METHOD_GET:
+ srpt_mgmt_method_get(sport, mad_wc->recv_buf.mad, dm_mad);
+ break;
+ case IB_MGMT_METHOD_SET:
+ dm_mad->mad_hdr.status =
+ cpu_to_be16(DM_MAD_STATUS_UNSUP_METHOD_ATTR);
+ break;
+ default:
+ dm_mad->mad_hdr.status =
+ cpu_to_be16(DM_MAD_STATUS_UNSUP_METHOD);
+ break;
+ }
+
+ if (!ib_post_send_mad(rsp, NULL)) {
+ ib_free_recv_mad(mad_wc);
+ /* will destroy_ah & free_send_mad in send completion */
+ return;
+ }
+
+ ib_free_send_mad(rsp);
+
+err_rsp:
+ ib_destroy_ah(ah);
+err:
+ ib_free_recv_mad(mad_wc);
+}
+
+/**
+ * srpt_refresh_port() - Configure a HCA port.
+ *
+ * Enable InfiniBand management datagram processing, update the cached sm_lid,
+ * lid and gid values, and register a callback function for processing MADs
+ * on the specified port.
+ *
+ * Note: It is safe to call this function more than once for the same port.
+ */
+static int srpt_refresh_port(struct srpt_port *sport)
+{
+ struct ib_mad_reg_req reg_req;
+ struct ib_port_modify port_modify;
+ struct ib_port_attr port_attr;
+ int ret;
+
+ memset(&port_modify, 0, sizeof port_modify);
+ port_modify.set_port_cap_mask = IB_PORT_DEVICE_MGMT_SUP;
+ port_modify.clr_port_cap_mask = 0;
+
+ ret = ib_modify_port(sport->sdev->device, sport->port, 0, &port_modify);
+ if (ret)
+ goto err_mod_port;
+
+ ret = ib_query_port(sport->sdev->device, sport->port, &port_attr);
+ if (ret)
+ goto err_query_port;
+
+ sport->sm_lid = port_attr.sm_lid;
+ sport->lid = port_attr.lid;
+
+ ret = ib_query_gid(sport->sdev->device, sport->port, 0, &sport->gid);
+ if (ret)
+ goto err_query_port;
+
+ if (!sport->mad_agent) {
+ memset(®_req, 0, sizeof reg_req);
+ reg_req.mgmt_class = IB_MGMT_CLASS_DEVICE_MGMT;
+ reg_req.mgmt_class_version = IB_MGMT_BASE_VERSION;
+ set_bit(IB_MGMT_METHOD_GET, reg_req.method_mask);
+ set_bit(IB_MGMT_METHOD_SET, reg_req.method_mask);
+
+ sport->mad_agent = ib_register_mad_agent(sport->sdev->device,
+ sport->port,
+ IB_QPT_GSI,
+ ®_req, 0,
+ srpt_mad_send_handler,
+ srpt_mad_recv_handler,
+ sport);
+ if (IS_ERR(sport->mad_agent)) {
+ ret = PTR_ERR(sport->mad_agent);
+ sport->mad_agent = NULL;
+ goto err_query_port;
+ }
+ }
+
+ return 0;
+
+err_query_port:
+
+ port_modify.set_port_cap_mask = 0;
+ port_modify.clr_port_cap_mask = IB_PORT_DEVICE_MGMT_SUP;
+ ib_modify_port(sport->sdev->device, sport->port, 0, &port_modify);
+
+err_mod_port:
+
+ return ret;
+}
+
+/**
+ * srpt_unregister_mad_agent() - Unregister MAD callback functions.
+ *
+ * Note: It is safe to call this function more than once for the same device.
+ */
+static void srpt_unregister_mad_agent(struct srpt_device *sdev)
+{
+ struct ib_port_modify port_modify = {
+ .clr_port_cap_mask = IB_PORT_DEVICE_MGMT_SUP,
+ };
+ struct srpt_port *sport;
+ int i;
+
+ for (i = 1; i <= sdev->device->phys_port_cnt; i++) {
+ sport = &sdev->port[i - 1];
+ WARN_ON(sport->port != i);
+ if (ib_modify_port(sdev->device, i, 0, &port_modify) < 0)
+ PRINT_ERROR("%s", "disabling MAD processing failed.");
+ if (sport->mad_agent) {
+ ib_unregister_mad_agent(sport->mad_agent);
+ sport->mad_agent = NULL;
+ }
+ }
+}
+
+/**
+ * srpt_alloc_ioctx() - Allocate and initialize an SRPT I/O context structure.
+ */
+static struct srpt_ioctx *srpt_alloc_ioctx(struct srpt_device *sdev)
+{
+ struct srpt_ioctx *ioctx;
+
+ ioctx = kmalloc(sizeof *ioctx, GFP_KERNEL);
+ if (!ioctx)
+ goto out;
+
+ ioctx->buf = kzalloc(srp_max_message_size, GFP_KERNEL);
+ if (!ioctx->buf)
+ goto out_free_ioctx;
+
+ ioctx->dma = ib_dma_map_single(sdev->device, ioctx->buf,
+ srp_max_message_size, DMA_BIDIRECTIONAL);
+ if (ib_dma_mapping_error(sdev->device, ioctx->dma))
+ goto out_free_buf;
+
+ return ioctx;
+
+out_free_buf:
+ kfree(ioctx->buf);
+out_free_ioctx:
+ kfree(ioctx);
+out:
+ return NULL;
+}
+
+/**
+ * srpt_free_ioctx() - Deallocate an SRPT I/O context structure.
+ */
+static void srpt_free_ioctx(struct srpt_device *sdev, struct srpt_ioctx *ioctx)
+{
+ if (!ioctx)
+ return;
+
+ ib_dma_unmap_single(sdev->device, ioctx->dma,
+ srp_max_message_size, DMA_BIDIRECTIONAL);
+ kfree(ioctx->buf);
+ kfree(ioctx);
+}
+
+/**
+ * srpt_alloc_ioctx_ring() - Allocate a ring of SRPT I/O context structures.
+ * @sdev: Device to allocate the I/O context ring for.
+ * @ioctx_ring: Pointer to an array of I/O contexts.
+ * @ring_size: Number of elements in the I/O context ring.
+ * @flags: Flags to be set in the ring index.
+ */
+static int srpt_alloc_ioctx_ring(struct srpt_device *sdev,
+ struct srpt_ioctx **ioctx_ring,
+ int ring_size,
+ int flags)
+{
+ int res;
+ int i;
+
+ res = -ENOMEM;
+ for (i = 0; i < ring_size; ++i) {
+ ioctx_ring[i] = srpt_alloc_ioctx(sdev);
+
+ if (!ioctx_ring[i])
+ goto err;
+
+ EXTRACHECKS_WARN_ON(i & flags);
+ ioctx_ring[i]->index = i | flags;
+ }
+ res = 0;
+ goto out;
+
+err:
+ while (--i >= 0) {
+ srpt_free_ioctx(sdev, ioctx_ring[i]);
+ ioctx_ring[i] = NULL;
+ }
+out:
+ return res;
+}
+
+/**
+ * srpt_free_ioctx_ring() - Free the ring of SRPT I/O context structures.
+ */
+static void srpt_free_ioctx_ring(struct srpt_device *sdev,
+ struct srpt_ioctx **ioctx_ring,
+ int ring_size)
+{
+ int i;
+
+ for (i = 0; i < ring_size; ++i) {
+ srpt_free_ioctx(sdev, ioctx_ring[i]);
+ ioctx_ring[i] = NULL;
+ }
+}
+
+/**
+ * srpt_alloc_tti_ring() - Allocate target-to-initiator I/O contexts.
+ */
+static int srpt_alloc_tti_ioctx(struct srpt_rdma_ch *ch)
+{
+ return srpt_alloc_ioctx_ring(ch->sport->sdev, ch->tti_ioctx,
+ ARRAY_SIZE(ch->tti_ioctx),
+ SRPT_OP_TTI);
+}
+
+/**
+ * srpt_free_tti_ring() - Free target-to-initiator I/O contexts.
+ */
+static void srpt_free_tti_ioctx(struct srpt_rdma_ch *ch)
+{
+ srpt_free_ioctx_ring(ch->sport->sdev, ch->tti_ioctx,
+ ARRAY_SIZE(ch->tti_ioctx));
+}
+
+/**
+ * srpt_get_tti_ioctx() - Get a target-to-initiator I/O context.
+ */
+static struct srpt_ioctx *srpt_get_tti_ioctx(struct srpt_rdma_ch *ch)
+{
+ struct srpt_ioctx *ioctx;
+ struct srpt_device *sdev;
+ unsigned long flags;
+
+ sdev = ch->sport->sdev;
+ spin_lock_irqsave(&sdev->spinlock, flags);
+ EXTRACHECKS_WARN_ON(ch->tti_head - ch->tti_tail < 0);
+ if (ch->tti_head - ch->tti_tail < TTI_IOCTX_COUNT)
+ ioctx = ch->tti_ioctx[ch->tti_head++ & TTI_IOCTX_MASK];
+ else
+ ioctx = NULL;
+ spin_unlock_irqrestore(&sdev->spinlock, flags);
+ return ioctx;
+}
+
+/**
+ * srpt_put_tti_ioctx() - Put back a target-to-initiator I/O context.
+ */
+static void srpt_put_tti_ioctx(struct srpt_rdma_ch *ch)
+{
+ struct srpt_device *sdev;
+ unsigned long flags;
+
+ sdev = ch->sport->sdev;
+ spin_lock_irqsave(&sdev->spinlock, flags);
+ EXTRACHECKS_WARN_ON(ch->tti_head - ch->tti_tail < 0);
+ ch->tti_tail++;
+ EXTRACHECKS_WARN_ON(ch->tti_head - ch->tti_tail < 0);
+ spin_unlock_irqrestore(&sdev->spinlock, flags);
+}
+
+/**
+ * srpt_get_cmd_state() - Get the state of a SCSI command.
+ */
+static enum srpt_command_state srpt_get_cmd_state(struct srpt_ioctx *ioctx)
+{
+ BUG_ON(!ioctx);
+
+ return atomic_read(&ioctx->state);
+}
+
+/**
+ * srpt_set_cmd_state() - Set the state of a SCSI command.
+ * @new: New state to be set.
+ *
+ * Does not modify the state of aborted commands. Returns the previous command
+ * state.
+ */
+static enum srpt_command_state srpt_set_cmd_state(struct srpt_ioctx *ioctx,
+ enum srpt_command_state new)
+{
+ enum srpt_command_state previous;
+
+ BUG_ON(!ioctx);
+
+ do {
+ previous = atomic_read(&ioctx->state);
+ } while (previous != SRPT_STATE_DONE
+ && atomic_cmpxchg(&ioctx->state, previous, new) != previous);
+
+ return previous;
+}
+
+/**
+ * srpt_test_and_set_cmd_state() - Test and set the state of a command.
+ * @old: State to compare against.
+ * @new: New state to be set if the current state matches 'old'.
+ *
+ * Returns the previous command state.
+ */
+static enum srpt_command_state
+srpt_test_and_set_cmd_state(struct srpt_ioctx *ioctx,
+ enum srpt_command_state old,
+ enum srpt_command_state new)
+{
+ WARN_ON(!ioctx);
+ WARN_ON(old == SRPT_STATE_DONE);
+ WARN_ON(new == SRPT_STATE_NEW);
+
+ return atomic_cmpxchg(&ioctx->state, old, new);
+}
+
+/**
+ * srpt_post_recv() - Post an IB receive request.
+ */
+static int srpt_post_recv(struct srpt_device *sdev, struct srpt_ioctx *ioctx)
+{
+ struct ib_sge list;
+ struct ib_recv_wr wr, *bad_wr;
+
+ wr.wr_id = ioctx->index | SRPT_OP_RECV;
+
+ list.addr = ioctx->dma;
+ list.length = srp_max_message_size;
+ list.lkey = sdev->mr->lkey;
+
+ wr.next = NULL;
+ wr.sg_list = &list;
+ wr.num_sge = 1;
+
+ return ib_post_srq_recv(sdev->srq, &wr, &bad_wr);
+}
+
+/**
+ * srpt_post_send() - Post an IB send request.
+ * @ch: RDMA channel to post the send request on.
+ * @ioctx: I/O context of the send request.
+ * @len: length of the request to be sent in bytes.
+ *
+ * Returns zero upon success and a non-zero value upon failure.
+ */
+static int srpt_post_send(struct srpt_rdma_ch *ch, struct srpt_ioctx *ioctx,
+ int len)
+{
+ struct ib_sge list;
+ struct ib_send_wr wr, *bad_wr;
+ struct srpt_device *sdev = ch->sport->sdev;
+ int ret;
+
+ ret = -ENOMEM;
+ if (atomic_dec_return(&ch->sq_wr_avail) < 0) {
+ PRINT_ERROR("%s[%d]: send queue full", __func__, __LINE__);
+ goto out;
+ }
+
+ ib_dma_sync_single_for_device(sdev->device, ioctx->dma,
+ len, DMA_TO_DEVICE);
+
+ list.addr = ioctx->dma;
+ list.length = len;
+ list.lkey = sdev->mr->lkey;
+
+ wr.next = NULL;
+ wr.wr_id = ioctx->index;
+ wr.sg_list = &list;
+ wr.num_sge = 1;
+ wr.opcode = IB_WR_SEND;
+ wr.send_flags = IB_SEND_SIGNALED;
+
+ ret = ib_post_send(ch->qp, &wr, &bad_wr);
+
+out:
+ if (ret < 0)
+ atomic_inc(&ch->sq_wr_avail);
+ return ret;
+}
+
+/**
+ * srpt_get_desc_tbl() - Parse the data descriptors of an SRP_CMD request.
+ * @ioctx: Pointer to the I/O context associated with the request.
+ * @srp_cmd: Pointer to the SRP_CMD request data.
+ * @dir: Pointer to the variable to which the transfer direction will be
+ * written.
+ * @data_len: Pointer to the variable to which the total data length of all
+ * descriptors in the SRP_CMD request will be written.
+ *
+ * This function initializes ioctx->nrbuf and ioctx->r_bufs.
+ *
+ * Returns -EINVAL when the SRP_CMD request contains inconsistent descriptors;
+ * -ENOMEM when memory allocation fails and zero upon success.
+ */
+static int srpt_get_desc_tbl(struct srpt_ioctx *ioctx, struct srp_cmd *srp_cmd,
+ scst_data_direction *dir, u64 *data_len)
+{
+ struct srp_indirect_buf *idb;
+ struct srp_direct_buf *db;
+ unsigned add_cdb_offset;
+ int ret;
+
+ /*
+ * The pointer computations below will only be compiled correctly
+ * if srp_cmd::add_data is declared as s8*, u8*, s8[] or u8[], so check
+ * whether srp_cmd::add_data has been declared as a byte pointer.
+ */
+ BUILD_BUG_ON(!__same_type(srp_cmd->add_data[0], (s8)0)
+ && !__same_type(srp_cmd->add_data[0], (u8)0));
+
+ BUG_ON(!dir);
+ BUG_ON(!data_len);
+
+ ret = 0;
+ *data_len = 0;
+
+ /*
+ * The lower four bits of the buffer format field contain the DATA-IN
+ * buffer descriptor format, and the highest four bits contain the
+ * DATA-OUT buffer descriptor format.
+ */
+ *dir = SCST_DATA_NONE;
+ if (srp_cmd->buf_fmt & 0xf)
+ /* DATA-IN: transfer data from target to initiator. */
+ *dir = SCST_DATA_READ;
+ else if (srp_cmd->buf_fmt >> 4)
+ /* DATA-OUT: transfer data from initiator to target. */
+ *dir = SCST_DATA_WRITE;
+
+ /*
+ * According to the SRP spec, the lower two bits of the 'ADDITIONAL
+ * CDB LENGTH' field are reserved and the size in bytes of this field
+ * is four times the value specified in bits 3..7. Hence the "& ~3".
+ */
+ add_cdb_offset = srp_cmd->add_cdb_len & ~3;
+ if (((srp_cmd->buf_fmt & 0xf) == SRP_DATA_DESC_DIRECT) ||
+ ((srp_cmd->buf_fmt >> 4) == SRP_DATA_DESC_DIRECT)) {
+ ioctx->n_rbuf = 1;
+ ioctx->rbufs = &ioctx->single_rbuf;
+
+ db = (struct srp_direct_buf *)(srp_cmd->add_data
+ + add_cdb_offset);
+ memcpy(ioctx->rbufs, db, sizeof *db);
+ *data_len = be32_to_cpu(db->len);
+ } else if (((srp_cmd->buf_fmt & 0xf) == SRP_DATA_DESC_INDIRECT) ||
+ ((srp_cmd->buf_fmt >> 4) == SRP_DATA_DESC_INDIRECT)) {
+ idb = (struct srp_indirect_buf *)(srp_cmd->add_data
+ + add_cdb_offset);
+
+ ioctx->n_rbuf = be32_to_cpu(idb->table_desc.len) / sizeof *db;
+
+ if (ioctx->n_rbuf >
+ (srp_cmd->data_out_desc_cnt + srp_cmd->data_in_desc_cnt)) {
+ PRINT_ERROR("received unsupported SRP_CMD request type"
+ " (%u out + %u in != %u / %zu)",
+ srp_cmd->data_out_desc_cnt,
+ srp_cmd->data_in_desc_cnt,
+ be32_to_cpu(idb->table_desc.len),
+ sizeof(*db));
+ ioctx->n_rbuf = 0;
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (ioctx->n_rbuf == 1)
+ ioctx->rbufs = &ioctx->single_rbuf;
+ else {
+ ioctx->rbufs =
+ kmalloc(ioctx->n_rbuf * sizeof *db, GFP_ATOMIC);
+ if (!ioctx->rbufs) {
+ ioctx->n_rbuf = 0;
+ ret = -ENOMEM;
+ goto out;
+ }
+ }
+
+ db = idb->desc_list;
+ memcpy(ioctx->rbufs, db, ioctx->n_rbuf * sizeof *db);
+ *data_len = be32_to_cpu(idb->len);
+ }
+out:
+ return ret;
+}
+
+/**
+ * srpt_init_ch_qp() - Initialize queue pair attributes.
+ *
+ * Initialized the attributes of queue pair 'qp' by allowing local write,
+ * remote read and remote write. Also transitions 'qp' to state IB_QPS_INIT.
+ */
+static int srpt_init_ch_qp(struct srpt_rdma_ch *ch, struct ib_qp *qp)
+{
+ struct ib_qp_attr *attr;
+ int ret;
+
+ attr = kzalloc(sizeof *attr, GFP_KERNEL);
+ if (!attr)
+ return -ENOMEM;
+
+ attr->qp_state = IB_QPS_INIT;
+ attr->qp_access_flags = IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_READ |
+ IB_ACCESS_REMOTE_WRITE;
+ attr->port_num = ch->sport->port;
+ attr->pkey_index = 0;
+
+ ret = ib_modify_qp(qp, attr,
+ IB_QP_STATE | IB_QP_ACCESS_FLAGS | IB_QP_PORT |
+ IB_QP_PKEY_INDEX);
+
+ kfree(attr);
+ return ret;
+}
+
+/**
+ * srpt_ch_qp_rtr() - Change the state of a channel to 'ready to receive' (RTR).
+ * @ch: channel of the queue pair.
+ * @qp: queue pair to change the state of.
+ *
+ * Returns zero upon success and a negative value upon failure.
+ *
+ * Note: currently a struct ib_qp_attr takes 136 bytes on a 64-bit system.
+ * If this structure ever becomes larger, it might be necessary to allocate
+ * it dynamically instead of on the stack.
+ */
+static int srpt_ch_qp_rtr(struct srpt_rdma_ch *ch, struct ib_qp *qp)
+{
+ struct ib_qp_attr qp_attr;
+ int attr_mask;
+ int ret;
+
+ qp_attr.qp_state = IB_QPS_RTR;
+ ret = ib_cm_init_qp_attr(ch->cm_id, &qp_attr, &attr_mask);
+ if (ret)
+ goto out;
+
+ qp_attr.max_dest_rd_atomic = 4;
+
+ ret = ib_modify_qp(qp, &qp_attr, attr_mask);
+
+out:
+ return ret;
+}
+
+/**
+ * srpt_ch_qp_rts() - Change the state of a channel to 'ready to send' (RTS).
+ * @ch: channel of the queue pair.
+ * @qp: queue pair to change the state of.
+ *
+ * Returns zero upon success and a negative value upon failure.
+ *
+ * Note: currently a struct ib_qp_attr takes 136 bytes on a 64-bit system.
+ * If this structure ever becomes larger, it might be necessary to allocate
+ * it dynamically instead of on the stack.
+ */
+static int srpt_ch_qp_rts(struct srpt_rdma_ch *ch, struct ib_qp *qp)
+{
+ struct ib_qp_attr qp_attr;
+ int attr_mask;
+ int ret;
+
+ qp_attr.qp_state = IB_QPS_RTS;
+ ret = ib_cm_init_qp_attr(ch->cm_id, &qp_attr, &attr_mask);
+ if (ret)
+ goto out;
+
+ qp_attr.max_rd_atomic = 4;
+
+ ret = ib_modify_qp(qp, &qp_attr, attr_mask);
+
+out:
+ return ret;
+}
+
+/**
+ * srpt_req_lim_delta() - Compute req_lim delta.
+ *
+ * Compute by how much req_lim changed since the last time this function has
+ * been called. This value is necessary for filling in the REQUEST LIMIT DELTA
+ * field of an SRP_RSP response.
+ *
+ * Side Effect:
+ * Resets ch->req_lim_delta.
+ *
+ * Note:
+ * The caller must either pass the returned value to the initiator in the
+ * REQUEST LIMIT DELTA field of an SRP information unit or pass the returned
+ * value to srpt_undo_req_lim_delta(). Any other approach will result in an
+ * SRP protocol violation.
+ */
+static int srpt_req_lim_delta(struct srpt_rdma_ch *ch)
+{
+ return atomic_xchg(&ch->req_lim_delta, 0);
+}
+
+/**
+ * srpt_undo_req_lim_delta() - Undo the side effect of srpt_req_lim_delta().
+ * @ch: Channel pointer.
+ * @delta: return value of srpt_req_lim_delta().
+ */
+static void srpt_undo_req_lim_delta(struct srpt_rdma_ch *ch, int delta)
+{
+ atomic_add(delta, &ch->req_lim_delta);
+}
+
+/**
+ * srpt_send_cred_req() - Send an SRP_CRED_REQ IU to the initiator.
+ *
+ * The previous value of ch->req_lim_delta is restored if sending fails
+ * synchronously or asynchronously.
+ */
+static void srpt_send_cred_req(struct srpt_rdma_ch *ch, s32 req_lim_delta)
+{
+ struct srpt_ioctx *ioctx;
+ struct srp_cred_req *srp_cred_req;
+ int res;
+
+ ioctx = srpt_get_tti_ioctx(ch);
+ if (!ioctx) {
+ PRINT_ERROR("%s",
+ "Sending SRP_CRED_REQ failed -- no I/O context"
+ " available ! This will sooner or later result"
+ " in an initiator lockup.");
+ goto err;
+ }
+
+ BUG_ON(!ch);
+ srp_cred_req = ioctx->buf;
+ BUG_ON(!srp_cred_req);
+ memset(srp_cred_req, 0, sizeof(*srp_cred_req));
+ srp_cred_req->opcode = SRP_CRED_REQ;
+ srp_cred_req->req_lim_delta = cpu_to_be32(req_lim_delta);
+ srp_cred_req->tag = cpu_to_be64(0);
+ res = srpt_post_send(ch, ioctx, sizeof(*srp_cred_req));
+ if (res) {
+ PRINT_ERROR("sending SRP_CRED_REQ failed (res = %d)", res);
+ goto err_put;
+ }
+
+ TRACE_DBG("Sent SRP_CRED_REQ with req_lim_delta = %d and tag %lld",
+ req_lim_delta, 0ULL);
+
+ goto out;
+
+err_put:
+ srpt_put_tti_ioctx(ch);
+err:
+ srpt_undo_req_lim_delta(ch, req_lim_delta);
+out:
+ return;
+}
+
+/**
+ * srpt_reset_ioctx() - Free up resources and post again for receiving.
+ *
+ * Note: Do NOT modify *ioctx after this function has finished. Otherwise a
+ * race condition will be triggered between srpt_rcv_completion() and the
+ * caller of this function on *ioctx.
+ */
+static void srpt_reset_ioctx(struct srpt_rdma_ch *ch, struct srpt_ioctx *ioctx,
+ bool inc_req_lim)
+{
+ BUG_ON(!ch);
+ BUG_ON(!ioctx);
+
+ WARN_ON(srpt_get_cmd_state(ioctx) != SRPT_STATE_DONE);
+
+ ioctx->scmnd = NULL;
+ ioctx->ch = NULL;
+
+ /*
+ * If the WARN_ON() below gets triggered this means that
+ * srpt_unmap_sg_to_ib_sge() has not been called before
+ * scst_tgt_cmd_done().
+ */
+ WARN_ON(ioctx->mapped_sg_count);
+
+ if (ioctx->n_rbuf > 1) {
+ kfree(ioctx->rbufs);
+ ioctx->rbufs = NULL;
+ ioctx->n_rbuf = 0;
+ }
+
+ if (srpt_post_recv(ch->sport->sdev, ioctx))
+ PRINT_ERROR("%s", "SRQ post_recv failed - this is serious.");
+ else if (inc_req_lim) {
+ int req_lim;
+
+ atomic_inc(&ch->req_lim_delta);
+ req_lim = atomic_inc_return(&ch->req_lim);
+ if (req_lim < 0 || req_lim > ch->rq_size)
+ PRINT_ERROR("req_lim = %d out of range %d .. %d",
+ req_lim, 0, ch->rq_size);
+ if (atomic_read(&ch->supports_cred_req)) {
+ if (req_lim == ch->rq_size / 2
+ && atomic_read(&ch->req_lim_delta) > ch->rq_size/4)
+ srpt_send_cred_req(ch, srpt_req_lim_delta(ch));
+ } else {
+ if (atomic_add_unless(&ch->req_lim_waiter_count, -1, 0))
+ complete(&ch->req_lim_compl);
+ }
+ }
+}
+
+/**
+ * srpt_abort_scst_cmd() - Abort a SCSI command.
+ * @ioctx: I/O context associated with the SCSI command.
+ * @context: Preferred execution context.
+ */
+static void srpt_abort_scst_cmd(struct srpt_ioctx *ioctx,
+ enum scst_exec_context context)
+{
+ struct scst_cmd *scmnd;
+ enum srpt_command_state state;
+
+ BUG_ON(!ioctx);
+
+ /*
+ * If the command is in a state where the SCST core is waiting for the
+ * ib_srpt driver, change the state to the next state. Changing the
+ * state of the command from SRPT_NEED_DATA to SRPT_STATE_DATA_IN
+ * ensures that srpt_xmit_response() will call this function a second
+ * time.
+ */
+ state = srpt_test_and_set_cmd_state(ioctx, SRPT_STATE_NEED_DATA,
+ SRPT_STATE_DATA_IN);
+ if (state != SRPT_STATE_NEED_DATA) {
+ state = srpt_test_and_set_cmd_state(ioctx, SRPT_STATE_DATA_IN,
+ SRPT_STATE_DONE);
+ if (state != SRPT_STATE_DATA_IN) {
+ state = srpt_test_and_set_cmd_state(ioctx,
+ SRPT_STATE_CMD_RSP_SENT, SRPT_STATE_DONE);
+ if (state != SRPT_STATE_CMD_RSP_SENT)
+ state = srpt_test_and_set_cmd_state(ioctx,
+ SRPT_STATE_MGMT_RSP_SENT,
+ SRPT_STATE_DONE);
+ }
+ }
+ if (state == SRPT_STATE_DONE)
+ goto out;
+
+ scmnd = ioctx->scmnd;
+ WARN_ON(!scmnd);
+ if (!scmnd)
+ goto out;
+
+ WARN_ON(ioctx != scst_cmd_get_tgt_priv(scmnd));
+
+ TRACE_DBG("Aborting cmd with state %d and tag %lld",
+ state, scst_cmd_get_tag(scmnd));
+
+ switch (state) {
+ case SRPT_STATE_NEW:
+ /*
+ * Do nothing - defer abort processing until
+ * srpt_xmit_response() is invoked.
+ */
+ WARN_ON(!scst_cmd_aborted(scmnd));
+ break;
+ case SRPT_STATE_DATA_IN:
+ /*
+ * Invocation of srpt_pending_cmd_timeout() after
+ * srpt_handle_rdma_comp() set the state to SRPT_STATE_DATA_IN
+ * and before srpt_xmit_response() set the state to
+ * SRPT_STATE_CMD_RSP_SENT. Ignore the timeout and let
+ * srpt_handle_xmit_response() proceed.
+ */
+ break;
+ case SRPT_STATE_NEED_DATA:
+ /* SCST_DATA_WRITE - RDMA read error or RDMA read timeout. */
+ scst_rx_data(ioctx->scmnd, SCST_RX_STATUS_ERROR, context);
+ break;
+ case SRPT_STATE_CMD_RSP_SENT:
+ /*
+ * SRP_RSP sending failed or the SRP_RSP send completion has
+ * not been received in time.
+ */
+ srpt_unmap_sg_to_ib_sge(ioctx->ch, ioctx);
+ scst_set_delivery_status(scmnd, SCST_CMD_DELIVERY_ABORTED);
+ scst_tgt_cmd_done(scmnd, context);
+ break;
+ case SRPT_STATE_MGMT_RSP_SENT:
+ /*
+ * Management command response sending failed. This state is
+ * never reached since there is no scmnd associated with
+ * management commands. Note: the SCST core frees these
+ * commands immediately after srpt_tsk_mgmt_done() returned.
+ */
+ WARN_ON("ERROR: unexpected command state");
+ break;
+ default:
+ WARN_ON("ERROR: unexpected command state");
+ break;
+ }
+
+out:
+ ;
+}
+
+/**
+ * srpt_handle_send_err_comp() - Process an IB_WC_SEND or RDMA error completion.
+ */
+static void srpt_handle_send_err_comp(struct srpt_rdma_ch *ch, u64 wr_id,
+ enum scst_exec_context context)
+{
+ struct srpt_ioctx *ioctx;
+ struct srpt_device *sdev = ch->sport->sdev;
+ enum srpt_command_state state;
+ struct scst_cmd *scmnd;
+
+ EXTRACHECKS_WARN_ON(wr_id & SRPT_OP_RECV);
+
+ ioctx = sdev->ioctx_ring[wr_id & ~SRPT_OP_TTI];
+
+ if ((wr_id & SRPT_OP_TTI) == 0) {
+ state = srpt_get_cmd_state(ioctx);
+ scmnd = ioctx->scmnd;
+
+ EXTRACHECKS_WARN_ON(state != SRPT_STATE_CMD_RSP_SENT
+ && state != SRPT_STATE_MGMT_RSP_SENT
+ && state != SRPT_STATE_NEED_DATA
+ && state != SRPT_STATE_DONE);
+
+ if (state != SRPT_STATE_DONE) {
+ if (scmnd)
+ srpt_abort_scst_cmd(ioctx, context);
+ else {
+ srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+ srpt_reset_ioctx(ch, ioctx, 1);
+ }
+ } else
+ PRINT_ERROR("Received more than one IB error completion"
+ " for wr_id = %u.", (unsigned)wr_id);
+ } else {
+ struct srp_cred_req *srp_cred_req;
+ s32 req_lim_delta;
+
+ srp_cred_req = ioctx->buf;
+ req_lim_delta = be32_to_cpu(srp_cred_req->req_lim_delta);
+ srpt_undo_req_lim_delta(ch, req_lim_delta);
+ srpt_put_tti_ioctx(ch);
+ PRINT_ERROR("Sending SRP_CRED_REQ with delta = %d failed.",
+ req_lim_delta);
+ }
+}
+
+/**
+ * srpt_handle_send_comp() - Process an IB send completion notification.
+ */
+static void srpt_handle_send_comp(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx,
+ enum scst_exec_context context)
+{
+ enum srpt_command_state state;
+
+ atomic_inc(&ch->sq_wr_avail);
+
+ state = srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+
+ EXTRACHECKS_WARN_ON(state != SRPT_STATE_CMD_RSP_SENT
+ && state != SRPT_STATE_MGMT_RSP_SENT
+ && state != SRPT_STATE_DONE);
+
+ if (state != SRPT_STATE_DONE) {
+ struct scst_cmd *scmnd;
+
+ scmnd = ioctx->scmnd;
+ EXTRACHECKS_WARN_ON((state == SRPT_STATE_MGMT_RSP_SENT)
+ != (scmnd == NULL));
+ if (scmnd) {
+ srpt_unmap_sg_to_ib_sge(ch, ioctx);
+ scst_tgt_cmd_done(scmnd, context);
+ } else
+ srpt_reset_ioctx(ch, ioctx, 1);
+ } else {
+ PRINT_ERROR("IB completion has been received too late for"
+ " wr_id = %u.", ioctx->index);
+ }
+}
+
+/**
+ * srpt_handle_rdma_comp() - Process an IB RDMA completion notification.
+ */
+static void srpt_handle_rdma_comp(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx,
+ enum scst_exec_context context)
+{
+ enum srpt_command_state state;
+ struct scst_cmd *scmnd;
+
+ EXTRACHECKS_WARN_ON(ioctx->n_rdma <= 0);
+ atomic_add(ioctx->n_rdma, &ch->sq_wr_avail);
+
+ scmnd = ioctx->scmnd;
+ if (scmnd) {
+ state = srpt_test_and_set_cmd_state(ioctx, SRPT_STATE_NEED_DATA,
+ SRPT_STATE_DATA_IN);
+
+ EXTRACHECKS_WARN_ON(state != SRPT_STATE_NEED_DATA);
+
+ scst_rx_data(ioctx->scmnd, SCST_RX_STATUS_SUCCESS, context);
+ } else
+ PRINT_ERROR("%s[%d]: scmnd == NULL", __func__, __LINE__);
+}
+
+/**
+ * srpt_build_cmd_rsp() - Build an SRP_RSP response.
+ * @ch: RDMA channel through which the request has been received.
+ * @ioctx: I/O context associated with the SRP_CMD request. The response will
+ * be built in the buffer ioctx->buf points at and hence this function will
+ * overwrite the request data.
+ * @tag: tag of the request for which this response is being generated.
+ * @status: value for the STATUS field of the SRP_RSP information unit.
+ * @sense_data: pointer to sense data to be included in the response.
+ * @sense_data_len: length in bytes of the sense data.
+ *
+ * Returns the size in bytes of the SRP_RSP response.
+ *
+ * An SRP_RSP response contains a SCSI status or service response. See also
+ * section 6.9 in the SRP r16a document for the format of an SRP_RSP
+ * response. See also SPC-2 for more information about sense data.
+ */
+static int srpt_build_cmd_rsp(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx, s32 req_lim_delta,
+ u64 tag, int status,
+ const u8 *sense_data, int sense_data_len)
+{
+ struct srp_rsp *srp_rsp;
+ int max_sense_len;
+
+ /*
+ * The lowest bit of all SAM-3 status codes is zero (see also
+ * paragraph 5.3 in SAM-3).
+ */
+ EXTRACHECKS_WARN_ON(status & 1);
+
+ srp_rsp = ioctx->buf;
+ BUG_ON(!srp_rsp);
+ memset(srp_rsp, 0, sizeof *srp_rsp);
+
+ srp_rsp->opcode = SRP_RSP;
+ srp_rsp->req_lim_delta = cpu_to_be32(req_lim_delta);
+ srp_rsp->tag = tag;
+ srp_rsp->status = status;
+
+ if (!SCST_SENSE_VALID(sense_data))
+ sense_data_len = 0;
+ else {
+ BUILD_BUG_ON(MIN_MAX_MESSAGE_SIZE <= sizeof(*srp_rsp));
+ max_sense_len = ch->max_ti_iu_len - sizeof(*srp_rsp);
+ if (sense_data_len > max_sense_len) {
+ PRINT_WARNING("truncated sense data from %d to %d"
+ " bytes", sense_data_len,
+ max_sense_len);
+ sense_data_len = max_sense_len;
+ }
+
+ srp_rsp->flags |= SRP_RSP_FLAG_SNSVALID;
+ srp_rsp->sense_data_len = cpu_to_be32(sense_data_len);
+ memcpy(srp_rsp + 1, sense_data, sense_data_len);
+ }
+
+ return sizeof(*srp_rsp) + sense_data_len;
+}
+
+/**
+ * srpt_build_tskmgmt_rsp() - Build a task management response.
+ * @ch: RDMA channel through which the request has been received.
+ * @ioctx: I/O context in which the SRP_RSP response will be built.
+ * @rsp_code: RSP_CODE that will be stored in the response.
+ * @tag: Tag of the request for which this response is being generated.
+ *
+ * Returns the size in bytes of the SRP_RSP response.
+ *
+ * An SRP_RSP response contains a SCSI status or service response. See also
+ * section 6.9 in the SRP r16a document for the format of an SRP_RSP
+ * response.
+ */
+static int srpt_build_tskmgmt_rsp(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx, s32 req_lim_delta,
+ u8 rsp_code, u64 tag)
+{
+ struct srp_rsp *srp_rsp;
+ int resp_data_len;
+ int resp_len;
+
+ resp_data_len = (rsp_code == SRP_TSK_MGMT_SUCCESS) ? 0 : 4;
+ resp_len = sizeof(*srp_rsp) + resp_data_len;
+
+ srp_rsp = ioctx->buf;
+ memset(srp_rsp, 0, sizeof *srp_rsp);
+
+ srp_rsp->opcode = SRP_RSP;
+ srp_rsp->req_lim_delta = cpu_to_be32(req_lim_delta);
+ srp_rsp->tag = tag;
+
+ if (rsp_code != SRP_TSK_MGMT_SUCCESS) {
+ srp_rsp->flags |= SRP_RSP_FLAG_RSPVALID;
+ srp_rsp->resp_data_len = cpu_to_be32(resp_data_len);
+ srp_rsp->data[3] = rsp_code;
+ }
+
+ return resp_len;
+}
+
+/**
+ * srpt_handle_cmd() - Process SRP_CMD.
+ */
+static int srpt_handle_cmd(struct srpt_rdma_ch *ch, struct srpt_ioctx *ioctx,
+ enum scst_exec_context context)
+{
+ struct scst_cmd *scmnd;
+ struct srp_cmd *srp_cmd;
+ scst_data_direction dir;
+ u64 data_len;
+ int ret;
+
+ srp_cmd = ioctx->buf;
+
+ scmnd = scst_rx_cmd(ch->scst_sess, (u8 *) &srp_cmd->lun,
+ sizeof srp_cmd->lun, srp_cmd->cdb,
+ sizeof srp_cmd->cdb, context);
+ if (!scmnd)
+ goto err;
+
+ ioctx->scmnd = scmnd;
+
+ ret = srpt_get_desc_tbl(ioctx, srp_cmd, &dir, &data_len);
+ if (ret) {
+ scst_set_cmd_error(scmnd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto err;
+ }
+
+ switch (srp_cmd->task_attr) {
+ case SRP_CMD_HEAD_OF_Q:
+ scst_cmd_set_queue_type(scmnd, SCST_CMD_QUEUE_HEAD_OF_QUEUE);
+ break;
+ case SRP_CMD_ORDERED_Q:
+ scst_cmd_set_queue_type(scmnd, SCST_CMD_QUEUE_ORDERED);
+ break;
+ case SRP_CMD_SIMPLE_Q:
+ scst_cmd_set_queue_type(scmnd, SCST_CMD_QUEUE_SIMPLE);
+ break;
+ case SRP_CMD_ACA:
+ scst_cmd_set_queue_type(scmnd, SCST_CMD_QUEUE_ACA);
+ break;
+ default:
+ scst_cmd_set_queue_type(scmnd, SCST_CMD_QUEUE_ORDERED);
+ break;
+ }
+
+ scst_cmd_set_tag(scmnd, srp_cmd->tag);
+ scst_cmd_set_tgt_priv(scmnd, ioctx);
+ scst_cmd_set_expected(scmnd, dir, data_len);
+ scst_cmd_init_done(scmnd, context);
+
+ return 0;
+
+err:
+ return -1;
+}
+
+/**
+ * srpt_handle_tsk_mgmt() - Process an SRP_TSK_MGMT information unit.
+ *
+ * Returns SCST_MGMT_STATUS_SUCCESS upon success.
+ *
+ * Each task management function is performed by calling one of the
+ * scst_rx_mgmt_fn*() functions. These functions will either report failure
+ * or process the task management function asynchronously. The function
+ * srpt_tsk_mgmt_done() will be called by the SCST core upon completion of the
+ * task management function. When srpt_handle_tsk_mgmt() reports failure
+ * (i.e. returns -1) a response will have been built in ioctx->buf. This
+ * information unit has to be sent back by the caller.
+ *
+ * For more information about SRP_TSK_MGMT information units, see also section
+ * 6.7 in the SRP r16a document.
+ */
+static u8 srpt_handle_tsk_mgmt(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx)
+{
+ struct srp_tsk_mgmt *srp_tsk;
+ struct srpt_mgmt_ioctx *mgmt_ioctx;
+ int ret;
+
+ srp_tsk = ioctx->buf;
+
+ TRACE_DBG("recv_tsk_mgmt= %d for task_tag= %lld"
+ " using tag= %lld cm_id= %p sess= %p",
+ srp_tsk->tsk_mgmt_func, srp_tsk->task_tag, srp_tsk->tag,
+ ch->cm_id, ch->scst_sess);
+
+ ret = SCST_MGMT_STATUS_FAILED;
+ mgmt_ioctx = kmalloc(sizeof *mgmt_ioctx, GFP_ATOMIC);
+ if (!mgmt_ioctx)
+ goto err;
+
+ mgmt_ioctx->ioctx = ioctx;
+ mgmt_ioctx->ch = ch;
+ mgmt_ioctx->tag = srp_tsk->tag;
+
+ switch (srp_tsk->tsk_mgmt_func) {
+ case SRP_TSK_ABORT_TASK:
+ TRACE_DBG("%s", "Processing SRP_TSK_ABORT_TASK");
+ ret = scst_rx_mgmt_fn_tag(ch->scst_sess,
+ SCST_ABORT_TASK,
+ srp_tsk->task_tag,
+ SCST_ATOMIC, mgmt_ioctx);
+ break;
+ case SRP_TSK_ABORT_TASK_SET:
+ TRACE_DBG("%s", "Processing SRP_TSK_ABORT_TASK_SET");
+ ret = scst_rx_mgmt_fn_lun(ch->scst_sess,
+ SCST_ABORT_TASK_SET,
+ (u8 *) &srp_tsk->lun,
+ sizeof srp_tsk->lun,
+ SCST_ATOMIC, mgmt_ioctx);
+ break;
+ case SRP_TSK_CLEAR_TASK_SET:
+ TRACE_DBG("%s", "Processing SRP_TSK_CLEAR_TASK_SET");
+ ret = scst_rx_mgmt_fn_lun(ch->scst_sess,
+ SCST_CLEAR_TASK_SET,
+ (u8 *) &srp_tsk->lun,
+ sizeof srp_tsk->lun,
+ SCST_ATOMIC, mgmt_ioctx);
+ break;
+ case SRP_TSK_LUN_RESET:
+ TRACE_DBG("%s", "Processing SRP_TSK_LUN_RESET");
+ ret = scst_rx_mgmt_fn_lun(ch->scst_sess,
+ SCST_LUN_RESET,
+ (u8 *) &srp_tsk->lun,
+ sizeof srp_tsk->lun,
+ SCST_ATOMIC, mgmt_ioctx);
+ break;
+ case SRP_TSK_CLEAR_ACA:
+ TRACE_DBG("%s", "Processing SRP_TSK_CLEAR_ACA");
+ ret = scst_rx_mgmt_fn_lun(ch->scst_sess,
+ SCST_CLEAR_ACA,
+ (u8 *) &srp_tsk->lun,
+ sizeof srp_tsk->lun,
+ SCST_ATOMIC, mgmt_ioctx);
+ break;
+ default:
+ TRACE_DBG("%s", "Unsupported task management function.");
+ ret = SCST_MGMT_STATUS_FN_NOT_SUPPORTED;
+ }
+
+ if (ret != SCST_MGMT_STATUS_SUCCESS)
+ goto err;
+ return ret;
+
+err:
+ kfree(mgmt_ioctx);
+ return ret;
+}
+
+static void srpt_handle_cred_rsp(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx)
+{
+ int max_lun_commands;
+ int req_lim_delta;
+
+ if (!atomic_read(&ch->supports_cred_req)) {
+ atomic_set(&ch->supports_cred_req, true);
+ PRINT_INFO("Enabled SRP_CRED_REQ support for session %s",
+ ch->sess_name);
+
+ max_lun_commands = scst_get_max_lun_commands(NULL, 0);
+ if (4 <= max_lun_commands && max_lun_commands < ch->rq_size) {
+ req_lim_delta = ch->rq_size - max_lun_commands;
+ PRINT_INFO("Decreasing initiator request limit from %d"
+ " to %d", ch->rq_size, max_lun_commands);
+ /*
+ * Note: at least in theory this may make the req_lim
+ * variable managed by the initiator temporarily
+ * negative.
+ */
+ ch->rq_size -= req_lim_delta;
+ atomic_sub(req_lim_delta, &ch->req_lim);
+ atomic_sub(req_lim_delta, &ch->req_lim_delta);
+ }
+ }
+}
+
+static u8 scst_to_srp_tsk_mgmt_status(const int scst_mgmt_status)
+{
+ switch (scst_mgmt_status) {
+ case SCST_MGMT_STATUS_SUCCESS:
+ return SRP_TSK_MGMT_SUCCESS;
+ case SCST_MGMT_STATUS_FN_NOT_SUPPORTED:
+ return SRP_TSK_MGMT_FUNC_NOT_SUPP;
+ case SCST_MGMT_STATUS_TASK_NOT_EXIST:
+ case SCST_MGMT_STATUS_LUN_NOT_EXIST:
+ case SCST_MGMT_STATUS_REJECTED:
+ case SCST_MGMT_STATUS_FAILED:
+ default:
+ break;
+ }
+ return SRP_TSK_MGMT_FAILED;
+}
+
+/**
+ * srpt_handle_new_iu() - Process a newly received information unit.
+ * @ch: RDMA channel through which the information unit has been received.
+ * @ioctx: SRPT I/O context associated with the information unit.
+ */
+static void srpt_handle_new_iu(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx,
+ enum scst_exec_context context)
+{
+ struct srp_cmd *srp_cmd;
+ struct scst_cmd *scmnd;
+ enum rdma_ch_state ch_state;
+ u8 srp_response_status;
+ int tsk_mgmt_status;
+ int len;
+ int send_rsp_res;
+
+ ch_state = atomic_read(&ch->state);
+ if (ch_state == RDMA_CHANNEL_CONNECTING) {
+ list_add_tail(&ioctx->wait_list, &ch->cmd_wait_list);
+ return;
+ }
+
+ ioctx->n_rbuf = 0;
+ ioctx->rbufs = NULL;
+ ioctx->n_rdma = 0;
+ ioctx->n_rdma_ius = 0;
+ ioctx->rdma_ius = NULL;
+ ioctx->mapped_sg_count = 0;
+ ioctx->scmnd = NULL;
+ ioctx->ch = ch;
+ atomic_set(&ioctx->state, SRPT_STATE_NEW);
+
+ if (unlikely(ch_state == RDMA_CHANNEL_DISCONNECTING)) {
+ srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+ srpt_reset_ioctx(ch, ioctx, 0);
+ return;
+ }
+
+ WARN_ON(ch_state != RDMA_CHANNEL_LIVE);
+
+ scmnd = NULL;
+
+ srp_response_status = SAM_STAT_BUSY;
+ /* To keep the compiler happy. */
+ tsk_mgmt_status = SCST_MGMT_STATUS_FAILED;
+
+ ib_dma_sync_single_for_cpu(ch->sport->sdev->device,
+ ioctx->dma, srp_max_message_size,
+ DMA_FROM_DEVICE);
+
+ srp_cmd = ioctx->buf;
+
+ if (srp_cmd->opcode == SRP_CMD || srp_cmd->opcode == SRP_TSK_MGMT
+ || srp_cmd->opcode == SRP_I_LOGOUT) {
+ int req_lim;
+
+ req_lim = atomic_dec_return(&ch->req_lim);
+ if (unlikely(req_lim < 0))
+ PRINT_ERROR("req_lim = %d < 0", req_lim);
+ }
+
+ switch (srp_cmd->opcode) {
+ case SRP_CMD:
+ if (srpt_handle_cmd(ch, ioctx, context) < 0) {
+ scmnd = ioctx->scmnd;
+ if (scmnd)
+ srp_response_status =
+ scst_cmd_get_status(scmnd);
+ goto err;
+ }
+ break;
+
+ case SRP_TSK_MGMT:
+ tsk_mgmt_status = srpt_handle_tsk_mgmt(ch, ioctx);
+ if (tsk_mgmt_status != SCST_MGMT_STATUS_SUCCESS)
+ goto err;
+ break;
+
+ case SRP_I_LOGOUT:
+ goto err;
+
+ case SRP_CRED_RSP:
+ TRACE_DBG("%s", "received SRP_CRED_RSP");
+ srpt_handle_cred_rsp(ch, ioctx);
+ srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+ srpt_reset_ioctx(ch, ioctx, 0);
+ break;
+
+ case SRP_AER_RSP:
+ TRACE_DBG("%s", "received SRP_AER_RSP");
+ srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+ srpt_reset_ioctx(ch, ioctx, 0);
+ break;
+
+ case SRP_RSP:
+ default:
+ PRINT_ERROR("received IU with unknown opcode 0x%x",
+ srp_cmd->opcode);
+ srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+ srpt_reset_ioctx(ch, ioctx, 0);
+ break;
+ }
+
+ return;
+
+err:
+ send_rsp_res = -ENOTCONN;
+
+ if (atomic_read(&ch->state) != RDMA_CHANNEL_LIVE) {
+ /* Give up if another thread modified the channel state. */
+ PRINT_ERROR("%s", "channel is no longer in connected state.");
+ } else {
+ s32 req_lim_delta;
+
+ req_lim_delta = srpt_req_lim_delta(ch);
+ if (srp_cmd->opcode == SRP_TSK_MGMT)
+ len = srpt_build_tskmgmt_rsp(ch, ioctx, req_lim_delta,
+ scst_to_srp_tsk_mgmt_status(tsk_mgmt_status),
+ ((struct srp_tsk_mgmt *)srp_cmd)->tag);
+ else if (scmnd)
+ len = srpt_build_cmd_rsp(ch, ioctx, req_lim_delta,
+ srp_cmd->tag, srp_response_status,
+ scst_cmd_get_sense_buffer(scmnd),
+ scst_cmd_get_sense_buffer_len(scmnd));
+ else
+ len = srpt_build_cmd_rsp(ch, ioctx, srp_cmd->tag,
+ req_lim_delta,
+ srp_response_status,
+ NULL, 0);
+ srpt_set_cmd_state(ioctx,
+ srp_cmd->opcode == SRP_TSK_MGMT
+ ? SRPT_STATE_MGMT_RSP_SENT
+ : SRPT_STATE_CMD_RSP_SENT);
+ send_rsp_res = srpt_post_send(ch, ioctx, len);
+ if (send_rsp_res) {
+ PRINT_ERROR("%s", "Sending SRP_RSP response failed.");
+ srpt_undo_req_lim_delta(ch, req_lim_delta - 1);
+ }
+ }
+ if (send_rsp_res) {
+ if (scmnd)
+ srpt_abort_scst_cmd(ioctx, context);
+ else {
+ srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+ srpt_reset_ioctx(ch, ioctx, 1);
+ }
+ }
+}
+
+static void srpt_process_rcv_completion(struct ib_cq *cq,
+ struct srpt_rdma_ch *ch,
+ enum scst_exec_context context,
+ struct ib_wc *wc)
+{
+ struct srpt_device *sdev = ch->sport->sdev;
+ struct srpt_ioctx *ioctx;
+
+ EXTRACHECKS_WARN_ON((wc->wr_id & SRPT_OP_RECV) == 0);
+ EXTRACHECKS_WARN_ON((wc->wr_id & SRPT_OP_TTI) != 0);
+
+ if (wc->status == IB_WC_SUCCESS) {
+ ioctx = sdev->ioctx_ring[wc->wr_id & ~SRPT_OP_RECV];
+ srpt_handle_new_iu(ch, ioctx, context);
+ } else {
+ PRINT_INFO("receiving wr_id %u failed with status %d",
+ (unsigned)(wc->wr_id & ~SRPT_OP_RECV), wc->status);
+ }
+}
+
+static void srpt_process_send_completion(struct ib_cq *cq,
+ struct srpt_rdma_ch *ch,
+ enum scst_exec_context context,
+ struct ib_wc *wc)
+{
+ struct srpt_device *sdev = ch->sport->sdev;
+ struct srpt_ioctx *ioctx;
+
+ EXTRACHECKS_WARN_ON((wc->wr_id & SRPT_OP_RECV) != 0);
+
+ if (wc->status == IB_WC_SUCCESS) {
+ if ((wc->wr_id & SRPT_OP_TTI) == 0) {
+ ioctx = sdev->ioctx_ring[wc->wr_id];
+ if (wc->opcode == IB_WC_SEND)
+ srpt_handle_send_comp(ch, ioctx, context);
+ else {
+ EXTRACHECKS_WARN_ON(wc->opcode
+ != IB_WC_RDMA_READ);
+ srpt_handle_rdma_comp(ch, ioctx, context);
+ }
+ } else
+ srpt_put_tti_ioctx(ch);
+ } else {
+ PRINT_INFO("sending %s for wr_id %u failed with status %d",
+ wc->wr_id & SRPT_OP_TTI ? "request" : "response",
+ (unsigned)(wc->wr_id & ~SRPT_OP_FLAGS), wc->status);
+ srpt_handle_send_err_comp(ch, wc->wr_id, context);
+ }
+}
+
+static void srpt_process_completion(struct ib_cq *cq,
+ struct srpt_rdma_ch *ch,
+ enum scst_exec_context context)
+{
+ struct ib_wc wc[16];
+ int i, n;
+
+ EXTRACHECKS_WARN_ON(cq != ch->cq);
+
+ do {
+ while ((n = ib_poll_cq(cq, ARRAY_SIZE(wc), wc)) > 0) {
+ for (i = 0; i < n; i++) {
+ if (wc[i].wr_id & SRPT_OP_RECV)
+ srpt_process_rcv_completion(cq, ch,
+ context,
+ &wc[i]);
+ else
+ srpt_process_send_completion(cq, ch,
+ context,
+ &wc[i]);
+ }
+ }
+ } while (ib_req_notify_cq(cq, IB_CQ_NEXT_COMP
+ | IB_CQ_REPORT_MISSED_EVENTS) > 0);
+}
+
+/**
+ * srpt_completion() - IB completion queue callback function.
+ *
+ * Notes:
+ * - It is guaranteed that a completion handler will never be invoked
+ * concurrently on two different CPUs for the same completion queue. See also
+ * Documentation/infiniband/core_locking.txt and the implementation of
+ * handle_edge_irq() in kernel/irq/chip.c.
+ * - When threaded IRQs are enabled, completion handlers are invoked in thread
+ * context instead of interrupt context.
+ */
+static void srpt_completion(struct ib_cq *cq, void *ctx)
+{
+ struct srpt_rdma_ch *ch = ctx;
+
+ atomic_inc(&ch->processing_compl);
+ switch (thread) {
+ case MODE_IB_COMPLETION_IN_THREAD:
+ wake_up_interruptible(&ch->wait_queue);
+ break;
+ case MODE_IB_COMPLETION_IN_SIRQ:
+ srpt_process_completion(cq, ch, SCST_CONTEXT_THREAD);
+ break;
+ case MODE_ALL_IN_SIRQ:
+ srpt_process_completion(cq, ch, SCST_CONTEXT_TASKLET);
+ break;
+ }
+ atomic_dec(&ch->processing_compl);
+}
+
+static int srpt_compl_thread(void *arg)
+{
+ struct srpt_rdma_ch *ch;
+
+ /* Hibernation / freezing of the SRPT kernel thread is not supported. */
+ current->flags |= PF_NOFREEZE;
+
+ ch = arg;
+ BUG_ON(!ch);
+ PRINT_INFO("Session %s: kernel thread %s (PID %d) started",
+ ch->sess_name, ch->thread->comm, current->pid);
+ while (!kthread_should_stop()) {
+ wait_event_interruptible(ch->wait_queue,
+ (srpt_process_completion(ch->cq, ch,
+ SCST_CONTEXT_THREAD),
+ kthread_should_stop()));
+ }
+ PRINT_INFO("Session %s: kernel thread %s (PID %d) stopped",
+ ch->sess_name, ch->thread->comm, current->pid);
+ return 0;
+}
+
+/**
+ * srpt_create_ch_ib() - Create receive and send completion queues.
+ */
+static int srpt_create_ch_ib(struct srpt_rdma_ch *ch)
+{
+ struct ib_qp_init_attr *qp_init;
+ struct srpt_device *sdev = ch->sport->sdev;
+ int ret;
+
+ EXTRACHECKS_WARN_ON(ch->rq_size < 1);
+
+ ret = -ENOMEM;
+ qp_init = kzalloc(sizeof *qp_init, GFP_KERNEL);
+ if (!qp_init)
+ goto out;
+
+ ch->cq = ib_create_cq(sdev->device, srpt_completion, NULL, ch,
+ ch->rq_size + srpt_sq_size, 0);
+ if (IS_ERR(ch->cq)) {
+ ret = PTR_ERR(ch->cq);
+ PRINT_ERROR("failed to create CQ cqe= %d ret= %d",
+ ch->rq_size + srpt_sq_size, ret);
+ goto out;
+ }
+
+ qp_init->qp_context = (void *)ch;
+ qp_init->event_handler
+ = (void(*)(struct ib_event *, void*))srpt_qp_event;
+ qp_init->send_cq = ch->cq;
+ qp_init->recv_cq = ch->cq;
+ qp_init->srq = sdev->srq;
+ qp_init->sq_sig_type = IB_SIGNAL_REQ_WR;
+ qp_init->qp_type = IB_QPT_RC;
+ qp_init->cap.max_send_wr = srpt_sq_size;
+ qp_init->cap.max_send_sge = SRPT_DEF_SG_PER_WQE;
+
+ ch->qp = ib_create_qp(sdev->pd, qp_init);
+ if (IS_ERR(ch->qp)) {
+ ret = PTR_ERR(ch->qp);
+ PRINT_ERROR("failed to create_qp ret= %d", ret);
+ goto err_destroy_cq;
+ }
+
+ atomic_set(&ch->sq_wr_avail, qp_init->cap.max_send_wr);
+
+ TRACE_DBG("%s: max_cqe= %d max_sge= %d sq_size = %d"
+ " cm_id= %p", __func__, ch->cq->cqe,
+ qp_init->cap.max_send_sge, qp_init->cap.max_send_wr,
+ ch->cm_id);
+
+ ret = srpt_init_ch_qp(ch, ch->qp);
+ if (ret)
+ goto err_destroy_qp;
+
+ if (thread == MODE_IB_COMPLETION_IN_THREAD) {
+ init_waitqueue_head(&ch->wait_queue);
+
+ TRACE_DBG("creating IB completion thread for session %s",
+ ch->sess_name);
+
+ ch->thread = kthread_run(srpt_compl_thread, ch,
+ "ib_srpt_compl");
+ if (IS_ERR(ch->thread)) {
+ PRINT_ERROR("failed to create kernel thread %ld",
+ PTR_ERR(ch->thread));
+ ch->thread = NULL;
+ goto err_destroy_qp;
+ }
+ } else
+ ib_req_notify_cq(ch->cq, IB_CQ_NEXT_COMP);
+
+out:
+ kfree(qp_init);
+ return ret;
+
+err_destroy_qp:
+ ib_destroy_qp(ch->qp);
+err_destroy_cq:
+ ib_destroy_cq(ch->cq);
+ goto out;
+}
+
+static void srpt_destroy_ch_ib(struct srpt_rdma_ch *ch)
+{
+ struct ib_qp_attr qp_attr;
+ int ret;
+
+ if (ch->thread)
+ kthread_stop(ch->thread);
+
+ qp_attr.qp_state = IB_QPS_RESET;
+ ret = ib_modify_qp(ch->qp, &qp_attr, IB_QP_STATE);
+ if (ret < 0)
+ PRINT_ERROR("Resetting queue pair state failed: %d", ret);
+
+ while (atomic_read(&ch->processing_compl))
+ ;
+
+ ib_destroy_qp(ch->qp);
+ ib_destroy_cq(ch->cq);
+}
+
+/**
+ * srpt_unregister_channel() - Start RDMA channel disconnection.
+ *
+ * Note: The caller must hold ch->sdev->spinlock.
+ */
+static void srpt_unregister_channel(struct srpt_rdma_ch *ch)
+ __acquires(&ch->sport->sdev->spinlock)
+ __releases(&ch->sport->sdev->spinlock)
+{
+ struct srpt_device *sdev;
+
+ sdev = ch->sport->sdev;
+ list_del(&ch->list);
+ atomic_set(&ch->state, RDMA_CHANNEL_DISCONNECTING);
+ spin_unlock_irq(&sdev->spinlock);
+
+ /*
+ * At this point it is guaranteed that no new commands will be sent to
+ * the SCST core for channel ch, which is a requirement for
+ * scst_unregister_session().
+ */
+
+ TRACE_DBG("unregistering session %p", ch->scst_sess);
+ scst_unregister_session(ch->scst_sess, 0, srpt_release_channel);
+ spin_lock_irq(&sdev->spinlock);
+}
+
+/**
+ * srpt_release_channel_by_cmid() - Release a channel.
+ * @cm_id: Pointer to the CM ID of the channel to be released.
+ *
+ * Note: Must be called from inside srpt_cm_handler to avoid a race between
+ * accessing sdev->spinlock and the call to kfree(sdev) in srpt_remove_one()
+ * (the caller of srpt_cm_handler holds the cm_id spinlock; srpt_remove_one()
+ * waits until all SCST sessions for the associated IB device have been
+ * unregistered and SCST session registration involves a call to
+ * ib_destroy_cm_id(), which locks the cm_id spinlock and hence waits until
+ * this function has finished).
+ */
+static void srpt_release_channel_by_cmid(struct ib_cm_id *cm_id)
+{
+ struct srpt_device *sdev;
+ struct srpt_rdma_ch *ch;
+
+ EXTRACHECKS_WARN_ON_ONCE(irqs_disabled());
+
+ sdev = cm_id->context;
+ BUG_ON(!sdev);
+ spin_lock_irq(&sdev->spinlock);
+ list_for_each_entry(ch, &sdev->rch_list, list) {
+ if (ch->cm_id == cm_id) {
+ srpt_unregister_channel(ch);
+ break;
+ }
+ }
+ spin_unlock_irq(&sdev->spinlock);
+}
+
+/**
+ * srpt_find_channel() - Look up an RDMA channel.
+ * @cm_id: Pointer to the CM ID of the channel to be looked up.
+ *
+ * Return NULL if no matching RDMA channel has been found.
+ */
+static struct srpt_rdma_ch *srpt_find_channel(struct srpt_device *sdev,
+ struct ib_cm_id *cm_id)
+{
+ struct srpt_rdma_ch *ch;
+ bool found;
+
+ EXTRACHECKS_WARN_ON_ONCE(irqs_disabled());
+ BUG_ON(!sdev);
+
+ found = false;
+ spin_lock_irq(&sdev->spinlock);
+ list_for_each_entry(ch, &sdev->rch_list, list) {
+ if (ch->cm_id == cm_id) {
+ found = true;
+ break;
+ }
+ }
+ spin_unlock_irq(&sdev->spinlock);
+
+ return found ? ch : NULL;
+}
+
+/**
+ * srpt_release_channel() - Release all resources associated with an RDMA channel.
+ *
+ * Notes:
+ * - The caller must have removed the channel from the channel list before
+ * calling this function.
+ * - Must be called as a callback function via scst_unregister_session(). Never
+ * call this function directly because doing so would trigger several race
+ * conditions.
+ * - Do not access ch->sport or ch->sport->sdev in this function because the
+ * memory that was allocated for the sport and/or sdev data structures may
+ * already have been freed at the time this function is called.
+ */
+static void srpt_release_channel(struct scst_session *scst_sess)
+{
+ struct srpt_rdma_ch *ch;
+
+ ch = scst_sess_get_tgt_priv(scst_sess);
+ BUG_ON(!ch);
+ WARN_ON(atomic_read(&ch->state) != RDMA_CHANNEL_DISCONNECTING);
+
+ TRACE_DBG("destroying cm_id %p", ch->cm_id);
+ BUG_ON(!ch->cm_id);
+ ib_destroy_cm_id(ch->cm_id);
+
+ srpt_destroy_ch_ib(ch);
+ srpt_free_tti_ioctx(ch);
+
+ kfree(ch);
+}
+
+/**
+ * srpt_enable_target() - Allows to enable a target via sysfs.
+ */
+static int srpt_enable_target(struct scst_tgt *scst_tgt, bool enable)
+{
+ struct srpt_device *sdev = scst_tgt_get_tgt_priv(scst_tgt);
+
+ EXTRACHECKS_WARN_ON_ONCE(irqs_disabled());
+
+ TRACE_DBG("%s target %s", enable ? "Enabling" : "Disabling",
+ sdev->device->name);
+
+ spin_lock_irq(&sdev->spinlock);
+ sdev->enabled = enable;
+ spin_unlock_irq(&sdev->spinlock);
+
+ return 0;
+}
+
+/**
+ * srpt_is_target_enabled() - Allows to query a targets status via sysfs.
+ */
+static bool srpt_is_target_enabled(struct scst_tgt *scst_tgt)
+{
+ struct srpt_device *sdev = scst_tgt_get_tgt_priv(scst_tgt);
+ bool res;
+
+ EXTRACHECKS_WARN_ON_ONCE(irqs_disabled());
+
+ spin_lock_irq(&sdev->spinlock);
+ res = sdev->enabled;
+ spin_unlock_irq(&sdev->spinlock);
+ return res;
+}
+
+/**
+ * srpt_cm_req_recv() - Process the event IB_CM_REQ_RECEIVED.
+ *
+ * Ownership of the cm_id is transferred to the SCST session if this functions
+ * returns zero. Otherwise the caller remains the owner of cm_id.
+ */
+static int srpt_cm_req_recv(struct ib_cm_id *cm_id,
+ struct ib_cm_req_event_param *param,
+ void *private_data)
+{
+ struct srpt_device *sdev = cm_id->context;
+ struct srp_login_req *req;
+ struct srp_login_rsp *rsp;
+ struct srp_login_rej *rej;
+ struct ib_cm_rep_param *rep_param;
+ struct srpt_rdma_ch *ch, *tmp_ch;
+ u32 it_iu_len;
+ int ret = 0;
+
+ EXTRACHECKS_WARN_ON_ONCE(irqs_disabled());
+
+ if (WARN_ON(!sdev || !private_data))
+ return -EINVAL;
+
+ req = (struct srp_login_req *)private_data;
+
+ it_iu_len = be32_to_cpu(req->req_it_iu_len);
+
+ PRINT_INFO("Received SRP_LOGIN_REQ with"
+ " i_port_id 0x%llx:0x%llx, t_port_id 0x%llx:0x%llx and it_iu_len %d"
+ " on port %d (guid=0x%llx:0x%llx)",
+ be64_to_cpu(*(__be64 *)&req->initiator_port_id[0]),
+ be64_to_cpu(*(__be64 *)&req->initiator_port_id[8]),
+ be64_to_cpu(*(__be64 *)&req->target_port_id[0]),
+ be64_to_cpu(*(__be64 *)&req->target_port_id[8]),
+ it_iu_len,
+ param->port,
+ be64_to_cpu(*(__be64 *)&sdev->port[param->port - 1].gid.raw[0]),
+ be64_to_cpu(*(__be64 *)&sdev->port[param->port - 1].gid.raw[8]));
+
+ rsp = kzalloc(sizeof *rsp, GFP_KERNEL);
+ rej = kzalloc(sizeof *rej, GFP_KERNEL);
+ rep_param = kzalloc(sizeof *rep_param, GFP_KERNEL);
+
+ if (!rsp || !rej || !rep_param) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ if (it_iu_len > srp_max_message_size || it_iu_len < 64) {
+ rej->reason =
+ cpu_to_be32(SRP_LOGIN_REJ_REQ_IT_IU_LENGTH_TOO_LARGE);
+ ret = -EINVAL;
+ PRINT_ERROR("rejected SRP_LOGIN_REQ because its"
+ " length (%d bytes) is out of range (%d .. %d)",
+ it_iu_len, 64, srp_max_message_size);
+ goto reject;
+ }
+
+ if (!srpt_is_target_enabled(sdev->scst_tgt)) {
+ rej->reason =
+ cpu_to_be32(SRP_LOGIN_REJ_INSUFFICIENT_RESOURCES);
+ ret = -EINVAL;
+ PRINT_ERROR("rejected SRP_LOGIN_REQ because the target %s"
+ " has not yet been enabled", sdev->device->name);
+ goto reject;
+ }
+
+ if ((req->req_flags & SRP_MTCH_ACTION) == SRP_MULTICHAN_SINGLE) {
+ rsp->rsp_flags = SRP_LOGIN_RSP_MULTICHAN_NO_CHAN;
+
+ spin_lock_irq(&sdev->spinlock);
+
+ list_for_each_entry_safe(ch, tmp_ch, &sdev->rch_list, list) {
+ if (!memcmp(ch->i_port_id, req->initiator_port_id, 16)
+ && !memcmp(ch->t_port_id, req->target_port_id, 16)
+ && param->port == ch->sport->port
+ && param->listen_id == ch->sport->sdev->cm_id
+ && ch->cm_id) {
+ enum rdma_ch_state prev_state;
+
+ /* found an existing channel */
+ TRACE_DBG("Found existing channel name= %s"
+ " cm_id= %p state= %d",
+ ch->sess_name, ch->cm_id,
+ atomic_read(&ch->state));
+
+ prev_state = atomic_xchg(&ch->state,
+ RDMA_CHANNEL_DISCONNECTING);
+ if (prev_state == RDMA_CHANNEL_CONNECTING)
+ srpt_unregister_channel(ch);
+
+ spin_unlock_irq(&sdev->spinlock);
+
+ rsp->rsp_flags =
+ SRP_LOGIN_RSP_MULTICHAN_TERMINATED;
+
+ if (prev_state == RDMA_CHANNEL_LIVE) {
+ ib_send_cm_dreq(ch->cm_id, NULL, 0);
+ PRINT_INFO("disconnected"
+ " session %s because a new"
+ " SRP_LOGIN_REQ has been received.",
+ ch->sess_name);
+ } else if (prev_state ==
+ RDMA_CHANNEL_CONNECTING) {
+ PRINT_ERROR("%s", "rejected"
+ " SRP_LOGIN_REQ because another login"
+ " request is being processed.");
+ ib_send_cm_rej(ch->cm_id,
+ IB_CM_REJ_NO_RESOURCES,
+ NULL, 0, NULL, 0);
+ }
+
+ spin_lock_irq(&sdev->spinlock);
+ }
+ }
+
+ spin_unlock_irq(&sdev->spinlock);
+
+ } else
+ rsp->rsp_flags = SRP_LOGIN_RSP_MULTICHAN_MAINTAINED;
+
+ if (*(__be64 *)req->target_port_id != cpu_to_be64(srpt_service_guid)
+ || *(__be64 *)(req->target_port_id + 8) !=
+ cpu_to_be64(srpt_service_guid)) {
+ rej->reason =
+ cpu_to_be32(SRP_LOGIN_REJ_UNABLE_ASSOCIATE_CHANNEL);
+ ret = -ENOMEM;
+ PRINT_ERROR("%s", "rejected SRP_LOGIN_REQ because it"
+ " has an invalid target port identifier.");
+ goto reject;
+ }
+
+ ch = kzalloc(sizeof *ch, GFP_KERNEL);
+ if (!ch) {
+ rej->reason = cpu_to_be32(SRP_LOGIN_REJ_INSUFFICIENT_RESOURCES);
+ PRINT_ERROR("%s",
+ "rejected SRP_LOGIN_REQ because out of memory.");
+ ret = -ENOMEM;
+ goto reject;
+ }
+
+ memcpy(ch->i_port_id, req->initiator_port_id, 16);
+ memcpy(ch->t_port_id, req->target_port_id, 16);
+ ch->sport = &sdev->port[param->port - 1];
+ ch->cm_id = cm_id;
+ ch->rq_size = max(SRPT_RQ_SIZE, scst_get_max_lun_commands(NULL, 0));
+ atomic_set(&ch->processing_compl, 0);
+ atomic_set(&ch->state, RDMA_CHANNEL_CONNECTING);
+ INIT_LIST_HEAD(&ch->cmd_wait_list);
+
+ ch->tti_head = 0;
+ ch->tti_tail = 0;
+ ret = srpt_alloc_tti_ioctx(ch);
+ if (ret) {
+ PRINT_ERROR("%s", "send ring allocation failed");
+ goto free_ch;
+ }
+
+ ret = srpt_create_ch_ib(ch);
+ if (ret) {
+ rej->reason = cpu_to_be32(SRP_LOGIN_REJ_INSUFFICIENT_RESOURCES);
+ PRINT_ERROR("%s", "rejected SRP_LOGIN_REQ because creating"
+ " a new RDMA channel failed.");
+ goto free_req_ring;
+ }
+
+ ret = srpt_ch_qp_rtr(ch, ch->qp);
+ if (ret) {
+ rej->reason = cpu_to_be32(SRP_LOGIN_REJ_INSUFFICIENT_RESOURCES);
+ PRINT_ERROR("rejected SRP_LOGIN_REQ because enabling"
+ " RTR failed (error code = %d)", ret);
+ goto destroy_ib;
+ }
+
+ if (use_port_guid_in_session_name) {
+ /*
+ * If the kernel module parameter use_port_guid_in_session_name
+ * has been specified, use a combination of the target port
+ * GUID and the initiator port ID as the session name. This
+ * was the original behavior of the SRP target implementation
+ * (i.e. before the SRPT was included in OFED 1.3).
+ */
+ snprintf(ch->sess_name, sizeof(ch->sess_name),
+ "0x%016llx%016llx",
+ be64_to_cpu(*(__be64 *)
+ &sdev->port[param->port - 1].gid.raw[8]),
+ be64_to_cpu(*(__be64 *)(ch->i_port_id + 8)));
+ } else {
+ /*
+ * Default behavior: use the initator port identifier as the
+ * session name.
+ */
+ snprintf(ch->sess_name, sizeof(ch->sess_name),
+ "0x%016llx%016llx",
+ be64_to_cpu(*(__be64 *)ch->i_port_id),
+ be64_to_cpu(*(__be64 *)(ch->i_port_id + 8)));
+ }
+
+ TRACE_DBG("registering session %s", ch->sess_name);
+
+ BUG_ON(!sdev->scst_tgt);
+ ch->scst_sess = scst_register_session(sdev->scst_tgt, 0, ch->sess_name,
+ ch, NULL, NULL);
+ if (!ch->scst_sess) {
+ rej->reason = cpu_to_be32(SRP_LOGIN_REJ_INSUFFICIENT_RESOURCES);
+ TRACE_DBG("%s", "Failed to create SCST session");
+ goto release_channel;
+ }
+
+ TRACE_DBG("Establish connection sess=%p name=%s cm_id=%p",
+ ch->scst_sess, ch->sess_name, ch->cm_id);
+
+ /* create srp_login_response */
+ rsp->opcode = SRP_LOGIN_RSP;
+ rsp->tag = req->tag;
+ rsp->max_it_iu_len = req->req_it_iu_len;
+ rsp->max_ti_iu_len = req->req_it_iu_len;
+ ch->max_ti_iu_len = it_iu_len;
+ atomic_set(&ch->supports_cred_req, false);
+ rsp->buf_fmt =
+ cpu_to_be16(SRP_BUF_FORMAT_DIRECT | SRP_BUF_FORMAT_INDIRECT);
+ rsp->req_lim_delta = cpu_to_be32(ch->rq_size);
+ atomic_set(&ch->req_lim, ch->rq_size);
+ atomic_set(&ch->req_lim_delta, 0);
+ atomic_set(&ch->req_lim_waiter_count, 0);
+ init_completion(&ch->req_lim_compl);
+
+ /* create cm reply */
+ rep_param->qp_num = ch->qp->qp_num;
+ rep_param->private_data = (void *)rsp;
+ rep_param->private_data_len = sizeof *rsp;
+ rep_param->rnr_retry_count = 7;
+ rep_param->flow_control = 1;
+ rep_param->failover_accepted = 0;
+ rep_param->srq = 1;
+ rep_param->responder_resources = 4;
+ rep_param->initiator_depth = 4;
+
+ ret = ib_send_cm_rep(cm_id, rep_param);
+ if (ret) {
+ PRINT_ERROR("sending SRP_LOGIN_REQ response failed"
+ " (error code = %d)", ret);
+ goto release_channel;
+ }
+
+ spin_lock_irq(&sdev->spinlock);
+ list_add_tail(&ch->list, &sdev->rch_list);
+ spin_unlock_irq(&sdev->spinlock);
+
+ goto out;
+
+release_channel:
+ atomic_set(&ch->state, RDMA_CHANNEL_DISCONNECTING);
+ scst_unregister_session(ch->scst_sess, 0, NULL);
+ ch->scst_sess = NULL;
+
+destroy_ib:
+ srpt_destroy_ch_ib(ch);
+
+free_req_ring:
+ srpt_free_tti_ioctx(ch);
+
+free_ch:
+ kfree(ch);
+
+reject:
+ rej->opcode = SRP_LOGIN_REJ;
+ rej->tag = req->tag;
+ rej->buf_fmt =
+ cpu_to_be16(SRP_BUF_FORMAT_DIRECT | SRP_BUF_FORMAT_INDIRECT);
+
+ ib_send_cm_rej(cm_id, IB_CM_REJ_CONSUMER_DEFINED, NULL, 0,
+ (void *)rej, sizeof *rej);
+
+out:
+ kfree(rep_param);
+ kfree(rsp);
+ kfree(rej);
+
+ return ret;
+}
+
+static void srpt_cm_rej_recv(struct ib_cm_id *cm_id)
+{
+ PRINT_INFO("Received InfiniBand REJ packet for cm_id %p.", cm_id);
+ srpt_release_channel_by_cmid(cm_id);
+}
+
+/**
+ * srpt_cm_rtu_recv() - Process an IB_CM_RTU_RECEIVED or IB_CM_USER_ESTABLISHED event.
+ *
+ * An IB_CM_RTU_RECEIVED message indicates that the connection is established
+ * and that the recipient may begin transmitting (RTU = ready to use).
+ */
+static void srpt_cm_rtu_recv(struct ib_cm_id *cm_id)
+{
+ struct srpt_rdma_ch *ch;
+ int ret;
+
+ ch = srpt_find_channel(cm_id->context, cm_id);
+ WARN_ON(!ch);
+ if (!ch)
+ goto out;
+
+ if (srpt_test_and_set_channel_state(ch, RDMA_CHANNEL_CONNECTING,
+ RDMA_CHANNEL_LIVE) == RDMA_CHANNEL_CONNECTING) {
+ struct srpt_ioctx *ioctx, *ioctx_tmp;
+
+ ret = srpt_ch_qp_rts(ch, ch->qp);
+
+ if (srpt_autodetect_cred_req)
+ srpt_send_cred_req(ch, 0);
+
+ list_for_each_entry_safe(ioctx, ioctx_tmp, &ch->cmd_wait_list,
+ wait_list) {
+ list_del(&ioctx->wait_list);
+ srpt_handle_new_iu(ch, ioctx, SCST_CONTEXT_THREAD);
+ }
+ if (ret && srpt_test_and_set_channel_state(ch,
+ RDMA_CHANNEL_LIVE,
+ RDMA_CHANNEL_DISCONNECTING) == RDMA_CHANNEL_LIVE) {
+ TRACE_DBG("cm_id=%p sess_name=%s state=%d",
+ cm_id, ch->sess_name,
+ atomic_read(&ch->state));
+ ib_send_cm_dreq(ch->cm_id, NULL, 0);
+ }
+ }
+
+out:
+ ;
+}
+
+static void srpt_cm_timewait_exit(struct ib_cm_id *cm_id)
+{
+ PRINT_INFO("Received InfiniBand TimeWait exit for cm_id %p.", cm_id);
+ srpt_release_channel_by_cmid(cm_id);
+}
+
+static void srpt_cm_rep_error(struct ib_cm_id *cm_id)
+{
+ PRINT_INFO("Received InfiniBand REP error for cm_id %p.", cm_id);
+ srpt_release_channel_by_cmid(cm_id);
+}
+
+/**
+ * srpt_cm_dreq_recv() - Process reception of a DREQ message.
+ */
+static void srpt_cm_dreq_recv(struct ib_cm_id *cm_id)
+{
+ struct srpt_rdma_ch *ch;
+
+ ch = srpt_find_channel(cm_id->context, cm_id);
+ if (!ch) {
+ TRACE_DBG("Received DREQ for channel %p which is already"
+ " being unregistered.", cm_id);
+ goto out;
+ }
+
+ TRACE_DBG("cm_id= %p ch->state= %d", cm_id, atomic_read(&ch->state));
+
+ switch (atomic_read(&ch->state)) {
+ case RDMA_CHANNEL_LIVE:
+ case RDMA_CHANNEL_CONNECTING:
+ ib_send_cm_drep(ch->cm_id, NULL, 0);
+ PRINT_INFO("Received DREQ and sent DREP for session %s.",
+ ch->sess_name);
+ break;
+ case RDMA_CHANNEL_DISCONNECTING:
+ default:
+ break;
+ }
+
+out:
+ ;
+}
+
+/**
+ * srpt_cm_drep_recv() - Process reception of a DREP message.
+ */
+static void srpt_cm_drep_recv(struct ib_cm_id *cm_id)
+{
+ PRINT_INFO("Received InfiniBand DREP message for cm_id %p.", cm_id);
+ srpt_release_channel_by_cmid(cm_id);
+}
+
+/**
+ * srpt_cm_handler() - IB connection manager callback function.
+ *
+ * A non-zero return value will cause the caller destroy the CM ID.
+ *
+ * Note: srpt_cm_handler() must only return a non-zero value when transferring
+ * ownership of the cm_id to a channel by srpt_cm_req_recv() failed. Returning
+ * a non-zero value in any other case will trigger a race with the
+ * ib_destroy_cm_id() call in srpt_release_channel().
+ */
+static int srpt_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
+{
+ int ret;
+
+ ret = 0;
+ switch (event->event) {
+ case IB_CM_REQ_RECEIVED:
+ ret = srpt_cm_req_recv(cm_id, &event->param.req_rcvd,
+ event->private_data);
+ break;
+ case IB_CM_REJ_RECEIVED:
+ srpt_cm_rej_recv(cm_id);
+ break;
+ case IB_CM_RTU_RECEIVED:
+ case IB_CM_USER_ESTABLISHED:
+ srpt_cm_rtu_recv(cm_id);
+ break;
+ case IB_CM_DREQ_RECEIVED:
+ srpt_cm_dreq_recv(cm_id);
+ break;
+ case IB_CM_DREP_RECEIVED:
+ srpt_cm_drep_recv(cm_id);
+ break;
+ case IB_CM_TIMEWAIT_EXIT:
+ srpt_cm_timewait_exit(cm_id);
+ break;
+ case IB_CM_REP_ERROR:
+ srpt_cm_rep_error(cm_id);
+ break;
+ case IB_CM_DREQ_ERROR:
+ PRINT_INFO("%s", "Received IB DREQ ERROR event.");
+ break;
+ case IB_CM_MRA_RECEIVED:
+ PRINT_INFO("%s", "Received IB MRA event");
+ break;
+ default:
+ PRINT_ERROR("received unrecognized IB CM event %d",
+ event->event);
+ break;
+ }
+
+ return ret;
+}
+
+/**
+ * srpt_map_sg_to_ib_sge() - Map an SG list to an IB SGE list.
+ */
+static int srpt_map_sg_to_ib_sge(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx,
+ struct scst_cmd *scmnd)
+{
+ struct scatterlist *scat;
+ scst_data_direction dir;
+ struct rdma_iu *riu;
+ struct srp_direct_buf *db;
+ dma_addr_t dma_addr;
+ struct ib_sge *sge;
+ u64 raddr;
+ u32 rsize;
+ u32 tsize;
+ u32 dma_len;
+ int count, nrdma;
+ int i, j, k;
+
+ BUG_ON(!ch);
+ BUG_ON(!ioctx);
+ BUG_ON(!scmnd);
+ scat = scst_cmd_get_sg(scmnd);
+ WARN_ON(!scat);
+ dir = scst_cmd_get_data_direction(scmnd);
+ BUG_ON(dir == SCST_DATA_NONE);
+ /*
+ * Cache 'dir' because it is needed in srpt_unmap_sg_to_ib_sge()
+ * and because scst_set_cmd_error_status() resets scmnd->data_direction.
+ */
+ ioctx->dir = dir;
+ count = ib_dma_map_sg(ch->sport->sdev->device, scat,
+ scst_cmd_get_sg_cnt(scmnd),
+ scst_to_tgt_dma_dir(dir));
+ if (unlikely(!count))
+ return -EBUSY;
+
+ ioctx->mapped_sg_count = count;
+
+ if (ioctx->rdma_ius && ioctx->n_rdma_ius)
+ nrdma = ioctx->n_rdma_ius;
+ else {
+ nrdma = count / SRPT_DEF_SG_PER_WQE + ioctx->n_rbuf;
+
+ ioctx->rdma_ius = kzalloc(nrdma * sizeof *riu,
+ scst_cmd_atomic(scmnd)
+ ? GFP_ATOMIC : GFP_KERNEL);
+ if (!ioctx->rdma_ius)
+ goto free_mem;
+
+ ioctx->n_rdma_ius = nrdma;
+ }
+
+ db = ioctx->rbufs;
+ tsize = (dir == SCST_DATA_READ) ?
+ scst_cmd_get_resp_data_len(scmnd) : scst_cmd_get_bufflen(scmnd);
+ dma_len = sg_dma_len(&scat[0]);
+ riu = ioctx->rdma_ius;
+
+ /*
+ * For each remote desc - calculate the #ib_sge.
+ * If #ib_sge < SRPT_DEF_SG_PER_WQE per rdma operation then
+ * each remote desc rdma_iu is required a rdma wr;
+ * else
+ * we need to allocate extra rdma_iu to carry extra #ib_sge in
+ * another rdma wr
+ */
+ for (i = 0, j = 0;
+ j < count && i < ioctx->n_rbuf && tsize > 0; ++i, ++riu, ++db) {
+ rsize = be32_to_cpu(db->len);
+ raddr = be64_to_cpu(db->va);
+ riu->raddr = raddr;
+ riu->rkey = be32_to_cpu(db->key);
+ riu->sge_cnt = 0;
+
+ /* calculate how many sge required for this remote_buf */
+ while (rsize > 0 && tsize > 0) {
+
+ if (rsize >= dma_len) {
+ tsize -= dma_len;
+ rsize -= dma_len;
+ raddr += dma_len;
+
+ if (tsize > 0) {
+ ++j;
+ if (j < count)
+ dma_len = sg_dma_len(&scat[j]);
+ }
+ } else {
+ tsize -= rsize;
+ dma_len -= rsize;
+ rsize = 0;
+ }
+
+ ++riu->sge_cnt;
+
+ if (rsize > 0 && riu->sge_cnt == SRPT_DEF_SG_PER_WQE) {
+ ++ioctx->n_rdma;
+ riu->sge =
+ kmalloc(riu->sge_cnt * sizeof *riu->sge,
+ scst_cmd_atomic(scmnd)
+ ? GFP_ATOMIC : GFP_KERNEL);
+ if (!riu->sge)
+ goto free_mem;
+
+ ++riu;
+ riu->sge_cnt = 0;
+ riu->raddr = raddr;
+ riu->rkey = be32_to_cpu(db->key);
+ }
+ }
+
+ ++ioctx->n_rdma;
+ riu->sge = kmalloc(riu->sge_cnt * sizeof *riu->sge,
+ scst_cmd_atomic(scmnd)
+ ? GFP_ATOMIC : GFP_KERNEL);
+ if (!riu->sge)
+ goto free_mem;
+ }
+
+ db = ioctx->rbufs;
+ scat = scst_cmd_get_sg(scmnd);
+ tsize = (dir == SCST_DATA_READ) ?
+ scst_cmd_get_resp_data_len(scmnd) : scst_cmd_get_bufflen(scmnd);
+ riu = ioctx->rdma_ius;
+ dma_len = sg_dma_len(&scat[0]);
+ dma_addr = sg_dma_address(&scat[0]);
+
+ /* this second loop is really mapped sg_addres to rdma_iu->ib_sge */
+ for (i = 0, j = 0;
+ j < count && i < ioctx->n_rbuf && tsize > 0; ++i, ++riu, ++db) {
+ rsize = be32_to_cpu(db->len);
+ sge = riu->sge;
+ k = 0;
+
+ while (rsize > 0 && tsize > 0) {
+ sge->addr = dma_addr;
+ sge->lkey = ch->sport->sdev->mr->lkey;
+
+ if (rsize >= dma_len) {
+ sge->length =
+ (tsize < dma_len) ? tsize : dma_len;
+ tsize -= dma_len;
+ rsize -= dma_len;
+
+ if (tsize > 0) {
+ ++j;
+ if (j < count) {
+ dma_len = sg_dma_len(&scat[j]);
+ dma_addr =
+ sg_dma_address(&scat[j]);
+ }
+ }
+ } else {
+ sge->length = (tsize < rsize) ? tsize : rsize;
+ tsize -= rsize;
+ dma_len -= rsize;
+ dma_addr += rsize;
+ rsize = 0;
+ }
+
+ ++k;
+ if (k == riu->sge_cnt && rsize > 0) {
+ ++riu;
+ sge = riu->sge;
+ k = 0;
+ } else if (rsize > 0)
+ ++sge;
+ }
+ }
+
+ return 0;
+
+free_mem:
+ srpt_unmap_sg_to_ib_sge(ch, ioctx);
+
+ return -ENOMEM;
+}
+
+/**
+ * srpt_unmap_sg_to_ib_sge() - Unmap an IB SGE list.
+ */
+static void srpt_unmap_sg_to_ib_sge(struct srpt_rdma_ch *ch,
+ struct srpt_ioctx *ioctx)
+{
+ struct scst_cmd *scmnd;
+ struct scatterlist *scat;
+ scst_data_direction dir;
+
+ EXTRACHECKS_BUG_ON(!ch);
+ EXTRACHECKS_BUG_ON(!ioctx);
+ EXTRACHECKS_BUG_ON(ioctx->n_rdma && !ioctx->rdma_ius);
+
+ while (ioctx->n_rdma)
+ kfree(ioctx->rdma_ius[--ioctx->n_rdma].sge);
+
+ kfree(ioctx->rdma_ius);
+ ioctx->rdma_ius = NULL;
+
+ if (ioctx->mapped_sg_count) {
+ scmnd = ioctx->scmnd;
+ EXTRACHECKS_BUG_ON(!scmnd);
+ EXTRACHECKS_WARN_ON(ioctx->scmnd != scmnd);
+ EXTRACHECKS_WARN_ON(ioctx != scst_cmd_get_tgt_priv(scmnd));
+ scat = scst_cmd_get_sg(scmnd);
+ EXTRACHECKS_WARN_ON(!scat);
+ dir = ioctx->dir;
+ EXTRACHECKS_BUG_ON(dir == SCST_DATA_NONE);
+ ib_dma_unmap_sg(ch->sport->sdev->device, scat,
+ scst_cmd_get_sg_cnt(scmnd),
+ scst_to_tgt_dma_dir(dir));
+ ioctx->mapped_sg_count = 0;
+ }
+}
+
+/**
+ * srpt_perform_rdmas() - Perform IB RDMA.
+ */
+static int srpt_perform_rdmas(struct srpt_rdma_ch *ch, struct srpt_ioctx *ioctx,
+ scst_data_direction dir)
+{
+ struct ib_send_wr wr;
+ struct ib_send_wr *bad_wr;
+ struct rdma_iu *riu;
+ int i;
+ int ret;
+ int sq_wr_avail;
+
+ if (dir == SCST_DATA_WRITE) {
+ ret = -ENOMEM;
+ sq_wr_avail = atomic_sub_return(ioctx->n_rdma,
+ &ch->sq_wr_avail);
+ if (sq_wr_avail < 0) {
+ atomic_add(ioctx->n_rdma, &ch->sq_wr_avail);
+ PRINT_ERROR("%s[%d]: send queue full",
+ __func__, __LINE__);
+ goto out;
+ }
+ }
+
+ ret = 0;
+ riu = ioctx->rdma_ius;
+ memset(&wr, 0, sizeof wr);
+
+ for (i = 0; i < ioctx->n_rdma; ++i, ++riu) {
+ wr.opcode = (dir == SCST_DATA_READ) ?
+ IB_WR_RDMA_WRITE : IB_WR_RDMA_READ;
+ wr.next = NULL;
+ wr.wr_id = ioctx->index;
+ wr.wr.rdma.remote_addr = riu->raddr;
+ wr.wr.rdma.rkey = riu->rkey;
+ wr.num_sge = riu->sge_cnt;
+ wr.sg_list = riu->sge;
+
+ /* only get completion event for the last rdma wr */
+ if (i == (ioctx->n_rdma - 1) && dir == SCST_DATA_WRITE)
+ wr.send_flags = IB_SEND_SIGNALED;
+
+ ret = ib_post_send(ch->qp, &wr, &bad_wr);
+ if (ret)
+ goto out;
+ }
+
+out:
+ return ret;
+}
+
+/**
+ * srpt_xfer_data() - Start data transfer from initiator to target.
+ *
+ * Note: Must not block.
+ */
+static int srpt_xfer_data(struct srpt_rdma_ch *ch, struct srpt_ioctx *ioctx,
+ struct scst_cmd *scmnd)
+{
+ int ret;
+
+ ret = srpt_map_sg_to_ib_sge(ch, ioctx, scmnd);
+ if (ret) {
+ PRINT_ERROR("%s[%d] ret=%d", __func__, __LINE__, ret);
+ ret = SCST_TGT_RES_QUEUE_FULL;
+ goto out;
+ }
+
+ ret = srpt_perform_rdmas(ch, ioctx, scst_cmd_get_data_direction(scmnd));
+ if (ret) {
+ if (ret == -EAGAIN || ret == -ENOMEM) {
+ PRINT_INFO("%s[%d] queue full -- ret=%d",
+ __func__, __LINE__, ret);
+ ret = SCST_TGT_RES_QUEUE_FULL;
+ } else {
+ PRINT_ERROR("%s[%d] fatal error -- ret=%d",
+ __func__, __LINE__, ret);
+ ret = SCST_TGT_RES_FATAL_ERROR;
+ }
+ goto out_unmap;
+ }
+
+ ret = SCST_TGT_RES_SUCCESS;
+
+out:
+ return ret;
+out_unmap:
+ srpt_unmap_sg_to_ib_sge(ch, ioctx);
+ goto out;
+}
+
+/**
+ * srpt_pending_cmd_timeout() - SCST command hw processing timeout callback.
+ *
+ * Called by the SCST core if no IB completion notification has been received
+ * within max_hw_pending_time seconds.
+ */
+static void srpt_pending_cmd_timeout(struct scst_cmd *scmnd)
+{
+ struct srpt_ioctx *ioctx;
+ enum srpt_command_state state;
+
+ ioctx = scst_cmd_get_tgt_priv(scmnd);
+ BUG_ON(!ioctx);
+
+ state = srpt_get_cmd_state(ioctx);
+ switch (state) {
+ case SRPT_STATE_NEW:
+ case SRPT_STATE_DATA_IN:
+ case SRPT_STATE_DONE:
+ /*
+ * srpt_pending_cmd_timeout() should never be invoked for
+ * commands in this state.
+ */
+ PRINT_ERROR("Processing SCST command %p (SRPT state %d) took"
+ " too long -- aborting", scmnd, state);
+ break;
+ case SRPT_STATE_NEED_DATA:
+ case SRPT_STATE_CMD_RSP_SENT:
+ case SRPT_STATE_MGMT_RSP_SENT:
+ default:
+ PRINT_ERROR("Command %p: IB completion for wr_id %u has not"
+ " been received in time (SRPT command state %d)",
+ scmnd, ioctx->index, state);
+ break;
+ }
+
+ srpt_abort_scst_cmd(ioctx, SCST_CONTEXT_SAME);
+}
+
+/**
+ * srpt_rdy_to_xfer() - Transfers data from initiator to target.
+ *
+ * Called by the SCST core to transfer data from the initiator to the target
+ * (SCST_DATA_WRITE). Must not block.
+ */
+static int srpt_rdy_to_xfer(struct scst_cmd *scmnd)
+{
+ struct srpt_rdma_ch *ch;
+ struct srpt_ioctx *ioctx;
+ enum srpt_command_state new_state;
+ enum rdma_ch_state ch_state;
+ int ret;
+
+ ioctx = scst_cmd_get_tgt_priv(scmnd);
+ BUG_ON(!ioctx);
+
+ new_state = srpt_set_cmd_state(ioctx, SRPT_STATE_NEED_DATA);
+ WARN_ON(new_state == SRPT_STATE_DONE);
+
+ ch = ioctx->ch;
+ WARN_ON(ch != scst_sess_get_tgt_priv(scst_cmd_get_session(scmnd)));
+ BUG_ON(!ch);
+
+ ch_state = atomic_read(&ch->state);
+ if (ch_state == RDMA_CHANNEL_DISCONNECTING) {
+ TRACE_DBG("cmd with tag %lld: channel disconnecting",
+ scst_cmd_get_tag(scmnd));
+ ret = SCST_TGT_RES_FATAL_ERROR;
+ goto out;
+ } else if (ch_state == RDMA_CHANNEL_CONNECTING) {
+ ret = SCST_TGT_RES_QUEUE_FULL;
+ goto out;
+ }
+ ret = srpt_xfer_data(ch, ioctx, scmnd);
+
+out:
+ return ret;
+}
+
+/**
+ * srpt_must_wait_for_cred() - Whether or not the target must wait with
+ * sending a response towards the initiator in order to avoid that the
+ * initiator locks up. The Linux SRP initiator locks up when
+ * initiator.req_lim <= req_lim_min (req_lim_min equals 1 for SRP_CMD and
+ * equals 0 for SRP_TSK_MGMT) and either no new SRP_RSP will be received by the
+ * initiator or none of the received SRP_RSP responses increases
+ * initiator.req_lim. One possible strategy to avoid an initiator lockup is
+ * that the target does not send an SRP_RSP that makes initiator.req_lim <=
+ * req_lim_min. While the target does not know the value of initiator.req_lim,
+ * one can deduce from the credit mechanism specified in the SRP standard that
+ * when target.req_lim == req_lim_min, initiator.req_lim must also equal
+ * req_lim_min. Hence wait with sending a response when target.req_lim <=
+ * req_lim_min if that response would not increase initiator.req_lim. The last
+ * condition is equivalent to srpt_req_lim_delta(ch) <= 0.
+ *
+ * If this function returns false, the caller must either send a response to
+ * the initiator with the REQUEST LIMIT DELTA field set to delta or call
+ * srpt_undo_req_lim_delta(ch, delta); where delta is the value written to
+ * the address that is the third argument of this function.
+ *
+ * Note: The constant 'compensation' compensates for the fact that multiple
+ * threads are processing SRP commands simultaneously.
+ *
+ * See also: For more information about how to reproduce the initiator lockup,
+ * see also http://bugzilla.kernel.org/show_bug.cgi?id=14235.
+ */
+static bool srpt_must_wait_for_cred(struct srpt_rdma_ch *ch, int req_lim_min,
+ int *req_lim_delta)
+{
+ int delta;
+ bool res;
+ int compensation;
+ enum { default_vdisk_threads = 8 };
+
+ EXTRACHECKS_BUG_ON(!req_lim_delta);
+
+ compensation = min_t(int, default_vdisk_threads, num_online_cpus()) + 1;
+ res = true;
+ if (atomic_read(&ch->supports_cred_req)
+ || atomic_read(&ch->req_lim) > req_lim_min + compensation) {
+ res = false;
+ *req_lim_delta = srpt_req_lim_delta(ch);
+ } else {
+ bool again;
+ do {
+ again = false;
+ delta = atomic_read(&ch->req_lim_delta);
+ if (delta > 0) {
+ if (atomic_cmpxchg(&ch->req_lim_delta, delta, 0)
+ == delta) {
+ res = false;
+ *req_lim_delta = delta;
+ } else
+ again = true;
+ }
+ } while (again);
+ }
+ return res;
+}
+
+/**
+ * srpt_wait_for_cred() - Wait until sending a response won't lock up the
+ * initiator.
+ *
+ * The caller must either send a response to the initiator with the REQUEST
+ * LIMIT DELTA field set to delta + 1 or call srpt_undo_req_lim_delta(ch,
+ * delta); where delta is the return value of this function.
+ */
+static int srpt_wait_for_cred(struct srpt_rdma_ch *ch, int req_lim_min)
+{
+ int delta;
+
+#if 0
+ bool debug_print = atomic_read(&ch->req_lim) <= req_lim_min + 1;
+ if (debug_print)
+ PRINT_INFO("srpt_wait_for_cred(): min %d, req_lim %d,"
+ " req_lim_delta %d", req_lim_min,
+ atomic_read(&ch->req_lim),
+ atomic_read(&ch->req_lim_delta));
+#endif
+#if defined(CONFIG_SCST_DEBUG)
+ if (processing_delay_in_us <= MAX_UDELAY_MS * 1000)
+ udelay(processing_delay_in_us);
+#endif
+ delta = 0; /* superfluous -- to keep sparse happy */
+ while (unlikely(srpt_must_wait_for_cred(ch, req_lim_min, &delta))) {
+ atomic_inc(&ch->req_lim_waiter_count);
+ wait_for_completion(&ch->req_lim_compl);
+ }
+#if 0
+ if (debug_print)
+ PRINT_INFO("srpt_wait_for_cred() returns %d", delta);
+#endif
+ return delta;
+}
+
+/**
+ * srpt_xmit_response() - Transmits the response to a SCSI command.
+ *
+ * Callback function called by the SCST core. Must not block. Must ensure that
+ * scst_tgt_cmd_done() will get invoked when returning SCST_TGT_RES_SUCCESS.
+ */
+static int srpt_xmit_response(struct scst_cmd *scmnd)
+{
+ struct srpt_rdma_ch *ch;
+ struct srpt_ioctx *ioctx;
+ enum srpt_command_state state;
+ s32 req_lim_delta;
+ int ret;
+ scst_data_direction dir;
+ int resp_len;
+
+ ret = SCST_TGT_RES_SUCCESS;
+
+ ioctx = scst_cmd_get_tgt_priv(scmnd);
+ BUG_ON(!ioctx);
+
+ ch = scst_sess_get_tgt_priv(scst_cmd_get_session(scmnd));
+ BUG_ON(!ch);
+
+ state = srpt_test_and_set_cmd_state(ioctx, SRPT_STATE_NEW,
+ SRPT_STATE_CMD_RSP_SENT);
+ if (state != SRPT_STATE_NEW) {
+ state = srpt_test_and_set_cmd_state(ioctx, SRPT_STATE_DATA_IN,
+ SRPT_STATE_CMD_RSP_SENT);
+ if (state != SRPT_STATE_DATA_IN)
+ PRINT_ERROR("Unexpected command state %d",
+ srpt_get_cmd_state(ioctx));
+ }
+
+ if (unlikely(scst_cmd_aborted(scmnd))) {
+ srpt_abort_scst_cmd(ioctx, SCST_CONTEXT_SAME);
+ goto out;
+ }
+
+ EXTRACHECKS_BUG_ON(scst_cmd_atomic(scmnd));
+
+ dir = scst_cmd_get_data_direction(scmnd);
+
+ /* For read commands, transfer the data to the initiator. */
+ if (dir == SCST_DATA_READ && scst_cmd_get_resp_data_len(scmnd)) {
+ ret = srpt_xfer_data(ch, ioctx, scmnd);
+ if (ret != SCST_TGT_RES_SUCCESS) {
+ PRINT_ERROR("%s: tag= %llu xfer_data failed",
+ __func__, scst_cmd_get_tag(scmnd));
+ goto out;
+ }
+ }
+
+ req_lim_delta = srpt_wait_for_cred(ch, 1);
+
+ resp_len = srpt_build_cmd_rsp(ch, ioctx, req_lim_delta,
+ scst_cmd_get_tag(scmnd),
+ scst_cmd_get_status(scmnd),
+ scst_cmd_get_sense_buffer(scmnd),
+ scst_cmd_get_sense_buffer_len(scmnd));
+
+ if (srpt_post_send(ch, ioctx, resp_len)) {
+ srpt_unmap_sg_to_ib_sge(ch, ioctx);
+ srpt_set_cmd_state(ioctx, state);
+ scst_set_delivery_status(scmnd, SCST_CMD_DELIVERY_FAILED);
+ PRINT_ERROR("%s[%d]: ch->state %d cmd state %d tag %llu",
+ __func__, __LINE__, atomic_read(&ch->state),
+ state, scst_cmd_get_tag(scmnd));
+ srpt_undo_req_lim_delta(ch, req_lim_delta - 1);
+ ret = SCST_TGT_RES_QUEUE_FULL;
+ }
+
+out:
+ return ret;
+}
+
+/**
+ * srpt_tsk_mgmt_done() - SCST callback function that sends back the response
+ * for a task management request.
+ *
+ * Must not block.
+ */
+static void srpt_tsk_mgmt_done(struct scst_mgmt_cmd *mcmnd)
+{
+ struct srpt_rdma_ch *ch;
+ struct srpt_mgmt_ioctx *mgmt_ioctx;
+ struct srpt_ioctx *ioctx;
+ enum srpt_command_state new_state;
+ s32 req_lim_delta;
+ int rsp_len;
+
+ mgmt_ioctx = scst_mgmt_cmd_get_tgt_priv(mcmnd);
+ BUG_ON(!mgmt_ioctx);
+
+ ch = mgmt_ioctx->ch;
+ BUG_ON(!ch);
+
+ ioctx = mgmt_ioctx->ioctx;
+ BUG_ON(!ioctx);
+
+ TRACE_DBG("%s: tsk_mgmt_done for tag= %lld status=%d",
+ __func__, mgmt_ioctx->tag, scst_mgmt_cmd_get_status(mcmnd));
+
+ WARN_ON(in_irq());
+
+ new_state = srpt_set_cmd_state(ioctx, SRPT_STATE_MGMT_RSP_SENT);
+ WARN_ON(new_state == SRPT_STATE_DONE);
+
+ req_lim_delta = srpt_wait_for_cred(ch, 0);
+
+ rsp_len = srpt_build_tskmgmt_rsp(ch, ioctx, req_lim_delta,
+ scst_to_srp_tsk_mgmt_status(
+ scst_mgmt_cmd_get_status(mcmnd)),
+ mgmt_ioctx->tag);
+ /*
+ * Note: the srpt_post_send() call below sends the task management
+ * response asynchronously. It is possible that the SCST core has
+ * already freed the struct scst_mgmt_cmd structure before the
+ * response is sent. This is fine.
+ */
+ if (srpt_post_send(ch, ioctx, rsp_len)) {
+ PRINT_ERROR("%s", "Sending SRP_RSP response failed.");
+ srpt_undo_req_lim_delta(ch, req_lim_delta - 1);
+ }
+
+ scst_mgmt_cmd_set_tgt_priv(mcmnd, NULL);
+
+ kfree(mgmt_ioctx);
+}
+
+/**
+ * srpt_get_initiator_port_transport_id() - SCST TransportID callback function.
+ *
+ * See also SPC-3, section 7.5.4.5, TransportID for initiator ports using SRP.
+ */
+static int srpt_get_initiator_port_transport_id(struct scst_session *scst_sess,
+ uint8_t **transport_id)
+{
+ struct srpt_rdma_ch *ch;
+ struct spc_rdma_transport_id {
+ uint8_t protocol_identifier;
+ uint8_t reserved[7];
+ uint8_t i_port_id[16];
+ };
+ struct spc_rdma_transport_id *tr_id;
+ int res;
+
+ if (!scst_sess) {
+ res = SCSI_TRANSPORTID_PROTOCOLID_SRP;
+ goto out;
+ }
+
+ ch = scst_sess_get_tgt_priv(scst_sess);
+ BUG_ON(!ch);
+
+ BUILD_BUG_ON(sizeof(*tr_id) != 24);
+
+ tr_id = kzalloc(sizeof(struct spc_rdma_transport_id), GFP_KERNEL);
+ if (!tr_id) {
+ PRINT_ERROR("%s", "Allocation of TransportID failed");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ res = 0;
+ tr_id->protocol_identifier = SCSI_TRANSPORTID_PROTOCOLID_SRP;
+ memcpy(tr_id->i_port_id, ch->i_port_id, sizeof(ch->i_port_id));
+
+ *transport_id = (uint8_t *)tr_id;
+
+out:
+ return res;
+}
+
+/**
+ * srpt_on_free_cmd() - Free command-private data.
+ *
+ * Called by the SCST core. May be called in IRQ context.
+ */
+static void srpt_on_free_cmd(struct scst_cmd *scmnd)
+{
+ struct srpt_rdma_ch *ch;
+ struct srpt_ioctx *ioctx;
+
+ ioctx = scst_cmd_get_tgt_priv(scmnd);
+ BUG_ON(!ioctx);
+
+ WARN_ON(srpt_get_cmd_state(ioctx) != SRPT_STATE_DONE);
+
+ ch = ioctx->ch;
+ BUG_ON(!ch);
+
+ srpt_reset_ioctx(ch, ioctx, 1);
+}
+
+static void srpt_refresh_port_work(struct work_struct *work)
+{
+ struct srpt_port *sport = container_of(work, struct srpt_port, work);
+
+ srpt_refresh_port(sport);
+}
+
+/**
+ * srpt_detect() - Returns the number of target adapters.
+ *
+ * Callback function called by the SCST core.
+ */
+static int srpt_detect(struct scst_tgt_template *tp)
+{
+ int device_count;
+
+ device_count = atomic_read(&srpt_device_count);
+
+ return device_count;
+}
+
+/**
+ * srpt_release() - Free the resources associated with an SCST target.
+ *
+ * Callback function called by the SCST core from scst_unregister_target().
+ */
+static int srpt_release(struct scst_tgt *scst_tgt)
+{
+ struct srpt_device *sdev = scst_tgt_get_tgt_priv(scst_tgt);
+ struct srpt_rdma_ch *ch;
+
+ EXTRACHECKS_WARN_ON_ONCE(irqs_disabled());
+
+ BUG_ON(!scst_tgt);
+ if (WARN_ON(!sdev))
+ return -ENODEV;
+
+ spin_lock_irq(&sdev->spinlock);
+ while (!list_empty(&sdev->rch_list)) {
+ ch = list_first_entry(&sdev->rch_list, typeof(*ch), list);
+ srpt_unregister_channel(ch);
+ }
+ spin_unlock_irq(&sdev->spinlock);
+
+ scst_tgt_set_tgt_priv(scst_tgt, NULL);
+
+ return 0;
+}
+
+/**
+ * srpt_get_scsi_transport_version() - Returns the SCSI transport version.
+ * This function is called from scst_pres.c, the code that implements
+ * persistent reservation support.
+ */
+static uint16_t srpt_get_scsi_transport_version(struct scst_tgt *scst_tgt)
+{
+ return 0x0940; /* SRP */
+}
+
+/* SCST target template for the SRP target implementation. */
+static struct scst_tgt_template srpt_template = {
+ .name = DRV_NAME,
+ .sg_tablesize = SRPT_DEF_SG_TABLESIZE,
+ .max_hw_pending_time = 60/*seconds*/,
+ .enable_target = srpt_enable_target,
+ .is_target_enabled = srpt_is_target_enabled,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = DEFAULT_SRPT_TRACE_FLAGS,
+ .trace_flags = &trace_flag,
+#endif
+ .detect = srpt_detect,
+ .release = srpt_release,
+ .xmit_response = srpt_xmit_response,
+ .rdy_to_xfer = srpt_rdy_to_xfer,
+ .on_hw_pending_cmd_timeout = srpt_pending_cmd_timeout,
+ .on_free_cmd = srpt_on_free_cmd,
+ .task_mgmt_fn_done = srpt_tsk_mgmt_done,
+ .get_initiator_port_transport_id = srpt_get_initiator_port_transport_id,
+ .get_scsi_transport_version = srpt_get_scsi_transport_version,
+};
+
+/**
+ * srpt_dev_release() - Device release callback function.
+ *
+ * The callback function srpt_dev_release() is called whenever a
+ * device is removed from the /sys/class/infiniband_srpt device class.
+ * Although this function has been left empty, a release function has been
+ * defined such that upon module removal no complaint is logged about a
+ * missing release function.
+ */
+static void srpt_dev_release(struct device *dev)
+{
+}
+
+static ssize_t show_login_info(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct srpt_device *sdev =
+ container_of(dev, struct srpt_device, dev);
+ struct srpt_port *sport;
+ int i;
+ int len = 0;
+
+ for (i = 0; i < sdev->device->phys_port_cnt; i++) {
+ sport = &sdev->port[i];
+
+ len += sprintf(buf + len,
+ "tid_ext=%016llx,ioc_guid=%016llx,pkey=ffff,"
+ "dgid=%04x%04x%04x%04x%04x%04x%04x%04x,"
+ "service_id=%016llx\n",
+ srpt_service_guid,
+ srpt_service_guid,
+ be16_to_cpu(((__be16 *) sport->gid.raw)[0]),
+ be16_to_cpu(((__be16 *) sport->gid.raw)[1]),
+ be16_to_cpu(((__be16 *) sport->gid.raw)[2]),
+ be16_to_cpu(((__be16 *) sport->gid.raw)[3]),
+ be16_to_cpu(((__be16 *) sport->gid.raw)[4]),
+ be16_to_cpu(((__be16 *) sport->gid.raw)[5]),
+ be16_to_cpu(((__be16 *) sport->gid.raw)[6]),
+ be16_to_cpu(((__be16 *) sport->gid.raw)[7]),
+ srpt_service_guid);
+ }
+
+ return len;
+}
+
+static struct class_attribute srpt_class_attrs[] = {
+ __ATTR_NULL,
+};
+
+static struct device_attribute srpt_dev_attrs[] = {
+ __ATTR(login_info, S_IRUGO, show_login_info, NULL),
+ __ATTR_NULL,
+};
+
+static struct class srpt_class = {
+ .name = "infiniband_srpt",
+ .dev_release = srpt_dev_release,
+ .class_attrs = srpt_class_attrs,
+ .dev_attrs = srpt_dev_attrs,
+};
+
+/**
+ * srpt_add_one() - Infiniband device addition callback function.
+ */
+static void srpt_add_one(struct ib_device *device)
+{
+ struct srpt_device *sdev;
+ struct srpt_port *sport;
+ struct ib_srq_init_attr srq_attr;
+ int i;
+
+ TRACE_DBG("device = %p, device->dma_ops = %p", device, device->dma_ops);
+
+ sdev = kzalloc(sizeof *sdev, GFP_KERNEL);
+ if (!sdev)
+ return;
+
+ sdev->scst_tgt = scst_register_target(&srpt_template, NULL);
+ if (!sdev->scst_tgt) {
+ PRINT_ERROR("SCST registration failed for %s.",
+ sdev->device->name);
+ goto free_dev;
+ }
+
+ scst_tgt_set_tgt_priv(sdev->scst_tgt, sdev);
+
+ sdev->device = device;
+
+ sdev->dev.class = &srpt_class;
+ sdev->dev.parent = device->dma_device;
+ dev_set_name(&sdev->dev, "srpt-%s", device->name);
+
+ if (device_register(&sdev->dev))
+ goto unregister_tgt;
+
+ if (ib_query_device(device, &sdev->dev_attr))
+ goto err_dev;
+
+ sdev->pd = ib_alloc_pd(device);
+ if (IS_ERR(sdev->pd))
+ goto err_dev;
+
+ sdev->mr = ib_get_dma_mr(sdev->pd, IB_ACCESS_LOCAL_WRITE);
+ if (IS_ERR(sdev->mr))
+ goto err_pd;
+
+ sdev->srq_size = min(srpt_srq_size, sdev->dev_attr.max_srq_wr);
+
+ srq_attr.event_handler = srpt_srq_event;
+ srq_attr.srq_context = (void *)sdev;
+ srq_attr.attr.max_wr = sdev->srq_size;
+ srq_attr.attr.max_sge = 1;
+ srq_attr.attr.srq_limit = 0;
+
+ sdev->srq = ib_create_srq(sdev->pd, &srq_attr);
+ if (IS_ERR(sdev->srq))
+ goto err_mr;
+
+ TRACE_DBG("%s: create SRQ #wr= %d max_allow=%d dev= %s",
+ __func__, sdev->srq_size,
+ sdev->dev_attr.max_srq_wr, device->name);
+
+ if (!srpt_service_guid)
+ srpt_service_guid = be64_to_cpu(device->node_guid);
+
+ sdev->cm_id = ib_create_cm_id(device, srpt_cm_handler, sdev);
+ if (IS_ERR(sdev->cm_id))
+ goto err_srq;
+
+ /* print out target login information */
+ TRACE_DBG("Target login info: id_ext=%016llx,"
+ "ioc_guid=%016llx,pkey=ffff,service_id=%016llx",
+ srpt_service_guid, srpt_service_guid, srpt_service_guid);
+
+ /*
+ * We do not have a consistent service_id (ie. also id_ext of target_id)
+ * to identify this target. We currently use the guid of the first HCA
+ * in the system as service_id; therefore, the target_id will change
+ * if this HCA is gone bad and replaced by different HCA
+ */
+ if (ib_cm_listen(sdev->cm_id, cpu_to_be64(srpt_service_guid), 0, NULL))
+ goto err_cm;
+
+ INIT_IB_EVENT_HANDLER(&sdev->event_handler, sdev->device,
+ srpt_event_handler);
+ if (ib_register_event_handler(&sdev->event_handler))
+ goto err_cm;
+
+ sdev->ioctx_ring = kmalloc(sdev->srq_size * sizeof sdev->ioctx_ring[0],
+ GFP_KERNEL);
+ if (!sdev->ioctx_ring)
+ goto err_event;
+
+ if (srpt_alloc_ioctx_ring(sdev, sdev->ioctx_ring, sdev->srq_size, 0))
+ goto err_alloc_ring;
+
+ INIT_LIST_HEAD(&sdev->rch_list);
+ spin_lock_init(&sdev->spinlock);
+
+ for (i = 0; i < sdev->srq_size; ++i)
+ srpt_post_recv(sdev, sdev->ioctx_ring[i]);
+
+ ib_set_client_data(device, &srpt_client, sdev);
+
+ WARN_ON(sdev->device->phys_port_cnt
+ > sizeof(sdev->port)/sizeof(sdev->port[0]));
+
+ for (i = 1; i <= sdev->device->phys_port_cnt; i++) {
+ sport = &sdev->port[i - 1];
+ sport->sdev = sdev;
+ sport->port = i;
+ INIT_WORK(&sport->work, srpt_refresh_port_work);
+ if (srpt_refresh_port(sport)) {
+ PRINT_ERROR("MAD registration failed for %s-%d.",
+ sdev->device->name, i);
+ goto err_ring;
+ }
+ }
+
+ atomic_inc(&srpt_device_count);
+
+ return;
+
+err_ring:
+ ib_set_client_data(device, &srpt_client, NULL);
+ srpt_free_ioctx_ring(sdev, sdev->ioctx_ring, sdev->srq_size);
+err_alloc_ring:
+ kfree(sdev->ioctx_ring);
+err_event:
+ ib_unregister_event_handler(&sdev->event_handler);
+err_cm:
+ ib_destroy_cm_id(sdev->cm_id);
+err_srq:
+ ib_destroy_srq(sdev->srq);
+err_mr:
+ ib_dereg_mr(sdev->mr);
+err_pd:
+ ib_dealloc_pd(sdev->pd);
+err_dev:
+ device_unregister(&sdev->dev);
+unregister_tgt:
+ scst_unregister_target(sdev->scst_tgt);
+free_dev:
+ kfree(sdev);
+}
+
+/**
+ * srpt_remove_one() - InfiniBand device removal callback function.
+ */
+static void srpt_remove_one(struct ib_device *device)
+{
+ int i;
+ struct srpt_device *sdev;
+
+ sdev = ib_get_client_data(device, &srpt_client);
+ if (WARN_ON(!sdev))
+ return;
+
+ srpt_unregister_mad_agent(sdev);
+
+ ib_unregister_event_handler(&sdev->event_handler);
+
+ /* Cancel any work queued by the just unregistered IB event handler. */
+ for (i = 0; i < sdev->device->phys_port_cnt; i++)
+ cancel_work_sync(&sdev->port[i].work);
+
+ ib_destroy_cm_id(sdev->cm_id);
+ ib_destroy_srq(sdev->srq);
+ ib_dereg_mr(sdev->mr);
+ ib_dealloc_pd(sdev->pd);
+
+ device_unregister(&sdev->dev);
+
+ /*
+ * Unregistering an SCST target must happen after destroying sdev->cm_id
+ * such that no new SRP_LOGIN_REQ information units can arrive while
+ * destroying the SCST target.
+ */
+ scst_unregister_target(sdev->scst_tgt);
+ sdev->scst_tgt = NULL;
+
+ srpt_free_ioctx_ring(sdev, sdev->ioctx_ring, sdev->srq_size);
+ kfree(sdev->ioctx_ring);
+ sdev->ioctx_ring = NULL;
+ kfree(sdev);
+}
+
+/**
+ * srpt_init_module() - Kernel module initialization.
+ *
+ * Note: Since ib_register_client() registers callback functions, and since at
+ * least one of these callback functions (srpt_add_one()) calls SCST functions,
+ * the SCST target template must be registered before ib_register_client() is
+ * called.
+ */
+static int __init srpt_init_module(void)
+{
+ int ret;
+
+ ret = -EINVAL;
+ if (srp_max_message_size < MIN_MAX_MESSAGE_SIZE) {
+ PRINT_ERROR("invalid value %d for kernel module parameter"
+ " srp_max_message_size -- must be at least %d.",
+ srp_max_message_size,
+ MIN_MAX_MESSAGE_SIZE);
+ goto out;
+ }
+
+ if (srpt_srq_size < MIN_SRPT_SRQ_SIZE
+ || srpt_srq_size > MAX_SRPT_SRQ_SIZE) {
+ PRINT_ERROR("invalid value %d for kernel module parameter"
+ " srpt_srq_size -- must be in the range [%d..%d].",
+ srpt_srq_size, MIN_SRPT_SRQ_SIZE,
+ MAX_SRPT_SRQ_SIZE);
+ goto out;
+ }
+
+ if (srpt_sq_size < MIN_SRPT_SQ_SIZE) {
+ PRINT_ERROR("invalid value %d for kernel module parameter"
+ " srpt_sq_size -- must be at least %d.",
+ srpt_srq_size, MIN_SRPT_SQ_SIZE);
+ goto out;
+ }
+
+ ret = class_register(&srpt_class);
+ if (ret) {
+ PRINT_ERROR("%s", "couldn't register class ib_srpt");
+ goto out;
+ }
+
+ switch (thread) {
+ case MODE_ALL_IN_SIRQ:
+ /*
+ * Process both IB completions and SCST commands in SIRQ
+ * context. May lead to soft lockups and other scary behavior
+ * under sufficient load.
+ */
+ srpt_template.rdy_to_xfer_atomic = true;
+ break;
+ case MODE_IB_COMPLETION_IN_THREAD:
+ /*
+ * Process IB completions in the kernel thread associated with
+ * the RDMA channel, and process SCST commands in the kernel
+ * threads created by the SCST core.
+ */
+ srpt_template.rdy_to_xfer_atomic = false;
+ break;
+ case MODE_IB_COMPLETION_IN_SIRQ:
+ default:
+ /*
+ * Process IB completions in SIRQ context and SCST commands in
+ * the kernel threads created by the SCST core.
+ */
+ srpt_template.rdy_to_xfer_atomic = false;
+ break;
+ }
+
+ ret = scst_register_target_template(&srpt_template);
+ if (ret < 0) {
+ PRINT_ERROR("%s", "couldn't register with scst");
+ ret = -ENODEV;
+ goto out_unregister_class;
+ }
+
+ ret = ib_register_client(&srpt_client);
+ if (ret) {
+ PRINT_ERROR("%s", "couldn't register IB client");
+ goto out_unregister_procfs;
+ }
+
+ return 0;
+
+out_unregister_procfs:
+ scst_unregister_target_template(&srpt_template);
+out_unregister_class:
+ class_unregister(&srpt_class);
+out:
+ return ret;
+}
+
+static void __exit srpt_cleanup_module(void)
+{
+
+ ib_unregister_client(&srpt_client);
+ scst_unregister_target_template(&srpt_template);
+ class_unregister(&srpt_class);
+}
+
+module_init(srpt_init_module);
+module_exit(srpt_cleanup_module);
+
+/*
+ * Local variables:
+ * c-basic-offset: 8
+ * indent-tabs-mode: t
+ * End:
+ */
diff -uprN orig/linux-2.6.35/drivers/scst/srpt/ib_srpt.h linux-2.6.35/drivers/scst/srpt/ib_srpt.h
--- orig/linux-2.6.35/drivers/scst/srpt/ib_srpt.h
+++ linux-2.6.35/drivers/scst/srpt/ib_srpt.h
@@ -0,0 +1,368 @@
+/*
+ * Copyright (c) 2006 - 2009 Mellanox Technology Inc. All rights reserved.
+ * Copyright (C) 2009 - 2010 Bart Van Assche <bart.vanassche@gmail.com>
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#ifndef IB_SRPT_H
+#define IB_SRPT_H
+
+#include <linux/version.h>
+#include <linux/types.h>
+#include <linux/list.h>
+
+#include <rdma/ib_verbs.h>
+#include <rdma/ib_sa.h>
+#include <rdma/ib_cm.h>
+
+#include <scsi/srp.h>
+
+#include <scst/scst.h>
+
+#include "ib_dm_mad.h"
+
+/*
+ * The prefix the ServiceName field must start with in the device management
+ * ServiceEntries attribute pair. See also the SRP r16a document.
+ */
+#define SRP_SERVICE_NAME_PREFIX "SRP.T10:"
+
+enum {
+ /*
+ * SRP IOControllerProfile attributes for SRP target ports that have
+ * not been defined in <scsi/srp.h>. Source: section B.7, table B.7
+ * in the SRP r16a document.
+ */
+ SRP_PROTOCOL = 0x0108,
+ SRP_PROTOCOL_VERSION = 0x0001,
+ SRP_IO_SUBCLASS = 0x609e,
+ SRP_SEND_TO_IOC = 0x01,
+ SRP_SEND_FROM_IOC = 0x02,
+ SRP_RDMA_READ_FROM_IOC = 0x08,
+ SRP_RDMA_WRITE_FROM_IOC = 0x20,
+
+ /*
+ * srp_login_cmd::req_flags bitmasks. See also table 9 in the SRP r16a
+ * document.
+ */
+ SRP_MTCH_ACTION = 0x03, /* MULTI-CHANNEL ACTION */
+ SRP_LOSOLNT = 0x10, /* logout solicited notification */
+ SRP_CRSOLNT = 0x20, /* credit request solicited notification */
+ SRP_AESOLNT = 0x40, /* asynchronous event solicited notification */
+
+ /*
+ * srp_cmd::sol_nt / srp_tsk_mgmt::sol_not bitmasks. See also tables
+ * 18 and 20 in the T10 r16a document.
+ */
+ SRP_SCSOLNT = 0x02, /* SCSOLNT = successful solicited notification */
+ SRP_UCSOLNT = 0x04, /* UCSOLNT = unsuccessful solicited notification */
+
+ /*
+ * srp_rsp::sol_not / srp_t_logout::sol_not bitmasks. See also tables
+ * 16 and 22 in the T10 r16a document.
+ */
+ SRP_SOLNT = 0x01, /* SOLNT = solicited notification */
+
+ /* See also table 24 in the T10 r16a document. */
+ SRP_TSK_MGMT_SUCCESS = 0x00,
+ SRP_TSK_MGMT_FUNC_NOT_SUPP = 0x04,
+ SRP_TSK_MGMT_FAILED = 0x05,
+
+ /* See also table 21 in the T10 r16a document. */
+ SRP_CMD_SIMPLE_Q = 0x0,
+ SRP_CMD_HEAD_OF_Q = 0x1,
+ SRP_CMD_ORDERED_Q = 0x2,
+ SRP_CMD_ACA = 0x4,
+
+ SRP_LOGIN_RSP_MULTICHAN_NO_CHAN = 0x0,
+ SRP_LOGIN_RSP_MULTICHAN_TERMINATED = 0x1,
+ SRP_LOGIN_RSP_MULTICHAN_MAINTAINED = 0x2,
+
+ SRPT_DEF_SG_TABLESIZE = 128,
+ SRPT_DEF_SG_PER_WQE = 16,
+
+ MIN_SRPT_SQ_SIZE = 16,
+ DEF_SRPT_SQ_SIZE = 4096,
+ SRPT_RQ_SIZE = 128,
+ MIN_SRPT_SRQ_SIZE = 4,
+ DEFAULT_SRPT_SRQ_SIZE = 4095,
+ MAX_SRPT_SRQ_SIZE = 65535,
+
+ MIN_MAX_MESSAGE_SIZE = 996,
+ DEFAULT_MAX_MESSAGE_SIZE
+ = sizeof(struct srp_cmd)/*48*/
+ + sizeof(struct srp_indirect_buf)/*20*/
+ + 128 * sizeof(struct srp_direct_buf)/*16*/,
+
+ DEFAULT_MAX_RDMA_SIZE = 65536,
+
+ /*
+ * Number of I/O contexts to be allocated for sending back requests
+ * from the target to the initiator. Must be a power of two.
+ */
+ TTI_IOCTX_COUNT = 2,
+ TTI_IOCTX_MASK = TTI_IOCTX_COUNT - 1,
+};
+
+/**
+ * @SRPT_OP_TTI: wr_id flag for marking requests sent by the target to the
+ * initiator.
+ * @SRPT_OP_RECV: wr_id flag for marking receive operations.
+ */
+enum {
+ SRPT_OP_TTI = (1 << 30),
+ SRPT_OP_RECV = (1 << 31),
+
+ SRPT_OP_FLAGS = SRPT_OP_TTI | SRPT_OP_RECV,
+};
+
+/*
+ * SRP_CRED_REQ information unit, as defined in section 6.10 of the T10 SRP
+ * r16a document.
+ */
+struct srp_cred_req {
+ u8 opcode;
+ u8 sol_not;
+ u8 reserved[2];
+ __be32 req_lim_delta;
+ __be64 tag;
+};
+
+struct rdma_iu {
+ u64 raddr;
+ u32 rkey;
+ struct ib_sge *sge;
+ u32 sge_cnt;
+ int mem_id;
+};
+
+/**
+ * enum srpt_command_state - SCSI command states managed by SRPT.
+ * @SRPT_STATE_NEW: New command arrived and is being processed.
+ * @SRPT_STATE_NEED_DATA: Processing a write or bidir command and waiting
+ * for data arrival.
+ * @SRPT_STATE_DATA_IN: Data for the write or bidir command arrived and is
+ * being processed.
+ * @SRPT_STATE_CMD_RSP_SENT: SRP_RSP for SRP_CMD has been sent.
+ * @SRPT_STATE_MGMT_RSP_SENT: SRP_RSP for SRP_TSK_MGMT has been sent.
+ * @SRPT_STATE_DONE: Command processing finished successfully, command
+ * processing has been aborted or command processing
+ * failed.
+ */
+enum srpt_command_state {
+ SRPT_STATE_NEW = 0,
+ SRPT_STATE_NEED_DATA = 1,
+ SRPT_STATE_DATA_IN = 2,
+ SRPT_STATE_CMD_RSP_SENT = 3,
+ SRPT_STATE_MGMT_RSP_SENT = 4,
+ SRPT_STATE_DONE = 5,
+};
+
+/**
+ * struct srpt_ioctx - SRPT-private data associated with a struct scst_cmd.
+ * @index: Index of the I/O context in ioctx_ring.
+ * @buf: Pointer to the message transferred via this I/O context.
+ * @dma: DMA address of buf.
+ * @wait_list: Node for insertion in srpt_rdma_ch::cmd_wait_list.
+ * @state: I/O context state. See also enum srpt_command_state.
+ */
+struct srpt_ioctx {
+ int index;
+ void *buf;
+ dma_addr_t dma;
+ struct rdma_iu *rdma_ius;
+ struct srp_direct_buf *rbufs;
+ struct srp_direct_buf single_rbuf;
+ struct list_head wait_list;
+ int mapped_sg_count;
+ u16 n_rdma_ius;
+ u8 n_rdma;
+ u8 n_rbuf;
+
+ u64 wr_id;
+ enum ib_wc_status status;
+ enum ib_wc_opcode opcode;
+ struct srpt_rdma_ch *ch;
+ struct scst_cmd *scmnd;
+ scst_data_direction dir;
+ atomic_t state;
+};
+
+/**
+ * struct srpt_mgmt_ioctx - SCST management command context information.
+ * @ioctx: SRPT I/O context associated with the management command.
+ * @ch: RDMA channel over which the management command has been received.
+ * @tag: SCSI tag of the management command.
+ */
+struct srpt_mgmt_ioctx {
+ struct srpt_ioctx *ioctx;
+ struct srpt_rdma_ch *ch;
+ u64 tag;
+};
+
+/**
+ * enum rdma_ch_state - SRP channel state.
+ */
+enum rdma_ch_state {
+ RDMA_CHANNEL_CONNECTING,
+ RDMA_CHANNEL_LIVE,
+ RDMA_CHANNEL_DISCONNECTING
+};
+
+/**
+ * struct srpt_rdma_ch - RDMA channel.
+ * @wait_queue: Allows the kernel thread to wait for more work.
+ * @thread: Kernel thread that processes the IB queues associated with
+ * the channel.
+ * @cm_id: IB CM ID associated with the channel.
+ * @rq_size: IB receive queue size.
+ * @processing_compl: whether or not an IB completion is being processed.
+ * @qp: IB queue pair used for communicating over this channel.
+ * @sq_wr_avail: number of work requests available in the send queue.
+ * @cq: IB completion queue for this channel.
+ * @sport: pointer to the information of the HCA port used by this
+ * channel.
+ * @i_port_id: 128-bit initiator port identifier copied from SRP_LOGIN_REQ.
+ * @t_port_id: 128-bit target port identifier copied from SRP_LOGIN_REQ.
+ * @max_ti_iu_len: maximum target-to-initiator information unit length.
+ * @supports_cred_req: whether or not the initiator supports SRP_CRED_REQ.
+ * @req_lim: request limit: maximum number of requests that may be sent
+ * by the initiator without having received a response or
+ * SRP_CRED_REQ.
+ * @req_lim_delta: req_lim_delta to be sent in the next SRP_RSP.
+ * @req_lim_waiter_count: number of threads waiting on req_lim_wait.
+ * @req_lim_compl: completion variable that is signalled every time req_lim
+ * has been incremented.
+ * @state: channel state. See also enum rdma_ch_state.
+ * @list: node for insertion in the srpt_device::rch_list list.
+ * @cmd_wait_list: list of SCST commands that arrived before the RTU event. This
+ * list contains struct srpt_ioctx elements and is protected
+ * against concurrent modification by the cm_id spinlock.
+ * @tti_head: Index of first element of tti_ioctx that is not in use.
+ * @tti_tail: Index of first element of tti_ioctx that is in use.
+ * @tti_ioctx: Circular buffer with I/O contexts for sending requests from
+ * target to initiator.
+ * @scst_sess: SCST session information associated with this SRP channel.
+ * @sess_name: SCST session name.
+ */
+struct srpt_rdma_ch {
+ wait_queue_head_t wait_queue;
+ struct task_struct *thread;
+ struct ib_cm_id *cm_id;
+ struct ib_qp *qp;
+ int rq_size;
+ atomic_t processing_compl;
+ struct ib_cq *cq;
+ atomic_t sq_wr_avail;
+ struct srpt_port *sport;
+ u8 i_port_id[16];
+ u8 t_port_id[16];
+ int max_ti_iu_len;
+ atomic_t supports_cred_req;
+ atomic_t req_lim;
+ atomic_t req_lim_delta;
+ atomic_t req_lim_waiter_count;
+ struct completion req_lim_compl;
+ atomic_t state;
+ struct list_head list;
+ struct list_head cmd_wait_list;
+ int tti_head;
+ int tti_tail;
+ struct srpt_ioctx *tti_ioctx[TTI_IOCTX_COUNT];
+
+ struct scst_session *scst_sess;
+ u8 sess_name[36];
+};
+
+/**
+ * struct srpt_port - Information associated by SRPT with a single IB port.
+ * @sdev: backpointer to the HCA information.
+ * @mad_agent: per-port management datagram processing information.
+ * @port: one-based port number.
+ * @sm_lid: cached value of the port's sm_lid.
+ * @lid: cached value of the port's lid.
+ * @gid: cached value of the port's gid.
+ * @work: work structure for refreshing the aforementioned cached values.
+ */
+struct srpt_port {
+ struct srpt_device *sdev;
+ struct ib_mad_agent *mad_agent;
+ u8 port;
+ u16 sm_lid;
+ u16 lid;
+ union ib_gid gid;
+ struct work_struct work;
+};
+
+/**
+ * struct srpt_device - Information associated by SRPT with a single HCA.
+ * @device: backpointer to the struct ib_device managed by the IB core.
+ * @pd: IB protection domain.
+ * @mr: L_Key (local key) with write access to all local memory.
+ * @srq: Per-HCA SRQ (shared receive queue).
+ * @cm_id: connection identifier.
+ * @dev_attr: attributes of the InfiniBand device as obtained during the
+ * ib_client::add() callback.
+ * @ioctx_ring: Per-HCA I/O context ring.
+ * @rch_list: per-device channel list -- see also srpt_rdma_ch::list.
+ * @spinlock: protects rch_list.
+ * @srpt_port: information about the ports owned by this HCA.
+ * @event_handler: per-HCA asynchronous IB event handler.
+ * @dev: per-port srpt-<portname> device instance.
+ * @scst_tgt: SCST target information associated with this HCA.
+ * @enabled: Whether or not this SCST target is enabled.
+ */
+struct srpt_device {
+ struct ib_device *device;
+ struct ib_pd *pd;
+ struct ib_mr *mr;
+ struct ib_srq *srq;
+ struct ib_cm_id *cm_id;
+ struct ib_device_attr dev_attr;
+ int srq_size;
+ struct srpt_ioctx **ioctx_ring;
+ struct list_head rch_list;
+ spinlock_t spinlock;
+ struct srpt_port port[2];
+ struct ib_event_handler event_handler;
+ struct device dev;
+ struct scst_tgt *scst_tgt;
+ bool enabled;
+};
+
+#endif /* IB_SRPT_H */
+
+/*
+ * Local variables:
+ * c-basic-offset: 8
+ * indent-tabs-mode: t
+ * End:
+ */
Signed-off-by:
diff -uprN orig/linux-2.6.35/Documentation/scst/README.srpt linux-2.6.35/Documentation/scst/README.srpt
--- orig/linux-2.6.35/Documentation/scst/README.srpt
+++ linux-2.6.35/Documentation/scst/README.srpt
@@ -0,0 +1,109 @@
+SCSI RDMA Protocol (SRP) Target driver for Linux
+=================================================
+
+The SRP Target driver is designed to work directly on top of the
+OpenFabrics OFED-1.x software stack (http://www.openfabrics.org) or
+the Infiniband drivers in the Linux kernel tree
+(http://www.kernel.org). The SRP target driver also interfaces with
+the generic SCSI target mid-level driver called SCST
+(http://scst.sourceforge.net).
+
+How-to run
+-----------
+
+A. On srp target machine
+1. Please refer to SCST's README for loading scst driver and its
+dev_handlers drivers (scst_disk, scst_vdisk block or file IO mode, nullio, ...)
+
+Example 1: working with real back-end scsi disks
+a. modprobe scst
+b. modprobe scst_disk
+c. cat /proc/scsi_tgt/scsi_tgt
+
+ibstor00:~ # cat /proc/scsi_tgt/scsi_tgt
+Device (host:ch:id:lun or name) Device handler
+0:0:0:0 dev_disk
+4:0:0:0 dev_disk
+5:0:0:0 dev_disk
+6:0:0:0 dev_disk
+7:0:0:0 dev_disk
+
+Now you want to exclude the first scsi disk and expose the last 4 scsi disks as
+IB/SRP luns for I/O
+echo "add 4:0:0:0 0" >/proc/scsi_tgt/groups/Default/devices
+echo "add 5:0:0:0 1" >/proc/scsi_tgt/groups/Default/devices
+echo "add 6:0:0:0 2" >/proc/scsi_tgt/groups/Default/devices
+echo "add 7:0:0:0 3" >/proc/scsi_tgt/groups/Default/devices
+
+Example 2: working with VDISK FILEIO mode (using md0 device and file 10G-file)
+a. modprobe scst
+b. modprobe scst_vdisk
+c. echo "open vdisk0 /dev/md0" > /proc/scsi_tgt/vdisk/vdisk
+d. echo "open vdisk1 /10G-file" > /proc/scsi_tgt/vdisk/vdisk
+e. echo "add vdisk0 0" >/proc/scsi_tgt/groups/Default/devices
+f. echo "add vdisk1 1" >/proc/scsi_tgt/groups/Default/devices
+
+Example 3: working with VDISK BLOCKIO mode (using md0 device, sda, and cciss/c1d0)
+a. modprobe scst
+b. modprobe scst_vdisk
+c. echo "open vdisk0 /dev/md0 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk
+d. echo "open vdisk1 /dev/sda BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk
+e. echo "open vdisk2 /dev/cciss/c1d0 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk
+f. echo "add vdisk0 0" >/proc/scsi_tgt/groups/Default/devices
+g. echo "add vdisk1 1" >/proc/scsi_tgt/groups/Default/devices
+h. echo "add vdisk2 2" >/proc/scsi_tgt/groups/Default/devices
+
+2. modprobe ib_srpt
+
+B. On initiator machines you can manualy do the following steps:
+1. modprobe ib_srp
+2. ibsrpdm -c (to discover new SRP target)
+3. echo <new target info> > /sys/class/infiniband_srp/srp-mthca0-1/add_target
+4. fdisk -l (will show new discovered scsi disks)
+
+Example:
+Assume that you use port 1 of first HCA in the system ie. mthca0
+
+[root@lab104 ~]# ibsrpdm -c -d /dev/infiniband/umad0
+id_ext=0002c90200226cf4,ioc_guid=0002c90200226cf4,
+dgid=fe800000000000000002c90200226cf5,pkey=ffff,service_id=0002c90200226cf4
+[root@lab104 ~]# echo id_ext=0002c90200226cf4,ioc_guid=0002c90200226cf4,
+dgid=fe800000000000000002c90200226cf5,pkey=ffff,service_id=0002c90200226cf4 >
+/sys/class/infiniband_srp/srp-mthca0-1/add_target
+
+OR
+
++ You can edit /etc/infiniband/openib.conf to load srp driver and srp HA daemon
+automatically ie. set SRP_LOAD=yes, and SRPHA_ENABLE=yes
++ To set up and use high availability feature you need dm-multipath driver
+and multipath tool
++ Please refer to OFED-1.x SRP's user manual for more in-details instructions
+on how-to enable/use HA feature
+
+To minimize QUEUE_FULL conditions, you can apply scst_increase_max_tgt_cmds
+patch from SRPT package from http://sourceforge.net/project/showfiles.php?group_id=110471
+
+Performance notes
+-----------------
+
+In some cases, for instance working with SSD devices, which consume 100%
+of a single CPU load for data transfers in their internal threads, to
+maximize IOPS it can be needed to assign for those threads dedicated
+CPUs using Linux CPU affinity facilities. No IRQ processing should be
+done on those CPUs. Check that using /proc/interrupts. See taskset
+command and Documentation/IRQ-affinity.txt in your kernel's source tree
+for how to assign CPU affinity to tasks and IRQs.
+
+The reason for that is that processing of coming commands in SIRQ context
+can be done on the same CPUs as SSD devices' threads doing data
+transfers. As the result, those threads won't receive all the CPU power
+and perform worse.
+
+Alternatively to CPU affinity assignment, you can try to enable SRP
+target's internal thread. It will allows Linux CPU scheduler to better
+distribute load among available CPUs. To enable SRP target driver's
+internal thread you should load ib_srpt module with parameter
+"thread=1".
+
+Send questions about this driver to scst-devel@lists.sourceforge.net, CC:
+Vu Pham <vuhuong@mellanox.com> and Bart Van Assche <bart.vanassche@gmail.com>.
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers
2010-09-14 14:36 [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Vladislav Bolkhovitin
` (16 preceding siblings ...)
2010-09-14 14:51 ` [PATCH 17/17]: SCST InfiniBand SRP " Vladislav Bolkhovitin
@ 2010-09-15 5:56 ` Dmitry Torokhov
2010-09-15 14:52 ` Vladislav Bolkhovitin
2010-09-19 9:54 ` Bart Van Assche
17 siblings, 2 replies; 22+ messages in thread
From: Dmitry Torokhov @ 2010-09-15 5:56 UTC (permalink / raw)
To: Vladislav Bolkhovitin
Cc: linux-scsi, linux-kernel, scst-devel, James Bottomley,
Andrew Morton, FUJITA Tomonori, Mike Christie, Jeff Garzik,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
Chetan Loke, Hannes Reinecke, Richard Sharpe
Hi Vladislav,
On Tue, Sep 14, 2010 at 06:36:30PM +0400, Vladislav Bolkhovitin wrote:
> Hi All,
>
> Please review this second iteration of the patch set of the new
> (although, perhaps, the oldest) SCSI target framework for Linux SCST
> with a set of dev handlers and 2 target drivers: for local access
> (scst_local) and for Infiniband SRP (srpt).
>
I am afraid that the way the code was split into the patches will hinder
the review process. Normally every patch consists of a usable on its
own piece of code (or a logically compete change). This way the unit of
work (coding, testing or reviewing) is clearly defined and easier to
comprehend.
Your patch series, unfortunately, splits Makefile and Kconfig changes
separately from the drivers (which will cause build breakages should it
be applied as is and someone needs to bisect), introduces header files
in their entirety even when some of the data there is not needed till
much later, uses facilities (for example sysfs bindings) that have not
been introduced yet... I am afraid you'd have to rework the splitting
process.
Also patch descriptions should be improved:
"This patch contains SYSFS interface implementation"
- what are you trying to solve?
- main points?
- why offloading of actions to a separate thread?
This should be in the patch description.
BTW, why are you using an exclusive thread instead of exclusive
workqueue for this?
Also, kobject_del() + kobject_put() == kobject_unregister().
And too many "naked returns", I believe common style is to only have
return for non-void functions.
Thanks.
--
Dmitry
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers
2010-09-15 5:56 ` [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Dmitry Torokhov
@ 2010-09-15 14:52 ` Vladislav Bolkhovitin
2010-09-15 18:07 ` Vladislav Bolkhovitin
2010-09-19 9:54 ` Bart Van Assche
1 sibling, 1 reply; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-15 14:52 UTC (permalink / raw)
To: Dmitry Torokhov
Cc: linux-scsi, linux-kernel, scst-devel, James Bottomley,
Andrew Morton, FUJITA Tomonori, Mike Christie, Jeff Garzik,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
Chetan Loke, Hannes Reinecke, Richard Sharpe
Hi Dmitry,
Dmitry Torokhov, on 09/15/2010 09:56 AM wrote:
>> Please review this second iteration of the patch set of the new
>> (although, perhaps, the oldest) SCSI target framework for Linux SCST
>> with a set of dev handlers and 2 target drivers: for local access
>> (scst_local) and for Infiniband SRP (srpt).
>
> I am afraid that the way the code was split into the patches will hinder
> the review process. Normally every patch consists of a usable on its
> own piece of code (or a logically compete change). This way the unit of
> work (coding, testing or reviewing) is clearly defined and easier to
> comprehend.
>
> Your patch series, unfortunately, splits Makefile and Kconfig changes
> separately from the drivers (which will cause build breakages should it
> be applied as is and someone needs to bisect), introduces header files
> in their entirety even when some of the data there is not needed till
> much later, uses facilities (for example sysfs bindings) that have not
> been introduced yet... I am afraid you'd have to rework the splitting
> process.
I can do any splitting you like. Generally, I oriented on the similarly
large work recently accepted into the kernel: DRBD. DRBD's patches were
split on per file basis. So, I did, basically, the same, only combined
some small such files into a single patch (e.g. "[PATCH 1/17]: Integration
of SCST into the Linux kernel tree" or "[PATCH 4/17]: SCST main management
files and private headers"). Making Makefile and Kconfig as a
separate patch is just a part of this approach. Unfortunately, SCST is too
big to be allowed to be sent to LKML, where the limit is 300KB, so splitting
it on separate patches and the corresponding bisecting breakage seems to be
unavoidable.
> Also patch descriptions should be improved:
>
> "This patch contains SYSFS interface implementation"
SCST overall as well the SYSFS interface are thoroughly documented in the
SCST README and SysfsRules files, so I thought it would be unnecessary
to duplicate this text in the patch description, especially considering
that the SYSFS interface is quite big, so its description is big as well.
Would it be OK if I add to the next patches' descriptions references
where to find the corresponding documentation?
> - what are you trying to solve?
> - main points?
This interface provides possibility for a user to configure his/her SCST
server configuration: add/delete/manage target drivers, targets, dev handlers,
virtual devices and access control to them as well as it allows to see current
SCST configuration with necessary statistical and debug info (e.g. SGV cache
statistics).
> - why offloading of actions to a separate thread?
See below.
> This should be in the patch description.
OK
> BTW, why are you using an exclusive thread instead of exclusive
> workqueue for this?
This is a bit tricky. Basically, I documented it in scst.h before
struct scst_sysfs_work_item definition.
All internal SCST management is done for simplicity under scst_mutex.
It's simple, robust and worked well even under the highest load for ages.
But in 2.6.35 sysfs was improved to make lockdep to check s_active related
deadlocks and we discovered potential circular locking dependency
between scst_mutex and s_active. On some management operations lockdep triggered
output like:
[ 2036.926891] =======================================================
[ 2036.927670] [ INFO: possible circular locking dependency detected ]
[ 2036.927670] 2.6.35-scst-dbg #15
[ 2036.927670] -------------------------------------------------------
[ 2036.927670] rmmod/4715 is trying to acquire lock:
[ 2036.927670] (s_active#230){++++.+}, at: [<78240a24>] sysfs_hash_and_remove+0x63/0x67
[ 2036.927670]
[ 2036.927670] but task is already holding lock:
[ 2036.927670] (&scst_mutex){+.+.+.}, at: [<fefd7fe2>] scst_unregister_virtual_device+0x58/0x216 [scst]
[ 2036.927670]
[ 2036.927670] which lock already depends on the new lock.
[ 2036.927670]
[ 2036.927670]
[ 2036.927670] the existing dependency chain (in reverse order) is:
[ 2036.927670]
[ 2036.927670] -> #2 (&scst_mutex){+.+.+.}:
[ 2036.927670] [<78168d67>] lock_acquire+0x76/0x129
[ 2036.927670] [<78467619>] __mutex_lock_common+0x58/0x3fc
[ 2036.927670] [<78467a6d>] mutex_lock_nested+0x36/0x3d
[ 2036.927670] [<f8ecec91>] vcdrom_change+0x1b9/0x500 [scst_vdisk]
[ 2036.927670] [<f8ecf030>] vcdrom_sysfs_filename_store+0x58/0xd8 [scst_vdisk]
[ 2036.927670] [<feffd139>] scst_dev_attr_store+0x44/0x5d [scst]
[ 2036.927670] [<7824104f>] sysfs_write_file+0x9e/0xe8
[ 2036.927670] [<781ee836>] vfs_write+0x91/0x17e
[ 2036.927670] [<781ef213>] sys_write+0x42/0x69
[ 2036.927670] [<78102d13>] sysenter_do_call+0x12/0x32
[ 2036.927670]
[ 2036.927670] -> #1 (&virt_dev->vdev_sysfs_mutex){+.+.+.}:
[ 2036.927670] [<78168d67>] lock_acquire+0x76/0x129
[ 2036.927670] [<78467619>] __mutex_lock_common+0x58/0x3fc
[ 2036.927670] [<784679f3>] mutex_lock_interruptible_nested+0x36/0x3d
[ 2036.927670] [<f8ecebd6>] vcdrom_change+0xfe/0x500 [scst_vdisk]
[ 2036.927670] [<f8ecf030>] vcdrom_sysfs_filename_store+0x58/0xd8 [scst_vdisk]
[ 2036.927670] [<feffd139>] scst_dev_attr_store+0x44/0x5d [scst]
[ 2036.927670] [<7824104f>] sysfs_write_file+0x9e/0xe8
[ 2036.927670] [<781ee836>] vfs_write+0x91/0x17e
[ 2036.927670] [<781ef213>] sys_write+0x42/0x69
[ 2036.927670] [<78102d13>] sysenter_do_call+0x12/0x32
[ 2036.927670]
[ 2036.927670] -> #0 (s_active#230){++++.+}:
[ 2036.927670] [<78168af4>] __lock_acquire+0x1013/0x1210
[ 2036.927670] [<78168d67>] lock_acquire+0x76/0x129
[ 2036.927670] [<78242417>] sysfs_addrm_finish+0x100/0x150
[ 2036.927670] [<78240a24>] sysfs_hash_and_remove+0x63/0x67
[ 2036.927670] [<782415b6>] sysfs_remove_file+0x14/0x16
[ 2036.927670] [<feffdb29>] scst_devt_dev_sysfs_put+0x75/0x133 [scst]
[ 2036.927670] [<fefd6410>] scst_assign_dev_handler+0x109/0x5b6 [scst]
[ 2036.927670] [<fefd80ce>] scst_unregister_virtual_device+0x144/0x216 [scst]
[ 2036.927670] [<f8ed06f3>] vdev_del_device+0x47/0xd4 [scst_vdisk]
[ 2036.927670] [<f8ed6701>] exit_scst_vdisk+0x60/0xe6 [scst_vdisk]
[ 2036.927670] [<f8ed67b1>] exit_scst_vdisk_driver+0x12/0x46 [scst_vdisk]
[ 2036.927670] [<7817253a>] sys_delete_module+0x139/0x214
[ 2036.927670] [<78102d13>] sysenter_do_call+0x12/0x32
[ 2036.927670]
[ 2036.927670] other info that might help us debug this:
[ 2036.927670]
[ 2036.927670] 2 locks held by rmmod/4715:
[ 2036.927670] #0: (scst_vdisk_mutex){+.+.+.}, at: [<f8ed66f0>] exit_scst_vdisk+0x4f/0xe6 [scst_vdisk]
[ 2036.927670] #1: (&scst_mutex){+.+.+.}, at: [<fefd7fe2>] scst_unregister_virtual_device+0x58/0x216 [scst]
[ 2036.927670]
[ 2036.927670] stack backtrace:
[ 2036.927670] Pid: 4715, comm: rmmod Not tainted 2.6.35-scst-dbg #15
[ 2036.927670] Call Trace:
[ 2036.927670] [<784660a3>] ? printk+0x2d/0x32
[ 2036.927670] [<78166cbc>] print_circular_bug+0xb4/0xb9
[ 2036.927670] [<78168af4>] __lock_acquire+0x1013/0x1210
[ 2036.927670] [<78168d67>] lock_acquire+0x76/0x129
[ 2036.927670] [<78240a24>] ? sysfs_hash_and_remove+0x63/0x67
[ 2036.927670] [<78242417>] sysfs_addrm_finish+0x100/0x150
[ 2036.927670] [<78240a24>] ? sysfs_hash_and_remove+0x63/0x67
[ 2036.927670] [<78240a24>] sysfs_hash_and_remove+0x63/0x67
[ 2036.927670] [<782415b6>] sysfs_remove_file+0x14/0x16
[ 2036.927670] [<feffdb29>] scst_devt_dev_sysfs_put+0x75/0x133 [scst]
[ 2036.927670] [<fefd5b20>] ? scst_stop_dev_threads+0x77/0x111 [scst]
[ 2036.927670] [<f8ece3c2>] ? vdisk_detach+0x88/0x133 [scst_vdisk]
[ 2036.927670] [<fefd6410>] scst_assign_dev_handler+0x109/0x5b6 [scst]
[ 2036.927670] [<ff007368>] ? scst_pr_clear_dev+0x8e/0xfc [scst]
[ 2036.927670] [<fefd80ce>] scst_unregister_virtual_device+0x144/0x216 [scst]
[ 2036.927670] [<f8ed06f3>] vdev_del_device+0x47/0xd4 [scst_vdisk]
[ 2036.927670] [<f8ed6701>] exit_scst_vdisk+0x60/0xe6 [scst_vdisk]
[ 2036.927670] [<f8ed67b1>] exit_scst_vdisk_driver+0x12/0x46 [scst_vdisk]
[ 2036.927670] [<7817253a>] sys_delete_module+0x139/0x214
[ 2036.927670] [<7846c87e>] ? sub_preempt_count+0x7e/0xad
[ 2036.927670] [<78102d42>] ? sysenter_exit+0xf/0x1a
[ 2036.927670] [<7816789d>] ? trace_hardirqs_on_caller+0x10c/0x14d
[ 2036.927670] [<78102d13>] sysenter_do_call+0x12/0x32
[ 2036.927670] [<7846007b>] ? init_intel_cacheinfo+0x317/0x38b
It is caused by a chicken and egg problem: SCST objects, including their sysfs hierarchy
(kobjects) are created under scst_mutex, but for some of them (ACGs and their names
attributes, ACNs, are the most problematic objects) the creation is triggered from
inside the SYSFS.
I spent a LOT of time trying to rule out this problem in an acceptable manner.
Particularly, I analyzed splitting creation of SCST objects and their kobjects, so the
latter would be created outside of scst_mutex, and making a fine grain locking for
the SCST management instead of the single scst_mutex, but all the options lead to
unacceptably complicated code. So, I have chosen the use of the separate thread for all
the SYSFS management operation with scst_mutex/s_active deadlock detecting (see
scst_sysfs_queue_wait_work()) and, if the deadlock possibility detected, returning EAGAIN
asking the user space to poll completion of the command using last_sysfs_mgmt_res
attribute. It is documented in the README and scstadmin is doing that. It, definitely,
isn't a piece of beauty, but it's simple and works, so, I believe, good enough. User space,
anyway, is supposes to hide all the complexities of the direct SYSFS manipulations under
higher level management tools like scstadmin.
> Also, kobject_del() + kobject_put() == kobject_unregister().
What do you mean? I can't find kobject_unregister() in the kernel code.
> And too many "naked returns", I believe common style is to only have
> return for non-void functions.
Hmm, I have not found this in the CodingStyle file. I personally started doing that
after I, adding later error recovery cases in returning void functions, several times
forgot to add the corresponding empty "return" before them, hence had them erroneously
triggered. Like:
was:
void f(void)
{
...
}
new:
void f(void)
{
...
out_error:
error recovery code and exit
}
should be:
void f(void)
{
...
return;
out_error:
error recovery code and exit
}
So, I'd prefer to keep those returns to save from such mistakes in future.
Thank you,
Vlad
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers
2010-09-15 14:52 ` Vladislav Bolkhovitin
@ 2010-09-15 18:07 ` Vladislav Bolkhovitin
0 siblings, 0 replies; 22+ messages in thread
From: Vladislav Bolkhovitin @ 2010-09-15 18:07 UTC (permalink / raw)
To: Dmitry Torokhov
Cc: linux-scsi, linux-kernel, scst-devel, James Bottomley,
Andrew Morton, FUJITA Tomonori, Mike Christie, Jeff Garzik,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
Chetan Loke, Hannes Reinecke, Richard Sharpe
Vladislav Bolkhovitin, on 09/15/2010 06:52 PM wrote:
> It, definitely,
> isn't a piece of beauty, but it's simple and works, so, I believe, good enough. User space,
> anyway, is supposes to hide all the complexities of the direct SYSFS manipulations under
> higher level management tools like scstadmin.
There's one more point I forgot to note why having the separate SYSFS
management thread is good: it naturally serializes all SYSFS management
operations, so allows to simplify locking in target drivers and dev
handlers. For instance, it allows in add_target() callback don't bother
if del_target() or another add_target() for the same target name called
simultaneously.
Thanks,
Vlad
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers
2010-09-15 5:56 ` [PATCH 0/17]: New SCSI target framework (SCST) with dev handlers and 2 target drivers Dmitry Torokhov
2010-09-15 14:52 ` Vladislav Bolkhovitin
@ 2010-09-19 9:54 ` Bart Van Assche
1 sibling, 0 replies; 22+ messages in thread
From: Bart Van Assche @ 2010-09-19 9:54 UTC (permalink / raw)
To: Dmitry Torokhov
Cc: Vladislav Bolkhovitin, linux-scsi, linux-kernel, scst-devel,
James Bottomley, Andrew Morton, FUJITA Tomonori, Mike Christie,
Jeff Garzik, Vu Pham, James Smart, Joe Eykholt, Andy Yan,
Chetan Loke, Hannes Reinecke, Richard Sharpe
On Wed, Sep 15, 2010 at 7:56 AM, Dmitry Torokhov
<dmitry.torokhov@gmail.com> wrote:
>
> Hi Vladislav,
>
> On Tue, Sep 14, 2010 at 06:36:30PM +0400, Vladislav Bolkhovitin wrote:
> > Hi All,
> >
> > Please review this second iteration of the patch set of the new
> > (although, perhaps, the oldest) SCSI target framework for Linux SCST
> > with a set of dev handlers and 2 target drivers: for local access
> > (scst_local) and for Infiniband SRP (srpt).
> >
>
> [ ... ]
>
> And too many "naked returns", I believe common style is to only have
> return for non-void functions.
Hello Dmitry,
If there is agreement among kernel maintainers that return statements
at the end of functions that do not return a value should be avoided,
it might be a good idea to let checkpatch complain about such return
statements. Currently checkpatch doesn't complain about such return
statements.
Bart.
^ permalink raw reply [flat|nested] 22+ messages in thread