[PATCH V5 0/5] Add XEN pvSCSI support

[PATCH V5 0/5] Add XEN pvSCSI support

[jgross]

This series adds XEN pvSCSI support. With pvSCSI it is possible to use physical
SCSI devices from a XEN domain.

The support consists of a backend in the privileged Domain-0 doing the real
I/O and a frontend in the unprivileged domU passing I/O-requests to the backend.

The code is taken (and adapted) from the original pvSCSI implementation done
for Linux 2.6 in 2008 by Fujitsu.

[PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels
[PATCH V5 2/5] Add XEN pvSCSI protocol description
[PATCH V5 3/5] Introduce xen-scsifront module
[PATCH V5 4/5] Introduce XEN scsiback module
[PATCH V5 5/5] add xen pvscsi maintainer

Changes in V5:
- Added patch to support threaded irqs for interdomain event channels
- several changes in xen-scsifront after comments from Christoph Hellwig
- several changes in xen-scsiback after comments from Christoph Hellwig
- several changes in xen-scsiback after comments from James Bottomley

Changes in V4:
- Re-add define for VSCSIIF_ACT_SCSI_SG_PRESET to vscsiif.h to indicate this
  action value should not be used in future enhancements

Changes in V3:
- added some comments to the protocol header file
- removed the CDB emulation from xen-scsiback, handled by core target
  infrastructure
- several changes in xen-scsifront after comments from Christoph Hellwig

Changes in V2:
- use core target infrastructure by backend instead of pure SCSI passthrough
- add support for larger SG lists by putting them in grant page(s)
- add command abort capability

[PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels

[jgross]

From: Juergen Gross <jgross@suse.com>

Export bind_interdomain_evtchn_to_irq() so drivers can use threaded
interrupt handlers with:

 irq = bind_interdomain_evtchn_to_irq(remote_dom, remote_port);
 if (irq < 0)
     /* error */
 ret = request_threaded_irq(...);

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: David Vrabel <david.vrabel@citrix.com>
---
 drivers/xen/events/events_base.c | 5 +++--
 include/xen/events.h             | 2 ++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
index 5b5c5ff..b4bca2d 100644
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -900,8 +900,8 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)
 	return irq;
 }
 
-static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
-					  unsigned int remote_port)
+int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
+				   unsigned int remote_port)
 {
 	struct evtchn_bind_interdomain bind_interdomain;
 	int err;
@@ -914,6 +914,7 @@ static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
 
 	return err ? : bind_evtchn_to_irq(bind_interdomain.local_port);
 }
+EXPORT_SYMBOL_GPL(bind_interdomain_evtchn_to_irq);
 
 static int find_virq(unsigned int virq, unsigned int cpu)
 {
diff --git a/include/xen/events.h b/include/xen/events.h
index 8bee7a7..5321cd9 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -28,6 +28,8 @@ int bind_ipi_to_irqhandler(enum ipi_vector ipi,
 			   unsigned long irqflags,
 			   const char *devname,
 			   void *dev_id);
+int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
+				   unsigned int remote_port);
 int bind_interdomain_evtchn_to_irqhandler(unsigned int remote_domain,
 					  unsigned int remote_port,
 					  irq_handler_t handler,
-- 
1.8.4.5

Re: [Xen-devel] [PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels

[Konrad Rzeszutek Wilk]

On Mon, Aug 18, 2014 at 11:31:46AM +0200, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Export bind_interdomain_evtchn_to_irq() so drivers can use threaded
> interrupt handlers with:
> 
>  irq = bind_interdomain_evtchn_to_irq(remote_dom, remote_port);
>  if (irq < 0)
>      /* error */
>  ret = request_threaded_irq(...);
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: David Vrabel <david.vrabel@citrix.com>

Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  drivers/xen/events/events_base.c | 5 +++--
>  include/xen/events.h             | 2 ++
>  2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
> index 5b5c5ff..b4bca2d 100644
> --- a/drivers/xen/events/events_base.c
> +++ b/drivers/xen/events/events_base.c
> @@ -900,8 +900,8 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)
>  	return irq;
>  }
>  
> -static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
> -					  unsigned int remote_port)
> +int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
> +				   unsigned int remote_port)
>  {
>  	struct evtchn_bind_interdomain bind_interdomain;
>  	int err;
> @@ -914,6 +914,7 @@ static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
>  
>  	return err ? : bind_evtchn_to_irq(bind_interdomain.local_port);
>  }
> +EXPORT_SYMBOL_GPL(bind_interdomain_evtchn_to_irq);
>  
>  static int find_virq(unsigned int virq, unsigned int cpu)
>  {
> diff --git a/include/xen/events.h b/include/xen/events.h
> index 8bee7a7..5321cd9 100644
> --- a/include/xen/events.h
> +++ b/include/xen/events.h
> @@ -28,6 +28,8 @@ int bind_ipi_to_irqhandler(enum ipi_vector ipi,
>  			   unsigned long irqflags,
>  			   const char *devname,
>  			   void *dev_id);
> +int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
> +				   unsigned int remote_port);
>  int bind_interdomain_evtchn_to_irqhandler(unsigned int remote_domain,
>  					  unsigned int remote_port,
>  					  irq_handler_t handler,
> -- 
> 1.8.4.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels

[Konrad Rzeszutek Wilk]

On Mon, Aug 18, 2014 at 11:31:46AM +0200, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Export bind_interdomain_evtchn_to_irq() so drivers can use threaded
> interrupt handlers with:
> 
>  irq = bind_interdomain_evtchn_to_irq(remote_dom, remote_port);
>  if (irq < 0)
>      /* error */
>  ret = request_threaded_irq(...);
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: David Vrabel <david.vrabel@citrix.com>

Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  drivers/xen/events/events_base.c | 5 +++--
>  include/xen/events.h             | 2 ++
>  2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
> index 5b5c5ff..b4bca2d 100644
> --- a/drivers/xen/events/events_base.c
> +++ b/drivers/xen/events/events_base.c
> @@ -900,8 +900,8 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)
>  	return irq;
>  }
>  
> -static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
> -					  unsigned int remote_port)
> +int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
> +				   unsigned int remote_port)
>  {
>  	struct evtchn_bind_interdomain bind_interdomain;
>  	int err;
> @@ -914,6 +914,7 @@ static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
>  
>  	return err ? : bind_evtchn_to_irq(bind_interdomain.local_port);
>  }
> +EXPORT_SYMBOL_GPL(bind_interdomain_evtchn_to_irq);
>  
>  static int find_virq(unsigned int virq, unsigned int cpu)
>  {
> diff --git a/include/xen/events.h b/include/xen/events.h
> index 8bee7a7..5321cd9 100644
> --- a/include/xen/events.h
> +++ b/include/xen/events.h
> @@ -28,6 +28,8 @@ int bind_ipi_to_irqhandler(enum ipi_vector ipi,
>  			   unsigned long irqflags,
>  			   const char *devname,
>  			   void *dev_id);
> +int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
> +				   unsigned int remote_port);
>  int bind_interdomain_evtchn_to_irqhandler(unsigned int remote_domain,
>  					  unsigned int remote_port,
>  					  irq_handler_t handler,
> -- 
> 1.8.4.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

[PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels

[jgross]

From: Juergen Gross <jgross@suse.com>

Export bind_interdomain_evtchn_to_irq() so drivers can use threaded
interrupt handlers with:

 irq = bind_interdomain_evtchn_to_irq(remote_dom, remote_port);
 if (irq < 0)
     /* error */
 ret = request_threaded_irq(...);

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: David Vrabel <david.vrabel@citrix.com>
---
 drivers/xen/events/events_base.c | 5 +++--
 include/xen/events.h             | 2 ++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
index 5b5c5ff..b4bca2d 100644
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -900,8 +900,8 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)
 	return irq;
 }
 
-static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
-					  unsigned int remote_port)
+int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
+				   unsigned int remote_port)
 {
 	struct evtchn_bind_interdomain bind_interdomain;
 	int err;
@@ -914,6 +914,7 @@ static int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
 
 	return err ? : bind_evtchn_to_irq(bind_interdomain.local_port);
 }
+EXPORT_SYMBOL_GPL(bind_interdomain_evtchn_to_irq);
 
 static int find_virq(unsigned int virq, unsigned int cpu)
 {
diff --git a/include/xen/events.h b/include/xen/events.h
index 8bee7a7..5321cd9 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -28,6 +28,8 @@ int bind_ipi_to_irqhandler(enum ipi_vector ipi,
 			   unsigned long irqflags,
 			   const char *devname,
 			   void *dev_id);
+int bind_interdomain_evtchn_to_irq(unsigned int remote_domain,
+				   unsigned int remote_port);
 int bind_interdomain_evtchn_to_irqhandler(unsigned int remote_domain,
 					  unsigned int remote_port,
 					  irq_handler_t handler,
-- 
1.8.4.5

[PATCH V5 2/5] Add XEN pvSCSI protocol description

[jgross]

From: Juergen Gross <jgross@suse.com>

Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).

This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
Changes are:
- added comment
- adapt to Linux style guide
- add support for larger SG-lists by putting them in an own granted page
- remove stale definitions

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
 1 file changed, 214 insertions(+)
 create mode 100644 include/xen/interface/io/vscsiif.h

diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
new file mode 100644
index 0000000..4291889
--- /dev/null
+++ b/include/xen/interface/io/vscsiif.h
@@ -0,0 +1,214 @@
+/******************************************************************************
+ * vscsiif.h
+ *
+ * Based on the blkif.h code.
+ *
+ * This interface is to be regarded as a stable API between XEN domains
+ * running potentially different Linux kernel versions.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright(c) FUJITSU Limited 2008.
+ */
+
+#ifndef __XEN__PUBLIC_IO_SCSI_H__
+#define __XEN__PUBLIC_IO_SCSI_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ * Front->back notifications: When enqueuing a new request, sending a
+ * notification can be made conditional on req_event (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: When enqueuing a new response, sending a
+ * notification can be made conditional on rsp_event (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ */
+
+/*
+ * Feature and Parameter Negotiation
+ * =================================
+ * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
+ * communicate capabilities and to negotiate operating parameters.  This
+ * section enumerates these nodes which reside in the respective front and
+ * backend portions of the XenStore, following the XenBus convention.
+ *
+ * All data in the XenStore is stored as strings.  Nodes specifying numeric
+ * values are encoded in decimal.  Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formated node string, without loss of information.
+ *
+ * Any specified default value is in effect if the corresponding XenBus node
+ * is not present in the XenStore.
+ *
+ * XenStore nodes in sections marked "PRIVATE" are solely for use by the
+ * driver side whose XenBus tree contains them.
+ *
+ *****************************************************************************
+ *                            Backend XenBus Nodes
+ *****************************************************************************
+ *
+ *------------------ Backend Device Identification (PRIVATE) ------------------
+ *
+ * p-devname
+ *      Values:         string
+ *
+ *      A free string used to identify the physical device (e.g. a disk name).
+ *
+ * p-dev
+ *      Values:         string
+ *
+ *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
+ *      (host, controller, target, lun, all integers), or a WWN (e.g.
+ *      "naa.60014054ac780582").
+ *
+ * v-dev
+ *      Values:         string
+ *
+ *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
+ *      (host, controller, target, lun, all integers).
+ *
+ *--------------------------------- Features ---------------------------------
+ *
+ * feature-sg-grant
+ *      Values:         <uint16_t>
+ *      Default Value:  0
+ *
+ *      Specifies the maximum number of scatter/gather elements in grant pages
+ *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
+ *      SG elements specified directly in the request.
+ *
+ *****************************************************************************
+ *                            Frontend XenBus Nodes
+ *****************************************************************************
+ *
+ *----------------------- Request Transport Parameters -----------------------
+ *
+ * event-channel
+ *      Values:         <uint32_t>
+ *
+ *      The identifier of the Xen event channel used to signal activity
+ *      in the ring buffer.
+ *
+ * ring-ref
+ *      Values:         <uint32_t>
+ *
+ * protocol
+ *      Values:         string (XEN_IO_PROTO_ABI_*)
+ *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
+ *
+ *      The machine ABI rules governing the format of all ring request and
+ *      response structures.
+ */
+
+/* Requests from the frontend to the backend */
+
+/*
+ * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
+ * The target is specified via channel, id and lun.
+ */
+#define VSCSIIF_ACT_SCSI_CDB		1
+
+/*
+ * Request abort of a running operation for the specified target given by
+ * channel, id, lun and the operation's rqid in ref_rqid.
+ */
+#define VSCSIIF_ACT_SCSI_ABORT		2
+
+/*
+ * Request a device reset of the specified target (channel and id).
+ */
+#define VSCSIIF_ACT_SCSI_RESET		3
+
+/*
+ * Preset scatter/gather elements for a following request. Deprecated.
+ * Keeping the define only to avoid usage of the value "4" for other actions.
+ */
+#define VSCSIIF_ACT_SCSI_SG_PRESET	4
+
+/*
+ * Maximum scatter/gather segments per request.
+ *
+ * Considering balance between allocating at least 16 "vscsiif_request"
+ * structures on one page (4096 bytes) and the number of scatter/gather
+ * elements needed, we decided to use 26 as a magic number.
+ *
+ * If "feature-sg-grant" is set, more scatter/gather elements can be specified
+ * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
+ * In this case the vscsiif_request seg elements don't contain references to
+ * the user data, but to the SG elements referencing the user data.
+ */
+#define VSCSIIF_SG_TABLESIZE		26
+
+/*
+ * based on Linux kernel 2.6.18, still valid
+ * Changing these values requires support of multiple protocols via the rings
+ * as "old clients" will blindly use these values and the resulting structure
+ * sizes.
+ */
+#define VSCSIIF_MAX_COMMAND_SIZE	16
+#define VSCSIIF_SENSE_BUFFERSIZE	96
+
+struct scsiif_request_segment {
+	grant_ref_t gref;
+	uint16_t offset;
+	uint16_t length;
+};
+
+/* Size of one request is 252 bytes */
+struct vscsiif_request {
+	uint16_t rqid;		/* private guest value, echoed in resp  */
+	uint8_t act;		/* command between backend and frontend */
+	uint8_t cmd_len;	/* valid CDB bytes */
+
+	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
+	uint16_t timeout_per_command;
+	uint16_t channel, id, lun;	/* (virtual) device specification */
+	uint16_t ref_rqid;		/* command abort reference */
+	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
+					   DMA_FROM_DEVICE(2)
+					   DMA_NONE(3) requests */
+	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
+#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
+					/* nr_segments counts grant pages with
+					   SG elements
+					   usable if "feature-sg-grant" set */
+
+	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
+	uint32_t reserved[3];
+};
+
+struct vscsiif_response {
+	uint16_t rqid;		/* identifies request */
+	uint8_t padding;
+	uint8_t sense_len;
+	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
+	int32_t rslt;
+	uint32_t residual_len;	/* request bufflen -
+				   return the value from physical device */
+	uint32_t reserved[36];
+};
+
+DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
+
+#endif /*__XEN__PUBLIC_IO_SCSI_H__*/
-- 
1.8.4.5

Re: [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Konrad Rzeszutek Wilk]

On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
> XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).
> 
> This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
> Changes are:
> - added comment
> - adapt to Linux style guide
> - add support for larger SG-lists by putting them in an own granted page
> - remove stale definitions
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 214 insertions(+)
>  create mode 100644 include/xen/interface/io/vscsiif.h
> 
> diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
> new file mode 100644
> index 0000000..4291889
> --- /dev/null
> +++ b/include/xen/interface/io/vscsiif.h
> @@ -0,0 +1,214 @@
> +/******************************************************************************
> + * vscsiif.h
> + *
> + * Based on the blkif.h code.
> + *
> + * This interface is to be regarded as a stable API between XEN domains
> + * running potentially different Linux kernel versions.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to
> + * deal in the Software without restriction, including without limitation the
> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
> + * sell copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> + * DEALINGS IN THE SOFTWARE.
> + *
> + * Copyright(c) FUJITSU Limited 2008.
> + */
> +
> +#ifndef __XEN__PUBLIC_IO_SCSI_H__
> +#define __XEN__PUBLIC_IO_SCSI_H__
> +
> +#include "ring.h"
> +#include "../grant_table.h"
> +
> +/*
> + * Front->back notifications: When enqueuing a new request, sending a
> + * notification can be made conditional on req_event (i.e., the generic
> + * hold-off mechanism provided by the ring macros). Backends must set
> + * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
> + *
> + * Back->front notifications: When enqueuing a new response, sending a
> + * notification can be made conditional on rsp_event (i.e., the generic
> + * hold-off mechanism provided by the ring macros). Frontends must set
> + * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
> + */
> +
> +/*
> + * Feature and Parameter Negotiation
> + * =================================
> + * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
> + * communicate capabilities and to negotiate operating parameters.  This
> + * section enumerates these nodes which reside in the respective front and
> + * backend portions of the XenStore, following the XenBus convention.
> + *
> + * All data in the XenStore is stored as strings.  Nodes specifying numeric
> + * values are encoded in decimal.  Integer value ranges listed below are
> + * expressed as fixed sized integer types capable of storing the conversion
> + * of a properly formated node string, without loss of information.
> + *
> + * Any specified default value is in effect if the corresponding XenBus node
> + * is not present in the XenStore.
> + *
> + * XenStore nodes in sections marked "PRIVATE" are solely for use by the
> + * driver side whose XenBus tree contains them.
> + *
> + *****************************************************************************
> + *                            Backend XenBus Nodes
> + *****************************************************************************
> + *
> + *------------------ Backend Device Identification (PRIVATE) ------------------
> + *
> + * p-devname
> + *      Values:         string
> + *
> + *      A free string used to identify the physical device (e.g. a disk name).
> + *
> + * p-dev
> + *      Values:         string
> + *
> + *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
> + *      (host, controller, target, lun, all integers), or a WWN (e.g.
> + *      "naa.60014054ac780582").
> + *
> + * v-dev
> + *      Values:         string
> + *
> + *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
> + *      (host, controller, target, lun, all integers).
> + *
> + *--------------------------------- Features ---------------------------------
> + *
> + * feature-sg-grant
> + *      Values:         <uint16_t>
> + *      Default Value:  0
> + *
> + *      Specifies the maximum number of scatter/gather elements in grant pages
> + *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
> + *      SG elements specified directly in the request.
> + *
> + *****************************************************************************
> + *                            Frontend XenBus Nodes
> + *****************************************************************************
> + *
> + *----------------------- Request Transport Parameters -----------------------
> + *
> + * event-channel
> + *      Values:         <uint32_t>
> + *
> + *      The identifier of the Xen event channel used to signal activity
> + *      in the ring buffer.
> + *
> + * ring-ref
> + *      Values:         <uint32_t>

Should there a bit of explanation here about what 'ring ref' is?

> + *
> + * protocol
> + *      Values:         string (XEN_IO_PROTO_ABI_*)
> + *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
> + *
> + *      The machine ABI rules governing the format of all ring request and
> + *      response structures.
> + */
> +
> +/* Requests from the frontend to the backend */
> +
> +/*
> + * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
> + * The target is specified via channel, id and lun.
> + */
> +#define VSCSIIF_ACT_SCSI_CDB		1
> +
> +/*
> + * Request abort of a running operation for the specified target given by
> + * channel, id, lun and the operation's rqid in ref_rqid.
> + */
> +#define VSCSIIF_ACT_SCSI_ABORT		2
> +
> +/*
> + * Request a device reset of the specified target (channel and id).
> + */
> +#define VSCSIIF_ACT_SCSI_RESET		3
> +
> +/*
> + * Preset scatter/gather elements for a following request. Deprecated.
> + * Keeping the define only to avoid usage of the value "4" for other actions.
> + */
> +#define VSCSIIF_ACT_SCSI_SG_PRESET	4
> +
> +/*
> + * Maximum scatter/gather segments per request.
> + *
> + * Considering balance between allocating at least 16 "vscsiif_request"
> + * structures on one page (4096 bytes) and the number of scatter/gather
> + * elements needed, we decided to use 26 as a magic number.
> + *
> + * If "feature-sg-grant" is set, more scatter/gather elements can be specified
> + * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
> + * In this case the vscsiif_request seg elements don't contain references to
> + * the user data, but to the SG elements referencing the user data.

That sounds like the indirect descriptors that virtio and xen-block has.
More questions about that below.
> + */
> +#define VSCSIIF_SG_TABLESIZE		26
> +
> +/*
> + * based on Linux kernel 2.6.18, still valid
> + * Changing these values requires support of multiple protocols via the rings
> + * as "old clients" will blindly use these values and the resulting structure
> + * sizes.
> + */
> +#define VSCSIIF_MAX_COMMAND_SIZE	16
> +#define VSCSIIF_SENSE_BUFFERSIZE	96
> +
> +struct scsiif_request_segment {
> +	grant_ref_t gref;
> +	uint16_t offset;
> +	uint16_t length;
> +};

Yeey, 8 byte data structure!
> +
> +/* Size of one request is 252 bytes */
> +struct vscsiif_request {
> +	uint16_t rqid;		/* private guest value, echoed in resp  */
> +	uint8_t act;		/* command between backend and frontend */
> +	uint8_t cmd_len;	/* valid CDB bytes */
> +
> +	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
> +	uint16_t timeout_per_command;
> +	uint16_t channel, id, lun;	/* (virtual) device specification */
> +	uint16_t ref_rqid;		/* command abort reference */
> +	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
> +					   DMA_FROM_DEVICE(2)
> +					   DMA_NONE(3) requests */
> +	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
> +#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
> +					/* nr_segments counts grant pages with
> +					   SG elements

Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
count of grant pages with SG elements. Does that mean the req.seg[0].gref points
to an page which will have an array of grant references? And each grant reference
will point to a data page? What about the 'offset' and 'length' of them?

Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
'struct scsiif_request_segment' ? If so, where would the could of those
segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
go? Won't we collide?


> +					   usable if "feature-sg-grant" set */
> +
> +	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
> +	uint32_t reserved[3];

Or is the flag suppose to show up here?
> +};
> +

The previous structure had an comment about the size of the structure.
Should it be here as well?

> +struct vscsiif_response {
> +	uint16_t rqid;		/* identifies request */
> +	uint8_t padding;
> +	uint8_t sense_len;
> +	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
> +	int32_t rslt;
> +	uint32_t residual_len;	/* request bufflen -
> +				   return the value from physical device */
> +	uint32_t reserved[36];
> +};
> +
> +DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
> +
> +#endif /*__XEN__PUBLIC_IO_SCSI_H__*/
> -- 
> 1.8.4.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Konrad Rzeszutek Wilk]

On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
> XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).
> 
> This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
> Changes are:
> - added comment
> - adapt to Linux style guide
> - add support for larger SG-lists by putting them in an own granted page
> - remove stale definitions
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 214 insertions(+)
>  create mode 100644 include/xen/interface/io/vscsiif.h
> 
> diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
> new file mode 100644
> index 0000000..4291889
> --- /dev/null
> +++ b/include/xen/interface/io/vscsiif.h
> @@ -0,0 +1,214 @@
> +/******************************************************************************
> + * vscsiif.h
> + *
> + * Based on the blkif.h code.
> + *
> + * This interface is to be regarded as a stable API between XEN domains
> + * running potentially different Linux kernel versions.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to
> + * deal in the Software without restriction, including without limitation the
> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
> + * sell copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> + * DEALINGS IN THE SOFTWARE.
> + *
> + * Copyright(c) FUJITSU Limited 2008.
> + */
> +
> +#ifndef __XEN__PUBLIC_IO_SCSI_H__
> +#define __XEN__PUBLIC_IO_SCSI_H__
> +
> +#include "ring.h"
> +#include "../grant_table.h"
> +
> +/*
> + * Front->back notifications: When enqueuing a new request, sending a
> + * notification can be made conditional on req_event (i.e., the generic
> + * hold-off mechanism provided by the ring macros). Backends must set
> + * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
> + *
> + * Back->front notifications: When enqueuing a new response, sending a
> + * notification can be made conditional on rsp_event (i.e., the generic
> + * hold-off mechanism provided by the ring macros). Frontends must set
> + * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
> + */
> +
> +/*
> + * Feature and Parameter Negotiation
> + * =================================
> + * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
> + * communicate capabilities and to negotiate operating parameters.  This
> + * section enumerates these nodes which reside in the respective front and
> + * backend portions of the XenStore, following the XenBus convention.
> + *
> + * All data in the XenStore is stored as strings.  Nodes specifying numeric
> + * values are encoded in decimal.  Integer value ranges listed below are
> + * expressed as fixed sized integer types capable of storing the conversion
> + * of a properly formated node string, without loss of information.
> + *
> + * Any specified default value is in effect if the corresponding XenBus node
> + * is not present in the XenStore.
> + *
> + * XenStore nodes in sections marked "PRIVATE" are solely for use by the
> + * driver side whose XenBus tree contains them.
> + *
> + *****************************************************************************
> + *                            Backend XenBus Nodes
> + *****************************************************************************
> + *
> + *------------------ Backend Device Identification (PRIVATE) ------------------
> + *
> + * p-devname
> + *      Values:         string
> + *
> + *      A free string used to identify the physical device (e.g. a disk name).
> + *
> + * p-dev
> + *      Values:         string
> + *
> + *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
> + *      (host, controller, target, lun, all integers), or a WWN (e.g.
> + *      "naa.60014054ac780582").
> + *
> + * v-dev
> + *      Values:         string
> + *
> + *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
> + *      (host, controller, target, lun, all integers).
> + *
> + *--------------------------------- Features ---------------------------------
> + *
> + * feature-sg-grant
> + *      Values:         <uint16_t>
> + *      Default Value:  0
> + *
> + *      Specifies the maximum number of scatter/gather elements in grant pages
> + *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
> + *      SG elements specified directly in the request.
> + *
> + *****************************************************************************
> + *                            Frontend XenBus Nodes
> + *****************************************************************************
> + *
> + *----------------------- Request Transport Parameters -----------------------
> + *
> + * event-channel
> + *      Values:         <uint32_t>
> + *
> + *      The identifier of the Xen event channel used to signal activity
> + *      in the ring buffer.
> + *
> + * ring-ref
> + *      Values:         <uint32_t>

Should there a bit of explanation here about what 'ring ref' is?

> + *
> + * protocol
> + *      Values:         string (XEN_IO_PROTO_ABI_*)
> + *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
> + *
> + *      The machine ABI rules governing the format of all ring request and
> + *      response structures.
> + */
> +
> +/* Requests from the frontend to the backend */
> +
> +/*
> + * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
> + * The target is specified via channel, id and lun.
> + */
> +#define VSCSIIF_ACT_SCSI_CDB		1
> +
> +/*
> + * Request abort of a running operation for the specified target given by
> + * channel, id, lun and the operation's rqid in ref_rqid.
> + */
> +#define VSCSIIF_ACT_SCSI_ABORT		2
> +
> +/*
> + * Request a device reset of the specified target (channel and id).
> + */
> +#define VSCSIIF_ACT_SCSI_RESET		3
> +
> +/*
> + * Preset scatter/gather elements for a following request. Deprecated.
> + * Keeping the define only to avoid usage of the value "4" for other actions.
> + */
> +#define VSCSIIF_ACT_SCSI_SG_PRESET	4
> +
> +/*
> + * Maximum scatter/gather segments per request.
> + *
> + * Considering balance between allocating at least 16 "vscsiif_request"
> + * structures on one page (4096 bytes) and the number of scatter/gather
> + * elements needed, we decided to use 26 as a magic number.
> + *
> + * If "feature-sg-grant" is set, more scatter/gather elements can be specified
> + * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
> + * In this case the vscsiif_request seg elements don't contain references to
> + * the user data, but to the SG elements referencing the user data.

That sounds like the indirect descriptors that virtio and xen-block has.
More questions about that below.
> + */
> +#define VSCSIIF_SG_TABLESIZE		26
> +
> +/*
> + * based on Linux kernel 2.6.18, still valid
> + * Changing these values requires support of multiple protocols via the rings
> + * as "old clients" will blindly use these values and the resulting structure
> + * sizes.
> + */
> +#define VSCSIIF_MAX_COMMAND_SIZE	16
> +#define VSCSIIF_SENSE_BUFFERSIZE	96
> +
> +struct scsiif_request_segment {
> +	grant_ref_t gref;
> +	uint16_t offset;
> +	uint16_t length;
> +};

Yeey, 8 byte data structure!
> +
> +/* Size of one request is 252 bytes */
> +struct vscsiif_request {
> +	uint16_t rqid;		/* private guest value, echoed in resp  */
> +	uint8_t act;		/* command between backend and frontend */
> +	uint8_t cmd_len;	/* valid CDB bytes */
> +
> +	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
> +	uint16_t timeout_per_command;
> +	uint16_t channel, id, lun;	/* (virtual) device specification */
> +	uint16_t ref_rqid;		/* command abort reference */
> +	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
> +					   DMA_FROM_DEVICE(2)
> +					   DMA_NONE(3) requests */
> +	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
> +#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
> +					/* nr_segments counts grant pages with
> +					   SG elements

Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
count of grant pages with SG elements. Does that mean the req.seg[0].gref points
to an page which will have an array of grant references? And each grant reference
will point to a data page? What about the 'offset' and 'length' of them?

Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
'struct scsiif_request_segment' ? If so, where would the could of those
segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
go? Won't we collide?


> +					   usable if "feature-sg-grant" set */
> +
> +	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
> +	uint32_t reserved[3];

Or is the flag suppose to show up here?
> +};
> +

The previous structure had an comment about the size of the structure.
Should it be here as well?

> +struct vscsiif_response {
> +	uint16_t rqid;		/* identifies request */
> +	uint8_t padding;
> +	uint8_t sense_len;
> +	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
> +	int32_t rslt;
> +	uint32_t residual_len;	/* request bufflen -
> +				   return the value from physical device */
> +	uint32_t reserved[36];
> +};
> +
> +DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
> +
> +#endif /*__XEN__PUBLIC_IO_SCSI_H__*/
> -- 
> 1.8.4.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Juergen Gross]

On 08/20/2014 03:25 PM, Konrad Rzeszutek Wilk wrote:
> On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
>> From: Juergen Gross <jgross@suse.com>
>>
>> Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
>> XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).
>>
>> This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
>> Changes are:
>> - added comment
>> - adapt to Linux style guide
>> - add support for larger SG-lists by putting them in an own granted page
>> - remove stale definitions
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
>>   1 file changed, 214 insertions(+)
>>   create mode 100644 include/xen/interface/io/vscsiif.h
>>
>> diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
>> new file mode 100644
>> index 0000000..4291889
>> --- /dev/null
>> +++ b/include/xen/interface/io/vscsiif.h
>> @@ -0,0 +1,214 @@
>> +/******************************************************************************
>> + * vscsiif.h
>> + *
>> + * Based on the blkif.h code.
>> + *
>> + * This interface is to be regarded as a stable API between XEN domains
>> + * running potentially different Linux kernel versions.
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to
>> + * deal in the Software without restriction, including without limitation the
>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>> + * sell copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>> + * DEALINGS IN THE SOFTWARE.
>> + *
>> + * Copyright(c) FUJITSU Limited 2008.
>> + */
>> +
>> +#ifndef __XEN__PUBLIC_IO_SCSI_H__
>> +#define __XEN__PUBLIC_IO_SCSI_H__
>> +
>> +#include "ring.h"
>> +#include "../grant_table.h"
>> +
>> +/*
>> + * Front->back notifications: When enqueuing a new request, sending a
>> + * notification can be made conditional on req_event (i.e., the generic
>> + * hold-off mechanism provided by the ring macros). Backends must set
>> + * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
>> + *
>> + * Back->front notifications: When enqueuing a new response, sending a
>> + * notification can be made conditional on rsp_event (i.e., the generic
>> + * hold-off mechanism provided by the ring macros). Frontends must set
>> + * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
>> + */
>> +
>> +/*
>> + * Feature and Parameter Negotiation
>> + * =================================
>> + * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
>> + * communicate capabilities and to negotiate operating parameters.  This
>> + * section enumerates these nodes which reside in the respective front and
>> + * backend portions of the XenStore, following the XenBus convention.
>> + *
>> + * All data in the XenStore is stored as strings.  Nodes specifying numeric
>> + * values are encoded in decimal.  Integer value ranges listed below are
>> + * expressed as fixed sized integer types capable of storing the conversion
>> + * of a properly formated node string, without loss of information.
>> + *
>> + * Any specified default value is in effect if the corresponding XenBus node
>> + * is not present in the XenStore.
>> + *
>> + * XenStore nodes in sections marked "PRIVATE" are solely for use by the
>> + * driver side whose XenBus tree contains them.
>> + *
>> + *****************************************************************************
>> + *                            Backend XenBus Nodes
>> + *****************************************************************************
>> + *
>> + *------------------ Backend Device Identification (PRIVATE) ------------------
>> + *
>> + * p-devname
>> + *      Values:         string
>> + *
>> + *      A free string used to identify the physical device (e.g. a disk name).
>> + *
>> + * p-dev
>> + *      Values:         string
>> + *
>> + *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
>> + *      (host, controller, target, lun, all integers), or a WWN (e.g.
>> + *      "naa.60014054ac780582").
>> + *
>> + * v-dev
>> + *      Values:         string
>> + *
>> + *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
>> + *      (host, controller, target, lun, all integers).
>> + *
>> + *--------------------------------- Features ---------------------------------
>> + *
>> + * feature-sg-grant
>> + *      Values:         <uint16_t>
>> + *      Default Value:  0
>> + *
>> + *      Specifies the maximum number of scatter/gather elements in grant pages
>> + *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
>> + *      SG elements specified directly in the request.
>> + *
>> + *****************************************************************************
>> + *                            Frontend XenBus Nodes
>> + *****************************************************************************
>> + *
>> + *----------------------- Request Transport Parameters -----------------------
>> + *
>> + * event-channel
>> + *      Values:         <uint32_t>
>> + *
>> + *      The identifier of the Xen event channel used to signal activity
>> + *      in the ring buffer.
>> + *
>> + * ring-ref
>> + *      Values:         <uint32_t>
>
> Should there a bit of explanation here about what 'ring ref' is?

Uuh, yes. I'll add it.

>
>> + *
>> + * protocol
>> + *      Values:         string (XEN_IO_PROTO_ABI_*)
>> + *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
>> + *
>> + *      The machine ABI rules governing the format of all ring request and
>> + *      response structures.
>> + */
>> +
>> +/* Requests from the frontend to the backend */
>> +
>> +/*
>> + * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
>> + * The target is specified via channel, id and lun.
>> + */
>> +#define VSCSIIF_ACT_SCSI_CDB		1
>> +
>> +/*
>> + * Request abort of a running operation for the specified target given by
>> + * channel, id, lun and the operation's rqid in ref_rqid.
>> + */
>> +#define VSCSIIF_ACT_SCSI_ABORT		2
>> +
>> +/*
>> + * Request a device reset of the specified target (channel and id).
>> + */
>> +#define VSCSIIF_ACT_SCSI_RESET		3
>> +
>> +/*
>> + * Preset scatter/gather elements for a following request. Deprecated.
>> + * Keeping the define only to avoid usage of the value "4" for other actions.
>> + */
>> +#define VSCSIIF_ACT_SCSI_SG_PRESET	4
>> +
>> +/*
>> + * Maximum scatter/gather segments per request.
>> + *
>> + * Considering balance between allocating at least 16 "vscsiif_request"
>> + * structures on one page (4096 bytes) and the number of scatter/gather
>> + * elements needed, we decided to use 26 as a magic number.
>> + *
>> + * If "feature-sg-grant" is set, more scatter/gather elements can be specified
>> + * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
>> + * In this case the vscsiif_request seg elements don't contain references to
>> + * the user data, but to the SG elements referencing the user data.
>
> That sounds like the indirect descriptors that virtio and xen-block has.
> More questions about that below.
>> + */
>> +#define VSCSIIF_SG_TABLESIZE		26
>> +
>> +/*
>> + * based on Linux kernel 2.6.18, still valid
>> + * Changing these values requires support of multiple protocols via the rings
>> + * as "old clients" will blindly use these values and the resulting structure
>> + * sizes.
>> + */
>> +#define VSCSIIF_MAX_COMMAND_SIZE	16
>> +#define VSCSIIF_SENSE_BUFFERSIZE	96
>> +
>> +struct scsiif_request_segment {
>> +	grant_ref_t gref;
>> +	uint16_t offset;
>> +	uint16_t length;
>> +};
>
> Yeey, 8 byte data structure!
>> +
>> +/* Size of one request is 252 bytes */
>> +struct vscsiif_request {
>> +	uint16_t rqid;		/* private guest value, echoed in resp  */
>> +	uint8_t act;		/* command between backend and frontend */
>> +	uint8_t cmd_len;	/* valid CDB bytes */
>> +
>> +	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
>> +	uint16_t timeout_per_command;
>> +	uint16_t channel, id, lun;	/* (virtual) device specification */
>> +	uint16_t ref_rqid;		/* command abort reference */
>> +	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
>> +					   DMA_FROM_DEVICE(2)
>> +					   DMA_NONE(3) requests */
>> +	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
>> +#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
>> +					/* nr_segments counts grant pages with
>> +					   SG elements
>
> Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
> count of grant pages with SG elements. Does that mean the req.seg[0].gref points
> to an page which will have an array of grant references? And each grant reference
> will point to a data page? What about the 'offset' and 'length' of them?
>
> Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
> 'struct scsiif_request_segment' ? If so, where would the could of those
> segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
> go? Won't we collide?

nr_segments (without the VSCSIIF_SG_GRANT bit) always counts the number
of populated seg[] entries. If VSCSIIF_SG_GRANT is set, each seg[] entry
references another array of struct scsiif_request_segment using a grant
ref, offset into the granted page and a length of the array in bytes.

The resulting number of struct scsiif_request_segment is the sum of
seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).

>
>
>> +					   usable if "feature-sg-grant" set */
>> +
>> +	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
>> +	uint32_t reserved[3];
>
> Or is the flag suppose to show up here?

No. There is no guarantee an "old" domU not aware of "feature-sg-grant"
will have set reserved[] to zero.

>> +};
>> +
>
> The previous structure had an comment about the size of the structure.
> Should it be here as well?

I can add one.

>
>> +struct vscsiif_response {
>> +	uint16_t rqid;		/* identifies request */
>> +	uint8_t padding;
>> +	uint8_t sense_len;
>> +	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
>> +	int32_t rslt;
>> +	uint32_t residual_len;	/* request bufflen -
>> +				   return the value from physical device */
>> +	uint32_t reserved[36];
>> +};
>> +
>> +DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
>> +
>> +#endif /*__XEN__PUBLIC_IO_SCSI_H__*/

Re: [Xen-devel] [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Juergen Gross]

On 08/20/2014 03:25 PM, Konrad Rzeszutek Wilk wrote:
> On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
>> From: Juergen Gross <jgross@suse.com>
>>
>> Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
>> XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).
>>
>> This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
>> Changes are:
>> - added comment
>> - adapt to Linux style guide
>> - add support for larger SG-lists by putting them in an own granted page
>> - remove stale definitions
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
>>   1 file changed, 214 insertions(+)
>>   create mode 100644 include/xen/interface/io/vscsiif.h
>>
>> diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
>> new file mode 100644
>> index 0000000..4291889
>> --- /dev/null
>> +++ b/include/xen/interface/io/vscsiif.h
>> @@ -0,0 +1,214 @@
>> +/******************************************************************************
>> + * vscsiif.h
>> + *
>> + * Based on the blkif.h code.
>> + *
>> + * This interface is to be regarded as a stable API between XEN domains
>> + * running potentially different Linux kernel versions.
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to
>> + * deal in the Software without restriction, including without limitation the
>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>> + * sell copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>> + * DEALINGS IN THE SOFTWARE.
>> + *
>> + * Copyright(c) FUJITSU Limited 2008.
>> + */
>> +
>> +#ifndef __XEN__PUBLIC_IO_SCSI_H__
>> +#define __XEN__PUBLIC_IO_SCSI_H__
>> +
>> +#include "ring.h"
>> +#include "../grant_table.h"
>> +
>> +/*
>> + * Front->back notifications: When enqueuing a new request, sending a
>> + * notification can be made conditional on req_event (i.e., the generic
>> + * hold-off mechanism provided by the ring macros). Backends must set
>> + * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
>> + *
>> + * Back->front notifications: When enqueuing a new response, sending a
>> + * notification can be made conditional on rsp_event (i.e., the generic
>> + * hold-off mechanism provided by the ring macros). Frontends must set
>> + * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
>> + */
>> +
>> +/*
>> + * Feature and Parameter Negotiation
>> + * =================================
>> + * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
>> + * communicate capabilities and to negotiate operating parameters.  This
>> + * section enumerates these nodes which reside in the respective front and
>> + * backend portions of the XenStore, following the XenBus convention.
>> + *
>> + * All data in the XenStore is stored as strings.  Nodes specifying numeric
>> + * values are encoded in decimal.  Integer value ranges listed below are
>> + * expressed as fixed sized integer types capable of storing the conversion
>> + * of a properly formated node string, without loss of information.
>> + *
>> + * Any specified default value is in effect if the corresponding XenBus node
>> + * is not present in the XenStore.
>> + *
>> + * XenStore nodes in sections marked "PRIVATE" are solely for use by the
>> + * driver side whose XenBus tree contains them.
>> + *
>> + *****************************************************************************
>> + *                            Backend XenBus Nodes
>> + *****************************************************************************
>> + *
>> + *------------------ Backend Device Identification (PRIVATE) ------------------
>> + *
>> + * p-devname
>> + *      Values:         string
>> + *
>> + *      A free string used to identify the physical device (e.g. a disk name).
>> + *
>> + * p-dev
>> + *      Values:         string
>> + *
>> + *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
>> + *      (host, controller, target, lun, all integers), or a WWN (e.g.
>> + *      "naa.60014054ac780582").
>> + *
>> + * v-dev
>> + *      Values:         string
>> + *
>> + *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
>> + *      (host, controller, target, lun, all integers).
>> + *
>> + *--------------------------------- Features ---------------------------------
>> + *
>> + * feature-sg-grant
>> + *      Values:         <uint16_t>
>> + *      Default Value:  0
>> + *
>> + *      Specifies the maximum number of scatter/gather elements in grant pages
>> + *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
>> + *      SG elements specified directly in the request.
>> + *
>> + *****************************************************************************
>> + *                            Frontend XenBus Nodes
>> + *****************************************************************************
>> + *
>> + *----------------------- Request Transport Parameters -----------------------
>> + *
>> + * event-channel
>> + *      Values:         <uint32_t>
>> + *
>> + *      The identifier of the Xen event channel used to signal activity
>> + *      in the ring buffer.
>> + *
>> + * ring-ref
>> + *      Values:         <uint32_t>
>
> Should there a bit of explanation here about what 'ring ref' is?

Uuh, yes. I'll add it.

>
>> + *
>> + * protocol
>> + *      Values:         string (XEN_IO_PROTO_ABI_*)
>> + *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
>> + *
>> + *      The machine ABI rules governing the format of all ring request and
>> + *      response structures.
>> + */
>> +
>> +/* Requests from the frontend to the backend */
>> +
>> +/*
>> + * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
>> + * The target is specified via channel, id and lun.
>> + */
>> +#define VSCSIIF_ACT_SCSI_CDB		1
>> +
>> +/*
>> + * Request abort of a running operation for the specified target given by
>> + * channel, id, lun and the operation's rqid in ref_rqid.
>> + */
>> +#define VSCSIIF_ACT_SCSI_ABORT		2
>> +
>> +/*
>> + * Request a device reset of the specified target (channel and id).
>> + */
>> +#define VSCSIIF_ACT_SCSI_RESET		3
>> +
>> +/*
>> + * Preset scatter/gather elements for a following request. Deprecated.
>> + * Keeping the define only to avoid usage of the value "4" for other actions.
>> + */
>> +#define VSCSIIF_ACT_SCSI_SG_PRESET	4
>> +
>> +/*
>> + * Maximum scatter/gather segments per request.
>> + *
>> + * Considering balance between allocating at least 16 "vscsiif_request"
>> + * structures on one page (4096 bytes) and the number of scatter/gather
>> + * elements needed, we decided to use 26 as a magic number.
>> + *
>> + * If "feature-sg-grant" is set, more scatter/gather elements can be specified
>> + * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
>> + * In this case the vscsiif_request seg elements don't contain references to
>> + * the user data, but to the SG elements referencing the user data.
>
> That sounds like the indirect descriptors that virtio and xen-block has.
> More questions about that below.
>> + */
>> +#define VSCSIIF_SG_TABLESIZE		26
>> +
>> +/*
>> + * based on Linux kernel 2.6.18, still valid
>> + * Changing these values requires support of multiple protocols via the rings
>> + * as "old clients" will blindly use these values and the resulting structure
>> + * sizes.
>> + */
>> +#define VSCSIIF_MAX_COMMAND_SIZE	16
>> +#define VSCSIIF_SENSE_BUFFERSIZE	96
>> +
>> +struct scsiif_request_segment {
>> +	grant_ref_t gref;
>> +	uint16_t offset;
>> +	uint16_t length;
>> +};
>
> Yeey, 8 byte data structure!
>> +
>> +/* Size of one request is 252 bytes */
>> +struct vscsiif_request {
>> +	uint16_t rqid;		/* private guest value, echoed in resp  */
>> +	uint8_t act;		/* command between backend and frontend */
>> +	uint8_t cmd_len;	/* valid CDB bytes */
>> +
>> +	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
>> +	uint16_t timeout_per_command;
>> +	uint16_t channel, id, lun;	/* (virtual) device specification */
>> +	uint16_t ref_rqid;		/* command abort reference */
>> +	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
>> +					   DMA_FROM_DEVICE(2)
>> +					   DMA_NONE(3) requests */
>> +	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
>> +#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
>> +					/* nr_segments counts grant pages with
>> +					   SG elements
>
> Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
> count of grant pages with SG elements. Does that mean the req.seg[0].gref points
> to an page which will have an array of grant references? And each grant reference
> will point to a data page? What about the 'offset' and 'length' of them?
>
> Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
> 'struct scsiif_request_segment' ? If so, where would the could of those
> segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
> go? Won't we collide?

nr_segments (without the VSCSIIF_SG_GRANT bit) always counts the number
of populated seg[] entries. If VSCSIIF_SG_GRANT is set, each seg[] entry
references another array of struct scsiif_request_segment using a grant
ref, offset into the granted page and a length of the array in bytes.

The resulting number of struct scsiif_request_segment is the sum of
seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).

>
>
>> +					   usable if "feature-sg-grant" set */
>> +
>> +	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
>> +	uint32_t reserved[3];
>
> Or is the flag suppose to show up here?

No. There is no guarantee an "old" domU not aware of "feature-sg-grant"
will have set reserved[] to zero.

>> +};
>> +
>
> The previous structure had an comment about the size of the structure.
> Should it be here as well?

I can add one.

>
>> +struct vscsiif_response {
>> +	uint16_t rqid;		/* identifies request */
>> +	uint8_t padding;
>> +	uint8_t sense_len;
>> +	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
>> +	int32_t rslt;
>> +	uint32_t residual_len;	/* request bufflen -
>> +				   return the value from physical device */
>> +	uint32_t reserved[36];
>> +};
>> +
>> +DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
>> +
>> +#endif /*__XEN__PUBLIC_IO_SCSI_H__*/

Re: [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Konrad Rzeszutek Wilk]

On Wed, Aug 20, 2014 at 04:01:57PM +0200, Juergen Gross wrote:
> On 08/20/2014 03:25 PM, Konrad Rzeszutek Wilk wrote:
> >On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
> >>From: Juergen Gross <jgross@suse.com>
> >>
> >>Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
> >>XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).
> >>
> >>This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
> >>Changes are:
> >>- added comment
> >>- adapt to Linux style guide
> >>- add support for larger SG-lists by putting them in an own granted page
> >>- remove stale definitions
> >>
> >>Signed-off-by: Juergen Gross <jgross@suse.com>
> >>---
> >>  include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
> >>  1 file changed, 214 insertions(+)
> >>  create mode 100644 include/xen/interface/io/vscsiif.h
> >>
> >>diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
> >>new file mode 100644
> >>index 0000000..4291889
> >>--- /dev/null
> >>+++ b/include/xen/interface/io/vscsiif.h
> >>@@ -0,0 +1,214 @@
> >>+/******************************************************************************
> >>+ * vscsiif.h
> >>+ *
> >>+ * Based on the blkif.h code.
> >>+ *
> >>+ * This interface is to be regarded as a stable API between XEN domains
> >>+ * running potentially different Linux kernel versions.
> >>+ *
> >>+ * Permission is hereby granted, free of charge, to any person obtaining a copy
> >>+ * of this software and associated documentation files (the "Software"), to
> >>+ * deal in the Software without restriction, including without limitation the
> >>+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
> >>+ * sell copies of the Software, and to permit persons to whom the Software is
> >>+ * furnished to do so, subject to the following conditions:
> >>+ *
> >>+ * The above copyright notice and this permission notice shall be included in
> >>+ * all copies or substantial portions of the Software.
> >>+ *
> >>+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> >>+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> >>+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> >>+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> >>+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> >>+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> >>+ * DEALINGS IN THE SOFTWARE.
> >>+ *
> >>+ * Copyright(c) FUJITSU Limited 2008.
> >>+ */
> >>+
> >>+#ifndef __XEN__PUBLIC_IO_SCSI_H__
> >>+#define __XEN__PUBLIC_IO_SCSI_H__
> >>+
> >>+#include "ring.h"
> >>+#include "../grant_table.h"
> >>+
> >>+/*
> >>+ * Front->back notifications: When enqueuing a new request, sending a
> >>+ * notification can be made conditional on req_event (i.e., the generic
> >>+ * hold-off mechanism provided by the ring macros). Backends must set
> >>+ * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
> >>+ *
> >>+ * Back->front notifications: When enqueuing a new response, sending a
> >>+ * notification can be made conditional on rsp_event (i.e., the generic
> >>+ * hold-off mechanism provided by the ring macros). Frontends must set
> >>+ * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
> >>+ */
> >>+
> >>+/*
> >>+ * Feature and Parameter Negotiation
> >>+ * =================================
> >>+ * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
> >>+ * communicate capabilities and to negotiate operating parameters.  This
> >>+ * section enumerates these nodes which reside in the respective front and
> >>+ * backend portions of the XenStore, following the XenBus convention.
> >>+ *
> >>+ * All data in the XenStore is stored as strings.  Nodes specifying numeric
> >>+ * values are encoded in decimal.  Integer value ranges listed below are
> >>+ * expressed as fixed sized integer types capable of storing the conversion
> >>+ * of a properly formated node string, without loss of information.
> >>+ *
> >>+ * Any specified default value is in effect if the corresponding XenBus node
> >>+ * is not present in the XenStore.
> >>+ *
> >>+ * XenStore nodes in sections marked "PRIVATE" are solely for use by the
> >>+ * driver side whose XenBus tree contains them.
> >>+ *
> >>+ *****************************************************************************
> >>+ *                            Backend XenBus Nodes
> >>+ *****************************************************************************
> >>+ *
> >>+ *------------------ Backend Device Identification (PRIVATE) ------------------
> >>+ *
> >>+ * p-devname
> >>+ *      Values:         string
> >>+ *
> >>+ *      A free string used to identify the physical device (e.g. a disk name).
> >>+ *
> >>+ * p-dev
> >>+ *      Values:         string
> >>+ *
> >>+ *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
> >>+ *      (host, controller, target, lun, all integers), or a WWN (e.g.
> >>+ *      "naa.60014054ac780582").
> >>+ *
> >>+ * v-dev
> >>+ *      Values:         string
> >>+ *
> >>+ *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
> >>+ *      (host, controller, target, lun, all integers).
> >>+ *
> >>+ *--------------------------------- Features ---------------------------------
> >>+ *
> >>+ * feature-sg-grant
> >>+ *      Values:         <uint16_t>
> >>+ *      Default Value:  0
> >>+ *
> >>+ *      Specifies the maximum number of scatter/gather elements in grant pages
> >>+ *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
> >>+ *      SG elements specified directly in the request.
> >>+ *
> >>+ *****************************************************************************
> >>+ *                            Frontend XenBus Nodes
> >>+ *****************************************************************************
> >>+ *
> >>+ *----------------------- Request Transport Parameters -----------------------
> >>+ *
> >>+ * event-channel
> >>+ *      Values:         <uint32_t>
> >>+ *
> >>+ *      The identifier of the Xen event channel used to signal activity
> >>+ *      in the ring buffer.
> >>+ *
> >>+ * ring-ref
> >>+ *      Values:         <uint32_t>
> >
> >Should there a bit of explanation here about what 'ring ref' is?
> 
> Uuh, yes. I'll add it.
> 
> >
> >>+ *
> >>+ * protocol
> >>+ *      Values:         string (XEN_IO_PROTO_ABI_*)
> >>+ *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
> >>+ *
> >>+ *      The machine ABI rules governing the format of all ring request and
> >>+ *      response structures.
> >>+ */
> >>+
> >>+/* Requests from the frontend to the backend */
> >>+
> >>+/*
> >>+ * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
> >>+ * The target is specified via channel, id and lun.
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_CDB		1
> >>+
> >>+/*
> >>+ * Request abort of a running operation for the specified target given by
> >>+ * channel, id, lun and the operation's rqid in ref_rqid.
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_ABORT		2
> >>+
> >>+/*
> >>+ * Request a device reset of the specified target (channel and id).
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_RESET		3
> >>+
> >>+/*
> >>+ * Preset scatter/gather elements for a following request. Deprecated.
> >>+ * Keeping the define only to avoid usage of the value "4" for other actions.
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_SG_PRESET	4
> >>+
> >>+/*
> >>+ * Maximum scatter/gather segments per request.
> >>+ *
> >>+ * Considering balance between allocating at least 16 "vscsiif_request"
> >>+ * structures on one page (4096 bytes) and the number of scatter/gather
> >>+ * elements needed, we decided to use 26 as a magic number.
> >>+ *
> >>+ * If "feature-sg-grant" is set, more scatter/gather elements can be specified
> >>+ * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
> >>+ * In this case the vscsiif_request seg elements don't contain references to
> >>+ * the user data, but to the SG elements referencing the user data.
> >
> >That sounds like the indirect descriptors that virtio and xen-block has.
> >More questions about that below.
> >>+ */
> >>+#define VSCSIIF_SG_TABLESIZE		26
> >>+
> >>+/*
> >>+ * based on Linux kernel 2.6.18, still valid
> >>+ * Changing these values requires support of multiple protocols via the rings
> >>+ * as "old clients" will blindly use these values and the resulting structure
> >>+ * sizes.
> >>+ */
> >>+#define VSCSIIF_MAX_COMMAND_SIZE	16
> >>+#define VSCSIIF_SENSE_BUFFERSIZE	96
> >>+
> >>+struct scsiif_request_segment {
> >>+	grant_ref_t gref;
> >>+	uint16_t offset;
> >>+	uint16_t length;
> >>+};
> >
> >Yeey, 8 byte data structure!
> >>+
> >>+/* Size of one request is 252 bytes */
> >>+struct vscsiif_request {
> >>+	uint16_t rqid;		/* private guest value, echoed in resp  */
> >>+	uint8_t act;		/* command between backend and frontend */
> >>+	uint8_t cmd_len;	/* valid CDB bytes */
> >>+
> >>+	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
> >>+	uint16_t timeout_per_command;
> >>+	uint16_t channel, id, lun;	/* (virtual) device specification */
> >>+	uint16_t ref_rqid;		/* command abort reference */
> >>+	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
> >>+					   DMA_FROM_DEVICE(2)
> >>+					   DMA_NONE(3) requests */
> >>+	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
> >>+#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
> >>+					/* nr_segments counts grant pages with
> >>+					   SG elements
> >
> >Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
> >count of grant pages with SG elements. Does that mean the req.seg[0].gref points
> >to an page which will have an array of grant references? And each grant reference
> >will point to a data page? What about the 'offset' and 'length' of them?
> >
> >Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
> >'struct scsiif_request_segment' ? If so, where would the could of those
> >segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
> >go? Won't we collide?
> 
> nr_segments (without the VSCSIIF_SG_GRANT bit) always counts the number
> of populated seg[] entries. If VSCSIIF_SG_GRANT is set, each seg[] entry
> references another array of struct scsiif_request_segment using a grant
> ref, offset into the granted page and a length of the array in bytes.

Shouldn't there be a comment about that? The blkif.h has a pretty lengthy
one when it comes to indirect descriptors that could be copied as it
sounds exactly like the same thing.
> 
> The resulting number of struct scsiif_request_segment is the sum of
> seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).
> 

Where the nr_segments can only go Up to VSCSIIF_SG_TABLESIZE, so the max
total SG entries you can is 13312 ( 4096 / 8 = 512 max per page, times
26).
> >
> >
> >>+					   usable if "feature-sg-grant" set */
> >>+
> >>+	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
> >>+	uint32_t reserved[3];
> >
> >Or is the flag suppose to show up here?
> 
> No. There is no guarantee an "old" domU not aware of "feature-sg-grant"
> will have set reserved[] to zero.

<nods>
> 
> >>+};
> >>+
> >
> >The previous structure had an comment about the size of the structure.
> >Should it be here as well?
> 
> I can add one.

Thank you!
> 
> >
> >>+struct vscsiif_response {
> >>+	uint16_t rqid;		/* identifies request */
> >>+	uint8_t padding;
> >>+	uint8_t sense_len;
> >>+	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
> >>+	int32_t rslt;
> >>+	uint32_t residual_len;	/* request bufflen -
> >>+				   return the value from physical device */
> >>+	uint32_t reserved[36];
> >>+};
> >>+
> >>+DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
> >>+
> >>+#endif /*__XEN__PUBLIC_IO_SCSI_H__*/
> 

Re: [Xen-devel] [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Konrad Rzeszutek Wilk]

On Wed, Aug 20, 2014 at 04:01:57PM +0200, Juergen Gross wrote:
> On 08/20/2014 03:25 PM, Konrad Rzeszutek Wilk wrote:
> >On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
> >>From: Juergen Gross <jgross@suse.com>
> >>
> >>Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
> >>XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).
> >>
> >>This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
> >>Changes are:
> >>- added comment
> >>- adapt to Linux style guide
> >>- add support for larger SG-lists by putting them in an own granted page
> >>- remove stale definitions
> >>
> >>Signed-off-by: Juergen Gross <jgross@suse.com>
> >>---
> >>  include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
> >>  1 file changed, 214 insertions(+)
> >>  create mode 100644 include/xen/interface/io/vscsiif.h
> >>
> >>diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
> >>new file mode 100644
> >>index 0000000..4291889
> >>--- /dev/null
> >>+++ b/include/xen/interface/io/vscsiif.h
> >>@@ -0,0 +1,214 @@
> >>+/******************************************************************************
> >>+ * vscsiif.h
> >>+ *
> >>+ * Based on the blkif.h code.
> >>+ *
> >>+ * This interface is to be regarded as a stable API between XEN domains
> >>+ * running potentially different Linux kernel versions.
> >>+ *
> >>+ * Permission is hereby granted, free of charge, to any person obtaining a copy
> >>+ * of this software and associated documentation files (the "Software"), to
> >>+ * deal in the Software without restriction, including without limitation the
> >>+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
> >>+ * sell copies of the Software, and to permit persons to whom the Software is
> >>+ * furnished to do so, subject to the following conditions:
> >>+ *
> >>+ * The above copyright notice and this permission notice shall be included in
> >>+ * all copies or substantial portions of the Software.
> >>+ *
> >>+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> >>+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> >>+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> >>+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> >>+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> >>+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> >>+ * DEALINGS IN THE SOFTWARE.
> >>+ *
> >>+ * Copyright(c) FUJITSU Limited 2008.
> >>+ */
> >>+
> >>+#ifndef __XEN__PUBLIC_IO_SCSI_H__
> >>+#define __XEN__PUBLIC_IO_SCSI_H__
> >>+
> >>+#include "ring.h"
> >>+#include "../grant_table.h"
> >>+
> >>+/*
> >>+ * Front->back notifications: When enqueuing a new request, sending a
> >>+ * notification can be made conditional on req_event (i.e., the generic
> >>+ * hold-off mechanism provided by the ring macros). Backends must set
> >>+ * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
> >>+ *
> >>+ * Back->front notifications: When enqueuing a new response, sending a
> >>+ * notification can be made conditional on rsp_event (i.e., the generic
> >>+ * hold-off mechanism provided by the ring macros). Frontends must set
> >>+ * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
> >>+ */
> >>+
> >>+/*
> >>+ * Feature and Parameter Negotiation
> >>+ * =================================
> >>+ * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
> >>+ * communicate capabilities and to negotiate operating parameters.  This
> >>+ * section enumerates these nodes which reside in the respective front and
> >>+ * backend portions of the XenStore, following the XenBus convention.
> >>+ *
> >>+ * All data in the XenStore is stored as strings.  Nodes specifying numeric
> >>+ * values are encoded in decimal.  Integer value ranges listed below are
> >>+ * expressed as fixed sized integer types capable of storing the conversion
> >>+ * of a properly formated node string, without loss of information.
> >>+ *
> >>+ * Any specified default value is in effect if the corresponding XenBus node
> >>+ * is not present in the XenStore.
> >>+ *
> >>+ * XenStore nodes in sections marked "PRIVATE" are solely for use by the
> >>+ * driver side whose XenBus tree contains them.
> >>+ *
> >>+ *****************************************************************************
> >>+ *                            Backend XenBus Nodes
> >>+ *****************************************************************************
> >>+ *
> >>+ *------------------ Backend Device Identification (PRIVATE) ------------------
> >>+ *
> >>+ * p-devname
> >>+ *      Values:         string
> >>+ *
> >>+ *      A free string used to identify the physical device (e.g. a disk name).
> >>+ *
> >>+ * p-dev
> >>+ *      Values:         string
> >>+ *
> >>+ *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
> >>+ *      (host, controller, target, lun, all integers), or a WWN (e.g.
> >>+ *      "naa.60014054ac780582").
> >>+ *
> >>+ * v-dev
> >>+ *      Values:         string
> >>+ *
> >>+ *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
> >>+ *      (host, controller, target, lun, all integers).
> >>+ *
> >>+ *--------------------------------- Features ---------------------------------
> >>+ *
> >>+ * feature-sg-grant
> >>+ *      Values:         <uint16_t>
> >>+ *      Default Value:  0
> >>+ *
> >>+ *      Specifies the maximum number of scatter/gather elements in grant pages
> >>+ *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
> >>+ *      SG elements specified directly in the request.
> >>+ *
> >>+ *****************************************************************************
> >>+ *                            Frontend XenBus Nodes
> >>+ *****************************************************************************
> >>+ *
> >>+ *----------------------- Request Transport Parameters -----------------------
> >>+ *
> >>+ * event-channel
> >>+ *      Values:         <uint32_t>
> >>+ *
> >>+ *      The identifier of the Xen event channel used to signal activity
> >>+ *      in the ring buffer.
> >>+ *
> >>+ * ring-ref
> >>+ *      Values:         <uint32_t>
> >
> >Should there a bit of explanation here about what 'ring ref' is?
> 
> Uuh, yes. I'll add it.
> 
> >
> >>+ *
> >>+ * protocol
> >>+ *      Values:         string (XEN_IO_PROTO_ABI_*)
> >>+ *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
> >>+ *
> >>+ *      The machine ABI rules governing the format of all ring request and
> >>+ *      response structures.
> >>+ */
> >>+
> >>+/* Requests from the frontend to the backend */
> >>+
> >>+/*
> >>+ * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
> >>+ * The target is specified via channel, id and lun.
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_CDB		1
> >>+
> >>+/*
> >>+ * Request abort of a running operation for the specified target given by
> >>+ * channel, id, lun and the operation's rqid in ref_rqid.
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_ABORT		2
> >>+
> >>+/*
> >>+ * Request a device reset of the specified target (channel and id).
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_RESET		3
> >>+
> >>+/*
> >>+ * Preset scatter/gather elements for a following request. Deprecated.
> >>+ * Keeping the define only to avoid usage of the value "4" for other actions.
> >>+ */
> >>+#define VSCSIIF_ACT_SCSI_SG_PRESET	4
> >>+
> >>+/*
> >>+ * Maximum scatter/gather segments per request.
> >>+ *
> >>+ * Considering balance between allocating at least 16 "vscsiif_request"
> >>+ * structures on one page (4096 bytes) and the number of scatter/gather
> >>+ * elements needed, we decided to use 26 as a magic number.
> >>+ *
> >>+ * If "feature-sg-grant" is set, more scatter/gather elements can be specified
> >>+ * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
> >>+ * In this case the vscsiif_request seg elements don't contain references to
> >>+ * the user data, but to the SG elements referencing the user data.
> >
> >That sounds like the indirect descriptors that virtio and xen-block has.
> >More questions about that below.
> >>+ */
> >>+#define VSCSIIF_SG_TABLESIZE		26
> >>+
> >>+/*
> >>+ * based on Linux kernel 2.6.18, still valid
> >>+ * Changing these values requires support of multiple protocols via the rings
> >>+ * as "old clients" will blindly use these values and the resulting structure
> >>+ * sizes.
> >>+ */
> >>+#define VSCSIIF_MAX_COMMAND_SIZE	16
> >>+#define VSCSIIF_SENSE_BUFFERSIZE	96
> >>+
> >>+struct scsiif_request_segment {
> >>+	grant_ref_t gref;
> >>+	uint16_t offset;
> >>+	uint16_t length;
> >>+};
> >
> >Yeey, 8 byte data structure!
> >>+
> >>+/* Size of one request is 252 bytes */
> >>+struct vscsiif_request {
> >>+	uint16_t rqid;		/* private guest value, echoed in resp  */
> >>+	uint8_t act;		/* command between backend and frontend */
> >>+	uint8_t cmd_len;	/* valid CDB bytes */
> >>+
> >>+	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
> >>+	uint16_t timeout_per_command;
> >>+	uint16_t channel, id, lun;	/* (virtual) device specification */
> >>+	uint16_t ref_rqid;		/* command abort reference */
> >>+	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
> >>+					   DMA_FROM_DEVICE(2)
> >>+					   DMA_NONE(3) requests */
> >>+	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
> >>+#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
> >>+					/* nr_segments counts grant pages with
> >>+					   SG elements
> >
> >Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
> >count of grant pages with SG elements. Does that mean the req.seg[0].gref points
> >to an page which will have an array of grant references? And each grant reference
> >will point to a data page? What about the 'offset' and 'length' of them?
> >
> >Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
> >'struct scsiif_request_segment' ? If so, where would the could of those
> >segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
> >go? Won't we collide?
> 
> nr_segments (without the VSCSIIF_SG_GRANT bit) always counts the number
> of populated seg[] entries. If VSCSIIF_SG_GRANT is set, each seg[] entry
> references another array of struct scsiif_request_segment using a grant
> ref, offset into the granted page and a length of the array in bytes.

Shouldn't there be a comment about that? The blkif.h has a pretty lengthy
one when it comes to indirect descriptors that could be copied as it
sounds exactly like the same thing.
> 
> The resulting number of struct scsiif_request_segment is the sum of
> seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).
> 

Where the nr_segments can only go Up to VSCSIIF_SG_TABLESIZE, so the max
total SG entries you can is 13312 ( 4096 / 8 = 512 max per page, times
26).
> >
> >
> >>+					   usable if "feature-sg-grant" set */
> >>+
> >>+	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
> >>+	uint32_t reserved[3];
> >
> >Or is the flag suppose to show up here?
> 
> No. There is no guarantee an "old" domU not aware of "feature-sg-grant"
> will have set reserved[] to zero.

<nods>
> 
> >>+};
> >>+
> >
> >The previous structure had an comment about the size of the structure.
> >Should it be here as well?
> 
> I can add one.

Thank you!
> 
> >
> >>+struct vscsiif_response {
> >>+	uint16_t rqid;		/* identifies request */
> >>+	uint8_t padding;
> >>+	uint8_t sense_len;
> >>+	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
> >>+	int32_t rslt;
> >>+	uint32_t residual_len;	/* request bufflen -
> >>+				   return the value from physical device */
> >>+	uint32_t reserved[36];
> >>+};
> >>+
> >>+DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
> >>+
> >>+#endif /*__XEN__PUBLIC_IO_SCSI_H__*/
> 

Re: [Xen-devel] [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Juergen Gross]

On 08/21/2014 09:26 PM, Konrad Rzeszutek Wilk wrote:
> On Wed, Aug 20, 2014 at 04:01:57PM +0200, Juergen Gross wrote:
>> On 08/20/2014 03:25 PM, Konrad Rzeszutek Wilk wrote:
>>> On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
...
>>>> +struct vscsiif_request {
>>>> +	uint16_t rqid;		/* private guest value, echoed in resp  */
>>>> +	uint8_t act;		/* command between backend and frontend */
>>>> +	uint8_t cmd_len;	/* valid CDB bytes */
>>>> +
>>>> +	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
>>>> +	uint16_t timeout_per_command;
>>>> +	uint16_t channel, id, lun;	/* (virtual) device specification */
>>>> +	uint16_t ref_rqid;		/* command abort reference */
>>>> +	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
>>>> +					   DMA_FROM_DEVICE(2)
>>>> +					   DMA_NONE(3) requests */
>>>> +	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
>>>> +#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
>>>> +					/* nr_segments counts grant pages with
>>>> +					   SG elements
>>>
>>> Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
>>> count of grant pages with SG elements. Does that mean the req.seg[0].gref points
>>> to an page which will have an array of grant references? And each grant reference
>>> will point to a data page? What about the 'offset' and 'length' of them?
>>>
>>> Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
>>> 'struct scsiif_request_segment' ? If so, where would the could of those
>>> segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
>>> go? Won't we collide?
>>
>> nr_segments (without the VSCSIIF_SG_GRANT bit) always counts the number
>> of populated seg[] entries. If VSCSIIF_SG_GRANT is set, each seg[] entry
>> references another array of struct scsiif_request_segment using a grant
>> ref, offset into the granted page and a length of the array in bytes.
>
> Shouldn't there be a comment about that? The blkif.h has a pretty lengthy
> one when it comes to indirect descriptors that could be copied as it
> sounds exactly like the same thing.

I already added the following:

/*
  * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
  * The target is specified via channel, id and lun.
  * The operation to be performed is specified via a CDB in cmnd[], the 
length
  * of the CDB is in cmd_len. sc_data_direction specifies the direction 
of data
  * (to the device, from the device, or none at all).
  * If data is to be transferred to or from the device the buffer(s) in the
  * guest memory is/are specified via one or multiple scsiif_request_segment
  * descriptors each specifying a memory page via a grant_ref_t, a 
offset into
  * the page and the length of the area in that page. All 
scsiif_request_segment
  * areas concatenated form the resulting data buffer used by the operation.
  * If the number of scsiif_request_segment areas is not too large (less 
than
  * or equal VSCSIIF_SG_TABLESIZE) the areas can be specified directly 
in the
  * seg[] array and the number of valid scsiif_request_segment elements 
is to be
  * set in nr_segments.
  * If "feature-sg-grant" in the Xenstore is set it is possible to 
specify more
  * than VSCSIIF_SG_TABLESIZE scsiif_request_segment elements via 
indirection.
  * The maximum number of allowed scsiif_request_segment elements is the 
value
  * of the "feature-sg-grant" entry from Xenstore. When using 
indirection the
  * seg[] array doesn't contain specifications of the data buffers, but
  * references to scsiif_request_segment arrays, which in turn reference the
  * data buffers. While nr_segments holds the number of populated seg[] 
entries
  * (plus the set VSCSIIF_SG_GRANT bit), the number of 
scsiif_request_segment
  * elements referencing the target data buffers is calculated from the 
lengths
  * of the seg[] elements (the sum of all valid seg[].length divided by the
  * size of one scsiif_request_segment structure).
  */
#define VSCSIIF_ACT_SCSI_CDB           1

>>
>> The resulting number of struct scsiif_request_segment is the sum of
>> seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).
>>
>
> Where the nr_segments can only go Up to VSCSIIF_SG_TABLESIZE, so the max
> total SG entries you can is 13312 ( 4096 / 8 = 512 max per page, times
> 26).

In theory, yes. SG_ALL (being 128 today) is limiting this value.


Juergen

Re: [Xen-devel] [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Christoph Hellwig]

On Fri, Aug 22, 2014 at 06:18:03AM +0200, Juergen Gross wrote:
> >>The resulting number of struct scsiif_request_segment is the sum of
> >>seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).
> >>
> >
> >Where the nr_segments can only go Up to VSCSIIF_SG_TABLESIZE, so the max
> >total SG entries you can is 13312 ( 4096 / 8 = 512 max per page, times
> >26).
> 
> In theory, yes. SG_ALL (being 128 today) is limiting this value.

Note that SG_ALL (or SCSI_MAX_SG_SEGMENTS which is a better name for it)
is only the maximum if your driver doesn't support S/G chaining.  Given
that it and the Xen infrastructure seems to use the proper s/g list
helpers it should support SCSI_MAX_SG_CHAIN_SEGMENTS easily.

Re: [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Christoph Hellwig]

On Fri, Aug 22, 2014 at 06:18:03AM +0200, Juergen Gross wrote:
> >>The resulting number of struct scsiif_request_segment is the sum of
> >>seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).
> >>
> >
> >Where the nr_segments can only go Up to VSCSIIF_SG_TABLESIZE, so the max
> >total SG entries you can is 13312 ( 4096 / 8 = 512 max per page, times
> >26).
> 
> In theory, yes. SG_ALL (being 128 today) is limiting this value.

Note that SG_ALL (or SCSI_MAX_SG_SEGMENTS which is a better name for it)
is only the maximum if your driver doesn't support S/G chaining.  Given
that it and the Xen infrastructure seems to use the proper s/g list
helpers it should support SCSI_MAX_SG_CHAIN_SEGMENTS easily.

Re: [PATCH V5 2/5] Add XEN pvSCSI protocol description

[Juergen Gross]

On 08/21/2014 09:26 PM, Konrad Rzeszutek Wilk wrote:
> On Wed, Aug 20, 2014 at 04:01:57PM +0200, Juergen Gross wrote:
>> On 08/20/2014 03:25 PM, Konrad Rzeszutek Wilk wrote:
>>> On Mon, Aug 18, 2014 at 11:31:47AM +0200, jgross@suse.com wrote:
...
>>>> +struct vscsiif_request {
>>>> +	uint16_t rqid;		/* private guest value, echoed in resp  */
>>>> +	uint8_t act;		/* command between backend and frontend */
>>>> +	uint8_t cmd_len;	/* valid CDB bytes */
>>>> +
>>>> +	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
>>>> +	uint16_t timeout_per_command;
>>>> +	uint16_t channel, id, lun;	/* (virtual) device specification */
>>>> +	uint16_t ref_rqid;		/* command abort reference */
>>>> +	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
>>>> +					   DMA_FROM_DEVICE(2)
>>>> +					   DMA_NONE(3) requests */
>>>> +	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
>>>> +#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
>>>> +					/* nr_segments counts grant pages with
>>>> +					   SG elements
>>>
>>> Stop missing. However I am a bit lost. It says that the 'nr_segments' will have the
>>> count of grant pages with SG elements. Does that mean the req.seg[0].gref points
>>> to an page which will have an array of grant references? And each grant reference
>>> will point to a data page? What about the 'offset' and 'length' of them?
>>>
>>> Or does it mean that the 'reg.seg[0].gref' points an page that is filled with
>>> 'struct scsiif_request_segment' ? If so, where would the could of those
>>> segments be in? In 'nr_segments'? If that is so where does the VSCSIIF_SG_GRANT
>>> go? Won't we collide?
>>
>> nr_segments (without the VSCSIIF_SG_GRANT bit) always counts the number
>> of populated seg[] entries. If VSCSIIF_SG_GRANT is set, each seg[] entry
>> references another array of struct scsiif_request_segment using a grant
>> ref, offset into the granted page and a length of the array in bytes.
>
> Shouldn't there be a comment about that? The blkif.h has a pretty lengthy
> one when it comes to indirect descriptors that could be copied as it
> sounds exactly like the same thing.

I already added the following:

/*
  * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
  * The target is specified via channel, id and lun.
  * The operation to be performed is specified via a CDB in cmnd[], the 
length
  * of the CDB is in cmd_len. sc_data_direction specifies the direction 
of data
  * (to the device, from the device, or none at all).
  * If data is to be transferred to or from the device the buffer(s) in the
  * guest memory is/are specified via one or multiple scsiif_request_segment
  * descriptors each specifying a memory page via a grant_ref_t, a 
offset into
  * the page and the length of the area in that page. All 
scsiif_request_segment
  * areas concatenated form the resulting data buffer used by the operation.
  * If the number of scsiif_request_segment areas is not too large (less 
than
  * or equal VSCSIIF_SG_TABLESIZE) the areas can be specified directly 
in the
  * seg[] array and the number of valid scsiif_request_segment elements 
is to be
  * set in nr_segments.
  * If "feature-sg-grant" in the Xenstore is set it is possible to 
specify more
  * than VSCSIIF_SG_TABLESIZE scsiif_request_segment elements via 
indirection.
  * The maximum number of allowed scsiif_request_segment elements is the 
value
  * of the "feature-sg-grant" entry from Xenstore. When using 
indirection the
  * seg[] array doesn't contain specifications of the data buffers, but
  * references to scsiif_request_segment arrays, which in turn reference the
  * data buffers. While nr_segments holds the number of populated seg[] 
entries
  * (plus the set VSCSIIF_SG_GRANT bit), the number of 
scsiif_request_segment
  * elements referencing the target data buffers is calculated from the 
lengths
  * of the seg[] elements (the sum of all valid seg[].length divided by the
  * size of one scsiif_request_segment structure).
  */
#define VSCSIIF_ACT_SCSI_CDB           1

>>
>> The resulting number of struct scsiif_request_segment is the sum of
>> seg[0..nr_segments-1].length / sizeof(struct scsiif_request_segment).
>>
>
> Where the nr_segments can only go Up to VSCSIIF_SG_TABLESIZE, so the max
> total SG entries you can is 13312 ( 4096 / 8 = 512 max per page, times
> 26).

In theory, yes. SG_ALL (being 128 today) is limiting this value.


Juergen

Re: [PATCH V5 2/5] Add XEN pvSCSI protocol description

[David Vrabel]

On 18/08/14 10:31, jgross@suse.com wrote:
> - add support for larger SG-lists by putting them in an own granted page

As a reminder, this can't be acked until this protocol change has been
applied to Xen copy of this header.

David

Re: [PATCH V5 2/5] Add XEN pvSCSI protocol description

[David Vrabel]

On 18/08/14 10:31, jgross@suse.com wrote:
> - add support for larger SG-lists by putting them in an own granted page

As a reminder, this can't be acked until this protocol change has been
applied to Xen copy of this header.

David

[PATCH V5 2/5] Add XEN pvSCSI protocol description

[jgross]

From: Juergen Gross <jgross@suse.com>

Add the definition of pvSCSI protocol used between the pvSCSI frontend in a
XEN domU and the pvSCSI backend in a XEN driver domain (usually Dom0).

This header was originally provided by Fujitsu for XEN based on Linux 2.6.18.
Changes are:
- added comment
- adapt to Linux style guide
- add support for larger SG-lists by putting them in an own granted page
- remove stale definitions

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 include/xen/interface/io/vscsiif.h | 214 +++++++++++++++++++++++++++++++++++++
 1 file changed, 214 insertions(+)
 create mode 100644 include/xen/interface/io/vscsiif.h

diff --git a/include/xen/interface/io/vscsiif.h b/include/xen/interface/io/vscsiif.h
new file mode 100644
index 0000000..4291889
--- /dev/null
+++ b/include/xen/interface/io/vscsiif.h
@@ -0,0 +1,214 @@
+/******************************************************************************
+ * vscsiif.h
+ *
+ * Based on the blkif.h code.
+ *
+ * This interface is to be regarded as a stable API between XEN domains
+ * running potentially different Linux kernel versions.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright(c) FUJITSU Limited 2008.
+ */
+
+#ifndef __XEN__PUBLIC_IO_SCSI_H__
+#define __XEN__PUBLIC_IO_SCSI_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ * Front->back notifications: When enqueuing a new request, sending a
+ * notification can be made conditional on req_event (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * req_event appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: When enqueuing a new response, sending a
+ * notification can be made conditional on rsp_event (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * rsp_event appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ */
+
+/*
+ * Feature and Parameter Negotiation
+ * =================================
+ * The two halves of a Xen pvSCSI driver utilize nodes within the XenStore to
+ * communicate capabilities and to negotiate operating parameters.  This
+ * section enumerates these nodes which reside in the respective front and
+ * backend portions of the XenStore, following the XenBus convention.
+ *
+ * All data in the XenStore is stored as strings.  Nodes specifying numeric
+ * values are encoded in decimal.  Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formated node string, without loss of information.
+ *
+ * Any specified default value is in effect if the corresponding XenBus node
+ * is not present in the XenStore.
+ *
+ * XenStore nodes in sections marked "PRIVATE" are solely for use by the
+ * driver side whose XenBus tree contains them.
+ *
+ *****************************************************************************
+ *                            Backend XenBus Nodes
+ *****************************************************************************
+ *
+ *------------------ Backend Device Identification (PRIVATE) ------------------
+ *
+ * p-devname
+ *      Values:         string
+ *
+ *      A free string used to identify the physical device (e.g. a disk name).
+ *
+ * p-dev
+ *      Values:         string
+ *
+ *      A string specifying the backend device: either a 4-tuple "h:c:t:l"
+ *      (host, controller, target, lun, all integers), or a WWN (e.g.
+ *      "naa.60014054ac780582").
+ *
+ * v-dev
+ *      Values:         string
+ *
+ *      A string specifying the frontend device in form of a 4-tuple "h:c:t:l"
+ *      (host, controller, target, lun, all integers).
+ *
+ *--------------------------------- Features ---------------------------------
+ *
+ * feature-sg-grant
+ *      Values:         <uint16_t>
+ *      Default Value:  0
+ *
+ *      Specifies the maximum number of scatter/gather elements in grant pages
+ *      supported. If not set, the backend supports up to VSCSIIF_SG_TABLESIZE
+ *      SG elements specified directly in the request.
+ *
+ *****************************************************************************
+ *                            Frontend XenBus Nodes
+ *****************************************************************************
+ *
+ *----------------------- Request Transport Parameters -----------------------
+ *
+ * event-channel
+ *      Values:         <uint32_t>
+ *
+ *      The identifier of the Xen event channel used to signal activity
+ *      in the ring buffer.
+ *
+ * ring-ref
+ *      Values:         <uint32_t>
+ *
+ * protocol
+ *      Values:         string (XEN_IO_PROTO_ABI_*)
+ *      Default Value:  XEN_IO_PROTO_ABI_NATIVE
+ *
+ *      The machine ABI rules governing the format of all ring request and
+ *      response structures.
+ */
+
+/* Requests from the frontend to the backend */
+
+/*
+ * Request a SCSI operation specified via a CDB in vscsiif_request.cmnd.
+ * The target is specified via channel, id and lun.
+ */
+#define VSCSIIF_ACT_SCSI_CDB		1
+
+/*
+ * Request abort of a running operation for the specified target given by
+ * channel, id, lun and the operation's rqid in ref_rqid.
+ */
+#define VSCSIIF_ACT_SCSI_ABORT		2
+
+/*
+ * Request a device reset of the specified target (channel and id).
+ */
+#define VSCSIIF_ACT_SCSI_RESET		3
+
+/*
+ * Preset scatter/gather elements for a following request. Deprecated.
+ * Keeping the define only to avoid usage of the value "4" for other actions.
+ */
+#define VSCSIIF_ACT_SCSI_SG_PRESET	4
+
+/*
+ * Maximum scatter/gather segments per request.
+ *
+ * Considering balance between allocating at least 16 "vscsiif_request"
+ * structures on one page (4096 bytes) and the number of scatter/gather
+ * elements needed, we decided to use 26 as a magic number.
+ *
+ * If "feature-sg-grant" is set, more scatter/gather elements can be specified
+ * by placing them in one or more (up to VSCSIIF_SG_TABLESIZE) granted pages.
+ * In this case the vscsiif_request seg elements don't contain references to
+ * the user data, but to the SG elements referencing the user data.
+ */
+#define VSCSIIF_SG_TABLESIZE		26
+
+/*
+ * based on Linux kernel 2.6.18, still valid
+ * Changing these values requires support of multiple protocols via the rings
+ * as "old clients" will blindly use these values and the resulting structure
+ * sizes.
+ */
+#define VSCSIIF_MAX_COMMAND_SIZE	16
+#define VSCSIIF_SENSE_BUFFERSIZE	96
+
+struct scsiif_request_segment {
+	grant_ref_t gref;
+	uint16_t offset;
+	uint16_t length;
+};
+
+/* Size of one request is 252 bytes */
+struct vscsiif_request {
+	uint16_t rqid;		/* private guest value, echoed in resp  */
+	uint8_t act;		/* command between backend and frontend */
+	uint8_t cmd_len;	/* valid CDB bytes */
+
+	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];	/* the CDB */
+	uint16_t timeout_per_command;
+	uint16_t channel, id, lun;	/* (virtual) device specification */
+	uint16_t ref_rqid;		/* command abort reference */
+	uint8_t sc_data_direction;	/* for DMA_TO_DEVICE(1)
+					   DMA_FROM_DEVICE(2)
+					   DMA_NONE(3) requests */
+	uint8_t nr_segments;		/* Number of pieces of scatter-gather */
+#define VSCSIIF_SG_GRANT	0x80	/* flag: SG elements via grant page */
+					/* nr_segments counts grant pages with
+					   SG elements
+					   usable if "feature-sg-grant" set */
+
+	struct scsiif_request_segment seg[VSCSIIF_SG_TABLESIZE];
+	uint32_t reserved[3];
+};
+
+struct vscsiif_response {
+	uint16_t rqid;		/* identifies request */
+	uint8_t padding;
+	uint8_t sense_len;
+	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
+	int32_t rslt;
+	uint32_t residual_len;	/* request bufflen -
+				   return the value from physical device */
+	uint32_t reserved[36];
+};
+
+DEFINE_RING_TYPES(vscsiif, struct vscsiif_request, struct vscsiif_response);
+
+#endif /*__XEN__PUBLIC_IO_SCSI_H__*/
-- 
1.8.4.5

[PATCH V5 3/5] Introduce xen-scsifront module

[jgross]

From: Juergen Gross <jgross@suse.com>

Introduces the XEN pvSCSI frontend. With pvSCSI it is possible for a XEN domU
to issue SCSI commands to a SCSI LUN assigned to that domU. The SCSI commands
are passed to the pvSCSI backend in a driver domain (usually Dom0) which is
owner of the physical device. This allows e.g. to use SCSI tape drives in a
XEN domU.

The code is taken from the pvSCSI implementation in XEN done by Fujitsu based
on Linux kernel 2.6.18.

Changes from the original version are:
- port to upstream kernel
- put all code in just one source file
- move module to appropriate location in kernel tree
- adapt to Linux style guide
- some minor code simplifications
- replace constants with defines
- remove not used defines
- add support for larger SG lists by putting them in a granted page

Signed-off-by: Juergen Gross <jgross@suse.com>

Xen related parts
Acked-by: David Vrabel <david.vrabel@citrix.com>
---
 drivers/scsi/Kconfig         |    9 +
 drivers/scsi/Makefile        |    1 +
 drivers/scsi/xen-scsifront.c | 1017 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 1027 insertions(+)
 create mode 100644 drivers/scsi/xen-scsifront.c

diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index 18a3358..9130df1 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -576,6 +576,15 @@ config VMWARE_PVSCSI
 	  To compile this driver as a module, choose M here: the
 	  module will be called vmw_pvscsi.
 
+config XEN_SCSI_FRONTEND
+	tristate "XEN SCSI frontend driver"
+	depends on SCSI && XEN
+	help
+	  The XEN SCSI frontend driver allows the kernel to access SCSI Devices
+	  within another guest OS (usually Dom0).
+	  Only needed if the kernel is running in a XEN guest and generic
+	  SCSI access to a device is needed.
+
 config HYPERV_STORAGE
 	tristate "Microsoft Hyper-V virtual storage driver"
 	depends on SCSI && HYPERV
diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile
index 5f0d299..59f1ce6 100644
--- a/drivers/scsi/Makefile
+++ b/drivers/scsi/Makefile
@@ -141,6 +141,7 @@ obj-$(CONFIG_SCSI_ESAS2R)	+= esas2r/
 obj-$(CONFIG_SCSI_PMCRAID)	+= pmcraid.o
 obj-$(CONFIG_SCSI_VIRTIO)	+= virtio_scsi.o
 obj-$(CONFIG_VMWARE_PVSCSI)	+= vmw_pvscsi.o
+obj-$(CONFIG_XEN_SCSI_FRONTEND)	+= xen-scsifront.o
 obj-$(CONFIG_HYPERV_STORAGE)	+= hv_storvsc.o
 
 obj-$(CONFIG_ARM)		+= arm/
diff --git a/drivers/scsi/xen-scsifront.c b/drivers/scsi/xen-scsifront.c
new file mode 100644
index 0000000..f54312e
--- /dev/null
+++ b/drivers/scsi/xen-scsifront.c
@@ -0,0 +1,1017 @@
+/*
+ * Xen SCSI frontend driver
+ *
+ * Copyright (c) 2008, FUJITSU Limited
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#define DEBUG
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/wait.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/sched.h>
+#include <linux/blkdev.h>
+#include <linux/pfn.h>
+#include <linux/slab.h>
+
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_host.h>
+
+#include <xen/xen.h>
+#include <xen/xenbus.h>
+#include <xen/grant_table.h>
+#include <xen/events.h>
+#include <xen/page.h>
+
+#include <xen/interface/grant_table.h>
+#include <xen/interface/io/vscsiif.h>
+#include <xen/interface/io/protocols.h>
+
+#include <asm/xen/hypervisor.h>
+
+
+#define GRANT_INVALID_REF	0
+
+#define VSCSIFRONT_OP_ADD_LUN	1
+#define VSCSIFRONT_OP_DEL_LUN	2
+
+#define DEFAULT_TASK_COMM_LEN	TASK_COMM_LEN
+
+/* tuning point*/
+#define VSCSIIF_DEFAULT_CMD_PER_LUN 10
+#define VSCSIIF_MAX_TARGET          64
+#define VSCSIIF_MAX_LUN             255
+
+#define VSCSIIF_RING_SIZE	__CONST_RING_SIZE(vscsiif, PAGE_SIZE)
+#define VSCSIIF_MAX_REQS	VSCSIIF_RING_SIZE
+
+#define vscsiif_grants_sg(_sg)	(PFN_UP((_sg) *		\
+				sizeof(struct scsiif_request_segment)))
+
+struct vscsifrnt_shadow {
+	/* command between backend and frontend */
+	unsigned char act;
+	uint16_t rqid;
+
+	/* Number of pieces of scatter-gather */
+	unsigned int nr_grants;
+	struct scsiif_request_segment *sg;
+
+	/* do reset or abort function */
+	wait_queue_head_t wq_reset;	/* reset work queue           */
+	int wait_reset;			/* reset work queue condition */
+	int32_t rslt_reset;		/* reset response status      */
+					/* (SUCESS or FAILED)         */
+
+	/* requested struct scsi_cmnd is stored from kernel */
+	struct scsi_cmnd *sc;
+	int gref[vscsiif_grants_sg(SG_ALL) + SG_ALL];
+};
+
+struct vscsifrnt_info {
+	struct xenbus_device *dev;
+
+	struct Scsi_Host *host;
+	int host_active;
+
+	spinlock_t shadow_lock;
+	unsigned int evtchn;
+	unsigned int irq;
+
+	grant_ref_t ring_ref;
+	struct vscsiif_front_ring ring;
+	struct vscsiif_response	ring_res;
+
+	unsigned long shadow_free;
+	struct vscsifrnt_shadow *shadow[VSCSIIF_MAX_REQS];
+
+	wait_queue_head_t wq_sync;
+	unsigned int waiting_sync:1;
+
+	char dev_state_path[64];
+	struct task_struct *curr;
+};
+
+#define DPRINTK(_f, _a...)				\
+	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)
+
+#define PREFIX(lvl) KERN_##lvl "scsifront: "
+
+static void scsifront_wake_up(struct vscsifrnt_info *info)
+{
+	info->waiting_sync = 0;
+	wake_up(&info->wq_sync);
+}
+
+static int scsifront_get_rqid(struct vscsifrnt_info *info)
+{
+	unsigned long flags;
+	int free;
+
+	spin_lock_irqsave(&info->shadow_lock, flags);
+
+	free = find_first_bit(&info->shadow_free, VSCSIIF_MAX_REQS);
+	info->shadow_free &= ~(1UL << free);
+
+	spin_unlock_irqrestore(&info->shadow_lock, flags);
+
+	return free;
+}
+
+static int _scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
+{
+	info->shadow_free |= 1UL << id;
+	info->shadow[id] = NULL;
+
+	return (info->shadow_free == 1UL << id || info->waiting_sync);
+}
+
+static void scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
+{
+	unsigned long flags;
+	int was_empty;
+
+	spin_lock_irqsave(&info->shadow_lock, flags);
+	was_empty = _scsifront_put_rqid(info, id);
+	spin_unlock_irqrestore(&info->shadow_lock, flags);
+
+	if (was_empty)
+		scsifront_wake_up(info);
+}
+
+static struct vscsiif_request *scsifront_pre_req(struct vscsifrnt_info *info)
+{
+	struct vscsiif_front_ring *ring = &(info->ring);
+	struct vscsiif_request *ring_req;
+	uint32_t id;
+
+	id = scsifront_get_rqid(info);	/* use id by response */
+	if (id >= VSCSIIF_MAX_REQS)
+		return NULL;
+
+	ring_req = RING_GET_REQUEST(&(info->ring), ring->req_prod_pvt);
+
+	ring->req_prod_pvt++;
+
+	ring_req->rqid = (uint16_t)id;
+
+	return ring_req;
+}
+
+static void scsifront_do_request(struct vscsifrnt_info *info)
+{
+	struct vscsiif_front_ring *ring = &(info->ring);
+	int notify;
+
+	RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(ring, notify);
+	if (notify)
+		notify_remote_via_irq(info->irq);
+}
+
+static void scsifront_gnttab_done(struct vscsifrnt_info *info, uint32_t id)
+{
+	struct vscsifrnt_shadow *s = info->shadow[id];
+	int i;
+
+	if (s->sc->sc_data_direction == DMA_NONE)
+		return;
+
+	for (i = 0; i < s->nr_grants; i++) {
+		if (unlikely(gnttab_query_foreign_access(s->gref[i]) != 0)) {
+			shost_printk(PREFIX(ALERT), info->host,
+				     "grant still in use by backend\n");
+			BUG();
+		}
+		gnttab_end_foreign_access(s->gref[i], 0, 0UL);
+	}
+
+	kfree(s->sg);
+}
+
+static void scsifront_cdb_cmd_done(struct vscsifrnt_info *info,
+				   struct vscsiif_response *ring_res)
+{
+	struct scsi_cmnd *sc;
+	uint32_t id;
+	uint8_t sense_len;
+
+	id = ring_res->rqid;
+	sc = info->shadow[id]->sc;
+
+	BUG_ON(sc == NULL);
+
+	scsifront_gnttab_done(info, id);
+	scsifront_put_rqid(info, id);
+
+	sc->result = ring_res->rslt;
+	scsi_set_resid(sc, ring_res->residual_len);
+
+	sense_len = min_t(uint8_t, VSCSIIF_SENSE_BUFFERSIZE,
+			  ring_res->sense_len);
+
+	if (sense_len)
+		memcpy(sc->sense_buffer, ring_res->sense_buffer, sense_len);
+
+	sc->scsi_done(sc);
+}
+
+static void scsifront_sync_cmd_done(struct vscsifrnt_info *info,
+				    struct vscsiif_response *ring_res)
+{
+	uint16_t id = ring_res->rqid;
+	unsigned long flags;
+	struct vscsifrnt_shadow *shadow = info->shadow[id];
+	int was_empty;
+
+	spin_lock_irqsave(&info->shadow_lock, flags);
+	shadow->wait_reset = 1;
+	switch (shadow->rslt_reset) {
+	case 0:
+		shadow->rslt_reset = ring_res->rslt;
+		break;
+	case -1:
+		was_empty = _scsifront_put_rqid(info, id);
+		spin_unlock_irqrestore(&info->shadow_lock, flags);
+		kfree(shadow);
+		if (was_empty)
+			scsifront_wake_up(info);
+		return;
+	default:
+		shost_printk(PREFIX(ERR), info->host,
+			     "bad reset state %d, possibly leaking %u\n",
+			     shadow->rslt_reset, id);
+		break;
+	}
+	spin_unlock_irqrestore(&info->shadow_lock, flags);
+
+	wake_up(&shadow->wq_reset);
+}
+
+static int scsifront_cmd_done(struct vscsifrnt_info *info)
+{
+	struct vscsiif_response *ring_res;
+	RING_IDX i, rp;
+	int more_to_do = 0;
+	unsigned long flags;
+
+	spin_lock_irqsave(info->host->host_lock, flags);
+
+	rp = info->ring.sring->rsp_prod;
+	rmb();	/* ordering required respective to dom0 */
+	for (i = info->ring.rsp_cons; i != rp; i++) {
+
+		ring_res = RING_GET_RESPONSE(&info->ring, i);
+
+		if (info->shadow[ring_res->rqid]->act == VSCSIIF_ACT_SCSI_CDB)
+			scsifront_cdb_cmd_done(info, ring_res);
+		else
+			scsifront_sync_cmd_done(info, ring_res);
+	}
+
+	info->ring.rsp_cons = i;
+
+	if (i != info->ring.req_prod_pvt)
+		RING_FINAL_CHECK_FOR_RESPONSES(&info->ring, more_to_do);
+	else
+		info->ring.sring->rsp_event = i + 1;
+
+	info->waiting_sync = 0;
+
+	spin_unlock_irqrestore(info->host->host_lock, flags);
+
+	wake_up(&info->wq_sync);
+
+	return more_to_do;
+}
+
+static irqreturn_t scsifront_irq_fn(int irq, void *dev_id)
+{
+	struct vscsifrnt_info *info = dev_id;
+
+	while (scsifront_cmd_done(info))
+		/* Yield point for this unbounded loop. */
+		cond_resched();
+
+	return IRQ_HANDLED;
+}
+
+static int map_data_for_request(struct vscsifrnt_info *info,
+				struct scsi_cmnd *sc,
+				struct vscsiif_request *ring_req,
+				struct vscsifrnt_shadow *shadow)
+{
+	grant_ref_t gref_head;
+	struct page *page;
+	int err, ref, ref_cnt = 0;
+	int write = (sc->sc_data_direction == DMA_TO_DEVICE);
+	unsigned int i, off, len, bytes;
+	unsigned int data_len = scsi_bufflen(sc);
+	unsigned int data_grants = 0, seg_grants = 0;
+	struct scatterlist *sg;
+	unsigned long mfn;
+	struct scsiif_request_segment *seg;
+
+	ring_req->nr_segments = 0;
+	if (sc->sc_data_direction == DMA_NONE || !data_len)
+		return 0;
+
+	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i)
+		data_grants += PFN_UP(sg->offset + sg->length);
+
+	if (data_grants > VSCSIIF_SG_TABLESIZE) {
+		if (data_grants > info->host->sg_tablesize) {
+			shost_printk(PREFIX(ERR), info->host,
+			     "Unable to map request_buffer for command!\n");
+			return -E2BIG;
+		}
+		seg_grants = vscsiif_grants_sg(data_grants);
+		shadow->sg = kcalloc(data_grants,
+			sizeof(struct scsiif_request_segment), GFP_NOIO);
+		if (!shadow->sg)
+			return -ENOMEM;
+	}
+	seg = shadow->sg ? : ring_req->seg;
+
+	err = gnttab_alloc_grant_references(seg_grants + data_grants,
+					    &gref_head);
+	if (err) {
+		kfree(shadow->sg);
+		shost_printk(PREFIX(ERR), info->host,
+			     "gnttab_alloc_grant_references() error\n");
+		return -ENOMEM;
+	}
+
+	if (seg_grants) {
+		page = virt_to_page(seg);
+		off = (unsigned long)seg & ~PAGE_MASK;
+		len = sizeof(struct scsiif_request_segment) * data_grants;
+		while (len > 0) {
+			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
+
+			ref = gnttab_claim_grant_reference(&gref_head);
+			BUG_ON(ref == -ENOSPC);
+
+			mfn = pfn_to_mfn(page_to_pfn(page));
+			gnttab_grant_foreign_access_ref(ref,
+				info->dev->otherend_id, mfn, 1);
+			shadow->gref[ref_cnt] = ref;
+			ring_req->seg[ref_cnt].gref   = ref;
+			ring_req->seg[ref_cnt].offset = (uint16_t)off;
+			ring_req->seg[ref_cnt].length = (uint16_t)bytes;
+
+			page++;
+			len -= bytes;
+			off = 0;
+			ref_cnt++;
+		}
+		BUG_ON(seg_grants < ref_cnt);
+		seg_grants = ref_cnt;
+	}
+
+	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i) {
+		page = sg_page(sg);
+		off = sg->offset;
+		len = sg->length;
+
+		while (len > 0 && data_len > 0) {
+			/*
+			 * sg sends a scatterlist that is larger than
+			 * the data_len it wants transferred for certain
+			 * IO sizes
+			 */
+			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
+			bytes = min(bytes, data_len);
+
+			ref = gnttab_claim_grant_reference(&gref_head);
+			BUG_ON(ref == -ENOSPC);
+
+			mfn = pfn_to_mfn(page_to_pfn(page));
+			gnttab_grant_foreign_access_ref(ref,
+				info->dev->otherend_id, mfn, write);
+
+			shadow->gref[ref_cnt] = ref;
+			seg->gref   = ref;
+			seg->offset = (uint16_t)off;
+			seg->length = (uint16_t)bytes;
+
+			page++;
+			seg++;
+			len -= bytes;
+			data_len -= bytes;
+			off = 0;
+			ref_cnt++;
+		}
+	}
+
+	if (seg_grants)
+		ring_req->nr_segments = VSCSIIF_SG_GRANT | seg_grants;
+	else
+		ring_req->nr_segments = (uint8_t)ref_cnt;
+	shadow->nr_grants = ref_cnt;
+
+	return 0;
+}
+
+static struct vscsiif_request *scsifront_command2ring(
+		struct vscsifrnt_info *info, struct scsi_cmnd *sc,
+		struct vscsifrnt_shadow *shadow)
+{
+	struct vscsiif_request *ring_req;
+
+	memset(shadow, 0, sizeof(*shadow));
+
+	ring_req = scsifront_pre_req(info);
+	if (!ring_req)
+		return NULL;
+
+	info->shadow[ring_req->rqid] = shadow;
+	shadow->rqid = ring_req->rqid;
+
+	ring_req->id      = sc->device->id;
+	ring_req->lun     = sc->device->lun;
+	ring_req->channel = sc->device->channel;
+	ring_req->cmd_len = sc->cmd_len;
+
+	BUG_ON(sc->cmd_len > VSCSIIF_MAX_COMMAND_SIZE);
+
+	memcpy(ring_req->cmnd, sc->cmnd, sc->cmd_len);
+
+	ring_req->sc_data_direction   = (uint8_t)sc->sc_data_direction;
+	ring_req->timeout_per_command = sc->request->timeout / HZ;
+
+	return ring_req;
+}
+
+static int scsifront_queuecommand(struct Scsi_Host *shost,
+				  struct scsi_cmnd *sc)
+{
+	struct vscsifrnt_info *info = shost_priv(shost);
+	struct vscsiif_request *ring_req;
+	struct vscsifrnt_shadow *shadow = scsi_cmd_priv(sc);
+	unsigned long flags;
+	int err;
+	uint16_t rqid;
+
+	spin_lock_irqsave(shost->host_lock, flags);
+	if (RING_FULL(&info->ring))
+		goto busy;
+
+	ring_req = scsifront_command2ring(info, sc, shadow);
+	if (!ring_req)
+		goto busy;
+
+	sc->result    = 0;
+
+	rqid              = ring_req->rqid;
+	ring_req->act     = VSCSIIF_ACT_SCSI_CDB;
+
+	shadow->sc  = sc;
+	shadow->act = VSCSIIF_ACT_SCSI_CDB;
+
+	err = map_data_for_request(info, sc, ring_req, shadow);
+	if (err < 0) {
+		DPRINTK("%s: err %d\n", __func__, err);
+		scsifront_put_rqid(info, rqid);
+		spin_unlock_irqrestore(shost->host_lock, flags);
+		if (err == -ENOMEM)
+			return SCSI_MLQUEUE_HOST_BUSY;
+		sc->result = DID_ERROR << 16;
+		sc->scsi_done(sc);
+		return 0;
+	}
+
+	scsifront_do_request(info);
+	spin_unlock_irqrestore(shost->host_lock, flags);
+
+	return 0;
+
+busy:
+	spin_unlock_irqrestore(shost->host_lock, flags);
+	DPRINTK("%s: busy\n", __func__);
+	return SCSI_MLQUEUE_HOST_BUSY;
+}
+
+/*
+ * Any exception handling (reset or abort) must be forwarded to the backend.
+ * We have to wait until an answer is returned. This answer contains the
+ * result to be returned to the requestor.
+ */
+static int scsifront_action_handler(struct scsi_cmnd *sc, uint8_t act)
+{
+	struct Scsi_Host *host = sc->device->host;
+	struct vscsifrnt_info *info = shost_priv(host);
+	struct vscsifrnt_shadow *shadow, *s = scsi_cmd_priv(sc);
+	struct vscsiif_request *ring_req;
+	int err = 0;
+
+	shadow = kmalloc(sizeof(*shadow), GFP_NOIO);
+	if (!shadow)
+		return FAILED;
+
+	for (;;) {
+		spin_lock_irq(host->host_lock);
+		if (!RING_FULL(&info->ring)) {
+			ring_req = scsifront_command2ring(info, sc, shadow);
+			if (ring_req)
+				break;
+		}
+		if (err) {
+			spin_unlock_irq(host->host_lock);
+			kfree(shadow);
+			return FAILED;
+		}
+		info->waiting_sync = 1;
+		spin_unlock_irq(host->host_lock);
+		err = wait_event_interruptible(info->wq_sync,
+					       !info->waiting_sync);
+		spin_lock_irq(host->host_lock);
+	}
+
+	ring_req->act = act;
+	ring_req->ref_rqid = s->rqid;
+
+	shadow->act = act;
+	shadow->rslt_reset = 0;
+	init_waitqueue_head(&shadow->wq_reset);
+
+	ring_req->nr_segments         = 0;
+
+	scsifront_do_request(info);
+
+	spin_unlock_irq(host->host_lock);
+	err = wait_event_interruptible(shadow->wq_reset, shadow->wait_reset);
+	spin_lock_irq(host->host_lock);
+
+	if (!err) {
+		err = shadow->rslt_reset;
+		scsifront_put_rqid(info, shadow->rqid);
+		kfree(shadow);
+	} else {
+		spin_lock(&info->shadow_lock);
+		shadow->rslt_reset = -1;
+		spin_unlock(&info->shadow_lock);
+		err = FAILED;
+	}
+
+	spin_unlock_irq(host->host_lock);
+	return err;
+}
+
+static int scsifront_eh_abort_handler(struct scsi_cmnd *sc)
+{
+	DPRINTK("%s\n", __func__);
+	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_ABORT);
+}
+
+static int scsifront_dev_reset_handler(struct scsi_cmnd *sc)
+{
+	DPRINTK("%s\n", __func__);
+	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_RESET);
+}
+
+static int scsifront_sdev_configure(struct scsi_device *sdev)
+{
+	struct vscsifrnt_info *info = shost_priv(sdev->host);
+
+	if (info && current == info->curr)
+		xenbus_printf(XBT_NIL, info->dev->nodename,
+			      info->dev_state_path, "%d", XenbusStateConnected);
+
+	return 0;
+}
+
+static void scsifront_sdev_destroy(struct scsi_device *sdev)
+{
+	struct vscsifrnt_info *info = shost_priv(sdev->host);
+
+	if (info && current == info->curr)
+		xenbus_printf(XBT_NIL, info->dev->nodename,
+			      info->dev_state_path, "%d", XenbusStateClosed);
+}
+
+static struct scsi_host_template scsifront_sht = {
+	.module			= THIS_MODULE,
+	.name			= "Xen SCSI frontend driver",
+	.queuecommand		= scsifront_queuecommand,
+	.eh_abort_handler	= scsifront_eh_abort_handler,
+	.eh_device_reset_handler = scsifront_dev_reset_handler,
+	.slave_configure	= scsifront_sdev_configure,
+	.slave_destroy		= scsifront_sdev_destroy,
+	.cmd_per_lun		= VSCSIIF_DEFAULT_CMD_PER_LUN,
+	.can_queue		= VSCSIIF_MAX_REQS,
+	.this_id		= -1,
+	.cmd_size		= sizeof(struct vscsifrnt_shadow),
+	.sg_tablesize		= VSCSIIF_SG_TABLESIZE,
+	.use_clustering		= DISABLE_CLUSTERING,
+	.proc_name		= "scsifront",
+};
+
+static int scsifront_alloc_ring(struct vscsifrnt_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct vscsiif_sring *sring;
+	int err = -ENOMEM;
+
+	/***** Frontend to Backend ring start *****/
+	sring = (struct vscsiif_sring *) __get_free_page(GFP_KERNEL);
+	if (!sring) {
+		xenbus_dev_fatal(dev, err,
+			"fail to allocate shared ring (Front to Back)");
+		return err;
+	}
+	SHARED_RING_INIT(sring);
+	FRONT_RING_INIT(&info->ring, sring, PAGE_SIZE);
+
+	err = xenbus_grant_ring(dev, virt_to_mfn(sring));
+	if (err < 0) {
+		free_page((unsigned long) sring);
+		xenbus_dev_fatal(dev, err,
+			"fail to grant shared ring (Front to Back)");
+		return err;
+	}
+	info->ring_ref = err;
+
+	err = xenbus_alloc_evtchn(dev, &info->evtchn);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "xenbus_alloc_evtchn");
+		goto free_gnttab;
+	}
+
+	err = bind_evtchn_to_irq(info->evtchn);
+	if (err <= 0) {
+		xenbus_dev_fatal(dev, err, "bind_evtchn_to_irq");
+		goto free_gnttab;
+	}
+
+	info->irq = err;
+
+	err = request_threaded_irq(info->irq, NULL, scsifront_irq_fn,
+				   IRQF_ONESHOT, "scsifront", info);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "request_threaded_irq");
+		goto free_irq;
+	}
+
+	return 0;
+
+/* free resource */
+free_irq:
+	unbind_from_irqhandler(info->irq, info);
+free_gnttab:
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+
+	return err;
+}
+
+static int scsifront_init_ring(struct vscsifrnt_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct xenbus_transaction xbt;
+	int err;
+
+	DPRINTK("%s\n", __func__);
+
+	err = scsifront_alloc_ring(info);
+	if (err)
+		return err;
+	DPRINTK("%u %u\n", info->ring_ref, info->evtchn);
+
+again:
+	err = xenbus_transaction_start(&xbt);
+	if (err)
+		xenbus_dev_fatal(dev, err, "starting transaction");
+
+	err = xenbus_printf(xbt, dev->nodename, "ring-ref", "%u",
+			    info->ring_ref);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "%s", "writing ring-ref");
+		goto fail;
+	}
+
+	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
+			    info->evtchn);
+
+	if (err) {
+		xenbus_dev_fatal(dev, err, "%s", "writing event-channel");
+		goto fail;
+	}
+
+	err = xenbus_transaction_end(xbt, 0);
+	if (err) {
+		if (err == -EAGAIN)
+			goto again;
+		xenbus_dev_fatal(dev, err, "completing transaction");
+		goto free_sring;
+	}
+
+	return 0;
+
+fail:
+	xenbus_transaction_end(xbt, 1);
+free_sring:
+	unbind_from_irqhandler(info->irq, info);
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+
+	return err;
+}
+
+
+static int scsifront_probe(struct xenbus_device *dev,
+			   const struct xenbus_device_id *id)
+{
+	struct vscsifrnt_info *info;
+	struct Scsi_Host *host;
+	int err = -ENOMEM;
+	char name[DEFAULT_TASK_COMM_LEN];
+
+	host = scsi_host_alloc(&scsifront_sht, sizeof(*info));
+	if (!host) {
+		xenbus_dev_fatal(dev, err, "fail to allocate scsi host");
+		return err;
+	}
+	info = (struct vscsifrnt_info *)host->hostdata;
+
+	dev_set_drvdata(&dev->dev, info);
+	info->dev  = dev;
+
+	info->shadow_free = (1UL << VSCSIIF_MAX_REQS) - 1;
+
+	err = scsifront_init_ring(info);
+	if (err) {
+		scsi_host_put(host);
+		return err;
+	}
+
+	init_waitqueue_head(&info->wq_sync);
+	spin_lock_init(&info->shadow_lock);
+
+	snprintf(name, DEFAULT_TASK_COMM_LEN, "vscsiif.%d", host->host_no);
+
+	host->max_id      = VSCSIIF_MAX_TARGET;
+	host->max_channel = 0;
+	host->max_lun     = VSCSIIF_MAX_LUN;
+	host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
+	host->max_cmd_len = VSCSIIF_MAX_COMMAND_SIZE;
+
+	err = scsi_add_host(host, &dev->dev);
+	if (err) {
+		dev_err(&dev->dev, "fail to add scsi host %d\n", err);
+		goto free_sring;
+	}
+	info->host = host;
+	info->host_active = 1;
+
+	xenbus_switch_state(dev, XenbusStateInitialised);
+
+	return 0;
+
+free_sring:
+	unbind_from_irqhandler(info->irq, info);
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+	scsi_host_put(host);
+	return err;
+}
+
+static int scsifront_remove(struct xenbus_device *dev)
+{
+	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
+
+	DPRINTK("%s: %s removed\n", __func__, dev->nodename);
+
+	if (info->host_active) {
+		/* Scsi_host not yet removed */
+		scsi_remove_host(info->host);
+		info->host_active = 0;
+	}
+
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+	unbind_from_irqhandler(info->irq, info);
+
+	scsi_host_put(info->host);
+
+	return 0;
+}
+
+static void scsifront_disconnect(struct vscsifrnt_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct Scsi_Host *host = info->host;
+
+	DPRINTK("%s: %s disconnect\n", __func__, dev->nodename);
+
+	/*
+	 * When this function is executed, all devices of
+	 * Frontend have been deleted.
+	 * Therefore, it need not block I/O before remove_host.
+	 */
+
+	if (info->host_active)
+		scsi_remove_host(host);
+	info->host_active = 0;
+
+	xenbus_frontend_closed(dev);
+}
+
+static void scsifront_do_lun_hotplug(struct vscsifrnt_info *info, int op)
+{
+	struct xenbus_device *dev = info->dev;
+	int i, err = 0;
+	char str[64];
+	char **dir;
+	unsigned int dir_n = 0;
+	unsigned int device_state;
+	unsigned int hst, chn, tgt, lun;
+	struct scsi_device *sdev;
+
+	dir = xenbus_directory(XBT_NIL, dev->otherend, "vscsi-devs", &dir_n);
+	if (IS_ERR(dir))
+		return;
+
+	/* mark current task as the one allowed to modify device states */
+	BUG_ON(info->curr);
+	info->curr = current;
+
+	for (i = 0; i < dir_n; i++) {
+		/* read status */
+		snprintf(str, sizeof(str), "vscsi-devs/%s/state", dir[i]);
+		err = xenbus_scanf(XBT_NIL, dev->otherend, str, "%u",
+				   &device_state);
+		if (XENBUS_EXIST_ERR(err))
+			continue;
+
+		/* virtual SCSI device */
+		snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", dir[i]);
+		err = xenbus_scanf(XBT_NIL, dev->otherend, str,
+				   "%u:%u:%u:%u", &hst, &chn, &tgt, &lun);
+		if (XENBUS_EXIST_ERR(err))
+			continue;
+
+		/*
+		 * Front device state path, used in slave_configure called
+		 * on successfull scsi_add_device, and in slave_destroy called
+		 * on remove of a device.
+		 */
+		snprintf(info->dev_state_path, sizeof(info->dev_state_path),
+			 "vscsi-devs/%s/state", dir[i]);
+
+		switch (op) {
+		case VSCSIFRONT_OP_ADD_LUN:
+			if (device_state == XenbusStateInitialised) {
+				err = scsi_add_device(info->host, chn, tgt,
+						      lun);
+
+				if (err) {
+					dev_err(&dev->dev, "scsi_add_device\n");
+					xenbus_printf(XBT_NIL, dev->nodename,
+						      info->dev_state_path,
+						      "%d", XenbusStateClosed);
+				}
+			}
+			break;
+		case VSCSIFRONT_OP_DEL_LUN:
+			if (device_state == XenbusStateClosing) {
+				sdev = scsi_device_lookup(info->host, chn, tgt,
+							  lun);
+				if (sdev) {
+					scsi_remove_device(sdev);
+					scsi_device_put(sdev);
+				}
+			}
+			break;
+		default:
+			break;
+		}
+	}
+
+	info->curr = NULL;
+
+	kfree(dir);
+}
+
+static void scsifront_read_backend_params(struct xenbus_device *dev,
+					  struct vscsifrnt_info *info)
+{
+	unsigned int sg_grant;
+	int ret;
+	struct Scsi_Host *host = info->host;
+
+	ret = xenbus_scanf(XBT_NIL, dev->otherend, "feature-sg-grant", "%u",
+			   &sg_grant);
+	if (ret == 1 && sg_grant) {
+		sg_grant = min_t(unsigned int, sg_grant, SG_ALL);
+		host->sg_tablesize = min_t(unsigned int, sg_grant,
+			VSCSIIF_SG_TABLESIZE * PAGE_SIZE /
+			sizeof(struct scsiif_request_segment));
+		dev_info(&dev->dev, "using up to %d SG entries\n",
+			 host->sg_tablesize);
+		host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
+	}
+}
+
+static void scsifront_backend_changed(struct xenbus_device *dev,
+				      enum xenbus_state backend_state)
+{
+	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
+
+	DPRINTK("%p %u %u\n", dev, dev->state, backend_state);
+
+	switch (backend_state) {
+	case XenbusStateUnknown:
+	case XenbusStateInitialising:
+	case XenbusStateInitWait:
+	case XenbusStateInitialised:
+		break;
+
+	case XenbusStateConnected:
+		scsifront_read_backend_params(dev, info);
+		if (xenbus_read_driver_state(dev->nodename) ==
+			XenbusStateInitialised) {
+			scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
+		}
+
+		if (dev->state != XenbusStateConnected)
+			xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+
+	case XenbusStateClosed:
+		if (dev->state == XenbusStateClosed)
+			break;
+		/* Missed the backend's Closing state -- fallthrough */
+	case XenbusStateClosing:
+		scsifront_disconnect(info);
+		break;
+
+	case XenbusStateReconfiguring:
+		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_DEL_LUN);
+		xenbus_switch_state(dev, XenbusStateReconfiguring);
+		break;
+
+	case XenbusStateReconfigured:
+		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
+		xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+	}
+}
+
+static const struct xenbus_device_id scsifront_ids[] = {
+	{ "vscsi" },
+	{ "" }
+};
+
+static DEFINE_XENBUS_DRIVER(scsifront, ,
+	.probe			= scsifront_probe,
+	.remove			= scsifront_remove,
+	.otherend_changed	= scsifront_backend_changed,
+);
+
+static int __init scsifront_init(void)
+{
+	if (!xen_domain())
+		return -ENODEV;
+
+	return xenbus_register_frontend(&scsifront_driver);
+}
+module_init(scsifront_init);
+
+static void __exit scsifront_exit(void)
+{
+	xenbus_unregister_driver(&scsifront_driver);
+}
+module_exit(scsifront_exit);
+
+MODULE_DESCRIPTION("Xen SCSI frontend driver");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("xen:vscsi");
-- 
1.8.4.5

[PATCH V5 3/5] Introduce xen-scsifront module

[jgross]

From: Juergen Gross <jgross@suse.com>

Introduces the XEN pvSCSI frontend. With pvSCSI it is possible for a XEN domU
to issue SCSI commands to a SCSI LUN assigned to that domU. The SCSI commands
are passed to the pvSCSI backend in a driver domain (usually Dom0) which is
owner of the physical device. This allows e.g. to use SCSI tape drives in a
XEN domU.

The code is taken from the pvSCSI implementation in XEN done by Fujitsu based
on Linux kernel 2.6.18.

Changes from the original version are:
- port to upstream kernel
- put all code in just one source file
- move module to appropriate location in kernel tree
- adapt to Linux style guide
- some minor code simplifications
- replace constants with defines
- remove not used defines
- add support for larger SG lists by putting them in a granted page

Signed-off-by: Juergen Gross <jgross@suse.com>

Xen related parts
Acked-by: David Vrabel <david.vrabel@citrix.com>
---
 drivers/scsi/Kconfig         |    9 +
 drivers/scsi/Makefile        |    1 +
 drivers/scsi/xen-scsifront.c | 1017 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 1027 insertions(+)
 create mode 100644 drivers/scsi/xen-scsifront.c

diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index 18a3358..9130df1 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -576,6 +576,15 @@ config VMWARE_PVSCSI
 	  To compile this driver as a module, choose M here: the
 	  module will be called vmw_pvscsi.
 
+config XEN_SCSI_FRONTEND
+	tristate "XEN SCSI frontend driver"
+	depends on SCSI && XEN
+	help
+	  The XEN SCSI frontend driver allows the kernel to access SCSI Devices
+	  within another guest OS (usually Dom0).
+	  Only needed if the kernel is running in a XEN guest and generic
+	  SCSI access to a device is needed.
+
 config HYPERV_STORAGE
 	tristate "Microsoft Hyper-V virtual storage driver"
 	depends on SCSI && HYPERV
diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile
index 5f0d299..59f1ce6 100644
--- a/drivers/scsi/Makefile
+++ b/drivers/scsi/Makefile
@@ -141,6 +141,7 @@ obj-$(CONFIG_SCSI_ESAS2R)	+= esas2r/
 obj-$(CONFIG_SCSI_PMCRAID)	+= pmcraid.o
 obj-$(CONFIG_SCSI_VIRTIO)	+= virtio_scsi.o
 obj-$(CONFIG_VMWARE_PVSCSI)	+= vmw_pvscsi.o
+obj-$(CONFIG_XEN_SCSI_FRONTEND)	+= xen-scsifront.o
 obj-$(CONFIG_HYPERV_STORAGE)	+= hv_storvsc.o
 
 obj-$(CONFIG_ARM)		+= arm/
diff --git a/drivers/scsi/xen-scsifront.c b/drivers/scsi/xen-scsifront.c
new file mode 100644
index 0000000..f54312e
--- /dev/null
+++ b/drivers/scsi/xen-scsifront.c
@@ -0,0 +1,1017 @@
+/*
+ * Xen SCSI frontend driver
+ *
+ * Copyright (c) 2008, FUJITSU Limited
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#define DEBUG
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/wait.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/sched.h>
+#include <linux/blkdev.h>
+#include <linux/pfn.h>
+#include <linux/slab.h>
+
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_host.h>
+
+#include <xen/xen.h>
+#include <xen/xenbus.h>
+#include <xen/grant_table.h>
+#include <xen/events.h>
+#include <xen/page.h>
+
+#include <xen/interface/grant_table.h>
+#include <xen/interface/io/vscsiif.h>
+#include <xen/interface/io/protocols.h>
+
+#include <asm/xen/hypervisor.h>
+
+
+#define GRANT_INVALID_REF	0
+
+#define VSCSIFRONT_OP_ADD_LUN	1
+#define VSCSIFRONT_OP_DEL_LUN	2
+
+#define DEFAULT_TASK_COMM_LEN	TASK_COMM_LEN
+
+/* tuning point*/
+#define VSCSIIF_DEFAULT_CMD_PER_LUN 10
+#define VSCSIIF_MAX_TARGET          64
+#define VSCSIIF_MAX_LUN             255
+
+#define VSCSIIF_RING_SIZE	__CONST_RING_SIZE(vscsiif, PAGE_SIZE)
+#define VSCSIIF_MAX_REQS	VSCSIIF_RING_SIZE
+
+#define vscsiif_grants_sg(_sg)	(PFN_UP((_sg) *		\
+				sizeof(struct scsiif_request_segment)))
+
+struct vscsifrnt_shadow {
+	/* command between backend and frontend */
+	unsigned char act;
+	uint16_t rqid;
+
+	/* Number of pieces of scatter-gather */
+	unsigned int nr_grants;
+	struct scsiif_request_segment *sg;
+
+	/* do reset or abort function */
+	wait_queue_head_t wq_reset;	/* reset work queue           */
+	int wait_reset;			/* reset work queue condition */
+	int32_t rslt_reset;		/* reset response status      */
+					/* (SUCESS or FAILED)         */
+
+	/* requested struct scsi_cmnd is stored from kernel */
+	struct scsi_cmnd *sc;
+	int gref[vscsiif_grants_sg(SG_ALL) + SG_ALL];
+};
+
+struct vscsifrnt_info {
+	struct xenbus_device *dev;
+
+	struct Scsi_Host *host;
+	int host_active;
+
+	spinlock_t shadow_lock;
+	unsigned int evtchn;
+	unsigned int irq;
+
+	grant_ref_t ring_ref;
+	struct vscsiif_front_ring ring;
+	struct vscsiif_response	ring_res;
+
+	unsigned long shadow_free;
+	struct vscsifrnt_shadow *shadow[VSCSIIF_MAX_REQS];
+
+	wait_queue_head_t wq_sync;
+	unsigned int waiting_sync:1;
+
+	char dev_state_path[64];
+	struct task_struct *curr;
+};
+
+#define DPRINTK(_f, _a...)				\
+	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)
+
+#define PREFIX(lvl) KERN_##lvl "scsifront: "
+
+static void scsifront_wake_up(struct vscsifrnt_info *info)
+{
+	info->waiting_sync = 0;
+	wake_up(&info->wq_sync);
+}
+
+static int scsifront_get_rqid(struct vscsifrnt_info *info)
+{
+	unsigned long flags;
+	int free;
+
+	spin_lock_irqsave(&info->shadow_lock, flags);
+
+	free = find_first_bit(&info->shadow_free, VSCSIIF_MAX_REQS);
+	info->shadow_free &= ~(1UL << free);
+
+	spin_unlock_irqrestore(&info->shadow_lock, flags);
+
+	return free;
+}
+
+static int _scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
+{
+	info->shadow_free |= 1UL << id;
+	info->shadow[id] = NULL;
+
+	return (info->shadow_free == 1UL << id || info->waiting_sync);
+}
+
+static void scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
+{
+	unsigned long flags;
+	int was_empty;
+
+	spin_lock_irqsave(&info->shadow_lock, flags);
+	was_empty = _scsifront_put_rqid(info, id);
+	spin_unlock_irqrestore(&info->shadow_lock, flags);
+
+	if (was_empty)
+		scsifront_wake_up(info);
+}
+
+static struct vscsiif_request *scsifront_pre_req(struct vscsifrnt_info *info)
+{
+	struct vscsiif_front_ring *ring = &(info->ring);
+	struct vscsiif_request *ring_req;
+	uint32_t id;
+
+	id = scsifront_get_rqid(info);	/* use id by response */
+	if (id >= VSCSIIF_MAX_REQS)
+		return NULL;
+
+	ring_req = RING_GET_REQUEST(&(info->ring), ring->req_prod_pvt);
+
+	ring->req_prod_pvt++;
+
+	ring_req->rqid = (uint16_t)id;
+
+	return ring_req;
+}
+
+static void scsifront_do_request(struct vscsifrnt_info *info)
+{
+	struct vscsiif_front_ring *ring = &(info->ring);
+	int notify;
+
+	RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(ring, notify);
+	if (notify)
+		notify_remote_via_irq(info->irq);
+}
+
+static void scsifront_gnttab_done(struct vscsifrnt_info *info, uint32_t id)
+{
+	struct vscsifrnt_shadow *s = info->shadow[id];
+	int i;
+
+	if (s->sc->sc_data_direction == DMA_NONE)
+		return;
+
+	for (i = 0; i < s->nr_grants; i++) {
+		if (unlikely(gnttab_query_foreign_access(s->gref[i]) != 0)) {
+			shost_printk(PREFIX(ALERT), info->host,
+				     "grant still in use by backend\n");
+			BUG();
+		}
+		gnttab_end_foreign_access(s->gref[i], 0, 0UL);
+	}
+
+	kfree(s->sg);
+}
+
+static void scsifront_cdb_cmd_done(struct vscsifrnt_info *info,
+				   struct vscsiif_response *ring_res)
+{
+	struct scsi_cmnd *sc;
+	uint32_t id;
+	uint8_t sense_len;
+
+	id = ring_res->rqid;
+	sc = info->shadow[id]->sc;
+
+	BUG_ON(sc == NULL);
+
+	scsifront_gnttab_done(info, id);
+	scsifront_put_rqid(info, id);
+
+	sc->result = ring_res->rslt;
+	scsi_set_resid(sc, ring_res->residual_len);
+
+	sense_len = min_t(uint8_t, VSCSIIF_SENSE_BUFFERSIZE,
+			  ring_res->sense_len);
+
+	if (sense_len)
+		memcpy(sc->sense_buffer, ring_res->sense_buffer, sense_len);
+
+	sc->scsi_done(sc);
+}
+
+static void scsifront_sync_cmd_done(struct vscsifrnt_info *info,
+				    struct vscsiif_response *ring_res)
+{
+	uint16_t id = ring_res->rqid;
+	unsigned long flags;
+	struct vscsifrnt_shadow *shadow = info->shadow[id];
+	int was_empty;
+
+	spin_lock_irqsave(&info->shadow_lock, flags);
+	shadow->wait_reset = 1;
+	switch (shadow->rslt_reset) {
+	case 0:
+		shadow->rslt_reset = ring_res->rslt;
+		break;
+	case -1:
+		was_empty = _scsifront_put_rqid(info, id);
+		spin_unlock_irqrestore(&info->shadow_lock, flags);
+		kfree(shadow);
+		if (was_empty)
+			scsifront_wake_up(info);
+		return;
+	default:
+		shost_printk(PREFIX(ERR), info->host,
+			     "bad reset state %d, possibly leaking %u\n",
+			     shadow->rslt_reset, id);
+		break;
+	}
+	spin_unlock_irqrestore(&info->shadow_lock, flags);
+
+	wake_up(&shadow->wq_reset);
+}
+
+static int scsifront_cmd_done(struct vscsifrnt_info *info)
+{
+	struct vscsiif_response *ring_res;
+	RING_IDX i, rp;
+	int more_to_do = 0;
+	unsigned long flags;
+
+	spin_lock_irqsave(info->host->host_lock, flags);
+
+	rp = info->ring.sring->rsp_prod;
+	rmb();	/* ordering required respective to dom0 */
+	for (i = info->ring.rsp_cons; i != rp; i++) {
+
+		ring_res = RING_GET_RESPONSE(&info->ring, i);
+
+		if (info->shadow[ring_res->rqid]->act == VSCSIIF_ACT_SCSI_CDB)
+			scsifront_cdb_cmd_done(info, ring_res);
+		else
+			scsifront_sync_cmd_done(info, ring_res);
+	}
+
+	info->ring.rsp_cons = i;
+
+	if (i != info->ring.req_prod_pvt)
+		RING_FINAL_CHECK_FOR_RESPONSES(&info->ring, more_to_do);
+	else
+		info->ring.sring->rsp_event = i + 1;
+
+	info->waiting_sync = 0;
+
+	spin_unlock_irqrestore(info->host->host_lock, flags);
+
+	wake_up(&info->wq_sync);
+
+	return more_to_do;
+}
+
+static irqreturn_t scsifront_irq_fn(int irq, void *dev_id)
+{
+	struct vscsifrnt_info *info = dev_id;
+
+	while (scsifront_cmd_done(info))
+		/* Yield point for this unbounded loop. */
+		cond_resched();
+
+	return IRQ_HANDLED;
+}
+
+static int map_data_for_request(struct vscsifrnt_info *info,
+				struct scsi_cmnd *sc,
+				struct vscsiif_request *ring_req,
+				struct vscsifrnt_shadow *shadow)
+{
+	grant_ref_t gref_head;
+	struct page *page;
+	int err, ref, ref_cnt = 0;
+	int write = (sc->sc_data_direction == DMA_TO_DEVICE);
+	unsigned int i, off, len, bytes;
+	unsigned int data_len = scsi_bufflen(sc);
+	unsigned int data_grants = 0, seg_grants = 0;
+	struct scatterlist *sg;
+	unsigned long mfn;
+	struct scsiif_request_segment *seg;
+
+	ring_req->nr_segments = 0;
+	if (sc->sc_data_direction == DMA_NONE || !data_len)
+		return 0;
+
+	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i)
+		data_grants += PFN_UP(sg->offset + sg->length);
+
+	if (data_grants > VSCSIIF_SG_TABLESIZE) {
+		if (data_grants > info->host->sg_tablesize) {
+			shost_printk(PREFIX(ERR), info->host,
+			     "Unable to map request_buffer for command!\n");
+			return -E2BIG;
+		}
+		seg_grants = vscsiif_grants_sg(data_grants);
+		shadow->sg = kcalloc(data_grants,
+			sizeof(struct scsiif_request_segment), GFP_NOIO);
+		if (!shadow->sg)
+			return -ENOMEM;
+	}
+	seg = shadow->sg ? : ring_req->seg;
+
+	err = gnttab_alloc_grant_references(seg_grants + data_grants,
+					    &gref_head);
+	if (err) {
+		kfree(shadow->sg);
+		shost_printk(PREFIX(ERR), info->host,
+			     "gnttab_alloc_grant_references() error\n");
+		return -ENOMEM;
+	}
+
+	if (seg_grants) {
+		page = virt_to_page(seg);
+		off = (unsigned long)seg & ~PAGE_MASK;
+		len = sizeof(struct scsiif_request_segment) * data_grants;
+		while (len > 0) {
+			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
+
+			ref = gnttab_claim_grant_reference(&gref_head);
+			BUG_ON(ref == -ENOSPC);
+
+			mfn = pfn_to_mfn(page_to_pfn(page));
+			gnttab_grant_foreign_access_ref(ref,
+				info->dev->otherend_id, mfn, 1);
+			shadow->gref[ref_cnt] = ref;
+			ring_req->seg[ref_cnt].gref   = ref;
+			ring_req->seg[ref_cnt].offset = (uint16_t)off;
+			ring_req->seg[ref_cnt].length = (uint16_t)bytes;
+
+			page++;
+			len -= bytes;
+			off = 0;
+			ref_cnt++;
+		}
+		BUG_ON(seg_grants < ref_cnt);
+		seg_grants = ref_cnt;
+	}
+
+	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i) {
+		page = sg_page(sg);
+		off = sg->offset;
+		len = sg->length;
+
+		while (len > 0 && data_len > 0) {
+			/*
+			 * sg sends a scatterlist that is larger than
+			 * the data_len it wants transferred for certain
+			 * IO sizes
+			 */
+			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
+			bytes = min(bytes, data_len);
+
+			ref = gnttab_claim_grant_reference(&gref_head);
+			BUG_ON(ref == -ENOSPC);
+
+			mfn = pfn_to_mfn(page_to_pfn(page));
+			gnttab_grant_foreign_access_ref(ref,
+				info->dev->otherend_id, mfn, write);
+
+			shadow->gref[ref_cnt] = ref;
+			seg->gref   = ref;
+			seg->offset = (uint16_t)off;
+			seg->length = (uint16_t)bytes;
+
+			page++;
+			seg++;
+			len -= bytes;
+			data_len -= bytes;
+			off = 0;
+			ref_cnt++;
+		}
+	}
+
+	if (seg_grants)
+		ring_req->nr_segments = VSCSIIF_SG_GRANT | seg_grants;
+	else
+		ring_req->nr_segments = (uint8_t)ref_cnt;
+	shadow->nr_grants = ref_cnt;
+
+	return 0;
+}
+
+static struct vscsiif_request *scsifront_command2ring(
+		struct vscsifrnt_info *info, struct scsi_cmnd *sc,
+		struct vscsifrnt_shadow *shadow)
+{
+	struct vscsiif_request *ring_req;
+
+	memset(shadow, 0, sizeof(*shadow));
+
+	ring_req = scsifront_pre_req(info);
+	if (!ring_req)
+		return NULL;
+
+	info->shadow[ring_req->rqid] = shadow;
+	shadow->rqid = ring_req->rqid;
+
+	ring_req->id      = sc->device->id;
+	ring_req->lun     = sc->device->lun;
+	ring_req->channel = sc->device->channel;
+	ring_req->cmd_len = sc->cmd_len;
+
+	BUG_ON(sc->cmd_len > VSCSIIF_MAX_COMMAND_SIZE);
+
+	memcpy(ring_req->cmnd, sc->cmnd, sc->cmd_len);
+
+	ring_req->sc_data_direction   = (uint8_t)sc->sc_data_direction;
+	ring_req->timeout_per_command = sc->request->timeout / HZ;
+
+	return ring_req;
+}
+
+static int scsifront_queuecommand(struct Scsi_Host *shost,
+				  struct scsi_cmnd *sc)
+{
+	struct vscsifrnt_info *info = shost_priv(shost);
+	struct vscsiif_request *ring_req;
+	struct vscsifrnt_shadow *shadow = scsi_cmd_priv(sc);
+	unsigned long flags;
+	int err;
+	uint16_t rqid;
+
+	spin_lock_irqsave(shost->host_lock, flags);
+	if (RING_FULL(&info->ring))
+		goto busy;
+
+	ring_req = scsifront_command2ring(info, sc, shadow);
+	if (!ring_req)
+		goto busy;
+
+	sc->result    = 0;
+
+	rqid              = ring_req->rqid;
+	ring_req->act     = VSCSIIF_ACT_SCSI_CDB;
+
+	shadow->sc  = sc;
+	shadow->act = VSCSIIF_ACT_SCSI_CDB;
+
+	err = map_data_for_request(info, sc, ring_req, shadow);
+	if (err < 0) {
+		DPRINTK("%s: err %d\n", __func__, err);
+		scsifront_put_rqid(info, rqid);
+		spin_unlock_irqrestore(shost->host_lock, flags);
+		if (err == -ENOMEM)
+			return SCSI_MLQUEUE_HOST_BUSY;
+		sc->result = DID_ERROR << 16;
+		sc->scsi_done(sc);
+		return 0;
+	}
+
+	scsifront_do_request(info);
+	spin_unlock_irqrestore(shost->host_lock, flags);
+
+	return 0;
+
+busy:
+	spin_unlock_irqrestore(shost->host_lock, flags);
+	DPRINTK("%s: busy\n", __func__);
+	return SCSI_MLQUEUE_HOST_BUSY;
+}
+
+/*
+ * Any exception handling (reset or abort) must be forwarded to the backend.
+ * We have to wait until an answer is returned. This answer contains the
+ * result to be returned to the requestor.
+ */
+static int scsifront_action_handler(struct scsi_cmnd *sc, uint8_t act)
+{
+	struct Scsi_Host *host = sc->device->host;
+	struct vscsifrnt_info *info = shost_priv(host);
+	struct vscsifrnt_shadow *shadow, *s = scsi_cmd_priv(sc);
+	struct vscsiif_request *ring_req;
+	int err = 0;
+
+	shadow = kmalloc(sizeof(*shadow), GFP_NOIO);
+	if (!shadow)
+		return FAILED;
+
+	for (;;) {
+		spin_lock_irq(host->host_lock);
+		if (!RING_FULL(&info->ring)) {
+			ring_req = scsifront_command2ring(info, sc, shadow);
+			if (ring_req)
+				break;
+		}
+		if (err) {
+			spin_unlock_irq(host->host_lock);
+			kfree(shadow);
+			return FAILED;
+		}
+		info->waiting_sync = 1;
+		spin_unlock_irq(host->host_lock);
+		err = wait_event_interruptible(info->wq_sync,
+					       !info->waiting_sync);
+		spin_lock_irq(host->host_lock);
+	}
+
+	ring_req->act = act;
+	ring_req->ref_rqid = s->rqid;
+
+	shadow->act = act;
+	shadow->rslt_reset = 0;
+	init_waitqueue_head(&shadow->wq_reset);
+
+	ring_req->nr_segments         = 0;
+
+	scsifront_do_request(info);
+
+	spin_unlock_irq(host->host_lock);
+	err = wait_event_interruptible(shadow->wq_reset, shadow->wait_reset);
+	spin_lock_irq(host->host_lock);
+
+	if (!err) {
+		err = shadow->rslt_reset;
+		scsifront_put_rqid(info, shadow->rqid);
+		kfree(shadow);
+	} else {
+		spin_lock(&info->shadow_lock);
+		shadow->rslt_reset = -1;
+		spin_unlock(&info->shadow_lock);
+		err = FAILED;
+	}
+
+	spin_unlock_irq(host->host_lock);
+	return err;
+}
+
+static int scsifront_eh_abort_handler(struct scsi_cmnd *sc)
+{
+	DPRINTK("%s\n", __func__);
+	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_ABORT);
+}
+
+static int scsifront_dev_reset_handler(struct scsi_cmnd *sc)
+{
+	DPRINTK("%s\n", __func__);
+	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_RESET);
+}
+
+static int scsifront_sdev_configure(struct scsi_device *sdev)
+{
+	struct vscsifrnt_info *info = shost_priv(sdev->host);
+
+	if (info && current == info->curr)
+		xenbus_printf(XBT_NIL, info->dev->nodename,
+			      info->dev_state_path, "%d", XenbusStateConnected);
+
+	return 0;
+}
+
+static void scsifront_sdev_destroy(struct scsi_device *sdev)
+{
+	struct vscsifrnt_info *info = shost_priv(sdev->host);
+
+	if (info && current == info->curr)
+		xenbus_printf(XBT_NIL, info->dev->nodename,
+			      info->dev_state_path, "%d", XenbusStateClosed);
+}
+
+static struct scsi_host_template scsifront_sht = {
+	.module			= THIS_MODULE,
+	.name			= "Xen SCSI frontend driver",
+	.queuecommand		= scsifront_queuecommand,
+	.eh_abort_handler	= scsifront_eh_abort_handler,
+	.eh_device_reset_handler = scsifront_dev_reset_handler,
+	.slave_configure	= scsifront_sdev_configure,
+	.slave_destroy		= scsifront_sdev_destroy,
+	.cmd_per_lun		= VSCSIIF_DEFAULT_CMD_PER_LUN,
+	.can_queue		= VSCSIIF_MAX_REQS,
+	.this_id		= -1,
+	.cmd_size		= sizeof(struct vscsifrnt_shadow),
+	.sg_tablesize		= VSCSIIF_SG_TABLESIZE,
+	.use_clustering		= DISABLE_CLUSTERING,
+	.proc_name		= "scsifront",
+};
+
+static int scsifront_alloc_ring(struct vscsifrnt_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct vscsiif_sring *sring;
+	int err = -ENOMEM;
+
+	/***** Frontend to Backend ring start *****/
+	sring = (struct vscsiif_sring *) __get_free_page(GFP_KERNEL);
+	if (!sring) {
+		xenbus_dev_fatal(dev, err,
+			"fail to allocate shared ring (Front to Back)");
+		return err;
+	}
+	SHARED_RING_INIT(sring);
+	FRONT_RING_INIT(&info->ring, sring, PAGE_SIZE);
+
+	err = xenbus_grant_ring(dev, virt_to_mfn(sring));
+	if (err < 0) {
+		free_page((unsigned long) sring);
+		xenbus_dev_fatal(dev, err,
+			"fail to grant shared ring (Front to Back)");
+		return err;
+	}
+	info->ring_ref = err;
+
+	err = xenbus_alloc_evtchn(dev, &info->evtchn);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "xenbus_alloc_evtchn");
+		goto free_gnttab;
+	}
+
+	err = bind_evtchn_to_irq(info->evtchn);
+	if (err <= 0) {
+		xenbus_dev_fatal(dev, err, "bind_evtchn_to_irq");
+		goto free_gnttab;
+	}
+
+	info->irq = err;
+
+	err = request_threaded_irq(info->irq, NULL, scsifront_irq_fn,
+				   IRQF_ONESHOT, "scsifront", info);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "request_threaded_irq");
+		goto free_irq;
+	}
+
+	return 0;
+
+/* free resource */
+free_irq:
+	unbind_from_irqhandler(info->irq, info);
+free_gnttab:
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+
+	return err;
+}
+
+static int scsifront_init_ring(struct vscsifrnt_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct xenbus_transaction xbt;
+	int err;
+
+	DPRINTK("%s\n", __func__);
+
+	err = scsifront_alloc_ring(info);
+	if (err)
+		return err;
+	DPRINTK("%u %u\n", info->ring_ref, info->evtchn);
+
+again:
+	err = xenbus_transaction_start(&xbt);
+	if (err)
+		xenbus_dev_fatal(dev, err, "starting transaction");
+
+	err = xenbus_printf(xbt, dev->nodename, "ring-ref", "%u",
+			    info->ring_ref);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "%s", "writing ring-ref");
+		goto fail;
+	}
+
+	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
+			    info->evtchn);
+
+	if (err) {
+		xenbus_dev_fatal(dev, err, "%s", "writing event-channel");
+		goto fail;
+	}
+
+	err = xenbus_transaction_end(xbt, 0);
+	if (err) {
+		if (err == -EAGAIN)
+			goto again;
+		xenbus_dev_fatal(dev, err, "completing transaction");
+		goto free_sring;
+	}
+
+	return 0;
+
+fail:
+	xenbus_transaction_end(xbt, 1);
+free_sring:
+	unbind_from_irqhandler(info->irq, info);
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+
+	return err;
+}
+
+
+static int scsifront_probe(struct xenbus_device *dev,
+			   const struct xenbus_device_id *id)
+{
+	struct vscsifrnt_info *info;
+	struct Scsi_Host *host;
+	int err = -ENOMEM;
+	char name[DEFAULT_TASK_COMM_LEN];
+
+	host = scsi_host_alloc(&scsifront_sht, sizeof(*info));
+	if (!host) {
+		xenbus_dev_fatal(dev, err, "fail to allocate scsi host");
+		return err;
+	}
+	info = (struct vscsifrnt_info *)host->hostdata;
+
+	dev_set_drvdata(&dev->dev, info);
+	info->dev  = dev;
+
+	info->shadow_free = (1UL << VSCSIIF_MAX_REQS) - 1;
+
+	err = scsifront_init_ring(info);
+	if (err) {
+		scsi_host_put(host);
+		return err;
+	}
+
+	init_waitqueue_head(&info->wq_sync);
+	spin_lock_init(&info->shadow_lock);
+
+	snprintf(name, DEFAULT_TASK_COMM_LEN, "vscsiif.%d", host->host_no);
+
+	host->max_id      = VSCSIIF_MAX_TARGET;
+	host->max_channel = 0;
+	host->max_lun     = VSCSIIF_MAX_LUN;
+	host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
+	host->max_cmd_len = VSCSIIF_MAX_COMMAND_SIZE;
+
+	err = scsi_add_host(host, &dev->dev);
+	if (err) {
+		dev_err(&dev->dev, "fail to add scsi host %d\n", err);
+		goto free_sring;
+	}
+	info->host = host;
+	info->host_active = 1;
+
+	xenbus_switch_state(dev, XenbusStateInitialised);
+
+	return 0;
+
+free_sring:
+	unbind_from_irqhandler(info->irq, info);
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+	scsi_host_put(host);
+	return err;
+}
+
+static int scsifront_remove(struct xenbus_device *dev)
+{
+	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
+
+	DPRINTK("%s: %s removed\n", __func__, dev->nodename);
+
+	if (info->host_active) {
+		/* Scsi_host not yet removed */
+		scsi_remove_host(info->host);
+		info->host_active = 0;
+	}
+
+	gnttab_end_foreign_access(info->ring_ref, 0,
+				  (unsigned long)info->ring.sring);
+	unbind_from_irqhandler(info->irq, info);
+
+	scsi_host_put(info->host);
+
+	return 0;
+}
+
+static void scsifront_disconnect(struct vscsifrnt_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct Scsi_Host *host = info->host;
+
+	DPRINTK("%s: %s disconnect\n", __func__, dev->nodename);
+
+	/*
+	 * When this function is executed, all devices of
+	 * Frontend have been deleted.
+	 * Therefore, it need not block I/O before remove_host.
+	 */
+
+	if (info->host_active)
+		scsi_remove_host(host);
+	info->host_active = 0;
+
+	xenbus_frontend_closed(dev);
+}
+
+static void scsifront_do_lun_hotplug(struct vscsifrnt_info *info, int op)
+{
+	struct xenbus_device *dev = info->dev;
+	int i, err = 0;
+	char str[64];
+	char **dir;
+	unsigned int dir_n = 0;
+	unsigned int device_state;
+	unsigned int hst, chn, tgt, lun;
+	struct scsi_device *sdev;
+
+	dir = xenbus_directory(XBT_NIL, dev->otherend, "vscsi-devs", &dir_n);
+	if (IS_ERR(dir))
+		return;
+
+	/* mark current task as the one allowed to modify device states */
+	BUG_ON(info->curr);
+	info->curr = current;
+
+	for (i = 0; i < dir_n; i++) {
+		/* read status */
+		snprintf(str, sizeof(str), "vscsi-devs/%s/state", dir[i]);
+		err = xenbus_scanf(XBT_NIL, dev->otherend, str, "%u",
+				   &device_state);
+		if (XENBUS_EXIST_ERR(err))
+			continue;
+
+		/* virtual SCSI device */
+		snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", dir[i]);
+		err = xenbus_scanf(XBT_NIL, dev->otherend, str,
+				   "%u:%u:%u:%u", &hst, &chn, &tgt, &lun);
+		if (XENBUS_EXIST_ERR(err))
+			continue;
+
+		/*
+		 * Front device state path, used in slave_configure called
+		 * on successfull scsi_add_device, and in slave_destroy called
+		 * on remove of a device.
+		 */
+		snprintf(info->dev_state_path, sizeof(info->dev_state_path),
+			 "vscsi-devs/%s/state", dir[i]);
+
+		switch (op) {
+		case VSCSIFRONT_OP_ADD_LUN:
+			if (device_state == XenbusStateInitialised) {
+				err = scsi_add_device(info->host, chn, tgt,
+						      lun);
+
+				if (err) {
+					dev_err(&dev->dev, "scsi_add_device\n");
+					xenbus_printf(XBT_NIL, dev->nodename,
+						      info->dev_state_path,
+						      "%d", XenbusStateClosed);
+				}
+			}
+			break;
+		case VSCSIFRONT_OP_DEL_LUN:
+			if (device_state == XenbusStateClosing) {
+				sdev = scsi_device_lookup(info->host, chn, tgt,
+							  lun);
+				if (sdev) {
+					scsi_remove_device(sdev);
+					scsi_device_put(sdev);
+				}
+			}
+			break;
+		default:
+			break;
+		}
+	}
+
+	info->curr = NULL;
+
+	kfree(dir);
+}
+
+static void scsifront_read_backend_params(struct xenbus_device *dev,
+					  struct vscsifrnt_info *info)
+{
+	unsigned int sg_grant;
+	int ret;
+	struct Scsi_Host *host = info->host;
+
+	ret = xenbus_scanf(XBT_NIL, dev->otherend, "feature-sg-grant", "%u",
+			   &sg_grant);
+	if (ret == 1 && sg_grant) {
+		sg_grant = min_t(unsigned int, sg_grant, SG_ALL);
+		host->sg_tablesize = min_t(unsigned int, sg_grant,
+			VSCSIIF_SG_TABLESIZE * PAGE_SIZE /
+			sizeof(struct scsiif_request_segment));
+		dev_info(&dev->dev, "using up to %d SG entries\n",
+			 host->sg_tablesize);
+		host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
+	}
+}
+
+static void scsifront_backend_changed(struct xenbus_device *dev,
+				      enum xenbus_state backend_state)
+{
+	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
+
+	DPRINTK("%p %u %u\n", dev, dev->state, backend_state);
+
+	switch (backend_state) {
+	case XenbusStateUnknown:
+	case XenbusStateInitialising:
+	case XenbusStateInitWait:
+	case XenbusStateInitialised:
+		break;
+
+	case XenbusStateConnected:
+		scsifront_read_backend_params(dev, info);
+		if (xenbus_read_driver_state(dev->nodename) ==
+			XenbusStateInitialised) {
+			scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
+		}
+
+		if (dev->state != XenbusStateConnected)
+			xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+
+	case XenbusStateClosed:
+		if (dev->state == XenbusStateClosed)
+			break;
+		/* Missed the backend's Closing state -- fallthrough */
+	case XenbusStateClosing:
+		scsifront_disconnect(info);
+		break;
+
+	case XenbusStateReconfiguring:
+		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_DEL_LUN);
+		xenbus_switch_state(dev, XenbusStateReconfiguring);
+		break;
+
+	case XenbusStateReconfigured:
+		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
+		xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+	}
+}
+
+static const struct xenbus_device_id scsifront_ids[] = {
+	{ "vscsi" },
+	{ "" }
+};
+
+static DEFINE_XENBUS_DRIVER(scsifront, ,
+	.probe			= scsifront_probe,
+	.remove			= scsifront_remove,
+	.otherend_changed	= scsifront_backend_changed,
+);
+
+static int __init scsifront_init(void)
+{
+	if (!xen_domain())
+		return -ENODEV;
+
+	return xenbus_register_frontend(&scsifront_driver);
+}
+module_init(scsifront_init);
+
+static void __exit scsifront_exit(void)
+{
+	xenbus_unregister_driver(&scsifront_driver);
+}
+module_exit(scsifront_exit);
+
+MODULE_DESCRIPTION("Xen SCSI frontend driver");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("xen:vscsi");
-- 
1.8.4.5

Re: [Xen-devel] [PATCH V5 3/5] Introduce xen-scsifront module

[Konrad Rzeszutek Wilk]

> --- /dev/null
> +++ b/drivers/scsi/xen-scsifront.c
> @@ -0,0 +1,1017 @@
> +/*
> + * Xen SCSI frontend driver
> + *
> + * Copyright (c) 2008, FUJITSU Limited
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version 2
> + * as published by the Free Software Foundation; or, when distributed
> + * separately from the Linux kernel or incorporated into other
> + * software packages, subject to the following license:
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this source file (the "Software"), to deal in the Software without
> + * restriction, including without limitation the rights to use, copy, modify,
> + * merge, publish, distribute, sublicense, and/or sell copies of the Software,
> + * and to permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + */
> +
> +#define DEBUG

:-) I think you don't want that in the driver.

> +
> +#include <linux/module.h>
> +#include <linux/kernel.h>
> +#include <linux/device.h>
> +#include <linux/wait.h>
> +#include <linux/interrupt.h>
> +#include <linux/spinlock.h>
> +#include <linux/sched.h>
> +#include <linux/blkdev.h>
> +#include <linux/pfn.h>
> +#include <linux/slab.h>
> +
> +#include <scsi/scsi_cmnd.h>
> +#include <scsi/scsi_device.h>
> +#include <scsi/scsi.h>
> +#include <scsi/scsi_host.h>
> +
> +#include <xen/xen.h>
> +#include <xen/xenbus.h>
> +#include <xen/grant_table.h>
> +#include <xen/events.h>
> +#include <xen/page.h>
> +
> +#include <xen/interface/grant_table.h>
> +#include <xen/interface/io/vscsiif.h>
> +#include <xen/interface/io/protocols.h>
> +
> +#include <asm/xen/hypervisor.h>
> +
> +
> +#define GRANT_INVALID_REF	0
> +
> +#define VSCSIFRONT_OP_ADD_LUN	1
> +#define VSCSIFRONT_OP_DEL_LUN	2

Shouldn't those be defined in the vscsiff.h file?

> +
> +#define DEFAULT_TASK_COMM_LEN	TASK_COMM_LEN

Not sure why you need the DEFAULT_ ? Could you use TASK_COMM_LEN?

> +
> +/* tuning point*/

Missing stop and a space after the 't':

/* Tuning point. */

> +#define VSCSIIF_DEFAULT_CMD_PER_LUN 10
> +#define VSCSIIF_MAX_TARGET          64
> +#define VSCSIIF_MAX_LUN             255
> +
> +#define VSCSIIF_RING_SIZE	__CONST_RING_SIZE(vscsiif, PAGE_SIZE)
> +#define VSCSIIF_MAX_REQS	VSCSIIF_RING_SIZE
> +
> +#define vscsiif_grants_sg(_sg)	(PFN_UP((_sg) *		\
> +				sizeof(struct scsiif_request_segment)))
> +
> +struct vscsifrnt_shadow {
> +	/* command between backend and frontend */
> +	unsigned char act;

act? How about 'op' ? Or 'cmd_op'?

> +	uint16_t rqid;

rqid? Could you have a comment explaining what that acronym is?
Oh wait - request id? How about just req_id?

> +
> +	/* Number of pieces of scatter-gather */
> +	unsigned int nr_grants;

s/nr_grants/nr_sg/ ?

> +	struct scsiif_request_segment *sg;
> +
> +	/* do reset or abort function */

Full stop missing.
> +	wait_queue_head_t wq_reset;	/* reset work queue           */
> +	int wait_reset;			/* reset work queue condition */
> +	int32_t rslt_reset;		/* reset response status      */
> +					/* (SUCESS or FAILED)         */

Full stop missing. s/SUCESS/SUCCESS/
> +
> +	/* requested struct scsi_cmnd is stored from kernel */

Full stop missing.
> +	struct scsi_cmnd *sc;
> +	int gref[vscsiif_grants_sg(SG_ALL) + SG_ALL];
> +};
> +
> +struct vscsifrnt_info {
> +	struct xenbus_device *dev;
> +
> +	struct Scsi_Host *host;
> +	int host_active;

This 'host_active' seems to be a guard to call 'scsi_host_remove'
through either the disconnect (so backend told us to disconnect)
or the .remove (so XenStore keys are moved). Either way I think
it is possible to run _both_ of them and this being just
an 'int' is not sufficient.

I would recommend you change this to an ref_count.
> +
> +	spinlock_t shadow_lock;

Could you move this spinlock below - to where
you have tons of of 'shadow' values please?

> +	unsigned int evtchn;
> +	unsigned int irq;
> +
> +	grant_ref_t ring_ref;
> +	struct vscsiif_front_ring ring;
> +	struct vscsiif_response	ring_res;
> +
> +	unsigned long shadow_free;

A comment would help in explainig this. 'shadow_free' sounds 
a bit cryptic. Oh, and xen-blkfront has it as 'shadow_free'
too! Argh, 'shadow_free_bitmask' could be a better name?
Oh, what if we converted to an bitmask type?

Either way - if you think the existing one should be this way
then lets leave it as is. But if you think a better name
is suited I can change in xen-blkfront too.

> +	struct vscsifrnt_shadow *shadow[VSCSIIF_MAX_REQS];
> +
> +	wait_queue_head_t wq_sync;
> +	unsigned int waiting_sync:1;

That sounds like it needs a spinlock to protect it?

Could you also explain a bit of what its purpose is? What
is it suppose to wait for? It looks like it is tied to the
error handler, perhaps then it ought to be called:

err_handler_done


> +
> +	char dev_state_path[64];
> +	struct task_struct *curr;
> +};
> +
> +#define DPRINTK(_f, _a...)				\
> +	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)

Could all the DPRINTK be converted to pr_debug right away?

> +
> +#define PREFIX(lvl) KERN_##lvl "scsifront: "

Perhaps you want?

#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +static void scsifront_wake_up(struct vscsifrnt_info *info)
> +{
> +	info->waiting_sync = 0;
> +	wake_up(&info->wq_sync);
> +}
> +
> +static int scsifront_get_rqid(struct vscsifrnt_info *info)
> +{
> +	unsigned long flags;
> +	int free;
> +
> +	spin_lock_irqsave(&info->shadow_lock, flags);
> +
> +	free = find_first_bit(&info->shadow_free, VSCSIIF_MAX_REQS);
> +	info->shadow_free &= ~(1UL << free);
> +
> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
> +
> +	return free;
> +}
> +
> +static int _scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
> +{
> +	info->shadow_free |= 1UL << id;
> +	info->shadow[id] = NULL;
> +
> +	return (info->shadow_free == 1UL << id || info->waiting_sync);
> +}
> +
> +static void scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
> +{
> +	unsigned long flags;
> +	int was_empty;
> +
> +	spin_lock_irqsave(&info->shadow_lock, flags);
> +	was_empty = _scsifront_put_rqid(info, id);

I wouldn't call this 'was_empty' as it will also be true if the 'waiting_sync'
is turned on. Perhaps this should be called: 'kick' or 'wake_wq' ?

> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
> +
> +	if (was_empty)
> +		scsifront_wake_up(info);
> +}
> +
> +static struct vscsiif_request *scsifront_pre_req(struct vscsifrnt_info *info)
> +{
> +	struct vscsiif_front_ring *ring = &(info->ring);
> +	struct vscsiif_request *ring_req;
> +	uint32_t id;
> +
> +	id = scsifront_get_rqid(info);	/* use id by response */

s/by/in/

> +	if (id >= VSCSIIF_MAX_REQS)
> +		return NULL;
> +
> +	ring_req = RING_GET_REQUEST(&(info->ring), ring->req_prod_pvt);
> +
> +	ring->req_prod_pvt++;
> +
> +	ring_req->rqid = (uint16_t)id;
> +
> +	return ring_req;
> +}
> +
> +static void scsifront_do_request(struct vscsifrnt_info *info)
> +{
> +	struct vscsiif_front_ring *ring = &(info->ring);
> +	int notify;
> +
> +	RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(ring, notify);
> +	if (notify)
> +		notify_remote_via_irq(info->irq);
> +}
> +
> +static void scsifront_gnttab_done(struct vscsifrnt_info *info, uint32_t id)
> +{
> +	struct vscsifrnt_shadow *s = info->shadow[id];
> +	int i;
> +
> +	if (s->sc->sc_data_direction == DMA_NONE)
> +		return;
> +
> +	for (i = 0; i < s->nr_grants; i++) {
> +		if (unlikely(gnttab_query_foreign_access(s->gref[i]) != 0)) {
> +			shost_printk(PREFIX(ALERT), info->host,
> +				     "grant still in use by backend\n");
> +			BUG();
> +		}
> +		gnttab_end_foreign_access(s->gref[i], 0, 0UL);
> +	}
> +
> +	kfree(s->sg);
> +}
> +
> +static void scsifront_cdb_cmd_done(struct vscsifrnt_info *info,
> +				   struct vscsiif_response *ring_res)

s/ring_res/ring_rsp/ ?

> +{
> +	struct scsi_cmnd *sc;
> +	uint32_t id;
> +	uint8_t sense_len;
> +
> +	id = ring_res->rqid;
> +	sc = info->shadow[id]->sc;
> +
> +	BUG_ON(sc == NULL);
> +
> +	scsifront_gnttab_done(info, id);
> +	scsifront_put_rqid(info, id);
> +
> +	sc->result = ring_res->rslt;
> +	scsi_set_resid(sc, ring_res->residual_len);
> +
> +	sense_len = min_t(uint8_t, VSCSIIF_SENSE_BUFFERSIZE,
> +			  ring_res->sense_len);
> +
> +	if (sense_len)
> +		memcpy(sc->sense_buffer, ring_res->sense_buffer, sense_len);
> +
> +	sc->scsi_done(sc);
> +}
> +
> +static void scsifront_sync_cmd_done(struct vscsifrnt_info *info,
> +				    struct vscsiif_response *ring_res)
> +{
> +	uint16_t id = ring_res->rqid;
> +	unsigned long flags;
> +	struct vscsifrnt_shadow *shadow = info->shadow[id];
> +	int was_empty;
> +
> +	spin_lock_irqsave(&info->shadow_lock, flags);
> +	shadow->wait_reset = 1;
> +	switch (shadow->rslt_reset) {
> +	case 0:
> +		shadow->rslt_reset = ring_res->rslt;
> +		break;
> +	case -1:

Is there an #define for this? The comment at the top mentioned
SUCCESS and FAILED ?

> +		was_empty = _scsifront_put_rqid(info, id);

Perhaps call it 'kick' or 'wake_wq' ?

> +		spin_unlock_irqrestore(&info->shadow_lock, flags);
> +		kfree(shadow);
> +		if (was_empty)
> +			scsifront_wake_up(info);
> +		return;
> +	default:
> +		shost_printk(PREFIX(ERR), info->host,
> +			     "bad reset state %d, possibly leaking %u\n",
> +			     shadow->rslt_reset, id);
> +		break;
> +	}
> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
> +
> +	wake_up(&shadow->wq_reset);
> +}
> +
> +static int scsifront_cmd_done(struct vscsifrnt_info *info)
> +{
> +	struct vscsiif_response *ring_res;
> +	RING_IDX i, rp;
> +	int more_to_do = 0;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(info->host->host_lock, flags);
> +
> +	rp = info->ring.sring->rsp_prod;
> +	rmb();	/* ordering required respective to dom0 */
> +	for (i = info->ring.rsp_cons; i != rp; i++) {
> +
> +		ring_res = RING_GET_RESPONSE(&info->ring, i);

I would recommend you look at git commit 6878c32e5cc0e40980abe51d1f02fb453e27493e

As the 'rqid' value - might be corrupted by the backend. Which means
that the 'info->shadow[rubbish_val]->act' could blow up.

I would just add a check to make sure that the 'rqid' value is
within the expected values.

> +
> +		if (info->shadow[ring_res->rqid]->act == VSCSIIF_ACT_SCSI_CDB)
> +			scsifront_cdb_cmd_done(info, ring_res);
> +		else
> +			scsifront_sync_cmd_done(info, ring_res);
> +	}
> +
> +	info->ring.rsp_cons = i;
> +
> +	if (i != info->ring.req_prod_pvt)
> +		RING_FINAL_CHECK_FOR_RESPONSES(&info->ring, more_to_do);
> +	else
> +		info->ring.sring->rsp_event = i + 1;
> +
> +	info->waiting_sync = 0;
> +
> +	spin_unlock_irqrestore(info->host->host_lock, flags);
> +
> +	wake_up(&info->wq_sync);
> +
> +	return more_to_do;
> +}
> +
> +static irqreturn_t scsifront_irq_fn(int irq, void *dev_id)
> +{
> +	struct vscsifrnt_info *info = dev_id;
> +
> +	while (scsifront_cmd_done(info))
> +		/* Yield point for this unbounded loop. */
> +		cond_resched();
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int map_data_for_request(struct vscsifrnt_info *info,
> +				struct scsi_cmnd *sc,
> +				struct vscsiif_request *ring_req,
> +				struct vscsifrnt_shadow *shadow)
> +{
> +	grant_ref_t gref_head;
> +	struct page *page;
> +	int err, ref, ref_cnt = 0;
> +	int write = (sc->sc_data_direction == DMA_TO_DEVICE);

What if it is DMA_BIDIRECTIONAL ?

> +	unsigned int i, off, len, bytes;
> +	unsigned int data_len = scsi_bufflen(sc);
> +	unsigned int data_grants = 0, seg_grants = 0;
> +	struct scatterlist *sg;
> +	unsigned long mfn;
> +	struct scsiif_request_segment *seg;
> +
> +	ring_req->nr_segments = 0;
> +	if (sc->sc_data_direction == DMA_NONE || !data_len)
> +		return 0;
> +
> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i)
> +		data_grants += PFN_UP(sg->offset + sg->length);
> +
> +	if (data_grants > VSCSIIF_SG_TABLESIZE) {
> +		if (data_grants > info->host->sg_tablesize) {
> +			shost_printk(PREFIX(ERR), info->host,
> +			     "Unable to map request_buffer for command!\n");
> +			return -E2BIG;
> +		}
> +		seg_grants = vscsiif_grants_sg(data_grants);
> +		shadow->sg = kcalloc(data_grants,
> +			sizeof(struct scsiif_request_segment), GFP_NOIO);
> +		if (!shadow->sg)
> +			return -ENOMEM;
> +	}
> +	seg = shadow->sg ? : ring_req->seg;
> +
> +	err = gnttab_alloc_grant_references(seg_grants + data_grants,
> +					    &gref_head);
> +	if (err) {
> +		kfree(shadow->sg);
> +		shost_printk(PREFIX(ERR), info->host,
> +			     "gnttab_alloc_grant_references() error\n");
> +		return -ENOMEM;
> +	}
> +
> +	if (seg_grants) {
> +		page = virt_to_page(seg);
> +		off = (unsigned long)seg & ~PAGE_MASK;
> +		len = sizeof(struct scsiif_request_segment) * data_grants;
> +		while (len > 0) {
> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
> +
> +			ref = gnttab_claim_grant_reference(&gref_head);
> +			BUG_ON(ref == -ENOSPC);
> +
> +			mfn = pfn_to_mfn(page_to_pfn(page));
> +			gnttab_grant_foreign_access_ref(ref,
> +				info->dev->otherend_id, mfn, 1);
> +			shadow->gref[ref_cnt] = ref;
> +			ring_req->seg[ref_cnt].gref   = ref;
> +			ring_req->seg[ref_cnt].offset = (uint16_t)off;
> +			ring_req->seg[ref_cnt].length = (uint16_t)bytes;
> +
> +			page++;
> +			len -= bytes;
> +			off = 0;
> +			ref_cnt++;
> +		}
> +		BUG_ON(seg_grants < ref_cnt);
> +		seg_grants = ref_cnt;
> +	}
> +
> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i) {
> +		page = sg_page(sg);
> +		off = sg->offset;
> +		len = sg->length;
> +
> +		while (len > 0 && data_len > 0) {
> +			/*
> +			 * sg sends a scatterlist that is larger than
> +			 * the data_len it wants transferred for certain
> +			 * IO sizes

Full stop missing.
> +			 */
> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
> +			bytes = min(bytes, data_len);
> +
> +			ref = gnttab_claim_grant_reference(&gref_head);
> +			BUG_ON(ref == -ENOSPC);
> +
> +			mfn = pfn_to_mfn(page_to_pfn(page));
> +			gnttab_grant_foreign_access_ref(ref,
> +				info->dev->otherend_id, mfn, write);
> +
> +			shadow->gref[ref_cnt] = ref;
> +			seg->gref   = ref;
> +			seg->offset = (uint16_t)off;
> +			seg->length = (uint16_t)bytes;
> +
> +			page++;
> +			seg++;
> +			len -= bytes;
> +			data_len -= bytes;
> +			off = 0;
> +			ref_cnt++;
> +		}
> +	}
> +
> +	if (seg_grants)
> +		ring_req->nr_segments = VSCSIIF_SG_GRANT | seg_grants;
> +	else
> +		ring_req->nr_segments = (uint8_t)ref_cnt;
> +	shadow->nr_grants = ref_cnt;
> +
> +	return 0;
> +}
> +
> +static struct vscsiif_request *scsifront_command2ring(
> +		struct vscsifrnt_info *info, struct scsi_cmnd *sc,
> +		struct vscsifrnt_shadow *shadow)
> +{
> +	struct vscsiif_request *ring_req;
> +
> +	memset(shadow, 0, sizeof(*shadow));
> +
> +	ring_req = scsifront_pre_req(info);
> +	if (!ring_req)
> +		return NULL;
> +
> +	info->shadow[ring_req->rqid] = shadow;
> +	shadow->rqid = ring_req->rqid;
> +
> +	ring_req->id      = sc->device->id;
> +	ring_req->lun     = sc->device->lun;
> +	ring_req->channel = sc->device->channel;
> +	ring_req->cmd_len = sc->cmd_len;
> +
> +	BUG_ON(sc->cmd_len > VSCSIIF_MAX_COMMAND_SIZE);
> +
> +	memcpy(ring_req->cmnd, sc->cmnd, sc->cmd_len);
> +
> +	ring_req->sc_data_direction   = (uint8_t)sc->sc_data_direction;
> +	ring_req->timeout_per_command = sc->request->timeout / HZ;
> +
> +	return ring_req;
> +}
> +
> +static int scsifront_queuecommand(struct Scsi_Host *shost,
> +				  struct scsi_cmnd *sc)
> +{
> +	struct vscsifrnt_info *info = shost_priv(shost);
> +	struct vscsiif_request *ring_req;
> +	struct vscsifrnt_shadow *shadow = scsi_cmd_priv(sc);
> +	unsigned long flags;
> +	int err;
> +	uint16_t rqid;
> +

BUG_ON(sc->sc_data_direction == DMA_BIDIRECTIONAL); ?


> +	spin_lock_irqsave(shost->host_lock, flags);
> +	if (RING_FULL(&info->ring))
> +		goto busy;
> +
> +	ring_req = scsifront_command2ring(info, sc, shadow);
> +	if (!ring_req)
> +		goto busy;
> +
> +	sc->result    = 0;

This has some odd spacing? Is it suppose to be at the same
aligment as the ones below? Maybe it is my editor having
issues.

> +
> +	rqid              = ring_req->rqid;
> +	ring_req->act     = VSCSIIF_ACT_SCSI_CDB;
> +
> +	shadow->sc  = sc;
> +	shadow->act = VSCSIIF_ACT_SCSI_CDB;
> +
> +	err = map_data_for_request(info, sc, ring_req, shadow);
> +	if (err < 0) {
> +		DPRINTK("%s: err %d\n", __func__, err);
> +		scsifront_put_rqid(info, rqid);
> +		spin_unlock_irqrestore(shost->host_lock, flags);
> +		if (err == -ENOMEM)
> +			return SCSI_MLQUEUE_HOST_BUSY;
> +		sc->result = DID_ERROR << 16;
> +		sc->scsi_done(sc);
> +		return 0;
> +	}
> +
> +	scsifront_do_request(info);
> +	spin_unlock_irqrestore(shost->host_lock, flags);
> +
> +	return 0;
> +
> +busy:
> +	spin_unlock_irqrestore(shost->host_lock, flags);
> +	DPRINTK("%s: busy\n", __func__);
> +	return SCSI_MLQUEUE_HOST_BUSY;
> +}
> +
> +/*
> + * Any exception handling (reset or abort) must be forwarded to the backend.
> + * We have to wait until an answer is returned. This answer contains the
> + * result to be returned to the requestor.
> + */
> +static int scsifront_action_handler(struct scsi_cmnd *sc, uint8_t act)
> +{
> +	struct Scsi_Host *host = sc->device->host;
> +	struct vscsifrnt_info *info = shost_priv(host);
> +	struct vscsifrnt_shadow *shadow, *s = scsi_cmd_priv(sc);
> +	struct vscsiif_request *ring_req;
> +	int err = 0;
> +
> +	shadow = kmalloc(sizeof(*shadow), GFP_NOIO);
> +	if (!shadow)
> +		return FAILED;
> +
> +	for (;;) {
> +		spin_lock_irq(host->host_lock);
> +		if (!RING_FULL(&info->ring)) {
> +			ring_req = scsifront_command2ring(info, sc, shadow);
> +			if (ring_req)
> +				break;
> +		}
> +		if (err) {
> +			spin_unlock_irq(host->host_lock);
> +			kfree(shadow);
> +			return FAILED;
> +		}
> +		info->waiting_sync = 1;
> +		spin_unlock_irq(host->host_lock);
> +		err = wait_event_interruptible(info->wq_sync,
> +					       !info->waiting_sync);
> +		spin_lock_irq(host->host_lock);
> +	}
> +
> +	ring_req->act = act;
> +	ring_req->ref_rqid = s->rqid;
> +
> +	shadow->act = act;
> +	shadow->rslt_reset = 0;
> +	init_waitqueue_head(&shadow->wq_reset);
> +
> +	ring_req->nr_segments         = 0;

Something odd with the spaces here.

> +
> +	scsifront_do_request(info);
> +
> +	spin_unlock_irq(host->host_lock);
> +	err = wait_event_interruptible(shadow->wq_reset, shadow->wait_reset);
> +	spin_lock_irq(host->host_lock);
> +
> +	if (!err) {
> +		err = shadow->rslt_reset;
> +		scsifront_put_rqid(info, shadow->rqid);
> +		kfree(shadow);
> +	} else {
> +		spin_lock(&info->shadow_lock);
> +		shadow->rslt_reset = -1;

#define for -1?

> +		spin_unlock(&info->shadow_lock);
> +		err = FAILED;
> +	}
> +
> +	spin_unlock_irq(host->host_lock);
> +	return err;
> +}
> +
> +static int scsifront_eh_abort_handler(struct scsi_cmnd *sc)
> +{
> +	DPRINTK("%s\n", __func__);
> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_ABORT);
> +}
> +
> +static int scsifront_dev_reset_handler(struct scsi_cmnd *sc)
> +{
> +	DPRINTK("%s\n", __func__);
> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_RESET);
> +}
> +
> +static int scsifront_sdev_configure(struct scsi_device *sdev)
> +{
> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
> +
> +	if (info && current == info->curr)
> +		xenbus_printf(XBT_NIL, info->dev->nodename,
> +			      info->dev_state_path, "%d", XenbusStateConnected);
> +
> +	return 0;
> +}
> +
> +static void scsifront_sdev_destroy(struct scsi_device *sdev)
> +{
> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
> +
> +	if (info && current == info->curr)
> +		xenbus_printf(XBT_NIL, info->dev->nodename,
> +			      info->dev_state_path, "%d", XenbusStateClosed);
> +}
> +
> +static struct scsi_host_template scsifront_sht = {
> +	.module			= THIS_MODULE,
> +	.name			= "Xen SCSI frontend driver",
> +	.queuecommand		= scsifront_queuecommand,
> +	.eh_abort_handler	= scsifront_eh_abort_handler,
> +	.eh_device_reset_handler = scsifront_dev_reset_handler,
> +	.slave_configure	= scsifront_sdev_configure,
> +	.slave_destroy		= scsifront_sdev_destroy,
> +	.cmd_per_lun		= VSCSIIF_DEFAULT_CMD_PER_LUN,
> +	.can_queue		= VSCSIIF_MAX_REQS,
> +	.this_id		= -1,
> +	.cmd_size		= sizeof(struct vscsifrnt_shadow),
> +	.sg_tablesize		= VSCSIIF_SG_TABLESIZE,
> +	.use_clustering		= DISABLE_CLUSTERING,
> +	.proc_name		= "scsifront",
> +};
> +
> +static int scsifront_alloc_ring(struct vscsifrnt_info *info)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	struct vscsiif_sring *sring;
> +	int err = -ENOMEM;
> +
> +	/***** Frontend to Backend ring start *****/
> +	sring = (struct vscsiif_sring *) __get_free_page(GFP_KERNEL);
> +	if (!sring) {
> +		xenbus_dev_fatal(dev, err,
> +			"fail to allocate shared ring (Front to Back)");
> +		return err;
> +	}
> +	SHARED_RING_INIT(sring);
> +	FRONT_RING_INIT(&info->ring, sring, PAGE_SIZE);
> +
> +	err = xenbus_grant_ring(dev, virt_to_mfn(sring));
> +	if (err < 0) {
> +		free_page((unsigned long) sring);

You can remove the space there.

> +		xenbus_dev_fatal(dev, err,
> +			"fail to grant shared ring (Front to Back)");
> +		return err;
> +	}
> +	info->ring_ref = err;
> +
> +	err = xenbus_alloc_evtchn(dev, &info->evtchn);
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "xenbus_alloc_evtchn");
> +		goto free_gnttab;
> +	}
> +
> +	err = bind_evtchn_to_irq(info->evtchn);
> +	if (err <= 0) {
> +		xenbus_dev_fatal(dev, err, "bind_evtchn_to_irq");
> +		goto free_gnttab;
> +	}
> +
> +	info->irq = err;
> +
> +	err = request_threaded_irq(info->irq, NULL, scsifront_irq_fn,
> +				   IRQF_ONESHOT, "scsifront", info);
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "request_threaded_irq");
> +		goto free_irq;
> +	}
> +
> +	return 0;
> +
> +/* free resource */
> +free_irq:
> +	unbind_from_irqhandler(info->irq, info);
> +free_gnttab:
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +

free_page((unsigned long)sring);

> +	return err;
> +}
> +
> +static int scsifront_init_ring(struct vscsifrnt_info *info)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	struct xenbus_transaction xbt;
> +	int err;
> +
> +	DPRINTK("%s\n", __func__);
> +
> +	err = scsifront_alloc_ring(info);
> +	if (err)
> +		return err;
> +	DPRINTK("%u %u\n", info->ring_ref, info->evtchn);
> +
> +again:
> +	err = xenbus_transaction_start(&xbt);
> +	if (err)
> +		xenbus_dev_fatal(dev, err, "starting transaction");
> +
> +	err = xenbus_printf(xbt, dev->nodename, "ring-ref", "%u",
> +			    info->ring_ref);
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "%s", "writing ring-ref");
> +		goto fail;
> +	}
> +
> +	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
> +			    info->evtchn);
> +
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "%s", "writing event-channel");
> +		goto fail;
> +	}
> +
> +	err = xenbus_transaction_end(xbt, 0);
> +	if (err) {
> +		if (err == -EAGAIN)
> +			goto again;
> +		xenbus_dev_fatal(dev, err, "completing transaction");
> +		goto free_sring;
> +	}
> +
> +	return 0;
> +
> +fail:
> +	xenbus_transaction_end(xbt, 1);
> +free_sring:
> +	unbind_from_irqhandler(info->irq, info);
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +

The label says 'free_sring' but I am not seeing it being freed?

> +	return err;
> +}
> +
> +
> +static int scsifront_probe(struct xenbus_device *dev,
> +			   const struct xenbus_device_id *id)
> +{
> +	struct vscsifrnt_info *info;
> +	struct Scsi_Host *host;
> +	int err = -ENOMEM;
> +	char name[DEFAULT_TASK_COMM_LEN];
> +
> +	host = scsi_host_alloc(&scsifront_sht, sizeof(*info));
> +	if (!host) {
> +		xenbus_dev_fatal(dev, err, "fail to allocate scsi host");
> +		return err;
> +	}
> +	info = (struct vscsifrnt_info *)host->hostdata;
> +
> +	dev_set_drvdata(&dev->dev, info);
> +	info->dev  = dev;

Extra space.

> +
> +	info->shadow_free = (1UL << VSCSIIF_MAX_REQS) - 1;
> +
> +	err = scsifront_init_ring(info);
> +	if (err) {
> +		scsi_host_put(host);
> +		return err;
> +	}
> +
> +	init_waitqueue_head(&info->wq_sync);
> +	spin_lock_init(&info->shadow_lock);
> +
> +	snprintf(name, DEFAULT_TASK_COMM_LEN, "vscsiif.%d", host->host_no);
> +
> +	host->max_id      = VSCSIIF_MAX_TARGET;
> +	host->max_channel = 0;
> +	host->max_lun     = VSCSIIF_MAX_LUN;
> +	host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
> +	host->max_cmd_len = VSCSIIF_MAX_COMMAND_SIZE;
> +
> +	err = scsi_add_host(host, &dev->dev);
> +	if (err) {
> +		dev_err(&dev->dev, "fail to add scsi host %d\n", err);
> +		goto free_sring;
> +	}
> +	info->host = host;
> +	info->host_active = 1;
> +
> +	xenbus_switch_state(dev, XenbusStateInitialised);
> +
> +	return 0;
> +
> +free_sring:
> +	unbind_from_irqhandler(info->irq, info);
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +	scsi_host_put(host);

The label says 'free_sring' but I am not seeing it being freed?

Hm, could those operations - unbind_from_irqhandler, gnttab_end_foreign_access
and free_page be moved to a seperate function to call?

> +	return err;
> +}
> +
> +static int scsifront_remove(struct xenbus_device *dev)
> +{
> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
> +
> +	DPRINTK("%s: %s removed\n", __func__, dev->nodename);
> +
> +	if (info->host_active) {
> +		/* Scsi_host not yet removed */
> +		scsi_remove_host(info->host);
> +		info->host_active = 0;
> +	}
> +
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +	unbind_from_irqhandler(info->irq, info);
> +

Should you free the ring?

> +	scsi_host_put(info->host);
> +
> +	return 0;
> +}
> +
> +static void scsifront_disconnect(struct vscsifrnt_info *info)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	struct Scsi_Host *host = info->host;
> +
> +	DPRINTK("%s: %s disconnect\n", __func__, dev->nodename);
> +
> +	/*
> +	 * When this function is executed, all devices of
> +	 * Frontend have been deleted.
> +	 * Therefore, it need not block I/O before remove_host.
> +	 */
> +
> +	if (info->host_active)
> +		scsi_remove_host(host);
> +	info->host_active = 0;
> +
> +	xenbus_frontend_closed(dev);
> +}
> +
> +static void scsifront_do_lun_hotplug(struct vscsifrnt_info *info, int op)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	int i, err = 0;
> +	char str[64];
> +	char **dir;
> +	unsigned int dir_n = 0;
> +	unsigned int device_state;
> +	unsigned int hst, chn, tgt, lun;
> +	struct scsi_device *sdev;
> +
> +	dir = xenbus_directory(XBT_NIL, dev->otherend, "vscsi-devs", &dir_n);
> +	if (IS_ERR(dir))
> +		return;
> +
> +	/* mark current task as the one allowed to modify device states */
> +	BUG_ON(info->curr);
> +	info->curr = current;
> +
> +	for (i = 0; i < dir_n; i++) {
> +		/* read status */
> +		snprintf(str, sizeof(str), "vscsi-devs/%s/state", dir[i]);
> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str, "%u",
> +				   &device_state);
> +		if (XENBUS_EXIST_ERR(err))
> +			continue;
> +
> +		/* virtual SCSI device */
> +		snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", dir[i]);
> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str,
> +				   "%u:%u:%u:%u", &hst, &chn, &tgt, &lun);
> +		if (XENBUS_EXIST_ERR(err))
> +			continue;
> +
> +		/*
> +		 * Front device state path, used in slave_configure called
> +		 * on successfull scsi_add_device, and in slave_destroy called
> +		 * on remove of a device.
> +		 */
> +		snprintf(info->dev_state_path, sizeof(info->dev_state_path),
> +			 "vscsi-devs/%s/state", dir[i]);
> +
> +		switch (op) {
> +		case VSCSIFRONT_OP_ADD_LUN:
> +			if (device_state == XenbusStateInitialised) {

You could convert to make this a bit easier to read to do:

	if (device_state != XenbusStateInitialised)
		break;
	.. and then with the rest of the code.

> +				err = scsi_add_device(info->host, chn, tgt,
> +						      lun);

You can make that more than 80 characters long.
> +
> +				if (err) {
> +					dev_err(&dev->dev, "scsi_add_device\n");
> +					xenbus_printf(XBT_NIL, dev->nodename,
> +						      info->dev_state_path,
> +						      "%d", XenbusStateClosed);
> +				}
> +			}
> +			break;
> +		case VSCSIFRONT_OP_DEL_LUN:
> +			if (device_state == XenbusStateClosing) {

Ditto.
> +				sdev = scsi_device_lookup(info->host, chn, tgt,
> +							  lun);
> +				if (sdev) {
> +					scsi_remove_device(sdev);
> +					scsi_device_put(sdev);
> +				}
> +			}
> +			break;
> +		default:
> +			break;
> +		}
> +	}
> +
> +	info->curr = NULL;
> +
> +	kfree(dir);
> +}
> +
> +static void scsifront_read_backend_params(struct xenbus_device *dev,
> +					  struct vscsifrnt_info *info)
> +{
> +	unsigned int sg_grant;
> +	int ret;
> +	struct Scsi_Host *host = info->host;
> +
> +	ret = xenbus_scanf(XBT_NIL, dev->otherend, "feature-sg-grant", "%u",
> +			   &sg_grant);
> +	if (ret == 1 && sg_grant) {
> +		sg_grant = min_t(unsigned int, sg_grant, SG_ALL);
> +		host->sg_tablesize = min_t(unsigned int, sg_grant,
> +			VSCSIIF_SG_TABLESIZE * PAGE_SIZE /
> +			sizeof(struct scsiif_request_segment));
> +		dev_info(&dev->dev, "using up to %d SG entries\n",
> +			 host->sg_tablesize);

Perhaps this dev_info can be printed regardless of whether
the backend has advertised an value? Isn't this information also
visible in the SysFS? If so, should it be just removed?

> +		host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;

If the backend decides to give us an value less than optimal - say '2'
should we just ignore it? Perhaps have

if (sg_grant < VSCSIIF_SG_TABLESIZE)
	/* Pfffff. */
	return;


> +	}
> +}
> +
> +static void scsifront_backend_changed(struct xenbus_device *dev,
> +				      enum xenbus_state backend_state)
> +{
> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
> +
> +	DPRINTK("%p %u %u\n", dev, dev->state, backend_state);
> +
> +	switch (backend_state) {
> +	case XenbusStateUnknown:
> +	case XenbusStateInitialising:
> +	case XenbusStateInitWait:
> +	case XenbusStateInitialised:
> +		break;
> +
> +	case XenbusStateConnected:
> +		scsifront_read_backend_params(dev, info);
> +		if (xenbus_read_driver_state(dev->nodename) ==
> +			XenbusStateInitialised) {

You can make this more than 80 characters. And also remove the '{' and '}'


> +			scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
> +		}
> +
> +		if (dev->state != XenbusStateConnected)
> +			xenbus_switch_state(dev, XenbusStateConnected);
> +		break;
> +
> +	case XenbusStateClosed:
> +		if (dev->state == XenbusStateClosed)
> +			break;
> +		/* Missed the backend's Closing state -- fallthrough */
> +	case XenbusStateClosing:
> +		scsifront_disconnect(info);
> +		break;
> +
> +	case XenbusStateReconfiguring:
> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_DEL_LUN);
> +		xenbus_switch_state(dev, XenbusStateReconfiguring);
> +		break;
> +
> +	case XenbusStateReconfigured:
> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
> +		xenbus_switch_state(dev, XenbusStateConnected);
> +		break;
> +	}
> +}
> +
> +static const struct xenbus_device_id scsifront_ids[] = {
> +	{ "vscsi" },
> +	{ "" }
> +};
> +
> +static DEFINE_XENBUS_DRIVER(scsifront, ,
> +	.probe			= scsifront_probe,
> +	.remove			= scsifront_remove,
> +	.otherend_changed	= scsifront_backend_changed,
> +);
> +
> +static int __init scsifront_init(void)
> +{
> +	if (!xen_domain())
> +		return -ENODEV;
> +
> +	return xenbus_register_frontend(&scsifront_driver);
> +}
> +module_init(scsifront_init);
> +
> +static void __exit scsifront_exit(void)
> +{
> +	xenbus_unregister_driver(&scsifront_driver);
> +}
> +module_exit(scsifront_exit);
> +
> +MODULE_DESCRIPTION("Xen SCSI frontend driver");
> +MODULE_LICENSE("GPL");
> +MODULE_ALIAS("xen:vscsi");

MODULE_AUTHOR?

Re: [Xen-devel] [PATCH V5 3/5] Introduce xen-scsifront module

[Juergen Gross]

On 08/23/2014 12:25 AM, Konrad Rzeszutek Wilk wrote:
>> --- /dev/null
>> +++ b/drivers/scsi/xen-scsifront.c
>> @@ -0,0 +1,1017 @@
>> +/*
>> + * Xen SCSI frontend driver
>> + *
>> + * Copyright (c) 2008, FUJITSU Limited
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU General Public License version 2
>> + * as published by the Free Software Foundation; or, when distributed
>> + * separately from the Linux kernel or incorporated into other
>> + * software packages, subject to the following license:
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this source file (the "Software"), to deal in the Software without
>> + * restriction, including without limitation the rights to use, copy, modify,
>> + * merge, publish, distribute, sublicense, and/or sell copies of the Software,
>> + * and to permit persons to whom the Software is furnished to do so, subject to
>> + * the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
>> + * IN THE SOFTWARE.
>> + */
>> +
>> +#define DEBUG
>
> :-) I think you don't want that in the driver.

Correct. :-)

>
>> +
>> +#include <linux/module.h>
>> +#include <linux/kernel.h>
>> +#include <linux/device.h>
>> +#include <linux/wait.h>
>> +#include <linux/interrupt.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/sched.h>
>> +#include <linux/blkdev.h>
>> +#include <linux/pfn.h>
>> +#include <linux/slab.h>
>> +
>> +#include <scsi/scsi_cmnd.h>
>> +#include <scsi/scsi_device.h>
>> +#include <scsi/scsi.h>
>> +#include <scsi/scsi_host.h>
>> +
>> +#include <xen/xen.h>
>> +#include <xen/xenbus.h>
>> +#include <xen/grant_table.h>
>> +#include <xen/events.h>
>> +#include <xen/page.h>
>> +
>> +#include <xen/interface/grant_table.h>
>> +#include <xen/interface/io/vscsiif.h>
>> +#include <xen/interface/io/protocols.h>
>> +
>> +#include <asm/xen/hypervisor.h>
>> +
>> +
>> +#define GRANT_INVALID_REF	0
>> +
>> +#define VSCSIFRONT_OP_ADD_LUN	1
>> +#define VSCSIFRONT_OP_DEL_LUN	2
>
> Shouldn't those be defined in the vscsiff.h file?

No, they are private for the frontend.

>
>> +
>> +#define DEFAULT_TASK_COMM_LEN	TASK_COMM_LEN
>
> Not sure why you need the DEFAULT_ ? Could you use TASK_COMM_LEN?

Sure.

>
>> +
>> +/* tuning point*/
>
> Missing stop and a space after the 't':
>
> /* Tuning point. */

Okay.

>
>> +#define VSCSIIF_DEFAULT_CMD_PER_LUN 10
>> +#define VSCSIIF_MAX_TARGET          64
>> +#define VSCSIIF_MAX_LUN             255
>> +
>> +#define VSCSIIF_RING_SIZE	__CONST_RING_SIZE(vscsiif, PAGE_SIZE)
>> +#define VSCSIIF_MAX_REQS	VSCSIIF_RING_SIZE
>> +
>> +#define vscsiif_grants_sg(_sg)	(PFN_UP((_sg) *		\
>> +				sizeof(struct scsiif_request_segment)))
>> +
>> +struct vscsifrnt_shadow {
>> +	/* command between backend and frontend */
>> +	unsigned char act;
>
> act? How about 'op' ? Or 'cmd_op'?

I wanted to use the same name as the related element in vscsiif.h

>
>> +	uint16_t rqid;
>
> rqid? Could you have a comment explaining what that acronym is?
> Oh wait - request id? How about just req_id?

Same again. It's called rqid in vscsiiif.h

>
>> +
>> +	/* Number of pieces of scatter-gather */
>> +	unsigned int nr_grants;
>
> s/nr_grants/nr_sg/ ?

I'll update the comment, as this is really the number of grants.

>
>> +	struct scsiif_request_segment *sg;
>> +
>> +	/* do reset or abort function */
>
> Full stop missing.
>> +	wait_queue_head_t wq_reset;	/* reset work queue           */
>> +	int wait_reset;			/* reset work queue condition */
>> +	int32_t rslt_reset;		/* reset response status      */
>> +					/* (SUCESS or FAILED)         */
>
> Full stop missing. s/SUCESS/SUCCESS/

Okay.

>> +
>> +	/* requested struct scsi_cmnd is stored from kernel */
>
> Full stop missing.

Okay.

>> +	struct scsi_cmnd *sc;
>> +	int gref[vscsiif_grants_sg(SG_ALL) + SG_ALL];
>> +};
>> +
>> +struct vscsifrnt_info {
>> +	struct xenbus_device *dev;
>> +
>> +	struct Scsi_Host *host;
>> +	int host_active;
>
> This 'host_active' seems to be a guard to call 'scsi_host_remove'
> through either the disconnect (so backend told us to disconnect)
> or the .remove (so XenStore keys are moved). Either way I think
> it is possible to run _both_ of them and this being just
> an 'int' is not sufficient.
>
> I would recommend you change this to an ref_count.

Hmm, I think a lock is required here.

>> +
>> +	spinlock_t shadow_lock;
>
> Could you move this spinlock below - to where
> you have tons of of 'shadow' values please?

Okay.

>
>> +	unsigned int evtchn;
>> +	unsigned int irq;
>> +
>> +	grant_ref_t ring_ref;
>> +	struct vscsiif_front_ring ring;
>> +	struct vscsiif_response	ring_res;
>> +
>> +	unsigned long shadow_free;
>
> A comment would help in explainig this. 'shadow_free' sounds
> a bit cryptic. Oh, and xen-blkfront has it as 'shadow_free'
> too! Argh, 'shadow_free_bitmask' could be a better name?
> Oh, what if we converted to an bitmask type?
>
> Either way - if you think the existing one should be this way
> then lets leave it as is. But if you think a better name
> is suited I can change in xen-blkfront too.

I'll use the bitmap operations and call it shadow_free_bitmap.

>
>> +	struct vscsifrnt_shadow *shadow[VSCSIIF_MAX_REQS];
>> +
>> +	wait_queue_head_t wq_sync;
>> +	unsigned int waiting_sync:1;
>
> That sounds like it needs a spinlock to protect it?

It is protected by info->host->host_lock.

> Could you also explain a bit of what its purpose is? What
> is it suppose to wait for? It looks like it is tied to the
> error handler, perhaps then it ought to be called:
>
> err_handler_done

It is used by the error handler, yes. But this is only due to the
fact that the error handler can't return with SCSI_MLQUEUE_HOST_BUSY.

I think I rename it to wait_ring_available, because this is the real
meaning: someone is waiting until a request can be allocated in the
ring.

>
>
>> +
>> +	char dev_state_path[64];
>> +	struct task_struct *curr;
>> +};
>> +
>> +#define DPRINTK(_f, _a...)				\
>> +	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)
>
> Could all the DPRINTK be converted to pr_debug right away?

Yes, I'll do it.

>> +
>> +#define PREFIX(lvl) KERN_##lvl "scsifront: "
>
> Perhaps you want?
>
> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

Not quite, but I'll remove the PREFIX define.

>> +
>> +static void scsifront_wake_up(struct vscsifrnt_info *info)
>> +{
>> +	info->waiting_sync = 0;
>> +	wake_up(&info->wq_sync);
>> +}
>> +
>> +static int scsifront_get_rqid(struct vscsifrnt_info *info)
>> +{
>> +	unsigned long flags;
>> +	int free;
>> +
>> +	spin_lock_irqsave(&info->shadow_lock, flags);
>> +
>> +	free = find_first_bit(&info->shadow_free, VSCSIIF_MAX_REQS);
>> +	info->shadow_free &= ~(1UL << free);
>> +
>> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +
>> +	return free;
>> +}
>> +
>> +static int _scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
>> +{
>> +	info->shadow_free |= 1UL << id;
>> +	info->shadow[id] = NULL;
>> +
>> +	return (info->shadow_free == 1UL << id || info->waiting_sync);
>> +}
>> +
>> +static void scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
>> +{
>> +	unsigned long flags;
>> +	int was_empty;
>> +
>> +	spin_lock_irqsave(&info->shadow_lock, flags);
>> +	was_empty = _scsifront_put_rqid(info, id);
>
> I wouldn't call this 'was_empty' as it will also be true if the 'waiting_sync'
> is turned on. Perhaps this should be called: 'kick' or 'wake_wq' ?

kick is okay, I think.

>
>> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +
>> +	if (was_empty)
>> +		scsifront_wake_up(info);
>> +}
>> +
>> +static struct vscsiif_request *scsifront_pre_req(struct vscsifrnt_info *info)
>> +{
>> +	struct vscsiif_front_ring *ring = &(info->ring);
>> +	struct vscsiif_request *ring_req;
>> +	uint32_t id;
>> +
>> +	id = scsifront_get_rqid(info);	/* use id by response */
>
> s/by/in/

Okay.

>
>> +	if (id >= VSCSIIF_MAX_REQS)
>> +		return NULL;
>> +
>> +	ring_req = RING_GET_REQUEST(&(info->ring), ring->req_prod_pvt);
>> +
>> +	ring->req_prod_pvt++;
>> +
>> +	ring_req->rqid = (uint16_t)id;
>> +
>> +	return ring_req;
>> +}
>> +
>> +static void scsifront_do_request(struct vscsifrnt_info *info)
>> +{
>> +	struct vscsiif_front_ring *ring = &(info->ring);
>> +	int notify;
>> +
>> +	RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(ring, notify);
>> +	if (notify)
>> +		notify_remote_via_irq(info->irq);
>> +}
>> +
>> +static void scsifront_gnttab_done(struct vscsifrnt_info *info, uint32_t id)
>> +{
>> +	struct vscsifrnt_shadow *s = info->shadow[id];
>> +	int i;
>> +
>> +	if (s->sc->sc_data_direction == DMA_NONE)
>> +		return;
>> +
>> +	for (i = 0; i < s->nr_grants; i++) {
>> +		if (unlikely(gnttab_query_foreign_access(s->gref[i]) != 0)) {
>> +			shost_printk(PREFIX(ALERT), info->host,
>> +				     "grant still in use by backend\n");
>> +			BUG();
>> +		}
>> +		gnttab_end_foreign_access(s->gref[i], 0, 0UL);
>> +	}
>> +
>> +	kfree(s->sg);
>> +}
>> +
>> +static void scsifront_cdb_cmd_done(struct vscsifrnt_info *info,
>> +				   struct vscsiif_response *ring_res)
>
> s/ring_res/ring_rsp/ ?

Yes, that's better.

>
>> +{
>> +	struct scsi_cmnd *sc;
>> +	uint32_t id;
>> +	uint8_t sense_len;
>> +
>> +	id = ring_res->rqid;
>> +	sc = info->shadow[id]->sc;
>> +
>> +	BUG_ON(sc == NULL);
>> +
>> +	scsifront_gnttab_done(info, id);
>> +	scsifront_put_rqid(info, id);
>> +
>> +	sc->result = ring_res->rslt;
>> +	scsi_set_resid(sc, ring_res->residual_len);
>> +
>> +	sense_len = min_t(uint8_t, VSCSIIF_SENSE_BUFFERSIZE,
>> +			  ring_res->sense_len);
>> +
>> +	if (sense_len)
>> +		memcpy(sc->sense_buffer, ring_res->sense_buffer, sense_len);
>> +
>> +	sc->scsi_done(sc);
>> +}
>> +
>> +static void scsifront_sync_cmd_done(struct vscsifrnt_info *info,
>> +				    struct vscsiif_response *ring_res)
>> +{
>> +	uint16_t id = ring_res->rqid;
>> +	unsigned long flags;
>> +	struct vscsifrnt_shadow *shadow = info->shadow[id];
>> +	int was_empty;
>> +
>> +	spin_lock_irqsave(&info->shadow_lock, flags);
>> +	shadow->wait_reset = 1;
>> +	switch (shadow->rslt_reset) {
>> +	case 0:
>> +		shadow->rslt_reset = ring_res->rslt;
>> +		break;
>> +	case -1:
>
> Is there an #define for this? The comment at the top mentioned
> SUCCESS and FAILED ?

Adding defines.

>
>> +		was_empty = _scsifront_put_rqid(info, id);
>
> Perhaps call it 'kick' or 'wake_wq' ?

kick, again.

>
>> +		spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +		kfree(shadow);
>> +		if (was_empty)
>> +			scsifront_wake_up(info);
>> +		return;
>> +	default:
>> +		shost_printk(PREFIX(ERR), info->host,
>> +			     "bad reset state %d, possibly leaking %u\n",
>> +			     shadow->rslt_reset, id);
>> +		break;
>> +	}
>> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +
>> +	wake_up(&shadow->wq_reset);
>> +}
>> +
>> +static int scsifront_cmd_done(struct vscsifrnt_info *info)
>> +{
>> +	struct vscsiif_response *ring_res;
>> +	RING_IDX i, rp;
>> +	int more_to_do = 0;
>> +	unsigned long flags;
>> +
>> +	spin_lock_irqsave(info->host->host_lock, flags);
>> +
>> +	rp = info->ring.sring->rsp_prod;
>> +	rmb();	/* ordering required respective to dom0 */
>> +	for (i = info->ring.rsp_cons; i != rp; i++) {
>> +
>> +		ring_res = RING_GET_RESPONSE(&info->ring, i);
>
> I would recommend you look at git commit 6878c32e5cc0e40980abe51d1f02fb453e27493e
>
> As the 'rqid' value - might be corrupted by the backend. Which means
> that the 'info->shadow[rubbish_val]->act' could blow up.
>
> I would just add a check to make sure that the 'rqid' value is
> within the expected values.

I'll add a check (the rqid should be in use as well).

>
>> +
>> +		if (info->shadow[ring_res->rqid]->act == VSCSIIF_ACT_SCSI_CDB)
>> +			scsifront_cdb_cmd_done(info, ring_res);
>> +		else
>> +			scsifront_sync_cmd_done(info, ring_res);
>> +	}
>> +
>> +	info->ring.rsp_cons = i;
>> +
>> +	if (i != info->ring.req_prod_pvt)
>> +		RING_FINAL_CHECK_FOR_RESPONSES(&info->ring, more_to_do);
>> +	else
>> +		info->ring.sring->rsp_event = i + 1;
>> +
>> +	info->waiting_sync = 0;
>> +
>> +	spin_unlock_irqrestore(info->host->host_lock, flags);
>> +
>> +	wake_up(&info->wq_sync);
>> +
>> +	return more_to_do;
>> +}
>> +
>> +static irqreturn_t scsifront_irq_fn(int irq, void *dev_id)
>> +{
>> +	struct vscsifrnt_info *info = dev_id;
>> +
>> +	while (scsifront_cmd_done(info))
>> +		/* Yield point for this unbounded loop. */
>> +		cond_resched();
>> +
>> +	return IRQ_HANDLED;
>> +}
>> +
>> +static int map_data_for_request(struct vscsifrnt_info *info,
>> +				struct scsi_cmnd *sc,
>> +				struct vscsiif_request *ring_req,
>> +				struct vscsifrnt_shadow *shadow)
>> +{
>> +	grant_ref_t gref_head;
>> +	struct page *page;
>> +	int err, ref, ref_cnt = 0;
>> +	int write = (sc->sc_data_direction == DMA_TO_DEVICE);
>
> What if it is DMA_BIDIRECTIONAL ?

That's okay. DMA_TO_DEVICE is the only case where a grant is flagged as
read only. Perhaps I should rename 'write' to 'grant_ro'.

>
>> +	unsigned int i, off, len, bytes;
>> +	unsigned int data_len = scsi_bufflen(sc);
>> +	unsigned int data_grants = 0, seg_grants = 0;
>> +	struct scatterlist *sg;
>> +	unsigned long mfn;
>> +	struct scsiif_request_segment *seg;
>> +
>> +	ring_req->nr_segments = 0;
>> +	if (sc->sc_data_direction == DMA_NONE || !data_len)
>> +		return 0;
>> +
>> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i)
>> +		data_grants += PFN_UP(sg->offset + sg->length);
>> +
>> +	if (data_grants > VSCSIIF_SG_TABLESIZE) {
>> +		if (data_grants > info->host->sg_tablesize) {
>> +			shost_printk(PREFIX(ERR), info->host,
>> +			     "Unable to map request_buffer for command!\n");
>> +			return -E2BIG;
>> +		}
>> +		seg_grants = vscsiif_grants_sg(data_grants);
>> +		shadow->sg = kcalloc(data_grants,
>> +			sizeof(struct scsiif_request_segment), GFP_NOIO);
>> +		if (!shadow->sg)
>> +			return -ENOMEM;
>> +	}
>> +	seg = shadow->sg ? : ring_req->seg;
>> +
>> +	err = gnttab_alloc_grant_references(seg_grants + data_grants,
>> +					    &gref_head);
>> +	if (err) {
>> +		kfree(shadow->sg);
>> +		shost_printk(PREFIX(ERR), info->host,
>> +			     "gnttab_alloc_grant_references() error\n");
>> +		return -ENOMEM;
>> +	}
>> +
>> +	if (seg_grants) {
>> +		page = virt_to_page(seg);
>> +		off = (unsigned long)seg & ~PAGE_MASK;
>> +		len = sizeof(struct scsiif_request_segment) * data_grants;
>> +		while (len > 0) {
>> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
>> +
>> +			ref = gnttab_claim_grant_reference(&gref_head);
>> +			BUG_ON(ref == -ENOSPC);
>> +
>> +			mfn = pfn_to_mfn(page_to_pfn(page));
>> +			gnttab_grant_foreign_access_ref(ref,
>> +				info->dev->otherend_id, mfn, 1);
>> +			shadow->gref[ref_cnt] = ref;
>> +			ring_req->seg[ref_cnt].gref   = ref;
>> +			ring_req->seg[ref_cnt].offset = (uint16_t)off;
>> +			ring_req->seg[ref_cnt].length = (uint16_t)bytes;
>> +
>> +			page++;
>> +			len -= bytes;
>> +			off = 0;
>> +			ref_cnt++;
>> +		}
>> +		BUG_ON(seg_grants < ref_cnt);
>> +		seg_grants = ref_cnt;
>> +	}
>> +
>> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i) {
>> +		page = sg_page(sg);
>> +		off = sg->offset;
>> +		len = sg->length;
>> +
>> +		while (len > 0 && data_len > 0) {
>> +			/*
>> +			 * sg sends a scatterlist that is larger than
>> +			 * the data_len it wants transferred for certain
>> +			 * IO sizes
>
> Full stop missing.

Yep.

>> +			 */
>> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
>> +			bytes = min(bytes, data_len);
>> +
>> +			ref = gnttab_claim_grant_reference(&gref_head);
>> +			BUG_ON(ref == -ENOSPC);
>> +
>> +			mfn = pfn_to_mfn(page_to_pfn(page));
>> +			gnttab_grant_foreign_access_ref(ref,
>> +				info->dev->otherend_id, mfn, write);
>> +
>> +			shadow->gref[ref_cnt] = ref;
>> +			seg->gref   = ref;
>> +			seg->offset = (uint16_t)off;
>> +			seg->length = (uint16_t)bytes;
>> +
>> +			page++;
>> +			seg++;
>> +			len -= bytes;
>> +			data_len -= bytes;
>> +			off = 0;
>> +			ref_cnt++;
>> +		}
>> +	}
>> +
>> +	if (seg_grants)
>> +		ring_req->nr_segments = VSCSIIF_SG_GRANT | seg_grants;
>> +	else
>> +		ring_req->nr_segments = (uint8_t)ref_cnt;
>> +	shadow->nr_grants = ref_cnt;
>> +
>> +	return 0;
>> +}
>> +
>> +static struct vscsiif_request *scsifront_command2ring(
>> +		struct vscsifrnt_info *info, struct scsi_cmnd *sc,
>> +		struct vscsifrnt_shadow *shadow)
>> +{
>> +	struct vscsiif_request *ring_req;
>> +
>> +	memset(shadow, 0, sizeof(*shadow));
>> +
>> +	ring_req = scsifront_pre_req(info);
>> +	if (!ring_req)
>> +		return NULL;
>> +
>> +	info->shadow[ring_req->rqid] = shadow;
>> +	shadow->rqid = ring_req->rqid;
>> +
>> +	ring_req->id      = sc->device->id;
>> +	ring_req->lun     = sc->device->lun;
>> +	ring_req->channel = sc->device->channel;
>> +	ring_req->cmd_len = sc->cmd_len;
>> +
>> +	BUG_ON(sc->cmd_len > VSCSIIF_MAX_COMMAND_SIZE);
>> +
>> +	memcpy(ring_req->cmnd, sc->cmnd, sc->cmd_len);
>> +
>> +	ring_req->sc_data_direction   = (uint8_t)sc->sc_data_direction;
>> +	ring_req->timeout_per_command = sc->request->timeout / HZ;
>> +
>> +	return ring_req;
>> +}
>> +
>> +static int scsifront_queuecommand(struct Scsi_Host *shost,
>> +				  struct scsi_cmnd *sc)
>> +{
>> +	struct vscsifrnt_info *info = shost_priv(shost);
>> +	struct vscsiif_request *ring_req;
>> +	struct vscsifrnt_shadow *shadow = scsi_cmd_priv(sc);
>> +	unsigned long flags;
>> +	int err;
>> +	uint16_t rqid;
>> +
>
> BUG_ON(sc->sc_data_direction == DMA_BIDIRECTIONAL); ?

No.

>
>
>> +	spin_lock_irqsave(shost->host_lock, flags);
>> +	if (RING_FULL(&info->ring))
>> +		goto busy;
>> +
>> +	ring_req = scsifront_command2ring(info, sc, shadow);
>> +	if (!ring_req)
>> +		goto busy;
>> +
>> +	sc->result    = 0;
>
> This has some odd spacing? Is it suppose to be at the same
> aligment as the ones below? Maybe it is my editor having
> issues.

Historical reasons. :-)
I'll change it.

>
>> +
>> +	rqid              = ring_req->rqid;
>> +	ring_req->act     = VSCSIIF_ACT_SCSI_CDB;
>> +
>> +	shadow->sc  = sc;
>> +	shadow->act = VSCSIIF_ACT_SCSI_CDB;
>> +
>> +	err = map_data_for_request(info, sc, ring_req, shadow);
>> +	if (err < 0) {
>> +		DPRINTK("%s: err %d\n", __func__, err);
>> +		scsifront_put_rqid(info, rqid);
>> +		spin_unlock_irqrestore(shost->host_lock, flags);
>> +		if (err == -ENOMEM)
>> +			return SCSI_MLQUEUE_HOST_BUSY;
>> +		sc->result = DID_ERROR << 16;
>> +		sc->scsi_done(sc);
>> +		return 0;
>> +	}
>> +
>> +	scsifront_do_request(info);
>> +	spin_unlock_irqrestore(shost->host_lock, flags);
>> +
>> +	return 0;
>> +
>> +busy:
>> +	spin_unlock_irqrestore(shost->host_lock, flags);
>> +	DPRINTK("%s: busy\n", __func__);
>> +	return SCSI_MLQUEUE_HOST_BUSY;
>> +}
>> +
>> +/*
>> + * Any exception handling (reset or abort) must be forwarded to the backend.
>> + * We have to wait until an answer is returned. This answer contains the
>> + * result to be returned to the requestor.
>> + */
>> +static int scsifront_action_handler(struct scsi_cmnd *sc, uint8_t act)
>> +{
>> +	struct Scsi_Host *host = sc->device->host;
>> +	struct vscsifrnt_info *info = shost_priv(host);
>> +	struct vscsifrnt_shadow *shadow, *s = scsi_cmd_priv(sc);
>> +	struct vscsiif_request *ring_req;
>> +	int err = 0;
>> +
>> +	shadow = kmalloc(sizeof(*shadow), GFP_NOIO);
>> +	if (!shadow)
>> +		return FAILED;
>> +
>> +	for (;;) {
>> +		spin_lock_irq(host->host_lock);
>> +		if (!RING_FULL(&info->ring)) {
>> +			ring_req = scsifront_command2ring(info, sc, shadow);
>> +			if (ring_req)
>> +				break;
>> +		}
>> +		if (err) {
>> +			spin_unlock_irq(host->host_lock);
>> +			kfree(shadow);
>> +			return FAILED;
>> +		}
>> +		info->waiting_sync = 1;
>> +		spin_unlock_irq(host->host_lock);
>> +		err = wait_event_interruptible(info->wq_sync,
>> +					       !info->waiting_sync);
>> +		spin_lock_irq(host->host_lock);
>> +	}
>> +
>> +	ring_req->act = act;
>> +	ring_req->ref_rqid = s->rqid;
>> +
>> +	shadow->act = act;
>> +	shadow->rslt_reset = 0;
>> +	init_waitqueue_head(&shadow->wq_reset);
>> +
>> +	ring_req->nr_segments         = 0;
>
> Something odd with the spaces here.

Okay.

>
>> +
>> +	scsifront_do_request(info);
>> +
>> +	spin_unlock_irq(host->host_lock);
>> +	err = wait_event_interruptible(shadow->wq_reset, shadow->wait_reset);
>> +	spin_lock_irq(host->host_lock);
>> +
>> +	if (!err) {
>> +		err = shadow->rslt_reset;
>> +		scsifront_put_rqid(info, shadow->rqid);
>> +		kfree(shadow);
>> +	} else {
>> +		spin_lock(&info->shadow_lock);
>> +		shadow->rslt_reset = -1;
>
> #define for -1?

Done.

>
>> +		spin_unlock(&info->shadow_lock);
>> +		err = FAILED;
>> +	}
>> +
>> +	spin_unlock_irq(host->host_lock);
>> +	return err;
>> +}
>> +
>> +static int scsifront_eh_abort_handler(struct scsi_cmnd *sc)
>> +{
>> +	DPRINTK("%s\n", __func__);
>> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_ABORT);
>> +}
>> +
>> +static int scsifront_dev_reset_handler(struct scsi_cmnd *sc)
>> +{
>> +	DPRINTK("%s\n", __func__);
>> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_RESET);
>> +}
>> +
>> +static int scsifront_sdev_configure(struct scsi_device *sdev)
>> +{
>> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
>> +
>> +	if (info && current == info->curr)
>> +		xenbus_printf(XBT_NIL, info->dev->nodename,
>> +			      info->dev_state_path, "%d", XenbusStateConnected);
>> +
>> +	return 0;
>> +}
>> +
>> +static void scsifront_sdev_destroy(struct scsi_device *sdev)
>> +{
>> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
>> +
>> +	if (info && current == info->curr)
>> +		xenbus_printf(XBT_NIL, info->dev->nodename,
>> +			      info->dev_state_path, "%d", XenbusStateClosed);
>> +}
>> +
>> +static struct scsi_host_template scsifront_sht = {
>> +	.module			= THIS_MODULE,
>> +	.name			= "Xen SCSI frontend driver",
>> +	.queuecommand		= scsifront_queuecommand,
>> +	.eh_abort_handler	= scsifront_eh_abort_handler,
>> +	.eh_device_reset_handler = scsifront_dev_reset_handler,
>> +	.slave_configure	= scsifront_sdev_configure,
>> +	.slave_destroy		= scsifront_sdev_destroy,
>> +	.cmd_per_lun		= VSCSIIF_DEFAULT_CMD_PER_LUN,
>> +	.can_queue		= VSCSIIF_MAX_REQS,
>> +	.this_id		= -1,
>> +	.cmd_size		= sizeof(struct vscsifrnt_shadow),
>> +	.sg_tablesize		= VSCSIIF_SG_TABLESIZE,
>> +	.use_clustering		= DISABLE_CLUSTERING,
>> +	.proc_name		= "scsifront",
>> +};
>> +
>> +static int scsifront_alloc_ring(struct vscsifrnt_info *info)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	struct vscsiif_sring *sring;
>> +	int err = -ENOMEM;
>> +
>> +	/***** Frontend to Backend ring start *****/
>> +	sring = (struct vscsiif_sring *) __get_free_page(GFP_KERNEL);
>> +	if (!sring) {
>> +		xenbus_dev_fatal(dev, err,
>> +			"fail to allocate shared ring (Front to Back)");
>> +		return err;
>> +	}
>> +	SHARED_RING_INIT(sring);
>> +	FRONT_RING_INIT(&info->ring, sring, PAGE_SIZE);
>> +
>> +	err = xenbus_grant_ring(dev, virt_to_mfn(sring));
>> +	if (err < 0) {
>> +		free_page((unsigned long) sring);
>
> You can remove the space there.

Indeed.

>
>> +		xenbus_dev_fatal(dev, err,
>> +			"fail to grant shared ring (Front to Back)");
>> +		return err;
>> +	}
>> +	info->ring_ref = err;
>> +
>> +	err = xenbus_alloc_evtchn(dev, &info->evtchn);
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "xenbus_alloc_evtchn");
>> +		goto free_gnttab;
>> +	}
>> +
>> +	err = bind_evtchn_to_irq(info->evtchn);
>> +	if (err <= 0) {
>> +		xenbus_dev_fatal(dev, err, "bind_evtchn_to_irq");
>> +		goto free_gnttab;
>> +	}
>> +
>> +	info->irq = err;
>> +
>> +	err = request_threaded_irq(info->irq, NULL, scsifront_irq_fn,
>> +				   IRQF_ONESHOT, "scsifront", info);
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "request_threaded_irq");
>> +		goto free_irq;
>> +	}
>> +
>> +	return 0;
>> +
>> +/* free resource */
>> +free_irq:
>> +	unbind_from_irqhandler(info->irq, info);
>> +free_gnttab:
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +
>
> free_page((unsigned long)sring);

No, gnttab_end_foreign_access() is doing it.

>
>> +	return err;
>> +}
>> +
>> +static int scsifront_init_ring(struct vscsifrnt_info *info)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	struct xenbus_transaction xbt;
>> +	int err;
>> +
>> +	DPRINTK("%s\n", __func__);
>> +
>> +	err = scsifront_alloc_ring(info);
>> +	if (err)
>> +		return err;
>> +	DPRINTK("%u %u\n", info->ring_ref, info->evtchn);
>> +
>> +again:
>> +	err = xenbus_transaction_start(&xbt);
>> +	if (err)
>> +		xenbus_dev_fatal(dev, err, "starting transaction");
>> +
>> +	err = xenbus_printf(xbt, dev->nodename, "ring-ref", "%u",
>> +			    info->ring_ref);
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "%s", "writing ring-ref");
>> +		goto fail;
>> +	}
>> +
>> +	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
>> +			    info->evtchn);
>> +
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "%s", "writing event-channel");
>> +		goto fail;
>> +	}
>> +
>> +	err = xenbus_transaction_end(xbt, 0);
>> +	if (err) {
>> +		if (err == -EAGAIN)
>> +			goto again;
>> +		xenbus_dev_fatal(dev, err, "completing transaction");
>> +		goto free_sring;
>> +	}
>> +
>> +	return 0;
>> +
>> +fail:
>> +	xenbus_transaction_end(xbt, 1);
>> +free_sring:
>> +	unbind_from_irqhandler(info->irq, info);
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +
>
> The label says 'free_sring' but I am not seeing it being freed?

gnttab_end_foreign_access() is doing magic things. :-)

>
>> +	return err;
>> +}
>> +
>> +
>> +static int scsifront_probe(struct xenbus_device *dev,
>> +			   const struct xenbus_device_id *id)
>> +{
>> +	struct vscsifrnt_info *info;
>> +	struct Scsi_Host *host;
>> +	int err = -ENOMEM;
>> +	char name[DEFAULT_TASK_COMM_LEN];
>> +
>> +	host = scsi_host_alloc(&scsifront_sht, sizeof(*info));
>> +	if (!host) {
>> +		xenbus_dev_fatal(dev, err, "fail to allocate scsi host");
>> +		return err;
>> +	}
>> +	info = (struct vscsifrnt_info *)host->hostdata;
>> +
>> +	dev_set_drvdata(&dev->dev, info);
>> +	info->dev  = dev;
>
> Extra space.

They pile up already. :-)

>
>> +
>> +	info->shadow_free = (1UL << VSCSIIF_MAX_REQS) - 1;
>> +
>> +	err = scsifront_init_ring(info);
>> +	if (err) {
>> +		scsi_host_put(host);
>> +		return err;
>> +	}
>> +
>> +	init_waitqueue_head(&info->wq_sync);
>> +	spin_lock_init(&info->shadow_lock);
>> +
>> +	snprintf(name, DEFAULT_TASK_COMM_LEN, "vscsiif.%d", host->host_no);
>> +
>> +	host->max_id      = VSCSIIF_MAX_TARGET;
>> +	host->max_channel = 0;
>> +	host->max_lun     = VSCSIIF_MAX_LUN;
>> +	host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
>> +	host->max_cmd_len = VSCSIIF_MAX_COMMAND_SIZE;
>> +
>> +	err = scsi_add_host(host, &dev->dev);
>> +	if (err) {
>> +		dev_err(&dev->dev, "fail to add scsi host %d\n", err);
>> +		goto free_sring;
>> +	}
>> +	info->host = host;
>> +	info->host_active = 1;
>> +
>> +	xenbus_switch_state(dev, XenbusStateInitialised);
>> +
>> +	return 0;
>> +
>> +free_sring:
>> +	unbind_from_irqhandler(info->irq, info);
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +	scsi_host_put(host);
>
> The label says 'free_sring' but I am not seeing it being freed?

As above.

>
> Hm, could those operations - unbind_from_irqhandler, gnttab_end_foreign_access
> and free_page be moved to a seperate function to call?

I don't think it's necessary, those are only _two_ function calls.

>
>> +	return err;
>> +}
>> +
>> +static int scsifront_remove(struct xenbus_device *dev)
>> +{
>> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
>> +
>> +	DPRINTK("%s: %s removed\n", __func__, dev->nodename);
>> +
>> +	if (info->host_active) {
>> +		/* Scsi_host not yet removed */
>> +		scsi_remove_host(info->host);
>> +		info->host_active = 0;
>> +	}
>> +
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +	unbind_from_irqhandler(info->irq, info);
>> +
>
> Should you free the ring?

I did.

>
>> +	scsi_host_put(info->host);
>> +
>> +	return 0;
>> +}
>> +
>> +static void scsifront_disconnect(struct vscsifrnt_info *info)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	struct Scsi_Host *host = info->host;
>> +
>> +	DPRINTK("%s: %s disconnect\n", __func__, dev->nodename);
>> +
>> +	/*
>> +	 * When this function is executed, all devices of
>> +	 * Frontend have been deleted.
>> +	 * Therefore, it need not block I/O before remove_host.
>> +	 */
>> +
>> +	if (info->host_active)
>> +		scsi_remove_host(host);
>> +	info->host_active = 0;
>> +
>> +	xenbus_frontend_closed(dev);
>> +}
>> +
>> +static void scsifront_do_lun_hotplug(struct vscsifrnt_info *info, int op)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	int i, err = 0;
>> +	char str[64];
>> +	char **dir;
>> +	unsigned int dir_n = 0;
>> +	unsigned int device_state;
>> +	unsigned int hst, chn, tgt, lun;
>> +	struct scsi_device *sdev;
>> +
>> +	dir = xenbus_directory(XBT_NIL, dev->otherend, "vscsi-devs", &dir_n);
>> +	if (IS_ERR(dir))
>> +		return;
>> +
>> +	/* mark current task as the one allowed to modify device states */
>> +	BUG_ON(info->curr);
>> +	info->curr = current;
>> +
>> +	for (i = 0; i < dir_n; i++) {
>> +		/* read status */
>> +		snprintf(str, sizeof(str), "vscsi-devs/%s/state", dir[i]);
>> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str, "%u",
>> +				   &device_state);
>> +		if (XENBUS_EXIST_ERR(err))
>> +			continue;
>> +
>> +		/* virtual SCSI device */
>> +		snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", dir[i]);
>> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str,
>> +				   "%u:%u:%u:%u", &hst, &chn, &tgt, &lun);
>> +		if (XENBUS_EXIST_ERR(err))
>> +			continue;
>> +
>> +		/*
>> +		 * Front device state path, used in slave_configure called
>> +		 * on successfull scsi_add_device, and in slave_destroy called
>> +		 * on remove of a device.
>> +		 */
>> +		snprintf(info->dev_state_path, sizeof(info->dev_state_path),
>> +			 "vscsi-devs/%s/state", dir[i]);
>> +
>> +		switch (op) {
>> +		case VSCSIFRONT_OP_ADD_LUN:
>> +			if (device_state == XenbusStateInitialised) {
>
> You could convert to make this a bit easier to read to do:
>
> 	if (device_state != XenbusStateInitialised)
> 		break;
> 	.. and then with the rest of the code.

Yes, that's better.

>
>> +				err = scsi_add_device(info->host, chn, tgt,
>> +						      lun);
>
> You can make that more than 80 characters long.

It's shorter now.

>> +
>> +				if (err) {
>> +					dev_err(&dev->dev, "scsi_add_device\n");
>> +					xenbus_printf(XBT_NIL, dev->nodename,
>> +						      info->dev_state_path,
>> +						      "%d", XenbusStateClosed);
>> +				}
>> +			}
>> +			break;
>> +		case VSCSIFRONT_OP_DEL_LUN:
>> +			if (device_state == XenbusStateClosing) {
>
> Ditto.

Yep.

>> +				sdev = scsi_device_lookup(info->host, chn, tgt,
>> +							  lun);
>> +				if (sdev) {
>> +					scsi_remove_device(sdev);
>> +					scsi_device_put(sdev);
>> +				}
>> +			}
>> +			break;
>> +		default:
>> +			break;
>> +		}
>> +	}
>> +
>> +	info->curr = NULL;
>> +
>> +	kfree(dir);
>> +}
>> +
>> +static void scsifront_read_backend_params(struct xenbus_device *dev,
>> +					  struct vscsifrnt_info *info)
>> +{
>> +	unsigned int sg_grant;
>> +	int ret;
>> +	struct Scsi_Host *host = info->host;
>> +
>> +	ret = xenbus_scanf(XBT_NIL, dev->otherend, "feature-sg-grant", "%u",
>> +			   &sg_grant);
>> +	if (ret == 1 && sg_grant) {
>> +		sg_grant = min_t(unsigned int, sg_grant, SG_ALL);
>> +		host->sg_tablesize = min_t(unsigned int, sg_grant,
>> +			VSCSIIF_SG_TABLESIZE * PAGE_SIZE /
>> +			sizeof(struct scsiif_request_segment));
>> +		dev_info(&dev->dev, "using up to %d SG entries\n",
>> +			 host->sg_tablesize);
>
> Perhaps this dev_info can be printed regardless of whether
> the backend has advertised an value? Isn't this information also
> visible in the SysFS? If so, should it be just removed?

I don't know any entry in SysFS covering this information.

Always printing the information is possible, yes.

>
>> +		host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
>
> If the backend decides to give us an value less than optimal - say '2'
> should we just ignore it? Perhaps have
>
> if (sg_grant < VSCSIIF_SG_TABLESIZE)
> 	/* Pfffff. */
> 	return;

Something like that, yes.

>> +	}
>> +}
>> +
>> +static void scsifront_backend_changed(struct xenbus_device *dev,
>> +				      enum xenbus_state backend_state)
>> +{
>> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
>> +
>> +	DPRINTK("%p %u %u\n", dev, dev->state, backend_state);
>> +
>> +	switch (backend_state) {
>> +	case XenbusStateUnknown:
>> +	case XenbusStateInitialising:
>> +	case XenbusStateInitWait:
>> +	case XenbusStateInitialised:
>> +		break;
>> +
>> +	case XenbusStateConnected:
>> +		scsifront_read_backend_params(dev, info);
>> +		if (xenbus_read_driver_state(dev->nodename) ==
>> +			XenbusStateInitialised) {
>
> You can make this more than 80 characters. And also remove the '{' and '}'

Okay.

>
>
>> +			scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
>> +		}
>> +
>> +		if (dev->state != XenbusStateConnected)
>> +			xenbus_switch_state(dev, XenbusStateConnected);
>> +		break;
>> +
>> +	case XenbusStateClosed:
>> +		if (dev->state == XenbusStateClosed)
>> +			break;
>> +		/* Missed the backend's Closing state -- fallthrough */
>> +	case XenbusStateClosing:
>> +		scsifront_disconnect(info);
>> +		break;
>> +
>> +	case XenbusStateReconfiguring:
>> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_DEL_LUN);
>> +		xenbus_switch_state(dev, XenbusStateReconfiguring);
>> +		break;
>> +
>> +	case XenbusStateReconfigured:
>> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
>> +		xenbus_switch_state(dev, XenbusStateConnected);
>> +		break;
>> +	}
>> +}
>> +
>> +static const struct xenbus_device_id scsifront_ids[] = {
>> +	{ "vscsi" },
>> +	{ "" }
>> +};
>> +
>> +static DEFINE_XENBUS_DRIVER(scsifront, ,
>> +	.probe			= scsifront_probe,
>> +	.remove			= scsifront_remove,
>> +	.otherend_changed	= scsifront_backend_changed,
>> +);
>> +
>> +static int __init scsifront_init(void)
>> +{
>> +	if (!xen_domain())
>> +		return -ENODEV;
>> +
>> +	return xenbus_register_frontend(&scsifront_driver);
>> +}
>> +module_init(scsifront_init);
>> +
>> +static void __exit scsifront_exit(void)
>> +{
>> +	xenbus_unregister_driver(&scsifront_driver);
>> +}
>> +module_exit(scsifront_exit);
>> +
>> +MODULE_DESCRIPTION("Xen SCSI frontend driver");
>> +MODULE_LICENSE("GPL");
>> +MODULE_ALIAS("xen:vscsi");
>
> MODULE_AUTHOR?

I don't mind putting myself in there, but I'm not the original author...


Juergen

Re: [PATCH V5 3/5] Introduce xen-scsifront module

[Juergen Gross]

On 08/23/2014 12:25 AM, Konrad Rzeszutek Wilk wrote:
>> --- /dev/null
>> +++ b/drivers/scsi/xen-scsifront.c
>> @@ -0,0 +1,1017 @@
>> +/*
>> + * Xen SCSI frontend driver
>> + *
>> + * Copyright (c) 2008, FUJITSU Limited
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU General Public License version 2
>> + * as published by the Free Software Foundation; or, when distributed
>> + * separately from the Linux kernel or incorporated into other
>> + * software packages, subject to the following license:
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this source file (the "Software"), to deal in the Software without
>> + * restriction, including without limitation the rights to use, copy, modify,
>> + * merge, publish, distribute, sublicense, and/or sell copies of the Software,
>> + * and to permit persons to whom the Software is furnished to do so, subject to
>> + * the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
>> + * IN THE SOFTWARE.
>> + */
>> +
>> +#define DEBUG
>
> :-) I think you don't want that in the driver.

Correct. :-)

>
>> +
>> +#include <linux/module.h>
>> +#include <linux/kernel.h>
>> +#include <linux/device.h>
>> +#include <linux/wait.h>
>> +#include <linux/interrupt.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/sched.h>
>> +#include <linux/blkdev.h>
>> +#include <linux/pfn.h>
>> +#include <linux/slab.h>
>> +
>> +#include <scsi/scsi_cmnd.h>
>> +#include <scsi/scsi_device.h>
>> +#include <scsi/scsi.h>
>> +#include <scsi/scsi_host.h>
>> +
>> +#include <xen/xen.h>
>> +#include <xen/xenbus.h>
>> +#include <xen/grant_table.h>
>> +#include <xen/events.h>
>> +#include <xen/page.h>
>> +
>> +#include <xen/interface/grant_table.h>
>> +#include <xen/interface/io/vscsiif.h>
>> +#include <xen/interface/io/protocols.h>
>> +
>> +#include <asm/xen/hypervisor.h>
>> +
>> +
>> +#define GRANT_INVALID_REF	0
>> +
>> +#define VSCSIFRONT_OP_ADD_LUN	1
>> +#define VSCSIFRONT_OP_DEL_LUN	2
>
> Shouldn't those be defined in the vscsiff.h file?

No, they are private for the frontend.

>
>> +
>> +#define DEFAULT_TASK_COMM_LEN	TASK_COMM_LEN
>
> Not sure why you need the DEFAULT_ ? Could you use TASK_COMM_LEN?

Sure.

>
>> +
>> +/* tuning point*/
>
> Missing stop and a space after the 't':
>
> /* Tuning point. */

Okay.

>
>> +#define VSCSIIF_DEFAULT_CMD_PER_LUN 10
>> +#define VSCSIIF_MAX_TARGET          64
>> +#define VSCSIIF_MAX_LUN             255
>> +
>> +#define VSCSIIF_RING_SIZE	__CONST_RING_SIZE(vscsiif, PAGE_SIZE)
>> +#define VSCSIIF_MAX_REQS	VSCSIIF_RING_SIZE
>> +
>> +#define vscsiif_grants_sg(_sg)	(PFN_UP((_sg) *		\
>> +				sizeof(struct scsiif_request_segment)))
>> +
>> +struct vscsifrnt_shadow {
>> +	/* command between backend and frontend */
>> +	unsigned char act;
>
> act? How about 'op' ? Or 'cmd_op'?

I wanted to use the same name as the related element in vscsiif.h

>
>> +	uint16_t rqid;
>
> rqid? Could you have a comment explaining what that acronym is?
> Oh wait - request id? How about just req_id?

Same again. It's called rqid in vscsiiif.h

>
>> +
>> +	/* Number of pieces of scatter-gather */
>> +	unsigned int nr_grants;
>
> s/nr_grants/nr_sg/ ?

I'll update the comment, as this is really the number of grants.

>
>> +	struct scsiif_request_segment *sg;
>> +
>> +	/* do reset or abort function */
>
> Full stop missing.
>> +	wait_queue_head_t wq_reset;	/* reset work queue           */
>> +	int wait_reset;			/* reset work queue condition */
>> +	int32_t rslt_reset;		/* reset response status      */
>> +					/* (SUCESS or FAILED)         */
>
> Full stop missing. s/SUCESS/SUCCESS/

Okay.

>> +
>> +	/* requested struct scsi_cmnd is stored from kernel */
>
> Full stop missing.

Okay.

>> +	struct scsi_cmnd *sc;
>> +	int gref[vscsiif_grants_sg(SG_ALL) + SG_ALL];
>> +};
>> +
>> +struct vscsifrnt_info {
>> +	struct xenbus_device *dev;
>> +
>> +	struct Scsi_Host *host;
>> +	int host_active;
>
> This 'host_active' seems to be a guard to call 'scsi_host_remove'
> through either the disconnect (so backend told us to disconnect)
> or the .remove (so XenStore keys are moved). Either way I think
> it is possible to run _both_ of them and this being just
> an 'int' is not sufficient.
>
> I would recommend you change this to an ref_count.

Hmm, I think a lock is required here.

>> +
>> +	spinlock_t shadow_lock;
>
> Could you move this spinlock below - to where
> you have tons of of 'shadow' values please?

Okay.

>
>> +	unsigned int evtchn;
>> +	unsigned int irq;
>> +
>> +	grant_ref_t ring_ref;
>> +	struct vscsiif_front_ring ring;
>> +	struct vscsiif_response	ring_res;
>> +
>> +	unsigned long shadow_free;
>
> A comment would help in explainig this. 'shadow_free' sounds
> a bit cryptic. Oh, and xen-blkfront has it as 'shadow_free'
> too! Argh, 'shadow_free_bitmask' could be a better name?
> Oh, what if we converted to an bitmask type?
>
> Either way - if you think the existing one should be this way
> then lets leave it as is. But if you think a better name
> is suited I can change in xen-blkfront too.

I'll use the bitmap operations and call it shadow_free_bitmap.

>
>> +	struct vscsifrnt_shadow *shadow[VSCSIIF_MAX_REQS];
>> +
>> +	wait_queue_head_t wq_sync;
>> +	unsigned int waiting_sync:1;
>
> That sounds like it needs a spinlock to protect it?

It is protected by info->host->host_lock.

> Could you also explain a bit of what its purpose is? What
> is it suppose to wait for? It looks like it is tied to the
> error handler, perhaps then it ought to be called:
>
> err_handler_done

It is used by the error handler, yes. But this is only due to the
fact that the error handler can't return with SCSI_MLQUEUE_HOST_BUSY.

I think I rename it to wait_ring_available, because this is the real
meaning: someone is waiting until a request can be allocated in the
ring.

>
>
>> +
>> +	char dev_state_path[64];
>> +	struct task_struct *curr;
>> +};
>> +
>> +#define DPRINTK(_f, _a...)				\
>> +	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)
>
> Could all the DPRINTK be converted to pr_debug right away?

Yes, I'll do it.

>> +
>> +#define PREFIX(lvl) KERN_##lvl "scsifront: "
>
> Perhaps you want?
>
> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

Not quite, but I'll remove the PREFIX define.

>> +
>> +static void scsifront_wake_up(struct vscsifrnt_info *info)
>> +{
>> +	info->waiting_sync = 0;
>> +	wake_up(&info->wq_sync);
>> +}
>> +
>> +static int scsifront_get_rqid(struct vscsifrnt_info *info)
>> +{
>> +	unsigned long flags;
>> +	int free;
>> +
>> +	spin_lock_irqsave(&info->shadow_lock, flags);
>> +
>> +	free = find_first_bit(&info->shadow_free, VSCSIIF_MAX_REQS);
>> +	info->shadow_free &= ~(1UL << free);
>> +
>> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +
>> +	return free;
>> +}
>> +
>> +static int _scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
>> +{
>> +	info->shadow_free |= 1UL << id;
>> +	info->shadow[id] = NULL;
>> +
>> +	return (info->shadow_free == 1UL << id || info->waiting_sync);
>> +}
>> +
>> +static void scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
>> +{
>> +	unsigned long flags;
>> +	int was_empty;
>> +
>> +	spin_lock_irqsave(&info->shadow_lock, flags);
>> +	was_empty = _scsifront_put_rqid(info, id);
>
> I wouldn't call this 'was_empty' as it will also be true if the 'waiting_sync'
> is turned on. Perhaps this should be called: 'kick' or 'wake_wq' ?

kick is okay, I think.

>
>> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +
>> +	if (was_empty)
>> +		scsifront_wake_up(info);
>> +}
>> +
>> +static struct vscsiif_request *scsifront_pre_req(struct vscsifrnt_info *info)
>> +{
>> +	struct vscsiif_front_ring *ring = &(info->ring);
>> +	struct vscsiif_request *ring_req;
>> +	uint32_t id;
>> +
>> +	id = scsifront_get_rqid(info);	/* use id by response */
>
> s/by/in/

Okay.

>
>> +	if (id >= VSCSIIF_MAX_REQS)
>> +		return NULL;
>> +
>> +	ring_req = RING_GET_REQUEST(&(info->ring), ring->req_prod_pvt);
>> +
>> +	ring->req_prod_pvt++;
>> +
>> +	ring_req->rqid = (uint16_t)id;
>> +
>> +	return ring_req;
>> +}
>> +
>> +static void scsifront_do_request(struct vscsifrnt_info *info)
>> +{
>> +	struct vscsiif_front_ring *ring = &(info->ring);
>> +	int notify;
>> +
>> +	RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(ring, notify);
>> +	if (notify)
>> +		notify_remote_via_irq(info->irq);
>> +}
>> +
>> +static void scsifront_gnttab_done(struct vscsifrnt_info *info, uint32_t id)
>> +{
>> +	struct vscsifrnt_shadow *s = info->shadow[id];
>> +	int i;
>> +
>> +	if (s->sc->sc_data_direction == DMA_NONE)
>> +		return;
>> +
>> +	for (i = 0; i < s->nr_grants; i++) {
>> +		if (unlikely(gnttab_query_foreign_access(s->gref[i]) != 0)) {
>> +			shost_printk(PREFIX(ALERT), info->host,
>> +				     "grant still in use by backend\n");
>> +			BUG();
>> +		}
>> +		gnttab_end_foreign_access(s->gref[i], 0, 0UL);
>> +	}
>> +
>> +	kfree(s->sg);
>> +}
>> +
>> +static void scsifront_cdb_cmd_done(struct vscsifrnt_info *info,
>> +				   struct vscsiif_response *ring_res)
>
> s/ring_res/ring_rsp/ ?

Yes, that's better.

>
>> +{
>> +	struct scsi_cmnd *sc;
>> +	uint32_t id;
>> +	uint8_t sense_len;
>> +
>> +	id = ring_res->rqid;
>> +	sc = info->shadow[id]->sc;
>> +
>> +	BUG_ON(sc == NULL);
>> +
>> +	scsifront_gnttab_done(info, id);
>> +	scsifront_put_rqid(info, id);
>> +
>> +	sc->result = ring_res->rslt;
>> +	scsi_set_resid(sc, ring_res->residual_len);
>> +
>> +	sense_len = min_t(uint8_t, VSCSIIF_SENSE_BUFFERSIZE,
>> +			  ring_res->sense_len);
>> +
>> +	if (sense_len)
>> +		memcpy(sc->sense_buffer, ring_res->sense_buffer, sense_len);
>> +
>> +	sc->scsi_done(sc);
>> +}
>> +
>> +static void scsifront_sync_cmd_done(struct vscsifrnt_info *info,
>> +				    struct vscsiif_response *ring_res)
>> +{
>> +	uint16_t id = ring_res->rqid;
>> +	unsigned long flags;
>> +	struct vscsifrnt_shadow *shadow = info->shadow[id];
>> +	int was_empty;
>> +
>> +	spin_lock_irqsave(&info->shadow_lock, flags);
>> +	shadow->wait_reset = 1;
>> +	switch (shadow->rslt_reset) {
>> +	case 0:
>> +		shadow->rslt_reset = ring_res->rslt;
>> +		break;
>> +	case -1:
>
> Is there an #define for this? The comment at the top mentioned
> SUCCESS and FAILED ?

Adding defines.

>
>> +		was_empty = _scsifront_put_rqid(info, id);
>
> Perhaps call it 'kick' or 'wake_wq' ?

kick, again.

>
>> +		spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +		kfree(shadow);
>> +		if (was_empty)
>> +			scsifront_wake_up(info);
>> +		return;
>> +	default:
>> +		shost_printk(PREFIX(ERR), info->host,
>> +			     "bad reset state %d, possibly leaking %u\n",
>> +			     shadow->rslt_reset, id);
>> +		break;
>> +	}
>> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
>> +
>> +	wake_up(&shadow->wq_reset);
>> +}
>> +
>> +static int scsifront_cmd_done(struct vscsifrnt_info *info)
>> +{
>> +	struct vscsiif_response *ring_res;
>> +	RING_IDX i, rp;
>> +	int more_to_do = 0;
>> +	unsigned long flags;
>> +
>> +	spin_lock_irqsave(info->host->host_lock, flags);
>> +
>> +	rp = info->ring.sring->rsp_prod;
>> +	rmb();	/* ordering required respective to dom0 */
>> +	for (i = info->ring.rsp_cons; i != rp; i++) {
>> +
>> +		ring_res = RING_GET_RESPONSE(&info->ring, i);
>
> I would recommend you look at git commit 6878c32e5cc0e40980abe51d1f02fb453e27493e
>
> As the 'rqid' value - might be corrupted by the backend. Which means
> that the 'info->shadow[rubbish_val]->act' could blow up.
>
> I would just add a check to make sure that the 'rqid' value is
> within the expected values.

I'll add a check (the rqid should be in use as well).

>
>> +
>> +		if (info->shadow[ring_res->rqid]->act == VSCSIIF_ACT_SCSI_CDB)
>> +			scsifront_cdb_cmd_done(info, ring_res);
>> +		else
>> +			scsifront_sync_cmd_done(info, ring_res);
>> +	}
>> +
>> +	info->ring.rsp_cons = i;
>> +
>> +	if (i != info->ring.req_prod_pvt)
>> +		RING_FINAL_CHECK_FOR_RESPONSES(&info->ring, more_to_do);
>> +	else
>> +		info->ring.sring->rsp_event = i + 1;
>> +
>> +	info->waiting_sync = 0;
>> +
>> +	spin_unlock_irqrestore(info->host->host_lock, flags);
>> +
>> +	wake_up(&info->wq_sync);
>> +
>> +	return more_to_do;
>> +}
>> +
>> +static irqreturn_t scsifront_irq_fn(int irq, void *dev_id)
>> +{
>> +	struct vscsifrnt_info *info = dev_id;
>> +
>> +	while (scsifront_cmd_done(info))
>> +		/* Yield point for this unbounded loop. */
>> +		cond_resched();
>> +
>> +	return IRQ_HANDLED;
>> +}
>> +
>> +static int map_data_for_request(struct vscsifrnt_info *info,
>> +				struct scsi_cmnd *sc,
>> +				struct vscsiif_request *ring_req,
>> +				struct vscsifrnt_shadow *shadow)
>> +{
>> +	grant_ref_t gref_head;
>> +	struct page *page;
>> +	int err, ref, ref_cnt = 0;
>> +	int write = (sc->sc_data_direction == DMA_TO_DEVICE);
>
> What if it is DMA_BIDIRECTIONAL ?

That's okay. DMA_TO_DEVICE is the only case where a grant is flagged as
read only. Perhaps I should rename 'write' to 'grant_ro'.

>
>> +	unsigned int i, off, len, bytes;
>> +	unsigned int data_len = scsi_bufflen(sc);
>> +	unsigned int data_grants = 0, seg_grants = 0;
>> +	struct scatterlist *sg;
>> +	unsigned long mfn;
>> +	struct scsiif_request_segment *seg;
>> +
>> +	ring_req->nr_segments = 0;
>> +	if (sc->sc_data_direction == DMA_NONE || !data_len)
>> +		return 0;
>> +
>> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i)
>> +		data_grants += PFN_UP(sg->offset + sg->length);
>> +
>> +	if (data_grants > VSCSIIF_SG_TABLESIZE) {
>> +		if (data_grants > info->host->sg_tablesize) {
>> +			shost_printk(PREFIX(ERR), info->host,
>> +			     "Unable to map request_buffer for command!\n");
>> +			return -E2BIG;
>> +		}
>> +		seg_grants = vscsiif_grants_sg(data_grants);
>> +		shadow->sg = kcalloc(data_grants,
>> +			sizeof(struct scsiif_request_segment), GFP_NOIO);
>> +		if (!shadow->sg)
>> +			return -ENOMEM;
>> +	}
>> +	seg = shadow->sg ? : ring_req->seg;
>> +
>> +	err = gnttab_alloc_grant_references(seg_grants + data_grants,
>> +					    &gref_head);
>> +	if (err) {
>> +		kfree(shadow->sg);
>> +		shost_printk(PREFIX(ERR), info->host,
>> +			     "gnttab_alloc_grant_references() error\n");
>> +		return -ENOMEM;
>> +	}
>> +
>> +	if (seg_grants) {
>> +		page = virt_to_page(seg);
>> +		off = (unsigned long)seg & ~PAGE_MASK;
>> +		len = sizeof(struct scsiif_request_segment) * data_grants;
>> +		while (len > 0) {
>> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
>> +
>> +			ref = gnttab_claim_grant_reference(&gref_head);
>> +			BUG_ON(ref == -ENOSPC);
>> +
>> +			mfn = pfn_to_mfn(page_to_pfn(page));
>> +			gnttab_grant_foreign_access_ref(ref,
>> +				info->dev->otherend_id, mfn, 1);
>> +			shadow->gref[ref_cnt] = ref;
>> +			ring_req->seg[ref_cnt].gref   = ref;
>> +			ring_req->seg[ref_cnt].offset = (uint16_t)off;
>> +			ring_req->seg[ref_cnt].length = (uint16_t)bytes;
>> +
>> +			page++;
>> +			len -= bytes;
>> +			off = 0;
>> +			ref_cnt++;
>> +		}
>> +		BUG_ON(seg_grants < ref_cnt);
>> +		seg_grants = ref_cnt;
>> +	}
>> +
>> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i) {
>> +		page = sg_page(sg);
>> +		off = sg->offset;
>> +		len = sg->length;
>> +
>> +		while (len > 0 && data_len > 0) {
>> +			/*
>> +			 * sg sends a scatterlist that is larger than
>> +			 * the data_len it wants transferred for certain
>> +			 * IO sizes
>
> Full stop missing.

Yep.

>> +			 */
>> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
>> +			bytes = min(bytes, data_len);
>> +
>> +			ref = gnttab_claim_grant_reference(&gref_head);
>> +			BUG_ON(ref == -ENOSPC);
>> +
>> +			mfn = pfn_to_mfn(page_to_pfn(page));
>> +			gnttab_grant_foreign_access_ref(ref,
>> +				info->dev->otherend_id, mfn, write);
>> +
>> +			shadow->gref[ref_cnt] = ref;
>> +			seg->gref   = ref;
>> +			seg->offset = (uint16_t)off;
>> +			seg->length = (uint16_t)bytes;
>> +
>> +			page++;
>> +			seg++;
>> +			len -= bytes;
>> +			data_len -= bytes;
>> +			off = 0;
>> +			ref_cnt++;
>> +		}
>> +	}
>> +
>> +	if (seg_grants)
>> +		ring_req->nr_segments = VSCSIIF_SG_GRANT | seg_grants;
>> +	else
>> +		ring_req->nr_segments = (uint8_t)ref_cnt;
>> +	shadow->nr_grants = ref_cnt;
>> +
>> +	return 0;
>> +}
>> +
>> +static struct vscsiif_request *scsifront_command2ring(
>> +		struct vscsifrnt_info *info, struct scsi_cmnd *sc,
>> +		struct vscsifrnt_shadow *shadow)
>> +{
>> +	struct vscsiif_request *ring_req;
>> +
>> +	memset(shadow, 0, sizeof(*shadow));
>> +
>> +	ring_req = scsifront_pre_req(info);
>> +	if (!ring_req)
>> +		return NULL;
>> +
>> +	info->shadow[ring_req->rqid] = shadow;
>> +	shadow->rqid = ring_req->rqid;
>> +
>> +	ring_req->id      = sc->device->id;
>> +	ring_req->lun     = sc->device->lun;
>> +	ring_req->channel = sc->device->channel;
>> +	ring_req->cmd_len = sc->cmd_len;
>> +
>> +	BUG_ON(sc->cmd_len > VSCSIIF_MAX_COMMAND_SIZE);
>> +
>> +	memcpy(ring_req->cmnd, sc->cmnd, sc->cmd_len);
>> +
>> +	ring_req->sc_data_direction   = (uint8_t)sc->sc_data_direction;
>> +	ring_req->timeout_per_command = sc->request->timeout / HZ;
>> +
>> +	return ring_req;
>> +}
>> +
>> +static int scsifront_queuecommand(struct Scsi_Host *shost,
>> +				  struct scsi_cmnd *sc)
>> +{
>> +	struct vscsifrnt_info *info = shost_priv(shost);
>> +	struct vscsiif_request *ring_req;
>> +	struct vscsifrnt_shadow *shadow = scsi_cmd_priv(sc);
>> +	unsigned long flags;
>> +	int err;
>> +	uint16_t rqid;
>> +
>
> BUG_ON(sc->sc_data_direction == DMA_BIDIRECTIONAL); ?

No.

>
>
>> +	spin_lock_irqsave(shost->host_lock, flags);
>> +	if (RING_FULL(&info->ring))
>> +		goto busy;
>> +
>> +	ring_req = scsifront_command2ring(info, sc, shadow);
>> +	if (!ring_req)
>> +		goto busy;
>> +
>> +	sc->result    = 0;
>
> This has some odd spacing? Is it suppose to be at the same
> aligment as the ones below? Maybe it is my editor having
> issues.

Historical reasons. :-)
I'll change it.

>
>> +
>> +	rqid              = ring_req->rqid;
>> +	ring_req->act     = VSCSIIF_ACT_SCSI_CDB;
>> +
>> +	shadow->sc  = sc;
>> +	shadow->act = VSCSIIF_ACT_SCSI_CDB;
>> +
>> +	err = map_data_for_request(info, sc, ring_req, shadow);
>> +	if (err < 0) {
>> +		DPRINTK("%s: err %d\n", __func__, err);
>> +		scsifront_put_rqid(info, rqid);
>> +		spin_unlock_irqrestore(shost->host_lock, flags);
>> +		if (err == -ENOMEM)
>> +			return SCSI_MLQUEUE_HOST_BUSY;
>> +		sc->result = DID_ERROR << 16;
>> +		sc->scsi_done(sc);
>> +		return 0;
>> +	}
>> +
>> +	scsifront_do_request(info);
>> +	spin_unlock_irqrestore(shost->host_lock, flags);
>> +
>> +	return 0;
>> +
>> +busy:
>> +	spin_unlock_irqrestore(shost->host_lock, flags);
>> +	DPRINTK("%s: busy\n", __func__);
>> +	return SCSI_MLQUEUE_HOST_BUSY;
>> +}
>> +
>> +/*
>> + * Any exception handling (reset or abort) must be forwarded to the backend.
>> + * We have to wait until an answer is returned. This answer contains the
>> + * result to be returned to the requestor.
>> + */
>> +static int scsifront_action_handler(struct scsi_cmnd *sc, uint8_t act)
>> +{
>> +	struct Scsi_Host *host = sc->device->host;
>> +	struct vscsifrnt_info *info = shost_priv(host);
>> +	struct vscsifrnt_shadow *shadow, *s = scsi_cmd_priv(sc);
>> +	struct vscsiif_request *ring_req;
>> +	int err = 0;
>> +
>> +	shadow = kmalloc(sizeof(*shadow), GFP_NOIO);
>> +	if (!shadow)
>> +		return FAILED;
>> +
>> +	for (;;) {
>> +		spin_lock_irq(host->host_lock);
>> +		if (!RING_FULL(&info->ring)) {
>> +			ring_req = scsifront_command2ring(info, sc, shadow);
>> +			if (ring_req)
>> +				break;
>> +		}
>> +		if (err) {
>> +			spin_unlock_irq(host->host_lock);
>> +			kfree(shadow);
>> +			return FAILED;
>> +		}
>> +		info->waiting_sync = 1;
>> +		spin_unlock_irq(host->host_lock);
>> +		err = wait_event_interruptible(info->wq_sync,
>> +					       !info->waiting_sync);
>> +		spin_lock_irq(host->host_lock);
>> +	}
>> +
>> +	ring_req->act = act;
>> +	ring_req->ref_rqid = s->rqid;
>> +
>> +	shadow->act = act;
>> +	shadow->rslt_reset = 0;
>> +	init_waitqueue_head(&shadow->wq_reset);
>> +
>> +	ring_req->nr_segments         = 0;
>
> Something odd with the spaces here.

Okay.

>
>> +
>> +	scsifront_do_request(info);
>> +
>> +	spin_unlock_irq(host->host_lock);
>> +	err = wait_event_interruptible(shadow->wq_reset, shadow->wait_reset);
>> +	spin_lock_irq(host->host_lock);
>> +
>> +	if (!err) {
>> +		err = shadow->rslt_reset;
>> +		scsifront_put_rqid(info, shadow->rqid);
>> +		kfree(shadow);
>> +	} else {
>> +		spin_lock(&info->shadow_lock);
>> +		shadow->rslt_reset = -1;
>
> #define for -1?

Done.

>
>> +		spin_unlock(&info->shadow_lock);
>> +		err = FAILED;
>> +	}
>> +
>> +	spin_unlock_irq(host->host_lock);
>> +	return err;
>> +}
>> +
>> +static int scsifront_eh_abort_handler(struct scsi_cmnd *sc)
>> +{
>> +	DPRINTK("%s\n", __func__);
>> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_ABORT);
>> +}
>> +
>> +static int scsifront_dev_reset_handler(struct scsi_cmnd *sc)
>> +{
>> +	DPRINTK("%s\n", __func__);
>> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_RESET);
>> +}
>> +
>> +static int scsifront_sdev_configure(struct scsi_device *sdev)
>> +{
>> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
>> +
>> +	if (info && current == info->curr)
>> +		xenbus_printf(XBT_NIL, info->dev->nodename,
>> +			      info->dev_state_path, "%d", XenbusStateConnected);
>> +
>> +	return 0;
>> +}
>> +
>> +static void scsifront_sdev_destroy(struct scsi_device *sdev)
>> +{
>> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
>> +
>> +	if (info && current == info->curr)
>> +		xenbus_printf(XBT_NIL, info->dev->nodename,
>> +			      info->dev_state_path, "%d", XenbusStateClosed);
>> +}
>> +
>> +static struct scsi_host_template scsifront_sht = {
>> +	.module			= THIS_MODULE,
>> +	.name			= "Xen SCSI frontend driver",
>> +	.queuecommand		= scsifront_queuecommand,
>> +	.eh_abort_handler	= scsifront_eh_abort_handler,
>> +	.eh_device_reset_handler = scsifront_dev_reset_handler,
>> +	.slave_configure	= scsifront_sdev_configure,
>> +	.slave_destroy		= scsifront_sdev_destroy,
>> +	.cmd_per_lun		= VSCSIIF_DEFAULT_CMD_PER_LUN,
>> +	.can_queue		= VSCSIIF_MAX_REQS,
>> +	.this_id		= -1,
>> +	.cmd_size		= sizeof(struct vscsifrnt_shadow),
>> +	.sg_tablesize		= VSCSIIF_SG_TABLESIZE,
>> +	.use_clustering		= DISABLE_CLUSTERING,
>> +	.proc_name		= "scsifront",
>> +};
>> +
>> +static int scsifront_alloc_ring(struct vscsifrnt_info *info)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	struct vscsiif_sring *sring;
>> +	int err = -ENOMEM;
>> +
>> +	/***** Frontend to Backend ring start *****/
>> +	sring = (struct vscsiif_sring *) __get_free_page(GFP_KERNEL);
>> +	if (!sring) {
>> +		xenbus_dev_fatal(dev, err,
>> +			"fail to allocate shared ring (Front to Back)");
>> +		return err;
>> +	}
>> +	SHARED_RING_INIT(sring);
>> +	FRONT_RING_INIT(&info->ring, sring, PAGE_SIZE);
>> +
>> +	err = xenbus_grant_ring(dev, virt_to_mfn(sring));
>> +	if (err < 0) {
>> +		free_page((unsigned long) sring);
>
> You can remove the space there.

Indeed.

>
>> +		xenbus_dev_fatal(dev, err,
>> +			"fail to grant shared ring (Front to Back)");
>> +		return err;
>> +	}
>> +	info->ring_ref = err;
>> +
>> +	err = xenbus_alloc_evtchn(dev, &info->evtchn);
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "xenbus_alloc_evtchn");
>> +		goto free_gnttab;
>> +	}
>> +
>> +	err = bind_evtchn_to_irq(info->evtchn);
>> +	if (err <= 0) {
>> +		xenbus_dev_fatal(dev, err, "bind_evtchn_to_irq");
>> +		goto free_gnttab;
>> +	}
>> +
>> +	info->irq = err;
>> +
>> +	err = request_threaded_irq(info->irq, NULL, scsifront_irq_fn,
>> +				   IRQF_ONESHOT, "scsifront", info);
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "request_threaded_irq");
>> +		goto free_irq;
>> +	}
>> +
>> +	return 0;
>> +
>> +/* free resource */
>> +free_irq:
>> +	unbind_from_irqhandler(info->irq, info);
>> +free_gnttab:
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +
>
> free_page((unsigned long)sring);

No, gnttab_end_foreign_access() is doing it.

>
>> +	return err;
>> +}
>> +
>> +static int scsifront_init_ring(struct vscsifrnt_info *info)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	struct xenbus_transaction xbt;
>> +	int err;
>> +
>> +	DPRINTK("%s\n", __func__);
>> +
>> +	err = scsifront_alloc_ring(info);
>> +	if (err)
>> +		return err;
>> +	DPRINTK("%u %u\n", info->ring_ref, info->evtchn);
>> +
>> +again:
>> +	err = xenbus_transaction_start(&xbt);
>> +	if (err)
>> +		xenbus_dev_fatal(dev, err, "starting transaction");
>> +
>> +	err = xenbus_printf(xbt, dev->nodename, "ring-ref", "%u",
>> +			    info->ring_ref);
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "%s", "writing ring-ref");
>> +		goto fail;
>> +	}
>> +
>> +	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
>> +			    info->evtchn);
>> +
>> +	if (err) {
>> +		xenbus_dev_fatal(dev, err, "%s", "writing event-channel");
>> +		goto fail;
>> +	}
>> +
>> +	err = xenbus_transaction_end(xbt, 0);
>> +	if (err) {
>> +		if (err == -EAGAIN)
>> +			goto again;
>> +		xenbus_dev_fatal(dev, err, "completing transaction");
>> +		goto free_sring;
>> +	}
>> +
>> +	return 0;
>> +
>> +fail:
>> +	xenbus_transaction_end(xbt, 1);
>> +free_sring:
>> +	unbind_from_irqhandler(info->irq, info);
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +
>
> The label says 'free_sring' but I am not seeing it being freed?

gnttab_end_foreign_access() is doing magic things. :-)

>
>> +	return err;
>> +}
>> +
>> +
>> +static int scsifront_probe(struct xenbus_device *dev,
>> +			   const struct xenbus_device_id *id)
>> +{
>> +	struct vscsifrnt_info *info;
>> +	struct Scsi_Host *host;
>> +	int err = -ENOMEM;
>> +	char name[DEFAULT_TASK_COMM_LEN];
>> +
>> +	host = scsi_host_alloc(&scsifront_sht, sizeof(*info));
>> +	if (!host) {
>> +		xenbus_dev_fatal(dev, err, "fail to allocate scsi host");
>> +		return err;
>> +	}
>> +	info = (struct vscsifrnt_info *)host->hostdata;
>> +
>> +	dev_set_drvdata(&dev->dev, info);
>> +	info->dev  = dev;
>
> Extra space.

They pile up already. :-)

>
>> +
>> +	info->shadow_free = (1UL << VSCSIIF_MAX_REQS) - 1;
>> +
>> +	err = scsifront_init_ring(info);
>> +	if (err) {
>> +		scsi_host_put(host);
>> +		return err;
>> +	}
>> +
>> +	init_waitqueue_head(&info->wq_sync);
>> +	spin_lock_init(&info->shadow_lock);
>> +
>> +	snprintf(name, DEFAULT_TASK_COMM_LEN, "vscsiif.%d", host->host_no);
>> +
>> +	host->max_id      = VSCSIIF_MAX_TARGET;
>> +	host->max_channel = 0;
>> +	host->max_lun     = VSCSIIF_MAX_LUN;
>> +	host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
>> +	host->max_cmd_len = VSCSIIF_MAX_COMMAND_SIZE;
>> +
>> +	err = scsi_add_host(host, &dev->dev);
>> +	if (err) {
>> +		dev_err(&dev->dev, "fail to add scsi host %d\n", err);
>> +		goto free_sring;
>> +	}
>> +	info->host = host;
>> +	info->host_active = 1;
>> +
>> +	xenbus_switch_state(dev, XenbusStateInitialised);
>> +
>> +	return 0;
>> +
>> +free_sring:
>> +	unbind_from_irqhandler(info->irq, info);
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +	scsi_host_put(host);
>
> The label says 'free_sring' but I am not seeing it being freed?

As above.

>
> Hm, could those operations - unbind_from_irqhandler, gnttab_end_foreign_access
> and free_page be moved to a seperate function to call?

I don't think it's necessary, those are only _two_ function calls.

>
>> +	return err;
>> +}
>> +
>> +static int scsifront_remove(struct xenbus_device *dev)
>> +{
>> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
>> +
>> +	DPRINTK("%s: %s removed\n", __func__, dev->nodename);
>> +
>> +	if (info->host_active) {
>> +		/* Scsi_host not yet removed */
>> +		scsi_remove_host(info->host);
>> +		info->host_active = 0;
>> +	}
>> +
>> +	gnttab_end_foreign_access(info->ring_ref, 0,
>> +				  (unsigned long)info->ring.sring);
>> +	unbind_from_irqhandler(info->irq, info);
>> +
>
> Should you free the ring?

I did.

>
>> +	scsi_host_put(info->host);
>> +
>> +	return 0;
>> +}
>> +
>> +static void scsifront_disconnect(struct vscsifrnt_info *info)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	struct Scsi_Host *host = info->host;
>> +
>> +	DPRINTK("%s: %s disconnect\n", __func__, dev->nodename);
>> +
>> +	/*
>> +	 * When this function is executed, all devices of
>> +	 * Frontend have been deleted.
>> +	 * Therefore, it need not block I/O before remove_host.
>> +	 */
>> +
>> +	if (info->host_active)
>> +		scsi_remove_host(host);
>> +	info->host_active = 0;
>> +
>> +	xenbus_frontend_closed(dev);
>> +}
>> +
>> +static void scsifront_do_lun_hotplug(struct vscsifrnt_info *info, int op)
>> +{
>> +	struct xenbus_device *dev = info->dev;
>> +	int i, err = 0;
>> +	char str[64];
>> +	char **dir;
>> +	unsigned int dir_n = 0;
>> +	unsigned int device_state;
>> +	unsigned int hst, chn, tgt, lun;
>> +	struct scsi_device *sdev;
>> +
>> +	dir = xenbus_directory(XBT_NIL, dev->otherend, "vscsi-devs", &dir_n);
>> +	if (IS_ERR(dir))
>> +		return;
>> +
>> +	/* mark current task as the one allowed to modify device states */
>> +	BUG_ON(info->curr);
>> +	info->curr = current;
>> +
>> +	for (i = 0; i < dir_n; i++) {
>> +		/* read status */
>> +		snprintf(str, sizeof(str), "vscsi-devs/%s/state", dir[i]);
>> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str, "%u",
>> +				   &device_state);
>> +		if (XENBUS_EXIST_ERR(err))
>> +			continue;
>> +
>> +		/* virtual SCSI device */
>> +		snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", dir[i]);
>> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str,
>> +				   "%u:%u:%u:%u", &hst, &chn, &tgt, &lun);
>> +		if (XENBUS_EXIST_ERR(err))
>> +			continue;
>> +
>> +		/*
>> +		 * Front device state path, used in slave_configure called
>> +		 * on successfull scsi_add_device, and in slave_destroy called
>> +		 * on remove of a device.
>> +		 */
>> +		snprintf(info->dev_state_path, sizeof(info->dev_state_path),
>> +			 "vscsi-devs/%s/state", dir[i]);
>> +
>> +		switch (op) {
>> +		case VSCSIFRONT_OP_ADD_LUN:
>> +			if (device_state == XenbusStateInitialised) {
>
> You could convert to make this a bit easier to read to do:
>
> 	if (device_state != XenbusStateInitialised)
> 		break;
> 	.. and then with the rest of the code.

Yes, that's better.

>
>> +				err = scsi_add_device(info->host, chn, tgt,
>> +						      lun);
>
> You can make that more than 80 characters long.

It's shorter now.

>> +
>> +				if (err) {
>> +					dev_err(&dev->dev, "scsi_add_device\n");
>> +					xenbus_printf(XBT_NIL, dev->nodename,
>> +						      info->dev_state_path,
>> +						      "%d", XenbusStateClosed);
>> +				}
>> +			}
>> +			break;
>> +		case VSCSIFRONT_OP_DEL_LUN:
>> +			if (device_state == XenbusStateClosing) {
>
> Ditto.

Yep.

>> +				sdev = scsi_device_lookup(info->host, chn, tgt,
>> +							  lun);
>> +				if (sdev) {
>> +					scsi_remove_device(sdev);
>> +					scsi_device_put(sdev);
>> +				}
>> +			}
>> +			break;
>> +		default:
>> +			break;
>> +		}
>> +	}
>> +
>> +	info->curr = NULL;
>> +
>> +	kfree(dir);
>> +}
>> +
>> +static void scsifront_read_backend_params(struct xenbus_device *dev,
>> +					  struct vscsifrnt_info *info)
>> +{
>> +	unsigned int sg_grant;
>> +	int ret;
>> +	struct Scsi_Host *host = info->host;
>> +
>> +	ret = xenbus_scanf(XBT_NIL, dev->otherend, "feature-sg-grant", "%u",
>> +			   &sg_grant);
>> +	if (ret == 1 && sg_grant) {
>> +		sg_grant = min_t(unsigned int, sg_grant, SG_ALL);
>> +		host->sg_tablesize = min_t(unsigned int, sg_grant,
>> +			VSCSIIF_SG_TABLESIZE * PAGE_SIZE /
>> +			sizeof(struct scsiif_request_segment));
>> +		dev_info(&dev->dev, "using up to %d SG entries\n",
>> +			 host->sg_tablesize);
>
> Perhaps this dev_info can be printed regardless of whether
> the backend has advertised an value? Isn't this information also
> visible in the SysFS? If so, should it be just removed?

I don't know any entry in SysFS covering this information.

Always printing the information is possible, yes.

>
>> +		host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
>
> If the backend decides to give us an value less than optimal - say '2'
> should we just ignore it? Perhaps have
>
> if (sg_grant < VSCSIIF_SG_TABLESIZE)
> 	/* Pfffff. */
> 	return;

Something like that, yes.

>> +	}
>> +}
>> +
>> +static void scsifront_backend_changed(struct xenbus_device *dev,
>> +				      enum xenbus_state backend_state)
>> +{
>> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
>> +
>> +	DPRINTK("%p %u %u\n", dev, dev->state, backend_state);
>> +
>> +	switch (backend_state) {
>> +	case XenbusStateUnknown:
>> +	case XenbusStateInitialising:
>> +	case XenbusStateInitWait:
>> +	case XenbusStateInitialised:
>> +		break;
>> +
>> +	case XenbusStateConnected:
>> +		scsifront_read_backend_params(dev, info);
>> +		if (xenbus_read_driver_state(dev->nodename) ==
>> +			XenbusStateInitialised) {
>
> You can make this more than 80 characters. And also remove the '{' and '}'

Okay.

>
>
>> +			scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
>> +		}
>> +
>> +		if (dev->state != XenbusStateConnected)
>> +			xenbus_switch_state(dev, XenbusStateConnected);
>> +		break;
>> +
>> +	case XenbusStateClosed:
>> +		if (dev->state == XenbusStateClosed)
>> +			break;
>> +		/* Missed the backend's Closing state -- fallthrough */
>> +	case XenbusStateClosing:
>> +		scsifront_disconnect(info);
>> +		break;
>> +
>> +	case XenbusStateReconfiguring:
>> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_DEL_LUN);
>> +		xenbus_switch_state(dev, XenbusStateReconfiguring);
>> +		break;
>> +
>> +	case XenbusStateReconfigured:
>> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
>> +		xenbus_switch_state(dev, XenbusStateConnected);
>> +		break;
>> +	}
>> +}
>> +
>> +static const struct xenbus_device_id scsifront_ids[] = {
>> +	{ "vscsi" },
>> +	{ "" }
>> +};
>> +
>> +static DEFINE_XENBUS_DRIVER(scsifront, ,
>> +	.probe			= scsifront_probe,
>> +	.remove			= scsifront_remove,
>> +	.otherend_changed	= scsifront_backend_changed,
>> +);
>> +
>> +static int __init scsifront_init(void)
>> +{
>> +	if (!xen_domain())
>> +		return -ENODEV;
>> +
>> +	return xenbus_register_frontend(&scsifront_driver);
>> +}
>> +module_init(scsifront_init);
>> +
>> +static void __exit scsifront_exit(void)
>> +{
>> +	xenbus_unregister_driver(&scsifront_driver);
>> +}
>> +module_exit(scsifront_exit);
>> +
>> +MODULE_DESCRIPTION("Xen SCSI frontend driver");
>> +MODULE_LICENSE("GPL");
>> +MODULE_ALIAS("xen:vscsi");
>
> MODULE_AUTHOR?

I don't mind putting myself in there, but I'm not the original author...


Juergen

Re: [PATCH V5 3/5] Introduce xen-scsifront module

[Konrad Rzeszutek Wilk]

> --- /dev/null
> +++ b/drivers/scsi/xen-scsifront.c
> @@ -0,0 +1,1017 @@
> +/*
> + * Xen SCSI frontend driver
> + *
> + * Copyright (c) 2008, FUJITSU Limited
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version 2
> + * as published by the Free Software Foundation; or, when distributed
> + * separately from the Linux kernel or incorporated into other
> + * software packages, subject to the following license:
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this source file (the "Software"), to deal in the Software without
> + * restriction, including without limitation the rights to use, copy, modify,
> + * merge, publish, distribute, sublicense, and/or sell copies of the Software,
> + * and to permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + */
> +
> +#define DEBUG

:-) I think you don't want that in the driver.

> +
> +#include <linux/module.h>
> +#include <linux/kernel.h>
> +#include <linux/device.h>
> +#include <linux/wait.h>
> +#include <linux/interrupt.h>
> +#include <linux/spinlock.h>
> +#include <linux/sched.h>
> +#include <linux/blkdev.h>
> +#include <linux/pfn.h>
> +#include <linux/slab.h>
> +
> +#include <scsi/scsi_cmnd.h>
> +#include <scsi/scsi_device.h>
> +#include <scsi/scsi.h>
> +#include <scsi/scsi_host.h>
> +
> +#include <xen/xen.h>
> +#include <xen/xenbus.h>
> +#include <xen/grant_table.h>
> +#include <xen/events.h>
> +#include <xen/page.h>
> +
> +#include <xen/interface/grant_table.h>
> +#include <xen/interface/io/vscsiif.h>
> +#include <xen/interface/io/protocols.h>
> +
> +#include <asm/xen/hypervisor.h>
> +
> +
> +#define GRANT_INVALID_REF	0
> +
> +#define VSCSIFRONT_OP_ADD_LUN	1
> +#define VSCSIFRONT_OP_DEL_LUN	2

Shouldn't those be defined in the vscsiff.h file?

> +
> +#define DEFAULT_TASK_COMM_LEN	TASK_COMM_LEN

Not sure why you need the DEFAULT_ ? Could you use TASK_COMM_LEN?

> +
> +/* tuning point*/

Missing stop and a space after the 't':

/* Tuning point. */

> +#define VSCSIIF_DEFAULT_CMD_PER_LUN 10
> +#define VSCSIIF_MAX_TARGET          64
> +#define VSCSIIF_MAX_LUN             255
> +
> +#define VSCSIIF_RING_SIZE	__CONST_RING_SIZE(vscsiif, PAGE_SIZE)
> +#define VSCSIIF_MAX_REQS	VSCSIIF_RING_SIZE
> +
> +#define vscsiif_grants_sg(_sg)	(PFN_UP((_sg) *		\
> +				sizeof(struct scsiif_request_segment)))
> +
> +struct vscsifrnt_shadow {
> +	/* command between backend and frontend */
> +	unsigned char act;

act? How about 'op' ? Or 'cmd_op'?

> +	uint16_t rqid;

rqid? Could you have a comment explaining what that acronym is?
Oh wait - request id? How about just req_id?

> +
> +	/* Number of pieces of scatter-gather */
> +	unsigned int nr_grants;

s/nr_grants/nr_sg/ ?

> +	struct scsiif_request_segment *sg;
> +
> +	/* do reset or abort function */

Full stop missing.
> +	wait_queue_head_t wq_reset;	/* reset work queue           */
> +	int wait_reset;			/* reset work queue condition */
> +	int32_t rslt_reset;		/* reset response status      */
> +					/* (SUCESS or FAILED)         */

Full stop missing. s/SUCESS/SUCCESS/
> +
> +	/* requested struct scsi_cmnd is stored from kernel */

Full stop missing.
> +	struct scsi_cmnd *sc;
> +	int gref[vscsiif_grants_sg(SG_ALL) + SG_ALL];
> +};
> +
> +struct vscsifrnt_info {
> +	struct xenbus_device *dev;
> +
> +	struct Scsi_Host *host;
> +	int host_active;

This 'host_active' seems to be a guard to call 'scsi_host_remove'
through either the disconnect (so backend told us to disconnect)
or the .remove (so XenStore keys are moved). Either way I think
it is possible to run _both_ of them and this being just
an 'int' is not sufficient.

I would recommend you change this to an ref_count.
> +
> +	spinlock_t shadow_lock;

Could you move this spinlock below - to where
you have tons of of 'shadow' values please?

> +	unsigned int evtchn;
> +	unsigned int irq;
> +
> +	grant_ref_t ring_ref;
> +	struct vscsiif_front_ring ring;
> +	struct vscsiif_response	ring_res;
> +
> +	unsigned long shadow_free;

A comment would help in explainig this. 'shadow_free' sounds 
a bit cryptic. Oh, and xen-blkfront has it as 'shadow_free'
too! Argh, 'shadow_free_bitmask' could be a better name?
Oh, what if we converted to an bitmask type?

Either way - if you think the existing one should be this way
then lets leave it as is. But if you think a better name
is suited I can change in xen-blkfront too.

> +	struct vscsifrnt_shadow *shadow[VSCSIIF_MAX_REQS];
> +
> +	wait_queue_head_t wq_sync;
> +	unsigned int waiting_sync:1;

That sounds like it needs a spinlock to protect it?

Could you also explain a bit of what its purpose is? What
is it suppose to wait for? It looks like it is tied to the
error handler, perhaps then it ought to be called:

err_handler_done


> +
> +	char dev_state_path[64];
> +	struct task_struct *curr;
> +};
> +
> +#define DPRINTK(_f, _a...)				\
> +	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)

Could all the DPRINTK be converted to pr_debug right away?

> +
> +#define PREFIX(lvl) KERN_##lvl "scsifront: "

Perhaps you want?

#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +static void scsifront_wake_up(struct vscsifrnt_info *info)
> +{
> +	info->waiting_sync = 0;
> +	wake_up(&info->wq_sync);
> +}
> +
> +static int scsifront_get_rqid(struct vscsifrnt_info *info)
> +{
> +	unsigned long flags;
> +	int free;
> +
> +	spin_lock_irqsave(&info->shadow_lock, flags);
> +
> +	free = find_first_bit(&info->shadow_free, VSCSIIF_MAX_REQS);
> +	info->shadow_free &= ~(1UL << free);
> +
> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
> +
> +	return free;
> +}
> +
> +static int _scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
> +{
> +	info->shadow_free |= 1UL << id;
> +	info->shadow[id] = NULL;
> +
> +	return (info->shadow_free == 1UL << id || info->waiting_sync);
> +}
> +
> +static void scsifront_put_rqid(struct vscsifrnt_info *info, uint32_t id)
> +{
> +	unsigned long flags;
> +	int was_empty;
> +
> +	spin_lock_irqsave(&info->shadow_lock, flags);
> +	was_empty = _scsifront_put_rqid(info, id);

I wouldn't call this 'was_empty' as it will also be true if the 'waiting_sync'
is turned on. Perhaps this should be called: 'kick' or 'wake_wq' ?

> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
> +
> +	if (was_empty)
> +		scsifront_wake_up(info);
> +}
> +
> +static struct vscsiif_request *scsifront_pre_req(struct vscsifrnt_info *info)
> +{
> +	struct vscsiif_front_ring *ring = &(info->ring);
> +	struct vscsiif_request *ring_req;
> +	uint32_t id;
> +
> +	id = scsifront_get_rqid(info);	/* use id by response */

s/by/in/

> +	if (id >= VSCSIIF_MAX_REQS)
> +		return NULL;
> +
> +	ring_req = RING_GET_REQUEST(&(info->ring), ring->req_prod_pvt);
> +
> +	ring->req_prod_pvt++;
> +
> +	ring_req->rqid = (uint16_t)id;
> +
> +	return ring_req;
> +}
> +
> +static void scsifront_do_request(struct vscsifrnt_info *info)
> +{
> +	struct vscsiif_front_ring *ring = &(info->ring);
> +	int notify;
> +
> +	RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(ring, notify);
> +	if (notify)
> +		notify_remote_via_irq(info->irq);
> +}
> +
> +static void scsifront_gnttab_done(struct vscsifrnt_info *info, uint32_t id)
> +{
> +	struct vscsifrnt_shadow *s = info->shadow[id];
> +	int i;
> +
> +	if (s->sc->sc_data_direction == DMA_NONE)
> +		return;
> +
> +	for (i = 0; i < s->nr_grants; i++) {
> +		if (unlikely(gnttab_query_foreign_access(s->gref[i]) != 0)) {
> +			shost_printk(PREFIX(ALERT), info->host,
> +				     "grant still in use by backend\n");
> +			BUG();
> +		}
> +		gnttab_end_foreign_access(s->gref[i], 0, 0UL);
> +	}
> +
> +	kfree(s->sg);
> +}
> +
> +static void scsifront_cdb_cmd_done(struct vscsifrnt_info *info,
> +				   struct vscsiif_response *ring_res)

s/ring_res/ring_rsp/ ?

> +{
> +	struct scsi_cmnd *sc;
> +	uint32_t id;
> +	uint8_t sense_len;
> +
> +	id = ring_res->rqid;
> +	sc = info->shadow[id]->sc;
> +
> +	BUG_ON(sc == NULL);
> +
> +	scsifront_gnttab_done(info, id);
> +	scsifront_put_rqid(info, id);
> +
> +	sc->result = ring_res->rslt;
> +	scsi_set_resid(sc, ring_res->residual_len);
> +
> +	sense_len = min_t(uint8_t, VSCSIIF_SENSE_BUFFERSIZE,
> +			  ring_res->sense_len);
> +
> +	if (sense_len)
> +		memcpy(sc->sense_buffer, ring_res->sense_buffer, sense_len);
> +
> +	sc->scsi_done(sc);
> +}
> +
> +static void scsifront_sync_cmd_done(struct vscsifrnt_info *info,
> +				    struct vscsiif_response *ring_res)
> +{
> +	uint16_t id = ring_res->rqid;
> +	unsigned long flags;
> +	struct vscsifrnt_shadow *shadow = info->shadow[id];
> +	int was_empty;
> +
> +	spin_lock_irqsave(&info->shadow_lock, flags);
> +	shadow->wait_reset = 1;
> +	switch (shadow->rslt_reset) {
> +	case 0:
> +		shadow->rslt_reset = ring_res->rslt;
> +		break;
> +	case -1:

Is there an #define for this? The comment at the top mentioned
SUCCESS and FAILED ?

> +		was_empty = _scsifront_put_rqid(info, id);

Perhaps call it 'kick' or 'wake_wq' ?

> +		spin_unlock_irqrestore(&info->shadow_lock, flags);
> +		kfree(shadow);
> +		if (was_empty)
> +			scsifront_wake_up(info);
> +		return;
> +	default:
> +		shost_printk(PREFIX(ERR), info->host,
> +			     "bad reset state %d, possibly leaking %u\n",
> +			     shadow->rslt_reset, id);
> +		break;
> +	}
> +	spin_unlock_irqrestore(&info->shadow_lock, flags);
> +
> +	wake_up(&shadow->wq_reset);
> +}
> +
> +static int scsifront_cmd_done(struct vscsifrnt_info *info)
> +{
> +	struct vscsiif_response *ring_res;
> +	RING_IDX i, rp;
> +	int more_to_do = 0;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(info->host->host_lock, flags);
> +
> +	rp = info->ring.sring->rsp_prod;
> +	rmb();	/* ordering required respective to dom0 */
> +	for (i = info->ring.rsp_cons; i != rp; i++) {
> +
> +		ring_res = RING_GET_RESPONSE(&info->ring, i);

I would recommend you look at git commit 6878c32e5cc0e40980abe51d1f02fb453e27493e

As the 'rqid' value - might be corrupted by the backend. Which means
that the 'info->shadow[rubbish_val]->act' could blow up.

I would just add a check to make sure that the 'rqid' value is
within the expected values.

> +
> +		if (info->shadow[ring_res->rqid]->act == VSCSIIF_ACT_SCSI_CDB)
> +			scsifront_cdb_cmd_done(info, ring_res);
> +		else
> +			scsifront_sync_cmd_done(info, ring_res);
> +	}
> +
> +	info->ring.rsp_cons = i;
> +
> +	if (i != info->ring.req_prod_pvt)
> +		RING_FINAL_CHECK_FOR_RESPONSES(&info->ring, more_to_do);
> +	else
> +		info->ring.sring->rsp_event = i + 1;
> +
> +	info->waiting_sync = 0;
> +
> +	spin_unlock_irqrestore(info->host->host_lock, flags);
> +
> +	wake_up(&info->wq_sync);
> +
> +	return more_to_do;
> +}
> +
> +static irqreturn_t scsifront_irq_fn(int irq, void *dev_id)
> +{
> +	struct vscsifrnt_info *info = dev_id;
> +
> +	while (scsifront_cmd_done(info))
> +		/* Yield point for this unbounded loop. */
> +		cond_resched();
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int map_data_for_request(struct vscsifrnt_info *info,
> +				struct scsi_cmnd *sc,
> +				struct vscsiif_request *ring_req,
> +				struct vscsifrnt_shadow *shadow)
> +{
> +	grant_ref_t gref_head;
> +	struct page *page;
> +	int err, ref, ref_cnt = 0;
> +	int write = (sc->sc_data_direction == DMA_TO_DEVICE);

What if it is DMA_BIDIRECTIONAL ?

> +	unsigned int i, off, len, bytes;
> +	unsigned int data_len = scsi_bufflen(sc);
> +	unsigned int data_grants = 0, seg_grants = 0;
> +	struct scatterlist *sg;
> +	unsigned long mfn;
> +	struct scsiif_request_segment *seg;
> +
> +	ring_req->nr_segments = 0;
> +	if (sc->sc_data_direction == DMA_NONE || !data_len)
> +		return 0;
> +
> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i)
> +		data_grants += PFN_UP(sg->offset + sg->length);
> +
> +	if (data_grants > VSCSIIF_SG_TABLESIZE) {
> +		if (data_grants > info->host->sg_tablesize) {
> +			shost_printk(PREFIX(ERR), info->host,
> +			     "Unable to map request_buffer for command!\n");
> +			return -E2BIG;
> +		}
> +		seg_grants = vscsiif_grants_sg(data_grants);
> +		shadow->sg = kcalloc(data_grants,
> +			sizeof(struct scsiif_request_segment), GFP_NOIO);
> +		if (!shadow->sg)
> +			return -ENOMEM;
> +	}
> +	seg = shadow->sg ? : ring_req->seg;
> +
> +	err = gnttab_alloc_grant_references(seg_grants + data_grants,
> +					    &gref_head);
> +	if (err) {
> +		kfree(shadow->sg);
> +		shost_printk(PREFIX(ERR), info->host,
> +			     "gnttab_alloc_grant_references() error\n");
> +		return -ENOMEM;
> +	}
> +
> +	if (seg_grants) {
> +		page = virt_to_page(seg);
> +		off = (unsigned long)seg & ~PAGE_MASK;
> +		len = sizeof(struct scsiif_request_segment) * data_grants;
> +		while (len > 0) {
> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
> +
> +			ref = gnttab_claim_grant_reference(&gref_head);
> +			BUG_ON(ref == -ENOSPC);
> +
> +			mfn = pfn_to_mfn(page_to_pfn(page));
> +			gnttab_grant_foreign_access_ref(ref,
> +				info->dev->otherend_id, mfn, 1);
> +			shadow->gref[ref_cnt] = ref;
> +			ring_req->seg[ref_cnt].gref   = ref;
> +			ring_req->seg[ref_cnt].offset = (uint16_t)off;
> +			ring_req->seg[ref_cnt].length = (uint16_t)bytes;
> +
> +			page++;
> +			len -= bytes;
> +			off = 0;
> +			ref_cnt++;
> +		}
> +		BUG_ON(seg_grants < ref_cnt);
> +		seg_grants = ref_cnt;
> +	}
> +
> +	scsi_for_each_sg(sc, sg, scsi_sg_count(sc), i) {
> +		page = sg_page(sg);
> +		off = sg->offset;
> +		len = sg->length;
> +
> +		while (len > 0 && data_len > 0) {
> +			/*
> +			 * sg sends a scatterlist that is larger than
> +			 * the data_len it wants transferred for certain
> +			 * IO sizes

Full stop missing.
> +			 */
> +			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
> +			bytes = min(bytes, data_len);
> +
> +			ref = gnttab_claim_grant_reference(&gref_head);
> +			BUG_ON(ref == -ENOSPC);
> +
> +			mfn = pfn_to_mfn(page_to_pfn(page));
> +			gnttab_grant_foreign_access_ref(ref,
> +				info->dev->otherend_id, mfn, write);
> +
> +			shadow->gref[ref_cnt] = ref;
> +			seg->gref   = ref;
> +			seg->offset = (uint16_t)off;
> +			seg->length = (uint16_t)bytes;
> +
> +			page++;
> +			seg++;
> +			len -= bytes;
> +			data_len -= bytes;
> +			off = 0;
> +			ref_cnt++;
> +		}
> +	}
> +
> +	if (seg_grants)
> +		ring_req->nr_segments = VSCSIIF_SG_GRANT | seg_grants;
> +	else
> +		ring_req->nr_segments = (uint8_t)ref_cnt;
> +	shadow->nr_grants = ref_cnt;
> +
> +	return 0;
> +}
> +
> +static struct vscsiif_request *scsifront_command2ring(
> +		struct vscsifrnt_info *info, struct scsi_cmnd *sc,
> +		struct vscsifrnt_shadow *shadow)
> +{
> +	struct vscsiif_request *ring_req;
> +
> +	memset(shadow, 0, sizeof(*shadow));
> +
> +	ring_req = scsifront_pre_req(info);
> +	if (!ring_req)
> +		return NULL;
> +
> +	info->shadow[ring_req->rqid] = shadow;
> +	shadow->rqid = ring_req->rqid;
> +
> +	ring_req->id      = sc->device->id;
> +	ring_req->lun     = sc->device->lun;
> +	ring_req->channel = sc->device->channel;
> +	ring_req->cmd_len = sc->cmd_len;
> +
> +	BUG_ON(sc->cmd_len > VSCSIIF_MAX_COMMAND_SIZE);
> +
> +	memcpy(ring_req->cmnd, sc->cmnd, sc->cmd_len);
> +
> +	ring_req->sc_data_direction   = (uint8_t)sc->sc_data_direction;
> +	ring_req->timeout_per_command = sc->request->timeout / HZ;
> +
> +	return ring_req;
> +}
> +
> +static int scsifront_queuecommand(struct Scsi_Host *shost,
> +				  struct scsi_cmnd *sc)
> +{
> +	struct vscsifrnt_info *info = shost_priv(shost);
> +	struct vscsiif_request *ring_req;
> +	struct vscsifrnt_shadow *shadow = scsi_cmd_priv(sc);
> +	unsigned long flags;
> +	int err;
> +	uint16_t rqid;
> +

BUG_ON(sc->sc_data_direction == DMA_BIDIRECTIONAL); ?


> +	spin_lock_irqsave(shost->host_lock, flags);
> +	if (RING_FULL(&info->ring))
> +		goto busy;
> +
> +	ring_req = scsifront_command2ring(info, sc, shadow);
> +	if (!ring_req)
> +		goto busy;
> +
> +	sc->result    = 0;

This has some odd spacing? Is it suppose to be at the same
aligment as the ones below? Maybe it is my editor having
issues.

> +
> +	rqid              = ring_req->rqid;
> +	ring_req->act     = VSCSIIF_ACT_SCSI_CDB;
> +
> +	shadow->sc  = sc;
> +	shadow->act = VSCSIIF_ACT_SCSI_CDB;
> +
> +	err = map_data_for_request(info, sc, ring_req, shadow);
> +	if (err < 0) {
> +		DPRINTK("%s: err %d\n", __func__, err);
> +		scsifront_put_rqid(info, rqid);
> +		spin_unlock_irqrestore(shost->host_lock, flags);
> +		if (err == -ENOMEM)
> +			return SCSI_MLQUEUE_HOST_BUSY;
> +		sc->result = DID_ERROR << 16;
> +		sc->scsi_done(sc);
> +		return 0;
> +	}
> +
> +	scsifront_do_request(info);
> +	spin_unlock_irqrestore(shost->host_lock, flags);
> +
> +	return 0;
> +
> +busy:
> +	spin_unlock_irqrestore(shost->host_lock, flags);
> +	DPRINTK("%s: busy\n", __func__);
> +	return SCSI_MLQUEUE_HOST_BUSY;
> +}
> +
> +/*
> + * Any exception handling (reset or abort) must be forwarded to the backend.
> + * We have to wait until an answer is returned. This answer contains the
> + * result to be returned to the requestor.
> + */
> +static int scsifront_action_handler(struct scsi_cmnd *sc, uint8_t act)
> +{
> +	struct Scsi_Host *host = sc->device->host;
> +	struct vscsifrnt_info *info = shost_priv(host);
> +	struct vscsifrnt_shadow *shadow, *s = scsi_cmd_priv(sc);
> +	struct vscsiif_request *ring_req;
> +	int err = 0;
> +
> +	shadow = kmalloc(sizeof(*shadow), GFP_NOIO);
> +	if (!shadow)
> +		return FAILED;
> +
> +	for (;;) {
> +		spin_lock_irq(host->host_lock);
> +		if (!RING_FULL(&info->ring)) {
> +			ring_req = scsifront_command2ring(info, sc, shadow);
> +			if (ring_req)
> +				break;
> +		}
> +		if (err) {
> +			spin_unlock_irq(host->host_lock);
> +			kfree(shadow);
> +			return FAILED;
> +		}
> +		info->waiting_sync = 1;
> +		spin_unlock_irq(host->host_lock);
> +		err = wait_event_interruptible(info->wq_sync,
> +					       !info->waiting_sync);
> +		spin_lock_irq(host->host_lock);
> +	}
> +
> +	ring_req->act = act;
> +	ring_req->ref_rqid = s->rqid;
> +
> +	shadow->act = act;
> +	shadow->rslt_reset = 0;
> +	init_waitqueue_head(&shadow->wq_reset);
> +
> +	ring_req->nr_segments         = 0;

Something odd with the spaces here.

> +
> +	scsifront_do_request(info);
> +
> +	spin_unlock_irq(host->host_lock);
> +	err = wait_event_interruptible(shadow->wq_reset, shadow->wait_reset);
> +	spin_lock_irq(host->host_lock);
> +
> +	if (!err) {
> +		err = shadow->rslt_reset;
> +		scsifront_put_rqid(info, shadow->rqid);
> +		kfree(shadow);
> +	} else {
> +		spin_lock(&info->shadow_lock);
> +		shadow->rslt_reset = -1;

#define for -1?

> +		spin_unlock(&info->shadow_lock);
> +		err = FAILED;
> +	}
> +
> +	spin_unlock_irq(host->host_lock);
> +	return err;
> +}
> +
> +static int scsifront_eh_abort_handler(struct scsi_cmnd *sc)
> +{
> +	DPRINTK("%s\n", __func__);
> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_ABORT);
> +}
> +
> +static int scsifront_dev_reset_handler(struct scsi_cmnd *sc)
> +{
> +	DPRINTK("%s\n", __func__);
> +	return scsifront_action_handler(sc, VSCSIIF_ACT_SCSI_RESET);
> +}
> +
> +static int scsifront_sdev_configure(struct scsi_device *sdev)
> +{
> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
> +
> +	if (info && current == info->curr)
> +		xenbus_printf(XBT_NIL, info->dev->nodename,
> +			      info->dev_state_path, "%d", XenbusStateConnected);
> +
> +	return 0;
> +}
> +
> +static void scsifront_sdev_destroy(struct scsi_device *sdev)
> +{
> +	struct vscsifrnt_info *info = shost_priv(sdev->host);
> +
> +	if (info && current == info->curr)
> +		xenbus_printf(XBT_NIL, info->dev->nodename,
> +			      info->dev_state_path, "%d", XenbusStateClosed);
> +}
> +
> +static struct scsi_host_template scsifront_sht = {
> +	.module			= THIS_MODULE,
> +	.name			= "Xen SCSI frontend driver",
> +	.queuecommand		= scsifront_queuecommand,
> +	.eh_abort_handler	= scsifront_eh_abort_handler,
> +	.eh_device_reset_handler = scsifront_dev_reset_handler,
> +	.slave_configure	= scsifront_sdev_configure,
> +	.slave_destroy		= scsifront_sdev_destroy,
> +	.cmd_per_lun		= VSCSIIF_DEFAULT_CMD_PER_LUN,
> +	.can_queue		= VSCSIIF_MAX_REQS,
> +	.this_id		= -1,
> +	.cmd_size		= sizeof(struct vscsifrnt_shadow),
> +	.sg_tablesize		= VSCSIIF_SG_TABLESIZE,
> +	.use_clustering		= DISABLE_CLUSTERING,
> +	.proc_name		= "scsifront",
> +};
> +
> +static int scsifront_alloc_ring(struct vscsifrnt_info *info)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	struct vscsiif_sring *sring;
> +	int err = -ENOMEM;
> +
> +	/***** Frontend to Backend ring start *****/
> +	sring = (struct vscsiif_sring *) __get_free_page(GFP_KERNEL);
> +	if (!sring) {
> +		xenbus_dev_fatal(dev, err,
> +			"fail to allocate shared ring (Front to Back)");
> +		return err;
> +	}
> +	SHARED_RING_INIT(sring);
> +	FRONT_RING_INIT(&info->ring, sring, PAGE_SIZE);
> +
> +	err = xenbus_grant_ring(dev, virt_to_mfn(sring));
> +	if (err < 0) {
> +		free_page((unsigned long) sring);

You can remove the space there.

> +		xenbus_dev_fatal(dev, err,
> +			"fail to grant shared ring (Front to Back)");
> +		return err;
> +	}
> +	info->ring_ref = err;
> +
> +	err = xenbus_alloc_evtchn(dev, &info->evtchn);
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "xenbus_alloc_evtchn");
> +		goto free_gnttab;
> +	}
> +
> +	err = bind_evtchn_to_irq(info->evtchn);
> +	if (err <= 0) {
> +		xenbus_dev_fatal(dev, err, "bind_evtchn_to_irq");
> +		goto free_gnttab;
> +	}
> +
> +	info->irq = err;
> +
> +	err = request_threaded_irq(info->irq, NULL, scsifront_irq_fn,
> +				   IRQF_ONESHOT, "scsifront", info);
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "request_threaded_irq");
> +		goto free_irq;
> +	}
> +
> +	return 0;
> +
> +/* free resource */
> +free_irq:
> +	unbind_from_irqhandler(info->irq, info);
> +free_gnttab:
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +

free_page((unsigned long)sring);

> +	return err;
> +}
> +
> +static int scsifront_init_ring(struct vscsifrnt_info *info)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	struct xenbus_transaction xbt;
> +	int err;
> +
> +	DPRINTK("%s\n", __func__);
> +
> +	err = scsifront_alloc_ring(info);
> +	if (err)
> +		return err;
> +	DPRINTK("%u %u\n", info->ring_ref, info->evtchn);
> +
> +again:
> +	err = xenbus_transaction_start(&xbt);
> +	if (err)
> +		xenbus_dev_fatal(dev, err, "starting transaction");
> +
> +	err = xenbus_printf(xbt, dev->nodename, "ring-ref", "%u",
> +			    info->ring_ref);
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "%s", "writing ring-ref");
> +		goto fail;
> +	}
> +
> +	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
> +			    info->evtchn);
> +
> +	if (err) {
> +		xenbus_dev_fatal(dev, err, "%s", "writing event-channel");
> +		goto fail;
> +	}
> +
> +	err = xenbus_transaction_end(xbt, 0);
> +	if (err) {
> +		if (err == -EAGAIN)
> +			goto again;
> +		xenbus_dev_fatal(dev, err, "completing transaction");
> +		goto free_sring;
> +	}
> +
> +	return 0;
> +
> +fail:
> +	xenbus_transaction_end(xbt, 1);
> +free_sring:
> +	unbind_from_irqhandler(info->irq, info);
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +

The label says 'free_sring' but I am not seeing it being freed?

> +	return err;
> +}
> +
> +
> +static int scsifront_probe(struct xenbus_device *dev,
> +			   const struct xenbus_device_id *id)
> +{
> +	struct vscsifrnt_info *info;
> +	struct Scsi_Host *host;
> +	int err = -ENOMEM;
> +	char name[DEFAULT_TASK_COMM_LEN];
> +
> +	host = scsi_host_alloc(&scsifront_sht, sizeof(*info));
> +	if (!host) {
> +		xenbus_dev_fatal(dev, err, "fail to allocate scsi host");
> +		return err;
> +	}
> +	info = (struct vscsifrnt_info *)host->hostdata;
> +
> +	dev_set_drvdata(&dev->dev, info);
> +	info->dev  = dev;

Extra space.

> +
> +	info->shadow_free = (1UL << VSCSIIF_MAX_REQS) - 1;
> +
> +	err = scsifront_init_ring(info);
> +	if (err) {
> +		scsi_host_put(host);
> +		return err;
> +	}
> +
> +	init_waitqueue_head(&info->wq_sync);
> +	spin_lock_init(&info->shadow_lock);
> +
> +	snprintf(name, DEFAULT_TASK_COMM_LEN, "vscsiif.%d", host->host_no);
> +
> +	host->max_id      = VSCSIIF_MAX_TARGET;
> +	host->max_channel = 0;
> +	host->max_lun     = VSCSIIF_MAX_LUN;
> +	host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;
> +	host->max_cmd_len = VSCSIIF_MAX_COMMAND_SIZE;
> +
> +	err = scsi_add_host(host, &dev->dev);
> +	if (err) {
> +		dev_err(&dev->dev, "fail to add scsi host %d\n", err);
> +		goto free_sring;
> +	}
> +	info->host = host;
> +	info->host_active = 1;
> +
> +	xenbus_switch_state(dev, XenbusStateInitialised);
> +
> +	return 0;
> +
> +free_sring:
> +	unbind_from_irqhandler(info->irq, info);
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +	scsi_host_put(host);

The label says 'free_sring' but I am not seeing it being freed?

Hm, could those operations - unbind_from_irqhandler, gnttab_end_foreign_access
and free_page be moved to a seperate function to call?

> +	return err;
> +}
> +
> +static int scsifront_remove(struct xenbus_device *dev)
> +{
> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
> +
> +	DPRINTK("%s: %s removed\n", __func__, dev->nodename);
> +
> +	if (info->host_active) {
> +		/* Scsi_host not yet removed */
> +		scsi_remove_host(info->host);
> +		info->host_active = 0;
> +	}
> +
> +	gnttab_end_foreign_access(info->ring_ref, 0,
> +				  (unsigned long)info->ring.sring);
> +	unbind_from_irqhandler(info->irq, info);
> +

Should you free the ring?

> +	scsi_host_put(info->host);
> +
> +	return 0;
> +}
> +
> +static void scsifront_disconnect(struct vscsifrnt_info *info)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	struct Scsi_Host *host = info->host;
> +
> +	DPRINTK("%s: %s disconnect\n", __func__, dev->nodename);
> +
> +	/*
> +	 * When this function is executed, all devices of
> +	 * Frontend have been deleted.
> +	 * Therefore, it need not block I/O before remove_host.
> +	 */
> +
> +	if (info->host_active)
> +		scsi_remove_host(host);
> +	info->host_active = 0;
> +
> +	xenbus_frontend_closed(dev);
> +}
> +
> +static void scsifront_do_lun_hotplug(struct vscsifrnt_info *info, int op)
> +{
> +	struct xenbus_device *dev = info->dev;
> +	int i, err = 0;
> +	char str[64];
> +	char **dir;
> +	unsigned int dir_n = 0;
> +	unsigned int device_state;
> +	unsigned int hst, chn, tgt, lun;
> +	struct scsi_device *sdev;
> +
> +	dir = xenbus_directory(XBT_NIL, dev->otherend, "vscsi-devs", &dir_n);
> +	if (IS_ERR(dir))
> +		return;
> +
> +	/* mark current task as the one allowed to modify device states */
> +	BUG_ON(info->curr);
> +	info->curr = current;
> +
> +	for (i = 0; i < dir_n; i++) {
> +		/* read status */
> +		snprintf(str, sizeof(str), "vscsi-devs/%s/state", dir[i]);
> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str, "%u",
> +				   &device_state);
> +		if (XENBUS_EXIST_ERR(err))
> +			continue;
> +
> +		/* virtual SCSI device */
> +		snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", dir[i]);
> +		err = xenbus_scanf(XBT_NIL, dev->otherend, str,
> +				   "%u:%u:%u:%u", &hst, &chn, &tgt, &lun);
> +		if (XENBUS_EXIST_ERR(err))
> +			continue;
> +
> +		/*
> +		 * Front device state path, used in slave_configure called
> +		 * on successfull scsi_add_device, and in slave_destroy called
> +		 * on remove of a device.
> +		 */
> +		snprintf(info->dev_state_path, sizeof(info->dev_state_path),
> +			 "vscsi-devs/%s/state", dir[i]);
> +
> +		switch (op) {
> +		case VSCSIFRONT_OP_ADD_LUN:
> +			if (device_state == XenbusStateInitialised) {

You could convert to make this a bit easier to read to do:

	if (device_state != XenbusStateInitialised)
		break;
	.. and then with the rest of the code.

> +				err = scsi_add_device(info->host, chn, tgt,
> +						      lun);

You can make that more than 80 characters long.
> +
> +				if (err) {
> +					dev_err(&dev->dev, "scsi_add_device\n");
> +					xenbus_printf(XBT_NIL, dev->nodename,
> +						      info->dev_state_path,
> +						      "%d", XenbusStateClosed);
> +				}
> +			}
> +			break;
> +		case VSCSIFRONT_OP_DEL_LUN:
> +			if (device_state == XenbusStateClosing) {

Ditto.
> +				sdev = scsi_device_lookup(info->host, chn, tgt,
> +							  lun);
> +				if (sdev) {
> +					scsi_remove_device(sdev);
> +					scsi_device_put(sdev);
> +				}
> +			}
> +			break;
> +		default:
> +			break;
> +		}
> +	}
> +
> +	info->curr = NULL;
> +
> +	kfree(dir);
> +}
> +
> +static void scsifront_read_backend_params(struct xenbus_device *dev,
> +					  struct vscsifrnt_info *info)
> +{
> +	unsigned int sg_grant;
> +	int ret;
> +	struct Scsi_Host *host = info->host;
> +
> +	ret = xenbus_scanf(XBT_NIL, dev->otherend, "feature-sg-grant", "%u",
> +			   &sg_grant);
> +	if (ret == 1 && sg_grant) {
> +		sg_grant = min_t(unsigned int, sg_grant, SG_ALL);
> +		host->sg_tablesize = min_t(unsigned int, sg_grant,
> +			VSCSIIF_SG_TABLESIZE * PAGE_SIZE /
> +			sizeof(struct scsiif_request_segment));
> +		dev_info(&dev->dev, "using up to %d SG entries\n",
> +			 host->sg_tablesize);

Perhaps this dev_info can be printed regardless of whether
the backend has advertised an value? Isn't this information also
visible in the SysFS? If so, should it be just removed?

> +		host->max_sectors = (host->sg_tablesize - 1) * PAGE_SIZE / 512;

If the backend decides to give us an value less than optimal - say '2'
should we just ignore it? Perhaps have

if (sg_grant < VSCSIIF_SG_TABLESIZE)
	/* Pfffff. */
	return;


> +	}
> +}
> +
> +static void scsifront_backend_changed(struct xenbus_device *dev,
> +				      enum xenbus_state backend_state)
> +{
> +	struct vscsifrnt_info *info = dev_get_drvdata(&dev->dev);
> +
> +	DPRINTK("%p %u %u\n", dev, dev->state, backend_state);
> +
> +	switch (backend_state) {
> +	case XenbusStateUnknown:
> +	case XenbusStateInitialising:
> +	case XenbusStateInitWait:
> +	case XenbusStateInitialised:
> +		break;
> +
> +	case XenbusStateConnected:
> +		scsifront_read_backend_params(dev, info);
> +		if (xenbus_read_driver_state(dev->nodename) ==
> +			XenbusStateInitialised) {

You can make this more than 80 characters. And also remove the '{' and '}'


> +			scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
> +		}
> +
> +		if (dev->state != XenbusStateConnected)
> +			xenbus_switch_state(dev, XenbusStateConnected);
> +		break;
> +
> +	case XenbusStateClosed:
> +		if (dev->state == XenbusStateClosed)
> +			break;
> +		/* Missed the backend's Closing state -- fallthrough */
> +	case XenbusStateClosing:
> +		scsifront_disconnect(info);
> +		break;
> +
> +	case XenbusStateReconfiguring:
> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_DEL_LUN);
> +		xenbus_switch_state(dev, XenbusStateReconfiguring);
> +		break;
> +
> +	case XenbusStateReconfigured:
> +		scsifront_do_lun_hotplug(info, VSCSIFRONT_OP_ADD_LUN);
> +		xenbus_switch_state(dev, XenbusStateConnected);
> +		break;
> +	}
> +}
> +
> +static const struct xenbus_device_id scsifront_ids[] = {
> +	{ "vscsi" },
> +	{ "" }
> +};
> +
> +static DEFINE_XENBUS_DRIVER(scsifront, ,
> +	.probe			= scsifront_probe,
> +	.remove			= scsifront_remove,
> +	.otherend_changed	= scsifront_backend_changed,
> +);
> +
> +static int __init scsifront_init(void)
> +{
> +	if (!xen_domain())
> +		return -ENODEV;
> +
> +	return xenbus_register_frontend(&scsifront_driver);
> +}
> +module_init(scsifront_init);
> +
> +static void __exit scsifront_exit(void)
> +{
> +	xenbus_unregister_driver(&scsifront_driver);
> +}
> +module_exit(scsifront_exit);
> +
> +MODULE_DESCRIPTION("Xen SCSI frontend driver");
> +MODULE_LICENSE("GPL");
> +MODULE_ALIAS("xen:vscsi");

MODULE_AUTHOR?

[PATCH V5 4/5] Introduce XEN scsiback module

[jgross]

From: Juergen Gross <jgross@suse.com>

Introduces the XEN pvSCSI backend. With pvSCSI it is possible for a XEN domU
to issue SCSI commands to a SCSI LUN assigned to that domU. The SCSI commands
are passed to the pvSCSI backend in a driver domain (usually Dom0) which is
owner of the physical device. This allows e.g. to use SCSI tape drives in a
XEN domU.

The code is taken from the pvSCSI implementation in XEN done by Fujitsu based
on Linux kernel 2.6.18.

Changes from the original version are:
- port to upstream kernel
- put all code in just one source file
- adapt to Linux style guide
- use target core infrastructure instead doing pure pass-through
- enable module unloading
- support SG-list in grant page(s)
- support task abort
- remove redundant struct backend
- allocate resources dynamically
- correct minor error in scsiback_fast_flush_area
- free allocated resources in case of error during I/O preparation
- remove CDB emulation, now handled by target core infrastructure

Signed-off-by: Juergen Gross <jgross@suse.com>

Xen related parts
Acked-by: David Vrabel <david.vrabel@citrix.com>
---
 drivers/xen/Kconfig        |    9 +
 drivers/xen/Makefile       |    1 +
 drivers/xen/xen-scsiback.c | 2124 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 2134 insertions(+)
 create mode 100644 drivers/xen/xen-scsiback.c

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 8bc0183..b812462 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -172,6 +172,15 @@ config XEN_PCIDEV_BACKEND
 
 	  If in doubt, say m.
 
+config XEN_SCSI_BACKEND
+	tristate "XEN SCSI backend driver"
+	depends on XEN && XEN_BACKEND && TARGET_CORE
+	help
+	  The SCSI backend driver allows the kernel to export its SCSI Devices
+	  to other guests via a high-performance shared-memory interface.
+	  Only needed for systems running as XEN driver domains (e.g. Dom0) and
+	  if guests need generic access to SCSI devices.
+
 config XEN_PRIVCMD
 	tristate
 	depends on XEN
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 84044b5..2140398 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -36,6 +36,7 @@ obj-$(CONFIG_XEN_ACPI_HOTPLUG_MEMORY)	+= xen-acpi-memhotplug.o
 obj-$(CONFIG_XEN_ACPI_HOTPLUG_CPU)	+= xen-acpi-cpuhotplug.o
 obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
 obj-$(CONFIG_XEN_EFI)			+= efi.o
+obj-$(CONFIG_XEN_SCSI_BACKEND)		+= xen-scsiback.o
 xen-evtchn-y				:= evtchn.o
 xen-gntdev-y				:= gntdev.o
 xen-gntalloc-y				:= gntalloc.o
diff --git a/drivers/xen/xen-scsiback.c b/drivers/xen/xen-scsiback.c
new file mode 100644
index 0000000..ed7fc6f
--- /dev/null
+++ b/drivers/xen/xen-scsiback.c
@@ -0,0 +1,2124 @@
+/*
+ * Xen SCSI backend driver
+ *
+ * Copyright (c) 2008, FUJITSU Limited
+ *
+ * Based on the blkback driver code.
+ * Adaption to kernel taget core infrastructure taken from vhost/scsi.c
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include <stdarg.h>
+
+#include <linux/module.h>
+#include <linux/utsname.h>
+#include <linux/interrupt.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+#include <linux/sched.h>
+#include <linux/list.h>
+#include <linux/gfp.h>
+#include <linux/delay.h>
+#include <linux/spinlock.h>
+#include <linux/configfs.h>
+
+#include <generated/utsrelease.h>
+
+#include <scsi/scsi_dbg.h>
+#include <scsi/scsi_eh.h>
+#include <scsi/scsi_tcq.h>
+
+#include <target/target_core_base.h>
+#include <target/target_core_fabric.h>
+#include <target/target_core_configfs.h>
+#include <target/target_core_fabric_configfs.h>
+
+#include <asm/hypervisor.h>
+
+#include <xen/xen.h>
+#include <xen/balloon.h>
+#include <xen/events.h>
+#include <xen/xenbus.h>
+#include <xen/grant_table.h>
+#include <xen/page.h>
+
+#include <xen/interface/grant_table.h>
+#include <xen/interface/io/vscsiif.h>
+
+#define DPRINTK(_f, _a...)			\
+	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)
+
+#define VSCSI_VERSION	"v0.1"
+#define VSCSI_NAMELEN	32
+
+struct ids_tuple {
+	unsigned int hst;		/* host    */
+	unsigned int chn;		/* channel */
+	unsigned int tgt;		/* target  */
+	unsigned int lun;		/* LUN     */
+};
+
+struct v2p_entry {
+	struct ids_tuple v;		/* translate from */
+	struct scsiback_tpg *tpg;	/* translate to   */
+	unsigned int lun;
+	struct kref kref;
+	struct list_head l;
+};
+
+struct vscsibk_info {
+	struct xenbus_device *dev;
+
+	domid_t domid;
+	unsigned int irq;
+
+	struct vscsiif_back_ring ring;
+	int ring_error;
+
+	spinlock_t ring_lock;
+	atomic_t nr_unreplied_reqs;
+
+	spinlock_t v2p_lock;
+	struct list_head v2p_entry_lists;
+
+	wait_queue_head_t waiting_to_free;
+};
+
+/* theoretical maximum of grants for one request */
+#define VSCSI_MAX_GRANTS	(SG_ALL + VSCSIIF_SG_TABLESIZE)
+
+/*
+ * VSCSI_GRANT_BATCH is the maximum number of grants to be processed in one
+ * call to map/unmap grants. Don't choose it too large, as there are arrays
+ * with VSCSI_GRANT_BATCH elements allocated on the stack.
+ */
+#define VSCSI_GRANT_BATCH	16
+
+struct vscsibk_pend {
+	uint16_t rqid;
+
+	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];
+	uint8_t cmd_len;
+
+	uint8_t sc_data_direction;
+	uint16_t n_sg;		/* real length of SG list */
+	uint16_t n_grants;	/* SG pages and potentially SG list */
+	uint32_t data_len;
+	uint32_t result;
+
+	struct vscsibk_info *info;
+	struct v2p_entry *v2p;
+	struct scatterlist *sgl;
+
+	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
+
+	grant_handle_t grant_handles[VSCSI_MAX_GRANTS];
+	struct page *pages[VSCSI_MAX_GRANTS];
+
+	struct se_cmd se_cmd;
+};
+
+struct scsiback_tmr {
+	atomic_t tmr_complete;
+	wait_queue_head_t tmr_wait;
+};
+
+struct scsiback_nexus {
+	/* Pointer to TCM session for I_T Nexus */
+	struct se_session *tvn_se_sess;
+};
+
+struct scsiback_tport {
+	/* SCSI protocol the tport is providing */
+	u8 tport_proto_id;
+	/* Binary World Wide unique Port Name for pvscsi Target port */
+	u64 tport_wwpn;
+	/* ASCII formatted WWPN for pvscsi Target port */
+	char tport_name[VSCSI_NAMELEN];
+	/* Returned by scsiback_make_tport() */
+	struct se_wwn tport_wwn;
+};
+
+struct scsiback_tpg {
+	/* scsiback port target portal group tag for TCM */
+	u16 tport_tpgt;
+	/* track number of TPG Port/Lun Links wrt explicit I_T Nexus shutdown */
+	int tv_tpg_port_count;
+	/* xen-pvscsi references to tpg_nexus, protected by tv_tpg_mutex */
+	int tv_tpg_fe_count;
+	/* list for scsiback_list */
+	struct list_head tv_tpg_list;
+	/* Used to protect access for tpg_nexus */
+	struct mutex tv_tpg_mutex;
+	/* Pointer to the TCM pvscsi I_T Nexus for this TPG endpoint */
+	struct scsiback_nexus *tpg_nexus;
+	/* Pointer back to scsiback_tport */
+	struct scsiback_tport *tport;
+	/* Returned by scsiback_make_tpg() */
+	struct se_portal_group se_tpg;
+	/* alias used in xenstore */
+	char param_alias[VSCSI_NAMELEN];
+	/* list of info structures related to this target portal group */
+	struct list_head info_list;
+};
+
+#define SCSIBACK_INVALID_HANDLE (~0)
+
+static bool log_print_stat;
+module_param(log_print_stat, bool, 0644);
+
+static int scsiback_max_buffer_pages = 1024;
+module_param_named(max_buffer_pages, scsiback_max_buffer_pages, int, 0644);
+MODULE_PARM_DESC(max_buffer_pages,
+"Maximum number of free pages to keep in backend buffer");
+
+static struct kmem_cache *scsiback_cachep;
+static DEFINE_SPINLOCK(free_pages_lock);
+static int free_pages_num;
+static LIST_HEAD(scsiback_free_pages);
+
+/* Global spinlock to protect scsiback TPG list */
+static DEFINE_MUTEX(scsiback_mutex);
+static LIST_HEAD(scsiback_list);
+
+/* Local pointer to allocated TCM configfs fabric module */
+static struct target_fabric_configfs *scsiback_fabric_configfs;
+
+static void scsiback_get(struct vscsibk_info *info)
+{
+	atomic_inc(&info->nr_unreplied_reqs);
+}
+
+static void scsiback_put(struct vscsibk_info *info)
+{
+	if (atomic_dec_and_test(&info->nr_unreplied_reqs))
+		wake_up(&info->waiting_to_free);
+}
+
+static void put_free_pages(struct page **page, int num)
+{
+	unsigned long flags;
+	int i = free_pages_num + num, n = num;
+
+	if (num == 0)
+		return;
+	if (i > scsiback_max_buffer_pages) {
+		n = min(num, i - scsiback_max_buffer_pages);
+		free_xenballooned_pages(n, page + num - n);
+		n = num - n;
+	}
+	spin_lock_irqsave(&free_pages_lock, flags);
+	for (i = 0; i < n; i++)
+		list_add(&page[i]->lru, &scsiback_free_pages);
+	free_pages_num += n;
+	spin_unlock_irqrestore(&free_pages_lock, flags);
+}
+
+static int get_free_page(struct page **page)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&free_pages_lock, flags);
+	if (list_empty(&scsiback_free_pages)) {
+		spin_unlock_irqrestore(&free_pages_lock, flags);
+		return alloc_xenballooned_pages(1, page, false);
+	}
+	page[0] = list_first_entry(&scsiback_free_pages, struct page, lru);
+	list_del(&page[0]->lru);
+	free_pages_num--;
+	spin_unlock_irqrestore(&free_pages_lock, flags);
+	return 0;
+}
+
+static unsigned long vaddr_page(struct page *page)
+{
+	unsigned long pfn = page_to_pfn(page);
+
+	return (unsigned long)pfn_to_kaddr(pfn);
+}
+
+static unsigned long vaddr(struct vscsibk_pend *req, int seg)
+{
+	return vaddr_page(req->pages[seg]);
+}
+
+static void scsiback_print_status(char *sense_buffer, int errors,
+					struct vscsibk_pend *pending_req)
+{
+	struct scsiback_tpg *tpg = pending_req->v2p->tpg;
+
+	pr_err("xen-pvscsi[%s:%d] cmnd[0]=%02x -> st=%02x msg=%02x host=%02x drv=%02x\n",
+	       tpg->tport->tport_name, pending_req->v2p->lun,
+	       pending_req->cmnd[0], status_byte(errors), msg_byte(errors),
+	       host_byte(errors), driver_byte(errors));
+
+	if (CHECK_CONDITION & status_byte(errors))
+		__scsi_print_sense("xen-pvscsi", sense_buffer,
+				   SCSI_SENSE_BUFFERSIZE);
+}
+
+static void scsiback_fast_flush_area(struct vscsibk_pend *req)
+{
+	struct gnttab_unmap_grant_ref unmap[VSCSI_GRANT_BATCH];
+	struct page *pages[VSCSI_GRANT_BATCH];
+	unsigned int i, invcount = 0;
+	grant_handle_t handle;
+	int err;
+
+	kfree(req->sgl);
+	req->sgl = NULL;
+	req->n_sg = 0;
+
+	if (!req->n_grants)
+		return;
+
+	for (i = 0; i < req->n_grants; i++) {
+		handle = req->grant_handles[i];
+		if (handle == SCSIBACK_INVALID_HANDLE)
+			continue;
+		gnttab_set_unmap_op(&unmap[invcount], vaddr(req, i),
+				    GNTMAP_host_map, handle);
+		req->grant_handles[i] = SCSIBACK_INVALID_HANDLE;
+		pages[invcount] = req->pages[i];
+		put_page(pages[invcount]);
+		invcount++;
+		if (invcount < VSCSI_GRANT_BATCH)
+			continue;
+		err = gnttab_unmap_refs(unmap, NULL, pages, invcount);
+		BUG_ON(err);
+		invcount = 0;
+	}
+
+	if (invcount) {
+		err = gnttab_unmap_refs(unmap, NULL, pages, invcount);
+		BUG_ON(err);
+	}
+
+	put_free_pages(req->pages, req->n_grants);
+	req->n_grants = 0;
+}
+
+static void scsiback_free_translation_entry(struct kref *kref)
+{
+	struct v2p_entry *entry = container_of(kref, struct v2p_entry, kref);
+	struct scsiback_tpg *tpg = entry->tpg;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_fe_count--;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	kfree(entry);
+}
+
+static void scsiback_do_resp_with_sense(char *sense_buffer, int32_t result,
+			uint32_t resid, struct vscsibk_pend *pending_req)
+{
+	struct vscsiif_response *ring_res;
+	struct vscsibk_info *info = pending_req->info;
+	int notify;
+	struct scsi_sense_hdr sshdr;
+	unsigned long flags;
+	unsigned len;
+
+	spin_lock_irqsave(&info->ring_lock, flags);
+
+	ring_res = RING_GET_RESPONSE(&info->ring, info->ring.rsp_prod_pvt);
+	info->ring.rsp_prod_pvt++;
+
+	ring_res->rslt   = result;
+	ring_res->rqid   = pending_req->rqid;
+
+	if (sense_buffer != NULL &&
+	    scsi_normalize_sense(sense_buffer, VSCSIIF_SENSE_BUFFERSIZE,
+				 &sshdr)) {
+		len = min_t(unsigned, 8 + sense_buffer[7],
+			    VSCSIIF_SENSE_BUFFERSIZE);
+		memcpy(ring_res->sense_buffer, sense_buffer, len);
+		ring_res->sense_len = len;
+	} else {
+		ring_res->sense_len = 0;
+	}
+
+	ring_res->residual_len = resid;
+
+	RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&info->ring, notify);
+	spin_unlock_irqrestore(&info->ring_lock, flags);
+
+	if (notify)
+		notify_remote_via_irq(info->irq);
+
+	if (pending_req->v2p)
+		kref_put(&pending_req->v2p->kref,
+			 scsiback_free_translation_entry);
+}
+
+static void scsiback_cmd_done(struct vscsibk_pend *pending_req)
+{
+	struct vscsibk_info *info = pending_req->info;
+	unsigned char *sense_buffer;
+	unsigned int resid;
+	int errors;
+
+	sense_buffer = pending_req->sense_buffer;
+	resid        = pending_req->se_cmd.residual_count;
+	errors       = pending_req->result;
+
+	if (errors && log_print_stat)
+		scsiback_print_status(sense_buffer, errors, pending_req);
+
+	scsiback_fast_flush_area(pending_req);
+	scsiback_do_resp_with_sense(sense_buffer, errors, resid, pending_req);
+	scsiback_put(info);
+}
+
+static void scsiback_cmd_exec(struct vscsibk_pend *pending_req)
+{
+	struct se_cmd *se_cmd = &pending_req->se_cmd;
+	struct se_session *sess = pending_req->v2p->tpg->tpg_nexus->tvn_se_sess;
+	int rc;
+
+	memset(pending_req->sense_buffer, 0, VSCSIIF_SENSE_BUFFERSIZE);
+
+	memset(se_cmd, 0, sizeof(*se_cmd));
+
+	scsiback_get(pending_req->info);
+	rc = target_submit_cmd_map_sgls(se_cmd, sess, pending_req->cmnd,
+			pending_req->sense_buffer, pending_req->v2p->lun,
+			pending_req->data_len, 0,
+			pending_req->sc_data_direction, 0,
+			pending_req->sgl, pending_req->n_sg,
+			NULL, 0, NULL, 0);
+	if (rc < 0) {
+		transport_send_check_condition_and_sense(se_cmd,
+				TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE, 0);
+		transport_generic_free_cmd(se_cmd, 0);
+	}
+}
+
+static int scsiback_gnttab_data_map_batch(struct gnttab_map_grant_ref *map,
+	struct page **pg, grant_handle_t *grant, int cnt)
+{
+	int err, i;
+
+	if (!cnt)
+		return 0;
+
+	err = gnttab_map_refs(map, NULL, pg, cnt);
+	BUG_ON(err);
+	for (i = 0; i < cnt; i++) {
+		if (unlikely(map[i].status != GNTST_okay)) {
+			pr_err("xen-pvscsi: invalid buffer -- could not remap it\n");
+			map[i].handle = SCSIBACK_INVALID_HANDLE;
+			err = -ENOMEM;
+		} else {
+			get_page(pg[i]);
+		}
+		grant[i] = map[i].handle;
+	}
+	return err;
+}
+
+static int scsiback_gnttab_data_map_list(struct vscsibk_pend *pending_req,
+			struct scsiif_request_segment *seg, struct page **pg,
+			grant_handle_t *grant, int cnt, u32 flags)
+{
+	int mapcount = 0, i, err = 0;
+	struct gnttab_map_grant_ref map[VSCSI_GRANT_BATCH];
+	struct vscsibk_info *info = pending_req->info;
+
+	for (i = 0; i < cnt; i++) {
+		if (get_free_page(pg + mapcount)) {
+			put_free_pages(pg, mapcount);
+			pr_err("xen-pvscsi: no grant page\n");
+			return -ENOMEM;
+		}
+		gnttab_set_map_op(&map[mapcount], vaddr_page(pg[mapcount]),
+				  flags, seg[i].gref, info->domid);
+		mapcount++;
+		if (mapcount < VSCSI_GRANT_BATCH)
+			continue;
+		err = scsiback_gnttab_data_map_batch(map, pg, grant, mapcount);
+		pg += mapcount;
+		grant += mapcount;
+		pending_req->n_grants += mapcount;
+		if (err)
+			return err;
+		mapcount = 0;
+	}
+	err = scsiback_gnttab_data_map_batch(map, pg, grant, mapcount);
+	pending_req->n_grants += mapcount;
+	return err;
+}
+
+static int scsiback_gnttab_data_map(struct vscsiif_request *ring_req,
+					struct vscsibk_pend *pending_req)
+{
+	u32 flags;
+	int i, err, n_segs, i_seg = 0;
+	struct page **pg;
+	struct scsiif_request_segment *seg;
+	unsigned long end_seg = 0;
+	unsigned int nr_segments = (unsigned int)ring_req->nr_segments;
+	unsigned int nr_sgl = 0;
+	struct scatterlist *sg;
+	grant_handle_t *grant;
+
+	pending_req->n_sg = 0;
+	pending_req->n_grants = 0;
+	pending_req->data_len = 0;
+
+	nr_segments &= ~VSCSIIF_SG_GRANT;
+	if (!nr_segments)
+		return 0;
+
+	if (nr_segments > VSCSIIF_SG_TABLESIZE) {
+		DPRINTK("xen-pvscsi: invalid parameter nr_seg = %d\n",
+			ring_req->nr_segments);
+		return -EINVAL;
+	}
+
+	if (ring_req->nr_segments & VSCSIIF_SG_GRANT) {
+		err = scsiback_gnttab_data_map_list(pending_req, ring_req->seg,
+			pending_req->pages, pending_req->grant_handles,
+			nr_segments, GNTMAP_host_map | GNTMAP_readonly);
+		if (err)
+			return err;
+		nr_sgl = nr_segments;
+		nr_segments = 0;
+		for (i = 0; i < nr_sgl; i++) {
+			n_segs = ring_req->seg[i].length /
+				 sizeof(struct scsiif_request_segment);
+			if ((unsigned)ring_req->seg[i].offset +
+			    (unsigned)ring_req->seg[i].length > PAGE_SIZE ||
+			    n_segs * sizeof(struct scsiif_request_segment) !=
+			    ring_req->seg[i].length)
+				return -EINVAL;
+			nr_segments += n_segs;
+		}
+		if (nr_segments > SG_ALL) {
+			DPRINTK("xen-pvscsi: invalid nr_seg = %d\n",
+				nr_segments);
+			return -EINVAL;
+		}
+	}
+
+	/* free of (sgl) in fast_flush_area()*/
+	pending_req->sgl = kmalloc_array(nr_segments,
+					sizeof(struct scatterlist), GFP_KERNEL);
+	if (!pending_req->sgl)
+		return -ENOMEM;
+
+	sg_init_table(pending_req->sgl, nr_segments);
+	pending_req->n_sg = nr_segments;
+
+	flags = GNTMAP_host_map;
+	if (pending_req->sc_data_direction == DMA_TO_DEVICE)
+		flags |= GNTMAP_readonly;
+
+	pg = pending_req->pages + nr_sgl;
+	grant = pending_req->grant_handles + nr_sgl;
+	if (!nr_sgl) {
+		seg = ring_req->seg;
+		err = scsiback_gnttab_data_map_list(pending_req, seg,
+			pg, grant, nr_segments, flags);
+		if (err)
+			return err;
+	} else {
+		for (i = 0; i < nr_sgl; i++) {
+			seg = (struct scsiif_request_segment *)(
+			      vaddr(pending_req, i) + ring_req->seg[i].offset);
+			n_segs = ring_req->seg[i].length /
+				 sizeof(struct scsiif_request_segment);
+			err = scsiback_gnttab_data_map_list(pending_req, seg,
+				pg, grant, n_segs, flags);
+			if (err)
+				return err;
+			pg += n_segs;
+			grant += n_segs;
+		}
+		end_seg = vaddr(pending_req, 0) + ring_req->seg[0].offset;
+		seg = (struct scsiif_request_segment *)end_seg;
+		end_seg += ring_req->seg[0].length;
+		pg = pending_req->pages + nr_sgl;
+	}
+
+	for_each_sg(pending_req->sgl, sg, nr_segments, i) {
+		sg_set_page(sg, pg[i], seg->length, seg->offset);
+		pending_req->data_len += seg->length;
+		seg++;
+		if (nr_sgl && (unsigned long)seg >= end_seg) {
+			i_seg++;
+			end_seg = vaddr(pending_req, i_seg) +
+				  ring_req->seg[i_seg].offset;
+			seg = (struct scsiif_request_segment *)end_seg;
+			end_seg += ring_req->seg[i_seg].length;
+		}
+		if (sg->offset >= PAGE_SIZE ||
+		    sg->length > PAGE_SIZE ||
+		    sg->offset + sg->length > PAGE_SIZE)
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void scsiback_disconnect(struct vscsibk_info *info)
+{
+	wait_event(info->waiting_to_free,
+		atomic_read(&info->nr_unreplied_reqs) == 0);
+
+	unbind_from_irqhandler(info->irq, info);
+	info->irq = 0;
+	xenbus_unmap_ring_vfree(info->dev, info->ring.sring);
+}
+
+static void scsiback_device_action(struct vscsibk_pend *pending_req,
+	enum tcm_tmreq_table act, int tag)
+{
+	int rc, err = FAILED;
+	struct scsiback_tpg *tpg = pending_req->v2p->tpg;
+	struct se_cmd *se_cmd = &pending_req->se_cmd;
+	struct scsiback_tmr *tmr;
+
+	tmr = kzalloc(sizeof(struct scsiback_tmr), GFP_KERNEL);
+	if (!tmr)
+		goto out;
+
+	init_waitqueue_head(&tmr->tmr_wait);
+
+	transport_init_se_cmd(se_cmd, tpg->se_tpg.se_tpg_tfo,
+		tpg->tpg_nexus->tvn_se_sess, 0, DMA_NONE, MSG_SIMPLE_TAG,
+		&pending_req->sense_buffer[0]);
+
+	rc = core_tmr_alloc_req(se_cmd, tmr, act, GFP_KERNEL);
+	if (rc < 0)
+		goto out;
+
+	se_cmd->se_tmr_req->ref_task_tag = tag;
+
+	if (transport_lookup_tmr_lun(se_cmd, pending_req->v2p->lun) < 0)
+		goto out;
+
+	transport_generic_handle_tmr(se_cmd);
+	wait_event(tmr->tmr_wait, atomic_read(&tmr->tmr_complete));
+
+	err = (se_cmd->se_tmr_req->response == TMR_FUNCTION_COMPLETE) ?
+		SUCCESS : FAILED;
+
+out:
+	if (tmr) {
+		transport_generic_free_cmd(&pending_req->se_cmd, 1);
+		kfree(tmr);
+	}
+
+	scsiback_do_resp_with_sense(NULL, err, 0, pending_req);
+
+	kmem_cache_free(scsiback_cachep, pending_req);
+}
+
+/*
+  Perform virtual to physical translation
+*/
+static struct v2p_entry *scsiback_do_translation(struct vscsibk_info *info,
+			struct ids_tuple *v)
+{
+	struct v2p_entry *entry;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+	list_for_each_entry(entry, head, l) {
+		if ((entry->v.chn == v->chn) &&
+		    (entry->v.tgt == v->tgt) &&
+		    (entry->v.lun == v->lun)) {
+			kref_get(&entry->kref);
+			goto out;
+		}
+	}
+	entry = NULL;
+
+out:
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+	return entry;
+}
+
+static int prepare_pending_reqs(struct vscsibk_info *info,
+				struct vscsiif_request *ring_req,
+				struct vscsibk_pend *pending_req)
+{
+	struct v2p_entry *v2p;
+	struct ids_tuple vir;
+
+	pending_req->rqid       = ring_req->rqid;
+	pending_req->info       = info;
+
+	vir.chn = ring_req->channel;
+	vir.tgt = ring_req->id;
+	vir.lun = ring_req->lun;
+
+	v2p = scsiback_do_translation(info, &vir);
+	if (!v2p) {
+		pending_req->v2p = NULL;
+		DPRINTK("xen-pvscsi: doesn't exist.\n");
+		return -ENODEV;
+	}
+	pending_req->v2p = v2p;
+
+	/* request range check from frontend */
+	pending_req->sc_data_direction = ring_req->sc_data_direction;
+	if ((pending_req->sc_data_direction != DMA_BIDIRECTIONAL) &&
+		(pending_req->sc_data_direction != DMA_TO_DEVICE) &&
+		(pending_req->sc_data_direction != DMA_FROM_DEVICE) &&
+		(pending_req->sc_data_direction != DMA_NONE)) {
+		DPRINTK("xen-pvscsi: invalid parameter data_dir = %d\n",
+			pending_req->sc_data_direction);
+		return -EINVAL;
+	}
+
+	pending_req->cmd_len = ring_req->cmd_len;
+	if (pending_req->cmd_len > VSCSIIF_MAX_COMMAND_SIZE) {
+		DPRINTK("xen-pvscsi: invalid parameter cmd_len = %d\n",
+			pending_req->cmd_len);
+		return -EINVAL;
+	}
+	memcpy(pending_req->cmnd, ring_req->cmnd, pending_req->cmd_len);
+
+	return 0;
+}
+
+static int scsiback_do_cmd_fn(struct vscsibk_info *info)
+{
+	struct vscsiif_back_ring *ring = &info->ring;
+	struct vscsiif_request *ring_req;
+	struct vscsibk_pend *pending_req;
+	RING_IDX rc, rp;
+	int err, more_to_do;
+	uint32_t result;
+	uint8_t act;
+
+	rc = ring->req_cons;
+	rp = ring->sring->req_prod;
+	rmb();	/* guest system is accessing ring, too */
+
+	if (RING_REQUEST_PROD_OVERFLOW(ring, rp)) {
+		rc = ring->rsp_prod_pvt;
+		pr_warn("xen-pvscsi: Dom%d provided bogus ring requests (%#x - %#x = %u). Halting ring processing\n",
+			   info->domid, rp, rc, rp - rc);
+		info->ring_error = 1;
+		return 0;
+	}
+
+	while ((rc != rp)) {
+		if (RING_REQUEST_CONS_OVERFLOW(ring, rc))
+			break;
+		pending_req = kmem_cache_alloc(scsiback_cachep, GFP_KERNEL);
+		if (!pending_req)
+			return 1;
+
+		ring_req = RING_GET_REQUEST(ring, rc);
+		ring->req_cons = ++rc;
+
+		act = ring_req->act;
+		err = prepare_pending_reqs(info, ring_req, pending_req);
+		if (err) {
+			switch (err) {
+			case -ENODEV:
+				result = DID_NO_CONNECT;
+				break;
+			default:
+				result = DRIVER_ERROR;
+				break;
+			}
+			scsiback_do_resp_with_sense(NULL, result << 24, 0,
+						    pending_req);
+			kmem_cache_free(scsiback_cachep, pending_req);
+			return 1;
+		}
+
+		switch (act) {
+		case VSCSIIF_ACT_SCSI_CDB:
+			if (scsiback_gnttab_data_map(ring_req, pending_req)) {
+				scsiback_fast_flush_area(pending_req);
+				scsiback_do_resp_with_sense(NULL,
+					DRIVER_ERROR << 24, 0, pending_req);
+				kmem_cache_free(scsiback_cachep, pending_req);
+			} else {
+				scsiback_cmd_exec(pending_req);
+			}
+			break;
+		case VSCSIIF_ACT_SCSI_ABORT:
+			scsiback_device_action(pending_req, TMR_ABORT_TASK,
+				ring_req->ref_rqid);
+			break;
+		case VSCSIIF_ACT_SCSI_RESET:
+			scsiback_device_action(pending_req, TMR_LUN_RESET, 0);
+			break;
+		default:
+			pr_err_ratelimited("xen-pvscsi: invalid request\n");
+			scsiback_do_resp_with_sense(NULL, DRIVER_ERROR << 24,
+						    0, pending_req);
+			kmem_cache_free(scsiback_cachep, pending_req);
+			break;
+		}
+
+		/* Yield point for this unbounded loop. */
+		cond_resched();
+	}
+
+	RING_FINAL_CHECK_FOR_REQUESTS(&info->ring, more_to_do);
+	return more_to_do;
+}
+
+static irqreturn_t scsiback_irq_fn(int irq, void *dev_id)
+{
+	struct vscsibk_info *info = dev_id;
+
+	if (info->ring_error)
+		return IRQ_HANDLED;
+
+	while (scsiback_do_cmd_fn(info))
+		cond_resched();
+
+	return IRQ_HANDLED;
+}
+
+static int scsiback_init_sring(struct vscsibk_info *info, grant_ref_t ring_ref,
+			evtchn_port_t evtchn)
+{
+	void *area;
+	struct vscsiif_sring *sring;
+	int err;
+
+	if (info->irq)
+		return -1;
+
+	err = xenbus_map_ring_valloc(info->dev, ring_ref, &area);
+	if (err)
+		return err;
+
+	sring = (struct vscsiif_sring *)area;
+	BACK_RING_INIT(&info->ring, sring, PAGE_SIZE);
+
+	err = bind_interdomain_evtchn_to_irq(info->domid, evtchn);
+	if (err < 0)
+		goto unmap_page;
+
+	info->irq = err;
+
+	err = request_threaded_irq(info->irq, NULL, scsiback_irq_fn,
+				   IRQF_ONESHOT, "vscsiif-backend", info);
+	if (err)
+		goto free_irq;
+
+	return 0;
+
+free_irq:
+	unbind_from_irqhandler(info->irq, info);
+	info->irq = 0;
+unmap_page:
+	xenbus_unmap_ring_vfree(info->dev, area);
+
+	return err;
+}
+
+static int scsiback_map(struct vscsibk_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	unsigned int ring_ref, evtchn;
+	int err;
+
+	err = xenbus_gather(XBT_NIL, dev->otherend,
+			"ring-ref", "%u", &ring_ref,
+			"event-channel", "%u", &evtchn, NULL);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "reading %s ring", dev->otherend);
+		return err;
+	}
+
+	return scsiback_init_sring(info, ring_ref, evtchn);
+}
+
+/*
+  Add a new translation entry
+*/
+static int scsiback_add_translation_entry(struct vscsibk_info *info,
+					  char *phy, struct ids_tuple *v)
+{
+	int err = 0;
+	struct v2p_entry *entry;
+	struct v2p_entry *new;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+	char *lunp;
+	unsigned int lun;
+	struct scsiback_tpg *tpg_entry, *tpg = NULL;
+	char *error = "doesn't exist";
+
+	lunp = strrchr(phy, ':');
+	if (!lunp) {
+		pr_err("xen-pvscsi: illegal format of physical device %s\n",
+			phy);
+		return -EINVAL;
+	}
+	*lunp = 0;
+	lunp++;
+	if (kstrtouint(lunp, 10, &lun) || lun >= TRANSPORT_MAX_LUNS_PER_TPG) {
+		pr_err("xen-pvscsi: lun number not valid: %s\n", lunp);
+		return -EINVAL;
+	}
+
+	mutex_lock(&scsiback_mutex);
+	list_for_each_entry(tpg_entry, &scsiback_list, tv_tpg_list) {
+		if (!strcmp(phy, tpg_entry->tport->tport_name) ||
+		    !strcmp(phy, tpg_entry->param_alias)) {
+			spin_lock(&tpg_entry->se_tpg.tpg_lun_lock);
+			if (tpg_entry->se_tpg.tpg_lun_list[lun]->lun_status ==
+			    TRANSPORT_LUN_STATUS_ACTIVE) {
+				if (!tpg_entry->tpg_nexus)
+					error = "nexus undefined";
+				else
+					tpg = tpg_entry;
+			}
+			spin_unlock(&tpg_entry->se_tpg.tpg_lun_lock);
+			break;
+		}
+	}
+	if (tpg) {
+		mutex_lock(&tpg->tv_tpg_mutex);
+		tpg->tv_tpg_fe_count++;
+		mutex_unlock(&tpg->tv_tpg_mutex);
+	}
+	mutex_unlock(&scsiback_mutex);
+
+	if (!tpg) {
+		pr_err("xen-pvscsi: %s:%d %s\n", phy, lun, error);
+		return -ENODEV;
+	}
+
+	new = kmalloc(sizeof(struct v2p_entry), GFP_KERNEL);
+	if (new == NULL) {
+		err = -ENOMEM;
+		goto out_free;
+	}
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+
+	/* Check double assignment to identical virtual ID */
+	list_for_each_entry(entry, head, l) {
+		if ((entry->v.chn == v->chn) &&
+		    (entry->v.tgt == v->tgt) &&
+		    (entry->v.lun == v->lun)) {
+			pr_warn("xen-pvscsi: Virtual ID is already used. Assignment was not performed.\n");
+			err = -EEXIST;
+			goto out;
+		}
+
+	}
+
+	/* Create a new translation entry and add to the list */
+	kref_init(&new->kref);
+	new->v = *v;
+	new->tpg = tpg;
+	new->lun = lun;
+	list_add_tail(&new->l, head);
+
+out:
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+
+out_free:
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_fe_count--;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	if (err)
+		kfree(new);
+
+	return err;
+}
+
+static void __scsiback_del_translation_entry(struct v2p_entry *entry)
+{
+	list_del(&entry->l);
+	kref_put(&entry->kref, scsiback_free_translation_entry);
+	kfree(entry);
+}
+
+/*
+  Delete the translation entry specfied
+*/
+static int scsiback_del_translation_entry(struct vscsibk_info *info,
+					  struct ids_tuple *v)
+{
+	struct v2p_entry *entry;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+	/* Find out the translation entry specified */
+	list_for_each_entry(entry, head, l) {
+		if ((entry->v.chn == v->chn) &&
+		    (entry->v.tgt == v->tgt) &&
+		    (entry->v.lun == v->lun)) {
+			goto found;
+		}
+	}
+
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+	return 1;
+
+found:
+	/* Delete the translation entry specfied */
+	__scsiback_del_translation_entry(entry);
+
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+	return 0;
+}
+
+static void scsiback_do_add_lun(struct vscsibk_info *info, const char *state,
+				char *phy, struct ids_tuple *vir)
+{
+	if (!scsiback_add_translation_entry(info, phy, vir)) {
+		if (xenbus_printf(XBT_NIL, info->dev->nodename, state,
+				  "%d", XenbusStateInitialised)) {
+			pr_err("xen-pvscsi: xenbus_printf error %s\n", state);
+			scsiback_del_translation_entry(info, vir);
+		}
+	} else {
+		xenbus_printf(XBT_NIL, info->dev->nodename, state,
+			      "%d", XenbusStateClosed);
+	}
+}
+
+static void scsiback_do_del_lun(struct vscsibk_info *info, const char *state,
+				struct ids_tuple *vir)
+{
+	if (!scsiback_del_translation_entry(info, vir)) {
+		if (xenbus_printf(XBT_NIL, info->dev->nodename, state,
+				  "%d", XenbusStateClosed))
+			pr_err("xen-pvscsi: xenbus_printf error %s\n", state);
+	}
+}
+
+#define VSCSIBACK_OP_ADD_OR_DEL_LUN	1
+#define VSCSIBACK_OP_UPDATEDEV_STATE	2
+
+static void scsiback_do_1lun_hotplug(struct vscsibk_info *info, int op,
+				     char *ent)
+{
+	int err;
+	struct ids_tuple vir;
+	char *val;
+	int device_state;
+	char phy[VSCSI_NAMELEN];
+	char str[64];
+	char state[64];
+	struct xenbus_device *dev = info->dev;
+
+	/* read status */
+	snprintf(state, sizeof(state), "vscsi-devs/%s/state", ent);
+	err = xenbus_scanf(XBT_NIL, dev->nodename, state, "%u", &device_state);
+	if (XENBUS_EXIST_ERR(err))
+		return;
+
+	/* physical SCSI device */
+	snprintf(str, sizeof(str), "vscsi-devs/%s/p-dev", ent);
+	val = xenbus_read(XBT_NIL, dev->nodename, str, NULL);
+	if (IS_ERR(val)) {
+		xenbus_printf(XBT_NIL, dev->nodename, state,
+			      "%d", XenbusStateClosed);
+		return;
+	}
+	strlcpy(phy, val, VSCSI_NAMELEN);
+	kfree(val);
+
+	/* virtual SCSI device */
+	snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", ent);
+	err = xenbus_scanf(XBT_NIL, dev->nodename, str, "%u:%u:%u:%u",
+			   &vir.hst, &vir.chn, &vir.tgt, &vir.lun);
+	if (XENBUS_EXIST_ERR(err)) {
+		xenbus_printf(XBT_NIL, dev->nodename, state,
+			      "%d", XenbusStateClosed);
+		return;
+	}
+
+	switch (op) {
+	case VSCSIBACK_OP_ADD_OR_DEL_LUN:
+		if (device_state == XenbusStateInitialising)
+			scsiback_do_add_lun(info, state, phy, &vir);
+		if (device_state == XenbusStateClosing)
+			scsiback_do_del_lun(info, state, &vir);
+		break;
+
+	case VSCSIBACK_OP_UPDATEDEV_STATE:
+		if (device_state == XenbusStateInitialised) {
+			/* modify vscsi-devs/dev-x/state */
+			if (xenbus_printf(XBT_NIL, dev->nodename, state,
+					  "%d", XenbusStateConnected)) {
+				pr_err("xen-pvscsi: xenbus_printf error %s\n",
+				       str);
+				scsiback_del_translation_entry(info, &vir);
+				xenbus_printf(XBT_NIL, dev->nodename, state,
+					      "%d", XenbusStateClosed);
+			}
+		}
+		break;
+	/*When it is necessary, processing is added here.*/
+	default:
+		break;
+	}
+}
+
+static void scsiback_do_lun_hotplug(struct vscsibk_info *info, int op)
+{
+	int i;
+	char **dir;
+	unsigned int ndir = 0;
+
+	dir = xenbus_directory(XBT_NIL, info->dev->nodename, "vscsi-devs",
+			       &ndir);
+	if (IS_ERR(dir))
+		return;
+
+	for (i = 0; i < ndir; i++)
+		scsiback_do_1lun_hotplug(info, op, dir[i]);
+
+	kfree(dir);
+}
+
+static void scsiback_frontend_changed(struct xenbus_device *dev,
+					enum xenbus_state frontend_state)
+{
+	struct vscsibk_info *info = dev_get_drvdata(&dev->dev);
+
+	switch (frontend_state) {
+	case XenbusStateInitialising:
+		break;
+
+	case XenbusStateInitialised:
+		if (scsiback_map(info))
+			break;
+
+		scsiback_do_lun_hotplug(info, VSCSIBACK_OP_ADD_OR_DEL_LUN);
+		xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+
+	case XenbusStateConnected:
+		scsiback_do_lun_hotplug(info, VSCSIBACK_OP_UPDATEDEV_STATE);
+
+		if (dev->state == XenbusStateConnected)
+			break;
+
+		xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+
+	case XenbusStateClosing:
+		if (info->irq)
+			scsiback_disconnect(info);
+
+		xenbus_switch_state(dev, XenbusStateClosing);
+		break;
+
+	case XenbusStateClosed:
+		xenbus_switch_state(dev, XenbusStateClosed);
+		if (xenbus_dev_is_online(dev))
+			break;
+		/* fall through if not online */
+	case XenbusStateUnknown:
+		device_unregister(&dev->dev);
+		break;
+
+	case XenbusStateReconfiguring:
+		scsiback_do_lun_hotplug(info, VSCSIBACK_OP_ADD_OR_DEL_LUN);
+		xenbus_switch_state(dev, XenbusStateReconfigured);
+
+		break;
+
+	default:
+		xenbus_dev_fatal(dev, -EINVAL, "saw state %d at frontend",
+					frontend_state);
+		break;
+	}
+}
+
+/*
+  Release the translation entry specfied
+*/
+static void scsiback_release_translation_entry(struct vscsibk_info *info)
+{
+	struct v2p_entry *entry, *tmp;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+
+	list_for_each_entry_safe(entry, tmp, head, l)
+		__scsiback_del_translation_entry(entry);
+
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+}
+
+static int scsiback_remove(struct xenbus_device *dev)
+{
+	struct vscsibk_info *info = dev_get_drvdata(&dev->dev);
+
+	if (info->irq)
+		scsiback_disconnect(info);
+
+	scsiback_release_translation_entry(info);
+
+	dev_set_drvdata(&dev->dev, NULL);
+
+	return 0;
+}
+
+static int scsiback_probe(struct xenbus_device *dev,
+			   const struct xenbus_device_id *id)
+{
+	int err;
+
+	struct vscsibk_info *info = kzalloc(sizeof(struct vscsibk_info),
+					    GFP_KERNEL);
+
+	DPRINTK("%p %d\n", dev, dev->otherend_id);
+
+	if (!info) {
+		xenbus_dev_fatal(dev, -ENOMEM, "allocating backend structure");
+		return -ENOMEM;
+	}
+	info->dev = dev;
+	dev_set_drvdata(&dev->dev, info);
+
+	info->domid = dev->otherend_id;
+	spin_lock_init(&info->ring_lock);
+	info->ring_error = 0;
+	atomic_set(&info->nr_unreplied_reqs, 0);
+	init_waitqueue_head(&info->waiting_to_free);
+	info->dev = dev;
+	info->irq = 0;
+	INIT_LIST_HEAD(&info->v2p_entry_lists);
+	spin_lock_init(&info->v2p_lock);
+
+	err = xenbus_printf(XBT_NIL, dev->nodename, "feature-sg-grant", "%u",
+			    SG_ALL);
+	if (err)
+		xenbus_dev_error(dev, err, "writing feature-sg-grant");
+
+	err = xenbus_switch_state(dev, XenbusStateInitWait);
+	if (err)
+		goto fail;
+
+	return 0;
+
+fail:
+	pr_warn("xen-pvscsi: %s failed\n", __func__);
+	scsiback_remove(dev);
+
+	return err;
+}
+
+static char *scsiback_dump_proto_id(struct scsiback_tport *tport)
+{
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return "SAS";
+	case SCSI_PROTOCOL_FCP:
+		return "FCP";
+	case SCSI_PROTOCOL_ISCSI:
+		return "iSCSI";
+	default:
+		break;
+	}
+
+	return "Unknown";
+}
+
+static u8 scsiback_get_fabric_proto_ident(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_get_fabric_proto_ident(se_tpg);
+	case SCSI_PROTOCOL_FCP:
+		return fc_get_fabric_proto_ident(se_tpg);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_get_fabric_proto_ident(se_tpg);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_get_fabric_proto_ident(se_tpg);
+}
+
+static char *scsiback_get_fabric_wwn(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	return &tport->tport_name[0];
+}
+
+static u16 scsiback_get_tag(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	return tpg->tport_tpgt;
+}
+
+static u32 scsiback_get_default_depth(struct se_portal_group *se_tpg)
+{
+	return 1;
+}
+
+static u32
+scsiback_get_pr_transport_id(struct se_portal_group *se_tpg,
+			      struct se_node_acl *se_nacl,
+			      struct t10_pr_registration *pr_reg,
+			      int *format_code,
+			      unsigned char *buf)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+					format_code, buf);
+	case SCSI_PROTOCOL_FCP:
+		return fc_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+					format_code, buf);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+					format_code, buf);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+			format_code, buf);
+}
+
+static u32
+scsiback_get_pr_transport_id_len(struct se_portal_group *se_tpg,
+				  struct se_node_acl *se_nacl,
+				  struct t10_pr_registration *pr_reg,
+				  int *format_code)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+					format_code);
+	case SCSI_PROTOCOL_FCP:
+		return fc_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+					format_code);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+					format_code);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+			format_code);
+}
+
+static char *
+scsiback_parse_pr_out_transport_id(struct se_portal_group *se_tpg,
+				    const char *buf,
+				    u32 *out_tid_len,
+				    char **port_nexus_ptr)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+					port_nexus_ptr);
+	case SCSI_PROTOCOL_FCP:
+		return fc_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+					port_nexus_ptr);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+					port_nexus_ptr);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+			port_nexus_ptr);
+}
+
+static struct se_wwn *
+scsiback_make_tport(struct target_fabric_configfs *tf,
+		     struct config_group *group,
+		     const char *name)
+{
+	struct scsiback_tport *tport;
+	char *ptr;
+	u64 wwpn = 0;
+	int off = 0;
+
+	tport = kzalloc(sizeof(struct scsiback_tport), GFP_KERNEL);
+	if (!tport)
+		return ERR_PTR(-ENOMEM);
+
+	tport->tport_wwpn = wwpn;
+	/*
+	 * Determine the emulated Protocol Identifier and Target Port Name
+	 * based on the incoming configfs directory name.
+	 */
+	ptr = strstr(name, "naa.");
+	if (ptr) {
+		tport->tport_proto_id = SCSI_PROTOCOL_SAS;
+		goto check_len;
+	}
+	ptr = strstr(name, "fc.");
+	if (ptr) {
+		tport->tport_proto_id = SCSI_PROTOCOL_FCP;
+		off = 3; /* Skip over "fc." */
+		goto check_len;
+	}
+	ptr = strstr(name, "iqn.");
+	if (ptr) {
+		tport->tport_proto_id = SCSI_PROTOCOL_ISCSI;
+		goto check_len;
+	}
+
+	pr_err("Unable to locate prefix for emulated Target Port: %s\n", name);
+	kfree(tport);
+	return ERR_PTR(-EINVAL);
+
+check_len:
+	if (strlen(name) >= VSCSI_NAMELEN) {
+		pr_err("Emulated %s Address: %s, exceeds max: %d\n", name,
+			scsiback_dump_proto_id(tport), VSCSI_NAMELEN);
+		kfree(tport);
+		return ERR_PTR(-EINVAL);
+	}
+	snprintf(&tport->tport_name[0], VSCSI_NAMELEN, "%s", &name[off]);
+
+	pr_debug("xen-pvscsi: Allocated emulated Target %s Address: %s\n",
+		 scsiback_dump_proto_id(tport), name);
+
+	return &tport->tport_wwn;
+}
+
+static void scsiback_drop_tport(struct se_wwn *wwn)
+{
+	struct scsiback_tport *tport = container_of(wwn,
+				struct scsiback_tport, tport_wwn);
+
+	pr_debug("xen-pvscsi: Deallocating emulated Target %s Address: %s\n",
+		 scsiback_dump_proto_id(tport), tport->tport_name);
+
+	kfree(tport);
+}
+
+static struct se_node_acl *
+scsiback_alloc_fabric_acl(struct se_portal_group *se_tpg)
+{
+	return kzalloc(sizeof(struct se_node_acl), GFP_KERNEL);
+}
+
+static void
+scsiback_release_fabric_acl(struct se_portal_group *se_tpg,
+			     struct se_node_acl *se_nacl)
+{
+	kfree(se_nacl);
+}
+
+static u32 scsiback_tpg_get_inst_index(struct se_portal_group *se_tpg)
+{
+	return 1;
+}
+
+static int scsiback_check_stop_free(struct se_cmd *se_cmd)
+{
+	/*
+	 * Do not release struct se_cmd's containing a valid TMR
+	 * pointer.  These will be released directly in scsiback_device_action()
+	 * with transport_generic_free_cmd().
+	 */
+	if (se_cmd->se_cmd_flags & SCF_SCSI_TMR_CDB)
+		return 0;
+
+	transport_generic_free_cmd(se_cmd, 0);
+	return 1;
+}
+
+static void scsiback_release_cmd(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	kmem_cache_free(scsiback_cachep, pending_req);
+}
+
+static int scsiback_shutdown_session(struct se_session *se_sess)
+{
+	return 0;
+}
+
+static void scsiback_close_session(struct se_session *se_sess)
+{
+}
+
+static u32 scsiback_sess_get_index(struct se_session *se_sess)
+{
+	return 0;
+}
+
+static int scsiback_write_pending(struct se_cmd *se_cmd)
+{
+	/* Go ahead and process the write immediately */
+	target_execute_cmd(se_cmd);
+
+	return 0;
+}
+
+static int scsiback_write_pending_status(struct se_cmd *se_cmd)
+{
+	return 0;
+}
+
+static void scsiback_set_default_node_attrs(struct se_node_acl *nacl)
+{
+}
+
+static u32 scsiback_get_task_tag(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	return pending_req->rqid;
+}
+
+static int scsiback_get_cmd_state(struct se_cmd *se_cmd)
+{
+	return 0;
+}
+
+static int scsiback_queue_data_in(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	pending_req->result = SAM_STAT_GOOD;
+	scsiback_cmd_done(pending_req);
+	return 0;
+}
+
+static int scsiback_queue_status(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	if (se_cmd->sense_buffer &&
+	    ((se_cmd->se_cmd_flags & SCF_TRANSPORT_TASK_SENSE) ||
+	     (se_cmd->se_cmd_flags & SCF_EMULATED_TASK_SENSE)))
+		pending_req->result = (DRIVER_SENSE << 24) |
+				      SAM_STAT_CHECK_CONDITION;
+	else
+		pending_req->result = se_cmd->scsi_status;
+
+	scsiback_cmd_done(pending_req);
+	return 0;
+}
+
+static void scsiback_queue_tm_rsp(struct se_cmd *se_cmd)
+{
+	struct se_tmr_req *se_tmr = se_cmd->se_tmr_req;
+	struct scsiback_tmr *tmr = se_tmr->fabric_tmr_ptr;
+
+	atomic_set(&tmr->tmr_complete, 1);
+	wake_up(&tmr->tmr_wait);
+}
+
+static void scsiback_aborted_task(struct se_cmd *se_cmd)
+{
+}
+
+static ssize_t scsiback_tpg_param_show_alias(struct se_portal_group *se_tpg,
+					     char *page)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg, struct scsiback_tpg,
+						se_tpg);
+	ssize_t rb;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	rb = snprintf(page, PAGE_SIZE, "%s\n", tpg->param_alias);
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return rb;
+}
+
+static ssize_t scsiback_tpg_param_store_alias(struct se_portal_group *se_tpg,
+					      const char *page, size_t count)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg, struct scsiback_tpg,
+						se_tpg);
+	int len;
+
+	if (strlen(page) >= VSCSI_NAMELEN) {
+		pr_err("param alias: %s, exceeds max: %d\n", page,
+			VSCSI_NAMELEN);
+		return -EINVAL;
+	}
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	len = snprintf(tpg->param_alias, VSCSI_NAMELEN, "%s", page);
+	if (tpg->param_alias[len - 1] == '\n')
+		tpg->param_alias[len - 1] = '\0';
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return count;
+}
+
+TF_TPG_PARAM_ATTR(scsiback, alias, S_IRUGO | S_IWUSR);
+
+static struct configfs_attribute *scsiback_param_attrs[] = {
+	&scsiback_tpg_param_alias.attr,
+	NULL,
+};
+
+static int scsiback_make_nexus(struct scsiback_tpg *tpg,
+				const char *name)
+{
+	struct se_portal_group *se_tpg;
+	struct se_session *se_sess;
+	struct scsiback_nexus *tv_nexus;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	if (tpg->tpg_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_debug("tpg->tpg_nexus already exists\n");
+		return -EEXIST;
+	}
+	se_tpg = &tpg->se_tpg;
+
+	tv_nexus = kzalloc(sizeof(struct scsiback_nexus), GFP_KERNEL);
+	if (!tv_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENOMEM;
+	}
+	/*
+	 *  Initialize the struct se_session pointer
+	 */
+	tv_nexus->tvn_se_sess = transport_init_session(TARGET_PROT_NORMAL);
+	if (IS_ERR(tv_nexus->tvn_se_sess)) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		kfree(tv_nexus);
+		return -ENOMEM;
+	}
+	se_sess = tv_nexus->tvn_se_sess;
+	/*
+	 * Since we are running in 'demo mode' this call with generate a
+	 * struct se_node_acl for the scsiback struct se_portal_group with
+	 * the SCSI Initiator port name of the passed configfs group 'name'.
+	 */
+	tv_nexus->tvn_se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
+				se_tpg, (unsigned char *)name);
+	if (!tv_nexus->tvn_se_sess->se_node_acl) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_debug("core_tpg_check_initiator_node_acl() failed for %s\n",
+			 name);
+		goto out;
+	}
+	/*
+	 * Now register the TCM pvscsi virtual I_T Nexus as active with the
+	 * call to __transport_register_session()
+	 */
+	__transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
+			tv_nexus->tvn_se_sess, tv_nexus);
+	tpg->tpg_nexus = tv_nexus;
+
+	mutex_unlock(&tpg->tv_tpg_mutex);
+	return 0;
+
+out:
+	transport_free_session(se_sess);
+	kfree(tv_nexus);
+	return -ENOMEM;
+}
+
+static int scsiback_drop_nexus(struct scsiback_tpg *tpg)
+{
+	struct se_session *se_sess;
+	struct scsiback_nexus *tv_nexus;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tv_nexus = tpg->tpg_nexus;
+	if (!tv_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENODEV;
+	}
+
+	se_sess = tv_nexus->tvn_se_sess;
+	if (!se_sess) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENODEV;
+	}
+
+	if (tpg->tv_tpg_port_count != 0) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_err("Unable to remove xen-pvscsi I_T Nexus with active TPG port count: %d\n",
+			tpg->tv_tpg_port_count);
+		return -EBUSY;
+	}
+
+	if (tpg->tv_tpg_fe_count != 0) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_err("Unable to remove xen-pvscsi I_T Nexus with active TPG frontend count: %d\n",
+			tpg->tv_tpg_fe_count);
+		return -EBUSY;
+	}
+
+	pr_debug("xen-pvscsi: Removing I_T Nexus to emulated %s Initiator Port: %s\n",
+		scsiback_dump_proto_id(tpg->tport),
+		tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
+
+	/*
+	 * Release the SCSI I_T Nexus to the emulated xen-pvscsi Target Port
+	 */
+	transport_deregister_session(tv_nexus->tvn_se_sess);
+	tpg->tpg_nexus = NULL;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	kfree(tv_nexus);
+	return 0;
+}
+
+static ssize_t scsiback_tpg_show_nexus(struct se_portal_group *se_tpg,
+					char *page)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_nexus *tv_nexus;
+	ssize_t ret;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tv_nexus = tpg->tpg_nexus;
+	if (!tv_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENODEV;
+	}
+	ret = snprintf(page, PAGE_SIZE, "%s\n",
+			tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return ret;
+}
+
+static ssize_t scsiback_tpg_store_nexus(struct se_portal_group *se_tpg,
+					 const char *page,
+					 size_t count)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport_wwn = tpg->tport;
+	unsigned char i_port[VSCSI_NAMELEN], *ptr, *port_ptr;
+	int ret;
+	/*
+	 * Shutdown the active I_T nexus if 'NULL' is passed..
+	 */
+	if (!strncmp(page, "NULL", 4)) {
+		ret = scsiback_drop_nexus(tpg);
+		return (!ret) ? count : ret;
+	}
+	/*
+	 * Otherwise make sure the passed virtual Initiator port WWN matches
+	 * the fabric protocol_id set in scsiback_make_tport(), and call
+	 * scsiback_make_nexus().
+	 */
+	if (strlen(page) >= VSCSI_NAMELEN) {
+		pr_err("Emulated NAA Sas Address: %s, exceeds max: %d\n",
+			page, VSCSI_NAMELEN);
+		return -EINVAL;
+	}
+	snprintf(&i_port[0], VSCSI_NAMELEN, "%s", page);
+
+	ptr = strstr(i_port, "naa.");
+	if (ptr) {
+		if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_SAS) {
+			pr_err("Passed SAS Initiator Port %s does not match target port protoid: %s\n",
+				i_port, scsiback_dump_proto_id(tport_wwn));
+			return -EINVAL;
+		}
+		port_ptr = &i_port[0];
+		goto check_newline;
+	}
+	ptr = strstr(i_port, "fc.");
+	if (ptr) {
+		if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_FCP) {
+			pr_err("Passed FCP Initiator Port %s does not match target port protoid: %s\n",
+				i_port, scsiback_dump_proto_id(tport_wwn));
+			return -EINVAL;
+		}
+		port_ptr = &i_port[3]; /* Skip over "fc." */
+		goto check_newline;
+	}
+	ptr = strstr(i_port, "iqn.");
+	if (ptr) {
+		if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_ISCSI) {
+			pr_err("Passed iSCSI Initiator Port %s does not match target port protoid: %s\n",
+				i_port, scsiback_dump_proto_id(tport_wwn));
+			return -EINVAL;
+		}
+		port_ptr = &i_port[0];
+		goto check_newline;
+	}
+	pr_err("Unable to locate prefix for emulated Initiator Port: %s\n",
+		i_port);
+	return -EINVAL;
+	/*
+	 * Clear any trailing newline for the NAA WWN
+	 */
+check_newline:
+	if (i_port[strlen(i_port) - 1] == '\n')
+		i_port[strlen(i_port) - 1] = '\0';
+
+	ret = scsiback_make_nexus(tpg, port_ptr);
+	if (ret < 0)
+		return ret;
+
+	return count;
+}
+
+TF_TPG_BASE_ATTR(scsiback, nexus, S_IRUGO | S_IWUSR);
+
+static struct configfs_attribute *scsiback_tpg_attrs[] = {
+	&scsiback_tpg_nexus.attr,
+	NULL,
+};
+
+static ssize_t
+scsiback_wwn_show_attr_version(struct target_fabric_configfs *tf,
+				char *page)
+{
+	return sprintf(page, "xen-pvscsi fabric module %s on %s/%s on "
+		UTS_RELEASE"\n",
+		VSCSI_VERSION, utsname()->sysname, utsname()->machine);
+}
+
+TF_WWN_ATTR_RO(scsiback, version);
+
+static struct configfs_attribute *scsiback_wwn_attrs[] = {
+	&scsiback_wwn_version.attr,
+	NULL,
+};
+
+static char *scsiback_get_fabric_name(void)
+{
+	return "xen-pvscsi";
+}
+
+static int scsiback_port_link(struct se_portal_group *se_tpg,
+			       struct se_lun *lun)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_port_count++;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return 0;
+}
+
+static void scsiback_port_unlink(struct se_portal_group *se_tpg,
+				  struct se_lun *lun)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_port_count--;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+}
+
+static struct se_portal_group *
+scsiback_make_tpg(struct se_wwn *wwn,
+		   struct config_group *group,
+		   const char *name)
+{
+	struct scsiback_tport *tport = container_of(wwn,
+			struct scsiback_tport, tport_wwn);
+
+	struct scsiback_tpg *tpg;
+	unsigned long tpgt;
+	int ret;
+
+	if (strstr(name, "tpgt_") != name)
+		return ERR_PTR(-EINVAL);
+	if (kstrtoul(name + 5, 10, &tpgt) || tpgt > UINT_MAX)
+		return ERR_PTR(-EINVAL);
+
+	tpg = kzalloc(sizeof(struct scsiback_tpg), GFP_KERNEL);
+	if (!tpg)
+		return ERR_PTR(-ENOMEM);
+
+	mutex_init(&tpg->tv_tpg_mutex);
+	INIT_LIST_HEAD(&tpg->tv_tpg_list);
+	INIT_LIST_HEAD(&tpg->info_list);
+	tpg->tport = tport;
+	tpg->tport_tpgt = tpgt;
+
+	ret = core_tpg_register(&scsiback_fabric_configfs->tf_ops, wwn,
+				&tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
+	if (ret < 0) {
+		kfree(tpg);
+		return NULL;
+	}
+	mutex_lock(&scsiback_mutex);
+	list_add_tail(&tpg->tv_tpg_list, &scsiback_list);
+	mutex_unlock(&scsiback_mutex);
+
+	return &tpg->se_tpg;
+}
+
+static void scsiback_drop_tpg(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+
+	mutex_lock(&scsiback_mutex);
+	list_del(&tpg->tv_tpg_list);
+	mutex_unlock(&scsiback_mutex);
+	/*
+	 * Release the virtual I_T Nexus for this xen-pvscsi TPG
+	 */
+	scsiback_drop_nexus(tpg);
+	/*
+	 * Deregister the se_tpg from TCM..
+	 */
+	core_tpg_deregister(se_tpg);
+	kfree(tpg);
+}
+
+static int scsiback_check_true(struct se_portal_group *se_tpg)
+{
+	return 1;
+}
+
+static int scsiback_check_false(struct se_portal_group *se_tpg)
+{
+	return 0;
+}
+
+static struct target_core_fabric_ops scsiback_ops = {
+	.get_fabric_name		= scsiback_get_fabric_name,
+	.get_fabric_proto_ident		= scsiback_get_fabric_proto_ident,
+	.tpg_get_wwn			= scsiback_get_fabric_wwn,
+	.tpg_get_tag			= scsiback_get_tag,
+	.tpg_get_default_depth		= scsiback_get_default_depth,
+	.tpg_get_pr_transport_id	= scsiback_get_pr_transport_id,
+	.tpg_get_pr_transport_id_len	= scsiback_get_pr_transport_id_len,
+	.tpg_parse_pr_out_transport_id	= scsiback_parse_pr_out_transport_id,
+	.tpg_check_demo_mode		= scsiback_check_true,
+	.tpg_check_demo_mode_cache	= scsiback_check_true,
+	.tpg_check_demo_mode_write_protect = scsiback_check_false,
+	.tpg_check_prod_mode_write_protect = scsiback_check_false,
+	.tpg_alloc_fabric_acl		= scsiback_alloc_fabric_acl,
+	.tpg_release_fabric_acl		= scsiback_release_fabric_acl,
+	.tpg_get_inst_index		= scsiback_tpg_get_inst_index,
+	.check_stop_free		= scsiback_check_stop_free,
+	.release_cmd			= scsiback_release_cmd,
+	.put_session			= NULL,
+	.shutdown_session		= scsiback_shutdown_session,
+	.close_session			= scsiback_close_session,
+	.sess_get_index			= scsiback_sess_get_index,
+	.sess_get_initiator_sid		= NULL,
+	.write_pending			= scsiback_write_pending,
+	.write_pending_status		= scsiback_write_pending_status,
+	.set_default_node_attributes	= scsiback_set_default_node_attrs,
+	.get_task_tag			= scsiback_get_task_tag,
+	.get_cmd_state			= scsiback_get_cmd_state,
+	.queue_data_in			= scsiback_queue_data_in,
+	.queue_status			= scsiback_queue_status,
+	.queue_tm_rsp			= scsiback_queue_tm_rsp,
+	.aborted_task			= scsiback_aborted_task,
+	/*
+	 * Setup callers for generic logic in target_core_fabric_configfs.c
+	 */
+	.fabric_make_wwn		= scsiback_make_tport,
+	.fabric_drop_wwn		= scsiback_drop_tport,
+	.fabric_make_tpg		= scsiback_make_tpg,
+	.fabric_drop_tpg		= scsiback_drop_tpg,
+	.fabric_post_link		= scsiback_port_link,
+	.fabric_pre_unlink		= scsiback_port_unlink,
+	.fabric_make_np			= NULL,
+	.fabric_drop_np			= NULL,
+#if 0
+	.fabric_make_nodeacl		= scsiback_make_nodeacl,
+	.fabric_drop_nodeacl		= scsiback_drop_nodeacl,
+#endif
+};
+
+static int scsiback_register_configfs(void)
+{
+	struct target_fabric_configfs *fabric;
+	int ret;
+
+	pr_debug("xen-pvscsi: fabric module %s on %s/%s on "UTS_RELEASE"\n",
+		 VSCSI_VERSION, utsname()->sysname, utsname()->machine);
+	/*
+	 * Register the top level struct config_item_type with TCM core
+	 */
+	fabric = target_fabric_configfs_init(THIS_MODULE, "xen-pvscsi");
+	if (IS_ERR(fabric))
+		return PTR_ERR(fabric);
+
+	/*
+	 * Setup fabric->tf_ops from our local scsiback_ops
+	 */
+	fabric->tf_ops = scsiback_ops;
+	/*
+	 * Setup default attribute lists for various fabric->tf_cit_tmpl
+	 */
+	fabric->tf_cit_tmpl.tfc_wwn_cit.ct_attrs = scsiback_wwn_attrs;
+	fabric->tf_cit_tmpl.tfc_tpg_base_cit.ct_attrs = scsiback_tpg_attrs;
+	fabric->tf_cit_tmpl.tfc_tpg_attrib_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_param_cit.ct_attrs = scsiback_param_attrs;
+	fabric->tf_cit_tmpl.tfc_tpg_np_base_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_base_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_attrib_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_auth_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_param_cit.ct_attrs = NULL;
+	/*
+	 * Register the fabric for use within TCM
+	 */
+	ret = target_fabric_configfs_register(fabric);
+	if (ret < 0) {
+		target_fabric_configfs_free(fabric);
+		return ret;
+	}
+	/*
+	 * Setup our local pointer to *fabric
+	 */
+	scsiback_fabric_configfs = fabric;
+	pr_debug("xen-pvscsi: Set fabric -> scsiback_fabric_configfs\n");
+	return 0;
+};
+
+static void scsiback_deregister_configfs(void)
+{
+	if (!scsiback_fabric_configfs)
+		return;
+
+	target_fabric_configfs_deregister(scsiback_fabric_configfs);
+	scsiback_fabric_configfs = NULL;
+	pr_debug("xen-pvscsi: Cleared scsiback_fabric_configfs\n");
+};
+
+static const struct xenbus_device_id scsiback_ids[] = {
+	{ "vscsi" },
+	{ "" }
+};
+
+static DEFINE_XENBUS_DRIVER(scsiback, ,
+	.probe			= scsiback_probe,
+	.remove			= scsiback_remove,
+	.otherend_changed	= scsiback_frontend_changed
+);
+
+static void scsiback_init_pend(void *p)
+{
+	struct vscsibk_pend *pend = p;
+	int i;
+
+	memset(pend, 0, sizeof(*pend));
+	for (i = 0; i < VSCSI_MAX_GRANTS; i++)
+		pend->grant_handles[i] = SCSIBACK_INVALID_HANDLE;
+}
+
+static int __init scsiback_init(void)
+{
+	int ret;
+
+	if (!xen_domain())
+		return -ENODEV;
+
+	scsiback_cachep = kmem_cache_create("vscsiif_cache",
+		sizeof(struct vscsibk_pend), 0, 0, scsiback_init_pend);
+	if (!scsiback_cachep)
+		return -ENOMEM;
+
+	ret = xenbus_register_backend(&scsiback_driver);
+	if (ret)
+		goto out_cache_destroy;
+
+	ret = scsiback_register_configfs();
+	if (ret)
+		goto out_unregister_xenbus;
+
+	return 0;
+
+out_unregister_xenbus:
+	xenbus_unregister_driver(&scsiback_driver);
+out_cache_destroy:
+	kmem_cache_destroy(scsiback_cachep);
+	pr_err("xen-pvscsi: %s: error %d\n", __func__, ret);
+	return ret;
+}
+
+static void __exit scsiback_exit(void)
+{
+	struct page *page;
+
+	while (free_pages_num) {
+		if (get_free_page(&page))
+			BUG();
+		free_xenballooned_pages(1, &page);
+	}
+	scsiback_deregister_configfs();
+	xenbus_unregister_driver(&scsiback_driver);
+	kmem_cache_destroy(scsiback_cachep);
+}
+
+module_init(scsiback_init);
+module_exit(scsiback_exit);
+
+MODULE_DESCRIPTION("Xen SCSI backend driver");
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_ALIAS("xen-backend:vscsi");
-- 
1.8.4.5

[PATCH V5 4/5] Introduce XEN scsiback module

[jgross]

From: Juergen Gross <jgross@suse.com>

Introduces the XEN pvSCSI backend. With pvSCSI it is possible for a XEN domU
to issue SCSI commands to a SCSI LUN assigned to that domU. The SCSI commands
are passed to the pvSCSI backend in a driver domain (usually Dom0) which is
owner of the physical device. This allows e.g. to use SCSI tape drives in a
XEN domU.

The code is taken from the pvSCSI implementation in XEN done by Fujitsu based
on Linux kernel 2.6.18.

Changes from the original version are:
- port to upstream kernel
- put all code in just one source file
- adapt to Linux style guide
- use target core infrastructure instead doing pure pass-through
- enable module unloading
- support SG-list in grant page(s)
- support task abort
- remove redundant struct backend
- allocate resources dynamically
- correct minor error in scsiback_fast_flush_area
- free allocated resources in case of error during I/O preparation
- remove CDB emulation, now handled by target core infrastructure

Signed-off-by: Juergen Gross <jgross@suse.com>

Xen related parts
Acked-by: David Vrabel <david.vrabel@citrix.com>
---
 drivers/xen/Kconfig        |    9 +
 drivers/xen/Makefile       |    1 +
 drivers/xen/xen-scsiback.c | 2124 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 2134 insertions(+)
 create mode 100644 drivers/xen/xen-scsiback.c

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 8bc0183..b812462 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -172,6 +172,15 @@ config XEN_PCIDEV_BACKEND
 
 	  If in doubt, say m.
 
+config XEN_SCSI_BACKEND
+	tristate "XEN SCSI backend driver"
+	depends on XEN && XEN_BACKEND && TARGET_CORE
+	help
+	  The SCSI backend driver allows the kernel to export its SCSI Devices
+	  to other guests via a high-performance shared-memory interface.
+	  Only needed for systems running as XEN driver domains (e.g. Dom0) and
+	  if guests need generic access to SCSI devices.
+
 config XEN_PRIVCMD
 	tristate
 	depends on XEN
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 84044b5..2140398 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -36,6 +36,7 @@ obj-$(CONFIG_XEN_ACPI_HOTPLUG_MEMORY)	+= xen-acpi-memhotplug.o
 obj-$(CONFIG_XEN_ACPI_HOTPLUG_CPU)	+= xen-acpi-cpuhotplug.o
 obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
 obj-$(CONFIG_XEN_EFI)			+= efi.o
+obj-$(CONFIG_XEN_SCSI_BACKEND)		+= xen-scsiback.o
 xen-evtchn-y				:= evtchn.o
 xen-gntdev-y				:= gntdev.o
 xen-gntalloc-y				:= gntalloc.o
diff --git a/drivers/xen/xen-scsiback.c b/drivers/xen/xen-scsiback.c
new file mode 100644
index 0000000..ed7fc6f
--- /dev/null
+++ b/drivers/xen/xen-scsiback.c
@@ -0,0 +1,2124 @@
+/*
+ * Xen SCSI backend driver
+ *
+ * Copyright (c) 2008, FUJITSU Limited
+ *
+ * Based on the blkback driver code.
+ * Adaption to kernel taget core infrastructure taken from vhost/scsi.c
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include <stdarg.h>
+
+#include <linux/module.h>
+#include <linux/utsname.h>
+#include <linux/interrupt.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+#include <linux/sched.h>
+#include <linux/list.h>
+#include <linux/gfp.h>
+#include <linux/delay.h>
+#include <linux/spinlock.h>
+#include <linux/configfs.h>
+
+#include <generated/utsrelease.h>
+
+#include <scsi/scsi_dbg.h>
+#include <scsi/scsi_eh.h>
+#include <scsi/scsi_tcq.h>
+
+#include <target/target_core_base.h>
+#include <target/target_core_fabric.h>
+#include <target/target_core_configfs.h>
+#include <target/target_core_fabric_configfs.h>
+
+#include <asm/hypervisor.h>
+
+#include <xen/xen.h>
+#include <xen/balloon.h>
+#include <xen/events.h>
+#include <xen/xenbus.h>
+#include <xen/grant_table.h>
+#include <xen/page.h>
+
+#include <xen/interface/grant_table.h>
+#include <xen/interface/io/vscsiif.h>
+
+#define DPRINTK(_f, _a...)			\
+	pr_debug("(file=%s, line=%d) " _f, __FILE__ , __LINE__ , ## _a)
+
+#define VSCSI_VERSION	"v0.1"
+#define VSCSI_NAMELEN	32
+
+struct ids_tuple {
+	unsigned int hst;		/* host    */
+	unsigned int chn;		/* channel */
+	unsigned int tgt;		/* target  */
+	unsigned int lun;		/* LUN     */
+};
+
+struct v2p_entry {
+	struct ids_tuple v;		/* translate from */
+	struct scsiback_tpg *tpg;	/* translate to   */
+	unsigned int lun;
+	struct kref kref;
+	struct list_head l;
+};
+
+struct vscsibk_info {
+	struct xenbus_device *dev;
+
+	domid_t domid;
+	unsigned int irq;
+
+	struct vscsiif_back_ring ring;
+	int ring_error;
+
+	spinlock_t ring_lock;
+	atomic_t nr_unreplied_reqs;
+
+	spinlock_t v2p_lock;
+	struct list_head v2p_entry_lists;
+
+	wait_queue_head_t waiting_to_free;
+};
+
+/* theoretical maximum of grants for one request */
+#define VSCSI_MAX_GRANTS	(SG_ALL + VSCSIIF_SG_TABLESIZE)
+
+/*
+ * VSCSI_GRANT_BATCH is the maximum number of grants to be processed in one
+ * call to map/unmap grants. Don't choose it too large, as there are arrays
+ * with VSCSI_GRANT_BATCH elements allocated on the stack.
+ */
+#define VSCSI_GRANT_BATCH	16
+
+struct vscsibk_pend {
+	uint16_t rqid;
+
+	uint8_t cmnd[VSCSIIF_MAX_COMMAND_SIZE];
+	uint8_t cmd_len;
+
+	uint8_t sc_data_direction;
+	uint16_t n_sg;		/* real length of SG list */
+	uint16_t n_grants;	/* SG pages and potentially SG list */
+	uint32_t data_len;
+	uint32_t result;
+
+	struct vscsibk_info *info;
+	struct v2p_entry *v2p;
+	struct scatterlist *sgl;
+
+	uint8_t sense_buffer[VSCSIIF_SENSE_BUFFERSIZE];
+
+	grant_handle_t grant_handles[VSCSI_MAX_GRANTS];
+	struct page *pages[VSCSI_MAX_GRANTS];
+
+	struct se_cmd se_cmd;
+};
+
+struct scsiback_tmr {
+	atomic_t tmr_complete;
+	wait_queue_head_t tmr_wait;
+};
+
+struct scsiback_nexus {
+	/* Pointer to TCM session for I_T Nexus */
+	struct se_session *tvn_se_sess;
+};
+
+struct scsiback_tport {
+	/* SCSI protocol the tport is providing */
+	u8 tport_proto_id;
+	/* Binary World Wide unique Port Name for pvscsi Target port */
+	u64 tport_wwpn;
+	/* ASCII formatted WWPN for pvscsi Target port */
+	char tport_name[VSCSI_NAMELEN];
+	/* Returned by scsiback_make_tport() */
+	struct se_wwn tport_wwn;
+};
+
+struct scsiback_tpg {
+	/* scsiback port target portal group tag for TCM */
+	u16 tport_tpgt;
+	/* track number of TPG Port/Lun Links wrt explicit I_T Nexus shutdown */
+	int tv_tpg_port_count;
+	/* xen-pvscsi references to tpg_nexus, protected by tv_tpg_mutex */
+	int tv_tpg_fe_count;
+	/* list for scsiback_list */
+	struct list_head tv_tpg_list;
+	/* Used to protect access for tpg_nexus */
+	struct mutex tv_tpg_mutex;
+	/* Pointer to the TCM pvscsi I_T Nexus for this TPG endpoint */
+	struct scsiback_nexus *tpg_nexus;
+	/* Pointer back to scsiback_tport */
+	struct scsiback_tport *tport;
+	/* Returned by scsiback_make_tpg() */
+	struct se_portal_group se_tpg;
+	/* alias used in xenstore */
+	char param_alias[VSCSI_NAMELEN];
+	/* list of info structures related to this target portal group */
+	struct list_head info_list;
+};
+
+#define SCSIBACK_INVALID_HANDLE (~0)
+
+static bool log_print_stat;
+module_param(log_print_stat, bool, 0644);
+
+static int scsiback_max_buffer_pages = 1024;
+module_param_named(max_buffer_pages, scsiback_max_buffer_pages, int, 0644);
+MODULE_PARM_DESC(max_buffer_pages,
+"Maximum number of free pages to keep in backend buffer");
+
+static struct kmem_cache *scsiback_cachep;
+static DEFINE_SPINLOCK(free_pages_lock);
+static int free_pages_num;
+static LIST_HEAD(scsiback_free_pages);
+
+/* Global spinlock to protect scsiback TPG list */
+static DEFINE_MUTEX(scsiback_mutex);
+static LIST_HEAD(scsiback_list);
+
+/* Local pointer to allocated TCM configfs fabric module */
+static struct target_fabric_configfs *scsiback_fabric_configfs;
+
+static void scsiback_get(struct vscsibk_info *info)
+{
+	atomic_inc(&info->nr_unreplied_reqs);
+}
+
+static void scsiback_put(struct vscsibk_info *info)
+{
+	if (atomic_dec_and_test(&info->nr_unreplied_reqs))
+		wake_up(&info->waiting_to_free);
+}
+
+static void put_free_pages(struct page **page, int num)
+{
+	unsigned long flags;
+	int i = free_pages_num + num, n = num;
+
+	if (num == 0)
+		return;
+	if (i > scsiback_max_buffer_pages) {
+		n = min(num, i - scsiback_max_buffer_pages);
+		free_xenballooned_pages(n, page + num - n);
+		n = num - n;
+	}
+	spin_lock_irqsave(&free_pages_lock, flags);
+	for (i = 0; i < n; i++)
+		list_add(&page[i]->lru, &scsiback_free_pages);
+	free_pages_num += n;
+	spin_unlock_irqrestore(&free_pages_lock, flags);
+}
+
+static int get_free_page(struct page **page)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&free_pages_lock, flags);
+	if (list_empty(&scsiback_free_pages)) {
+		spin_unlock_irqrestore(&free_pages_lock, flags);
+		return alloc_xenballooned_pages(1, page, false);
+	}
+	page[0] = list_first_entry(&scsiback_free_pages, struct page, lru);
+	list_del(&page[0]->lru);
+	free_pages_num--;
+	spin_unlock_irqrestore(&free_pages_lock, flags);
+	return 0;
+}
+
+static unsigned long vaddr_page(struct page *page)
+{
+	unsigned long pfn = page_to_pfn(page);
+
+	return (unsigned long)pfn_to_kaddr(pfn);
+}
+
+static unsigned long vaddr(struct vscsibk_pend *req, int seg)
+{
+	return vaddr_page(req->pages[seg]);
+}
+
+static void scsiback_print_status(char *sense_buffer, int errors,
+					struct vscsibk_pend *pending_req)
+{
+	struct scsiback_tpg *tpg = pending_req->v2p->tpg;
+
+	pr_err("xen-pvscsi[%s:%d] cmnd[0]=%02x -> st=%02x msg=%02x host=%02x drv=%02x\n",
+	       tpg->tport->tport_name, pending_req->v2p->lun,
+	       pending_req->cmnd[0], status_byte(errors), msg_byte(errors),
+	       host_byte(errors), driver_byte(errors));
+
+	if (CHECK_CONDITION & status_byte(errors))
+		__scsi_print_sense("xen-pvscsi", sense_buffer,
+				   SCSI_SENSE_BUFFERSIZE);
+}
+
+static void scsiback_fast_flush_area(struct vscsibk_pend *req)
+{
+	struct gnttab_unmap_grant_ref unmap[VSCSI_GRANT_BATCH];
+	struct page *pages[VSCSI_GRANT_BATCH];
+	unsigned int i, invcount = 0;
+	grant_handle_t handle;
+	int err;
+
+	kfree(req->sgl);
+	req->sgl = NULL;
+	req->n_sg = 0;
+
+	if (!req->n_grants)
+		return;
+
+	for (i = 0; i < req->n_grants; i++) {
+		handle = req->grant_handles[i];
+		if (handle == SCSIBACK_INVALID_HANDLE)
+			continue;
+		gnttab_set_unmap_op(&unmap[invcount], vaddr(req, i),
+				    GNTMAP_host_map, handle);
+		req->grant_handles[i] = SCSIBACK_INVALID_HANDLE;
+		pages[invcount] = req->pages[i];
+		put_page(pages[invcount]);
+		invcount++;
+		if (invcount < VSCSI_GRANT_BATCH)
+			continue;
+		err = gnttab_unmap_refs(unmap, NULL, pages, invcount);
+		BUG_ON(err);
+		invcount = 0;
+	}
+
+	if (invcount) {
+		err = gnttab_unmap_refs(unmap, NULL, pages, invcount);
+		BUG_ON(err);
+	}
+
+	put_free_pages(req->pages, req->n_grants);
+	req->n_grants = 0;
+}
+
+static void scsiback_free_translation_entry(struct kref *kref)
+{
+	struct v2p_entry *entry = container_of(kref, struct v2p_entry, kref);
+	struct scsiback_tpg *tpg = entry->tpg;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_fe_count--;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	kfree(entry);
+}
+
+static void scsiback_do_resp_with_sense(char *sense_buffer, int32_t result,
+			uint32_t resid, struct vscsibk_pend *pending_req)
+{
+	struct vscsiif_response *ring_res;
+	struct vscsibk_info *info = pending_req->info;
+	int notify;
+	struct scsi_sense_hdr sshdr;
+	unsigned long flags;
+	unsigned len;
+
+	spin_lock_irqsave(&info->ring_lock, flags);
+
+	ring_res = RING_GET_RESPONSE(&info->ring, info->ring.rsp_prod_pvt);
+	info->ring.rsp_prod_pvt++;
+
+	ring_res->rslt   = result;
+	ring_res->rqid   = pending_req->rqid;
+
+	if (sense_buffer != NULL &&
+	    scsi_normalize_sense(sense_buffer, VSCSIIF_SENSE_BUFFERSIZE,
+				 &sshdr)) {
+		len = min_t(unsigned, 8 + sense_buffer[7],
+			    VSCSIIF_SENSE_BUFFERSIZE);
+		memcpy(ring_res->sense_buffer, sense_buffer, len);
+		ring_res->sense_len = len;
+	} else {
+		ring_res->sense_len = 0;
+	}
+
+	ring_res->residual_len = resid;
+
+	RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&info->ring, notify);
+	spin_unlock_irqrestore(&info->ring_lock, flags);
+
+	if (notify)
+		notify_remote_via_irq(info->irq);
+
+	if (pending_req->v2p)
+		kref_put(&pending_req->v2p->kref,
+			 scsiback_free_translation_entry);
+}
+
+static void scsiback_cmd_done(struct vscsibk_pend *pending_req)
+{
+	struct vscsibk_info *info = pending_req->info;
+	unsigned char *sense_buffer;
+	unsigned int resid;
+	int errors;
+
+	sense_buffer = pending_req->sense_buffer;
+	resid        = pending_req->se_cmd.residual_count;
+	errors       = pending_req->result;
+
+	if (errors && log_print_stat)
+		scsiback_print_status(sense_buffer, errors, pending_req);
+
+	scsiback_fast_flush_area(pending_req);
+	scsiback_do_resp_with_sense(sense_buffer, errors, resid, pending_req);
+	scsiback_put(info);
+}
+
+static void scsiback_cmd_exec(struct vscsibk_pend *pending_req)
+{
+	struct se_cmd *se_cmd = &pending_req->se_cmd;
+	struct se_session *sess = pending_req->v2p->tpg->tpg_nexus->tvn_se_sess;
+	int rc;
+
+	memset(pending_req->sense_buffer, 0, VSCSIIF_SENSE_BUFFERSIZE);
+
+	memset(se_cmd, 0, sizeof(*se_cmd));
+
+	scsiback_get(pending_req->info);
+	rc = target_submit_cmd_map_sgls(se_cmd, sess, pending_req->cmnd,
+			pending_req->sense_buffer, pending_req->v2p->lun,
+			pending_req->data_len, 0,
+			pending_req->sc_data_direction, 0,
+			pending_req->sgl, pending_req->n_sg,
+			NULL, 0, NULL, 0);
+	if (rc < 0) {
+		transport_send_check_condition_and_sense(se_cmd,
+				TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE, 0);
+		transport_generic_free_cmd(se_cmd, 0);
+	}
+}
+
+static int scsiback_gnttab_data_map_batch(struct gnttab_map_grant_ref *map,
+	struct page **pg, grant_handle_t *grant, int cnt)
+{
+	int err, i;
+
+	if (!cnt)
+		return 0;
+
+	err = gnttab_map_refs(map, NULL, pg, cnt);
+	BUG_ON(err);
+	for (i = 0; i < cnt; i++) {
+		if (unlikely(map[i].status != GNTST_okay)) {
+			pr_err("xen-pvscsi: invalid buffer -- could not remap it\n");
+			map[i].handle = SCSIBACK_INVALID_HANDLE;
+			err = -ENOMEM;
+		} else {
+			get_page(pg[i]);
+		}
+		grant[i] = map[i].handle;
+	}
+	return err;
+}
+
+static int scsiback_gnttab_data_map_list(struct vscsibk_pend *pending_req,
+			struct scsiif_request_segment *seg, struct page **pg,
+			grant_handle_t *grant, int cnt, u32 flags)
+{
+	int mapcount = 0, i, err = 0;
+	struct gnttab_map_grant_ref map[VSCSI_GRANT_BATCH];
+	struct vscsibk_info *info = pending_req->info;
+
+	for (i = 0; i < cnt; i++) {
+		if (get_free_page(pg + mapcount)) {
+			put_free_pages(pg, mapcount);
+			pr_err("xen-pvscsi: no grant page\n");
+			return -ENOMEM;
+		}
+		gnttab_set_map_op(&map[mapcount], vaddr_page(pg[mapcount]),
+				  flags, seg[i].gref, info->domid);
+		mapcount++;
+		if (mapcount < VSCSI_GRANT_BATCH)
+			continue;
+		err = scsiback_gnttab_data_map_batch(map, pg, grant, mapcount);
+		pg += mapcount;
+		grant += mapcount;
+		pending_req->n_grants += mapcount;
+		if (err)
+			return err;
+		mapcount = 0;
+	}
+	err = scsiback_gnttab_data_map_batch(map, pg, grant, mapcount);
+	pending_req->n_grants += mapcount;
+	return err;
+}
+
+static int scsiback_gnttab_data_map(struct vscsiif_request *ring_req,
+					struct vscsibk_pend *pending_req)
+{
+	u32 flags;
+	int i, err, n_segs, i_seg = 0;
+	struct page **pg;
+	struct scsiif_request_segment *seg;
+	unsigned long end_seg = 0;
+	unsigned int nr_segments = (unsigned int)ring_req->nr_segments;
+	unsigned int nr_sgl = 0;
+	struct scatterlist *sg;
+	grant_handle_t *grant;
+
+	pending_req->n_sg = 0;
+	pending_req->n_grants = 0;
+	pending_req->data_len = 0;
+
+	nr_segments &= ~VSCSIIF_SG_GRANT;
+	if (!nr_segments)
+		return 0;
+
+	if (nr_segments > VSCSIIF_SG_TABLESIZE) {
+		DPRINTK("xen-pvscsi: invalid parameter nr_seg = %d\n",
+			ring_req->nr_segments);
+		return -EINVAL;
+	}
+
+	if (ring_req->nr_segments & VSCSIIF_SG_GRANT) {
+		err = scsiback_gnttab_data_map_list(pending_req, ring_req->seg,
+			pending_req->pages, pending_req->grant_handles,
+			nr_segments, GNTMAP_host_map | GNTMAP_readonly);
+		if (err)
+			return err;
+		nr_sgl = nr_segments;
+		nr_segments = 0;
+		for (i = 0; i < nr_sgl; i++) {
+			n_segs = ring_req->seg[i].length /
+				 sizeof(struct scsiif_request_segment);
+			if ((unsigned)ring_req->seg[i].offset +
+			    (unsigned)ring_req->seg[i].length > PAGE_SIZE ||
+			    n_segs * sizeof(struct scsiif_request_segment) !=
+			    ring_req->seg[i].length)
+				return -EINVAL;
+			nr_segments += n_segs;
+		}
+		if (nr_segments > SG_ALL) {
+			DPRINTK("xen-pvscsi: invalid nr_seg = %d\n",
+				nr_segments);
+			return -EINVAL;
+		}
+	}
+
+	/* free of (sgl) in fast_flush_area()*/
+	pending_req->sgl = kmalloc_array(nr_segments,
+					sizeof(struct scatterlist), GFP_KERNEL);
+	if (!pending_req->sgl)
+		return -ENOMEM;
+
+	sg_init_table(pending_req->sgl, nr_segments);
+	pending_req->n_sg = nr_segments;
+
+	flags = GNTMAP_host_map;
+	if (pending_req->sc_data_direction == DMA_TO_DEVICE)
+		flags |= GNTMAP_readonly;
+
+	pg = pending_req->pages + nr_sgl;
+	grant = pending_req->grant_handles + nr_sgl;
+	if (!nr_sgl) {
+		seg = ring_req->seg;
+		err = scsiback_gnttab_data_map_list(pending_req, seg,
+			pg, grant, nr_segments, flags);
+		if (err)
+			return err;
+	} else {
+		for (i = 0; i < nr_sgl; i++) {
+			seg = (struct scsiif_request_segment *)(
+			      vaddr(pending_req, i) + ring_req->seg[i].offset);
+			n_segs = ring_req->seg[i].length /
+				 sizeof(struct scsiif_request_segment);
+			err = scsiback_gnttab_data_map_list(pending_req, seg,
+				pg, grant, n_segs, flags);
+			if (err)
+				return err;
+			pg += n_segs;
+			grant += n_segs;
+		}
+		end_seg = vaddr(pending_req, 0) + ring_req->seg[0].offset;
+		seg = (struct scsiif_request_segment *)end_seg;
+		end_seg += ring_req->seg[0].length;
+		pg = pending_req->pages + nr_sgl;
+	}
+
+	for_each_sg(pending_req->sgl, sg, nr_segments, i) {
+		sg_set_page(sg, pg[i], seg->length, seg->offset);
+		pending_req->data_len += seg->length;
+		seg++;
+		if (nr_sgl && (unsigned long)seg >= end_seg) {
+			i_seg++;
+			end_seg = vaddr(pending_req, i_seg) +
+				  ring_req->seg[i_seg].offset;
+			seg = (struct scsiif_request_segment *)end_seg;
+			end_seg += ring_req->seg[i_seg].length;
+		}
+		if (sg->offset >= PAGE_SIZE ||
+		    sg->length > PAGE_SIZE ||
+		    sg->offset + sg->length > PAGE_SIZE)
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void scsiback_disconnect(struct vscsibk_info *info)
+{
+	wait_event(info->waiting_to_free,
+		atomic_read(&info->nr_unreplied_reqs) == 0);
+
+	unbind_from_irqhandler(info->irq, info);
+	info->irq = 0;
+	xenbus_unmap_ring_vfree(info->dev, info->ring.sring);
+}
+
+static void scsiback_device_action(struct vscsibk_pend *pending_req,
+	enum tcm_tmreq_table act, int tag)
+{
+	int rc, err = FAILED;
+	struct scsiback_tpg *tpg = pending_req->v2p->tpg;
+	struct se_cmd *se_cmd = &pending_req->se_cmd;
+	struct scsiback_tmr *tmr;
+
+	tmr = kzalloc(sizeof(struct scsiback_tmr), GFP_KERNEL);
+	if (!tmr)
+		goto out;
+
+	init_waitqueue_head(&tmr->tmr_wait);
+
+	transport_init_se_cmd(se_cmd, tpg->se_tpg.se_tpg_tfo,
+		tpg->tpg_nexus->tvn_se_sess, 0, DMA_NONE, MSG_SIMPLE_TAG,
+		&pending_req->sense_buffer[0]);
+
+	rc = core_tmr_alloc_req(se_cmd, tmr, act, GFP_KERNEL);
+	if (rc < 0)
+		goto out;
+
+	se_cmd->se_tmr_req->ref_task_tag = tag;
+
+	if (transport_lookup_tmr_lun(se_cmd, pending_req->v2p->lun) < 0)
+		goto out;
+
+	transport_generic_handle_tmr(se_cmd);
+	wait_event(tmr->tmr_wait, atomic_read(&tmr->tmr_complete));
+
+	err = (se_cmd->se_tmr_req->response == TMR_FUNCTION_COMPLETE) ?
+		SUCCESS : FAILED;
+
+out:
+	if (tmr) {
+		transport_generic_free_cmd(&pending_req->se_cmd, 1);
+		kfree(tmr);
+	}
+
+	scsiback_do_resp_with_sense(NULL, err, 0, pending_req);
+
+	kmem_cache_free(scsiback_cachep, pending_req);
+}
+
+/*
+  Perform virtual to physical translation
+*/
+static struct v2p_entry *scsiback_do_translation(struct vscsibk_info *info,
+			struct ids_tuple *v)
+{
+	struct v2p_entry *entry;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+	list_for_each_entry(entry, head, l) {
+		if ((entry->v.chn == v->chn) &&
+		    (entry->v.tgt == v->tgt) &&
+		    (entry->v.lun == v->lun)) {
+			kref_get(&entry->kref);
+			goto out;
+		}
+	}
+	entry = NULL;
+
+out:
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+	return entry;
+}
+
+static int prepare_pending_reqs(struct vscsibk_info *info,
+				struct vscsiif_request *ring_req,
+				struct vscsibk_pend *pending_req)
+{
+	struct v2p_entry *v2p;
+	struct ids_tuple vir;
+
+	pending_req->rqid       = ring_req->rqid;
+	pending_req->info       = info;
+
+	vir.chn = ring_req->channel;
+	vir.tgt = ring_req->id;
+	vir.lun = ring_req->lun;
+
+	v2p = scsiback_do_translation(info, &vir);
+	if (!v2p) {
+		pending_req->v2p = NULL;
+		DPRINTK("xen-pvscsi: doesn't exist.\n");
+		return -ENODEV;
+	}
+	pending_req->v2p = v2p;
+
+	/* request range check from frontend */
+	pending_req->sc_data_direction = ring_req->sc_data_direction;
+	if ((pending_req->sc_data_direction != DMA_BIDIRECTIONAL) &&
+		(pending_req->sc_data_direction != DMA_TO_DEVICE) &&
+		(pending_req->sc_data_direction != DMA_FROM_DEVICE) &&
+		(pending_req->sc_data_direction != DMA_NONE)) {
+		DPRINTK("xen-pvscsi: invalid parameter data_dir = %d\n",
+			pending_req->sc_data_direction);
+		return -EINVAL;
+	}
+
+	pending_req->cmd_len = ring_req->cmd_len;
+	if (pending_req->cmd_len > VSCSIIF_MAX_COMMAND_SIZE) {
+		DPRINTK("xen-pvscsi: invalid parameter cmd_len = %d\n",
+			pending_req->cmd_len);
+		return -EINVAL;
+	}
+	memcpy(pending_req->cmnd, ring_req->cmnd, pending_req->cmd_len);
+
+	return 0;
+}
+
+static int scsiback_do_cmd_fn(struct vscsibk_info *info)
+{
+	struct vscsiif_back_ring *ring = &info->ring;
+	struct vscsiif_request *ring_req;
+	struct vscsibk_pend *pending_req;
+	RING_IDX rc, rp;
+	int err, more_to_do;
+	uint32_t result;
+	uint8_t act;
+
+	rc = ring->req_cons;
+	rp = ring->sring->req_prod;
+	rmb();	/* guest system is accessing ring, too */
+
+	if (RING_REQUEST_PROD_OVERFLOW(ring, rp)) {
+		rc = ring->rsp_prod_pvt;
+		pr_warn("xen-pvscsi: Dom%d provided bogus ring requests (%#x - %#x = %u). Halting ring processing\n",
+			   info->domid, rp, rc, rp - rc);
+		info->ring_error = 1;
+		return 0;
+	}
+
+	while ((rc != rp)) {
+		if (RING_REQUEST_CONS_OVERFLOW(ring, rc))
+			break;
+		pending_req = kmem_cache_alloc(scsiback_cachep, GFP_KERNEL);
+		if (!pending_req)
+			return 1;
+
+		ring_req = RING_GET_REQUEST(ring, rc);
+		ring->req_cons = ++rc;
+
+		act = ring_req->act;
+		err = prepare_pending_reqs(info, ring_req, pending_req);
+		if (err) {
+			switch (err) {
+			case -ENODEV:
+				result = DID_NO_CONNECT;
+				break;
+			default:
+				result = DRIVER_ERROR;
+				break;
+			}
+			scsiback_do_resp_with_sense(NULL, result << 24, 0,
+						    pending_req);
+			kmem_cache_free(scsiback_cachep, pending_req);
+			return 1;
+		}
+
+		switch (act) {
+		case VSCSIIF_ACT_SCSI_CDB:
+			if (scsiback_gnttab_data_map(ring_req, pending_req)) {
+				scsiback_fast_flush_area(pending_req);
+				scsiback_do_resp_with_sense(NULL,
+					DRIVER_ERROR << 24, 0, pending_req);
+				kmem_cache_free(scsiback_cachep, pending_req);
+			} else {
+				scsiback_cmd_exec(pending_req);
+			}
+			break;
+		case VSCSIIF_ACT_SCSI_ABORT:
+			scsiback_device_action(pending_req, TMR_ABORT_TASK,
+				ring_req->ref_rqid);
+			break;
+		case VSCSIIF_ACT_SCSI_RESET:
+			scsiback_device_action(pending_req, TMR_LUN_RESET, 0);
+			break;
+		default:
+			pr_err_ratelimited("xen-pvscsi: invalid request\n");
+			scsiback_do_resp_with_sense(NULL, DRIVER_ERROR << 24,
+						    0, pending_req);
+			kmem_cache_free(scsiback_cachep, pending_req);
+			break;
+		}
+
+		/* Yield point for this unbounded loop. */
+		cond_resched();
+	}
+
+	RING_FINAL_CHECK_FOR_REQUESTS(&info->ring, more_to_do);
+	return more_to_do;
+}
+
+static irqreturn_t scsiback_irq_fn(int irq, void *dev_id)
+{
+	struct vscsibk_info *info = dev_id;
+
+	if (info->ring_error)
+		return IRQ_HANDLED;
+
+	while (scsiback_do_cmd_fn(info))
+		cond_resched();
+
+	return IRQ_HANDLED;
+}
+
+static int scsiback_init_sring(struct vscsibk_info *info, grant_ref_t ring_ref,
+			evtchn_port_t evtchn)
+{
+	void *area;
+	struct vscsiif_sring *sring;
+	int err;
+
+	if (info->irq)
+		return -1;
+
+	err = xenbus_map_ring_valloc(info->dev, ring_ref, &area);
+	if (err)
+		return err;
+
+	sring = (struct vscsiif_sring *)area;
+	BACK_RING_INIT(&info->ring, sring, PAGE_SIZE);
+
+	err = bind_interdomain_evtchn_to_irq(info->domid, evtchn);
+	if (err < 0)
+		goto unmap_page;
+
+	info->irq = err;
+
+	err = request_threaded_irq(info->irq, NULL, scsiback_irq_fn,
+				   IRQF_ONESHOT, "vscsiif-backend", info);
+	if (err)
+		goto free_irq;
+
+	return 0;
+
+free_irq:
+	unbind_from_irqhandler(info->irq, info);
+	info->irq = 0;
+unmap_page:
+	xenbus_unmap_ring_vfree(info->dev, area);
+
+	return err;
+}
+
+static int scsiback_map(struct vscsibk_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	unsigned int ring_ref, evtchn;
+	int err;
+
+	err = xenbus_gather(XBT_NIL, dev->otherend,
+			"ring-ref", "%u", &ring_ref,
+			"event-channel", "%u", &evtchn, NULL);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "reading %s ring", dev->otherend);
+		return err;
+	}
+
+	return scsiback_init_sring(info, ring_ref, evtchn);
+}
+
+/*
+  Add a new translation entry
+*/
+static int scsiback_add_translation_entry(struct vscsibk_info *info,
+					  char *phy, struct ids_tuple *v)
+{
+	int err = 0;
+	struct v2p_entry *entry;
+	struct v2p_entry *new;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+	char *lunp;
+	unsigned int lun;
+	struct scsiback_tpg *tpg_entry, *tpg = NULL;
+	char *error = "doesn't exist";
+
+	lunp = strrchr(phy, ':');
+	if (!lunp) {
+		pr_err("xen-pvscsi: illegal format of physical device %s\n",
+			phy);
+		return -EINVAL;
+	}
+	*lunp = 0;
+	lunp++;
+	if (kstrtouint(lunp, 10, &lun) || lun >= TRANSPORT_MAX_LUNS_PER_TPG) {
+		pr_err("xen-pvscsi: lun number not valid: %s\n", lunp);
+		return -EINVAL;
+	}
+
+	mutex_lock(&scsiback_mutex);
+	list_for_each_entry(tpg_entry, &scsiback_list, tv_tpg_list) {
+		if (!strcmp(phy, tpg_entry->tport->tport_name) ||
+		    !strcmp(phy, tpg_entry->param_alias)) {
+			spin_lock(&tpg_entry->se_tpg.tpg_lun_lock);
+			if (tpg_entry->se_tpg.tpg_lun_list[lun]->lun_status ==
+			    TRANSPORT_LUN_STATUS_ACTIVE) {
+				if (!tpg_entry->tpg_nexus)
+					error = "nexus undefined";
+				else
+					tpg = tpg_entry;
+			}
+			spin_unlock(&tpg_entry->se_tpg.tpg_lun_lock);
+			break;
+		}
+	}
+	if (tpg) {
+		mutex_lock(&tpg->tv_tpg_mutex);
+		tpg->tv_tpg_fe_count++;
+		mutex_unlock(&tpg->tv_tpg_mutex);
+	}
+	mutex_unlock(&scsiback_mutex);
+
+	if (!tpg) {
+		pr_err("xen-pvscsi: %s:%d %s\n", phy, lun, error);
+		return -ENODEV;
+	}
+
+	new = kmalloc(sizeof(struct v2p_entry), GFP_KERNEL);
+	if (new == NULL) {
+		err = -ENOMEM;
+		goto out_free;
+	}
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+
+	/* Check double assignment to identical virtual ID */
+	list_for_each_entry(entry, head, l) {
+		if ((entry->v.chn == v->chn) &&
+		    (entry->v.tgt == v->tgt) &&
+		    (entry->v.lun == v->lun)) {
+			pr_warn("xen-pvscsi: Virtual ID is already used. Assignment was not performed.\n");
+			err = -EEXIST;
+			goto out;
+		}
+
+	}
+
+	/* Create a new translation entry and add to the list */
+	kref_init(&new->kref);
+	new->v = *v;
+	new->tpg = tpg;
+	new->lun = lun;
+	list_add_tail(&new->l, head);
+
+out:
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+
+out_free:
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_fe_count--;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	if (err)
+		kfree(new);
+
+	return err;
+}
+
+static void __scsiback_del_translation_entry(struct v2p_entry *entry)
+{
+	list_del(&entry->l);
+	kref_put(&entry->kref, scsiback_free_translation_entry);
+	kfree(entry);
+}
+
+/*
+  Delete the translation entry specfied
+*/
+static int scsiback_del_translation_entry(struct vscsibk_info *info,
+					  struct ids_tuple *v)
+{
+	struct v2p_entry *entry;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+	/* Find out the translation entry specified */
+	list_for_each_entry(entry, head, l) {
+		if ((entry->v.chn == v->chn) &&
+		    (entry->v.tgt == v->tgt) &&
+		    (entry->v.lun == v->lun)) {
+			goto found;
+		}
+	}
+
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+	return 1;
+
+found:
+	/* Delete the translation entry specfied */
+	__scsiback_del_translation_entry(entry);
+
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+	return 0;
+}
+
+static void scsiback_do_add_lun(struct vscsibk_info *info, const char *state,
+				char *phy, struct ids_tuple *vir)
+{
+	if (!scsiback_add_translation_entry(info, phy, vir)) {
+		if (xenbus_printf(XBT_NIL, info->dev->nodename, state,
+				  "%d", XenbusStateInitialised)) {
+			pr_err("xen-pvscsi: xenbus_printf error %s\n", state);
+			scsiback_del_translation_entry(info, vir);
+		}
+	} else {
+		xenbus_printf(XBT_NIL, info->dev->nodename, state,
+			      "%d", XenbusStateClosed);
+	}
+}
+
+static void scsiback_do_del_lun(struct vscsibk_info *info, const char *state,
+				struct ids_tuple *vir)
+{
+	if (!scsiback_del_translation_entry(info, vir)) {
+		if (xenbus_printf(XBT_NIL, info->dev->nodename, state,
+				  "%d", XenbusStateClosed))
+			pr_err("xen-pvscsi: xenbus_printf error %s\n", state);
+	}
+}
+
+#define VSCSIBACK_OP_ADD_OR_DEL_LUN	1
+#define VSCSIBACK_OP_UPDATEDEV_STATE	2
+
+static void scsiback_do_1lun_hotplug(struct vscsibk_info *info, int op,
+				     char *ent)
+{
+	int err;
+	struct ids_tuple vir;
+	char *val;
+	int device_state;
+	char phy[VSCSI_NAMELEN];
+	char str[64];
+	char state[64];
+	struct xenbus_device *dev = info->dev;
+
+	/* read status */
+	snprintf(state, sizeof(state), "vscsi-devs/%s/state", ent);
+	err = xenbus_scanf(XBT_NIL, dev->nodename, state, "%u", &device_state);
+	if (XENBUS_EXIST_ERR(err))
+		return;
+
+	/* physical SCSI device */
+	snprintf(str, sizeof(str), "vscsi-devs/%s/p-dev", ent);
+	val = xenbus_read(XBT_NIL, dev->nodename, str, NULL);
+	if (IS_ERR(val)) {
+		xenbus_printf(XBT_NIL, dev->nodename, state,
+			      "%d", XenbusStateClosed);
+		return;
+	}
+	strlcpy(phy, val, VSCSI_NAMELEN);
+	kfree(val);
+
+	/* virtual SCSI device */
+	snprintf(str, sizeof(str), "vscsi-devs/%s/v-dev", ent);
+	err = xenbus_scanf(XBT_NIL, dev->nodename, str, "%u:%u:%u:%u",
+			   &vir.hst, &vir.chn, &vir.tgt, &vir.lun);
+	if (XENBUS_EXIST_ERR(err)) {
+		xenbus_printf(XBT_NIL, dev->nodename, state,
+			      "%d", XenbusStateClosed);
+		return;
+	}
+
+	switch (op) {
+	case VSCSIBACK_OP_ADD_OR_DEL_LUN:
+		if (device_state == XenbusStateInitialising)
+			scsiback_do_add_lun(info, state, phy, &vir);
+		if (device_state == XenbusStateClosing)
+			scsiback_do_del_lun(info, state, &vir);
+		break;
+
+	case VSCSIBACK_OP_UPDATEDEV_STATE:
+		if (device_state == XenbusStateInitialised) {
+			/* modify vscsi-devs/dev-x/state */
+			if (xenbus_printf(XBT_NIL, dev->nodename, state,
+					  "%d", XenbusStateConnected)) {
+				pr_err("xen-pvscsi: xenbus_printf error %s\n",
+				       str);
+				scsiback_del_translation_entry(info, &vir);
+				xenbus_printf(XBT_NIL, dev->nodename, state,
+					      "%d", XenbusStateClosed);
+			}
+		}
+		break;
+	/*When it is necessary, processing is added here.*/
+	default:
+		break;
+	}
+}
+
+static void scsiback_do_lun_hotplug(struct vscsibk_info *info, int op)
+{
+	int i;
+	char **dir;
+	unsigned int ndir = 0;
+
+	dir = xenbus_directory(XBT_NIL, info->dev->nodename, "vscsi-devs",
+			       &ndir);
+	if (IS_ERR(dir))
+		return;
+
+	for (i = 0; i < ndir; i++)
+		scsiback_do_1lun_hotplug(info, op, dir[i]);
+
+	kfree(dir);
+}
+
+static void scsiback_frontend_changed(struct xenbus_device *dev,
+					enum xenbus_state frontend_state)
+{
+	struct vscsibk_info *info = dev_get_drvdata(&dev->dev);
+
+	switch (frontend_state) {
+	case XenbusStateInitialising:
+		break;
+
+	case XenbusStateInitialised:
+		if (scsiback_map(info))
+			break;
+
+		scsiback_do_lun_hotplug(info, VSCSIBACK_OP_ADD_OR_DEL_LUN);
+		xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+
+	case XenbusStateConnected:
+		scsiback_do_lun_hotplug(info, VSCSIBACK_OP_UPDATEDEV_STATE);
+
+		if (dev->state == XenbusStateConnected)
+			break;
+
+		xenbus_switch_state(dev, XenbusStateConnected);
+		break;
+
+	case XenbusStateClosing:
+		if (info->irq)
+			scsiback_disconnect(info);
+
+		xenbus_switch_state(dev, XenbusStateClosing);
+		break;
+
+	case XenbusStateClosed:
+		xenbus_switch_state(dev, XenbusStateClosed);
+		if (xenbus_dev_is_online(dev))
+			break;
+		/* fall through if not online */
+	case XenbusStateUnknown:
+		device_unregister(&dev->dev);
+		break;
+
+	case XenbusStateReconfiguring:
+		scsiback_do_lun_hotplug(info, VSCSIBACK_OP_ADD_OR_DEL_LUN);
+		xenbus_switch_state(dev, XenbusStateReconfigured);
+
+		break;
+
+	default:
+		xenbus_dev_fatal(dev, -EINVAL, "saw state %d at frontend",
+					frontend_state);
+		break;
+	}
+}
+
+/*
+  Release the translation entry specfied
+*/
+static void scsiback_release_translation_entry(struct vscsibk_info *info)
+{
+	struct v2p_entry *entry, *tmp;
+	struct list_head *head = &(info->v2p_entry_lists);
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->v2p_lock, flags);
+
+	list_for_each_entry_safe(entry, tmp, head, l)
+		__scsiback_del_translation_entry(entry);
+
+	spin_unlock_irqrestore(&info->v2p_lock, flags);
+}
+
+static int scsiback_remove(struct xenbus_device *dev)
+{
+	struct vscsibk_info *info = dev_get_drvdata(&dev->dev);
+
+	if (info->irq)
+		scsiback_disconnect(info);
+
+	scsiback_release_translation_entry(info);
+
+	dev_set_drvdata(&dev->dev, NULL);
+
+	return 0;
+}
+
+static int scsiback_probe(struct xenbus_device *dev,
+			   const struct xenbus_device_id *id)
+{
+	int err;
+
+	struct vscsibk_info *info = kzalloc(sizeof(struct vscsibk_info),
+					    GFP_KERNEL);
+
+	DPRINTK("%p %d\n", dev, dev->otherend_id);
+
+	if (!info) {
+		xenbus_dev_fatal(dev, -ENOMEM, "allocating backend structure");
+		return -ENOMEM;
+	}
+	info->dev = dev;
+	dev_set_drvdata(&dev->dev, info);
+
+	info->domid = dev->otherend_id;
+	spin_lock_init(&info->ring_lock);
+	info->ring_error = 0;
+	atomic_set(&info->nr_unreplied_reqs, 0);
+	init_waitqueue_head(&info->waiting_to_free);
+	info->dev = dev;
+	info->irq = 0;
+	INIT_LIST_HEAD(&info->v2p_entry_lists);
+	spin_lock_init(&info->v2p_lock);
+
+	err = xenbus_printf(XBT_NIL, dev->nodename, "feature-sg-grant", "%u",
+			    SG_ALL);
+	if (err)
+		xenbus_dev_error(dev, err, "writing feature-sg-grant");
+
+	err = xenbus_switch_state(dev, XenbusStateInitWait);
+	if (err)
+		goto fail;
+
+	return 0;
+
+fail:
+	pr_warn("xen-pvscsi: %s failed\n", __func__);
+	scsiback_remove(dev);
+
+	return err;
+}
+
+static char *scsiback_dump_proto_id(struct scsiback_tport *tport)
+{
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return "SAS";
+	case SCSI_PROTOCOL_FCP:
+		return "FCP";
+	case SCSI_PROTOCOL_ISCSI:
+		return "iSCSI";
+	default:
+		break;
+	}
+
+	return "Unknown";
+}
+
+static u8 scsiback_get_fabric_proto_ident(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_get_fabric_proto_ident(se_tpg);
+	case SCSI_PROTOCOL_FCP:
+		return fc_get_fabric_proto_ident(se_tpg);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_get_fabric_proto_ident(se_tpg);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_get_fabric_proto_ident(se_tpg);
+}
+
+static char *scsiback_get_fabric_wwn(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	return &tport->tport_name[0];
+}
+
+static u16 scsiback_get_tag(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	return tpg->tport_tpgt;
+}
+
+static u32 scsiback_get_default_depth(struct se_portal_group *se_tpg)
+{
+	return 1;
+}
+
+static u32
+scsiback_get_pr_transport_id(struct se_portal_group *se_tpg,
+			      struct se_node_acl *se_nacl,
+			      struct t10_pr_registration *pr_reg,
+			      int *format_code,
+			      unsigned char *buf)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+					format_code, buf);
+	case SCSI_PROTOCOL_FCP:
+		return fc_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+					format_code, buf);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+					format_code, buf);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+			format_code, buf);
+}
+
+static u32
+scsiback_get_pr_transport_id_len(struct se_portal_group *se_tpg,
+				  struct se_node_acl *se_nacl,
+				  struct t10_pr_registration *pr_reg,
+				  int *format_code)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+					format_code);
+	case SCSI_PROTOCOL_FCP:
+		return fc_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+					format_code);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+					format_code);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+			format_code);
+}
+
+static char *
+scsiback_parse_pr_out_transport_id(struct se_portal_group *se_tpg,
+				    const char *buf,
+				    u32 *out_tid_len,
+				    char **port_nexus_ptr)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport = tpg->tport;
+
+	switch (tport->tport_proto_id) {
+	case SCSI_PROTOCOL_SAS:
+		return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+					port_nexus_ptr);
+	case SCSI_PROTOCOL_FCP:
+		return fc_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+					port_nexus_ptr);
+	case SCSI_PROTOCOL_ISCSI:
+		return iscsi_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+					port_nexus_ptr);
+	default:
+		pr_err("Unknown tport_proto_id: 0x%02x, using SAS emulation\n",
+			tport->tport_proto_id);
+		break;
+	}
+
+	return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+			port_nexus_ptr);
+}
+
+static struct se_wwn *
+scsiback_make_tport(struct target_fabric_configfs *tf,
+		     struct config_group *group,
+		     const char *name)
+{
+	struct scsiback_tport *tport;
+	char *ptr;
+	u64 wwpn = 0;
+	int off = 0;
+
+	tport = kzalloc(sizeof(struct scsiback_tport), GFP_KERNEL);
+	if (!tport)
+		return ERR_PTR(-ENOMEM);
+
+	tport->tport_wwpn = wwpn;
+	/*
+	 * Determine the emulated Protocol Identifier and Target Port Name
+	 * based on the incoming configfs directory name.
+	 */
+	ptr = strstr(name, "naa.");
+	if (ptr) {
+		tport->tport_proto_id = SCSI_PROTOCOL_SAS;
+		goto check_len;
+	}
+	ptr = strstr(name, "fc.");
+	if (ptr) {
+		tport->tport_proto_id = SCSI_PROTOCOL_FCP;
+		off = 3; /* Skip over "fc." */
+		goto check_len;
+	}
+	ptr = strstr(name, "iqn.");
+	if (ptr) {
+		tport->tport_proto_id = SCSI_PROTOCOL_ISCSI;
+		goto check_len;
+	}
+
+	pr_err("Unable to locate prefix for emulated Target Port: %s\n", name);
+	kfree(tport);
+	return ERR_PTR(-EINVAL);
+
+check_len:
+	if (strlen(name) >= VSCSI_NAMELEN) {
+		pr_err("Emulated %s Address: %s, exceeds max: %d\n", name,
+			scsiback_dump_proto_id(tport), VSCSI_NAMELEN);
+		kfree(tport);
+		return ERR_PTR(-EINVAL);
+	}
+	snprintf(&tport->tport_name[0], VSCSI_NAMELEN, "%s", &name[off]);
+
+	pr_debug("xen-pvscsi: Allocated emulated Target %s Address: %s\n",
+		 scsiback_dump_proto_id(tport), name);
+
+	return &tport->tport_wwn;
+}
+
+static void scsiback_drop_tport(struct se_wwn *wwn)
+{
+	struct scsiback_tport *tport = container_of(wwn,
+				struct scsiback_tport, tport_wwn);
+
+	pr_debug("xen-pvscsi: Deallocating emulated Target %s Address: %s\n",
+		 scsiback_dump_proto_id(tport), tport->tport_name);
+
+	kfree(tport);
+}
+
+static struct se_node_acl *
+scsiback_alloc_fabric_acl(struct se_portal_group *se_tpg)
+{
+	return kzalloc(sizeof(struct se_node_acl), GFP_KERNEL);
+}
+
+static void
+scsiback_release_fabric_acl(struct se_portal_group *se_tpg,
+			     struct se_node_acl *se_nacl)
+{
+	kfree(se_nacl);
+}
+
+static u32 scsiback_tpg_get_inst_index(struct se_portal_group *se_tpg)
+{
+	return 1;
+}
+
+static int scsiback_check_stop_free(struct se_cmd *se_cmd)
+{
+	/*
+	 * Do not release struct se_cmd's containing a valid TMR
+	 * pointer.  These will be released directly in scsiback_device_action()
+	 * with transport_generic_free_cmd().
+	 */
+	if (se_cmd->se_cmd_flags & SCF_SCSI_TMR_CDB)
+		return 0;
+
+	transport_generic_free_cmd(se_cmd, 0);
+	return 1;
+}
+
+static void scsiback_release_cmd(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	kmem_cache_free(scsiback_cachep, pending_req);
+}
+
+static int scsiback_shutdown_session(struct se_session *se_sess)
+{
+	return 0;
+}
+
+static void scsiback_close_session(struct se_session *se_sess)
+{
+}
+
+static u32 scsiback_sess_get_index(struct se_session *se_sess)
+{
+	return 0;
+}
+
+static int scsiback_write_pending(struct se_cmd *se_cmd)
+{
+	/* Go ahead and process the write immediately */
+	target_execute_cmd(se_cmd);
+
+	return 0;
+}
+
+static int scsiback_write_pending_status(struct se_cmd *se_cmd)
+{
+	return 0;
+}
+
+static void scsiback_set_default_node_attrs(struct se_node_acl *nacl)
+{
+}
+
+static u32 scsiback_get_task_tag(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	return pending_req->rqid;
+}
+
+static int scsiback_get_cmd_state(struct se_cmd *se_cmd)
+{
+	return 0;
+}
+
+static int scsiback_queue_data_in(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	pending_req->result = SAM_STAT_GOOD;
+	scsiback_cmd_done(pending_req);
+	return 0;
+}
+
+static int scsiback_queue_status(struct se_cmd *se_cmd)
+{
+	struct vscsibk_pend *pending_req = container_of(se_cmd,
+				struct vscsibk_pend, se_cmd);
+
+	if (se_cmd->sense_buffer &&
+	    ((se_cmd->se_cmd_flags & SCF_TRANSPORT_TASK_SENSE) ||
+	     (se_cmd->se_cmd_flags & SCF_EMULATED_TASK_SENSE)))
+		pending_req->result = (DRIVER_SENSE << 24) |
+				      SAM_STAT_CHECK_CONDITION;
+	else
+		pending_req->result = se_cmd->scsi_status;
+
+	scsiback_cmd_done(pending_req);
+	return 0;
+}
+
+static void scsiback_queue_tm_rsp(struct se_cmd *se_cmd)
+{
+	struct se_tmr_req *se_tmr = se_cmd->se_tmr_req;
+	struct scsiback_tmr *tmr = se_tmr->fabric_tmr_ptr;
+
+	atomic_set(&tmr->tmr_complete, 1);
+	wake_up(&tmr->tmr_wait);
+}
+
+static void scsiback_aborted_task(struct se_cmd *se_cmd)
+{
+}
+
+static ssize_t scsiback_tpg_param_show_alias(struct se_portal_group *se_tpg,
+					     char *page)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg, struct scsiback_tpg,
+						se_tpg);
+	ssize_t rb;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	rb = snprintf(page, PAGE_SIZE, "%s\n", tpg->param_alias);
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return rb;
+}
+
+static ssize_t scsiback_tpg_param_store_alias(struct se_portal_group *se_tpg,
+					      const char *page, size_t count)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg, struct scsiback_tpg,
+						se_tpg);
+	int len;
+
+	if (strlen(page) >= VSCSI_NAMELEN) {
+		pr_err("param alias: %s, exceeds max: %d\n", page,
+			VSCSI_NAMELEN);
+		return -EINVAL;
+	}
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	len = snprintf(tpg->param_alias, VSCSI_NAMELEN, "%s", page);
+	if (tpg->param_alias[len - 1] == '\n')
+		tpg->param_alias[len - 1] = '\0';
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return count;
+}
+
+TF_TPG_PARAM_ATTR(scsiback, alias, S_IRUGO | S_IWUSR);
+
+static struct configfs_attribute *scsiback_param_attrs[] = {
+	&scsiback_tpg_param_alias.attr,
+	NULL,
+};
+
+static int scsiback_make_nexus(struct scsiback_tpg *tpg,
+				const char *name)
+{
+	struct se_portal_group *se_tpg;
+	struct se_session *se_sess;
+	struct scsiback_nexus *tv_nexus;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	if (tpg->tpg_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_debug("tpg->tpg_nexus already exists\n");
+		return -EEXIST;
+	}
+	se_tpg = &tpg->se_tpg;
+
+	tv_nexus = kzalloc(sizeof(struct scsiback_nexus), GFP_KERNEL);
+	if (!tv_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENOMEM;
+	}
+	/*
+	 *  Initialize the struct se_session pointer
+	 */
+	tv_nexus->tvn_se_sess = transport_init_session(TARGET_PROT_NORMAL);
+	if (IS_ERR(tv_nexus->tvn_se_sess)) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		kfree(tv_nexus);
+		return -ENOMEM;
+	}
+	se_sess = tv_nexus->tvn_se_sess;
+	/*
+	 * Since we are running in 'demo mode' this call with generate a
+	 * struct se_node_acl for the scsiback struct se_portal_group with
+	 * the SCSI Initiator port name of the passed configfs group 'name'.
+	 */
+	tv_nexus->tvn_se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
+				se_tpg, (unsigned char *)name);
+	if (!tv_nexus->tvn_se_sess->se_node_acl) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_debug("core_tpg_check_initiator_node_acl() failed for %s\n",
+			 name);
+		goto out;
+	}
+	/*
+	 * Now register the TCM pvscsi virtual I_T Nexus as active with the
+	 * call to __transport_register_session()
+	 */
+	__transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
+			tv_nexus->tvn_se_sess, tv_nexus);
+	tpg->tpg_nexus = tv_nexus;
+
+	mutex_unlock(&tpg->tv_tpg_mutex);
+	return 0;
+
+out:
+	transport_free_session(se_sess);
+	kfree(tv_nexus);
+	return -ENOMEM;
+}
+
+static int scsiback_drop_nexus(struct scsiback_tpg *tpg)
+{
+	struct se_session *se_sess;
+	struct scsiback_nexus *tv_nexus;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tv_nexus = tpg->tpg_nexus;
+	if (!tv_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENODEV;
+	}
+
+	se_sess = tv_nexus->tvn_se_sess;
+	if (!se_sess) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENODEV;
+	}
+
+	if (tpg->tv_tpg_port_count != 0) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_err("Unable to remove xen-pvscsi I_T Nexus with active TPG port count: %d\n",
+			tpg->tv_tpg_port_count);
+		return -EBUSY;
+	}
+
+	if (tpg->tv_tpg_fe_count != 0) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		pr_err("Unable to remove xen-pvscsi I_T Nexus with active TPG frontend count: %d\n",
+			tpg->tv_tpg_fe_count);
+		return -EBUSY;
+	}
+
+	pr_debug("xen-pvscsi: Removing I_T Nexus to emulated %s Initiator Port: %s\n",
+		scsiback_dump_proto_id(tpg->tport),
+		tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
+
+	/*
+	 * Release the SCSI I_T Nexus to the emulated xen-pvscsi Target Port
+	 */
+	transport_deregister_session(tv_nexus->tvn_se_sess);
+	tpg->tpg_nexus = NULL;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	kfree(tv_nexus);
+	return 0;
+}
+
+static ssize_t scsiback_tpg_show_nexus(struct se_portal_group *se_tpg,
+					char *page)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_nexus *tv_nexus;
+	ssize_t ret;
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tv_nexus = tpg->tpg_nexus;
+	if (!tv_nexus) {
+		mutex_unlock(&tpg->tv_tpg_mutex);
+		return -ENODEV;
+	}
+	ret = snprintf(page, PAGE_SIZE, "%s\n",
+			tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return ret;
+}
+
+static ssize_t scsiback_tpg_store_nexus(struct se_portal_group *se_tpg,
+					 const char *page,
+					 size_t count)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+	struct scsiback_tport *tport_wwn = tpg->tport;
+	unsigned char i_port[VSCSI_NAMELEN], *ptr, *port_ptr;
+	int ret;
+	/*
+	 * Shutdown the active I_T nexus if 'NULL' is passed..
+	 */
+	if (!strncmp(page, "NULL", 4)) {
+		ret = scsiback_drop_nexus(tpg);
+		return (!ret) ? count : ret;
+	}
+	/*
+	 * Otherwise make sure the passed virtual Initiator port WWN matches
+	 * the fabric protocol_id set in scsiback_make_tport(), and call
+	 * scsiback_make_nexus().
+	 */
+	if (strlen(page) >= VSCSI_NAMELEN) {
+		pr_err("Emulated NAA Sas Address: %s, exceeds max: %d\n",
+			page, VSCSI_NAMELEN);
+		return -EINVAL;
+	}
+	snprintf(&i_port[0], VSCSI_NAMELEN, "%s", page);
+
+	ptr = strstr(i_port, "naa.");
+	if (ptr) {
+		if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_SAS) {
+			pr_err("Passed SAS Initiator Port %s does not match target port protoid: %s\n",
+				i_port, scsiback_dump_proto_id(tport_wwn));
+			return -EINVAL;
+		}
+		port_ptr = &i_port[0];
+		goto check_newline;
+	}
+	ptr = strstr(i_port, "fc.");
+	if (ptr) {
+		if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_FCP) {
+			pr_err("Passed FCP Initiator Port %s does not match target port protoid: %s\n",
+				i_port, scsiback_dump_proto_id(tport_wwn));
+			return -EINVAL;
+		}
+		port_ptr = &i_port[3]; /* Skip over "fc." */
+		goto check_newline;
+	}
+	ptr = strstr(i_port, "iqn.");
+	if (ptr) {
+		if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_ISCSI) {
+			pr_err("Passed iSCSI Initiator Port %s does not match target port protoid: %s\n",
+				i_port, scsiback_dump_proto_id(tport_wwn));
+			return -EINVAL;
+		}
+		port_ptr = &i_port[0];
+		goto check_newline;
+	}
+	pr_err("Unable to locate prefix for emulated Initiator Port: %s\n",
+		i_port);
+	return -EINVAL;
+	/*
+	 * Clear any trailing newline for the NAA WWN
+	 */
+check_newline:
+	if (i_port[strlen(i_port) - 1] == '\n')
+		i_port[strlen(i_port) - 1] = '\0';
+
+	ret = scsiback_make_nexus(tpg, port_ptr);
+	if (ret < 0)
+		return ret;
+
+	return count;
+}
+
+TF_TPG_BASE_ATTR(scsiback, nexus, S_IRUGO | S_IWUSR);
+
+static struct configfs_attribute *scsiback_tpg_attrs[] = {
+	&scsiback_tpg_nexus.attr,
+	NULL,
+};
+
+static ssize_t
+scsiback_wwn_show_attr_version(struct target_fabric_configfs *tf,
+				char *page)
+{
+	return sprintf(page, "xen-pvscsi fabric module %s on %s/%s on "
+		UTS_RELEASE"\n",
+		VSCSI_VERSION, utsname()->sysname, utsname()->machine);
+}
+
+TF_WWN_ATTR_RO(scsiback, version);
+
+static struct configfs_attribute *scsiback_wwn_attrs[] = {
+	&scsiback_wwn_version.attr,
+	NULL,
+};
+
+static char *scsiback_get_fabric_name(void)
+{
+	return "xen-pvscsi";
+}
+
+static int scsiback_port_link(struct se_portal_group *se_tpg,
+			       struct se_lun *lun)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_port_count++;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+
+	return 0;
+}
+
+static void scsiback_port_unlink(struct se_portal_group *se_tpg,
+				  struct se_lun *lun)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+
+	mutex_lock(&tpg->tv_tpg_mutex);
+	tpg->tv_tpg_port_count--;
+	mutex_unlock(&tpg->tv_tpg_mutex);
+}
+
+static struct se_portal_group *
+scsiback_make_tpg(struct se_wwn *wwn,
+		   struct config_group *group,
+		   const char *name)
+{
+	struct scsiback_tport *tport = container_of(wwn,
+			struct scsiback_tport, tport_wwn);
+
+	struct scsiback_tpg *tpg;
+	unsigned long tpgt;
+	int ret;
+
+	if (strstr(name, "tpgt_") != name)
+		return ERR_PTR(-EINVAL);
+	if (kstrtoul(name + 5, 10, &tpgt) || tpgt > UINT_MAX)
+		return ERR_PTR(-EINVAL);
+
+	tpg = kzalloc(sizeof(struct scsiback_tpg), GFP_KERNEL);
+	if (!tpg)
+		return ERR_PTR(-ENOMEM);
+
+	mutex_init(&tpg->tv_tpg_mutex);
+	INIT_LIST_HEAD(&tpg->tv_tpg_list);
+	INIT_LIST_HEAD(&tpg->info_list);
+	tpg->tport = tport;
+	tpg->tport_tpgt = tpgt;
+
+	ret = core_tpg_register(&scsiback_fabric_configfs->tf_ops, wwn,
+				&tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
+	if (ret < 0) {
+		kfree(tpg);
+		return NULL;
+	}
+	mutex_lock(&scsiback_mutex);
+	list_add_tail(&tpg->tv_tpg_list, &scsiback_list);
+	mutex_unlock(&scsiback_mutex);
+
+	return &tpg->se_tpg;
+}
+
+static void scsiback_drop_tpg(struct se_portal_group *se_tpg)
+{
+	struct scsiback_tpg *tpg = container_of(se_tpg,
+				struct scsiback_tpg, se_tpg);
+
+	mutex_lock(&scsiback_mutex);
+	list_del(&tpg->tv_tpg_list);
+	mutex_unlock(&scsiback_mutex);
+	/*
+	 * Release the virtual I_T Nexus for this xen-pvscsi TPG
+	 */
+	scsiback_drop_nexus(tpg);
+	/*
+	 * Deregister the se_tpg from TCM..
+	 */
+	core_tpg_deregister(se_tpg);
+	kfree(tpg);
+}
+
+static int scsiback_check_true(struct se_portal_group *se_tpg)
+{
+	return 1;
+}
+
+static int scsiback_check_false(struct se_portal_group *se_tpg)
+{
+	return 0;
+}
+
+static struct target_core_fabric_ops scsiback_ops = {
+	.get_fabric_name		= scsiback_get_fabric_name,
+	.get_fabric_proto_ident		= scsiback_get_fabric_proto_ident,
+	.tpg_get_wwn			= scsiback_get_fabric_wwn,
+	.tpg_get_tag			= scsiback_get_tag,
+	.tpg_get_default_depth		= scsiback_get_default_depth,
+	.tpg_get_pr_transport_id	= scsiback_get_pr_transport_id,
+	.tpg_get_pr_transport_id_len	= scsiback_get_pr_transport_id_len,
+	.tpg_parse_pr_out_transport_id	= scsiback_parse_pr_out_transport_id,
+	.tpg_check_demo_mode		= scsiback_check_true,
+	.tpg_check_demo_mode_cache	= scsiback_check_true,
+	.tpg_check_demo_mode_write_protect = scsiback_check_false,
+	.tpg_check_prod_mode_write_protect = scsiback_check_false,
+	.tpg_alloc_fabric_acl		= scsiback_alloc_fabric_acl,
+	.tpg_release_fabric_acl		= scsiback_release_fabric_acl,
+	.tpg_get_inst_index		= scsiback_tpg_get_inst_index,
+	.check_stop_free		= scsiback_check_stop_free,
+	.release_cmd			= scsiback_release_cmd,
+	.put_session			= NULL,
+	.shutdown_session		= scsiback_shutdown_session,
+	.close_session			= scsiback_close_session,
+	.sess_get_index			= scsiback_sess_get_index,
+	.sess_get_initiator_sid		= NULL,
+	.write_pending			= scsiback_write_pending,
+	.write_pending_status		= scsiback_write_pending_status,
+	.set_default_node_attributes	= scsiback_set_default_node_attrs,
+	.get_task_tag			= scsiback_get_task_tag,
+	.get_cmd_state			= scsiback_get_cmd_state,
+	.queue_data_in			= scsiback_queue_data_in,
+	.queue_status			= scsiback_queue_status,
+	.queue_tm_rsp			= scsiback_queue_tm_rsp,
+	.aborted_task			= scsiback_aborted_task,
+	/*
+	 * Setup callers for generic logic in target_core_fabric_configfs.c
+	 */
+	.fabric_make_wwn		= scsiback_make_tport,
+	.fabric_drop_wwn		= scsiback_drop_tport,
+	.fabric_make_tpg		= scsiback_make_tpg,
+	.fabric_drop_tpg		= scsiback_drop_tpg,
+	.fabric_post_link		= scsiback_port_link,
+	.fabric_pre_unlink		= scsiback_port_unlink,
+	.fabric_make_np			= NULL,
+	.fabric_drop_np			= NULL,
+#if 0
+	.fabric_make_nodeacl		= scsiback_make_nodeacl,
+	.fabric_drop_nodeacl		= scsiback_drop_nodeacl,
+#endif
+};
+
+static int scsiback_register_configfs(void)
+{
+	struct target_fabric_configfs *fabric;
+	int ret;
+
+	pr_debug("xen-pvscsi: fabric module %s on %s/%s on "UTS_RELEASE"\n",
+		 VSCSI_VERSION, utsname()->sysname, utsname()->machine);
+	/*
+	 * Register the top level struct config_item_type with TCM core
+	 */
+	fabric = target_fabric_configfs_init(THIS_MODULE, "xen-pvscsi");
+	if (IS_ERR(fabric))
+		return PTR_ERR(fabric);
+
+	/*
+	 * Setup fabric->tf_ops from our local scsiback_ops
+	 */
+	fabric->tf_ops = scsiback_ops;
+	/*
+	 * Setup default attribute lists for various fabric->tf_cit_tmpl
+	 */
+	fabric->tf_cit_tmpl.tfc_wwn_cit.ct_attrs = scsiback_wwn_attrs;
+	fabric->tf_cit_tmpl.tfc_tpg_base_cit.ct_attrs = scsiback_tpg_attrs;
+	fabric->tf_cit_tmpl.tfc_tpg_attrib_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_param_cit.ct_attrs = scsiback_param_attrs;
+	fabric->tf_cit_tmpl.tfc_tpg_np_base_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_base_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_attrib_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_auth_cit.ct_attrs = NULL;
+	fabric->tf_cit_tmpl.tfc_tpg_nacl_param_cit.ct_attrs = NULL;
+	/*
+	 * Register the fabric for use within TCM
+	 */
+	ret = target_fabric_configfs_register(fabric);
+	if (ret < 0) {
+		target_fabric_configfs_free(fabric);
+		return ret;
+	}
+	/*
+	 * Setup our local pointer to *fabric
+	 */
+	scsiback_fabric_configfs = fabric;
+	pr_debug("xen-pvscsi: Set fabric -> scsiback_fabric_configfs\n");
+	return 0;
+};
+
+static void scsiback_deregister_configfs(void)
+{
+	if (!scsiback_fabric_configfs)
+		return;
+
+	target_fabric_configfs_deregister(scsiback_fabric_configfs);
+	scsiback_fabric_configfs = NULL;
+	pr_debug("xen-pvscsi: Cleared scsiback_fabric_configfs\n");
+};
+
+static const struct xenbus_device_id scsiback_ids[] = {
+	{ "vscsi" },
+	{ "" }
+};
+
+static DEFINE_XENBUS_DRIVER(scsiback, ,
+	.probe			= scsiback_probe,
+	.remove			= scsiback_remove,
+	.otherend_changed	= scsiback_frontend_changed
+);
+
+static void scsiback_init_pend(void *p)
+{
+	struct vscsibk_pend *pend = p;
+	int i;
+
+	memset(pend, 0, sizeof(*pend));
+	for (i = 0; i < VSCSI_MAX_GRANTS; i++)
+		pend->grant_handles[i] = SCSIBACK_INVALID_HANDLE;
+}
+
+static int __init scsiback_init(void)
+{
+	int ret;
+
+	if (!xen_domain())
+		return -ENODEV;
+
+	scsiback_cachep = kmem_cache_create("vscsiif_cache",
+		sizeof(struct vscsibk_pend), 0, 0, scsiback_init_pend);
+	if (!scsiback_cachep)
+		return -ENOMEM;
+
+	ret = xenbus_register_backend(&scsiback_driver);
+	if (ret)
+		goto out_cache_destroy;
+
+	ret = scsiback_register_configfs();
+	if (ret)
+		goto out_unregister_xenbus;
+
+	return 0;
+
+out_unregister_xenbus:
+	xenbus_unregister_driver(&scsiback_driver);
+out_cache_destroy:
+	kmem_cache_destroy(scsiback_cachep);
+	pr_err("xen-pvscsi: %s: error %d\n", __func__, ret);
+	return ret;
+}
+
+static void __exit scsiback_exit(void)
+{
+	struct page *page;
+
+	while (free_pages_num) {
+		if (get_free_page(&page))
+			BUG();
+		free_xenballooned_pages(1, &page);
+	}
+	scsiback_deregister_configfs();
+	xenbus_unregister_driver(&scsiback_driver);
+	kmem_cache_destroy(scsiback_cachep);
+}
+
+module_init(scsiback_init);
+module_exit(scsiback_exit);
+
+MODULE_DESCRIPTION("Xen SCSI backend driver");
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_ALIAS("xen-backend:vscsi");
-- 
1.8.4.5

[PATCH V5 5/5] add xen pvscsi maintainer

[jgross]

From: Juergen Gross <jgross@suse.com>

Add myself as maintainer for the Xen pvSCSI stuff.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index aefa948..360f86f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10086,6 +10086,14 @@ S:	Supported
 F:	drivers/block/xen-blkback/*
 F:	drivers/block/xen*
 
+XEN PVSCSI DRIVERS
+M:	Juergen Gross <jgross@suse.com>
+L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
+S:	Supported
+F:	drivers/scsi/xen-scsifront.c
+F:	drivers/xen/xen-scsiback.c
+F:	include/xen/interface/io/vscsiif.h
+
 XEN SWIOTLB SUBSYSTEM
 M:	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
 L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
-- 
1.8.4.5

Re: [Xen-devel] [PATCH V5 5/5] add xen pvscsi maintainer

[Konrad Rzeszutek Wilk]

On Mon, Aug 18, 2014 at 11:31:50AM +0200, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Add myself as maintainer for the Xen pvSCSI stuff.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  MAINTAINERS | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index aefa948..360f86f 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -10086,6 +10086,14 @@ S:	Supported
>  F:	drivers/block/xen-blkback/*
>  F:	drivers/block/xen*
>  
> +XEN PVSCSI DRIVERS
> +M:	Juergen Gross <jgross@suse.com>
> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> +S:	Supported
> +F:	drivers/scsi/xen-scsifront.c
> +F:	drivers/xen/xen-scsiback.c
> +F:	include/xen/interface/io/vscsiif.h
> +
>  XEN SWIOTLB SUBSYSTEM
>  M:	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>  L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> -- 
> 1.8.4.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH V5 5/5] add xen pvscsi maintainer

[Konrad Rzeszutek Wilk]

On Mon, Aug 18, 2014 at 11:31:50AM +0200, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Add myself as maintainer for the Xen pvSCSI stuff.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  MAINTAINERS | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index aefa948..360f86f 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -10086,6 +10086,14 @@ S:	Supported
>  F:	drivers/block/xen-blkback/*
>  F:	drivers/block/xen*
>  
> +XEN PVSCSI DRIVERS
> +M:	Juergen Gross <jgross@suse.com>
> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> +S:	Supported
> +F:	drivers/scsi/xen-scsifront.c
> +F:	drivers/xen/xen-scsiback.c
> +F:	include/xen/interface/io/vscsiif.h
> +
>  XEN SWIOTLB SUBSYSTEM
>  M:	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>  L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> -- 
> 1.8.4.5
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH V5 5/5] add xen pvscsi maintainer

[David Vrabel]

On 18/08/14 10:31, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Add myself as maintainer for the Xen pvSCSI stuff.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  MAINTAINERS | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index aefa948..360f86f 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -10086,6 +10086,14 @@ S:	Supported
>  F:	drivers/block/xen-blkback/*
>  F:	drivers/block/xen*
>  
> +XEN PVSCSI DRIVERS
> +M:	Juergen Gross <jgross@suse.com>
> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)

You should add the appropriate scsi list here
(linux-scsi@vger.kernel.org I presume?)

David

> +S:	Supported
> +F:	drivers/scsi/xen-scsifront.c
> +F:	drivers/xen/xen-scsiback.c
> +F:	include/xen/interface/io/vscsiif.h
> +
>  XEN SWIOTLB SUBSYSTEM
>  M:	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>  L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> 

Re: [Xen-devel] [PATCH V5 5/5] add xen pvscsi maintainer

[Juergen Gross]

On 08/26/2014 04:14 PM, David Vrabel wrote:
> On 18/08/14 10:31, jgross@suse.com wrote:
>> From: Juergen Gross <jgross@suse.com>
>>
>> Add myself as maintainer for the Xen pvSCSI stuff.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   MAINTAINERS | 8 ++++++++
>>   1 file changed, 8 insertions(+)
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index aefa948..360f86f 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -10086,6 +10086,14 @@ S:	Supported
>>   F:	drivers/block/xen-blkback/*
>>   F:	drivers/block/xen*
>>
>> +XEN PVSCSI DRIVERS
>> +M:	Juergen Gross <jgross@suse.com>
>> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
>
> You should add the appropriate scsi list here
> (linux-scsi@vger.kernel.org I presume?)

Really? Xen PCI and Xen Block susbsystems don't have an according entry.
James, do you have an opinion here? I don't mind adding linux-scsi, but
I think the Xen list is the appropriate one for pvSCSI.

>> +S:	Supported
>> +F:	drivers/scsi/xen-scsifront.c
>> +F:	drivers/xen/xen-scsiback.c
>> +F:	include/xen/interface/io/vscsiif.h
>> +


Juergen

Re: [PATCH V5 5/5] add xen pvscsi maintainer

[James Bottomley]

On Tue, 2014-08-26 at 16:23 +0200, Juergen Gross wrote:
> On 08/26/2014 04:14 PM, David Vrabel wrote:
> > On 18/08/14 10:31, jgross@suse.com wrote:
> >> From: Juergen Gross <jgross@suse.com>
> >>
> >> Add myself as maintainer for the Xen pvSCSI stuff.
> >>
> >> Signed-off-by: Juergen Gross <jgross@suse.com>
> >> ---
> >>   MAINTAINERS | 8 ++++++++
> >>   1 file changed, 8 insertions(+)
> >>
> >> diff --git a/MAINTAINERS b/MAINTAINERS
> >> index aefa948..360f86f 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -10086,6 +10086,14 @@ S:	Supported
> >>   F:	drivers/block/xen-blkback/*
> >>   F:	drivers/block/xen*
> >>
> >> +XEN PVSCSI DRIVERS
> >> +M:	Juergen Gross <jgross@suse.com>
> >> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> >
> > You should add the appropriate scsi list here
> > (linux-scsi@vger.kernel.org I presume?)
> 
> Really? Xen PCI and Xen Block susbsystems don't have an according entry.
> James, do you have an opinion here? I don't mind adding linux-scsi, but
> I think the Xen list is the appropriate one for pvSCSI.

It depends how you want to handle this, but the patches have to come on
to the SCSI list somehow.  You can have them directly posted by adding
the list, or you can repost them yourselves.  It's a question of
workflow.

James

Re: [Xen-devel] [PATCH V5 5/5] add xen pvscsi maintainer

[James Bottomley]

On Tue, 2014-08-26 at 16:23 +0200, Juergen Gross wrote:
> On 08/26/2014 04:14 PM, David Vrabel wrote:
> > On 18/08/14 10:31, jgross@suse.com wrote:
> >> From: Juergen Gross <jgross@suse.com>
> >>
> >> Add myself as maintainer for the Xen pvSCSI stuff.
> >>
> >> Signed-off-by: Juergen Gross <jgross@suse.com>
> >> ---
> >>   MAINTAINERS | 8 ++++++++
> >>   1 file changed, 8 insertions(+)
> >>
> >> diff --git a/MAINTAINERS b/MAINTAINERS
> >> index aefa948..360f86f 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -10086,6 +10086,14 @@ S:	Supported
> >>   F:	drivers/block/xen-blkback/*
> >>   F:	drivers/block/xen*
> >>
> >> +XEN PVSCSI DRIVERS
> >> +M:	Juergen Gross <jgross@suse.com>
> >> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> >
> > You should add the appropriate scsi list here
> > (linux-scsi@vger.kernel.org I presume?)
> 
> Really? Xen PCI and Xen Block susbsystems don't have an according entry.
> James, do you have an opinion here? I don't mind adding linux-scsi, but
> I think the Xen list is the appropriate one for pvSCSI.

It depends how you want to handle this, but the patches have to come on
to the SCSI list somehow.  You can have them directly posted by adding
the list, or you can repost them yourselves.  It's a question of
workflow.

James

Re: [Xen-devel] [PATCH V5 5/5] add xen pvscsi maintainer

[David Vrabel]

On 26/08/14 17:37, James Bottomley wrote:
> On Tue, 2014-08-26 at 16:23 +0200, Juergen Gross wrote:
>> On 08/26/2014 04:14 PM, David Vrabel wrote:
>>> On 18/08/14 10:31, jgross@suse.com wrote:
>>>> From: Juergen Gross <jgross@suse.com>
>>>>
>>>> Add myself as maintainer for the Xen pvSCSI stuff.
>>>>
>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>> ---
>>>>   MAINTAINERS | 8 ++++++++
>>>>   1 file changed, 8 insertions(+)
>>>>
>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>> index aefa948..360f86f 100644
>>>> --- a/MAINTAINERS
>>>> +++ b/MAINTAINERS
>>>> @@ -10086,6 +10086,14 @@ S:	Supported
>>>>   F:	drivers/block/xen-blkback/*
>>>>   F:	drivers/block/xen*
>>>>
>>>> +XEN PVSCSI DRIVERS
>>>> +M:	Juergen Gross <jgross@suse.com>
>>>> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
>>>
>>> You should add the appropriate scsi list here
>>> (linux-scsi@vger.kernel.org I presume?)
>>
>> Really? Xen PCI and Xen Block susbsystems don't have an according entry.
>> James, do you have an opinion here? I don't mind adding linux-scsi, but
>> I think the Xen list is the appropriate one for pvSCSI.
> 
> It depends how you want to handle this, but the patches have to come on
> to the SCSI list somehow.  You can have them directly posted by adding
> the list, or you can repost them yourselves.  It's a question of
> workflow.

I think these drivers have had (and will continue to have) the most
useful review from scsi people.  I think posting directly (hence
earlier) to linux-scsi will ensure the best quality.

This is the workflow used for the Xen network drivers and I think it
works well.

David

Re: [Xen-devel] [PATCH V5 5/5] add xen pvscsi maintainer

[Juergen Gross]

On 08/26/2014 07:12 PM, David Vrabel wrote:
> On 26/08/14 17:37, James Bottomley wrote:
>> On Tue, 2014-08-26 at 16:23 +0200, Juergen Gross wrote:
>>> On 08/26/2014 04:14 PM, David Vrabel wrote:
>>>> On 18/08/14 10:31, jgross@suse.com wrote:
>>>>> From: Juergen Gross <jgross@suse.com>
>>>>>
>>>>> Add myself as maintainer for the Xen pvSCSI stuff.
>>>>>
>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>> ---
>>>>>    MAINTAINERS | 8 ++++++++
>>>>>    1 file changed, 8 insertions(+)
>>>>>
>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>>> index aefa948..360f86f 100644
>>>>> --- a/MAINTAINERS
>>>>> +++ b/MAINTAINERS
>>>>> @@ -10086,6 +10086,14 @@ S:	Supported
>>>>>    F:	drivers/block/xen-blkback/*
>>>>>    F:	drivers/block/xen*
>>>>>
>>>>> +XEN PVSCSI DRIVERS
>>>>> +M:	Juergen Gross <jgross@suse.com>
>>>>> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
>>>>
>>>> You should add the appropriate scsi list here
>>>> (linux-scsi@vger.kernel.org I presume?)
>>>
>>> Really? Xen PCI and Xen Block susbsystems don't have an according entry.
>>> James, do you have an opinion here? I don't mind adding linux-scsi, but
>>> I think the Xen list is the appropriate one for pvSCSI.
>>
>> It depends how you want to handle this, but the patches have to come on
>> to the SCSI list somehow.  You can have them directly posted by adding
>> the list, or you can repost them yourselves.  It's a question of
>> workflow.
>
> I think these drivers have had (and will continue to have) the most
> useful review from scsi people.  I think posting directly (hence
> earlier) to linux-scsi will ensure the best quality.
>
> This is the workflow used for the Xen network drivers and I think it
> works well.

Okay, I'll add the scsi list.

Juergen

Re: [PATCH V5 5/5] add xen pvscsi maintainer

[Juergen Gross]

On 08/26/2014 07:12 PM, David Vrabel wrote:
> On 26/08/14 17:37, James Bottomley wrote:
>> On Tue, 2014-08-26 at 16:23 +0200, Juergen Gross wrote:
>>> On 08/26/2014 04:14 PM, David Vrabel wrote:
>>>> On 18/08/14 10:31, jgross@suse.com wrote:
>>>>> From: Juergen Gross <jgross@suse.com>
>>>>>
>>>>> Add myself as maintainer for the Xen pvSCSI stuff.
>>>>>
>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>> ---
>>>>>    MAINTAINERS | 8 ++++++++
>>>>>    1 file changed, 8 insertions(+)
>>>>>
>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>>> index aefa948..360f86f 100644
>>>>> --- a/MAINTAINERS
>>>>> +++ b/MAINTAINERS
>>>>> @@ -10086,6 +10086,14 @@ S:	Supported
>>>>>    F:	drivers/block/xen-blkback/*
>>>>>    F:	drivers/block/xen*
>>>>>
>>>>> +XEN PVSCSI DRIVERS
>>>>> +M:	Juergen Gross <jgross@suse.com>
>>>>> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
>>>>
>>>> You should add the appropriate scsi list here
>>>> (linux-scsi@vger.kernel.org I presume?)
>>>
>>> Really? Xen PCI and Xen Block susbsystems don't have an according entry.
>>> James, do you have an opinion here? I don't mind adding linux-scsi, but
>>> I think the Xen list is the appropriate one for pvSCSI.
>>
>> It depends how you want to handle this, but the patches have to come on
>> to the SCSI list somehow.  You can have them directly posted by adding
>> the list, or you can repost them yourselves.  It's a question of
>> workflow.
>
> I think these drivers have had (and will continue to have) the most
> useful review from scsi people.  I think posting directly (hence
> earlier) to linux-scsi will ensure the best quality.
>
> This is the workflow used for the Xen network drivers and I think it
> works well.

Okay, I'll add the scsi list.

Juergen

Re: [PATCH V5 5/5] add xen pvscsi maintainer

[David Vrabel]

On 26/08/14 17:37, James Bottomley wrote:
> On Tue, 2014-08-26 at 16:23 +0200, Juergen Gross wrote:
>> On 08/26/2014 04:14 PM, David Vrabel wrote:
>>> On 18/08/14 10:31, jgross@suse.com wrote:
>>>> From: Juergen Gross <jgross@suse.com>
>>>>
>>>> Add myself as maintainer for the Xen pvSCSI stuff.
>>>>
>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>> ---
>>>>   MAINTAINERS | 8 ++++++++
>>>>   1 file changed, 8 insertions(+)
>>>>
>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>> index aefa948..360f86f 100644
>>>> --- a/MAINTAINERS
>>>> +++ b/MAINTAINERS
>>>> @@ -10086,6 +10086,14 @@ S:	Supported
>>>>   F:	drivers/block/xen-blkback/*
>>>>   F:	drivers/block/xen*
>>>>
>>>> +XEN PVSCSI DRIVERS
>>>> +M:	Juergen Gross <jgross@suse.com>
>>>> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
>>>
>>> You should add the appropriate scsi list here
>>> (linux-scsi@vger.kernel.org I presume?)
>>
>> Really? Xen PCI and Xen Block susbsystems don't have an according entry.
>> James, do you have an opinion here? I don't mind adding linux-scsi, but
>> I think the Xen list is the appropriate one for pvSCSI.
> 
> It depends how you want to handle this, but the patches have to come on
> to the SCSI list somehow.  You can have them directly posted by adding
> the list, or you can repost them yourselves.  It's a question of
> workflow.

I think these drivers have had (and will continue to have) the most
useful review from scsi people.  I think posting directly (hence
earlier) to linux-scsi will ensure the best quality.

This is the workflow used for the Xen network drivers and I think it
works well.

David

Re: [PATCH V5 5/5] add xen pvscsi maintainer

[Juergen Gross]

On 08/26/2014 04:14 PM, David Vrabel wrote:
> On 18/08/14 10:31, jgross@suse.com wrote:
>> From: Juergen Gross <jgross@suse.com>
>>
>> Add myself as maintainer for the Xen pvSCSI stuff.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   MAINTAINERS | 8 ++++++++
>>   1 file changed, 8 insertions(+)
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index aefa948..360f86f 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -10086,6 +10086,14 @@ S:	Supported
>>   F:	drivers/block/xen-blkback/*
>>   F:	drivers/block/xen*
>>
>> +XEN PVSCSI DRIVERS
>> +M:	Juergen Gross <jgross@suse.com>
>> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
>
> You should add the appropriate scsi list here
> (linux-scsi@vger.kernel.org I presume?)

Really? Xen PCI and Xen Block susbsystems don't have an according entry.
James, do you have an opinion here? I don't mind adding linux-scsi, but
I think the Xen list is the appropriate one for pvSCSI.

>> +S:	Supported
>> +F:	drivers/scsi/xen-scsifront.c
>> +F:	drivers/xen/xen-scsiback.c
>> +F:	include/xen/interface/io/vscsiif.h
>> +


Juergen

Re: [PATCH V5 5/5] add xen pvscsi maintainer

[David Vrabel]

On 18/08/14 10:31, jgross@suse.com wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Add myself as maintainer for the Xen pvSCSI stuff.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  MAINTAINERS | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index aefa948..360f86f 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -10086,6 +10086,14 @@ S:	Supported
>  F:	drivers/block/xen-blkback/*
>  F:	drivers/block/xen*
>  
> +XEN PVSCSI DRIVERS
> +M:	Juergen Gross <jgross@suse.com>
> +L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)

You should add the appropriate scsi list here
(linux-scsi@vger.kernel.org I presume?)

David

> +S:	Supported
> +F:	drivers/scsi/xen-scsifront.c
> +F:	drivers/xen/xen-scsiback.c
> +F:	include/xen/interface/io/vscsiif.h
> +
>  XEN SWIOTLB SUBSYSTEM
>  M:	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>  L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
> 

[PATCH V5 5/5] add xen pvscsi maintainer

[jgross]

From: Juergen Gross <jgross@suse.com>

Add myself as maintainer for the Xen pvSCSI stuff.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index aefa948..360f86f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10086,6 +10086,14 @@ S:	Supported
 F:	drivers/block/xen-blkback/*
 F:	drivers/block/xen*
 
+XEN PVSCSI DRIVERS
+M:	Juergen Gross <jgross@suse.com>
+L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
+S:	Supported
+F:	drivers/scsi/xen-scsifront.c
+F:	drivers/xen/xen-scsiback.c
+F:	include/xen/interface/io/vscsiif.h
+
 XEN SWIOTLB SUBSYSTEM
 M:	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
 L:	xen-devel@lists.xenproject.org (moderated for non-subscribers)
-- 
1.8.4.5

Re: [PATCH V5 0/5] Add XEN pvSCSI support

[Nicholas A. Bellinger]

Hi Juergen & Co,

On Mon, 2014-08-18 at 11:31 +0200, jgross@suse.com wrote:
> This series adds XEN pvSCSI support. With pvSCSI it is possible to use physical
> SCSI devices from a XEN domain.
> 
> The support consists of a backend in the privileged Domain-0 doing the real
> I/O and a frontend in the unprivileged domU passing I/O-requests to the backend.
> 
> The code is taken (and adapted) from the original pvSCSI implementation done
> for Linux 2.6 in 2008 by Fujitsu.
> 
> [PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels
> [PATCH V5 2/5] Add XEN pvSCSI protocol description
> [PATCH V5 3/5] Introduce xen-scsifront module
> [PATCH V5 4/5] Introduce XEN scsiback module
> [PATCH V5 5/5] add xen pvscsi maintainer
> 
> Changes in V5:
> - Added patch to support threaded irqs for interdomain event channels
> - several changes in xen-scsifront after comments from Christoph Hellwig
> - several changes in xen-scsiback after comments from Christoph Hellwig
> - several changes in xen-scsiback after comments from James Bottomley
> 
> Changes in V4:
> - Re-add define for VSCSIIF_ACT_SCSI_SG_PRESET to vscsiif.h to indicate this
>   action value should not be used in future enhancements
> 
> Changes in V3:
> - added some comments to the protocol header file
> - removed the CDB emulation from xen-scsiback, handled by core target
>   infrastructure
> - several changes in xen-scsifront after comments from Christoph Hellwig
> 
> Changes in V2:
> - use core target infrastructure by backend instead of pure SCSI passthrough
> - add support for larger SG lists by putting them in grant page(s)
> - add command abort capability
> 

For the XEN scsiback parts as a new target fabric driver, feel free to
add my:

Reviewed-by: Nicholas Bellinger <nab@linux-iscsi.org>

So I assume this will be merged for v3.18 via the xen.git tree, yes..?

--nab

Re: [Xen-devel] [PATCH V5 0/5] Add XEN pvSCSI support

[Konrad Rzeszutek Wilk]

On Fri, Aug 22, 2014 at 02:21:55PM -0700, Nicholas A. Bellinger wrote:
> Hi Juergen & Co,
> 
> On Mon, 2014-08-18 at 11:31 +0200, jgross@suse.com wrote:
> > This series adds XEN pvSCSI support. With pvSCSI it is possible to use physical
> > SCSI devices from a XEN domain.
> > 
> > The support consists of a backend in the privileged Domain-0 doing the real
> > I/O and a frontend in the unprivileged domU passing I/O-requests to the backend.
> > 
> > The code is taken (and adapted) from the original pvSCSI implementation done
> > for Linux 2.6 in 2008 by Fujitsu.
> > 
> > [PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels
> > [PATCH V5 2/5] Add XEN pvSCSI protocol description
> > [PATCH V5 3/5] Introduce xen-scsifront module
> > [PATCH V5 4/5] Introduce XEN scsiback module
> > [PATCH V5 5/5] add xen pvscsi maintainer
> > 
> > Changes in V5:
> > - Added patch to support threaded irqs for interdomain event channels
> > - several changes in xen-scsifront after comments from Christoph Hellwig
> > - several changes in xen-scsiback after comments from Christoph Hellwig
> > - several changes in xen-scsiback after comments from James Bottomley
> > 
> > Changes in V4:
> > - Re-add define for VSCSIIF_ACT_SCSI_SG_PRESET to vscsiif.h to indicate this
> >   action value should not be used in future enhancements
> > 
> > Changes in V3:
> > - added some comments to the protocol header file
> > - removed the CDB emulation from xen-scsiback, handled by core target
> >   infrastructure
> > - several changes in xen-scsifront after comments from Christoph Hellwig
> > 
> > Changes in V2:
> > - use core target infrastructure by backend instead of pure SCSI passthrough
> > - add support for larger SG lists by putting them in grant page(s)
> > - add command abort capability
> > 
> 
> For the XEN scsiback parts as a new target fabric driver, feel free to
> add my:
> 
> Reviewed-by: Nicholas Bellinger <nab@linux-iscsi.org>
> 
> So I assume this will be merged for v3.18 via the xen.git tree, yes..?

<nods>
When I chatted with Christopher Hellwig he recommended it be done that way.

I am almost done with the Xen bits.
> 
> --nab
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH V5 0/5] Add XEN pvSCSI support

[David Vrabel]

On 23/08/14 01:40, Konrad Rzeszutek Wilk wrote:
> On Fri, Aug 22, 2014 at 02:21:55PM -0700, Nicholas A. Bellinger wrote:
>> 
>>
>> For the XEN scsiback parts as a new target fabric driver, feel free to
>> add my:
>>
>> Reviewed-by: Nicholas Bellinger <nab@linux-iscsi.org>
>>
>> So I assume this will be merged for v3.18 via the xen.git tree, yes..?

My preference is for drivers outside of drivers/xen/ to be merged via
their relevant subsystem tree.  This is how all the other Xen PV drivers
are handled (net, block, console, etc.).

> <nods>
> When I chatted with Christopher Hellwig he recommended it be done that way.

Did he give a reason?  The driver has more scsi bits in it than Xen bits
really.

David

Re: [Xen-devel] [PATCH V5 0/5] Add XEN pvSCSI support

[David Vrabel]

On 23/08/14 01:40, Konrad Rzeszutek Wilk wrote:
> On Fri, Aug 22, 2014 at 02:21:55PM -0700, Nicholas A. Bellinger wrote:
>> 
>>
>> For the XEN scsiback parts as a new target fabric driver, feel free to
>> add my:
>>
>> Reviewed-by: Nicholas Bellinger <nab@linux-iscsi.org>
>>
>> So I assume this will be merged for v3.18 via the xen.git tree, yes..?

My preference is for drivers outside of drivers/xen/ to be merged via
their relevant subsystem tree.  This is how all the other Xen PV drivers
are handled (net, block, console, etc.).

> <nods>
> When I chatted with Christopher Hellwig he recommended it be done that way.

Did he give a reason?  The driver has more scsi bits in it than Xen bits
really.

David

Re: [Xen-devel] [PATCH V5 0/5] Add XEN pvSCSI support

[Christoph Hellwig]

On Tue, Aug 26, 2014 at 11:13:15AM +0100, David Vrabel wrote:
> > <nods>
> > When I chatted with Christopher Hellwig he recommended it be done that way.
> 
> Did he give a reason?  The driver has more scsi bits in it than Xen bits
> really.

There's Xen core changes and a scsi initiator driver and a target driver
both depending on the core changes.  If we want to get all these in
in the same merge window without pre-merging trees the Xen tree is the
most sensible way to get it in.  I'll hapilly take patches for the
initiator driver for the following merge windows once it is in.

Re: [PATCH V5 0/5] Add XEN pvSCSI support

[Christoph Hellwig]

On Tue, Aug 26, 2014 at 11:13:15AM +0100, David Vrabel wrote:
> > <nods>
> > When I chatted with Christopher Hellwig he recommended it be done that way.
> 
> Did he give a reason?  The driver has more scsi bits in it than Xen bits
> really.

There's Xen core changes and a scsi initiator driver and a target driver
both depending on the core changes.  If we want to get all these in
in the same merge window without pre-merging trees the Xen tree is the
most sensible way to get it in.  I'll hapilly take patches for the
initiator driver for the following merge windows once it is in.

Re: [PATCH V5 0/5] Add XEN pvSCSI support

[Konrad Rzeszutek Wilk]

On Fri, Aug 22, 2014 at 02:21:55PM -0700, Nicholas A. Bellinger wrote:
> Hi Juergen & Co,
> 
> On Mon, 2014-08-18 at 11:31 +0200, jgross@suse.com wrote:
> > This series adds XEN pvSCSI support. With pvSCSI it is possible to use physical
> > SCSI devices from a XEN domain.
> > 
> > The support consists of a backend in the privileged Domain-0 doing the real
> > I/O and a frontend in the unprivileged domU passing I/O-requests to the backend.
> > 
> > The code is taken (and adapted) from the original pvSCSI implementation done
> > for Linux 2.6 in 2008 by Fujitsu.
> > 
> > [PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels
> > [PATCH V5 2/5] Add XEN pvSCSI protocol description
> > [PATCH V5 3/5] Introduce xen-scsifront module
> > [PATCH V5 4/5] Introduce XEN scsiback module
> > [PATCH V5 5/5] add xen pvscsi maintainer
> > 
> > Changes in V5:
> > - Added patch to support threaded irqs for interdomain event channels
> > - several changes in xen-scsifront after comments from Christoph Hellwig
> > - several changes in xen-scsiback after comments from Christoph Hellwig
> > - several changes in xen-scsiback after comments from James Bottomley
> > 
> > Changes in V4:
> > - Re-add define for VSCSIIF_ACT_SCSI_SG_PRESET to vscsiif.h to indicate this
> >   action value should not be used in future enhancements
> > 
> > Changes in V3:
> > - added some comments to the protocol header file
> > - removed the CDB emulation from xen-scsiback, handled by core target
> >   infrastructure
> > - several changes in xen-scsifront after comments from Christoph Hellwig
> > 
> > Changes in V2:
> > - use core target infrastructure by backend instead of pure SCSI passthrough
> > - add support for larger SG lists by putting them in grant page(s)
> > - add command abort capability
> > 
> 
> For the XEN scsiback parts as a new target fabric driver, feel free to
> add my:
> 
> Reviewed-by: Nicholas Bellinger <nab@linux-iscsi.org>
> 
> So I assume this will be merged for v3.18 via the xen.git tree, yes..?

<nods>
When I chatted with Christopher Hellwig he recommended it be done that way.

I am almost done with the Xen bits.
> 
> --nab
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH V5 0/5] Add XEN pvSCSI support

[Nicholas A. Bellinger]

Hi Juergen & Co,

On Mon, 2014-08-18 at 11:31 +0200, jgross@suse.com wrote:
> This series adds XEN pvSCSI support. With pvSCSI it is possible to use physical
> SCSI devices from a XEN domain.
> 
> The support consists of a backend in the privileged Domain-0 doing the real
> I/O and a frontend in the unprivileged domU passing I/O-requests to the backend.
> 
> The code is taken (and adapted) from the original pvSCSI implementation done
> for Linux 2.6 in 2008 by Fujitsu.
> 
> [PATCH V5 1/5] xen/events: support threaded irqs for interdomain event channels
> [PATCH V5 2/5] Add XEN pvSCSI protocol description
> [PATCH V5 3/5] Introduce xen-scsifront module
> [PATCH V5 4/5] Introduce XEN scsiback module
> [PATCH V5 5/5] add xen pvscsi maintainer
> 
> Changes in V5:
> - Added patch to support threaded irqs for interdomain event channels
> - several changes in xen-scsifront after comments from Christoph Hellwig
> - several changes in xen-scsiback after comments from Christoph Hellwig
> - several changes in xen-scsiback after comments from James Bottomley
> 
> Changes in V4:
> - Re-add define for VSCSIIF_ACT_SCSI_SG_PRESET to vscsiif.h to indicate this
>   action value should not be used in future enhancements
> 
> Changes in V3:
> - added some comments to the protocol header file
> - removed the CDB emulation from xen-scsiback, handled by core target
>   infrastructure
> - several changes in xen-scsifront after comments from Christoph Hellwig
> 
> Changes in V2:
> - use core target infrastructure by backend instead of pure SCSI passthrough
> - add support for larger SG lists by putting them in grant page(s)
> - add command abort capability
> 

For the XEN scsiback parts as a new target fabric driver, feel free to
add my:

Reviewed-by: Nicholas Bellinger <nab@linux-iscsi.org>

So I assume this will be merged for v3.18 via the xen.git tree, yes..?

--nab