Re: [PATCH 0/3] various update for sheepdog driver

Re: [PATCH 0/3] various update for sheepdog driver

[FUJITA Tomonori]

On Sun,  6 Jul 2014 23:30:29 +0900
Hitoshi Mitake <mitake.hitoshi@gmail.com> wrote:

> This patchset contain some update for sheepdog driver. The most
> important one is the first patch. With this patch, iSCSI target with
> sheepdog virtual disk can enable caching feature of sheepdog.
> 
> Others are mainly for documenting purpose.
> 
> Hitoshi Mitake (3):
>   sheepdog: turn on cache flag of write request
>   sheepdog: update layout of inode object header
>   sheepdog: add documentation
> 
>  doc/README.sheepdog | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  usr/bs_sheepdog.c   |  29 +++++++++++++--
>  2 files changed, 126 insertions(+), 3 deletions(-)
>  create mode 100644 doc/README.sheepdog

Applied, thanks.

[PATCH 0/3] various update for sheepdog driver

[Hitoshi Mitake]

This patchset contain some update for sheepdog driver. The most
important one is the first patch. With this patch, iSCSI target with
sheepdog virtual disk can enable caching feature of sheepdog.

Others are mainly for documenting purpose.

Hitoshi Mitake (3):
  sheepdog: turn on cache flag of write request
  sheepdog: update layout of inode object header
  sheepdog: add documentation

 doc/README.sheepdog | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 usr/bs_sheepdog.c   |  29 +++++++++++++--
 2 files changed, 126 insertions(+), 3 deletions(-)
 create mode 100644 doc/README.sheepdog

-- 
1.9.1

[PATCH 1/3] sheepdog: turn on cache flag of write request

[Hitoshi Mitake]

tgtd doesn't affect semantics of caching, so we can always turn on
cache feature of sheepdog layer.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
---
 usr/bs_sheepdog.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/usr/bs_sheepdog.c b/usr/bs_sheepdog.c
index 6d60bd6..a333675 100644
--- a/usr/bs_sheepdog.c
+++ b/usr/bs_sheepdog.c
@@ -795,13 +795,19 @@ retry:
 		old_oid = 0;
 
 		if (write) {
+			/*
+			 * tgt doesn't affect semantics of caching, so we can
+			 * always turn on cache of sheep layer
+			 */
+			flags = SD_FLAG_CMD_CACHE;
+
 			if (ai->inode.data_vdi_id[idx] != vid) {
 				create = 1;
 
 				if (ai->inode.data_vdi_id[idx]) {
 					/* COW */
 					old_oid = oid;
-					flags = SD_FLAG_CMD_COW;
+					flags |= SD_FLAG_CMD_COW;
 				}
 
 				oid = vid_to_data_oid(ai->inode.vdi_id, idx);
-- 
1.9.1

[PATCH 2/3] sheepdog: update layout of inode object header

[Hitoshi Mitake]

This change doesn't affect behavior of sheepdog driver. Just for
documentation purpose.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
---
 usr/bs_sheepdog.c | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/usr/bs_sheepdog.c b/usr/bs_sheepdog.c
index a333675..15f44d8 100644
--- a/usr/bs_sheepdog.c
+++ b/usr/bs_sheepdog.c
@@ -193,6 +193,19 @@ struct sheepdog_vdi_rsp {
 	uint32_t pad[5];
 };
 
+/*
+ * Historical notes: previous version of sheepdog (< v0.9.0) has a limit of
+ * maximum number of children which can be created from single VDI. So the inode
+ * object has an array for storing the IDs of the child VDIs. The constant
+ * OLD_MAX_CHILDREN represents it. Current sheepdog doesn't have the limitation,
+ * so we are recycling the area (4 * OLD_MAX_CHILDREN = 4KB) for storing new
+ * metadata.
+ *
+ * users of the released area:
+ * - uint32_t btree_counter
+ */
+#define OLD_MAX_CHILDREN 1024U
+
 struct sheepdog_inode {
 	char name[SD_MAX_VDI_LEN];
 	char tag[SD_MAX_VDI_TAG_LEN];
@@ -201,13 +214,17 @@ struct sheepdog_inode {
 	uint64_t vm_clock_nsec;
 	uint64_t vdi_size;
 	uint64_t vm_state_size;
-	uint16_t copy_policy;
+	uint8_t  copy_policy;
+	uint8_t  store_policy;
 	uint8_t nr_copies;
 	uint8_t block_size_shift;
 	uint32_t snap_id;
 	uint32_t vdi_id;
 	uint32_t parent_vdi_id;
-	uint32_t child_vdi_id[MAX_CHILDREN];
+
+	uint32_t btree_counter;
+	uint32_t __unused[OLD_MAX_CHILDREN - 1];
+
 	uint32_t data_vdi_id[MAX_DATA_OBJS];
 };
 
-- 
1.9.1

[PATCH 3/3] sheepdog: add documentation

[Hitoshi Mitake]

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
---
 doc/README.sheepdog | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 100 insertions(+)
 create mode 100644 doc/README.sheepdog

diff --git a/doc/README.sheepdog b/doc/README.sheepdog
new file mode 100644
index 0000000..64452fe
--- /dev/null
+++ b/doc/README.sheepdog
@@ -0,0 +1,100 @@
+Sheepdog is a distributed storage system for QEMU, iSCSI clients and
+RESTful services. It provides highly available block level storage
+volumes that can be attached to QEMU-based virtual machines. The
+volumes can also be attached to other virtual machines and operating
+systems run on baremetal hardware if they support iSCSI
+protocol. Sheepdog scales to several hundreds nodes, and supports
+advanced volume management features such as snapshot, cloning, and
+thin provisioning. Stuff like volumes, snapshots, QEMU's vm-state
+(from live snapshot), even ISO files can be stored in the Sheepdog
+cluster.
+
+With tgt, sheepdog's virtual disks can be used by iSCSI
+initiators. Below is a brief description of setup.
+
+1. Install and launch tgt
+
+$ git clone https://github.com/fujita/tgt.git
+$ cd tgt
+$ make
+# make install
+2. Create a sheepdog VDI
+
+There is no special procedures in this step.
+
+$ dog vdi create tgt0 100G
+Of course you can use existing VDIs as iSCSI logical units.
+
+3. Setup iSCSI target provided by tgt
+
+One logical unit corresponds to one VDI of sheepdog. In this step, we
+create iSCSI target and logical unit which can be seen by iSCSI
+initiator.
+
+# tgtd
+# tgtadm --op new --mode target --tid 1 --lld iscsi --targetname iqn.2013-10.org.sheepdog-project
+# tgtadm --op new --mode lu --tid 1 --lun 2 --bstype sheepdog --backing-store unix:/sheep_store/sock:tgt0
+# tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
+
+The most important parameter is --backing-store which is required by
+tgtadm when we create the logical unit in the target (the third line
+of the above commands). With this parameter, we tell the tgtd
+process how to connect to the sheep process, which VDI we use as the
+logical unit, which snapshot ID of the VDI we use. The form of the
+--backing-store option is like this:
+
+ unix:path_of_unix_domain_socket:vdi
+ unix:path_of_unix_domain_socket:vdi:tag
+ unix:path_of_unix_domain_socket:vdi:snapid
+ tcp:host:port:vdi
+ tcp:host:port:vdi:tag
+ tcp:host:port:vdi:snapid
+
+We use a special case of the first one in the above command
+sequence. It means "connect to sheep process via unix domain socket
+protocol, path of unix domain socket is /sheep_store/sock, use VDI
+named tgt0 as a backing store of newly created logical unit".
+
+You can also specify sheep which is running on remote host via
+TCP. But if tgtd process is running on a same host of sheep process,
+there is no benefit of using TCP. Unix domain socket can provide
+faster connection than TCP.
+
+You can also specify tag or ID of snapshots via suffix of the
+--backing-store option.
+
+4. Setup iSCSI session (example of the open-iscsi initiator on Linux)
+
+After setting up iSCSI target, you can use the VDIs from any virtual
+machines and operating systems which supports iSCSI initiator. Many of
+popular hypervisors and operating systems support it (e.g. VMware ESX
+Family, Linux, Windows, etc). In this example, the way of Linux +
+open-iscsi is described.
+
+At first, you have to install open-iscsi ( http://www.open-iscsi.org/
+) and launch it. Major linux distros provide their open-iscsi
+package. Below is a way of installation in Debian and Ubuntu based
+systems.
+
+# apt-get install open-iscsi
+# /etc/init.d/open-iscsi start
+
+Next, we need to let iscsid discover and login to the target we've
+already created in the above sequence. If the initiator is running
+on different host from the target, you have to change the IP
+addresses in the below commands.
+
+# iscsiadm -m discovery -t st -p 127.0.0.1
+# iscsiadm -m node --targetname iqn.2013-10.org.sheepdog-project --portal 127.0.0.1:3260 --login
+
+New device files, e.g. /dev/sdb, will be created on your system after
+login completion. Now your system can use the speepdog VDIs like
+ordinal HDDs.
+
+ToDo
+
+Currently, iSCSI multipath is not supported by sheepdog. It will be
+implemented in v0.9.0.
+
+The latest version of this guide can be found here:
+https://github.com/sheepdog/sheepdog/wiki/General-protocol-support-%28iSCSI-and-NBD%29
-- 
1.9.1