[PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Eric Wheeler]

Hi Joe,

Please review the patch below when you have a moment.  I am interested in 
your feedback, and also interested in having this functionality merged 
upstream.  This was written against thin-provisioning-tools.git tag 
v0.5.6.

We use thin_dump on live dm-thin metadata snapshots all the time.  In our 
case, we want to dump the XML for new (snapshot) volumes instead of 
dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 
minutes instead of ~5 seconds for a single volume with --device-id.

I started by adding --device-id to pass the `block_address dev_id` and 
skip the call to emit_mappings() within mapping_tree_emitter::visit() 
unless the dev_id matches --device-id as passed to thin_dump.  

It is sometimes nice to list all of the device supers without waiting for 
emit_mappings() so I added the --skip-mappings option.  This allows you to 
get the dev_id without reading the mappings:
	thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta

Like `bool repair`, I added skip_mappings and dev_id as arguments to 
thin_provisioning::metadata_dump() and passed them down to 
mapping_tree_emitter with new members set_skip_mappings() and set_dev_id() 
to set private attributes (default values are assigned in 
metadata_dumper.h in case of backward compatible calls).

We work with the device metadata in a simplified format, and without 
superblock information:

	origin_offset:length:tdata_offset (hence, "o:L:d" or "old") 

Therefore, the output --format 'old' was added.  I added the format 'null' 
as well for benchmarking purposes so the ostream needn't write large 
volumes of text to /dev/null.  

Benchmarks of the various formats on our sample metadata snapshot on an 
idle machine:

for i in xml human_readable old null; do
	time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null
done

	real		user		sys
xml	29:01.27	25:35.87	01:54.49
human	27:39.81	24:12.90	01:52.39
old	27:07.71	23:40.64	01:52.97
null	23:39.38	21:17.22	00:49.04

I have this as a branch in bitbucket if you prefer to see it online:
  
https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id
  or
git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id


--
Eric Wheeler


diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc
index 191acb5..7df5d85 100644
--- a/thin-provisioning/thin_dump.cc
+++ b/thin-provisioning/thin_dump.cc
@@ -22,6 +22,8 @@
 #include <libgen.h>
 
 #include "human_readable_format.h"
+#include "old_format.h"
+#include "null_format.h"
 #include "metadata_dumper.h"
 #include "metadata.h"
 #include "xml_format.h"
@@ -36,6 +38,8 @@ using namespace thin_provisioning;
 struct flags {
 	bool find_metadata_snap;
 	bool repair;
+	block_address dev_id;
+	bool skip_mappings;
 };
 
 namespace {
@@ -49,12 +53,16 @@ namespace {
 				e = create_xml_emitter(out);
 			else if (format == "human_readable")
 				e = create_human_readable_emitter(out);
+			else if (format == "old")
+				e = create_old_emitter(out);
+			else if (format == "null")
+				e = create_null_emitter(out);
 			else {
 				cerr << "unknown format '" << format << "'" << endl;
 				exit(1);
 			}
 
-			metadata_dump(md, e, flags.repair);
+			metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id);
 
 		} catch (std::exception &e) {
 			cerr << e.what() << endl;
@@ -77,10 +85,12 @@ namespace {
 		out << "Usage: " << cmd << " [options] {device|file}" << endl
 		    << "Options:" << endl
 		    << "  {-h|--help}" << endl
-		    << "  {-f|--format} {xml|human_readable}" << endl
+		    << "  {-f|--format} {xml|human_readable|old|null}" << endl
 		    << "  {-r|--repair}" << endl
 		    << "  {-m|--metadata-snap} [block#]" << endl
 		    << "  {-o <xml file>}" << endl
+		    << "  {-d device_id}" << endl
+		    << "  {-s|--skip-mappings}" << endl
 		    << "  {-V|--version}" << endl;
 	}
 }
@@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv)
 {
 	int c;
 	char const *output = NULL;
-	const char shortopts[] = "hm::o:f:rV";
+	const char shortopts[] = "hm::o:sd:f:rV";
 	char *end_ptr;
 	string format = "xml";
 	block_address metadata_snap = 0;
 	struct flags flags;
-	flags.find_metadata_snap = flags.repair = false;
+	flags.find_metadata_snap = flags.repair = flags.skip_mappings = false;
+	flags.dev_id = -1;
 
 	const struct option longopts[] = {
 		{ "help", no_argument, NULL, 'h'},
 		{ "metadata-snap", optional_argument, NULL, 'm' },
 		{ "output", required_argument, NULL, 'o'},
+		{ "skip-mappings", no_argument, NULL, 's'},
+		{ "device-id", required_argument, NULL, 'd'},
 		{ "format", required_argument, NULL, 'f' },
 		{ "repair", no_argument, NULL, 'r'},
 		{ "version", no_argument, NULL, 'V'},
@@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv)
 			flags.repair = true;
 			break;
 
+		case 's':
+			flags.skip_mappings = true;
+			break;
+
+		case 'd':
+			flags.dev_id = strtoull(optarg, &end_ptr, 10);
+			if (end_ptr == optarg) {
+				cerr << "couldn't parse <device_id>" << endl;
+				usage(cerr, basename(argv[0]));
+				return 1;
+			}
+			break;
+
 		case 'm':
 			if (optarg) {
 				metadata_snap = strtoull(optarg, &end_ptr, 10);
@@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv)
 		}
 	}
 
+	if (format != "old" && flags.dev_id < 0) {
+		cerr << "Output format 'old' must specify --device-id" << endl;
+		return 1;
+	}
+
+
 	if (argc == optind) {
 		cerr << "No input file provided." << endl;
 		usage(cerr, basename(argv[0]));
diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h
index c96d22e..9d9814e 100644
--- a/thin-provisioning/metadata_dumper.h
+++ b/thin-provisioning/metadata_dumper.h
@@ -28,7 +28,7 @@ namespace thin_provisioning {
 	// Set the @repair flag if your metadata is corrupt, and you'd like
 	// the dumper to do it's best to recover info.  If not set, any
 	// corruption encountered will cause an exception to be thrown.
-	void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);
+	void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1);
 }
 
 //----------------------------------------------------------------
diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc
index db656ee..55cbae8 100644
--- a/thin-provisioning/metadata_dumper.cc
+++ b/thin-provisioning/metadata_dumper.cc
@@ -171,23 +171,31 @@ namespace {
 			  dd_(dd),
 			  repair_(repair),
 			  damage_policy_(damage_policy) {
+				  dev_id_ = -1;
+				  skip_mappings_ = false;
 		}
 
+		void set_dev_id(block_address dev_id) { dev_id_ = dev_id; }
+		void set_skip_mappings(bool skip) { skip_mappings_ = skip; }
+
 		void visit(btree_path const &path, block_address tree_root) {
 			block_address dev_id = path[0];
 
 			dd_map::const_iterator it = dd_.find(path[0]);
 			if (it != dd_.end()) {
 				device_tree_detail::device_details const &d = it->second;
-				e_->begin_device(dev_id,
-						 d.mapped_blocks_,
-						 d.transaction_id_,
-						 d.creation_time_,
-						 d.snapshotted_time_);
+				if (dev_id_ == -1UL || dev_id == dev_id_) {
+					e_->begin_device(dev_id,
+							 d.mapped_blocks_,
+							 d.transaction_id_,
+							 d.creation_time_,
+							 d.snapshotted_time_);
 
-				emit_mappings(tree_root);
+					if (!skip_mappings_)
+						emit_mappings(tree_root);
 
-				e_->end_device();
+					e_->end_device();
+				}
 
 			} else if (!repair_) {
 				ostringstream msg;
@@ -209,6 +217,8 @@ namespace {
 		emitter::ptr e_;
 		dd_map const &dd_;
 		bool repair_;
+		block_address dev_id_;
+		bool skip_mappings_;
 		mapping_tree_detail::damage_visitor::ptr damage_policy_;
 	};
 }
@@ -216,7 +226,7 @@ namespace {
 //----------------------------------------------------------------
 
 void
-thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
+thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id)
 {
 	details_extractor de;
 	device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair));
@@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
 	{
 		mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair));
 		mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair));
+
+		mte.set_skip_mappings(skip_mappings);
+
+		if (dev_id >= 0)
+			mte.set_dev_id(dev_id);
 		walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy);
 	}
 
diff --git a/man8/thin_dump.8 b/man8/thin_dump.8
index 7a9f785..9e0bf9e 100644
--- a/man8/thin_dump.8
+++ b/man8/thin_dump.8
@@ -26,9 +26,19 @@ in order to put it back onto a metadata
 
 This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used.
 
-.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP".
+.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP".
 Print output in XML or human readable format.
-
+.sp
+The
+.B old
+format requires the --device-id option and emits in the
+following format, one record per line:
+.B offset:length:tdata_offset
+(hence, "o:L:d") with units in bytes.
+.sp
+The
+.B null
+format emits nothing and only walks the tree.
 .IP "\fB\-r, \-\-repair\fP".
 Repair the metadata whilst dumping it.
 
@@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#.
 See the thin provisioning target documentation on how to create or release
 a metadata snapshot and retrieve the block number from the kernel.
 
+.IP "\fB\-d, \-\-skip\-mappings\fP".
+Skip emission of the mappings.  This outputs nothing if format is
+either of
+.B old
+or
+.B null
+.
+
+.IP "\fB\-d, \-\-device\-id\fP".
+Specify the device_id to be dumped.
+
 .IP "\fB\-h, \-\-help\fP".
 Print help and exit.
 
diff --git a/Makefile.in b/Makefile.in
index e67b300..078bb53 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -73,6 +73,8 @@ SOURCE=\
 	persistent-data/validators.cc \
 	thin-provisioning/device_tree.cc \
 	thin-provisioning/human_readable_format.cc \
+	thin-provisioning/old_format.cc \
+	thin-provisioning/null_format.cc \
 	thin-provisioning/mapping_tree.cc \
 	thin-provisioning/metadata.cc \
 	thin-provisioning/metadata_checker.cc \
diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h
new file mode 100644
index 0000000..dba69e9
--- /dev/null
+++ b/thin-provisioning/old_format.h
@@ -0,0 +1,34 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools.  If not, see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef OLD_FORMAT_H
+#define OLD_FORMAT_H
+
+#include "emitter.h"
+
+#include <iosfwd>
+
+//----------------------------------------------------------------
+
+namespace thin_provisioning {
+	emitter::ptr create_old_emitter(std::ostream &out);
+}
+
+//----------------------------------------------------------------
+
+#endif
diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc
new file mode 100644
index 0000000..52056c8
--- /dev/null
+++ b/thin-provisioning/old_format.cc
@@ -0,0 +1,103 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools.  If not, see
+// <http://www.gnu.org/licenses/>.
+
+#include "old_format.h"
+
+#include <iostream>
+#include <boost/format.hpp>
+
+using namespace std;
+using namespace thin_provisioning;
+
+//----------------------------------------------------------------
+
+namespace {
+	template <typename T>
+	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
+		if (maybe)
+			out << *maybe;
+
+		return out;
+	}
+
+	class old_emitter : public emitter {
+	public:
+		old_emitter(ostream &out)
+			: out_(out) {
+		}
+
+		void begin_superblock(string const &uuid,
+				      uint64_t time,
+				      uint64_t trans_id,
+				      uint32_t data_block_size,
+				      uint64_t nr_data_blocks,
+				      boost::optional<uint64_t> metadata_snap) {
+			data_block_size_ = data_block_size;
+		}
+
+		void end_superblock() {
+		}
+
+		void begin_device(uint32_t dev_id,
+				  uint64_t mapped_blocks,
+				  uint64_t trans_id,
+				  uint64_t creation_time,
+				  uint64_t snap_time) {
+		}
+
+		void end_device() {
+		}
+
+		void begin_named_mapping(string const &name) {
+		}
+
+		void end_named_mapping() {
+		}
+
+		void identifier(string const &name) {
+		}
+
+		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
+			out_ << (data_block_size_ << 9)*origin_begin
+				<< ":" << (data_block_size_ << 9)*len
+				<< ":" << (data_block_size_ << 9)*data_begin
+				<< endl;
+		}
+
+		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
+			out_ << (data_block_size_ << 9)*origin_block
+				<< ":" << (data_block_size_ << 9)
+				<< ":" << (data_block_size_ << 9)*data_block
+			  << endl;
+		}
+
+	private:
+		ostream &out_;
+		uint64_t data_block_size_;
+	};
+}
+
+//----------------------------------------------------------------
+
+thin_provisioning::emitter::ptr
+thin_provisioning::create_old_emitter(ostream &out)
+{
+	return emitter::ptr(new old_emitter(out));
+}
+
+//----------------------------------------------------------------
diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
new file mode 100644
index 0000000..2688762
--- /dev/null
+++ b/thin-provisioning/null_format.h
@@ -0,0 +1,34 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools.  If not, see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef NULL_FORMAT_H
+#define NULL_FORMAT_H
+
+#include "emitter.h"
+
+#include <iosfwd>
+
+//----------------------------------------------------------------
+
+namespace thin_provisioning {
+	emitter::ptr create_null_emitter(std::ostream &out);
+}
+
+//----------------------------------------------------------------
+
+#endif
diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
new file mode 100644
index 0000000..b1baa7d
--- /dev/null
+++ b/thin-provisioning/null_format.cc
@@ -0,0 +1,92 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools.  If not, see
+// <http://www.gnu.org/licenses/>.
+
+#include "null_format.h"
+
+#include <iostream>
+
+using namespace std;
+using namespace thin_provisioning;
+
+//----------------------------------------------------------------
+
+namespace {
+	template <typename T>
+	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
+		if (maybe)
+			out << *maybe;
+
+		return out;
+	}
+
+	class null_emitter : public emitter {
+	public:
+		null_emitter(ostream &out)
+			: out_(out) {
+		}
+
+		void begin_superblock(string const &uuid,
+				      uint64_t time,
+				      uint64_t trans_id,
+				      uint32_t data_block_size,
+				      uint64_t nr_data_blocks,
+				      boost::optional<uint64_t> metadata_snap) {
+		}
+
+		void end_superblock() {
+		}
+
+		void begin_device(uint32_t dev_id,
+				  uint64_t mapped_blocks,
+				  uint64_t trans_id,
+				  uint64_t creation_time,
+				  uint64_t snap_time) {
+		}
+
+		void end_device() {
+		}
+
+		void begin_named_mapping(string const &name) {
+		}
+
+		void end_named_mapping() {
+		}
+
+		void identifier(string const &name) {
+		}
+
+		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
+		}
+
+		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
+		}
+
+	private:
+		ostream &out_;
+	};
+}
+
+//----------------------------------------------------------------
+
+thin_provisioning::emitter::ptr
+thin_provisioning::create_null_emitter(ostream &out)
+{
+	return emitter::ptr(new null_emitter(out));
+}
+
+//----------------------------------------------------------------
diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
new file mode 100644
index 0000000..2688762
--- /dev/null
+++ b/thin-provisioning/null_format.h
@@ -0,0 +1,34 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools.  If not, see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef NULL_FORMAT_H
+#define NULL_FORMAT_H
+
+#include "emitter.h"
+
+#include <iosfwd>
+
+//----------------------------------------------------------------
+
+namespace thin_provisioning {
+	emitter::ptr create_null_emitter(std::ostream &out);
+}
+
+//----------------------------------------------------------------
+
+#endif
diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
new file mode 100644
index 0000000..b1baa7d
--- /dev/null
+++ b/thin-provisioning/null_format.cc
@@ -0,0 +1,92 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools.  If not, see
+// <http://www.gnu.org/licenses/>.
+
+#include "null_format.h"
+
+#include <iostream>
+
+using namespace std;
+using namespace thin_provisioning;
+
+//----------------------------------------------------------------
+
+namespace {
+	template <typename T>
+	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
+		if (maybe)
+			out << *maybe;
+
+		return out;
+	}
+
+	class null_emitter : public emitter {
+	public:
+		null_emitter(ostream &out)
+			: out_(out) {
+		}
+
+		void begin_superblock(string const &uuid,
+				      uint64_t time,
+				      uint64_t trans_id,
+				      uint32_t data_block_size,
+				      uint64_t nr_data_blocks,
+				      boost::optional<uint64_t> metadata_snap) {
+		}
+
+		void end_superblock() {
+		}
+
+		void begin_device(uint32_t dev_id,
+				  uint64_t mapped_blocks,
+				  uint64_t trans_id,
+				  uint64_t creation_time,
+				  uint64_t snap_time) {
+		}
+
+		void end_device() {
+		}
+
+		void begin_named_mapping(string const &name) {
+		}
+
+		void end_named_mapping() {
+		}
+
+		void identifier(string const &name) {
+		}
+
+		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
+		}
+
+		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
+		}
+
+	private:
+		ostream &out_;
+	};
+}
+
+//----------------------------------------------------------------
+
+thin_provisioning::emitter::ptr
+thin_provisioning::create_null_emitter(ostream &out)
+{
+	return emitter::ptr(new null_emitter(out));
+}
+
+//----------------------------------------------------------------

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Thanos Makatos]

On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> Hi Joe,
>
> Please review the patch below when you have a moment.  I am interested in
> your feedback, and also interested in having this functionality merged
> upstream.  This was written against thin-provisioning-tools.git tag
> v0.5.6.
>
> We use thin_dump on live dm-thin metadata snapshots all the time.  In our
> case, we want to dump the XML for new (snapshot) volumes instead of
> dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> minutes instead of ~5 seconds for a single volume with --device-id.

I'm interested in extracting the mappings of a particular device, too.
In fact, I've implemented this (along with extracting the mappings in
binary format) and have only recently opened a PR:
https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
that we've replicated some work here, I'm not sure whether we're
supposed to send patches to this list or open PR on github.

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Eric Wheeler]

On Tue, 15 Mar 2016, Thanos Makatos wrote:

> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> > Hi Joe,
> >
> > Please review the patch below when you have a moment.  I am interested in
> > your feedback, and also interested in having this functionality merged
> > upstream.  This was written against thin-provisioning-tools.git tag
> > v0.5.6.
> >
> > We use thin_dump on live dm-thin metadata snapshots all the time.  In our
> > case, we want to dump the XML for new (snapshot) volumes instead of
> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> > minutes instead of ~5 seconds for a single volume with --device-id.
> 
> I'm interested in extracting the mappings of a particular device, too.
> In fact, I've implemented this (along with extracting the mappings in
> binary format) and have only recently opened a PR:
> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
> that we've replicated some work here, I'm not sure whether we're
> supposed to send patches to this list or open PR on github.

Interesting, it is neat to see a binary format too.

I think we need to come up with a consistent way to extend attributes 
being passed down into mapping_tree_emitter::visit() by way of 
thin_provisioning::metadata_dump().  What is preferred?

I noticed that caching::metadata_dump() has the same prototype, so do 
those need to remain compatable calls?  

Should we countinue to extend the arguments to metadata_dump() 
as Thanos and I have done, or, should we pop up the mte call to the caller 
and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let 
the caller configure the mte with setter functions or constructor calls?

I'm open to whatever---just making sure whatever we do stays clean and 
scales for the future.

--
Eric Wheeler

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Ming-Hung Tsai]

2016-03-16 1:51 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
> On Tue, 15 Mar 2016, Thanos Makatos wrote:
>
>> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
>> > Hi Joe,
>> >
>> > We use thin_dump on live dm-thin metadata snapshots all the time.  In our
>> > case, we want to dump the XML for new (snapshot) volumes instead of
>> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
>> > minutes instead of ~5 seconds for a single volume with --device-id.
>>
>> I'm interested in extracting the mappings of a particular device, too.
>> In fact, I've implemented this (along with extracting the mappings in
>> binary format) and have only recently opened a PR:
>> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
>> that we've replicated some work here, I'm not sure whether we're
>> supposed to send patches to this list or open PR on github.
>
> Interesting, it is neat to see a binary format too.
>
> I think we need to come up with a consistent way to extend attributes
> being passed down into mapping_tree_emitter::visit() by way of
> thin_provisioning::metadata_dump().  What is preferred?
>
> I noticed that caching::metadata_dump() has the same prototype, so do
> those need to remain compatable calls?
>
> Should we countinue to extend the arguments to metadata_dump()
> as Thanos and I have done, or, should we pop up the mte call to the caller
> and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let
> the caller configure the mte with setter functions or constructor calls?

Hi,

You are not alone, I did the same feature and used it for a long time.
My solution is similar to Thanos, I override thin_provisioning::metadata_dump()
to accept the additional dev_id:

void metadata_dump(metadata::ptr md, emitter::ptr e);
void metadata_dump(metadata::ptr md, emitter::ptr e, uint64_t dev_id);

Instead of passing dev_id to mapping_tree_emitter, I pass the dd_map
with only one device,
then let the mapping_tree_emitter to traverse the bottom-level tree directly:

void
thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e,
uint64_t dev_id)
{
    uint64_t key[1] = {dev_id};
    dev_tree::maybe_value single_mapping_tree_root =
md->mappings_top_level_->lookup(key);
    device_tree::maybe_value details = md->details_->lookup(key);

    // ... error handling

    details_extractor de;
    de.visit(dev_id, *details);  // de contains only one device

    e->begin_superblock(...);
    {
        mapping_tree_emitter mte(md, e, de.get_details(),
mapping_damage_policy(repair));
        std::vector<uint64_t> path(1, dev_id); //
btree_detail::btree_path is std::vector<uint64_t>
        mte.visit(path, *single_mapping_tree_root);
    }
    e->end_superblock();
}

But this looks still a bit messy: I should setup the visitors and
trees carefully to do what I want.

I also wrote some debugging feature for thin_dump. I can push the code
if you have interest.

1. thin_dump -b <metadata_dev> : Dump the device_details tree only. Do
not emit the data mappings.
(This could be replace by the new thin_ls)

output (no mappings):
<superblock ... >
  <device ... >
  </device>
  <device ... >
  </device>
</superblock>

2. thin_dump -r2 <metadata_dev>: dump the data mapping tree despite
the device_details tree was broken



Ming-Hung Tsai

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Eric Wheeler]

On Wed, 16 Mar 2016, Ming-Hung Tsai wrote:

> 2016-03-16 1:51 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
> > On Tue, 15 Mar 2016, Thanos Makatos wrote:
> >
> >> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> >> > Hi Joe,
> >> >
> >> > We use thin_dump on live dm-thin metadata snapshots all the time.  In our
> >> > case, we want to dump the XML for new (snapshot) volumes instead of
> >> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> >> > minutes instead of ~5 seconds for a single volume with --device-id.
> >>
> >> I'm interested in extracting the mappings of a particular device, too.
> >> In fact, I've implemented this (along with extracting the mappings in
> >> binary format) and have only recently opened a PR:
> >> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
> >> that we've replicated some work here, I'm not sure whether we're
> >> supposed to send patches to this list or open PR on github.
> >
> > Interesting, it is neat to see a binary format too.
> >
> > I think we need to come up with a consistent way to extend attributes
> > being passed down into mapping_tree_emitter::visit() by way of
> > thin_provisioning::metadata_dump().  What is preferred?
> >
> > I noticed that caching::metadata_dump() has the same prototype, so do
> > those need to remain compatable calls?
> >
> > Should we countinue to extend the arguments to metadata_dump()
> > as Thanos and I have done, or, should we pop up the mte call to the caller
> > and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let
> > the caller configure the mte with setter functions or constructor calls?
> 
> Hi,
> 
> You are not alone, I did the same feature and used it for a long time.
> My solution is similar to Thanos, I override thin_provisioning::metadata_dump()
> to accept the additional dev_id:
> 
> void metadata_dump(metadata::ptr md, emitter::ptr e);
> void metadata_dump(metadata::ptr md, emitter::ptr e, uint64_t dev_id);

That makes for 3 different implementations, we've all modified 
metadata_dump:


original:  ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);

Ming-Hung: void metadata_dump(metadata::ptr md, emitter::ptr e, [bool repair?]
+		uint64_t dev_id);

Thanos:    ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, 
+		const block_address *dev_id)

Me:        ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, 
+		bool skip_mappings, 
+		block_address dev_id)


--
Eric Wheeler



> 
> Instead of passing dev_id to mapping_tree_emitter, I pass the dd_map
> with only one device,
> then let the mapping_tree_emitter to traverse the bottom-level tree directly:
> 
> void
> thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e,
> uint64_t dev_id)
> {
>     uint64_t key[1] = {dev_id};
>     dev_tree::maybe_value single_mapping_tree_root =
> md->mappings_top_level_->lookup(key);
>     device_tree::maybe_value details = md->details_->lookup(key);
> 
>     // ... error handling
> 
>     details_extractor de;
>     de.visit(dev_id, *details);  // de contains only one device
> 
>     e->begin_superblock(...);
>     {
>         mapping_tree_emitter mte(md, e, de.get_details(),
> mapping_damage_policy(repair));
>         std::vector<uint64_t> path(1, dev_id); //
> btree_detail::btree_path is std::vector<uint64_t>
>         mte.visit(path, *single_mapping_tree_root);
>     }
>     e->end_superblock();
> }
> 
> But this looks still a bit messy: I should setup the visitors and
> trees carefully to do what I want.
> 
> I also wrote some debugging feature for thin_dump. I can push the code
> if you have interest.
> 
> 1. thin_dump -b <metadata_dev> : Dump the device_details tree only. Do
> not emit the data mappings.
> (This could be replace by the new thin_ls)
> 
> output (no mappings):
> <superblock ... >
>   <device ... >
>   </device>
>   <device ... >
>   </device>
> </superblock>
> 
> 2. thin_dump -r2 <metadata_dev>: dump the data mapping tree despite
> the device_details tree was broken
> 
> 
> 
> Ming-Hung Tsai
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
> 

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Joe Thornber]

On Tue, Mar 15, 2016 at 10:59:15AM +0000, Thanos Makatos wrote:
> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> > Hi Joe,
> >
> > Please review the patch below when you have a moment.  I am interested in
> > your feedback, and also interested in having this functionality merged
> > upstream.  This was written against thin-provisioning-tools.git tag
> > v0.5.6.
> >
> > We use thin_dump on live dm-thin metadata snapshots all the time.  In our
> > case, we want to dump the XML for new (snapshot) volumes instead of
> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> > minutes instead of ~5 seconds for a single volume with --device-id.
> 
> I'm interested in extracting the mappings of a particular device, too.
> In fact, I've implemented this (along with extracting the mappings in
> binary format) and have only recently opened a PR:
> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
> that we've replicated some work here, I'm not sure whether we're
> supposed to send patches to this list or open PR on github.

I've not forgotten you patches, just haven't got round to them yet.

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Joe Thornber]

If you're skipping the mappings does the new thin_ls provide enough
information for you?

- Joe

On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> Hi Joe,
> 
> Please review the patch below when you have a moment.  I am interested in 
> your feedback, and also interested in having this functionality merged 
> upstream.  This was written against thin-provisioning-tools.git tag 
> v0.5.6.
> 
> We use thin_dump on live dm-thin metadata snapshots all the time.  In our 
> case, we want to dump the XML for new (snapshot) volumes instead of 
> dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 
> minutes instead of ~5 seconds for a single volume with --device-id.
> 
> I started by adding --device-id to pass the `block_address dev_id` and 
> skip the call to emit_mappings() within mapping_tree_emitter::visit() 
> unless the dev_id matches --device-id as passed to thin_dump.  
> 
> It is sometimes nice to list all of the device supers without waiting for 
> emit_mappings() so I added the --skip-mappings option.  This allows you to 
> get the dev_id without reading the mappings:
> 	thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta
> 
> Like `bool repair`, I added skip_mappings and dev_id as arguments to 
> thin_provisioning::metadata_dump() and passed them down to 
> mapping_tree_emitter with new members set_skip_mappings() and set_dev_id() 
> to set private attributes (default values are assigned in 
> metadata_dumper.h in case of backward compatible calls).
> 
> We work with the device metadata in a simplified format, and without 
> superblock information:
> 
> 	origin_offset:length:tdata_offset (hence, "o:L:d" or "old") 
> 
> Therefore, the output --format 'old' was added.  I added the format 'null' 
> as well for benchmarking purposes so the ostream needn't write large 
> volumes of text to /dev/null.  
> 
> Benchmarks of the various formats on our sample metadata snapshot on an 
> idle machine:
> 
> for i in xml human_readable old null; do
> 	time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null
> done
> 
> 	real		user		sys
> xml	29:01.27	25:35.87	01:54.49
> human	27:39.81	24:12.90	01:52.39
> old	27:07.71	23:40.64	01:52.97
> null	23:39.38	21:17.22	00:49.04
> 
> I have this as a branch in bitbucket if you prefer to see it online:
>   
> https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id
>   or
> git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id
> 
> 
> --
> Eric Wheeler
> 
> 
> diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc
> index 191acb5..7df5d85 100644
> --- a/thin-provisioning/thin_dump.cc
> +++ b/thin-provisioning/thin_dump.cc
> @@ -22,6 +22,8 @@
>  #include <libgen.h>
>  
>  #include "human_readable_format.h"
> +#include "old_format.h"
> +#include "null_format.h"
>  #include "metadata_dumper.h"
>  #include "metadata.h"
>  #include "xml_format.h"
> @@ -36,6 +38,8 @@ using namespace thin_provisioning;
>  struct flags {
>  	bool find_metadata_snap;
>  	bool repair;
> +	block_address dev_id;
> +	bool skip_mappings;
>  };
>  
>  namespace {
> @@ -49,12 +53,16 @@ namespace {
>  				e = create_xml_emitter(out);
>  			else if (format == "human_readable")
>  				e = create_human_readable_emitter(out);
> +			else if (format == "old")
> +				e = create_old_emitter(out);
> +			else if (format == "null")
> +				e = create_null_emitter(out);
>  			else {
>  				cerr << "unknown format '" << format << "'" << endl;
>  				exit(1);
>  			}
>  
> -			metadata_dump(md, e, flags.repair);
> +			metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id);
>  
>  		} catch (std::exception &e) {
>  			cerr << e.what() << endl;
> @@ -77,10 +85,12 @@ namespace {
>  		out << "Usage: " << cmd << " [options] {device|file}" << endl
>  		    << "Options:" << endl
>  		    << "  {-h|--help}" << endl
> -		    << "  {-f|--format} {xml|human_readable}" << endl
> +		    << "  {-f|--format} {xml|human_readable|old|null}" << endl
>  		    << "  {-r|--repair}" << endl
>  		    << "  {-m|--metadata-snap} [block#]" << endl
>  		    << "  {-o <xml file>}" << endl
> +		    << "  {-d device_id}" << endl
> +		    << "  {-s|--skip-mappings}" << endl
>  		    << "  {-V|--version}" << endl;
>  	}
>  }
> @@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv)
>  {
>  	int c;
>  	char const *output = NULL;
> -	const char shortopts[] = "hm::o:f:rV";
> +	const char shortopts[] = "hm::o:sd:f:rV";
>  	char *end_ptr;
>  	string format = "xml";
>  	block_address metadata_snap = 0;
>  	struct flags flags;
> -	flags.find_metadata_snap = flags.repair = false;
> +	flags.find_metadata_snap = flags.repair = flags.skip_mappings = false;
> +	flags.dev_id = -1;
>  
>  	const struct option longopts[] = {
>  		{ "help", no_argument, NULL, 'h'},
>  		{ "metadata-snap", optional_argument, NULL, 'm' },
>  		{ "output", required_argument, NULL, 'o'},
> +		{ "skip-mappings", no_argument, NULL, 's'},
> +		{ "device-id", required_argument, NULL, 'd'},
>  		{ "format", required_argument, NULL, 'f' },
>  		{ "repair", no_argument, NULL, 'r'},
>  		{ "version", no_argument, NULL, 'V'},
> @@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv)
>  			flags.repair = true;
>  			break;
>  
> +		case 's':
> +			flags.skip_mappings = true;
> +			break;
> +
> +		case 'd':
> +			flags.dev_id = strtoull(optarg, &end_ptr, 10);
> +			if (end_ptr == optarg) {
> +				cerr << "couldn't parse <device_id>" << endl;
> +				usage(cerr, basename(argv[0]));
> +				return 1;
> +			}
> +			break;
> +
>  		case 'm':
>  			if (optarg) {
>  				metadata_snap = strtoull(optarg, &end_ptr, 10);
> @@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv)
>  		}
>  	}
>  
> +	if (format != "old" && flags.dev_id < 0) {
> +		cerr << "Output format 'old' must specify --device-id" << endl;
> +		return 1;
> +	}
> +
> +
>  	if (argc == optind) {
>  		cerr << "No input file provided." << endl;
>  		usage(cerr, basename(argv[0]));
> diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h
> index c96d22e..9d9814e 100644
> --- a/thin-provisioning/metadata_dumper.h
> +++ b/thin-provisioning/metadata_dumper.h
> @@ -28,7 +28,7 @@ namespace thin_provisioning {
>  	// Set the @repair flag if your metadata is corrupt, and you'd like
>  	// the dumper to do it's best to recover info.  If not set, any
>  	// corruption encountered will cause an exception to be thrown.
> -	void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);
> +	void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1);
>  }
>  
>  //----------------------------------------------------------------
> diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc
> index db656ee..55cbae8 100644
> --- a/thin-provisioning/metadata_dumper.cc
> +++ b/thin-provisioning/metadata_dumper.cc
> @@ -171,23 +171,31 @@ namespace {
>  			  dd_(dd),
>  			  repair_(repair),
>  			  damage_policy_(damage_policy) {
> +				  dev_id_ = -1;
> +				  skip_mappings_ = false;
>  		}
>  
> +		void set_dev_id(block_address dev_id) { dev_id_ = dev_id; }
> +		void set_skip_mappings(bool skip) { skip_mappings_ = skip; }
> +
>  		void visit(btree_path const &path, block_address tree_root) {
>  			block_address dev_id = path[0];
>  
>  			dd_map::const_iterator it = dd_.find(path[0]);
>  			if (it != dd_.end()) {
>  				device_tree_detail::device_details const &d = it->second;
> -				e_->begin_device(dev_id,
> -						 d.mapped_blocks_,
> -						 d.transaction_id_,
> -						 d.creation_time_,
> -						 d.snapshotted_time_);
> +				if (dev_id_ == -1UL || dev_id == dev_id_) {
> +					e_->begin_device(dev_id,
> +							 d.mapped_blocks_,
> +							 d.transaction_id_,
> +							 d.creation_time_,
> +							 d.snapshotted_time_);
>  
> -				emit_mappings(tree_root);
> +					if (!skip_mappings_)
> +						emit_mappings(tree_root);
>  
> -				e_->end_device();
> +					e_->end_device();
> +				}
>  
>  			} else if (!repair_) {
>  				ostringstream msg;
> @@ -209,6 +217,8 @@ namespace {
>  		emitter::ptr e_;
>  		dd_map const &dd_;
>  		bool repair_;
> +		block_address dev_id_;
> +		bool skip_mappings_;
>  		mapping_tree_detail::damage_visitor::ptr damage_policy_;
>  	};
>  }
> @@ -216,7 +226,7 @@ namespace {
>  //----------------------------------------------------------------
>  
>  void
> -thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
> +thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id)
>  {
>  	details_extractor de;
>  	device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair));
> @@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
>  	{
>  		mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair));
>  		mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair));
> +
> +		mte.set_skip_mappings(skip_mappings);
> +
> +		if (dev_id >= 0)
> +			mte.set_dev_id(dev_id);
>  		walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy);
>  	}
>  
> diff --git a/man8/thin_dump.8 b/man8/thin_dump.8
> index 7a9f785..9e0bf9e 100644
> --- a/man8/thin_dump.8
> +++ b/man8/thin_dump.8
> @@ -26,9 +26,19 @@ in order to put it back onto a metadata
>  
>  This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used.
>  
> -.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP".
> +.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP".
>  Print output in XML or human readable format.
> -
> +.sp
> +The
> +.B old
> +format requires the --device-id option and emits in the
> +following format, one record per line:
> +.B offset:length:tdata_offset
> +(hence, "o:L:d") with units in bytes.
> +.sp
> +The
> +.B null
> +format emits nothing and only walks the tree.
>  .IP "\fB\-r, \-\-repair\fP".
>  Repair the metadata whilst dumping it.
>  
> @@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#.
>  See the thin provisioning target documentation on how to create or release
>  a metadata snapshot and retrieve the block number from the kernel.
>  
> +.IP "\fB\-d, \-\-skip\-mappings\fP".
> +Skip emission of the mappings.  This outputs nothing if format is
> +either of
> +.B old
> +or
> +.B null
> +.
> +
> +.IP "\fB\-d, \-\-device\-id\fP".
> +Specify the device_id to be dumped.
> +
>  .IP "\fB\-h, \-\-help\fP".
>  Print help and exit.
>  
> diff --git a/Makefile.in b/Makefile.in
> index e67b300..078bb53 100644
> --- a/Makefile.in
> +++ b/Makefile.in
> @@ -73,6 +73,8 @@ SOURCE=\
>  	persistent-data/validators.cc \
>  	thin-provisioning/device_tree.cc \
>  	thin-provisioning/human_readable_format.cc \
> +	thin-provisioning/old_format.cc \
> +	thin-provisioning/null_format.cc \
>  	thin-provisioning/mapping_tree.cc \
>  	thin-provisioning/metadata.cc \
>  	thin-provisioning/metadata_checker.cc \
> diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h
> new file mode 100644
> index 0000000..dba69e9
> --- /dev/null
> +++ b/thin-provisioning/old_format.h
> @@ -0,0 +1,34 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools.  If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef OLD_FORMAT_H
> +#define OLD_FORMAT_H
> +
> +#include "emitter.h"
> +
> +#include <iosfwd>
> +
> +//----------------------------------------------------------------
> +
> +namespace thin_provisioning {
> +	emitter::ptr create_old_emitter(std::ostream &out);
> +}
> +
> +//----------------------------------------------------------------
> +
> +#endif
> diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc
> new file mode 100644
> index 0000000..52056c8
> --- /dev/null
> +++ b/thin-provisioning/old_format.cc
> @@ -0,0 +1,103 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools.  If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "old_format.h"
> +
> +#include <iostream>
> +#include <boost/format.hpp>
> +
> +using namespace std;
> +using namespace thin_provisioning;
> +
> +//----------------------------------------------------------------
> +
> +namespace {
> +	template <typename T>
> +	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> +		if (maybe)
> +			out << *maybe;
> +
> +		return out;
> +	}
> +
> +	class old_emitter : public emitter {
> +	public:
> +		old_emitter(ostream &out)
> +			: out_(out) {
> +		}
> +
> +		void begin_superblock(string const &uuid,
> +				      uint64_t time,
> +				      uint64_t trans_id,
> +				      uint32_t data_block_size,
> +				      uint64_t nr_data_blocks,
> +				      boost::optional<uint64_t> metadata_snap) {
> +			data_block_size_ = data_block_size;
> +		}
> +
> +		void end_superblock() {
> +		}
> +
> +		void begin_device(uint32_t dev_id,
> +				  uint64_t mapped_blocks,
> +				  uint64_t trans_id,
> +				  uint64_t creation_time,
> +				  uint64_t snap_time) {
> +		}
> +
> +		void end_device() {
> +		}
> +
> +		void begin_named_mapping(string const &name) {
> +		}
> +
> +		void end_named_mapping() {
> +		}
> +
> +		void identifier(string const &name) {
> +		}
> +
> +		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> +			out_ << (data_block_size_ << 9)*origin_begin
> +				<< ":" << (data_block_size_ << 9)*len
> +				<< ":" << (data_block_size_ << 9)*data_begin
> +				<< endl;
> +		}
> +
> +		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> +			out_ << (data_block_size_ << 9)*origin_block
> +				<< ":" << (data_block_size_ << 9)
> +				<< ":" << (data_block_size_ << 9)*data_block
> +			  << endl;
> +		}
> +
> +	private:
> +		ostream &out_;
> +		uint64_t data_block_size_;
> +	};
> +}
> +
> +//----------------------------------------------------------------
> +
> +thin_provisioning::emitter::ptr
> +thin_provisioning::create_old_emitter(ostream &out)
> +{
> +	return emitter::ptr(new old_emitter(out));
> +}
> +
> +//----------------------------------------------------------------
> diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> new file mode 100644
> index 0000000..2688762
> --- /dev/null
> +++ b/thin-provisioning/null_format.h
> @@ -0,0 +1,34 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools.  If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef NULL_FORMAT_H
> +#define NULL_FORMAT_H
> +
> +#include "emitter.h"
> +
> +#include <iosfwd>
> +
> +//----------------------------------------------------------------
> +
> +namespace thin_provisioning {
> +	emitter::ptr create_null_emitter(std::ostream &out);
> +}
> +
> +//----------------------------------------------------------------
> +
> +#endif
> diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> new file mode 100644
> index 0000000..b1baa7d
> --- /dev/null
> +++ b/thin-provisioning/null_format.cc
> @@ -0,0 +1,92 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools.  If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "null_format.h"
> +
> +#include <iostream>
> +
> +using namespace std;
> +using namespace thin_provisioning;
> +
> +//----------------------------------------------------------------
> +
> +namespace {
> +	template <typename T>
> +	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> +		if (maybe)
> +			out << *maybe;
> +
> +		return out;
> +	}
> +
> +	class null_emitter : public emitter {
> +	public:
> +		null_emitter(ostream &out)
> +			: out_(out) {
> +		}
> +
> +		void begin_superblock(string const &uuid,
> +				      uint64_t time,
> +				      uint64_t trans_id,
> +				      uint32_t data_block_size,
> +				      uint64_t nr_data_blocks,
> +				      boost::optional<uint64_t> metadata_snap) {
> +		}
> +
> +		void end_superblock() {
> +		}
> +
> +		void begin_device(uint32_t dev_id,
> +				  uint64_t mapped_blocks,
> +				  uint64_t trans_id,
> +				  uint64_t creation_time,
> +				  uint64_t snap_time) {
> +		}
> +
> +		void end_device() {
> +		}
> +
> +		void begin_named_mapping(string const &name) {
> +		}
> +
> +		void end_named_mapping() {
> +		}
> +
> +		void identifier(string const &name) {
> +		}
> +
> +		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> +		}
> +
> +		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> +		}
> +
> +	private:
> +		ostream &out_;
> +	};
> +}
> +
> +//----------------------------------------------------------------
> +
> +thin_provisioning::emitter::ptr
> +thin_provisioning::create_null_emitter(ostream &out)
> +{
> +	return emitter::ptr(new null_emitter(out));
> +}
> +
> +//----------------------------------------------------------------
> diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> new file mode 100644
> index 0000000..2688762
> --- /dev/null
> +++ b/thin-provisioning/null_format.h
> @@ -0,0 +1,34 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools.  If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef NULL_FORMAT_H
> +#define NULL_FORMAT_H
> +
> +#include "emitter.h"
> +
> +#include <iosfwd>
> +
> +//----------------------------------------------------------------
> +
> +namespace thin_provisioning {
> +	emitter::ptr create_null_emitter(std::ostream &out);
> +}
> +
> +//----------------------------------------------------------------
> +
> +#endif
> diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> new file mode 100644
> index 0000000..b1baa7d
> --- /dev/null
> +++ b/thin-provisioning/null_format.cc
> @@ -0,0 +1,92 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools.  If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "null_format.h"
> +
> +#include <iostream>
> +
> +using namespace std;
> +using namespace thin_provisioning;
> +
> +//----------------------------------------------------------------
> +
> +namespace {
> +	template <typename T>
> +	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> +		if (maybe)
> +			out << *maybe;
> +
> +		return out;
> +	}
> +
> +	class null_emitter : public emitter {
> +	public:
> +		null_emitter(ostream &out)
> +			: out_(out) {
> +		}
> +
> +		void begin_superblock(string const &uuid,
> +				      uint64_t time,
> +				      uint64_t trans_id,
> +				      uint32_t data_block_size,
> +				      uint64_t nr_data_blocks,
> +				      boost::optional<uint64_t> metadata_snap) {
> +		}
> +
> +		void end_superblock() {
> +		}
> +
> +		void begin_device(uint32_t dev_id,
> +				  uint64_t mapped_blocks,
> +				  uint64_t trans_id,
> +				  uint64_t creation_time,
> +				  uint64_t snap_time) {
> +		}
> +
> +		void end_device() {
> +		}
> +
> +		void begin_named_mapping(string const &name) {
> +		}
> +
> +		void end_named_mapping() {
> +		}
> +
> +		void identifier(string const &name) {
> +		}
> +
> +		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> +		}
> +
> +		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> +		}
> +
> +	private:
> +		ostream &out_;
> +	};
> +}
> +
> +//----------------------------------------------------------------
> +
> +thin_provisioning::emitter::ptr
> +thin_provisioning::create_null_emitter(ostream &out)
> +{
> +	return emitter::ptr(new null_emitter(out));
> +}
> +
> +//----------------------------------------------------------------
> 

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Eric Wheeler]

On Thu, 17 Mar 2016, Joe Thornber wrote:

> If you're skipping the mappings does the new thin_ls provide enough
> information for you?

Our primary need is the ability to specify --device-id in the o:L:d output 
format (old_format.cc) with mappings.  Our nightly dumps now take 50 
seconds instead of 40 minutes since we only dump the dev_id's that we 
need.

As for thin_ls, we're still in the 0.5.x branch, I couldn't get 0.6.x to 
build on el7:

# git checkout v0.6.2-rc6
Previous HEAD position was e46bdfd... v0.6.1
HEAD is now at afb71d8... bump version

# ./configure 
[...]

# make
    [CXX] base/application.cc
g++: error: @LFS_FLAGS@: No such file or directory
make: *** [base/application.o] Error 1


Am I missing something trivial here?



--
Eric Wheeler

> 
> - Joe
> 
> On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> > Hi Joe,
> > 
> > Please review the patch below when you have a moment.  I am interested in 
> > your feedback, and also interested in having this functionality merged 
> > upstream.  This was written against thin-provisioning-tools.git tag 
> > v0.5.6.
> > 
> > We use thin_dump on live dm-thin metadata snapshots all the time.  In our 
> > case, we want to dump the XML for new (snapshot) volumes instead of 
> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 
> > minutes instead of ~5 seconds for a single volume with --device-id.
> > 
> > I started by adding --device-id to pass the `block_address dev_id` and 
> > skip the call to emit_mappings() within mapping_tree_emitter::visit() 
> > unless the dev_id matches --device-id as passed to thin_dump.  
> > 
> > It is sometimes nice to list all of the device supers without waiting for 
> > emit_mappings() so I added the --skip-mappings option.  This allows you to 
> > get the dev_id without reading the mappings:
> > 	thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta
> > 
> > Like `bool repair`, I added skip_mappings and dev_id as arguments to 
> > thin_provisioning::metadata_dump() and passed them down to 
> > mapping_tree_emitter with new members set_skip_mappings() and set_dev_id() 
> > to set private attributes (default values are assigned in 
> > metadata_dumper.h in case of backward compatible calls).
> > 
> > We work with the device metadata in a simplified format, and without 
> > superblock information:
> > 
> > 	origin_offset:length:tdata_offset (hence, "o:L:d" or "old") 
> > 
> > Therefore, the output --format 'old' was added.  I added the format 'null' 
> > as well for benchmarking purposes so the ostream needn't write large 
> > volumes of text to /dev/null.  
> > 
> > Benchmarks of the various formats on our sample metadata snapshot on an 
> > idle machine:
> > 
> > for i in xml human_readable old null; do
> > 	time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null
> > done
> > 
> > 	real		user		sys
> > xml	29:01.27	25:35.87	01:54.49
> > human	27:39.81	24:12.90	01:52.39
> > old	27:07.71	23:40.64	01:52.97
> > null	23:39.38	21:17.22	00:49.04
> > 
> > I have this as a branch in bitbucket if you prefer to see it online:
> >   
> > https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id
> >   or
> > git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id
> > 
> > 
> > --
> > Eric Wheeler
> > 
> > 
> > diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc
> > index 191acb5..7df5d85 100644
> > --- a/thin-provisioning/thin_dump.cc
> > +++ b/thin-provisioning/thin_dump.cc
> > @@ -22,6 +22,8 @@
> >  #include <libgen.h>
> >  
> >  #include "human_readable_format.h"
> > +#include "old_format.h"
> > +#include "null_format.h"
> >  #include "metadata_dumper.h"
> >  #include "metadata.h"
> >  #include "xml_format.h"
> > @@ -36,6 +38,8 @@ using namespace thin_provisioning;
> >  struct flags {
> >  	bool find_metadata_snap;
> >  	bool repair;
> > +	block_address dev_id;
> > +	bool skip_mappings;
> >  };
> >  
> >  namespace {
> > @@ -49,12 +53,16 @@ namespace {
> >  				e = create_xml_emitter(out);
> >  			else if (format == "human_readable")
> >  				e = create_human_readable_emitter(out);
> > +			else if (format == "old")
> > +				e = create_old_emitter(out);
> > +			else if (format == "null")
> > +				e = create_null_emitter(out);
> >  			else {
> >  				cerr << "unknown format '" << format << "'" << endl;
> >  				exit(1);
> >  			}
> >  
> > -			metadata_dump(md, e, flags.repair);
> > +			metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id);
> >  
> >  		} catch (std::exception &e) {
> >  			cerr << e.what() << endl;
> > @@ -77,10 +85,12 @@ namespace {
> >  		out << "Usage: " << cmd << " [options] {device|file}" << endl
> >  		    << "Options:" << endl
> >  		    << "  {-h|--help}" << endl
> > -		    << "  {-f|--format} {xml|human_readable}" << endl
> > +		    << "  {-f|--format} {xml|human_readable|old|null}" << endl
> >  		    << "  {-r|--repair}" << endl
> >  		    << "  {-m|--metadata-snap} [block#]" << endl
> >  		    << "  {-o <xml file>}" << endl
> > +		    << "  {-d device_id}" << endl
> > +		    << "  {-s|--skip-mappings}" << endl
> >  		    << "  {-V|--version}" << endl;
> >  	}
> >  }
> > @@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv)
> >  {
> >  	int c;
> >  	char const *output = NULL;
> > -	const char shortopts[] = "hm::o:f:rV";
> > +	const char shortopts[] = "hm::o:sd:f:rV";
> >  	char *end_ptr;
> >  	string format = "xml";
> >  	block_address metadata_snap = 0;
> >  	struct flags flags;
> > -	flags.find_metadata_snap = flags.repair = false;
> > +	flags.find_metadata_snap = flags.repair = flags.skip_mappings = false;
> > +	flags.dev_id = -1;
> >  
> >  	const struct option longopts[] = {
> >  		{ "help", no_argument, NULL, 'h'},
> >  		{ "metadata-snap", optional_argument, NULL, 'm' },
> >  		{ "output", required_argument, NULL, 'o'},
> > +		{ "skip-mappings", no_argument, NULL, 's'},
> > +		{ "device-id", required_argument, NULL, 'd'},
> >  		{ "format", required_argument, NULL, 'f' },
> >  		{ "repair", no_argument, NULL, 'r'},
> >  		{ "version", no_argument, NULL, 'V'},
> > @@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv)
> >  			flags.repair = true;
> >  			break;
> >  
> > +		case 's':
> > +			flags.skip_mappings = true;
> > +			break;
> > +
> > +		case 'd':
> > +			flags.dev_id = strtoull(optarg, &end_ptr, 10);
> > +			if (end_ptr == optarg) {
> > +				cerr << "couldn't parse <device_id>" << endl;
> > +				usage(cerr, basename(argv[0]));
> > +				return 1;
> > +			}
> > +			break;
> > +
> >  		case 'm':
> >  			if (optarg) {
> >  				metadata_snap = strtoull(optarg, &end_ptr, 10);
> > @@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv)
> >  		}
> >  	}
> >  
> > +	if (format != "old" && flags.dev_id < 0) {
> > +		cerr << "Output format 'old' must specify --device-id" << endl;
> > +		return 1;
> > +	}
> > +
> > +
> >  	if (argc == optind) {
> >  		cerr << "No input file provided." << endl;
> >  		usage(cerr, basename(argv[0]));
> > diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h
> > index c96d22e..9d9814e 100644
> > --- a/thin-provisioning/metadata_dumper.h
> > +++ b/thin-provisioning/metadata_dumper.h
> > @@ -28,7 +28,7 @@ namespace thin_provisioning {
> >  	// Set the @repair flag if your metadata is corrupt, and you'd like
> >  	// the dumper to do it's best to recover info.  If not set, any
> >  	// corruption encountered will cause an exception to be thrown.
> > -	void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);
> > +	void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1);
> >  }
> >  
> >  //----------------------------------------------------------------
> > diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc
> > index db656ee..55cbae8 100644
> > --- a/thin-provisioning/metadata_dumper.cc
> > +++ b/thin-provisioning/metadata_dumper.cc
> > @@ -171,23 +171,31 @@ namespace {
> >  			  dd_(dd),
> >  			  repair_(repair),
> >  			  damage_policy_(damage_policy) {
> > +				  dev_id_ = -1;
> > +				  skip_mappings_ = false;
> >  		}
> >  
> > +		void set_dev_id(block_address dev_id) { dev_id_ = dev_id; }
> > +		void set_skip_mappings(bool skip) { skip_mappings_ = skip; }
> > +
> >  		void visit(btree_path const &path, block_address tree_root) {
> >  			block_address dev_id = path[0];
> >  
> >  			dd_map::const_iterator it = dd_.find(path[0]);
> >  			if (it != dd_.end()) {
> >  				device_tree_detail::device_details const &d = it->second;
> > -				e_->begin_device(dev_id,
> > -						 d.mapped_blocks_,
> > -						 d.transaction_id_,
> > -						 d.creation_time_,
> > -						 d.snapshotted_time_);
> > +				if (dev_id_ == -1UL || dev_id == dev_id_) {
> > +					e_->begin_device(dev_id,
> > +							 d.mapped_blocks_,
> > +							 d.transaction_id_,
> > +							 d.creation_time_,
> > +							 d.snapshotted_time_);
> >  
> > -				emit_mappings(tree_root);
> > +					if (!skip_mappings_)
> > +						emit_mappings(tree_root);
> >  
> > -				e_->end_device();
> > +					e_->end_device();
> > +				}
> >  
> >  			} else if (!repair_) {
> >  				ostringstream msg;
> > @@ -209,6 +217,8 @@ namespace {
> >  		emitter::ptr e_;
> >  		dd_map const &dd_;
> >  		bool repair_;
> > +		block_address dev_id_;
> > +		bool skip_mappings_;
> >  		mapping_tree_detail::damage_visitor::ptr damage_policy_;
> >  	};
> >  }
> > @@ -216,7 +226,7 @@ namespace {
> >  //----------------------------------------------------------------
> >  
> >  void
> > -thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
> > +thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id)
> >  {
> >  	details_extractor de;
> >  	device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair));
> > @@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
> >  	{
> >  		mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair));
> >  		mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair));
> > +
> > +		mte.set_skip_mappings(skip_mappings);
> > +
> > +		if (dev_id >= 0)
> > +			mte.set_dev_id(dev_id);
> >  		walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy);
> >  	}
> >  
> > diff --git a/man8/thin_dump.8 b/man8/thin_dump.8
> > index 7a9f785..9e0bf9e 100644
> > --- a/man8/thin_dump.8
> > +++ b/man8/thin_dump.8
> > @@ -26,9 +26,19 @@ in order to put it back onto a metadata
> >  
> >  This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used.
> >  
> > -.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP".
> > +.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP".
> >  Print output in XML or human readable format.
> > -
> > +.sp
> > +The
> > +.B old
> > +format requires the --device-id option and emits in the
> > +following format, one record per line:
> > +.B offset:length:tdata_offset
> > +(hence, "o:L:d") with units in bytes.
> > +.sp
> > +The
> > +.B null
> > +format emits nothing and only walks the tree.
> >  .IP "\fB\-r, \-\-repair\fP".
> >  Repair the metadata whilst dumping it.
> >  
> > @@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#.
> >  See the thin provisioning target documentation on how to create or release
> >  a metadata snapshot and retrieve the block number from the kernel.
> >  
> > +.IP "\fB\-d, \-\-skip\-mappings\fP".
> > +Skip emission of the mappings.  This outputs nothing if format is
> > +either of
> > +.B old
> > +or
> > +.B null
> > +.
> > +
> > +.IP "\fB\-d, \-\-device\-id\fP".
> > +Specify the device_id to be dumped.
> > +
> >  .IP "\fB\-h, \-\-help\fP".
> >  Print help and exit.
> >  
> > diff --git a/Makefile.in b/Makefile.in
> > index e67b300..078bb53 100644
> > --- a/Makefile.in
> > +++ b/Makefile.in
> > @@ -73,6 +73,8 @@ SOURCE=\
> >  	persistent-data/validators.cc \
> >  	thin-provisioning/device_tree.cc \
> >  	thin-provisioning/human_readable_format.cc \
> > +	thin-provisioning/old_format.cc \
> > +	thin-provisioning/null_format.cc \
> >  	thin-provisioning/mapping_tree.cc \
> >  	thin-provisioning/metadata.cc \
> >  	thin-provisioning/metadata_checker.cc \
> > diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h
> > new file mode 100644
> > index 0000000..dba69e9
> > --- /dev/null
> > +++ b/thin-provisioning/old_format.h
> > @@ -0,0 +1,34 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools.  If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#ifndef OLD_FORMAT_H
> > +#define OLD_FORMAT_H
> > +
> > +#include "emitter.h"
> > +
> > +#include <iosfwd>
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace thin_provisioning {
> > +	emitter::ptr create_old_emitter(std::ostream &out);
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +#endif
> > diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc
> > new file mode 100644
> > index 0000000..52056c8
> > --- /dev/null
> > +++ b/thin-provisioning/old_format.cc
> > @@ -0,0 +1,103 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools.  If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#include "old_format.h"
> > +
> > +#include <iostream>
> > +#include <boost/format.hpp>
> > +
> > +using namespace std;
> > +using namespace thin_provisioning;
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace {
> > +	template <typename T>
> > +	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> > +		if (maybe)
> > +			out << *maybe;
> > +
> > +		return out;
> > +	}
> > +
> > +	class old_emitter : public emitter {
> > +	public:
> > +		old_emitter(ostream &out)
> > +			: out_(out) {
> > +		}
> > +
> > +		void begin_superblock(string const &uuid,
> > +				      uint64_t time,
> > +				      uint64_t trans_id,
> > +				      uint32_t data_block_size,
> > +				      uint64_t nr_data_blocks,
> > +				      boost::optional<uint64_t> metadata_snap) {
> > +			data_block_size_ = data_block_size;
> > +		}
> > +
> > +		void end_superblock() {
> > +		}
> > +
> > +		void begin_device(uint32_t dev_id,
> > +				  uint64_t mapped_blocks,
> > +				  uint64_t trans_id,
> > +				  uint64_t creation_time,
> > +				  uint64_t snap_time) {
> > +		}
> > +
> > +		void end_device() {
> > +		}
> > +
> > +		void begin_named_mapping(string const &name) {
> > +		}
> > +
> > +		void end_named_mapping() {
> > +		}
> > +
> > +		void identifier(string const &name) {
> > +		}
> > +
> > +		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> > +			out_ << (data_block_size_ << 9)*origin_begin
> > +				<< ":" << (data_block_size_ << 9)*len
> > +				<< ":" << (data_block_size_ << 9)*data_begin
> > +				<< endl;
> > +		}
> > +
> > +		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> > +			out_ << (data_block_size_ << 9)*origin_block
> > +				<< ":" << (data_block_size_ << 9)
> > +				<< ":" << (data_block_size_ << 9)*data_block
> > +			  << endl;
> > +		}
> > +
> > +	private:
> > +		ostream &out_;
> > +		uint64_t data_block_size_;
> > +	};
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +thin_provisioning::emitter::ptr
> > +thin_provisioning::create_old_emitter(ostream &out)
> > +{
> > +	return emitter::ptr(new old_emitter(out));
> > +}
> > +
> > +//----------------------------------------------------------------
> > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> > new file mode 100644
> > index 0000000..2688762
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.h
> > @@ -0,0 +1,34 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools.  If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#ifndef NULL_FORMAT_H
> > +#define NULL_FORMAT_H
> > +
> > +#include "emitter.h"
> > +
> > +#include <iosfwd>
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace thin_provisioning {
> > +	emitter::ptr create_null_emitter(std::ostream &out);
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +#endif
> > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> > new file mode 100644
> > index 0000000..b1baa7d
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.cc
> > @@ -0,0 +1,92 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools.  If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#include "null_format.h"
> > +
> > +#include <iostream>
> > +
> > +using namespace std;
> > +using namespace thin_provisioning;
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace {
> > +	template <typename T>
> > +	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> > +		if (maybe)
> > +			out << *maybe;
> > +
> > +		return out;
> > +	}
> > +
> > +	class null_emitter : public emitter {
> > +	public:
> > +		null_emitter(ostream &out)
> > +			: out_(out) {
> > +		}
> > +
> > +		void begin_superblock(string const &uuid,
> > +				      uint64_t time,
> > +				      uint64_t trans_id,
> > +				      uint32_t data_block_size,
> > +				      uint64_t nr_data_blocks,
> > +				      boost::optional<uint64_t> metadata_snap) {
> > +		}
> > +
> > +		void end_superblock() {
> > +		}
> > +
> > +		void begin_device(uint32_t dev_id,
> > +				  uint64_t mapped_blocks,
> > +				  uint64_t trans_id,
> > +				  uint64_t creation_time,
> > +				  uint64_t snap_time) {
> > +		}
> > +
> > +		void end_device() {
> > +		}
> > +
> > +		void begin_named_mapping(string const &name) {
> > +		}
> > +
> > +		void end_named_mapping() {
> > +		}
> > +
> > +		void identifier(string const &name) {
> > +		}
> > +
> > +		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> > +		}
> > +
> > +		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> > +		}
> > +
> > +	private:
> > +		ostream &out_;
> > +	};
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +thin_provisioning::emitter::ptr
> > +thin_provisioning::create_null_emitter(ostream &out)
> > +{
> > +	return emitter::ptr(new null_emitter(out));
> > +}
> > +
> > +//----------------------------------------------------------------
> > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> > new file mode 100644
> > index 0000000..2688762
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.h
> > @@ -0,0 +1,34 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools.  If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#ifndef NULL_FORMAT_H
> > +#define NULL_FORMAT_H
> > +
> > +#include "emitter.h"
> > +
> > +#include <iosfwd>
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace thin_provisioning {
> > +	emitter::ptr create_null_emitter(std::ostream &out);
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +#endif
> > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> > new file mode 100644
> > index 0000000..b1baa7d
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.cc
> > @@ -0,0 +1,92 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools.  If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#include "null_format.h"
> > +
> > +#include <iostream>
> > +
> > +using namespace std;
> > +using namespace thin_provisioning;
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace {
> > +	template <typename T>
> > +	std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> > +		if (maybe)
> > +			out << *maybe;
> > +
> > +		return out;
> > +	}
> > +
> > +	class null_emitter : public emitter {
> > +	public:
> > +		null_emitter(ostream &out)
> > +			: out_(out) {
> > +		}
> > +
> > +		void begin_superblock(string const &uuid,
> > +				      uint64_t time,
> > +				      uint64_t trans_id,
> > +				      uint32_t data_block_size,
> > +				      uint64_t nr_data_blocks,
> > +				      boost::optional<uint64_t> metadata_snap) {
> > +		}
> > +
> > +		void end_superblock() {
> > +		}
> > +
> > +		void begin_device(uint32_t dev_id,
> > +				  uint64_t mapped_blocks,
> > +				  uint64_t trans_id,
> > +				  uint64_t creation_time,
> > +				  uint64_t snap_time) {
> > +		}
> > +
> > +		void end_device() {
> > +		}
> > +
> > +		void begin_named_mapping(string const &name) {
> > +		}
> > +
> > +		void end_named_mapping() {
> > +		}
> > +
> > +		void identifier(string const &name) {
> > +		}
> > +
> > +		void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> > +		}
> > +
> > +		void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> > +		}
> > +
> > +	private:
> > +		ostream &out_;
> > +	};
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +thin_provisioning::emitter::ptr
> > +thin_provisioning::create_null_emitter(ostream &out)
> > +{
> > +	return emitter::ptr(new null_emitter(out));
> > +}
> > +
> > +//----------------------------------------------------------------
> > 
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
> 

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Ming-Hung Tsai]

2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
>
> On Thu, 17 Mar 2016, Joe Thornber wrote:
>
>> If you're skipping the mappings does the new thin_ls provide enough
>> information for you?
>
> Our primary need is the ability to specify --device-id in the o:L:d output
> format (old_format.cc) with mappings.  Our nightly dumps now take 50
> seconds instead of 40 minutes since we only dump the dev_id's that we
> need.
>
> As for thin_ls, we're still in the 0.5.x branch, I couldn't get 0.6.x to
> build on el7:
>
> # git checkout v0.6.2-rc6
> Previous HEAD position was e46bdfd... v0.6.1
> HEAD is now at afb71d8... bump version
>
> # ./configure
> [...]
>
> # make
>     [CXX] base/application.cc
> g++: error: @LFS_FLAGS@: No such file or directory
> make: *** [base/application.o] Error 1
>
>
> Am I missing something trivial here?

Did you run autoconf to regenerate the configure script ? v0.6.x has
some changes in the configure script.

# make distclean
# autoconf
# ./configure
# make


Ming-Hung Tsai

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Ming-Hung Tsai]

2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
>
> On Thu, 17 Mar 2016, Joe Thornber wrote:
>
>> If you're skipping the mappings does the new thin_ls provide enough
>> information for you?
>
> Our primary need is the ability to specify --device-id in the o:L:d output
> format (old_format.cc) with mappings.  Our nightly dumps now take 50
> seconds instead of 40 minutes since we only dump the dev_id's that we
> need.
>

I have similar requirement. I did this by my own compact_emitter,
which is a simplified version of "human_readable_emitter", without showing
the arrows and parentheses:

output format:

begin superblock=>uuid:"",time:<time>,transaction:<tid>,flags:<flags>,version:<version>,data_block_size:<bs>,nr_data_blocks:<nr_blocks>
device=>dev_id:<dev_id>,mapped_blocks:<mapped_blocks>,transaction:<tid>,creation_time:<creation_time>,snap_time:<snap_time>
<origin_begin>,<data_begin>,<len>,<time>
<origin_begin>,<data_begin>,1,<time>
...
end superblock

e.g.,

begin superblock=>uuid:"",time:0,transaction:1,flags:0,version:1,data_block_size:2048,nr_data_blocks:5657156
device=>dev_id:10000,mapped_blocks:3864041,transaction:0,creation_time:0,snap_time:0
1000,2000,100,0
1100,3000,200,0
...
end superblock

The <time> field is preserved for snapshot comparison.

Maybe we could discuss on creating a flexible emitter that could
customize the output format:
- Mapping units: sectors or blocks
- Emit the superblock and device tags or not. I think that an end mark
might be better for network transmission,
  to determine whether the dump was ended or not.
- Other consideration: should the customized output compatible to thin_restore ?



Ming-Hung Tsai

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Ming-Hung Tsai]

2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
>
> On Thu, 17 Mar 2016, Joe Thornber wrote:
>
>> If you're skipping the mappings does the new thin_ls provide enough
>> information for you?
>
> Our primary need is the ability to specify --device-id in the o:L:d output
> format (old_format.cc) with mappings.  Our nightly dumps now take 50
> seconds instead of 40 minutes since we only dump the dev_id's that we
> need.

I think that the --device-id option is required for thin_dump, but the
necessity of --skip-mappings could be discussed. If thin_ls supports
--device-id, then thin_dump might not require --skip-mappings.

I put my code here for reference (based on v0.7-devel)
https://github.com/mingnus/thin-provisioning-tools/commits/v0.7-thin_dump-enhancement
I borrow the option name "--skip-mappings" and function name
"set_skip_mappings" from Eric, since that they sounds more clear.

Build method:
# autoconf; ./configure --enable-dev-tools; make

My use cases:
# thin_dump --device-id=<dev_id> --format=<f> [--skip-mappings]


Ming-Hung Tsai

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Joe Thornber]

On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> Hi Joe,
> 
> Please review the patch below when you have a moment.  I am interested in 
> your feedback, and also interested in having this functionality merged 
> upstream.  This was written against thin-provisioning-tools.git tag 
> v0.5.6.

These changes are now on the v0.7-devel branch:

i) --dev-id to filter thin_dump output on device id, multiple selections allowed.
ii) --skip-mappings
iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own.

     eg, 

     thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda


The contrib/ directory is for third party contributions.  Use 'make
contrib' to build it.  I've put your emitter and tmakatos' emitter in
there already.

I hope that covers everything you need.

- Joe

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Eric Wheeler]

On Thu, 24 Mar 2016, Joe Thornber wrote:
> On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> > Hi Joe,
> > 
> > Please review the patch below when you have a moment.  I am interested in 
> > your feedback, and also interested in having this functionality merged 
> > upstream.  This was written against thin-provisioning-tools.git tag 
> > v0.5.6.
> 
> These changes are now on the v0.7-devel branch:
> 
> i) --dev-id to filter thin_dump output on device id, multiple selections allowed.
> ii) --skip-mappings
> iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own.
> 
>      eg, 
> 
>      thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda
> 
> 
> The contrib/ directory is for third party contributions.  Use 'make
> contrib' to build it.  I've put your emitter and tmakatos' emitter in
> there already.
> 
> I hope that covers everything you need.

I think that will work nicely. Thank you!  

-Eric

> 
> - Joe
> 

Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

[Eric Wheeler]

On Thu, 24 Mar 2016, Joe Thornber wrote:

> On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> > Hi Joe,
> > 
> > Please review the patch below when you have a moment.  I am interested in 
> > your feedback, and also interested in having this functionality merged 
> > upstream.  This was written against thin-provisioning-tools.git tag 
> > v0.5.6.
> 
> These changes are now on the v0.7-devel branch:
> 
> i) --dev-id to filter thin_dump output on device id, multiple selections allowed.
> ii) --skip-mappings
> iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own.
> 
>      eg, 
> 
>      thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda
> 
> 
> The contrib/ directory is for third party contributions.  Use 'make
> contrib' to build it.  I've put your emitter and tmakatos' emitter in
> there already.


It took me a bit to figure out that I needed --enable-testing to get 
contrib/ to build.  You might remove the --enable-testing requirement 
within the Makefile if contrib will remain as an optional Makefile target 
that doesn't usually build with `make`.

Also FYI: I noticed that `-ldl' needed added to to the g++ linking 
line:

/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:15: undefined reference to `dlopen' 
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:19: undefined reference to `dlerror' 
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:20: undefined reference to `dlsym' 
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:22: undefined reference to `dlerror' 
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:17: undefined reference to `dlerror'

So I added the -ldl as below:

g++ -g -Wall -fPIC -fno-strict-aliasing -std=c++11 -O8  
-DSTRERROR_R_CHAR_P   -o bin/pdata_tools base/application.o base/base64.o 
base/disk_units.o base/endian_utils.o base/error_state.o 
base/error_string.o base/grid_layout.o base/progress_monitor.o 
base/rolling_hash.o base/xml_utils.o block-cache/block_cache.o 
caching/cache_check.o caching/cache_dump.o caching/cache_metadata_size.o 
caching/cache_repair.o caching/cache_restore.o caching/commands.o 
caching/hint_array.o caching/mapping_array.o caching/metadata.o 
caching/metadata_dump.o caching/restore_emitter.o caching/superblock.o 
caching/xml_format.o era/commands.o era/era_array.o era/era_check.o 
era/era_detail.o era/era_dump.o era/era_invalidate.o era/era_restore.o 
era/metadata.o era/metadata_dump.o era/restore_emitter.o era/superblock.o 
era/writeset_tree.o era/xml_format.o main.o persistent-data/checksum.o 
persistent-data/data-structures/bitset.o 
persistent-data/data-structures/bloom_filter.o 
persistent-data/data-structures/btree.o persistent-data/error_set.o 
persistent-data/file_utils.o persistent-data/hex_dump.o 
persistent-data/space-maps/careful_alloc.o 
persistent-data/space-maps/disk.o persistent-data/space-maps/recursive.o 
persistent-data/space_map.o persistent-data/transaction_manager.o 
persistent-data/validators.o thin-provisioning/commands.o 
thin-provisioning/cache_stream.o thin-provisioning/chunk_stream.o 
thin-provisioning/device_tree.o thin-provisioning/fixed_chunk_stream.o 
thin-provisioning/human_readable_format.o thin-provisioning/mapping_tree.o 
thin-provisioning/metadata.o thin-provisioning/metadata_checker.o 
thin-provisioning/metadata_counter.o thin-provisioning/metadata_dumper.o 
thin-provisioning/pool_stream.o thin-provisioning/restore_emitter.o 
thin-provisioning/rmap_visitor.o thin-provisioning/superblock.o 
thin-provisioning/shared_library_emitter.o thin-provisioning/thin_check.o 
thin-provisioning/thin_delta.o thin-provisioning/thin_dump.o 
thin-provisioning/thin_ls.o thin-provisioning/thin_metadata_size.o 
thin-provisioning/thin_pool.o thin-provisioning/thin_repair.o 
thin-provisioning/thin_restore.o thin-provisioning/thin_rmap.o 
thin-provisioning/thin_trim.o thin-provisioning/xml_format.o -laio -lexpat 
-lstdc++ -ldl
         ^^^^  ADDED


> 
> I hope that covers everything you need.
> 
> - Joe
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>