* [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
@ 2016-03-15 1:45 Eric Wheeler
2016-03-15 10:59 ` Thanos Makatos
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Eric Wheeler @ 2016-03-15 1:45 UTC (permalink / raw)
To: Joe Thornber; +Cc: dm-devel
Hi Joe,
Please review the patch below when you have a moment. I am interested in
your feedback, and also interested in having this functionality merged
upstream. This was written against thin-provisioning-tools.git tag
v0.5.6.
We use thin_dump on live dm-thin metadata snapshots all the time. In our
case, we want to dump the XML for new (snapshot) volumes instead of
dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
minutes instead of ~5 seconds for a single volume with --device-id.
I started by adding --device-id to pass the `block_address dev_id` and
skip the call to emit_mappings() within mapping_tree_emitter::visit()
unless the dev_id matches --device-id as passed to thin_dump.
It is sometimes nice to list all of the device supers without waiting for
emit_mappings() so I added the --skip-mappings option. This allows you to
get the dev_id without reading the mappings:
thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta
Like `bool repair`, I added skip_mappings and dev_id as arguments to
thin_provisioning::metadata_dump() and passed them down to
mapping_tree_emitter with new members set_skip_mappings() and set_dev_id()
to set private attributes (default values are assigned in
metadata_dumper.h in case of backward compatible calls).
We work with the device metadata in a simplified format, and without
superblock information:
origin_offset:length:tdata_offset (hence, "o:L:d" or "old")
Therefore, the output --format 'old' was added. I added the format 'null'
as well for benchmarking purposes so the ostream needn't write large
volumes of text to /dev/null.
Benchmarks of the various formats on our sample metadata snapshot on an
idle machine:
for i in xml human_readable old null; do
time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null
done
real user sys
xml 29:01.27 25:35.87 01:54.49
human 27:39.81 24:12.90 01:52.39
old 27:07.71 23:40.64 01:52.97
null 23:39.38 21:17.22 00:49.04
I have this as a branch in bitbucket if you prefer to see it online:
https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id
or
git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id
--
Eric Wheeler
diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc
index 191acb5..7df5d85 100644
--- a/thin-provisioning/thin_dump.cc
+++ b/thin-provisioning/thin_dump.cc
@@ -22,6 +22,8 @@
#include <libgen.h>
#include "human_readable_format.h"
+#include "old_format.h"
+#include "null_format.h"
#include "metadata_dumper.h"
#include "metadata.h"
#include "xml_format.h"
@@ -36,6 +38,8 @@ using namespace thin_provisioning;
struct flags {
bool find_metadata_snap;
bool repair;
+ block_address dev_id;
+ bool skip_mappings;
};
namespace {
@@ -49,12 +53,16 @@ namespace {
e = create_xml_emitter(out);
else if (format == "human_readable")
e = create_human_readable_emitter(out);
+ else if (format == "old")
+ e = create_old_emitter(out);
+ else if (format == "null")
+ e = create_null_emitter(out);
else {
cerr << "unknown format '" << format << "'" << endl;
exit(1);
}
- metadata_dump(md, e, flags.repair);
+ metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id);
} catch (std::exception &e) {
cerr << e.what() << endl;
@@ -77,10 +85,12 @@ namespace {
out << "Usage: " << cmd << " [options] {device|file}" << endl
<< "Options:" << endl
<< " {-h|--help}" << endl
- << " {-f|--format} {xml|human_readable}" << endl
+ << " {-f|--format} {xml|human_readable|old|null}" << endl
<< " {-r|--repair}" << endl
<< " {-m|--metadata-snap} [block#]" << endl
<< " {-o <xml file>}" << endl
+ << " {-d device_id}" << endl
+ << " {-s|--skip-mappings}" << endl
<< " {-V|--version}" << endl;
}
}
@@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv)
{
int c;
char const *output = NULL;
- const char shortopts[] = "hm::o:f:rV";
+ const char shortopts[] = "hm::o:sd:f:rV";
char *end_ptr;
string format = "xml";
block_address metadata_snap = 0;
struct flags flags;
- flags.find_metadata_snap = flags.repair = false;
+ flags.find_metadata_snap = flags.repair = flags.skip_mappings = false;
+ flags.dev_id = -1;
const struct option longopts[] = {
{ "help", no_argument, NULL, 'h'},
{ "metadata-snap", optional_argument, NULL, 'm' },
{ "output", required_argument, NULL, 'o'},
+ { "skip-mappings", no_argument, NULL, 's'},
+ { "device-id", required_argument, NULL, 'd'},
{ "format", required_argument, NULL, 'f' },
{ "repair", no_argument, NULL, 'r'},
{ "version", no_argument, NULL, 'V'},
@@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv)
flags.repair = true;
break;
+ case 's':
+ flags.skip_mappings = true;
+ break;
+
+ case 'd':
+ flags.dev_id = strtoull(optarg, &end_ptr, 10);
+ if (end_ptr == optarg) {
+ cerr << "couldn't parse <device_id>" << endl;
+ usage(cerr, basename(argv[0]));
+ return 1;
+ }
+ break;
+
case 'm':
if (optarg) {
metadata_snap = strtoull(optarg, &end_ptr, 10);
@@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv)
}
}
+ if (format != "old" && flags.dev_id < 0) {
+ cerr << "Output format 'old' must specify --device-id" << endl;
+ return 1;
+ }
+
+
if (argc == optind) {
cerr << "No input file provided." << endl;
usage(cerr, basename(argv[0]));
diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h
index c96d22e..9d9814e 100644
--- a/thin-provisioning/metadata_dumper.h
+++ b/thin-provisioning/metadata_dumper.h
@@ -28,7 +28,7 @@ namespace thin_provisioning {
// Set the @repair flag if your metadata is corrupt, and you'd like
// the dumper to do it's best to recover info. If not set, any
// corruption encountered will cause an exception to be thrown.
- void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);
+ void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1);
}
//----------------------------------------------------------------
diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc
index db656ee..55cbae8 100644
--- a/thin-provisioning/metadata_dumper.cc
+++ b/thin-provisioning/metadata_dumper.cc
@@ -171,23 +171,31 @@ namespace {
dd_(dd),
repair_(repair),
damage_policy_(damage_policy) {
+ dev_id_ = -1;
+ skip_mappings_ = false;
}
+ void set_dev_id(block_address dev_id) { dev_id_ = dev_id; }
+ void set_skip_mappings(bool skip) { skip_mappings_ = skip; }
+
void visit(btree_path const &path, block_address tree_root) {
block_address dev_id = path[0];
dd_map::const_iterator it = dd_.find(path[0]);
if (it != dd_.end()) {
device_tree_detail::device_details const &d = it->second;
- e_->begin_device(dev_id,
- d.mapped_blocks_,
- d.transaction_id_,
- d.creation_time_,
- d.snapshotted_time_);
+ if (dev_id_ == -1UL || dev_id == dev_id_) {
+ e_->begin_device(dev_id,
+ d.mapped_blocks_,
+ d.transaction_id_,
+ d.creation_time_,
+ d.snapshotted_time_);
- emit_mappings(tree_root);
+ if (!skip_mappings_)
+ emit_mappings(tree_root);
- e_->end_device();
+ e_->end_device();
+ }
} else if (!repair_) {
ostringstream msg;
@@ -209,6 +217,8 @@ namespace {
emitter::ptr e_;
dd_map const &dd_;
bool repair_;
+ block_address dev_id_;
+ bool skip_mappings_;
mapping_tree_detail::damage_visitor::ptr damage_policy_;
};
}
@@ -216,7 +226,7 @@ namespace {
//----------------------------------------------------------------
void
-thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
+thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id)
{
details_extractor de;
device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair));
@@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
{
mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair));
mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair));
+
+ mte.set_skip_mappings(skip_mappings);
+
+ if (dev_id >= 0)
+ mte.set_dev_id(dev_id);
walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy);
}
diff --git a/man8/thin_dump.8 b/man8/thin_dump.8
index 7a9f785..9e0bf9e 100644
--- a/man8/thin_dump.8
+++ b/man8/thin_dump.8
@@ -26,9 +26,19 @@ in order to put it back onto a metadata
This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used.
-.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP".
+.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP".
Print output in XML or human readable format.
-
+.sp
+The
+.B old
+format requires the --device-id option and emits in the
+following format, one record per line:
+.B offset:length:tdata_offset
+(hence, "o:L:d") with units in bytes.
+.sp
+The
+.B null
+format emits nothing and only walks the tree.
.IP "\fB\-r, \-\-repair\fP".
Repair the metadata whilst dumping it.
@@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#.
See the thin provisioning target documentation on how to create or release
a metadata snapshot and retrieve the block number from the kernel.
+.IP "\fB\-d, \-\-skip\-mappings\fP".
+Skip emission of the mappings. This outputs nothing if format is
+either of
+.B old
+or
+.B null
+.
+
+.IP "\fB\-d, \-\-device\-id\fP".
+Specify the device_id to be dumped.
+
.IP "\fB\-h, \-\-help\fP".
Print help and exit.
diff --git a/Makefile.in b/Makefile.in
index e67b300..078bb53 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -73,6 +73,8 @@ SOURCE=\
persistent-data/validators.cc \
thin-provisioning/device_tree.cc \
thin-provisioning/human_readable_format.cc \
+ thin-provisioning/old_format.cc \
+ thin-provisioning/null_format.cc \
thin-provisioning/mapping_tree.cc \
thin-provisioning/metadata.cc \
thin-provisioning/metadata_checker.cc \
diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h
new file mode 100644
index 0000000..dba69e9
--- /dev/null
+++ b/thin-provisioning/old_format.h
@@ -0,0 +1,34 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools. If not, see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef OLD_FORMAT_H
+#define OLD_FORMAT_H
+
+#include "emitter.h"
+
+#include <iosfwd>
+
+//----------------------------------------------------------------
+
+namespace thin_provisioning {
+ emitter::ptr create_old_emitter(std::ostream &out);
+}
+
+//----------------------------------------------------------------
+
+#endif
diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc
new file mode 100644
index 0000000..52056c8
--- /dev/null
+++ b/thin-provisioning/old_format.cc
@@ -0,0 +1,103 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools. If not, see
+// <http://www.gnu.org/licenses/>.
+
+#include "old_format.h"
+
+#include <iostream>
+#include <boost/format.hpp>
+
+using namespace std;
+using namespace thin_provisioning;
+
+//----------------------------------------------------------------
+
+namespace {
+ template <typename T>
+ std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
+ if (maybe)
+ out << *maybe;
+
+ return out;
+ }
+
+ class old_emitter : public emitter {
+ public:
+ old_emitter(ostream &out)
+ : out_(out) {
+ }
+
+ void begin_superblock(string const &uuid,
+ uint64_t time,
+ uint64_t trans_id,
+ uint32_t data_block_size,
+ uint64_t nr_data_blocks,
+ boost::optional<uint64_t> metadata_snap) {
+ data_block_size_ = data_block_size;
+ }
+
+ void end_superblock() {
+ }
+
+ void begin_device(uint32_t dev_id,
+ uint64_t mapped_blocks,
+ uint64_t trans_id,
+ uint64_t creation_time,
+ uint64_t snap_time) {
+ }
+
+ void end_device() {
+ }
+
+ void begin_named_mapping(string const &name) {
+ }
+
+ void end_named_mapping() {
+ }
+
+ void identifier(string const &name) {
+ }
+
+ void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
+ out_ << (data_block_size_ << 9)*origin_begin
+ << ":" << (data_block_size_ << 9)*len
+ << ":" << (data_block_size_ << 9)*data_begin
+ << endl;
+ }
+
+ void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
+ out_ << (data_block_size_ << 9)*origin_block
+ << ":" << (data_block_size_ << 9)
+ << ":" << (data_block_size_ << 9)*data_block
+ << endl;
+ }
+
+ private:
+ ostream &out_;
+ uint64_t data_block_size_;
+ };
+}
+
+//----------------------------------------------------------------
+
+thin_provisioning::emitter::ptr
+thin_provisioning::create_old_emitter(ostream &out)
+{
+ return emitter::ptr(new old_emitter(out));
+}
+
+//----------------------------------------------------------------
diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
new file mode 100644
index 0000000..2688762
--- /dev/null
+++ b/thin-provisioning/null_format.h
@@ -0,0 +1,34 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools. If not, see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef NULL_FORMAT_H
+#define NULL_FORMAT_H
+
+#include "emitter.h"
+
+#include <iosfwd>
+
+//----------------------------------------------------------------
+
+namespace thin_provisioning {
+ emitter::ptr create_null_emitter(std::ostream &out);
+}
+
+//----------------------------------------------------------------
+
+#endif
diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
new file mode 100644
index 0000000..b1baa7d
--- /dev/null
+++ b/thin-provisioning/null_format.cc
@@ -0,0 +1,92 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools. If not, see
+// <http://www.gnu.org/licenses/>.
+
+#include "null_format.h"
+
+#include <iostream>
+
+using namespace std;
+using namespace thin_provisioning;
+
+//----------------------------------------------------------------
+
+namespace {
+ template <typename T>
+ std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
+ if (maybe)
+ out << *maybe;
+
+ return out;
+ }
+
+ class null_emitter : public emitter {
+ public:
+ null_emitter(ostream &out)
+ : out_(out) {
+ }
+
+ void begin_superblock(string const &uuid,
+ uint64_t time,
+ uint64_t trans_id,
+ uint32_t data_block_size,
+ uint64_t nr_data_blocks,
+ boost::optional<uint64_t> metadata_snap) {
+ }
+
+ void end_superblock() {
+ }
+
+ void begin_device(uint32_t dev_id,
+ uint64_t mapped_blocks,
+ uint64_t trans_id,
+ uint64_t creation_time,
+ uint64_t snap_time) {
+ }
+
+ void end_device() {
+ }
+
+ void begin_named_mapping(string const &name) {
+ }
+
+ void end_named_mapping() {
+ }
+
+ void identifier(string const &name) {
+ }
+
+ void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
+ }
+
+ void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
+ }
+
+ private:
+ ostream &out_;
+ };
+}
+
+//----------------------------------------------------------------
+
+thin_provisioning::emitter::ptr
+thin_provisioning::create_null_emitter(ostream &out)
+{
+ return emitter::ptr(new null_emitter(out));
+}
+
+//----------------------------------------------------------------
diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
new file mode 100644
index 0000000..2688762
--- /dev/null
+++ b/thin-provisioning/null_format.h
@@ -0,0 +1,34 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools. If not, see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef NULL_FORMAT_H
+#define NULL_FORMAT_H
+
+#include "emitter.h"
+
+#include <iosfwd>
+
+//----------------------------------------------------------------
+
+namespace thin_provisioning {
+ emitter::ptr create_null_emitter(std::ostream &out);
+}
+
+//----------------------------------------------------------------
+
+#endif
diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
new file mode 100644
index 0000000..b1baa7d
--- /dev/null
+++ b/thin-provisioning/null_format.cc
@@ -0,0 +1,92 @@
+// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+//
+// This file is part of the thin-provisioning-tools source.
+//
+// thin-provisioning-tools is free software: you can redistribute it
+// and/or modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation, either version 3 of
+// the License, or (at your option) any later version.
+//
+// thin-provisioning-tools is distributed in the hope that it will be
+// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with thin-provisioning-tools. If not, see
+// <http://www.gnu.org/licenses/>.
+
+#include "null_format.h"
+
+#include <iostream>
+
+using namespace std;
+using namespace thin_provisioning;
+
+//----------------------------------------------------------------
+
+namespace {
+ template <typename T>
+ std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
+ if (maybe)
+ out << *maybe;
+
+ return out;
+ }
+
+ class null_emitter : public emitter {
+ public:
+ null_emitter(ostream &out)
+ : out_(out) {
+ }
+
+ void begin_superblock(string const &uuid,
+ uint64_t time,
+ uint64_t trans_id,
+ uint32_t data_block_size,
+ uint64_t nr_data_blocks,
+ boost::optional<uint64_t> metadata_snap) {
+ }
+
+ void end_superblock() {
+ }
+
+ void begin_device(uint32_t dev_id,
+ uint64_t mapped_blocks,
+ uint64_t trans_id,
+ uint64_t creation_time,
+ uint64_t snap_time) {
+ }
+
+ void end_device() {
+ }
+
+ void begin_named_mapping(string const &name) {
+ }
+
+ void end_named_mapping() {
+ }
+
+ void identifier(string const &name) {
+ }
+
+ void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
+ }
+
+ void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
+ }
+
+ private:
+ ostream &out_;
+ };
+}
+
+//----------------------------------------------------------------
+
+thin_provisioning::emitter::ptr
+thin_provisioning::create_null_emitter(ostream &out)
+{
+ return emitter::ptr(new null_emitter(out));
+}
+
+//----------------------------------------------------------------
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-15 1:45 [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's Eric Wheeler
@ 2016-03-15 10:59 ` Thanos Makatos
2016-03-15 17:51 ` Eric Wheeler
2016-03-17 15:44 ` Joe Thornber
2016-03-17 15:45 ` Joe Thornber
2016-03-24 15:35 ` Joe Thornber
2 siblings, 2 replies; 14+ messages in thread
From: Thanos Makatos @ 2016-03-15 10:59 UTC (permalink / raw)
To: Eric Wheeler; +Cc: device-mapper development, Joe Thornber
On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> Hi Joe,
>
> Please review the patch below when you have a moment. I am interested in
> your feedback, and also interested in having this functionality merged
> upstream. This was written against thin-provisioning-tools.git tag
> v0.5.6.
>
> We use thin_dump on live dm-thin metadata snapshots all the time. In our
> case, we want to dump the XML for new (snapshot) volumes instead of
> dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> minutes instead of ~5 seconds for a single volume with --device-id.
I'm interested in extracting the mappings of a particular device, too.
In fact, I've implemented this (along with extracting the mappings in
binary format) and have only recently opened a PR:
https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
that we've replicated some work here, I'm not sure whether we're
supposed to send patches to this list or open PR on github.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-15 10:59 ` Thanos Makatos
@ 2016-03-15 17:51 ` Eric Wheeler
2016-03-16 5:42 ` Ming-Hung Tsai
2016-03-17 15:44 ` Joe Thornber
1 sibling, 1 reply; 14+ messages in thread
From: Eric Wheeler @ 2016-03-15 17:51 UTC (permalink / raw)
To: Thanos Makatos; +Cc: device-mapper development, Joe Thornber
On Tue, 15 Mar 2016, Thanos Makatos wrote:
> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> > Hi Joe,
> >
> > Please review the patch below when you have a moment. I am interested in
> > your feedback, and also interested in having this functionality merged
> > upstream. This was written against thin-provisioning-tools.git tag
> > v0.5.6.
> >
> > We use thin_dump on live dm-thin metadata snapshots all the time. In our
> > case, we want to dump the XML for new (snapshot) volumes instead of
> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> > minutes instead of ~5 seconds for a single volume with --device-id.
>
> I'm interested in extracting the mappings of a particular device, too.
> In fact, I've implemented this (along with extracting the mappings in
> binary format) and have only recently opened a PR:
> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
> that we've replicated some work here, I'm not sure whether we're
> supposed to send patches to this list or open PR on github.
Interesting, it is neat to see a binary format too.
I think we need to come up with a consistent way to extend attributes
being passed down into mapping_tree_emitter::visit() by way of
thin_provisioning::metadata_dump(). What is preferred?
I noticed that caching::metadata_dump() has the same prototype, so do
those need to remain compatable calls?
Should we countinue to extend the arguments to metadata_dump()
as Thanos and I have done, or, should we pop up the mte call to the caller
and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let
the caller configure the mte with setter functions or constructor calls?
I'm open to whatever---just making sure whatever we do stays clean and
scales for the future.
--
Eric Wheeler
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-15 17:51 ` Eric Wheeler
@ 2016-03-16 5:42 ` Ming-Hung Tsai
2016-03-16 19:14 ` Eric Wheeler
0 siblings, 1 reply; 14+ messages in thread
From: Ming-Hung Tsai @ 2016-03-16 5:42 UTC (permalink / raw)
To: Eric Wheeler, Thanos Makatos; +Cc: device-mapper development, Joe Thornber
2016-03-16 1:51 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
> On Tue, 15 Mar 2016, Thanos Makatos wrote:
>
>> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
>> > Hi Joe,
>> >
>> > We use thin_dump on live dm-thin metadata snapshots all the time. In our
>> > case, we want to dump the XML for new (snapshot) volumes instead of
>> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
>> > minutes instead of ~5 seconds for a single volume with --device-id.
>>
>> I'm interested in extracting the mappings of a particular device, too.
>> In fact, I've implemented this (along with extracting the mappings in
>> binary format) and have only recently opened a PR:
>> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
>> that we've replicated some work here, I'm not sure whether we're
>> supposed to send patches to this list or open PR on github.
>
> Interesting, it is neat to see a binary format too.
>
> I think we need to come up with a consistent way to extend attributes
> being passed down into mapping_tree_emitter::visit() by way of
> thin_provisioning::metadata_dump(). What is preferred?
>
> I noticed that caching::metadata_dump() has the same prototype, so do
> those need to remain compatable calls?
>
> Should we countinue to extend the arguments to metadata_dump()
> as Thanos and I have done, or, should we pop up the mte call to the caller
> and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let
> the caller configure the mte with setter functions or constructor calls?
Hi,
You are not alone, I did the same feature and used it for a long time.
My solution is similar to Thanos, I override thin_provisioning::metadata_dump()
to accept the additional dev_id:
void metadata_dump(metadata::ptr md, emitter::ptr e);
void metadata_dump(metadata::ptr md, emitter::ptr e, uint64_t dev_id);
Instead of passing dev_id to mapping_tree_emitter, I pass the dd_map
with only one device,
then let the mapping_tree_emitter to traverse the bottom-level tree directly:
void
thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e,
uint64_t dev_id)
{
uint64_t key[1] = {dev_id};
dev_tree::maybe_value single_mapping_tree_root =
md->mappings_top_level_->lookup(key);
device_tree::maybe_value details = md->details_->lookup(key);
// ... error handling
details_extractor de;
de.visit(dev_id, *details); // de contains only one device
e->begin_superblock(...);
{
mapping_tree_emitter mte(md, e, de.get_details(),
mapping_damage_policy(repair));
std::vector<uint64_t> path(1, dev_id); //
btree_detail::btree_path is std::vector<uint64_t>
mte.visit(path, *single_mapping_tree_root);
}
e->end_superblock();
}
But this looks still a bit messy: I should setup the visitors and
trees carefully to do what I want.
I also wrote some debugging feature for thin_dump. I can push the code
if you have interest.
1. thin_dump -b <metadata_dev> : Dump the device_details tree only. Do
not emit the data mappings.
(This could be replace by the new thin_ls)
output (no mappings):
<superblock ... >
<device ... >
</device>
<device ... >
</device>
</superblock>
2. thin_dump -r2 <metadata_dev>: dump the data mapping tree despite
the device_details tree was broken
Ming-Hung Tsai
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-16 5:42 ` Ming-Hung Tsai
@ 2016-03-16 19:14 ` Eric Wheeler
0 siblings, 0 replies; 14+ messages in thread
From: Eric Wheeler @ 2016-03-16 19:14 UTC (permalink / raw)
To: Ming-Hung Tsai; +Cc: Thanos Makatos, device-mapper development, Joe Thornber
On Wed, 16 Mar 2016, Ming-Hung Tsai wrote:
> 2016-03-16 1:51 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
> > On Tue, 15 Mar 2016, Thanos Makatos wrote:
> >
> >> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> >> > Hi Joe,
> >> >
> >> > We use thin_dump on live dm-thin metadata snapshots all the time. In our
> >> > case, we want to dump the XML for new (snapshot) volumes instead of
> >> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> >> > minutes instead of ~5 seconds for a single volume with --device-id.
> >>
> >> I'm interested in extracting the mappings of a particular device, too.
> >> In fact, I've implemented this (along with extracting the mappings in
> >> binary format) and have only recently opened a PR:
> >> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
> >> that we've replicated some work here, I'm not sure whether we're
> >> supposed to send patches to this list or open PR on github.
> >
> > Interesting, it is neat to see a binary format too.
> >
> > I think we need to come up with a consistent way to extend attributes
> > being passed down into mapping_tree_emitter::visit() by way of
> > thin_provisioning::metadata_dump(). What is preferred?
> >
> > I noticed that caching::metadata_dump() has the same prototype, so do
> > those need to remain compatable calls?
> >
> > Should we countinue to extend the arguments to metadata_dump()
> > as Thanos and I have done, or, should we pop up the mte call to the caller
> > and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let
> > the caller configure the mte with setter functions or constructor calls?
>
> Hi,
>
> You are not alone, I did the same feature and used it for a long time.
> My solution is similar to Thanos, I override thin_provisioning::metadata_dump()
> to accept the additional dev_id:
>
> void metadata_dump(metadata::ptr md, emitter::ptr e);
> void metadata_dump(metadata::ptr md, emitter::ptr e, uint64_t dev_id);
That makes for 3 different implementations, we've all modified
metadata_dump:
original: ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);
Ming-Hung: void metadata_dump(metadata::ptr md, emitter::ptr e, [bool repair?]
+ uint64_t dev_id);
Thanos: ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair,
+ const block_address *dev_id)
Me: ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair,
+ bool skip_mappings,
+ block_address dev_id)
--
Eric Wheeler
>
> Instead of passing dev_id to mapping_tree_emitter, I pass the dd_map
> with only one device,
> then let the mapping_tree_emitter to traverse the bottom-level tree directly:
>
> void
> thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e,
> uint64_t dev_id)
> {
> uint64_t key[1] = {dev_id};
> dev_tree::maybe_value single_mapping_tree_root =
> md->mappings_top_level_->lookup(key);
> device_tree::maybe_value details = md->details_->lookup(key);
>
> // ... error handling
>
> details_extractor de;
> de.visit(dev_id, *details); // de contains only one device
>
> e->begin_superblock(...);
> {
> mapping_tree_emitter mte(md, e, de.get_details(),
> mapping_damage_policy(repair));
> std::vector<uint64_t> path(1, dev_id); //
> btree_detail::btree_path is std::vector<uint64_t>
> mte.visit(path, *single_mapping_tree_root);
> }
> e->end_superblock();
> }
>
> But this looks still a bit messy: I should setup the visitors and
> trees carefully to do what I want.
>
> I also wrote some debugging feature for thin_dump. I can push the code
> if you have interest.
>
> 1. thin_dump -b <metadata_dev> : Dump the device_details tree only. Do
> not emit the data mappings.
> (This could be replace by the new thin_ls)
>
> output (no mappings):
> <superblock ... >
> <device ... >
> </device>
> <device ... >
> </device>
> </superblock>
>
> 2. thin_dump -r2 <metadata_dev>: dump the data mapping tree despite
> the device_details tree was broken
>
>
>
> Ming-Hung Tsai
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-15 10:59 ` Thanos Makatos
2016-03-15 17:51 ` Eric Wheeler
@ 2016-03-17 15:44 ` Joe Thornber
1 sibling, 0 replies; 14+ messages in thread
From: Joe Thornber @ 2016-03-17 15:44 UTC (permalink / raw)
To: Thanos Makatos; +Cc: Eric Wheeler, device-mapper development
On Tue, Mar 15, 2016 at 10:59:15AM +0000, Thanos Makatos wrote:
> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote:
> > Hi Joe,
> >
> > Please review the patch below when you have a moment. I am interested in
> > your feedback, and also interested in having this functionality merged
> > upstream. This was written against thin-provisioning-tools.git tag
> > v0.5.6.
> >
> > We use thin_dump on live dm-thin metadata snapshots all the time. In our
> > case, we want to dump the XML for new (snapshot) volumes instead of
> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> > minutes instead of ~5 seconds for a single volume with --device-id.
>
> I'm interested in extracting the mappings of a particular device, too.
> In fact, I've implemented this (along with extracting the mappings in
> binary format) and have only recently opened a PR:
> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems
> that we've replicated some work here, I'm not sure whether we're
> supposed to send patches to this list or open PR on github.
I've not forgotten you patches, just haven't got round to them yet.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-15 1:45 [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's Eric Wheeler
2016-03-15 10:59 ` Thanos Makatos
@ 2016-03-17 15:45 ` Joe Thornber
2016-03-17 22:45 ` Eric Wheeler
2016-03-24 15:35 ` Joe Thornber
2 siblings, 1 reply; 14+ messages in thread
From: Joe Thornber @ 2016-03-17 15:45 UTC (permalink / raw)
To: Eric Wheeler; +Cc: dm-devel
If you're skipping the mappings does the new thin_ls provide enough
information for you?
- Joe
On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> Hi Joe,
>
> Please review the patch below when you have a moment. I am interested in
> your feedback, and also interested in having this functionality merged
> upstream. This was written against thin-provisioning-tools.git tag
> v0.5.6.
>
> We use thin_dump on live dm-thin metadata snapshots all the time. In our
> case, we want to dump the XML for new (snapshot) volumes instead of
> dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> minutes instead of ~5 seconds for a single volume with --device-id.
>
> I started by adding --device-id to pass the `block_address dev_id` and
> skip the call to emit_mappings() within mapping_tree_emitter::visit()
> unless the dev_id matches --device-id as passed to thin_dump.
>
> It is sometimes nice to list all of the device supers without waiting for
> emit_mappings() so I added the --skip-mappings option. This allows you to
> get the dev_id without reading the mappings:
> thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta
>
> Like `bool repair`, I added skip_mappings and dev_id as arguments to
> thin_provisioning::metadata_dump() and passed them down to
> mapping_tree_emitter with new members set_skip_mappings() and set_dev_id()
> to set private attributes (default values are assigned in
> metadata_dumper.h in case of backward compatible calls).
>
> We work with the device metadata in a simplified format, and without
> superblock information:
>
> origin_offset:length:tdata_offset (hence, "o:L:d" or "old")
>
> Therefore, the output --format 'old' was added. I added the format 'null'
> as well for benchmarking purposes so the ostream needn't write large
> volumes of text to /dev/null.
>
> Benchmarks of the various formats on our sample metadata snapshot on an
> idle machine:
>
> for i in xml human_readable old null; do
> time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null
> done
>
> real user sys
> xml 29:01.27 25:35.87 01:54.49
> human 27:39.81 24:12.90 01:52.39
> old 27:07.71 23:40.64 01:52.97
> null 23:39.38 21:17.22 00:49.04
>
> I have this as a branch in bitbucket if you prefer to see it online:
>
> https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id
> or
> git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id
>
>
> --
> Eric Wheeler
>
>
> diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc
> index 191acb5..7df5d85 100644
> --- a/thin-provisioning/thin_dump.cc
> +++ b/thin-provisioning/thin_dump.cc
> @@ -22,6 +22,8 @@
> #include <libgen.h>
>
> #include "human_readable_format.h"
> +#include "old_format.h"
> +#include "null_format.h"
> #include "metadata_dumper.h"
> #include "metadata.h"
> #include "xml_format.h"
> @@ -36,6 +38,8 @@ using namespace thin_provisioning;
> struct flags {
> bool find_metadata_snap;
> bool repair;
> + block_address dev_id;
> + bool skip_mappings;
> };
>
> namespace {
> @@ -49,12 +53,16 @@ namespace {
> e = create_xml_emitter(out);
> else if (format == "human_readable")
> e = create_human_readable_emitter(out);
> + else if (format == "old")
> + e = create_old_emitter(out);
> + else if (format == "null")
> + e = create_null_emitter(out);
> else {
> cerr << "unknown format '" << format << "'" << endl;
> exit(1);
> }
>
> - metadata_dump(md, e, flags.repair);
> + metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id);
>
> } catch (std::exception &e) {
> cerr << e.what() << endl;
> @@ -77,10 +85,12 @@ namespace {
> out << "Usage: " << cmd << " [options] {device|file}" << endl
> << "Options:" << endl
> << " {-h|--help}" << endl
> - << " {-f|--format} {xml|human_readable}" << endl
> + << " {-f|--format} {xml|human_readable|old|null}" << endl
> << " {-r|--repair}" << endl
> << " {-m|--metadata-snap} [block#]" << endl
> << " {-o <xml file>}" << endl
> + << " {-d device_id}" << endl
> + << " {-s|--skip-mappings}" << endl
> << " {-V|--version}" << endl;
> }
> }
> @@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv)
> {
> int c;
> char const *output = NULL;
> - const char shortopts[] = "hm::o:f:rV";
> + const char shortopts[] = "hm::o:sd:f:rV";
> char *end_ptr;
> string format = "xml";
> block_address metadata_snap = 0;
> struct flags flags;
> - flags.find_metadata_snap = flags.repair = false;
> + flags.find_metadata_snap = flags.repair = flags.skip_mappings = false;
> + flags.dev_id = -1;
>
> const struct option longopts[] = {
> { "help", no_argument, NULL, 'h'},
> { "metadata-snap", optional_argument, NULL, 'm' },
> { "output", required_argument, NULL, 'o'},
> + { "skip-mappings", no_argument, NULL, 's'},
> + { "device-id", required_argument, NULL, 'd'},
> { "format", required_argument, NULL, 'f' },
> { "repair", no_argument, NULL, 'r'},
> { "version", no_argument, NULL, 'V'},
> @@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv)
> flags.repair = true;
> break;
>
> + case 's':
> + flags.skip_mappings = true;
> + break;
> +
> + case 'd':
> + flags.dev_id = strtoull(optarg, &end_ptr, 10);
> + if (end_ptr == optarg) {
> + cerr << "couldn't parse <device_id>" << endl;
> + usage(cerr, basename(argv[0]));
> + return 1;
> + }
> + break;
> +
> case 'm':
> if (optarg) {
> metadata_snap = strtoull(optarg, &end_ptr, 10);
> @@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv)
> }
> }
>
> + if (format != "old" && flags.dev_id < 0) {
> + cerr << "Output format 'old' must specify --device-id" << endl;
> + return 1;
> + }
> +
> +
> if (argc == optind) {
> cerr << "No input file provided." << endl;
> usage(cerr, basename(argv[0]));
> diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h
> index c96d22e..9d9814e 100644
> --- a/thin-provisioning/metadata_dumper.h
> +++ b/thin-provisioning/metadata_dumper.h
> @@ -28,7 +28,7 @@ namespace thin_provisioning {
> // Set the @repair flag if your metadata is corrupt, and you'd like
> // the dumper to do it's best to recover info. If not set, any
> // corruption encountered will cause an exception to be thrown.
> - void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);
> + void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1);
> }
>
> //----------------------------------------------------------------
> diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc
> index db656ee..55cbae8 100644
> --- a/thin-provisioning/metadata_dumper.cc
> +++ b/thin-provisioning/metadata_dumper.cc
> @@ -171,23 +171,31 @@ namespace {
> dd_(dd),
> repair_(repair),
> damage_policy_(damage_policy) {
> + dev_id_ = -1;
> + skip_mappings_ = false;
> }
>
> + void set_dev_id(block_address dev_id) { dev_id_ = dev_id; }
> + void set_skip_mappings(bool skip) { skip_mappings_ = skip; }
> +
> void visit(btree_path const &path, block_address tree_root) {
> block_address dev_id = path[0];
>
> dd_map::const_iterator it = dd_.find(path[0]);
> if (it != dd_.end()) {
> device_tree_detail::device_details const &d = it->second;
> - e_->begin_device(dev_id,
> - d.mapped_blocks_,
> - d.transaction_id_,
> - d.creation_time_,
> - d.snapshotted_time_);
> + if (dev_id_ == -1UL || dev_id == dev_id_) {
> + e_->begin_device(dev_id,
> + d.mapped_blocks_,
> + d.transaction_id_,
> + d.creation_time_,
> + d.snapshotted_time_);
>
> - emit_mappings(tree_root);
> + if (!skip_mappings_)
> + emit_mappings(tree_root);
>
> - e_->end_device();
> + e_->end_device();
> + }
>
> } else if (!repair_) {
> ostringstream msg;
> @@ -209,6 +217,8 @@ namespace {
> emitter::ptr e_;
> dd_map const &dd_;
> bool repair_;
> + block_address dev_id_;
> + bool skip_mappings_;
> mapping_tree_detail::damage_visitor::ptr damage_policy_;
> };
> }
> @@ -216,7 +226,7 @@ namespace {
> //----------------------------------------------------------------
>
> void
> -thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
> +thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id)
> {
> details_extractor de;
> device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair));
> @@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
> {
> mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair));
> mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair));
> +
> + mte.set_skip_mappings(skip_mappings);
> +
> + if (dev_id >= 0)
> + mte.set_dev_id(dev_id);
> walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy);
> }
>
> diff --git a/man8/thin_dump.8 b/man8/thin_dump.8
> index 7a9f785..9e0bf9e 100644
> --- a/man8/thin_dump.8
> +++ b/man8/thin_dump.8
> @@ -26,9 +26,19 @@ in order to put it back onto a metadata
>
> This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used.
>
> -.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP".
> +.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP".
> Print output in XML or human readable format.
> -
> +.sp
> +The
> +.B old
> +format requires the --device-id option and emits in the
> +following format, one record per line:
> +.B offset:length:tdata_offset
> +(hence, "o:L:d") with units in bytes.
> +.sp
> +The
> +.B null
> +format emits nothing and only walks the tree.
> .IP "\fB\-r, \-\-repair\fP".
> Repair the metadata whilst dumping it.
>
> @@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#.
> See the thin provisioning target documentation on how to create or release
> a metadata snapshot and retrieve the block number from the kernel.
>
> +.IP "\fB\-d, \-\-skip\-mappings\fP".
> +Skip emission of the mappings. This outputs nothing if format is
> +either of
> +.B old
> +or
> +.B null
> +.
> +
> +.IP "\fB\-d, \-\-device\-id\fP".
> +Specify the device_id to be dumped.
> +
> .IP "\fB\-h, \-\-help\fP".
> Print help and exit.
>
> diff --git a/Makefile.in b/Makefile.in
> index e67b300..078bb53 100644
> --- a/Makefile.in
> +++ b/Makefile.in
> @@ -73,6 +73,8 @@ SOURCE=\
> persistent-data/validators.cc \
> thin-provisioning/device_tree.cc \
> thin-provisioning/human_readable_format.cc \
> + thin-provisioning/old_format.cc \
> + thin-provisioning/null_format.cc \
> thin-provisioning/mapping_tree.cc \
> thin-provisioning/metadata.cc \
> thin-provisioning/metadata_checker.cc \
> diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h
> new file mode 100644
> index 0000000..dba69e9
> --- /dev/null
> +++ b/thin-provisioning/old_format.h
> @@ -0,0 +1,34 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools. If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef OLD_FORMAT_H
> +#define OLD_FORMAT_H
> +
> +#include "emitter.h"
> +
> +#include <iosfwd>
> +
> +//----------------------------------------------------------------
> +
> +namespace thin_provisioning {
> + emitter::ptr create_old_emitter(std::ostream &out);
> +}
> +
> +//----------------------------------------------------------------
> +
> +#endif
> diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc
> new file mode 100644
> index 0000000..52056c8
> --- /dev/null
> +++ b/thin-provisioning/old_format.cc
> @@ -0,0 +1,103 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools. If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "old_format.h"
> +
> +#include <iostream>
> +#include <boost/format.hpp>
> +
> +using namespace std;
> +using namespace thin_provisioning;
> +
> +//----------------------------------------------------------------
> +
> +namespace {
> + template <typename T>
> + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> + if (maybe)
> + out << *maybe;
> +
> + return out;
> + }
> +
> + class old_emitter : public emitter {
> + public:
> + old_emitter(ostream &out)
> + : out_(out) {
> + }
> +
> + void begin_superblock(string const &uuid,
> + uint64_t time,
> + uint64_t trans_id,
> + uint32_t data_block_size,
> + uint64_t nr_data_blocks,
> + boost::optional<uint64_t> metadata_snap) {
> + data_block_size_ = data_block_size;
> + }
> +
> + void end_superblock() {
> + }
> +
> + void begin_device(uint32_t dev_id,
> + uint64_t mapped_blocks,
> + uint64_t trans_id,
> + uint64_t creation_time,
> + uint64_t snap_time) {
> + }
> +
> + void end_device() {
> + }
> +
> + void begin_named_mapping(string const &name) {
> + }
> +
> + void end_named_mapping() {
> + }
> +
> + void identifier(string const &name) {
> + }
> +
> + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> + out_ << (data_block_size_ << 9)*origin_begin
> + << ":" << (data_block_size_ << 9)*len
> + << ":" << (data_block_size_ << 9)*data_begin
> + << endl;
> + }
> +
> + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> + out_ << (data_block_size_ << 9)*origin_block
> + << ":" << (data_block_size_ << 9)
> + << ":" << (data_block_size_ << 9)*data_block
> + << endl;
> + }
> +
> + private:
> + ostream &out_;
> + uint64_t data_block_size_;
> + };
> +}
> +
> +//----------------------------------------------------------------
> +
> +thin_provisioning::emitter::ptr
> +thin_provisioning::create_old_emitter(ostream &out)
> +{
> + return emitter::ptr(new old_emitter(out));
> +}
> +
> +//----------------------------------------------------------------
> diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> new file mode 100644
> index 0000000..2688762
> --- /dev/null
> +++ b/thin-provisioning/null_format.h
> @@ -0,0 +1,34 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools. If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef NULL_FORMAT_H
> +#define NULL_FORMAT_H
> +
> +#include "emitter.h"
> +
> +#include <iosfwd>
> +
> +//----------------------------------------------------------------
> +
> +namespace thin_provisioning {
> + emitter::ptr create_null_emitter(std::ostream &out);
> +}
> +
> +//----------------------------------------------------------------
> +
> +#endif
> diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> new file mode 100644
> index 0000000..b1baa7d
> --- /dev/null
> +++ b/thin-provisioning/null_format.cc
> @@ -0,0 +1,92 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools. If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "null_format.h"
> +
> +#include <iostream>
> +
> +using namespace std;
> +using namespace thin_provisioning;
> +
> +//----------------------------------------------------------------
> +
> +namespace {
> + template <typename T>
> + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> + if (maybe)
> + out << *maybe;
> +
> + return out;
> + }
> +
> + class null_emitter : public emitter {
> + public:
> + null_emitter(ostream &out)
> + : out_(out) {
> + }
> +
> + void begin_superblock(string const &uuid,
> + uint64_t time,
> + uint64_t trans_id,
> + uint32_t data_block_size,
> + uint64_t nr_data_blocks,
> + boost::optional<uint64_t> metadata_snap) {
> + }
> +
> + void end_superblock() {
> + }
> +
> + void begin_device(uint32_t dev_id,
> + uint64_t mapped_blocks,
> + uint64_t trans_id,
> + uint64_t creation_time,
> + uint64_t snap_time) {
> + }
> +
> + void end_device() {
> + }
> +
> + void begin_named_mapping(string const &name) {
> + }
> +
> + void end_named_mapping() {
> + }
> +
> + void identifier(string const &name) {
> + }
> +
> + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> + }
> +
> + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> + }
> +
> + private:
> + ostream &out_;
> + };
> +}
> +
> +//----------------------------------------------------------------
> +
> +thin_provisioning::emitter::ptr
> +thin_provisioning::create_null_emitter(ostream &out)
> +{
> + return emitter::ptr(new null_emitter(out));
> +}
> +
> +//----------------------------------------------------------------
> diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> new file mode 100644
> index 0000000..2688762
> --- /dev/null
> +++ b/thin-provisioning/null_format.h
> @@ -0,0 +1,34 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools. If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef NULL_FORMAT_H
> +#define NULL_FORMAT_H
> +
> +#include "emitter.h"
> +
> +#include <iosfwd>
> +
> +//----------------------------------------------------------------
> +
> +namespace thin_provisioning {
> + emitter::ptr create_null_emitter(std::ostream &out);
> +}
> +
> +//----------------------------------------------------------------
> +
> +#endif
> diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> new file mode 100644
> index 0000000..b1baa7d
> --- /dev/null
> +++ b/thin-provisioning/null_format.cc
> @@ -0,0 +1,92 @@
> +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> +//
> +// This file is part of the thin-provisioning-tools source.
> +//
> +// thin-provisioning-tools is free software: you can redistribute it
> +// and/or modify it under the terms of the GNU General Public License
> +// as published by the Free Software Foundation, either version 3 of
> +// the License, or (at your option) any later version.
> +//
> +// thin-provisioning-tools is distributed in the hope that it will be
> +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License along
> +// with thin-provisioning-tools. If not, see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "null_format.h"
> +
> +#include <iostream>
> +
> +using namespace std;
> +using namespace thin_provisioning;
> +
> +//----------------------------------------------------------------
> +
> +namespace {
> + template <typename T>
> + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> + if (maybe)
> + out << *maybe;
> +
> + return out;
> + }
> +
> + class null_emitter : public emitter {
> + public:
> + null_emitter(ostream &out)
> + : out_(out) {
> + }
> +
> + void begin_superblock(string const &uuid,
> + uint64_t time,
> + uint64_t trans_id,
> + uint32_t data_block_size,
> + uint64_t nr_data_blocks,
> + boost::optional<uint64_t> metadata_snap) {
> + }
> +
> + void end_superblock() {
> + }
> +
> + void begin_device(uint32_t dev_id,
> + uint64_t mapped_blocks,
> + uint64_t trans_id,
> + uint64_t creation_time,
> + uint64_t snap_time) {
> + }
> +
> + void end_device() {
> + }
> +
> + void begin_named_mapping(string const &name) {
> + }
> +
> + void end_named_mapping() {
> + }
> +
> + void identifier(string const &name) {
> + }
> +
> + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> + }
> +
> + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> + }
> +
> + private:
> + ostream &out_;
> + };
> +}
> +
> +//----------------------------------------------------------------
> +
> +thin_provisioning::emitter::ptr
> +thin_provisioning::create_null_emitter(ostream &out)
> +{
> + return emitter::ptr(new null_emitter(out));
> +}
> +
> +//----------------------------------------------------------------
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-17 15:45 ` Joe Thornber
@ 2016-03-17 22:45 ` Eric Wheeler
2016-03-18 2:04 ` Ming-Hung Tsai
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Eric Wheeler @ 2016-03-17 22:45 UTC (permalink / raw)
To: Joe Thornber; +Cc: dm-devel
On Thu, 17 Mar 2016, Joe Thornber wrote:
> If you're skipping the mappings does the new thin_ls provide enough
> information for you?
Our primary need is the ability to specify --device-id in the o:L:d output
format (old_format.cc) with mappings. Our nightly dumps now take 50
seconds instead of 40 minutes since we only dump the dev_id's that we
need.
As for thin_ls, we're still in the 0.5.x branch, I couldn't get 0.6.x to
build on el7:
# git checkout v0.6.2-rc6
Previous HEAD position was e46bdfd... v0.6.1
HEAD is now at afb71d8... bump version
# ./configure
[...]
# make
[CXX] base/application.cc
g++: error: @LFS_FLAGS@: No such file or directory
make: *** [base/application.o] Error 1
Am I missing something trivial here?
--
Eric Wheeler
>
> - Joe
>
> On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> > Hi Joe,
> >
> > Please review the patch below when you have a moment. I am interested in
> > your feedback, and also interested in having this functionality merged
> > upstream. This was written against thin-provisioning-tools.git tag
> > v0.5.6.
> >
> > We use thin_dump on live dm-thin metadata snapshots all the time. In our
> > case, we want to dump the XML for new (snapshot) volumes instead of
> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30
> > minutes instead of ~5 seconds for a single volume with --device-id.
> >
> > I started by adding --device-id to pass the `block_address dev_id` and
> > skip the call to emit_mappings() within mapping_tree_emitter::visit()
> > unless the dev_id matches --device-id as passed to thin_dump.
> >
> > It is sometimes nice to list all of the device supers without waiting for
> > emit_mappings() so I added the --skip-mappings option. This allows you to
> > get the dev_id without reading the mappings:
> > thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta
> >
> > Like `bool repair`, I added skip_mappings and dev_id as arguments to
> > thin_provisioning::metadata_dump() and passed them down to
> > mapping_tree_emitter with new members set_skip_mappings() and set_dev_id()
> > to set private attributes (default values are assigned in
> > metadata_dumper.h in case of backward compatible calls).
> >
> > We work with the device metadata in a simplified format, and without
> > superblock information:
> >
> > origin_offset:length:tdata_offset (hence, "o:L:d" or "old")
> >
> > Therefore, the output --format 'old' was added. I added the format 'null'
> > as well for benchmarking purposes so the ostream needn't write large
> > volumes of text to /dev/null.
> >
> > Benchmarks of the various formats on our sample metadata snapshot on an
> > idle machine:
> >
> > for i in xml human_readable old null; do
> > time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null
> > done
> >
> > real user sys
> > xml 29:01.27 25:35.87 01:54.49
> > human 27:39.81 24:12.90 01:52.39
> > old 27:07.71 23:40.64 01:52.97
> > null 23:39.38 21:17.22 00:49.04
> >
> > I have this as a branch in bitbucket if you prefer to see it online:
> >
> > https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id
> > or
> > git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id
> >
> >
> > --
> > Eric Wheeler
> >
> >
> > diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc
> > index 191acb5..7df5d85 100644
> > --- a/thin-provisioning/thin_dump.cc
> > +++ b/thin-provisioning/thin_dump.cc
> > @@ -22,6 +22,8 @@
> > #include <libgen.h>
> >
> > #include "human_readable_format.h"
> > +#include "old_format.h"
> > +#include "null_format.h"
> > #include "metadata_dumper.h"
> > #include "metadata.h"
> > #include "xml_format.h"
> > @@ -36,6 +38,8 @@ using namespace thin_provisioning;
> > struct flags {
> > bool find_metadata_snap;
> > bool repair;
> > + block_address dev_id;
> > + bool skip_mappings;
> > };
> >
> > namespace {
> > @@ -49,12 +53,16 @@ namespace {
> > e = create_xml_emitter(out);
> > else if (format == "human_readable")
> > e = create_human_readable_emitter(out);
> > + else if (format == "old")
> > + e = create_old_emitter(out);
> > + else if (format == "null")
> > + e = create_null_emitter(out);
> > else {
> > cerr << "unknown format '" << format << "'" << endl;
> > exit(1);
> > }
> >
> > - metadata_dump(md, e, flags.repair);
> > + metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id);
> >
> > } catch (std::exception &e) {
> > cerr << e.what() << endl;
> > @@ -77,10 +85,12 @@ namespace {
> > out << "Usage: " << cmd << " [options] {device|file}" << endl
> > << "Options:" << endl
> > << " {-h|--help}" << endl
> > - << " {-f|--format} {xml|human_readable}" << endl
> > + << " {-f|--format} {xml|human_readable|old|null}" << endl
> > << " {-r|--repair}" << endl
> > << " {-m|--metadata-snap} [block#]" << endl
> > << " {-o <xml file>}" << endl
> > + << " {-d device_id}" << endl
> > + << " {-s|--skip-mappings}" << endl
> > << " {-V|--version}" << endl;
> > }
> > }
> > @@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv)
> > {
> > int c;
> > char const *output = NULL;
> > - const char shortopts[] = "hm::o:f:rV";
> > + const char shortopts[] = "hm::o:sd:f:rV";
> > char *end_ptr;
> > string format = "xml";
> > block_address metadata_snap = 0;
> > struct flags flags;
> > - flags.find_metadata_snap = flags.repair = false;
> > + flags.find_metadata_snap = flags.repair = flags.skip_mappings = false;
> > + flags.dev_id = -1;
> >
> > const struct option longopts[] = {
> > { "help", no_argument, NULL, 'h'},
> > { "metadata-snap", optional_argument, NULL, 'm' },
> > { "output", required_argument, NULL, 'o'},
> > + { "skip-mappings", no_argument, NULL, 's'},
> > + { "device-id", required_argument, NULL, 'd'},
> > { "format", required_argument, NULL, 'f' },
> > { "repair", no_argument, NULL, 'r'},
> > { "version", no_argument, NULL, 'V'},
> > @@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv)
> > flags.repair = true;
> > break;
> >
> > + case 's':
> > + flags.skip_mappings = true;
> > + break;
> > +
> > + case 'd':
> > + flags.dev_id = strtoull(optarg, &end_ptr, 10);
> > + if (end_ptr == optarg) {
> > + cerr << "couldn't parse <device_id>" << endl;
> > + usage(cerr, basename(argv[0]));
> > + return 1;
> > + }
> > + break;
> > +
> > case 'm':
> > if (optarg) {
> > metadata_snap = strtoull(optarg, &end_ptr, 10);
> > @@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv)
> > }
> > }
> >
> > + if (format != "old" && flags.dev_id < 0) {
> > + cerr << "Output format 'old' must specify --device-id" << endl;
> > + return 1;
> > + }
> > +
> > +
> > if (argc == optind) {
> > cerr << "No input file provided." << endl;
> > usage(cerr, basename(argv[0]));
> > diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h
> > index c96d22e..9d9814e 100644
> > --- a/thin-provisioning/metadata_dumper.h
> > +++ b/thin-provisioning/metadata_dumper.h
> > @@ -28,7 +28,7 @@ namespace thin_provisioning {
> > // Set the @repair flag if your metadata is corrupt, and you'd like
> > // the dumper to do it's best to recover info. If not set, any
> > // corruption encountered will cause an exception to be thrown.
> > - void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair);
> > + void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1);
> > }
> >
> > //----------------------------------------------------------------
> > diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc
> > index db656ee..55cbae8 100644
> > --- a/thin-provisioning/metadata_dumper.cc
> > +++ b/thin-provisioning/metadata_dumper.cc
> > @@ -171,23 +171,31 @@ namespace {
> > dd_(dd),
> > repair_(repair),
> > damage_policy_(damage_policy) {
> > + dev_id_ = -1;
> > + skip_mappings_ = false;
> > }
> >
> > + void set_dev_id(block_address dev_id) { dev_id_ = dev_id; }
> > + void set_skip_mappings(bool skip) { skip_mappings_ = skip; }
> > +
> > void visit(btree_path const &path, block_address tree_root) {
> > block_address dev_id = path[0];
> >
> > dd_map::const_iterator it = dd_.find(path[0]);
> > if (it != dd_.end()) {
> > device_tree_detail::device_details const &d = it->second;
> > - e_->begin_device(dev_id,
> > - d.mapped_blocks_,
> > - d.transaction_id_,
> > - d.creation_time_,
> > - d.snapshotted_time_);
> > + if (dev_id_ == -1UL || dev_id == dev_id_) {
> > + e_->begin_device(dev_id,
> > + d.mapped_blocks_,
> > + d.transaction_id_,
> > + d.creation_time_,
> > + d.snapshotted_time_);
> >
> > - emit_mappings(tree_root);
> > + if (!skip_mappings_)
> > + emit_mappings(tree_root);
> >
> > - e_->end_device();
> > + e_->end_device();
> > + }
> >
> > } else if (!repair_) {
> > ostringstream msg;
> > @@ -209,6 +217,8 @@ namespace {
> > emitter::ptr e_;
> > dd_map const &dd_;
> > bool repair_;
> > + block_address dev_id_;
> > + bool skip_mappings_;
> > mapping_tree_detail::damage_visitor::ptr damage_policy_;
> > };
> > }
> > @@ -216,7 +226,7 @@ namespace {
> > //----------------------------------------------------------------
> >
> > void
> > -thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
> > +thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id)
> > {
> > details_extractor de;
> > device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair));
> > @@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair)
> > {
> > mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair));
> > mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair));
> > +
> > + mte.set_skip_mappings(skip_mappings);
> > +
> > + if (dev_id >= 0)
> > + mte.set_dev_id(dev_id);
> > walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy);
> > }
> >
> > diff --git a/man8/thin_dump.8 b/man8/thin_dump.8
> > index 7a9f785..9e0bf9e 100644
> > --- a/man8/thin_dump.8
> > +++ b/man8/thin_dump.8
> > @@ -26,9 +26,19 @@ in order to put it back onto a metadata
> >
> > This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used.
> >
> > -.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP".
> > +.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP".
> > Print output in XML or human readable format.
> > -
> > +.sp
> > +The
> > +.B old
> > +format requires the --device-id option and emits in the
> > +following format, one record per line:
> > +.B offset:length:tdata_offset
> > +(hence, "o:L:d") with units in bytes.
> > +.sp
> > +The
> > +.B null
> > +format emits nothing and only walks the tree.
> > .IP "\fB\-r, \-\-repair\fP".
> > Repair the metadata whilst dumping it.
> >
> > @@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#.
> > See the thin provisioning target documentation on how to create or release
> > a metadata snapshot and retrieve the block number from the kernel.
> >
> > +.IP "\fB\-d, \-\-skip\-mappings\fP".
> > +Skip emission of the mappings. This outputs nothing if format is
> > +either of
> > +.B old
> > +or
> > +.B null
> > +.
> > +
> > +.IP "\fB\-d, \-\-device\-id\fP".
> > +Specify the device_id to be dumped.
> > +
> > .IP "\fB\-h, \-\-help\fP".
> > Print help and exit.
> >
> > diff --git a/Makefile.in b/Makefile.in
> > index e67b300..078bb53 100644
> > --- a/Makefile.in
> > +++ b/Makefile.in
> > @@ -73,6 +73,8 @@ SOURCE=\
> > persistent-data/validators.cc \
> > thin-provisioning/device_tree.cc \
> > thin-provisioning/human_readable_format.cc \
> > + thin-provisioning/old_format.cc \
> > + thin-provisioning/null_format.cc \
> > thin-provisioning/mapping_tree.cc \
> > thin-provisioning/metadata.cc \
> > thin-provisioning/metadata_checker.cc \
> > diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h
> > new file mode 100644
> > index 0000000..dba69e9
> > --- /dev/null
> > +++ b/thin-provisioning/old_format.h
> > @@ -0,0 +1,34 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools. If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#ifndef OLD_FORMAT_H
> > +#define OLD_FORMAT_H
> > +
> > +#include "emitter.h"
> > +
> > +#include <iosfwd>
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace thin_provisioning {
> > + emitter::ptr create_old_emitter(std::ostream &out);
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +#endif
> > diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc
> > new file mode 100644
> > index 0000000..52056c8
> > --- /dev/null
> > +++ b/thin-provisioning/old_format.cc
> > @@ -0,0 +1,103 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools. If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#include "old_format.h"
> > +
> > +#include <iostream>
> > +#include <boost/format.hpp>
> > +
> > +using namespace std;
> > +using namespace thin_provisioning;
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace {
> > + template <typename T>
> > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> > + if (maybe)
> > + out << *maybe;
> > +
> > + return out;
> > + }
> > +
> > + class old_emitter : public emitter {
> > + public:
> > + old_emitter(ostream &out)
> > + : out_(out) {
> > + }
> > +
> > + void begin_superblock(string const &uuid,
> > + uint64_t time,
> > + uint64_t trans_id,
> > + uint32_t data_block_size,
> > + uint64_t nr_data_blocks,
> > + boost::optional<uint64_t> metadata_snap) {
> > + data_block_size_ = data_block_size;
> > + }
> > +
> > + void end_superblock() {
> > + }
> > +
> > + void begin_device(uint32_t dev_id,
> > + uint64_t mapped_blocks,
> > + uint64_t trans_id,
> > + uint64_t creation_time,
> > + uint64_t snap_time) {
> > + }
> > +
> > + void end_device() {
> > + }
> > +
> > + void begin_named_mapping(string const &name) {
> > + }
> > +
> > + void end_named_mapping() {
> > + }
> > +
> > + void identifier(string const &name) {
> > + }
> > +
> > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> > + out_ << (data_block_size_ << 9)*origin_begin
> > + << ":" << (data_block_size_ << 9)*len
> > + << ":" << (data_block_size_ << 9)*data_begin
> > + << endl;
> > + }
> > +
> > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> > + out_ << (data_block_size_ << 9)*origin_block
> > + << ":" << (data_block_size_ << 9)
> > + << ":" << (data_block_size_ << 9)*data_block
> > + << endl;
> > + }
> > +
> > + private:
> > + ostream &out_;
> > + uint64_t data_block_size_;
> > + };
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +thin_provisioning::emitter::ptr
> > +thin_provisioning::create_old_emitter(ostream &out)
> > +{
> > + return emitter::ptr(new old_emitter(out));
> > +}
> > +
> > +//----------------------------------------------------------------
> > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> > new file mode 100644
> > index 0000000..2688762
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.h
> > @@ -0,0 +1,34 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools. If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#ifndef NULL_FORMAT_H
> > +#define NULL_FORMAT_H
> > +
> > +#include "emitter.h"
> > +
> > +#include <iosfwd>
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace thin_provisioning {
> > + emitter::ptr create_null_emitter(std::ostream &out);
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +#endif
> > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> > new file mode 100644
> > index 0000000..b1baa7d
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.cc
> > @@ -0,0 +1,92 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools. If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#include "null_format.h"
> > +
> > +#include <iostream>
> > +
> > +using namespace std;
> > +using namespace thin_provisioning;
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace {
> > + template <typename T>
> > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> > + if (maybe)
> > + out << *maybe;
> > +
> > + return out;
> > + }
> > +
> > + class null_emitter : public emitter {
> > + public:
> > + null_emitter(ostream &out)
> > + : out_(out) {
> > + }
> > +
> > + void begin_superblock(string const &uuid,
> > + uint64_t time,
> > + uint64_t trans_id,
> > + uint32_t data_block_size,
> > + uint64_t nr_data_blocks,
> > + boost::optional<uint64_t> metadata_snap) {
> > + }
> > +
> > + void end_superblock() {
> > + }
> > +
> > + void begin_device(uint32_t dev_id,
> > + uint64_t mapped_blocks,
> > + uint64_t trans_id,
> > + uint64_t creation_time,
> > + uint64_t snap_time) {
> > + }
> > +
> > + void end_device() {
> > + }
> > +
> > + void begin_named_mapping(string const &name) {
> > + }
> > +
> > + void end_named_mapping() {
> > + }
> > +
> > + void identifier(string const &name) {
> > + }
> > +
> > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> > + }
> > +
> > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> > + }
> > +
> > + private:
> > + ostream &out_;
> > + };
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +thin_provisioning::emitter::ptr
> > +thin_provisioning::create_null_emitter(ostream &out)
> > +{
> > + return emitter::ptr(new null_emitter(out));
> > +}
> > +
> > +//----------------------------------------------------------------
> > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h
> > new file mode 100644
> > index 0000000..2688762
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.h
> > @@ -0,0 +1,34 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools. If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#ifndef NULL_FORMAT_H
> > +#define NULL_FORMAT_H
> > +
> > +#include "emitter.h"
> > +
> > +#include <iosfwd>
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace thin_provisioning {
> > + emitter::ptr create_null_emitter(std::ostream &out);
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +#endif
> > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc
> > new file mode 100644
> > index 0000000..b1baa7d
> > --- /dev/null
> > +++ b/thin-provisioning/null_format.cc
> > @@ -0,0 +1,92 @@
> > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved.
> > +//
> > +// This file is part of the thin-provisioning-tools source.
> > +//
> > +// thin-provisioning-tools is free software: you can redistribute it
> > +// and/or modify it under the terms of the GNU General Public License
> > +// as published by the Free Software Foundation, either version 3 of
> > +// the License, or (at your option) any later version.
> > +//
> > +// thin-provisioning-tools is distributed in the hope that it will be
> > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty
> > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +// GNU General Public License for more details.
> > +//
> > +// You should have received a copy of the GNU General Public License along
> > +// with thin-provisioning-tools. If not, see
> > +// <http://www.gnu.org/licenses/>.
> > +
> > +#include "null_format.h"
> > +
> > +#include <iostream>
> > +
> > +using namespace std;
> > +using namespace thin_provisioning;
> > +
> > +//----------------------------------------------------------------
> > +
> > +namespace {
> > + template <typename T>
> > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) {
> > + if (maybe)
> > + out << *maybe;
> > +
> > + return out;
> > + }
> > +
> > + class null_emitter : public emitter {
> > + public:
> > + null_emitter(ostream &out)
> > + : out_(out) {
> > + }
> > +
> > + void begin_superblock(string const &uuid,
> > + uint64_t time,
> > + uint64_t trans_id,
> > + uint32_t data_block_size,
> > + uint64_t nr_data_blocks,
> > + boost::optional<uint64_t> metadata_snap) {
> > + }
> > +
> > + void end_superblock() {
> > + }
> > +
> > + void begin_device(uint32_t dev_id,
> > + uint64_t mapped_blocks,
> > + uint64_t trans_id,
> > + uint64_t creation_time,
> > + uint64_t snap_time) {
> > + }
> > +
> > + void end_device() {
> > + }
> > +
> > + void begin_named_mapping(string const &name) {
> > + }
> > +
> > + void end_named_mapping() {
> > + }
> > +
> > + void identifier(string const &name) {
> > + }
> > +
> > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) {
> > + }
> > +
> > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) {
> > + }
> > +
> > + private:
> > + ostream &out_;
> > + };
> > +}
> > +
> > +//----------------------------------------------------------------
> > +
> > +thin_provisioning::emitter::ptr
> > +thin_provisioning::create_null_emitter(ostream &out)
> > +{
> > + return emitter::ptr(new null_emitter(out));
> > +}
> > +
> > +//----------------------------------------------------------------
> >
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-17 22:45 ` Eric Wheeler
@ 2016-03-18 2:04 ` Ming-Hung Tsai
2016-03-18 6:45 ` Ming-Hung Tsai
2016-03-20 15:59 ` Ming-Hung Tsai
2 siblings, 0 replies; 14+ messages in thread
From: Ming-Hung Tsai @ 2016-03-18 2:04 UTC (permalink / raw)
To: Eric Wheeler; +Cc: device-mapper development
2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
>
> On Thu, 17 Mar 2016, Joe Thornber wrote:
>
>> If you're skipping the mappings does the new thin_ls provide enough
>> information for you?
>
> Our primary need is the ability to specify --device-id in the o:L:d output
> format (old_format.cc) with mappings. Our nightly dumps now take 50
> seconds instead of 40 minutes since we only dump the dev_id's that we
> need.
>
> As for thin_ls, we're still in the 0.5.x branch, I couldn't get 0.6.x to
> build on el7:
>
> # git checkout v0.6.2-rc6
> Previous HEAD position was e46bdfd... v0.6.1
> HEAD is now at afb71d8... bump version
>
> # ./configure
> [...]
>
> # make
> [CXX] base/application.cc
> g++: error: @LFS_FLAGS@: No such file or directory
> make: *** [base/application.o] Error 1
>
>
> Am I missing something trivial here?
Did you run autoconf to regenerate the configure script ? v0.6.x has
some changes in the configure script.
# make distclean
# autoconf
# ./configure
# make
Ming-Hung Tsai
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-17 22:45 ` Eric Wheeler
2016-03-18 2:04 ` Ming-Hung Tsai
@ 2016-03-18 6:45 ` Ming-Hung Tsai
2016-03-20 15:59 ` Ming-Hung Tsai
2 siblings, 0 replies; 14+ messages in thread
From: Ming-Hung Tsai @ 2016-03-18 6:45 UTC (permalink / raw)
To: Eric Wheeler; +Cc: device-mapper development
2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
>
> On Thu, 17 Mar 2016, Joe Thornber wrote:
>
>> If you're skipping the mappings does the new thin_ls provide enough
>> information for you?
>
> Our primary need is the ability to specify --device-id in the o:L:d output
> format (old_format.cc) with mappings. Our nightly dumps now take 50
> seconds instead of 40 minutes since we only dump the dev_id's that we
> need.
>
I have similar requirement. I did this by my own compact_emitter,
which is a simplified version of "human_readable_emitter", without showing
the arrows and parentheses:
output format:
begin superblock=>uuid:"",time:<time>,transaction:<tid>,flags:<flags>,version:<version>,data_block_size:<bs>,nr_data_blocks:<nr_blocks>
device=>dev_id:<dev_id>,mapped_blocks:<mapped_blocks>,transaction:<tid>,creation_time:<creation_time>,snap_time:<snap_time>
<origin_begin>,<data_begin>,<len>,<time>
<origin_begin>,<data_begin>,1,<time>
...
end superblock
e.g.,
begin superblock=>uuid:"",time:0,transaction:1,flags:0,version:1,data_block_size:2048,nr_data_blocks:5657156
device=>dev_id:10000,mapped_blocks:3864041,transaction:0,creation_time:0,snap_time:0
1000,2000,100,0
1100,3000,200,0
...
end superblock
The <time> field is preserved for snapshot comparison.
Maybe we could discuss on creating a flexible emitter that could
customize the output format:
- Mapping units: sectors or blocks
- Emit the superblock and device tags or not. I think that an end mark
might be better for network transmission,
to determine whether the dump was ended or not.
- Other consideration: should the customized output compatible to thin_restore ?
Ming-Hung Tsai
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-17 22:45 ` Eric Wheeler
2016-03-18 2:04 ` Ming-Hung Tsai
2016-03-18 6:45 ` Ming-Hung Tsai
@ 2016-03-20 15:59 ` Ming-Hung Tsai
2 siblings, 0 replies; 14+ messages in thread
From: Ming-Hung Tsai @ 2016-03-20 15:59 UTC (permalink / raw)
To: device-mapper development
2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>:
>
> On Thu, 17 Mar 2016, Joe Thornber wrote:
>
>> If you're skipping the mappings does the new thin_ls provide enough
>> information for you?
>
> Our primary need is the ability to specify --device-id in the o:L:d output
> format (old_format.cc) with mappings. Our nightly dumps now take 50
> seconds instead of 40 minutes since we only dump the dev_id's that we
> need.
I think that the --device-id option is required for thin_dump, but the
necessity of --skip-mappings could be discussed. If thin_ls supports
--device-id, then thin_dump might not require --skip-mappings.
I put my code here for reference (based on v0.7-devel)
https://github.com/mingnus/thin-provisioning-tools/commits/v0.7-thin_dump-enhancement
I borrow the option name "--skip-mappings" and function name
"set_skip_mappings" from Eric, since that they sounds more clear.
Build method:
# autoconf; ./configure --enable-dev-tools; make
My use cases:
# thin_dump --device-id=<dev_id> --format=<f> [--skip-mappings]
Ming-Hung Tsai
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-15 1:45 [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's Eric Wheeler
2016-03-15 10:59 ` Thanos Makatos
2016-03-17 15:45 ` Joe Thornber
@ 2016-03-24 15:35 ` Joe Thornber
2016-03-28 2:51 ` Eric Wheeler
2016-03-28 22:05 ` Eric Wheeler
2 siblings, 2 replies; 14+ messages in thread
From: Joe Thornber @ 2016-03-24 15:35 UTC (permalink / raw)
To: Eric Wheeler; +Cc: dm-devel
On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> Hi Joe,
>
> Please review the patch below when you have a moment. I am interested in
> your feedback, and also interested in having this functionality merged
> upstream. This was written against thin-provisioning-tools.git tag
> v0.5.6.
These changes are now on the v0.7-devel branch:
i) --dev-id to filter thin_dump output on device id, multiple selections allowed.
ii) --skip-mappings
iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own.
eg,
thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda
The contrib/ directory is for third party contributions. Use 'make
contrib' to build it. I've put your emitter and tmakatos' emitter in
there already.
I hope that covers everything you need.
- Joe
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-24 15:35 ` Joe Thornber
@ 2016-03-28 2:51 ` Eric Wheeler
2016-03-28 22:05 ` Eric Wheeler
1 sibling, 0 replies; 14+ messages in thread
From: Eric Wheeler @ 2016-03-28 2:51 UTC (permalink / raw)
To: Joe Thornber; +Cc: dm-devel
On Thu, 24 Mar 2016, Joe Thornber wrote:
> On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> > Hi Joe,
> >
> > Please review the patch below when you have a moment. I am interested in
> > your feedback, and also interested in having this functionality merged
> > upstream. This was written against thin-provisioning-tools.git tag
> > v0.5.6.
>
> These changes are now on the v0.7-devel branch:
>
> i) --dev-id to filter thin_dump output on device id, multiple selections allowed.
> ii) --skip-mappings
> iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own.
>
> eg,
>
> thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda
>
>
> The contrib/ directory is for third party contributions. Use 'make
> contrib' to build it. I've put your emitter and tmakatos' emitter in
> there already.
>
> I hope that covers everything you need.
I think that will work nicely. Thank you!
-Eric
>
> - Joe
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's
2016-03-24 15:35 ` Joe Thornber
2016-03-28 2:51 ` Eric Wheeler
@ 2016-03-28 22:05 ` Eric Wheeler
1 sibling, 0 replies; 14+ messages in thread
From: Eric Wheeler @ 2016-03-28 22:05 UTC (permalink / raw)
To: Joe Thornber; +Cc: dm-devel
On Thu, 24 Mar 2016, Joe Thornber wrote:
> On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote:
> > Hi Joe,
> >
> > Please review the patch below when you have a moment. I am interested in
> > your feedback, and also interested in having this functionality merged
> > upstream. This was written against thin-provisioning-tools.git tag
> > v0.5.6.
>
> These changes are now on the v0.7-devel branch:
>
> i) --dev-id to filter thin_dump output on device id, multiple selections allowed.
> ii) --skip-mappings
> iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own.
>
> eg,
>
> thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda
>
>
> The contrib/ directory is for third party contributions. Use 'make
> contrib' to build it. I've put your emitter and tmakatos' emitter in
> there already.
It took me a bit to figure out that I needed --enable-testing to get
contrib/ to build. You might remove the --enable-testing requirement
within the Makefile if contrib will remain as an optional Makefile target
that doesn't usually build with `make`.
Also FYI: I noticed that `-ldl' needed added to to the g++ linking
line:
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:15: undefined reference to `dlopen'
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:19: undefined reference to `dlerror'
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:20: undefined reference to `dlsym'
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:22: undefined reference to `dlerror'
/usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:17: undefined reference to `dlerror'
So I added the -ldl as below:
g++ -g -Wall -fPIC -fno-strict-aliasing -std=c++11 -O8
-DSTRERROR_R_CHAR_P -o bin/pdata_tools base/application.o base/base64.o
base/disk_units.o base/endian_utils.o base/error_state.o
base/error_string.o base/grid_layout.o base/progress_monitor.o
base/rolling_hash.o base/xml_utils.o block-cache/block_cache.o
caching/cache_check.o caching/cache_dump.o caching/cache_metadata_size.o
caching/cache_repair.o caching/cache_restore.o caching/commands.o
caching/hint_array.o caching/mapping_array.o caching/metadata.o
caching/metadata_dump.o caching/restore_emitter.o caching/superblock.o
caching/xml_format.o era/commands.o era/era_array.o era/era_check.o
era/era_detail.o era/era_dump.o era/era_invalidate.o era/era_restore.o
era/metadata.o era/metadata_dump.o era/restore_emitter.o era/superblock.o
era/writeset_tree.o era/xml_format.o main.o persistent-data/checksum.o
persistent-data/data-structures/bitset.o
persistent-data/data-structures/bloom_filter.o
persistent-data/data-structures/btree.o persistent-data/error_set.o
persistent-data/file_utils.o persistent-data/hex_dump.o
persistent-data/space-maps/careful_alloc.o
persistent-data/space-maps/disk.o persistent-data/space-maps/recursive.o
persistent-data/space_map.o persistent-data/transaction_manager.o
persistent-data/validators.o thin-provisioning/commands.o
thin-provisioning/cache_stream.o thin-provisioning/chunk_stream.o
thin-provisioning/device_tree.o thin-provisioning/fixed_chunk_stream.o
thin-provisioning/human_readable_format.o thin-provisioning/mapping_tree.o
thin-provisioning/metadata.o thin-provisioning/metadata_checker.o
thin-provisioning/metadata_counter.o thin-provisioning/metadata_dumper.o
thin-provisioning/pool_stream.o thin-provisioning/restore_emitter.o
thin-provisioning/rmap_visitor.o thin-provisioning/superblock.o
thin-provisioning/shared_library_emitter.o thin-provisioning/thin_check.o
thin-provisioning/thin_delta.o thin-provisioning/thin_dump.o
thin-provisioning/thin_ls.o thin-provisioning/thin_metadata_size.o
thin-provisioning/thin_pool.o thin-provisioning/thin_repair.o
thin-provisioning/thin_restore.o thin-provisioning/thin_rmap.o
thin-provisioning/thin_trim.o thin-provisioning/xml_format.o -laio -lexpat
-lstdc++ -ldl
^^^^ ADDED
>
> I hope that covers everything you need.
>
> - Joe
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2016-03-28 22:05 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-15 1:45 [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's Eric Wheeler
2016-03-15 10:59 ` Thanos Makatos
2016-03-15 17:51 ` Eric Wheeler
2016-03-16 5:42 ` Ming-Hung Tsai
2016-03-16 19:14 ` Eric Wheeler
2016-03-17 15:44 ` Joe Thornber
2016-03-17 15:45 ` Joe Thornber
2016-03-17 22:45 ` Eric Wheeler
2016-03-18 2:04 ` Ming-Hung Tsai
2016-03-18 6:45 ` Ming-Hung Tsai
2016-03-20 15:59 ` Ming-Hung Tsai
2016-03-24 15:35 ` Joe Thornber
2016-03-28 2:51 ` Eric Wheeler
2016-03-28 22:05 ` Eric Wheeler
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.