[PATCH 0/8] Write-intent bitmap support for IMSM metadata

[PATCH 0/8] Write-intent bitmap support for IMSM metadata

[Jakub Radtke]

This patchset is aimed to add write-intent bitmap support for IMSM
metadata.

Additional function in the superswitch (set_bitmap) is proposed as
a place where the internal bitmaps configuration for non-native
metadata can be performed (through sysfs).

Adding the bitmap to existing volumes is implemented similarly like
for the PPL functionality (through --update-subarray).

The patchset was tested on HW and qemu.

Jakub Radtke (8):
  Modify mdstat parsing for volumes with the bitmap
  Enable bitmap support for external metadata
  imsm: Write-intent bitmap support
  imsm: Adding a spare to an existing array with bitmap
  Add "bitmap" to allowed command-line values
  imsm: Update-subarray for write-intent bitmap
  Create: Block automatic enabling bitmap for external metadata
  Grow: Block reshape when external metadata and write-intent bitmap

 Assemble.c    |   6 +
 Create.c      |  11 +
 Grow.c        |  24 +-
 mdadm.8.in    |  13 +-
 mdadm.c       |   4 +-
 mdadm.h       |   3 +
 mdstat.c      |   6 +
 super-intel.c | 679 +++++++++++++++++++++++++++++++++++++++++++++++++-
 8 files changed, 725 insertions(+), 21 deletions(-)

-- 
2.26.2

[PATCH 1/8] Modify mdstat parsing for volumes with the bitmap

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

Current mdstat read functionality is not working correctly
for the volumes with the write-intent bitmap.
It affects rebuild and reshape use cases.

Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Change-Id: I5288fbfb7d6a85b2aa8452675a6e3559bf4fe934
---
 mdstat.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/mdstat.c b/mdstat.c
index dd96cca7..2fd792c5 100644
--- a/mdstat.c
+++ b/mdstat.c
@@ -191,6 +191,12 @@ struct mdstat_ent *mdstat_read(int hold, int start)
 			else if (strcmp(w, "inactive") == 0) {
 				ent->active = 0;
 				in_devs = 1;
+			} else if (strcmp(w, "bitmap:") == 0) {
+				/* We need to stop parsing here;
+				 * otherwise, ent->raid_disks will be
+				 * overwritten by the wrong value.
+				 */
+				break;
 			} else if (ent->active > 0 &&
 				 ent->level == NULL &&
 				 w[0] != '(' /*readonly*/) {
-- 
2.26.2

[PATCH 2/8] Enable bitmap support for external metadata

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

The patch enables the implementation of a write-intent bitmap for external
metadata.
Configuration of the internal bitmaps for non-native metadata requires the
extension in superswitch to perform an additional sysfs setup before the
array is activated.

Change-Id: I77be689b9f4f815dd00bb14ef52267686b273fb5
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
---
 Assemble.c |  6 ++++++
 Create.c   | 10 ++++++++++
 mdadm.h    |  3 +++
 3 files changed, 19 insertions(+)

diff --git a/Assemble.c b/Assemble.c
index ed0ddfb1..479ff5ec 100644
--- a/Assemble.c
+++ b/Assemble.c
@@ -2094,6 +2094,12 @@ int assemble_container_content(struct supertype *st, int mdfd,
 		}
 	}
 
+	/* Before activating the array, perform extra steps required to configure the internal
+	 * write-intent bitmap.
+	 */
+	if (content->consistency_policy == CONSISTENCY_POLICY_BITMAP &&
+	    st->ss->set_bitmap)
+		st->ss->set_bitmap(st, content);
 
 	if (start_reshape) {
 		int spare = content->array.raid_disks + expansion;
diff --git a/Create.c b/Create.c
index 51f8826b..751f18c1 100644
--- a/Create.c
+++ b/Create.c
@@ -989,6 +989,16 @@ int Create(struct supertype *st, char *mddev,
 				st->ss->free_super(st);
 				goto abort_locked;
 			}
+			/* Before activating the array, perform extra steps required
+			 * to configure the internal write-intent bitmap.
+			 */
+			if (info_new.consistency_policy ==
+				    CONSISTENCY_POLICY_BITMAP &&
+			    st->ss->set_bitmap &&
+			    st->ss->set_bitmap(st, &info)) {
+				st->ss->free_super(st);
+				goto abort_locked;
+			}
 
 			/* update parent container uuid */
 			if (me) {
diff --git a/mdadm.h b/mdadm.h
index 1ee6c92e..0cf94239 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1006,6 +1006,9 @@ extern struct superswitch {
 	int (*add_internal_bitmap)(struct supertype *st, int *chunkp,
 				   int delay, int write_behind,
 				   unsigned long long size, int may_change, int major);
+	/* Perform additional setup required to activate a bitmap.
+	 */
+	int (*set_bitmap)(struct supertype *st, struct mdinfo *info);
 	/* Seek 'fd' to start of write-intent-bitmap.  Must be an
 	 * md-native format bitmap
 	 */
-- 
2.26.2

[PATCH 3/8] imsm: Write-intent bitmap support

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

This patch enables the bitmap functionality with the IMSM metadata format.
The functionality covers "internal" bitmap.

There are the following limitation to "internal" metadata:
- creating the bitmap using --grow is not supported
- parameter --bitmap-chunk is ignored (default value 64MiB is used)

Change-Id: I22826f736bc013aa5516239e1e0c38d4eec32f62
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
---
 super-intel.c | 619 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 617 insertions(+), 2 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 715febf7..edb1c60e 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -96,6 +96,23 @@
 						   * mutliple PPL area
 						   */
 
+/*
+ * Internal Write-intent bitmap is stored in the same area where PPL.
+ * Both features are mutually exclusive, so it is not an issue.
+ * The first 8KiB of the area are reserved and shall not be used.
+ */
+#define IMSM_BITMAP_AREA_RESERVED_SIZE 8192
+
+#define IMSM_BITMAP_HEADER_OFFSET (IMSM_BITMAP_AREA_RESERVED_SIZE)
+#define IMSM_BITMAP_HEADER_SIZE MAX_SECTOR_SIZE
+
+#define IMSM_BITMAP_START_OFFSET (IMSM_BITMAP_HEADER_OFFSET + IMSM_BITMAP_HEADER_SIZE)
+#define IMSM_BITMAP_AREA_SIZE (MULTIPLE_PPL_AREA_SIZE_IMSM - IMSM_BITMAP_START_OFFSET)
+#define IMSM_BITMAP_AND_HEADER_SIZE (IMSM_BITMAP_AREA_SIZE + IMSM_BITMAP_HEADER_SIZE)
+
+#define IMSM_DEFAULT_BITMAP_CHUNKSIZE (64 * 1024 * 1024)
+#define IMSM_DEFAULT_BITMAP_DAEMON_SLEEP 5
+
 /*
  * This macro let's us ensure that no-one accidentally
  * changes the size of a struct
@@ -229,6 +246,7 @@ struct imsm_dev {
 #define RWH_MULTIPLE_DISTRIBUTED 3
 #define RWH_MULTIPLE_PPLS_JOURNALING_DRIVE 4
 #define RWH_MULTIPLE_OFF 5
+#define RWH_BITMAP 6
 	__u8  rwh_policy; /* Raid Write Hole Policy */
 	__u8  jd_serial[MAX_RAID_SERIAL_LEN]; /* Journal Drive serial number */
 	__u8  filler1;
@@ -1682,6 +1700,8 @@ static void print_imsm_dev(struct intel_super *super,
 		printf("Multiple distributed PPLs\n");
 	else if (dev->rwh_policy == RWH_MULTIPLE_PPLS_JOURNALING_DRIVE)
 		printf("Multiple PPLs on journaling drive\n");
+	else if (dev->rwh_policy == RWH_BITMAP)
+		printf("Write-intent bitmap\n");
 	else
 		printf("<unknown:%d>\n", dev->rwh_policy);
 
@@ -3310,6 +3330,53 @@ static unsigned long long imsm_component_size_alignment_check(int level,
 	return component_size;
 }
 
+/*******************************************************************************
+ * Function:	get_bitmap_header_sector
+ * Description:	Returns the sector where the bitmap header is placed.
+ * Parameters:
+ *	st		: supertype information
+ *	dev_idx		: index of the device with bitmap
+ *
+ * Returns:
+ *	 The sector where the bitmap header is placed
+ ******************************************************************************/
+static unsigned long long get_bitmap_header_sector(struct intel_super *super,
+						   int dev_idx)
+{
+	struct imsm_dev *dev = get_imsm_dev(super, dev_idx);
+	struct imsm_map *map = get_imsm_map(dev, MAP_0);
+
+	if (!super->sector_size) {
+		dprintf("sector size is not set\n");
+		return 0;
+	}
+
+	return pba_of_lba0(map) + calc_component_size(map, dev) +
+	       (IMSM_BITMAP_HEADER_OFFSET / super->sector_size);
+}
+
+/*******************************************************************************
+ * Function:	get_bitmap_sector
+ * Description:	Returns the sector where the bitmap is placed.
+ * Parameters:
+ *	st		: supertype information
+ *	dev_idx		: index of the device with bitmap
+ *
+ * Returns:
+ *	 The sector where the bitmap is placed
+ ******************************************************************************/
+static unsigned long long get_bitmap_sector(struct intel_super *super,
+					    int dev_idx)
+{
+	if (!super->sector_size) {
+		dprintf("sector size is not set\n");
+		return 0;
+	}
+
+	return get_bitmap_header_sector(super, dev_idx) +
+	       (IMSM_BITMAP_HEADER_SIZE / super->sector_size);
+}
+
 static unsigned long long get_ppl_sector(struct intel_super *super, int dev_idx)
 {
 	struct imsm_dev *dev = get_imsm_dev(super, dev_idx);
@@ -3430,7 +3497,12 @@ static void getinfo_super_imsm_volume(struct supertype *st, struct mdinfo *info,
 	} else if (info->array.level <= 0) {
 		info->consistency_policy = CONSISTENCY_POLICY_NONE;
 	} else {
-		info->consistency_policy = CONSISTENCY_POLICY_RESYNC;
+		if (dev->rwh_policy == RWH_BITMAP) {
+			info->bitmap_offset = get_bitmap_sector(super, super->current_vol);
+			info->consistency_policy = CONSISTENCY_POLICY_BITMAP;
+		} else {
+			info->consistency_policy = CONSISTENCY_POLICY_RESYNC;
+		}
 	}
 
 	info->reshape_progress = 0;
@@ -6465,6 +6537,60 @@ static int write_init_ppl_imsm_all(struct supertype *st, struct mdinfo *info)
 	return ret;
 }
 
+/*******************************************************************************
+ * Function:	write_init_bitmap_imsm_vol
+ * Description:	Write a bitmap header and prepares the area for the bitmap.
+ * Parameters:
+ *	st	: supertype information
+ *	vol_idx	: the volume index to use
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int write_init_bitmap_imsm_vol(struct supertype *st, int vol_idx)
+{
+	struct intel_super *super = st->sb;
+	int prev_current_vol = super->current_vol;
+	struct dl *d;
+	int ret = 0;
+
+	super->current_vol = vol_idx;
+	for (d = super->disks; d; d = d->next) {
+		if (d->index < 0 || is_failed(&d->disk))
+			continue;
+		ret = st->ss->write_bitmap(st, d->fd, NoUpdate);
+		if (ret)
+			break;
+	}
+	super->current_vol = prev_current_vol;
+	return ret;
+}
+
+/*******************************************************************************
+ * Function:	write_init_bitmap_imsm_all
+ * Description:	Write a bitmap header and prepares the area for the bitmap.
+ *		Operation is executed for volumes with CONSISTENCY_POLICY_BITMAP.
+ * Parameters:
+ *	st	: supertype information
+ *	info	: info about the volume where the bitmap should be written
+ *	vol_idx	: the volume index to use
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int write_init_bitmap_imsm_all(struct supertype *st, struct mdinfo *info,
+				      int vol_idx)
+{
+	int ret = 0;
+
+	if (info && (info->consistency_policy == CONSISTENCY_POLICY_BITMAP))
+		ret = write_init_bitmap_imsm_vol(st, vol_idx);
+
+	return ret;
+}
+
 static int write_init_super_imsm(struct supertype *st)
 {
 	struct intel_super *super = st->sb;
@@ -6488,7 +6614,10 @@ static int write_init_super_imsm(struct supertype *st)
 			 */
 			rv = mgmt_disk(st);
 		} else {
+			/* adding the second volume to the array */
 			rv = write_init_ppl_imsm_all(st, &info);
+			if (!rv)
+				rv = write_init_bitmap_imsm_all(st, &info, current_vol);
 			if (!rv)
 				rv = create_array(st, current_vol);
 		}
@@ -6496,8 +6625,12 @@ static int write_init_super_imsm(struct supertype *st)
 		struct dl *d;
 		for (d = super->disks; d; d = d->next)
 			Kill(d->devname, NULL, 0, -1, 1);
-		if (current_vol >= 0)
+		if (current_vol >= 0) {
 			rv = write_init_ppl_imsm_all(st, &info);
+			if (!rv)
+				rv = write_init_bitmap_imsm_all(st, &info, current_vol);
+		}
+
 		if (!rv)
 			rv = write_super_imsm(st, 1);
 	}
@@ -12306,6 +12439,483 @@ abort:
 	return ret_val;
 }
 
+/*******************************************************************************
+ * Function:	calculate_bitmap_min_chunksize
+ * Description:	Calculates the minimal valid bitmap chunk size
+ * Parameters:
+ *	max_bits	: indicate how many bits can be used for the bitmap
+ *	data_area_size	: the size of the data area covered by the bitmap
+ *
+ * Returns:
+ *	 The bitmap chunk size
+ ******************************************************************************/
+static unsigned long long
+calculate_bitmap_min_chunksize(unsigned long long max_bits,
+			       unsigned long long data_area_size)
+{
+	unsigned long long min_chunk =
+		4096; /* sub-page chunks don't work yet.. */
+	unsigned long long bits = data_area_size / min_chunk + 1;
+
+	while (bits > max_bits) {
+		min_chunk *= 2;
+		bits = (bits + 1) / 2;
+	}
+	return min_chunk;
+}
+
+/*******************************************************************************
+ * Function:	calculate_bitmap_chunksize
+ * Description:	Calculates the bitmap chunk size for the given device
+ * Parameters:
+ *	st	: supertype information
+ *	dev	: device for the bitmap
+ *
+ * Returns:
+ *	 The bitmap chunk size
+ ******************************************************************************/
+static unsigned long long calculate_bitmap_chunksize(struct supertype *st,
+						     struct imsm_dev *dev)
+{
+	struct intel_super *super = st->sb;
+	unsigned long long min_chunksize;
+	unsigned long long result = IMSM_DEFAULT_BITMAP_CHUNKSIZE;
+	size_t dev_size = imsm_dev_size(dev);
+
+	min_chunksize = calculate_bitmap_min_chunksize(
+		IMSM_BITMAP_AREA_SIZE * super->sector_size, dev_size);
+
+	if (result < min_chunksize)
+		result = min_chunksize;
+
+	return result;
+}
+
+/*******************************************************************************
+ * Function:	init_bitmap_header
+ * Description:	Initialize the bitmap header structure
+ * Parameters:
+ *	st	: supertype information
+ *	bms	: bitmap header struct to initialize
+ *	dev	: device for the bitmap
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int init_bitmap_header(struct supertype *st, struct bitmap_super_s *bms,
+			      struct imsm_dev *dev)
+{
+	int vol_uuid[4];
+
+	if (!bms || !dev)
+		return -1;
+
+	bms->magic = __cpu_to_le32(BITMAP_MAGIC);
+	bms->version = __cpu_to_le32(BITMAP_MAJOR_HI);
+	bms->daemon_sleep = __cpu_to_le32(IMSM_DEFAULT_BITMAP_DAEMON_SLEEP);
+	bms->sync_size = __cpu_to_le64(IMSM_BITMAP_AREA_SIZE);
+	bms->write_behind = __cpu_to_le32(0);
+
+	uuid_from_super_imsm(st, vol_uuid);
+	memcpy(bms->uuid, vol_uuid, 16);
+
+	bms->chunksize = calculate_bitmap_chunksize(st, dev);
+
+	return 0;
+}
+
+/*******************************************************************************
+ * Function:	validate_internal_bitmap_for_drive
+ * Description:	Verify if the bitmap header for a given drive.
+ * Parameters:
+ *	st	: supertype information
+ *	offset	: The offset from the beginning of the drive where to look for
+ *		  the bitmap header.
+ *	d	: the drive info
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int validate_internal_bitmap_for_drive(struct supertype *st,
+					      unsigned long long offset,
+					      struct dl *d)
+{
+	struct intel_super *super = st->sb;
+	int ret = -1;
+	int vol_uuid[4];
+	bitmap_super_t *bms;
+	int fd;
+
+	if (!d)
+		return -1;
+
+	void *read_buf;
+
+	if (posix_memalign(&read_buf, MAX_SECTOR_SIZE, IMSM_BITMAP_HEADER_SIZE))
+		return -1;
+
+	fd = d->fd;
+	if (fd < 0) {
+		fd = open(d->devname, O_RDONLY, 0);
+		if (fd < 0) {
+			dprintf("cannot open the device %s\n", d->devname);
+			goto abort;
+		}
+	}
+
+	if (lseek64(fd, offset * super->sector_size, SEEK_SET) < 0)
+		goto abort;
+	if (read(fd, read_buf, IMSM_BITMAP_HEADER_SIZE) !=
+	    IMSM_BITMAP_HEADER_SIZE)
+		goto abort;
+
+	uuid_from_super_imsm(st, vol_uuid);
+
+	bms = read_buf;
+	if ((bms->magic != __cpu_to_le32(BITMAP_MAGIC)) ||
+	    (bms->version != __cpu_to_le32(BITMAP_MAJOR_HI)) ||
+	    (!same_uuid((int *)bms->uuid, vol_uuid, st->ss->swapuuid))) {
+		dprintf("wrong bitmap header detected\n");
+		goto abort;
+	}
+
+	ret = 0;
+abort:
+	if ((d->fd < 0) && (fd >= 0))
+		close(fd);
+	if (read_buf)
+		free(read_buf);
+
+	return ret;
+}
+
+/*******************************************************************************
+ * Function:	validate_internal_bitmap_imsm
+ * Description:	Verify if the bitmap header is in place and with proper data.
+ * Parameters:
+ *	st	: supertype information
+ *
+ * Returns:
+ *	 0 : success or device w/o RWH_BITMAP
+ *	-1 : fail
+ ******************************************************************************/
+static int validate_internal_bitmap_imsm(struct supertype *st)
+{
+	struct intel_super *super = st->sb;
+	struct imsm_dev *dev = get_imsm_dev(super, super->current_vol);
+	unsigned long long offset;
+	struct dl *d;
+
+	if (!dev)
+		return -1;
+
+	if (dev->rwh_policy != RWH_BITMAP)
+		return 0;
+
+	offset = get_bitmap_header_sector(super, super->current_vol);
+	for (d = super->disks; d; d = d->next) {
+		if (d->index < 0 || is_failed(&d->disk))
+			continue;
+
+		if (validate_internal_bitmap_for_drive(st, offset, d)) {
+			pr_err("imsm: bitmap validation failed\n");
+			return -1;
+		}
+	}
+	return 0;
+}
+
+/*******************************************************************************
+ * Function:	add_internal_bitmap_imsm
+ * Description:	Mark the volume to use the bitmap and updates the chunk size value.
+ * Parameters:
+ *	st		: supertype information
+ *	chunkp		: bitmap chunk size
+ *	delay		: not used for imsm
+ *	write_behind	: not used for imsm
+ *	size		: not used for imsm
+ *	may_change	: not used for imsm
+ *	amajor		: not used for imsm
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int add_internal_bitmap_imsm(struct supertype *st, int *chunkp,
+				    int delay, int write_behind,
+				    unsigned long long size, int may_change,
+				    int amajor)
+{
+	struct intel_super *super = st->sb;
+	int vol_idx = super->current_vol;
+	struct imsm_dev *dev;
+
+	if (!super->devlist || vol_idx == -1 || !chunkp)
+		return -1;
+
+	dev = get_imsm_dev(super, vol_idx);
+
+	if (!dev) {
+		dprintf("cannot find the device for volume index %d\n",
+			vol_idx);
+		return -1;
+	}
+	dev->rwh_policy = RWH_BITMAP;
+
+	*chunkp = calculate_bitmap_chunksize(st, dev);
+
+	return 0;
+}
+
+/*******************************************************************************
+ * Function:	locate_bitmap_imsm
+ * Description:	Seek 'fd' to start of write-intent-bitmap.
+ * Parameters:
+ *	st		: supertype information
+ *	fd		: file descriptor for the device
+ *	node_num	: not used for imsm
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int locate_bitmap_imsm(struct supertype *st, int fd, int node_num)
+{
+	struct intel_super *super = st->sb;
+	unsigned long long offset;
+	int vol_idx = super->current_vol;
+
+	if (!super->devlist || vol_idx == -1)
+		return -1;
+
+	offset = get_bitmap_header_sector(super, super->current_vol);
+	dprintf("bitmap header offset is %llu\n", offset);
+
+	lseek64(fd, offset << 9, 0);
+
+	return 0;
+}
+
+/*******************************************************************************
+ * Function:	write_init_bitmap_imsm
+ * Description:	Write a bitmap header and prepares the area for the bitmap.
+ * Parameters:
+ *	st	: supertype information
+ *	fd	: file descriptor for the device
+ *	update	: not used for imsm
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int write_init_bitmap_imsm(struct supertype *st, int fd,
+				  enum bitmap_update update)
+{
+	struct intel_super *super = st->sb;
+	int vol_idx = super->current_vol;
+	int ret = 0;
+	unsigned long long offset;
+	bitmap_super_t bms = { 0 };
+	size_t written = 0;
+	size_t to_write;
+	ssize_t rv_num;
+	void *buf;
+
+	if (!super->devlist || !super->sector_size || vol_idx == -1)
+		return -1;
+
+	struct imsm_dev *dev = get_imsm_dev(super, vol_idx);
+
+	/* first clear the space for bitmap header */
+	unsigned long long bitmap_area_start =
+		get_bitmap_header_sector(super, vol_idx);
+
+	dprintf("zeroing area start (%llu) and size (%u)\n", bitmap_area_start,
+		IMSM_BITMAP_AND_HEADER_SIZE / super->sector_size);
+	if (zero_disk_range(fd, bitmap_area_start,
+			    IMSM_BITMAP_HEADER_SIZE / super->sector_size)) {
+		pr_err("imsm: cannot zeroing the space for the bitmap\n");
+		return -1;
+	}
+
+	/* The bitmap area should be filled with "1"s to perform initial
+	 * synchronization.
+	 */
+	if (posix_memalign(&buf, MAX_SECTOR_SIZE, MAX_SECTOR_SIZE))
+		return -1;
+	memset(buf, 0xFF, MAX_SECTOR_SIZE);
+	offset = get_bitmap_sector(super, vol_idx);
+	lseek64(fd, offset << 9, 0);
+	while (written < IMSM_BITMAP_AREA_SIZE) {
+		to_write = IMSM_BITMAP_AREA_SIZE - written;
+		if (to_write > MAX_SECTOR_SIZE)
+			to_write = MAX_SECTOR_SIZE;
+		rv_num = write(fd, buf, MAX_SECTOR_SIZE);
+		if (rv_num != MAX_SECTOR_SIZE) {
+			ret = -1;
+			dprintf("cannot initialize bitmap area\n");
+			goto abort;
+		}
+		written += rv_num;
+	}
+
+	/* write a bitmap header */
+	init_bitmap_header(st, &bms, dev);
+	memset(buf, 0, MAX_SECTOR_SIZE);
+	memcpy(buf, &bms, sizeof(bitmap_super_t));
+	if (locate_bitmap_imsm(st, fd, 0)) {
+		ret = -1;
+		dprintf("cannot locate the bitmap\n");
+		goto abort;
+	}
+	if (write(fd, buf, MAX_SECTOR_SIZE) != MAX_SECTOR_SIZE) {
+		ret = -1;
+		dprintf("cannot write the bitmap header\n");
+		goto abort;
+	}
+	fsync(fd);
+
+abort:
+	free(buf);
+
+	return ret;
+}
+
+/*******************************************************************************
+ * Function:	is_vol_to_setup_bitmap
+ * Description:	Checks if a bitmap should be activated on the dev.
+ * Parameters:
+ *	info	: info about the volume to setup the bitmap
+ *	dev	: the device to check against bitmap creation
+ *
+ * Returns:
+ *	 0 : bitmap should be set up on the device
+ *	-1 : otherwise
+ ******************************************************************************/
+static int is_vol_to_setup_bitmap(struct mdinfo *info, struct imsm_dev *dev)
+{
+	if (!dev || !info)
+		return -1;
+
+	if ((strcmp((char *)dev->volume, info->name) == 0) &&
+	    (dev->rwh_policy == RWH_BITMAP))
+		return -1;
+
+	return 0;
+}
+
+/*******************************************************************************
+ * Function:	set_bitmap_sysfs
+ * Description:	Set the sysfs atributes of a given volume to activate the bitmap.
+ * Parameters:
+ *	info		: info about the volume where the bitmap should be setup
+ *	chunksize	: bitmap chunk size
+ *	location	: location of the bitmap
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int set_bitmap_sysfs(struct mdinfo *info, unsigned long long chunksize,
+			    char *location)
+{
+	/* The bitmap/metadata is set to external to allow changing of value for
+	 * bitmap/location. When external is used, the kernel will treat an offset
+	 * related to the device's first lba (in opposition to the "internal" case
+	 * when this value is related to the beginning of the superblock).
+	 */
+	if (sysfs_set_str(info, NULL, "bitmap/metadata", "external")) {
+		dprintf("failed to set bitmap/metadata\n");
+		return -1;
+	}
+
+	/* It can only be changed when no bitmap is active.
+	 * Should be bigger than 512 and must be power of 2.
+	 * It is expecting the value in bytes.
+	 */
+	if (sysfs_set_num(info, NULL, "bitmap/chunksize",
+					  __cpu_to_le32(chunksize))) {
+		dprintf("failed to set bitmap/chunksize\n");
+		return -1;
+	}
+
+	/* It is expecting the value in sectors. */
+	if (sysfs_set_num(info, NULL, "bitmap/space",
+					  __cpu_to_le64(IMSM_BITMAP_AREA_SIZE))) {
+		dprintf("failed to set bitmap/space\n");
+		return -1;
+	}
+
+	/* Determines the delay between the bitmap updates.
+	 * It is expecting the value in seconds.
+	 */
+	if (sysfs_set_num(info, NULL, "bitmap/time_base",
+					  __cpu_to_le64(IMSM_DEFAULT_BITMAP_DAEMON_SLEEP))) {
+		dprintf("failed to set bitmap/time_base\n");
+		return -1;
+	}
+
+	/* It is expecting the value in sectors with a sign at the beginning. */
+	if (sysfs_set_str(info, NULL, "bitmap/location", location)) {
+		dprintf("failed to set bitmap/location\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+/*******************************************************************************
+ * Function:	set_bitmap_imsm
+ * Description:	Setup the bitmap for the given volume
+ * Parameters:
+ *	st	: supertype information
+ *	info	: info about the volume where the bitmap should be setup
+ *
+ * Returns:
+ *	 0 : success
+ *	-1 : fail
+ ******************************************************************************/
+static int set_bitmap_imsm(struct supertype *st, struct mdinfo *info)
+{
+	struct intel_super *super = st->sb;
+	int prev_current_vol = super->current_vol;
+	struct imsm_dev *dev;
+	int ret = -1;
+	char location[16] = "";
+	unsigned long long chunksize;
+	struct intel_dev *dev_it;
+
+	for (dev_it = super->devlist; dev_it; dev_it = dev_it->next) {
+		super->current_vol = dev_it->index;
+		dev = get_imsm_dev(super, super->current_vol);
+
+		if (is_vol_to_setup_bitmap(info, dev)) {
+			if (validate_internal_bitmap_imsm(st)) {
+				dprintf("bitmap header validation failed\n");
+				goto abort;
+			}
+
+			chunksize = calculate_bitmap_chunksize(st, dev);
+			dprintf("chunk size is %llu\n", chunksize);
+
+			snprintf(location, sizeof(location), "+%llu",
+				 get_bitmap_sector(super, super->current_vol));
+			dprintf("bitmap offset is %s\n", location);
+
+			if (set_bitmap_sysfs(info, chunksize, location)) {
+				dprintf("cannot setup the bitmap\n");
+				goto abort;
+			}
+		}
+	}
+	ret = 0;
+abort:
+	super->current_vol = prev_current_vol;
+	return ret;
+}
+
 struct superswitch super_imsm = {
 	.examine_super	= examine_super_imsm,
 	.brief_examine_super = brief_examine_super_imsm,
@@ -12347,6 +12957,11 @@ struct superswitch super_imsm = {
 	.container_content = container_content_imsm,
 	.validate_container = validate_container_imsm,
 
+	.add_internal_bitmap = add_internal_bitmap_imsm,
+	.locate_bitmap = locate_bitmap_imsm,
+	.write_bitmap = write_init_bitmap_imsm,
+	.set_bitmap = set_bitmap_imsm,
+
 	.write_init_ppl = write_init_ppl_imsm,
 	.validate_ppl	= validate_ppl_imsm,
 
-- 
2.26.2

[PATCH 4/8] imsm: Adding a spare to an existing array with bitmap

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

When adding a spare to an existing array with bitmap, an additional
initialization (adding bitmap header and preparing the bitmap area)
is required.

Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Change-Id: Icaced458f4a1dd7777fce60660b749301c8496e8
---
 super-intel.c | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/super-intel.c b/super-intel.c
index edb1c60e..eb5d39b9 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -9537,6 +9537,39 @@ static int apply_size_change_update(struct imsm_update_size_change *u,
 	return ret_val;
 }
 
+static int prepare_spare_to_activate(struct supertype *st,
+				     struct imsm_update_activate_spare *u)
+{
+	struct intel_super *super = st->sb;
+	int prev_current_vol = super->current_vol;
+	struct active_array *a;
+	int ret = 1;
+
+	for (a = st->arrays; a; a = a->next)
+		/*
+		 * Additional initialization (adding bitmap header, filling
+		 * the bitmap area with '1's to force initial rebuild for a whole
+		 * data-area) is required when adding the spare to the volume
+		 * with write-intent bitmap.
+		 */
+		if (a->info.container_member == u->array &&
+		    a->info.consistency_policy == CONSISTENCY_POLICY_BITMAP) {
+			struct dl *dl;
+
+			for (dl = super->disks; dl; dl = dl->next)
+				if (dl == u->dl)
+					break;
+			if (!dl)
+				break;
+
+			super->current_vol = u->array;
+			if (st->ss->write_bitmap(st, dl->fd, NoUpdate))
+				ret = 0;
+			super->current_vol = prev_current_vol;
+		}
+	return ret;
+}
+
 static int apply_update_activate_spare(struct imsm_update_activate_spare *u,
 				       struct intel_super *super,
 				       struct active_array *active_array)
@@ -9961,7 +9994,9 @@ static void imsm_process_update(struct supertype *st,
 	}
 	case update_activate_spare: {
 		struct imsm_update_activate_spare *u = (void *) update->buf;
-		if (apply_update_activate_spare(u, super, st->arrays))
+
+		if (prepare_spare_to_activate(st, u) &&
+		    apply_update_activate_spare(u, super, st->arrays))
 			super->updates_pending++;
 		break;
 	}
-- 
2.26.2

[PATCH 5/8] Add "bitmap" to allowed command-line values

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

Currently, the sysfs interface allows bitmap setup only when volume
is in an inactive state.
For external metadata to add bitmap to existing volume instead of
GROW, the UPDATE operation can be done.
The patch adds a "bitmap" argument to the allowed values for UPDATE.

Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Change-Id: Ic46fdc6cc39eba2097a7e4956c44f4a9970e7b05
---
 mdadm.8.in | 13 +++++++++++--
 mdadm.c    |  4 +++-
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/mdadm.8.in b/mdadm.8.in
index 34a93a8f..c89d901e 100644
--- a/mdadm.8.in
+++ b/mdadm.8.in
@@ -2408,9 +2408,11 @@ or
 .B \-\-update=
 option. The supported options are
 .BR name ,
-.B ppl
+.BR ppl ,
+.BR no\-ppl ,
+.BR bitmap
 and
-.BR no\-ppl .
+.BR no\-bitmap .
 
 The
 .B name
@@ -2428,6 +2430,13 @@ and
 options enable and disable PPL in the metadata. Currently supported only for
 IMSM subarrays.
 
+The
+.B bitmap
+and
+.B no\-bitmap
+options enable and disable write-intent bitmap in the metadata. Currently supported only for
+IMSM subarrays.
+
 .TP
 .B \-\-examine
 The device should be a component of an md array.
diff --git a/mdadm.c b/mdadm.c
index 493d70e4..9a4317d5 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -779,6 +779,8 @@ int main(int argc, char *argv[])
 				continue;
 			if (strcmp(c.update, "devicesize") == 0)
 				continue;
+			if (strcmp(c.update, "bitmap") == 0)
+				continue;
 			if (strcmp(c.update, "no-bitmap") == 0)
 				continue;
 			if (strcmp(c.update, "bbl") == 0)
@@ -827,7 +829,7 @@ int main(int argc, char *argv[])
 			fprintf(outf, "Valid --update options are:\n"
 		"     'sparc2.2', 'super-minor', 'uuid', 'name', 'nodes', 'resync',\n"
 		"     'summaries', 'homehost', 'home-cluster', 'byteorder', 'devicesize',\n"
-		"     'no-bitmap', 'metadata', 'revert-reshape'\n"
+		"     'bitmap', 'no-bitmap', 'metadata', 'revert-reshape'\n"
 		"     'bbl', 'no-bbl', 'force-no-bbl', 'ppl', 'no-ppl'\n"
 		"     'layout-original', 'layout-alternate', 'layout-unspecified'\n"
 				);
-- 
2.26.2

[PATCH 6/8] imsm: Update-subarray for write-intent bitmap

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

The patch updates the current bitmap functionality to handle adding
the bitmap on existing volumes.

Change-Id: I75c4a2a450858204457f45b4daeecfd807c83834
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
---
 super-intel.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index eb5d39b9..0f3bbdf2 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -7783,6 +7783,19 @@ static int kill_subarray_imsm(struct supertype *st, char *subarray_id)
 	return 0;
 }
 
+static int get_rwh_policy_from_update(char *update)
+{
+	if (strcmp(update, "ppl") == 0)
+		return RWH_MULTIPLE_DISTRIBUTED;
+	else if (strcmp(update, "no-ppl") == 0)
+		return RWH_MULTIPLE_OFF;
+	else if (strcmp(update, "bitmap") == 0)
+		return RWH_BITMAP;
+	else if (strcmp(update, "no-bitmap") == 0)
+		return RWH_OFF;
+	return -1;
+}
+
 static int update_subarray_imsm(struct supertype *st, char *subarray,
 				char *update, struct mddev_ident *ident)
 {
@@ -7829,8 +7842,7 @@ static int update_subarray_imsm(struct supertype *st, char *subarray,
 			}
 			super->updates_pending++;
 		}
-	} else if (strcmp(update, "ppl") == 0 ||
-		   strcmp(update, "no-ppl") == 0) {
+	} else if (get_rwh_policy_from_update(update) != -1) {
 		int new_policy;
 		char *ep;
 		int vol = strtoul(subarray, &ep, 10);
@@ -7838,10 +7850,7 @@ static int update_subarray_imsm(struct supertype *st, char *subarray,
 		if (*ep != '\0' || vol >= super->anchor->num_raid_devs)
 			return 2;
 
-		if (strcmp(update, "ppl") == 0)
-			new_policy = RWH_MULTIPLE_DISTRIBUTED;
-		else
-			new_policy = RWH_MULTIPLE_OFF;
+		new_policy = get_rwh_policy_from_update(update);
 
 		if (st->update_tail) {
 			struct imsm_update_rwh_policy *u = xmalloc(sizeof(*u));
@@ -7857,6 +7866,8 @@ static int update_subarray_imsm(struct supertype *st, char *subarray,
 			dev->rwh_policy = new_policy;
 			super->updates_pending++;
 		}
+		if (new_policy == RWH_BITMAP)
+			return write_init_bitmap_imsm_vol(st, vol);
 	} else
 		return 2;
 
-- 
2.26.2

[PATCH 7/8] Create: Block automatic enabling bitmap for external metadata

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

For external metadata, bitmap should be added only when
explicitly set by the administrator.
They could be additional requirements to consider before
enabling the external metadata's functionality
(e.g., kernel support).

Change-Id: I3973464b4c73f77b9c2ebc9cbb5f501a14d5eb02
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
---
 Create.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Create.c b/Create.c
index 751f18c1..7df3b7e2 100644
--- a/Create.c
+++ b/Create.c
@@ -540,6 +540,7 @@ int Create(struct supertype *st, char *mddev,
 	}
 
 	if (!s->bitmap_file &&
+	    !st->ss->external &&
 	    s->level >= 1 &&
 	    st->ss->add_internal_bitmap &&
 	    s->journaldisks == 0 &&
-- 
2.26.2

[PATCH 8/8] Grow: Block reshape when external metadata and write-intent bitmap

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

Current kernel sysfs interface for the bitmap is limited. It allows
the applying of the bitmap on non-active volumes only.
The reshape operation for a volume with a bitmap should be blocked.

Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Change-Id: I2b4f57d6934815cd63a9cf506328b89fa4aa3930
---
 Grow.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/Grow.c b/Grow.c
index 6b8321c5..a48789ce 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1838,15 +1838,14 @@ int Grow_reshape(char *devname, int fd,
 		pr_err("Cannot increase raid-disks on this array beyond %d\n", st->max_devs);
 		return 1;
 	}
-	if (s->level == 0 &&
-	    (array.state & (1<<MD_SB_BITMAP_PRESENT)) &&
-	    !(array.state & (1<<MD_SB_CLUSTERED))) {
-                array.state &= ~(1<<MD_SB_BITMAP_PRESENT);
-                if (md_set_array_info(fd, &array)!= 0) {
-                        pr_err("failed to remove internal bitmap.\n");
-                        return 1;
-                }
-        }
+	if (s->level == 0 && (array.state & (1 << MD_SB_BITMAP_PRESENT)) &&
+		!(array.state & (1 << MD_SB_CLUSTERED)) && !st->ss->external) {
+		array.state &= ~(1 << MD_SB_BITMAP_PRESENT);
+		if (md_set_array_info(fd, &array) != 0) {
+			pr_err("failed to remove internal bitmap.\n");
+			return 1;
+		}
+	}
 
 	/* in the external case we need to check that the requested reshape is
 	 * supported, and perform an initial check that the container holds the
@@ -1910,6 +1909,13 @@ int Grow_reshape(char *devname, int fd,
 					free(subarray);
 					return 1;
 				}
+				if (content->consistency_policy ==
+				    CONSISTENCY_POLICY_BITMAP) {
+					pr_err("Operation not supported when write-intent bitmap is enabled\n");
+					sysfs_free(cc);
+					free(subarray);
+					return 1;
+				}
 			}
 			sysfs_free(cc);
 		}
-- 
2.26.2

[PATCH 0/8] Write-intent bitmap support for IMSM metadata

[Jakub Radtke]

This patchset is aimed to add write-intent bitmap support for IMSM
metadata.

Additional function in the superswitch (set_bitmap) is proposed as
a place where the internal bitmaps configuration for non-native
metadata can be performed (through sysfs).

Adding the bitmap to existing volumes is implemented similarly like
for the PPL functionality (through --update-subarray).

The patchset was tested on HW and qemu.

Jakub Radtke (8):
  Modify mdstat parsing for volumes with the bitmap
  Enable bitmap support for external metadata
  imsm: Write-intent bitmap support
  imsm: Adding a spare to an existing array with bitmap
  Add "bitmap" to allowed command-line values
  imsm: Update-subarray for write-intent bitmap
  Create: Block automatic enabling bitmap for external metadata
  Grow: Block reshape when external metadata and write-intent bitmap

 Assemble.c    |   6 +
 Create.c      |  11 +
 Grow.c        |  24 +-
 mdadm.8.in    |  13 +-
 mdadm.c       |   4 +-
 mdadm.h       |   3 +
 mdstat.c      |   6 +
 super-intel.c | 679 +++++++++++++++++++++++++++++++++++++++++++++++++-
 8 files changed, 725 insertions(+), 21 deletions(-)

-- 
2.26.2

[PATCH 1/8] Modify mdstat parsing for volumes with the bitmap

[Jakub Radtke]

From: Jakub Radtke <jakub.radtke@intel.com>

Current mdstat read functionality is not working correctly
for the volumes with the write-intent bitmap.
It affects rebuild and reshape use cases.

Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Change-Id: I5288fbfb7d6a85b2aa8452675a6e3559bf4fe934
---
 mdstat.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/mdstat.c b/mdstat.c
index dd96cca7..2fd792c5 100644
--- a/mdstat.c
+++ b/mdstat.c
@@ -191,6 +191,12 @@ struct mdstat_ent *mdstat_read(int hold, int start)
 			else if (strcmp(w, "inactive") == 0) {
 				ent->active = 0;
 				in_devs = 1;
+			} else if (strcmp(w, "bitmap:") == 0) {
+				/* We need to stop parsing here;
+				 * otherwise, ent->raid_disks will be
+				 * overwritten by the wrong value.
+				 */
+				break;
 			} else if (ent->active > 0 &&
 				 ent->level == NULL &&
 				 w[0] != '(' /*readonly*/) {
-- 
2.26.2