All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal
@ 2017-03-15 18:28 Song Liu
  2017-03-15 18:28 ` [PATCH 2/2] mdadm: remove journal with "remove-journal" Song Liu
  2017-03-17 19:38 ` [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal jes.sorensen
  0 siblings, 2 replies; 10+ messages in thread
From: Song Liu @ 2017-03-15 18:28 UTC (permalink / raw)
  To: linux-raid
  Cc: shli, neilb, kernel-team, dan.j.williams, hch, jes.sorensen, Song Liu

We use to apply constraints to --add-journal:
  1. We only support recreate journal for arrays with missing journal;
  2. The array need to be readonly when the journal is added.

As the kernel code getting more mature, these constraints are no longer
necessary. This patch removes this constraints from mdadm.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Manage.c   | 29 ++++-------------------------
 md_p.h     |  5 +++++
 mdadm.8.in |  7 ++-----
 3 files changed, 11 insertions(+), 30 deletions(-)

diff --git a/Manage.c b/Manage.c
index 5c3d2b9..289b7ce 100644
--- a/Manage.c
+++ b/Manage.c
@@ -946,28 +946,13 @@ int Manage_add(int fd, int tfd, struct mddev_dev *dv,
 
 	/* only add journal to array that supports journaling */
 	if (dv->disposition == 'j') {
-		struct mdinfo mdi;
-		struct mdinfo *mdp;
-
-		mdp = sysfs_read(fd, NULL, GET_ARRAY_STATE);
-		if (!mdp) {
-			pr_err("%s unable to read array state.\n", devname);
-			return -1;
-		}
-
-		if (strncmp(mdp->sysfs_array_state, "readonly", 8) != 0) {
-			sysfs_free(mdp);
-			pr_err("%s is not readonly, cannot add journal.\n", devname);
+		if ((array->state & (1<<MD_SB_HAS_JOURNAL)) &&
+		    !(array->state & (1<<MD_SB_JOURNAL_REMOVABLE))) {
+			pr_err("%s has a working journal. Hot spare journal device is not supported.\n",
+			       devname);
 			return -1;
 		}
 
-		sysfs_free(mdp);
-
-		tst->ss->getinfo_super(tst, &mdi, NULL);
-		if (mdi.journal_device_required == 0) {
-			pr_err("%s does not support journal device.\n", devname);
-			return -1;
-		}
 		disc.raid_disk = 0;
 	}
 
@@ -1097,12 +1082,6 @@ int Manage_add(int fd, int tfd, struct mddev_dev *dv,
 				       dv->devname, j, strerror(errno));
 			return -1;
 		}
-		if (dv->disposition == 'j') {
-			pr_err("Journal added successfully, making %s read-write\n", devname);
-			if (Manage_ro(devname, fd, -1))
-				pr_err("Failed to make %s read-write\n", devname);
-		}
-
 	}
 	if (verbose >= 0)
 		pr_err("added %s\n", dv->devname);
diff --git a/md_p.h b/md_p.h
index dc9fec1..7081bd9 100644
--- a/md_p.h
+++ b/md_p.h
@@ -121,6 +121,11 @@ typedef struct mdp_device_descriptor_s {
 				   * in container can be activated */
 #define MD_SB_CLUSTERED		5 /* MD is clustered  */
 #define	MD_SB_BITMAP_PRESENT	8 /* bitmap may be present nearby */
+#define	MD_SB_HAS_JOURNAL	9
+#define	MD_SB_JOURNAL_REMOVABLE	10 /* journal _feature_ can be removed,
+				    * which means the journal is either
+				    * missing or Faulty
+				    */
 
 typedef struct mdp_superblock_s {
 	/*
diff --git a/mdadm.8.in b/mdadm.8.in
index df1d460..087f679 100644
--- a/mdadm.8.in
+++ b/mdadm.8.in
@@ -1474,11 +1474,8 @@ the device is found or <slot>:missing in case the device is not found.
 
 .TP
 .BR \-\-add-journal
-Recreate journal for RAID-4/5/6 array that lost a journal device. In the
-current implementation, this command cannot add a journal to an array
-that had a failed journal. To avoid interrupting on-going write opertions,
-.B \-\-add-journal
-only works for array in Read-Only state.
+Add journal to an existing array, or recreate journal for RAID-4/5/6 array
+that lost a journal device.
 
 .TP
 .BR \-\-failfast
-- 
2.9.3


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH 2/2] mdadm: remove journal with "remove-journal"
  2017-03-15 18:28 [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal Song Liu
@ 2017-03-15 18:28 ` Song Liu
  2017-03-17 19:39   ` jes.sorensen
  2017-03-17 19:38 ` [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal jes.sorensen
  1 sibling, 1 reply; 10+ messages in thread
From: Song Liu @ 2017-03-15 18:28 UTC (permalink / raw)
  To: linux-raid
  Cc: shli, neilb, kernel-team, dan.j.williams, hch, jes.sorensen, Song Liu

When journal device of an array fails, the array is forced into read-only
mode. To make the array normal without adding another journal device, we
need to remove journal _feature_ from the array.

This patch is the mdadm part of removing journal feature for an array with
failed or missing journal.

Two flags are added to GET_ARRAY_INFO:
  1. MD_SB_HAS_JOURNAL: meaning the array have journal feature;
  2. MD_SB_JOURNAL_REMOVABLE: meaning the journal is faulty or missing

When both flags are set, mdadm will use ioctl SET_ARRAY_INFO to clear
MD_SB_HAS_JOURNAL, and thus remove journal feature from the array.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 Manage.c | 26 ++++++++++++++++++++++++++
 ReadMe.c |  1 +
 mdadm.c  |  9 ++++++++-
 mdadm.h  |  3 +++
 4 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/Manage.c b/Manage.c
index 289b7ce..26c7dca 100644
--- a/Manage.c
+++ b/Manage.c
@@ -189,6 +189,32 @@ int Manage_run(char *devname, int fd, struct context *c)
 	return IncrementalScan(c, nm);
 }
 
+int Manage_remove_journal(char *devname, int fd, int verbose)
+{
+	mdu_array_info_t array;
+
+	if (ioctl(fd, GET_ARRAY_INFO, &array) != 0)
+		return 1;
+
+	if (!(array.state & (1<<MD_SB_HAS_JOURNAL))) {
+		pr_err("%s does not have journal. No need to remove.\n",
+		       devname);
+		return 1;
+	} else if (!(array.state & (1<<MD_SB_JOURNAL_REMOVABLE))) {
+		pr_err("Cannot remove active journal. Please fail it and try again.\n");
+		return 1;
+	}
+
+	array.state &= ~(1<<MD_SB_HAS_JOURNAL);
+
+	if (ioctl(fd, SET_ARRAY_INFO, &array)) {
+		pr_err("Failed to add remove journal feature.\n");
+		return 1;
+	}
+
+	return 0;
+}
+
 int Manage_stop(char *devname, int fd, int verbose, int will_retry)
 {
 	/* Stop the array.  Array must already be configured
diff --git a/ReadMe.c b/ReadMe.c
index 50d3807..acc4180 100644
--- a/ReadMe.c
+++ b/ReadMe.c
@@ -164,6 +164,7 @@ struct option long_options[] = {
     {"add-spare", 0, 0, AddSpare},
     {"add-journal", 0, 0, AddJournal},
     {"remove",    0, 0, Remove},
+    {"remove-journal",    0, 0, RemoveJournal},
     {"fail",      0, 0, Fail},
     {"set-faulty",0, 0, Fail},
     {"replace",   0, 0, Replace},
diff --git a/mdadm.c b/mdadm.c
index d6ad8dc..3ad413f 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -194,6 +194,7 @@ int main(int argc, char *argv[])
 		case AddJournal:
 		case 'r':
 		case Remove:
+		case RemoveJournal:
 		case Replace:
 		case With:
 		case 'f':
@@ -994,6 +995,9 @@ int main(int argc, char *argv[])
 			}
 			c.runstop = -1;
 			continue;
+		case O(MANAGE,RemoveJournal):
+			c.remove_journal = 1;
+			continue;
 		case O(MANAGE,'t'):
 			c.test = 1;
 			continue;
@@ -1387,6 +1391,9 @@ int main(int argc, char *argv[])
 			rv = Manage_run(devlist->devname, mdfd, &c);
 		if (!rv && c.runstop < 0)
 			rv = Manage_stop(devlist->devname, mdfd, c.verbose, 0);
+		if (!rv && c.remove_journal > 0)
+			rv = Manage_remove_journal(devlist->devname, mdfd,
+						   c.verbose);
 		break;
 	case ASSEMBLE:
 		if (devs_found == 1 && ident.uuid_set == 0 &&
@@ -1906,7 +1913,7 @@ static int misc_list(struct mddev_dev *devlist,
 				mdfd = open_dev(dv->devname);
 				if (mdfd >= 0) break;
 			case 1:
-				mdfd = open_mddev(dv->devname, 1);  
+				mdfd = open_mddev(dv->devname, 1);
 		}
 		if (mdfd>=0) {
 			switch(dv->disposition) {
diff --git a/mdadm.h b/mdadm.h
index 71b8afb..e5acc49 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -390,6 +390,7 @@ enum special_options {
 	AddSpare,
 	AddJournal,
 	Remove,
+	RemoveJournal,
 	Fail,
 	Replace,
 	With,
@@ -512,6 +513,7 @@ struct context {
 	char	*action;
 	int	nodes;
 	char	*homecluster;
+	int	remove_journal;
 };
 
 struct shape {
@@ -1293,6 +1295,7 @@ extern int Manage_ro(char *devname, int fd, int readonly);
 extern int Manage_run(char *devname, int fd, struct context *c);
 extern int Manage_stop(char *devname, int fd, int quiet,
 		       int will_retry);
+extern int Manage_remove_journal(char *devname, int fd, int verbose);
 extern int Manage_subdevs(char *devname, int fd,
 			  struct mddev_dev *devlist, int verbose, int test,
 			  char *update, int force);
-- 
2.9.3


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal
  2017-03-15 18:28 [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal Song Liu
  2017-03-15 18:28 ` [PATCH 2/2] mdadm: remove journal with "remove-journal" Song Liu
@ 2017-03-17 19:38 ` jes.sorensen
  1 sibling, 0 replies; 10+ messages in thread
From: jes.sorensen @ 2017-03-17 19:38 UTC (permalink / raw)
  To: Song Liu; +Cc: linux-raid, neilb, kernel-team

Song Liu <songliubraving@fb.com> writes:
> We use to apply constraints to --add-journal:
>   1. We only support recreate journal for arrays with missing journal;
>   2. The array need to be readonly when the journal is added.
>
> As the kernel code getting more mature, these constraints are no longer
> necessary. This patch removes this constraints from mdadm.
>
> Signed-off-by: Song Liu <songliubraving@fb.com>

Applied!

However, for multi-patch series, please always include a cover letter.
Just like with the kernel. Easier to repond to that than each patch
individually.

Thanks,
Jes

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 2/2] mdadm: remove journal with "remove-journal"
  2017-03-15 18:28 ` [PATCH 2/2] mdadm: remove journal with "remove-journal" Song Liu
@ 2017-03-17 19:39   ` jes.sorensen
  0 siblings, 0 replies; 10+ messages in thread
From: jes.sorensen @ 2017-03-17 19:39 UTC (permalink / raw)
  To: Song Liu; +Cc: linux-raid, shli, neilb, kernel-team, dan.j.williams, hch

Song Liu <songliubraving@fb.com> writes:
> When journal device of an array fails, the array is forced into read-only
> mode. To make the array normal without adding another journal device, we
> need to remove journal _feature_ from the array.
>
> This patch is the mdadm part of removing journal feature for an array with
> failed or missing journal.
>
> Two flags are added to GET_ARRAY_INFO:
>   1. MD_SB_HAS_JOURNAL: meaning the array have journal feature;
>   2. MD_SB_JOURNAL_REMOVABLE: meaning the journal is faulty or missing
>
> When both flags are set, mdadm will use ioctl SET_ARRAY_INFO to clear
> MD_SB_HAS_JOURNAL, and thus remove journal feature from the array.
>
> Signed-off-by: Song Liu <songliubraving@fb.com>
> ---
>  Manage.c | 26 ++++++++++++++++++++++++++
>  ReadMe.c |  1 +
>  mdadm.c  |  9 ++++++++-
>  mdadm.h  |  3 +++
>  4 files changed, 38 insertions(+), 1 deletion(-)

Applied!

Thanks,
Jes

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 2/2] mdadm: remove journal with "remove-journal"
  2017-11-16 17:55     ` Song Liu
@ 2017-11-20 16:04       ` Larkin Lowrey
  0 siblings, 0 replies; 10+ messages in thread
From: Larkin Lowrey @ 2017-11-20 16:04 UTC (permalink / raw)
  To: Song Liu; +Cc: linux-raid, shli, jes.sorensen

On 11/16/2017 12:55 PM, Song Liu wrote:
>> On Nov 16, 2017, at 9:35 AM, Larkin Lowrey <llowrey@gmail.com> wrote:
>>
>> # mdadm /dev/md124 --fail /dev/md127p4
>> mdadm: set /dev/md127p4 faulty in /dev/md124
>>
>> # mdadm /dev/md124 --remove /dev/md127p4
>> mdadm: hot removed /dev/md127p4 from /dev/md124
>>
>> # cat /sys/block/md124/md/consistency_policy
>> journal
>>
>> # echo resync > /sys/block/md124/md/consistency_policy
>>
>> # cat /sys/block/md124/md/consistency_policy
>> resync
> Could you please try the following command here:
>
> echo active > /sys/block/md124/md/array_state
>
>
>> # mdadm --stop /dev/md124
>> mdadm: stopped /dev/md124
>>
>> # mdadm --assemble /dev/md124 /dev/md12[23] /dev/sd[goqpjfl]1
>> mdadm: Not safe to assemble with missing or stale journal device, consider --force.

This worked, thanks!

--Larkin

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 2/2] mdadm: remove journal with "remove-journal"
  2017-11-16 17:35   ` Larkin Lowrey
@ 2017-11-16 17:55     ` Song Liu
  2017-11-20 16:04       ` Larkin Lowrey
  0 siblings, 1 reply; 10+ messages in thread
From: Song Liu @ 2017-11-16 17:55 UTC (permalink / raw)
  To: Larkin Lowrey; +Cc: linux-raid, shli, jes.sorensen


> On Nov 16, 2017, at 9:35 AM, Larkin Lowrey <llowrey@gmail.com> wrote:
> 
> Is it not possible to permanently remove a journal and return to resync mode?
> 
> Here's what I tried:
> 
> #  mdadm --readonly /dev/md124
> 
> # cat /proc/mdstat
> Personalities : [raid10] [raid0] [raid6] [raid5] [raid4]
> md124 : active (read-only) raid5 md123[7] md127p4[15](J) sdp1[13] sdl1[10] sdg1[12] sdf1[11] sdo1[14] sdj1[8] sdq1[9] md122[6]
>       62509129728 blocks super 1.2 level 5, 512k chunk, algorithm 2 [9/9] [UUUUUUUUU]
> 
> # mdadm /dev/md124 --fail /dev/md127p4
> mdadm: set /dev/md127p4 faulty in /dev/md124
> 
> # mdadm /dev/md124 --remove /dev/md127p4
> mdadm: hot removed /dev/md127p4 from /dev/md124
> 
> # cat /sys/block/md124/md/consistency_policy
> journal
> 
> # echo resync > /sys/block/md124/md/consistency_policy
> 
> # cat /sys/block/md124/md/consistency_policy
> resync

Could you please try the following command here:

echo active > /sys/block/md124/md/array_state


> # mdadm --stop /dev/md124
> mdadm: stopped /dev/md124
> 
> # mdadm --assemble /dev/md124 /dev/md12[23] /dev/sd[goqpjfl]1
> mdadm: Not safe to assemble with missing or stale journal device, consider --force.
> 



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 2/2] mdadm: remove journal with "remove-journal"
  2017-08-31 17:51 ` Song Liu
  2017-09-01 14:56   ` Larkin Lowrey
@ 2017-11-16 17:35   ` Larkin Lowrey
  2017-11-16 17:55     ` Song Liu
  1 sibling, 1 reply; 10+ messages in thread
From: Larkin Lowrey @ 2017-11-16 17:35 UTC (permalink / raw)
  To: Song Liu, linux-raid; +Cc: shli, jes.sorensen

Is it not possible to permanently remove a journal and return to resync 
mode?

Here's what I tried:

#  mdadm --readonly /dev/md124

# cat /proc/mdstat
Personalities : [raid10] [raid0] [raid6] [raid5] [raid4]
md124 : active (read-only) raid5 md123[7] md127p4[15](J) sdp1[13] 
sdl1[10] sdg1[12] sdf1[11] sdo1[14] sdj1[8] sdq1[9] md122[6]
       62509129728 blocks super 1.2 level 5, 512k chunk, algorithm 2 
[9/9] [UUUUUUUUU]

# mdadm /dev/md124 --fail /dev/md127p4
mdadm: set /dev/md127p4 faulty in /dev/md124

# mdadm /dev/md124 --remove /dev/md127p4
mdadm: hot removed /dev/md127p4 from /dev/md124

# cat /sys/block/md124/md/consistency_policy
journal

# echo resync > /sys/block/md124/md/consistency_policy

# cat /sys/block/md124/md/consistency_policy
resync

# mdadm --stop /dev/md124
mdadm: stopped /dev/md124

# mdadm --assemble /dev/md124 /dev/md12[23] /dev/sd[goqpjfl]1
mdadm: Not safe to assemble with missing or stale journal device, 
consider --force.

--Larkin

On 8/31/2017 1:51 PM, Song Liu wrote:
> Hi Larkin,
>
> We didn't ship this patch. Instead, we decide to use sysfs entry
>
> /sys/block/mdXXX/md/consistency_policy
>
> You can find the patch at:
>
> https://marc.info/?l=linux-raid&m=149063896208043
>
> Thanks,
> Song
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 2/2] mdadm: remove journal with "remove-journal"
  2017-08-31 17:51 ` Song Liu
@ 2017-09-01 14:56   ` Larkin Lowrey
  2017-11-16 17:35   ` Larkin Lowrey
  1 sibling, 0 replies; 10+ messages in thread
From: Larkin Lowrey @ 2017-09-01 14:56 UTC (permalink / raw)
  To: Song Liu, linux-raid; +Cc: shli, jes.sorensen

Thanks!

--Larkin

On 8/31/2017 1:51 PM, Song Liu wrote:
> Hi Larkin,
>
> We didn't ship this patch. Instead, we decide to use sysfs entry
>
> /sys/block/mdXXX/md/consistency_policy
>
> You can find the patch at:
>
> https://marc.info/?l=linux-raid&m=149063896208043
>
> Thanks,
> Song
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 2/2] mdadm: remove journal with "remove-journal"
  2017-08-30 15:54 [PATCH 2/2] mdadm: remove journal with "remove-journal" Larkin Lowrey
@ 2017-08-31 17:51 ` Song Liu
  2017-09-01 14:56   ` Larkin Lowrey
  2017-11-16 17:35   ` Larkin Lowrey
  0 siblings, 2 replies; 10+ messages in thread
From: Song Liu @ 2017-08-31 17:51 UTC (permalink / raw)
  To: linux-raid; +Cc: shli, jes.sorensen, llowrey, Song Liu

Hi Larkin,

We didn't ship this patch. Instead, we decide to use sysfs entry

/sys/block/mdXXX/md/consistency_policy

You can find the patch at:

https://marc.info/?l=linux-raid&m=149063896208043

Thanks,
Song

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 2/2] mdadm: remove journal with "remove-journal"
@ 2017-08-30 15:54 Larkin Lowrey
  2017-08-31 17:51 ` Song Liu
  0 siblings, 1 reply; 10+ messages in thread
From: Larkin Lowrey @ 2017-08-30 15:54 UTC (permalink / raw)
  To: linux-raid

I don't see any evidence that either of these two patches made it to 
git. Is this the correct repo?

git.kernel.org/pub/scm/utils/mdadm/mdadm.git

--Larkin

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2017-11-20 16:04 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-15 18:28 [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal Song Liu
2017-03-15 18:28 ` [PATCH 2/2] mdadm: remove journal with "remove-journal" Song Liu
2017-03-17 19:39   ` jes.sorensen
2017-03-17 19:38 ` [PATCH 1/2] mdadm/r5cache: allow adding journal to array without journal jes.sorensen
2017-08-30 15:54 [PATCH 2/2] mdadm: remove journal with "remove-journal" Larkin Lowrey
2017-08-31 17:51 ` Song Liu
2017-09-01 14:56   ` Larkin Lowrey
2017-11-16 17:35   ` Larkin Lowrey
2017-11-16 17:55     ` Song Liu
2017-11-20 16:04       ` Larkin Lowrey

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.