[PATCH] md: adding a new flag MD_DELETING

[PATCH] md: adding a new flag MD_DELETING

[Lidong Zhong]

The mddev data structure is freed in mddev_delayed_delete(), which is
schedualed after the array is deconfigured completely when stopping. So
there is a race window between md_open() and do_md_stop(), which leads
to /dev/mdX can still be opened by userspace even it's not accessible
any more. As a result, a DeviceDisappeared event will not be able to be
monitored by mdadm in monitor mode. This patch tries to fix it by adding
this new flag MD_DELETING.

Signed-off-by: Lidong Zhong <lidong.zhong@suse.com>
---
 drivers/md/md.c | 4 +++-
 drivers/md/md.h | 2 ++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index 21da0c48f6c2..566df2491318 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -6439,6 +6439,7 @@ static int do_md_stop(struct mddev *mddev, int mode,
 		md_clean(mddev);
 		if (mddev->hold_active == UNTIL_STOP)
 			mddev->hold_active = 0;
+		set_bit(MD_DELETING, &mddev->flags);
 	}
 	md_new_event(mddev);
 	sysfs_notify_dirent_safe(mddev->sysfs_state);
@@ -7829,7 +7830,8 @@ static int md_open(struct block_device *bdev, fmode_t mode)
 	if ((err = mutex_lock_interruptible(&mddev->open_mutex)))
 		goto out;
 
-	if (test_bit(MD_CLOSING, &mddev->flags)) {
+	if (test_bit(MD_CLOSING, &mddev->flags) ||
+            (test_bit(MD_DELETING, &mddev->flags) && mddev->pers == NULL)) {
 		mutex_unlock(&mddev->open_mutex);
 		err = -ENODEV;
 		goto out;
diff --git a/drivers/md/md.h b/drivers/md/md.h
index bcbba1b5ec4a..83c7aa61699f 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -262,6 +262,8 @@ enum mddev_flags {
 	MD_BROKEN,              /* This is used in RAID-0/LINEAR only, to stop
 				 * I/O in case an array member is gone/failed.
 				 */
+	MD_DELETING,		/* If set, we are deleting the array, do not open
+				 * it then */
 };
 
 enum mddev_sb_flags {
-- 
2.26.2

Re: [PATCH] md: adding a new flag MD_DELETING

[Zhong Lidong]

Hi Song,

Could you share your opinion about this patch please?

Thanks,
Lidong

On 4/28/21 4:29 PM, Lidong Zhong wrote:
> The mddev data structure is freed in mddev_delayed_delete(), which is
> schedualed after the array is deconfigured completely when stopping. So
> there is a race window between md_open() and do_md_stop(), which leads
> to /dev/mdX can still be opened by userspace even it's not accessible
> any more. As a result, a DeviceDisappeared event will not be able to be
> monitored by mdadm in monitor mode. This patch tries to fix it by adding
> this new flag MD_DELETING.
> 
> Signed-off-by: Lidong Zhong <lidong.zhong@suse.com>
> ---
>  drivers/md/md.c | 4 +++-
>  drivers/md/md.h | 2 ++
>  2 files changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 21da0c48f6c2..566df2491318 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -6439,6 +6439,7 @@ static int do_md_stop(struct mddev *mddev, int mode,
>  		md_clean(mddev);
>  		if (mddev->hold_active == UNTIL_STOP)
>  			mddev->hold_active = 0;
> +		set_bit(MD_DELETING, &mddev->flags);
>  	}
>  	md_new_event(mddev);
>  	sysfs_notify_dirent_safe(mddev->sysfs_state);
> @@ -7829,7 +7830,8 @@ static int md_open(struct block_device *bdev, fmode_t mode)
>  	if ((err = mutex_lock_interruptible(&mddev->open_mutex)))
>  		goto out;
>  
> -	if (test_bit(MD_CLOSING, &mddev->flags)) {
> +	if (test_bit(MD_CLOSING, &mddev->flags) ||
> +            (test_bit(MD_DELETING, &mddev->flags) && mddev->pers == NULL)) {
>  		mutex_unlock(&mddev->open_mutex);
>  		err = -ENODEV;
>  		goto out;
> diff --git a/drivers/md/md.h b/drivers/md/md.h
> index bcbba1b5ec4a..83c7aa61699f 100644
> --- a/drivers/md/md.h
> +++ b/drivers/md/md.h
> @@ -262,6 +262,8 @@ enum mddev_flags {
>  	MD_BROKEN,              /* This is used in RAID-0/LINEAR only, to stop
>  				 * I/O in case an array member is gone/failed.
>  				 */
> +	MD_DELETING,		/* If set, we are deleting the array, do not open
> +				 * it then */
>  };
>  
>  enum mddev_sb_flags {
> 

Re: [PATCH] md: adding a new flag MD_DELETING

[Song Liu]

Hi Lidong,

On Sat, May 8, 2021 at 12:41 AM Zhong Lidong <lidong.zhong@suse.com> wrote:
>
> Hi Song,
>
> Could you share your opinion about this patch please?
>

The patch looks good to me. I will process it (run some tests etc)
later this week.

Thanks,
Song


>
> On 4/28/21 4:29 PM, Lidong Zhong wrote:
> > The mddev data structure is freed in mddev_delayed_delete(), which is
> > schedualed after the array is deconfigured completely when stopping. So
> > there is a race window between md_open() and do_md_stop(), which leads
> > to /dev/mdX can still be opened by userspace even it's not accessible
> > any more. As a result, a DeviceDisappeared event will not be able to be
> > monitored by mdadm in monitor mode. This patch tries to fix it by adding
> > this new flag MD_DELETING.
> >
> > Signed-off-by: Lidong Zhong <lidong.zhong@suse.com>
> > ---
> >  drivers/md/md.c | 4 +++-
> >  drivers/md/md.h | 2 ++
> >  2 files changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/md/md.c b/drivers/md/md.c
> > index 21da0c48f6c2..566df2491318 100644
> > --- a/drivers/md/md.c
> > +++ b/drivers/md/md.c
> > @@ -6439,6 +6439,7 @@ static int do_md_stop(struct mddev *mddev, int mode,
> >               md_clean(mddev);
> >               if (mddev->hold_active == UNTIL_STOP)
> >                       mddev->hold_active = 0;
> > +             set_bit(MD_DELETING, &mddev->flags);
> >       }
> >       md_new_event(mddev);
> >       sysfs_notify_dirent_safe(mddev->sysfs_state);
> > @@ -7829,7 +7830,8 @@ static int md_open(struct block_device *bdev, fmode_t mode)
> >       if ((err = mutex_lock_interruptible(&mddev->open_mutex)))
> >               goto out;
> >
> > -     if (test_bit(MD_CLOSING, &mddev->flags)) {
> > +     if (test_bit(MD_CLOSING, &mddev->flags) ||
> > +            (test_bit(MD_DELETING, &mddev->flags) && mddev->pers == NULL)) {
> >               mutex_unlock(&mddev->open_mutex);
> >               err = -ENODEV;
> >               goto out;
> > diff --git a/drivers/md/md.h b/drivers/md/md.h
> > index bcbba1b5ec4a..83c7aa61699f 100644
> > --- a/drivers/md/md.h
> > +++ b/drivers/md/md.h
> > @@ -262,6 +262,8 @@ enum mddev_flags {
> >       MD_BROKEN,              /* This is used in RAID-0/LINEAR only, to stop
> >                                * I/O in case an array member is gone/failed.
> >                                */
> > +     MD_DELETING,            /* If set, we are deleting the array, do not open
> > +                              * it then */
> >  };
> >
> >  enum mddev_sb_flags {
> >
>

Re: [PATCH] md: adding a new flag MD_DELETING

[Song Liu]

On Wed, Apr 28, 2021 at 1:31 AM Lidong Zhong <lidong.zhong@suse.com> wrote:
>
> The mddev data structure is freed in mddev_delayed_delete(), which is
> schedualed after the array is deconfigured completely when stopping. So
> there is a race window between md_open() and do_md_stop(), which leads
> to /dev/mdX can still be opened by userspace even it's not accessible
> any more. As a result, a DeviceDisappeared event will not be able to be
> monitored by mdadm in monitor mode. This patch tries to fix it by adding
> this new flag MD_DELETING.
>
> Signed-off-by: Lidong Zhong <lidong.zhong@suse.com>

Sorry for the delay. I missed this one.

As I try to apply the patch, I found the patch is somehow corrupted. It contains
special patterns like:

 =09if ((err =3D mutex_lock_interruptible(&mddev->open_mutex)))
 =09=09goto out;
=20
-=09if (test_bit(MD_CLOSING, &mddev->flags)) {
+=09if (test_bit(MD_CLOSING, &mddev->flags) ||
+            (test_bit(MD_DELETING, &mddev->flags) && mddev->pers =3D=3D NU=
LL)) {

Could you please try resend it?

Thanks,
Song

Re: [PATCH] md: adding a new flag MD_DELETING

[Zhong Lidong]

On 6/4/21 9:23 AM, Song Liu wrote:
> On Wed, Apr 28, 2021 at 1:31 AM Lidong Zhong <lidong.zhong@suse.com> wrote:
>>
>> The mddev data structure is freed in mddev_delayed_delete(), which is
>> schedualed after the array is deconfigured completely when stopping. So
>> there is a race window between md_open() and do_md_stop(), which leads
>> to /dev/mdX can still be opened by userspace even it's not accessible
>> any more. As a result, a DeviceDisappeared event will not be able to be
>> monitored by mdadm in monitor mode. This patch tries to fix it by adding
>> this new flag MD_DELETING.
>>
>> Signed-off-by: Lidong Zhong <lidong.zhong@suse.com>
> 
> Sorry for the delay. I missed this one.
> 
> As I try to apply the patch, I found the patch is somehow corrupted. It contains
> special patterns like:
> 
>  =09if ((err =3D mutex_lock_interruptible(&mddev->open_mutex)))
>  =09=09goto out;
> =20
> -=09if (test_bit(MD_CLOSING, &mddev->flags)) {
> +=09if (test_bit(MD_CLOSING, &mddev->flags) ||
> +            (test_bit(MD_DELETING, &mddev->flags) && mddev->pers =3D=3D NU=
> LL)) {
> 
> Could you please try resend it?
> 
Hi Song,

I'll resend the patch once my email client gets back to work.

Thanks,
Lidong