* FAILED: patch "[PATCH] null_blk: Fix zone reset all tracing" failed to apply to 5.9-stable tree @ 2020-11-03 14:24 gregkh 2020-11-03 23:37 ` Damien Le Moal 2020-11-04 5:29 ` [PATCH] null_blk: Fix zone reset all tracing Damien Le Moal 0 siblings, 2 replies; 15+ messages in thread From: gregkh @ 2020-11-03 14:24 UTC (permalink / raw) To: damien.lemoal, axboe; +Cc: stable The patch below does not apply to the 5.9-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable@vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ From f9c9104288da543cd64f186f9e2fba389f415630 Mon Sep 17 00:00:00 2001 From: Damien Le Moal <damien.lemoal@wdc.com> Date: Thu, 29 Oct 2020 20:04:59 +0900 Subject: [PATCH] null_blk: Fix zone reset all tracing In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is ignored and the operation is applied to all sequential zones. For these commands, tracing the effect of the command using the command sector to determine the target zone is thus incorrect. Fix null_zone_mgmt() zone condition tracing in the case of REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are not already empty. Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Cc: stable@vger.kernel.org Signed-off-by: Jens Axboe <axboe@kernel.dk> diff --git a/drivers/block/null_blk_zoned.c b/drivers/block/null_blk_zoned.c index 98056c88926b..b637b16a5f54 100644 --- a/drivers/block/null_blk_zoned.c +++ b/drivers/block/null_blk_zoned.c @@ -475,9 +475,14 @@ static blk_status_t null_zone_mgmt(struct nullb_cmd *cmd, enum req_opf op, switch (op) { case REQ_OP_ZONE_RESET_ALL: - for (i = dev->zone_nr_conv; i < dev->nr_zones; i++) - null_reset_zone(dev, &dev->zones[i]); - break; + for (i = dev->zone_nr_conv; i < dev->nr_zones; i++) { + zone = &dev->zones[i]; + if (zone->cond != BLK_ZONE_COND_EMPTY) { + null_reset_zone(dev, zone); + trace_nullb_zone_op(cmd, i, zone->cond); + } + } + return BLK_STS_OK; case REQ_OP_ZONE_RESET: ret = null_reset_zone(dev, zone); break; ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: FAILED: patch "[PATCH] null_blk: Fix zone reset all tracing" failed to apply to 5.9-stable tree 2020-11-03 14:24 FAILED: patch "[PATCH] null_blk: Fix zone reset all tracing" failed to apply to 5.9-stable tree gregkh @ 2020-11-03 23:37 ` Damien Le Moal 2020-11-04 5:29 ` [PATCH] null_blk: Fix zone reset all tracing Damien Le Moal 1 sibling, 0 replies; 15+ messages in thread From: Damien Le Moal @ 2020-11-03 23:37 UTC (permalink / raw) To: gregkh, axboe; +Cc: stable On 2020/11/03 23:23, gregkh@linuxfoundation.org wrote: > > The patch below does not apply to the 5.9-stable tree. > If someone wants it applied there, or to any other stable or longterm > tree, then please email the backport, including the original git commit > id to <stable@vger.kernel.org>. I will send a backported patch shortly. Thanks. > > thanks, > > greg k-h > > ------------------ original commit in Linus's tree ------------------ > > From f9c9104288da543cd64f186f9e2fba389f415630 Mon Sep 17 00:00:00 2001 > From: Damien Le Moal <damien.lemoal@wdc.com> > Date: Thu, 29 Oct 2020 20:04:59 +0900 > Subject: [PATCH] null_blk: Fix zone reset all tracing > > In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is > ignored and the operation is applied to all sequential zones. For these > commands, tracing the effect of the command using the command sector to > determine the target zone is thus incorrect. > > Fix null_zone_mgmt() zone condition tracing in the case of > REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are > not already empty. > > Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") > Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> > Cc: stable@vger.kernel.org > Signed-off-by: Jens Axboe <axboe@kernel.dk> > > diff --git a/drivers/block/null_blk_zoned.c b/drivers/block/null_blk_zoned.c > index 98056c88926b..b637b16a5f54 100644 > --- a/drivers/block/null_blk_zoned.c > +++ b/drivers/block/null_blk_zoned.c > @@ -475,9 +475,14 @@ static blk_status_t null_zone_mgmt(struct nullb_cmd *cmd, enum req_opf op, > > switch (op) { > case REQ_OP_ZONE_RESET_ALL: > - for (i = dev->zone_nr_conv; i < dev->nr_zones; i++) > - null_reset_zone(dev, &dev->zones[i]); > - break; > + for (i = dev->zone_nr_conv; i < dev->nr_zones; i++) { > + zone = &dev->zones[i]; > + if (zone->cond != BLK_ZONE_COND_EMPTY) { > + null_reset_zone(dev, zone); > + trace_nullb_zone_op(cmd, i, zone->cond); > + } > + } > + return BLK_STS_OK; > case REQ_OP_ZONE_RESET: > ret = null_reset_zone(dev, zone); > break; > > -- Damien Le Moal Western Digital Research ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH] null_blk: Fix zone reset all tracing 2020-11-03 14:24 FAILED: patch "[PATCH] null_blk: Fix zone reset all tracing" failed to apply to 5.9-stable tree gregkh 2020-11-03 23:37 ` Damien Le Moal @ 2020-11-04 5:29 ` Damien Le Moal 2020-11-04 9:10 ` Greg Kroah-Hartman 1 sibling, 1 reply; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 5:29 UTC (permalink / raw) To: Greg Kroah-Hartman, stable; +Cc: Jens Axboe commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is ignored and the operation is applied to all sequential zones. For these commands, tracing the effect of the command using the command sector to determine the target zone is thus incorrect. Fix null_zone_mgmt() zone condition tracing in the case of REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are not already empty. Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Cc: stable@vger.kernel.org Signed-off-by: Jens Axboe <axboe@kernel.dk> --- drivers/block/null_blk_zoned.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/block/null_blk_zoned.c b/drivers/block/null_blk_zoned.c index 3d25c9ad2383..fcbbe2e4ef2d 100644 --- a/drivers/block/null_blk_zoned.c +++ b/drivers/block/null_blk_zoned.c @@ -226,13 +226,15 @@ static blk_status_t null_zone_mgmt(struct nullb_cmd *cmd, enum req_opf op, switch (op) { case REQ_OP_ZONE_RESET_ALL: - for (i = 0; i < dev->nr_zones; i++) { - if (zone[i].type == BLK_ZONE_TYPE_CONVENTIONAL) - continue; - zone[i].cond = BLK_ZONE_COND_EMPTY; - zone[i].wp = zone[i].start; + for (i = dev->zone_nr_conv; i < dev->nr_zones; i++) { + zone = &dev->zones[i]; + if (zone->cond != BLK_ZONE_COND_EMPTY) { + zone->cond = BLK_ZONE_COND_EMPTY; + zone->wp = zone->start; + trace_nullb_zone_op(cmd, i, zone->cond); + } } - break; + return BLK_STS_OK; case REQ_OP_ZONE_RESET: if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) return BLK_STS_IOERR; -- 2.26.2 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH] null_blk: Fix zone reset all tracing 2020-11-04 5:29 ` [PATCH] null_blk: Fix zone reset all tracing Damien Le Moal @ 2020-11-04 9:10 ` Greg Kroah-Hartman 2020-11-04 9:15 ` Greg Kroah-Hartman 0 siblings, 1 reply; 15+ messages in thread From: Greg Kroah-Hartman @ 2020-11-04 9:10 UTC (permalink / raw) To: Damien Le Moal; +Cc: stable, Jens Axboe On Wed, Nov 04, 2020 at 02:29:14PM +0900, Damien Le Moal wrote: > commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. > > In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is > ignored and the operation is applied to all sequential zones. For these > commands, tracing the effect of the command using the command sector to > determine the target zone is thus incorrect. > > Fix null_zone_mgmt() zone condition tracing in the case of > REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are > not already empty. > > Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") > Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> > Cc: stable@vger.kernel.org > Signed-off-by: Jens Axboe <axboe@kernel.dk> > --- > drivers/block/null_blk_zoned.c | 14 ++++++++------ > 1 file changed, 8 insertions(+), 6 deletions(-) Now queued up, thanks. greg k-h ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] null_blk: Fix zone reset all tracing 2020-11-04 9:10 ` Greg Kroah-Hartman @ 2020-11-04 9:15 ` Greg Kroah-Hartman 2020-11-04 9:21 ` Damien Le Moal 0 siblings, 1 reply; 15+ messages in thread From: Greg Kroah-Hartman @ 2020-11-04 9:15 UTC (permalink / raw) To: Damien Le Moal; +Cc: stable, Jens Axboe On Wed, Nov 04, 2020 at 10:10:15AM +0100, Greg Kroah-Hartman wrote: > On Wed, Nov 04, 2020 at 02:29:14PM +0900, Damien Le Moal wrote: > > commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. > > > > In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is > > ignored and the operation is applied to all sequential zones. For these > > commands, tracing the effect of the command using the command sector to > > determine the target zone is thus incorrect. > > > > Fix null_zone_mgmt() zone condition tracing in the case of > > REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are > > not already empty. > > > > Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") > > Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> > > Cc: stable@vger.kernel.org > > Signed-off-by: Jens Axboe <axboe@kernel.dk> > > --- > > drivers/block/null_blk_zoned.c | 14 ++++++++------ > > 1 file changed, 8 insertions(+), 6 deletions(-) > > Now queued up, thanks. Wait, no, I'll delay this one until the next round as it's not fixing something introduced in this -rc series. thanks, greg k-h ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] null_blk: Fix zone reset all tracing 2020-11-04 9:15 ` Greg Kroah-Hartman @ 2020-11-04 9:21 ` Damien Le Moal 2020-11-04 9:27 ` Greg Kroah-Hartman 0 siblings, 1 reply; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 9:21 UTC (permalink / raw) To: Greg Kroah-Hartman; +Cc: stable, Jens Axboe On 2020/11/04 18:14, Greg Kroah-Hartman wrote: > On Wed, Nov 04, 2020 at 10:10:15AM +0100, Greg Kroah-Hartman wrote: >> On Wed, Nov 04, 2020 at 02:29:14PM +0900, Damien Le Moal wrote: >>> commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. >>> >>> In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is >>> ignored and the operation is applied to all sequential zones. For these >>> commands, tracing the effect of the command using the command sector to >>> determine the target zone is thus incorrect. >>> >>> Fix null_zone_mgmt() zone condition tracing in the case of >>> REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are >>> not already empty. >>> >>> Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") >>> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> >>> Cc: stable@vger.kernel.org >>> Signed-off-by: Jens Axboe <axboe@kernel.dk> >>> --- >>> drivers/block/null_blk_zoned.c | 14 ++++++++------ >>> 1 file changed, 8 insertions(+), 6 deletions(-) >> >> Now queued up, thanks. > > Wait, no, I'll delay this one until the next round as it's not fixing > something introduced in this -rc series. Yes, that problem is older. The lock fix I sent goes on top of this one though. I can send the backport for the lock fix without this patch applied. Is that OK ? > > thanks, > > greg k-h > -- Damien Le Moal Western Digital Research ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] null_blk: Fix zone reset all tracing 2020-11-04 9:21 ` Damien Le Moal @ 2020-11-04 9:27 ` Greg Kroah-Hartman 2020-11-04 9:31 ` Damien Le Moal 0 siblings, 1 reply; 15+ messages in thread From: Greg Kroah-Hartman @ 2020-11-04 9:27 UTC (permalink / raw) To: Damien Le Moal; +Cc: stable, Jens Axboe On Wed, Nov 04, 2020 at 09:21:27AM +0000, Damien Le Moal wrote: > On 2020/11/04 18:14, Greg Kroah-Hartman wrote: > > On Wed, Nov 04, 2020 at 10:10:15AM +0100, Greg Kroah-Hartman wrote: > >> On Wed, Nov 04, 2020 at 02:29:14PM +0900, Damien Le Moal wrote: > >>> commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. > >>> > >>> In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is > >>> ignored and the operation is applied to all sequential zones. For these > >>> commands, tracing the effect of the command using the command sector to > >>> determine the target zone is thus incorrect. > >>> > >>> Fix null_zone_mgmt() zone condition tracing in the case of > >>> REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are > >>> not already empty. > >>> > >>> Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") > >>> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> > >>> Cc: stable@vger.kernel.org > >>> Signed-off-by: Jens Axboe <axboe@kernel.dk> > >>> --- > >>> drivers/block/null_blk_zoned.c | 14 ++++++++------ > >>> 1 file changed, 8 insertions(+), 6 deletions(-) > >> > >> Now queued up, thanks. > > > > Wait, no, I'll delay this one until the next round as it's not fixing > > something introduced in this -rc series. > > Yes, that problem is older. > The lock fix I sent goes on top of this one though. I can send the backport for > the lock fix without this patch applied. Is that OK ? If the order of the patches is needed, then yes, I can take both, please submit them as a patch series so that I know this is needed. thanks, greg k-h ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] null_blk: Fix zone reset all tracing 2020-11-04 9:27 ` Greg Kroah-Hartman @ 2020-11-04 9:31 ` Damien Le Moal 2020-11-04 9:51 ` Greg Kroah-Hartman 0 siblings, 1 reply; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 9:31 UTC (permalink / raw) To: Greg Kroah-Hartman; +Cc: stable, Jens Axboe On 2020/11/04 18:26, Greg Kroah-Hartman wrote: > On Wed, Nov 04, 2020 at 09:21:27AM +0000, Damien Le Moal wrote: >> On 2020/11/04 18:14, Greg Kroah-Hartman wrote: >>> On Wed, Nov 04, 2020 at 10:10:15AM +0100, Greg Kroah-Hartman wrote: >>>> On Wed, Nov 04, 2020 at 02:29:14PM +0900, Damien Le Moal wrote: >>>>> commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. >>>>> >>>>> In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is >>>>> ignored and the operation is applied to all sequential zones. For these >>>>> commands, tracing the effect of the command using the command sector to >>>>> determine the target zone is thus incorrect. >>>>> >>>>> Fix null_zone_mgmt() zone condition tracing in the case of >>>>> REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are >>>>> not already empty. >>>>> >>>>> Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") >>>>> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> >>>>> Cc: stable@vger.kernel.org >>>>> Signed-off-by: Jens Axboe <axboe@kernel.dk> >>>>> --- >>>>> drivers/block/null_blk_zoned.c | 14 ++++++++------ >>>>> 1 file changed, 8 insertions(+), 6 deletions(-) >>>> >>>> Now queued up, thanks. >>> >>> Wait, no, I'll delay this one until the next round as it's not fixing >>> something introduced in this -rc series. >> >> Yes, that problem is older. >> The lock fix I sent goes on top of this one though. I can send the backport for >> the lock fix without this patch applied. Is that OK ? > > If the order of the patches is needed, then yes, I can take both, please > submit them as a patch series so that I know this is needed. OK. Sending that. Note that I still do not see Kanchan patch applied in stable 5.9.y branch, so I will do the backport assuming it is applied. Or I can send all 3 patches as the series. Which do you prefer ? > > thanks, > > greg k-h > -- Damien Le Moal Western Digital Research ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] null_blk: Fix zone reset all tracing 2020-11-04 9:31 ` Damien Le Moal @ 2020-11-04 9:51 ` Greg Kroah-Hartman 2020-11-04 9:54 ` Damien Le Moal 2020-11-04 11:27 ` [PATCH 0/3] null_blk fixes for 5.9 stable Damien Le Moal 0 siblings, 2 replies; 15+ messages in thread From: Greg Kroah-Hartman @ 2020-11-04 9:51 UTC (permalink / raw) To: Damien Le Moal; +Cc: stable, Jens Axboe On Wed, Nov 04, 2020 at 09:31:40AM +0000, Damien Le Moal wrote: > On 2020/11/04 18:26, Greg Kroah-Hartman wrote: > > On Wed, Nov 04, 2020 at 09:21:27AM +0000, Damien Le Moal wrote: > >> On 2020/11/04 18:14, Greg Kroah-Hartman wrote: > >>> On Wed, Nov 04, 2020 at 10:10:15AM +0100, Greg Kroah-Hartman wrote: > >>>> On Wed, Nov 04, 2020 at 02:29:14PM +0900, Damien Le Moal wrote: > >>>>> commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. > >>>>> > >>>>> In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is > >>>>> ignored and the operation is applied to all sequential zones. For these > >>>>> commands, tracing the effect of the command using the command sector to > >>>>> determine the target zone is thus incorrect. > >>>>> > >>>>> Fix null_zone_mgmt() zone condition tracing in the case of > >>>>> REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are > >>>>> not already empty. > >>>>> > >>>>> Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") > >>>>> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> > >>>>> Cc: stable@vger.kernel.org > >>>>> Signed-off-by: Jens Axboe <axboe@kernel.dk> > >>>>> --- > >>>>> drivers/block/null_blk_zoned.c | 14 ++++++++------ > >>>>> 1 file changed, 8 insertions(+), 6 deletions(-) > >>>> > >>>> Now queued up, thanks. > >>> > >>> Wait, no, I'll delay this one until the next round as it's not fixing > >>> something introduced in this -rc series. > >> > >> Yes, that problem is older. > >> The lock fix I sent goes on top of this one though. I can send the backport for > >> the lock fix without this patch applied. Is that OK ? > > > > If the order of the patches is needed, then yes, I can take both, please > > submit them as a patch series so that I know this is needed. > > OK. Sending that. Note that I still do not see Kanchan patch applied in stable > 5.9.y branch, so I will do the backport assuming it is applied. Or I can send > all 3 patches as the series. Which do you prefer ? All 3 is great, to ensure I have them all as I don't know what you mean by "Kanchan patch". thanks, greg k-h ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] null_blk: Fix zone reset all tracing 2020-11-04 9:51 ` Greg Kroah-Hartman @ 2020-11-04 9:54 ` Damien Le Moal 2020-11-04 11:27 ` [PATCH 0/3] null_blk fixes for 5.9 stable Damien Le Moal 1 sibling, 0 replies; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 9:54 UTC (permalink / raw) To: Greg Kroah-Hartman; +Cc: stable, Jens Axboe On 2020/11/04 18:50, Greg Kroah-Hartman wrote: > On Wed, Nov 04, 2020 at 09:31:40AM +0000, Damien Le Moal wrote: >> On 2020/11/04 18:26, Greg Kroah-Hartman wrote: >>> On Wed, Nov 04, 2020 at 09:21:27AM +0000, Damien Le Moal wrote: >>>> On 2020/11/04 18:14, Greg Kroah-Hartman wrote: >>>>> On Wed, Nov 04, 2020 at 10:10:15AM +0100, Greg Kroah-Hartman wrote: >>>>>> On Wed, Nov 04, 2020 at 02:29:14PM +0900, Damien Le Moal wrote: >>>>>>> commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. >>>>>>> >>>>>>> In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is >>>>>>> ignored and the operation is applied to all sequential zones. For these >>>>>>> commands, tracing the effect of the command using the command sector to >>>>>>> determine the target zone is thus incorrect. >>>>>>> >>>>>>> Fix null_zone_mgmt() zone condition tracing in the case of >>>>>>> REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are >>>>>>> not already empty. >>>>>>> >>>>>>> Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") >>>>>>> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> >>>>>>> Cc: stable@vger.kernel.org >>>>>>> Signed-off-by: Jens Axboe <axboe@kernel.dk> >>>>>>> --- >>>>>>> drivers/block/null_blk_zoned.c | 14 ++++++++------ >>>>>>> 1 file changed, 8 insertions(+), 6 deletions(-) >>>>>> >>>>>> Now queued up, thanks. >>>>> >>>>> Wait, no, I'll delay this one until the next round as it's not fixing >>>>> something introduced in this -rc series. >>>> >>>> Yes, that problem is older. >>>> The lock fix I sent goes on top of this one though. I can send the backport for >>>> the lock fix without this patch applied. Is that OK ? >>> >>> If the order of the patches is needed, then yes, I can take both, please >>> submit them as a patch series so that I know this is needed. >> >> OK. Sending that. Note that I still do not see Kanchan patch applied in stable >> 5.9.y branch, so I will do the backport assuming it is applied. Or I can send >> all 3 patches as the series. Which do you prefer ? > > All 3 is great, to ensure I have them all as I don't know what you mean > by "Kanchan patch". I was talking about " commit 35bc10b2eafbb701064b94f283b77c54d3304842 upstream." that you already applied to 5.9.y. Sending all 3 patches backported in a series to be sure. SInce the first patchin the series will be the above mentioned patch, everything should still apply cleanly on your side. Thanks! > > thanks, > > greg k-h > -- Damien Le Moal Western Digital Research ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 0/3] null_blk fixes for 5.9 stable 2020-11-04 9:51 ` Greg Kroah-Hartman 2020-11-04 9:54 ` Damien Le Moal @ 2020-11-04 11:27 ` Damien Le Moal 2020-11-04 11:27 ` [PATCH 1/3] null_blk: synchronization fix for zoned device Damien Le Moal ` (3 more replies) 1 sibling, 4 replies; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 11:27 UTC (permalink / raw) To: Greg Kroah-Hartman, stable; +Cc: Jens Axboe, Kanchan Joshi Greg, Here are three backported patches from this cycle for 5.9 stable. The upstream patches are: 1) 35bc10b2eafbb701064b94f283b77c54d3304842 2) f9c9104288da543cd64f186f9e2fba389f415630 3) aa1c09cb65e2ed17cb8e652bc7ec84e0af1229eb Thanks. Damien Le Moal (2): null_blk: Fix zone reset all tracing null_blk: Fix locking in zoned mode Kanchan Joshi (1): null_blk: synchronization fix for zoned device drivers/block/null_blk.h | 1 + drivers/block/null_blk_zoned.c | 157 ++++++++++++++++++++++++--------- 2 files changed, 118 insertions(+), 40 deletions(-) -- 2.26.2 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 1/3] null_blk: synchronization fix for zoned device 2020-11-04 11:27 ` [PATCH 0/3] null_blk fixes for 5.9 stable Damien Le Moal @ 2020-11-04 11:27 ` Damien Le Moal 2020-11-04 11:27 ` [PATCH 2/3] null_blk: Fix zone reset all tracing Damien Le Moal ` (2 subsequent siblings) 3 siblings, 0 replies; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 11:27 UTC (permalink / raw) To: Greg Kroah-Hartman, stable; +Cc: Jens Axboe, Kanchan Joshi From: Kanchan Joshi <joshi.k@samsung.com> commit 35bc10b2eafbb701064b94f283b77c54d3304842 upstream. Parallel write,read,zone-mgmt operations accessing/altering zone state and write-pointer may get into race. Avoid the situation by using a new spinlock for zoned device. Concurrent zone-appends (on a zone) returning same write-pointer issue is also avoided using this lock. Cc: stable@vger.kernel.org Fixes: e0489ed5daeb ("null_blk: Support REQ_OP_ZONE_APPEND") Signed-off-by: Kanchan Joshi <joshi.k@samsung.com> Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> --- drivers/block/null_blk.h | 1 + drivers/block/null_blk_zoned.c | 22 ++++++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/drivers/block/null_blk.h b/drivers/block/null_blk.h index daed4a9c3436..28099be50395 100644 --- a/drivers/block/null_blk.h +++ b/drivers/block/null_blk.h @@ -44,6 +44,7 @@ struct nullb_device { unsigned int nr_zones; struct blk_zone *zones; sector_t zone_size_sects; + spinlock_t zone_lock; unsigned long size; /* device size in MB */ unsigned long completion_nsec; /* time in ns to complete a request */ diff --git a/drivers/block/null_blk_zoned.c b/drivers/block/null_blk_zoned.c index 3d25c9ad2383..e8d8b13aaa5a 100644 --- a/drivers/block/null_blk_zoned.c +++ b/drivers/block/null_blk_zoned.c @@ -45,6 +45,7 @@ int null_init_zoned_dev(struct nullb_device *dev, struct request_queue *q) if (!dev->zones) return -ENOMEM; + spin_lock_init(&dev->zone_lock); if (dev->zone_nr_conv >= dev->nr_zones) { dev->zone_nr_conv = dev->nr_zones - 1; pr_info("changed the number of conventional zones to %u", @@ -131,8 +132,11 @@ int null_report_zones(struct gendisk *disk, sector_t sector, * So use a local copy to avoid corruption of the device zone * array. */ + spin_lock_irq(&dev->zone_lock); memcpy(&zone, &dev->zones[first_zone + i], sizeof(struct blk_zone)); + spin_unlock_irq(&dev->zone_lock); + error = cb(&zone, i, data); if (error) return error; @@ -277,18 +281,28 @@ static blk_status_t null_zone_mgmt(struct nullb_cmd *cmd, enum req_opf op, blk_status_t null_process_zoned_cmd(struct nullb_cmd *cmd, enum req_opf op, sector_t sector, sector_t nr_sectors) { + blk_status_t sts; + struct nullb_device *dev = cmd->nq->dev; + + spin_lock_irq(&dev->zone_lock); switch (op) { case REQ_OP_WRITE: - return null_zone_write(cmd, sector, nr_sectors, false); + sts = null_zone_write(cmd, sector, nr_sectors, false); + break; case REQ_OP_ZONE_APPEND: - return null_zone_write(cmd, sector, nr_sectors, true); + sts = null_zone_write(cmd, sector, nr_sectors, true); + break; case REQ_OP_ZONE_RESET: case REQ_OP_ZONE_RESET_ALL: case REQ_OP_ZONE_OPEN: case REQ_OP_ZONE_CLOSE: case REQ_OP_ZONE_FINISH: - return null_zone_mgmt(cmd, op, sector); + sts = null_zone_mgmt(cmd, op, sector); + break; default: - return null_process_cmd(cmd, op, sector, nr_sectors); + sts = null_process_cmd(cmd, op, sector, nr_sectors); } + spin_unlock_irq(&dev->zone_lock); + + return sts; } -- 2.26.2 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/3] null_blk: Fix zone reset all tracing 2020-11-04 11:27 ` [PATCH 0/3] null_blk fixes for 5.9 stable Damien Le Moal 2020-11-04 11:27 ` [PATCH 1/3] null_blk: synchronization fix for zoned device Damien Le Moal @ 2020-11-04 11:27 ` Damien Le Moal 2020-11-04 11:27 ` [PATCH 3/3] null_blk: Fix locking in zoned mode Damien Le Moal 2020-11-04 12:24 ` [PATCH 0/3] null_blk fixes for 5.9 stable Greg Kroah-Hartman 3 siblings, 0 replies; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 11:27 UTC (permalink / raw) To: Greg Kroah-Hartman, stable; +Cc: Jens Axboe, Kanchan Joshi commit f9c9104288da543cd64f186f9e2fba389f415630 upstream. In the cae of the REQ_OP_ZONE_RESET_ALL operation, the command sector is ignored and the operation is applied to all sequential zones. For these commands, tracing the effect of the command using the command sector to determine the target zone is thus incorrect. Fix null_zone_mgmt() zone condition tracing in the case of REQ_OP_ZONE_RESET_ALL to apply tracing to all sequential zones that are not already empty. Fixes: 766c3297d7e1 ("null_blk: add trace in null_blk_zoned.c") Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Cc: stable@vger.kernel.org Signed-off-by: Jens Axboe <axboe@kernel.dk> --- drivers/block/null_blk_zoned.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/block/null_blk_zoned.c b/drivers/block/null_blk_zoned.c index e8d8b13aaa5a..0342696db0ce 100644 --- a/drivers/block/null_blk_zoned.c +++ b/drivers/block/null_blk_zoned.c @@ -230,13 +230,15 @@ static blk_status_t null_zone_mgmt(struct nullb_cmd *cmd, enum req_opf op, switch (op) { case REQ_OP_ZONE_RESET_ALL: - for (i = 0; i < dev->nr_zones; i++) { - if (zone[i].type == BLK_ZONE_TYPE_CONVENTIONAL) - continue; - zone[i].cond = BLK_ZONE_COND_EMPTY; - zone[i].wp = zone[i].start; + for (i = dev->zone_nr_conv; i < dev->nr_zones; i++) { + zone = &dev->zones[i]; + if (zone->cond != BLK_ZONE_COND_EMPTY) { + zone->cond = BLK_ZONE_COND_EMPTY; + zone->wp = zone->start; + trace_nullb_zone_op(cmd, i, zone->cond); + } } - break; + return BLK_STS_OK; case REQ_OP_ZONE_RESET: if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) return BLK_STS_IOERR; -- 2.26.2 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/3] null_blk: Fix locking in zoned mode 2020-11-04 11:27 ` [PATCH 0/3] null_blk fixes for 5.9 stable Damien Le Moal 2020-11-04 11:27 ` [PATCH 1/3] null_blk: synchronization fix for zoned device Damien Le Moal 2020-11-04 11:27 ` [PATCH 2/3] null_blk: Fix zone reset all tracing Damien Le Moal @ 2020-11-04 11:27 ` Damien Le Moal 2020-11-04 12:24 ` [PATCH 0/3] null_blk fixes for 5.9 stable Greg Kroah-Hartman 3 siblings, 0 replies; 15+ messages in thread From: Damien Le Moal @ 2020-11-04 11:27 UTC (permalink / raw) To: Greg Kroah-Hartman, stable; +Cc: Jens Axboe, Kanchan Joshi commit aa1c09cb65e2ed17cb8e652bc7ec84e0af1229eb upstream. When the zoned mode is enabled in null_blk, Serializing read, write and zone management operations for each zone is necessary to protect device level information for managing zone resources (zone open and closed counters) as well as each zone condition and write pointer position. Commit 35bc10b2eafb ("null_blk: synchronization fix for zoned device") introduced a spinlock to implement this serialization. However, when memory backing is also enabled, GFP_NOIO memory allocations are executed under the spinlock, resulting in might_sleep() warnings. Furthermore, the zone_lock spinlock is locked/unlocked using spin_lock_irq/spin_unlock_irq, similarly to the memory backing code with the nullb->lock spinlock. This nested use of irq locks wrecks the irq enabled/disabled state. Fix all this by introducing a bitmap for per-zone lock, with locking implemented using wait_on_bit_lock_io() and clear_and_wake_up_bit(). This locking mechanism allows keeping a zone locked while executing null_process_cmd(), serializing all operations to the zone while allowing to sleep during memory backing allocation with GFP_NOIO. Device level zone resource management information is protected using a spinlock which is not held while executing null_process_cmd(); Fixes: 35bc10b2eafb ("null_blk: synchronization fix for zoned device") Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> --- drivers/block/null_blk.h | 2 +- drivers/block/null_blk_zoned.c | 133 ++++++++++++++++++++++++--------- 2 files changed, 98 insertions(+), 37 deletions(-) diff --git a/drivers/block/null_blk.h b/drivers/block/null_blk.h index 28099be50395..206309ecc7e4 100644 --- a/drivers/block/null_blk.h +++ b/drivers/block/null_blk.h @@ -44,7 +44,7 @@ struct nullb_device { unsigned int nr_zones; struct blk_zone *zones; sector_t zone_size_sects; - spinlock_t zone_lock; + unsigned long *zone_locks; unsigned long size; /* device size in MB */ unsigned long completion_nsec; /* time in ns to complete a request */ diff --git a/drivers/block/null_blk_zoned.c b/drivers/block/null_blk_zoned.c index 0342696db0ce..495713d6c989 100644 --- a/drivers/block/null_blk_zoned.c +++ b/drivers/block/null_blk_zoned.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #include <linux/vmalloc.h> +#include <linux/bitmap.h> #include "null_blk.h" #define CREATE_TRACE_POINTS @@ -45,7 +46,12 @@ int null_init_zoned_dev(struct nullb_device *dev, struct request_queue *q) if (!dev->zones) return -ENOMEM; - spin_lock_init(&dev->zone_lock); + dev->zone_locks = bitmap_zalloc(dev->nr_zones, GFP_KERNEL); + if (!dev->zone_locks) { + kvfree(dev->zones); + return -ENOMEM; + } + if (dev->zone_nr_conv >= dev->nr_zones) { dev->zone_nr_conv = dev->nr_zones - 1; pr_info("changed the number of conventional zones to %u", @@ -106,15 +112,26 @@ int null_register_zoned_dev(struct nullb *nullb) void null_free_zoned_dev(struct nullb_device *dev) { + bitmap_free(dev->zone_locks); kvfree(dev->zones); } +static inline void null_lock_zone(struct nullb_device *dev, unsigned int zno) +{ + wait_on_bit_lock_io(dev->zone_locks, zno, TASK_UNINTERRUPTIBLE); +} + +static inline void null_unlock_zone(struct nullb_device *dev, unsigned int zno) +{ + clear_and_wake_up_bit(zno, dev->zone_locks); +} + int null_report_zones(struct gendisk *disk, sector_t sector, unsigned int nr_zones, report_zones_cb cb, void *data) { struct nullb *nullb = disk->private_data; struct nullb_device *dev = nullb->dev; - unsigned int first_zone, i; + unsigned int first_zone, i, zno; struct blk_zone zone; int error; @@ -125,17 +142,17 @@ int null_report_zones(struct gendisk *disk, sector_t sector, nr_zones = min(nr_zones, dev->nr_zones - first_zone); trace_nullb_report_zones(nullb, nr_zones); - for (i = 0; i < nr_zones; i++) { + zno = first_zone; + for (i = 0; i < nr_zones; i++, zno++) { /* * Stacked DM target drivers will remap the zone information by * modifying the zone information passed to the report callback. * So use a local copy to avoid corruption of the device zone * array. */ - spin_lock_irq(&dev->zone_lock); - memcpy(&zone, &dev->zones[first_zone + i], - sizeof(struct blk_zone)); - spin_unlock_irq(&dev->zone_lock); + null_lock_zone(dev, zno); + memcpy(&zone, &dev->zones[zno], sizeof(struct blk_zone)); + null_unlock_zone(dev, zno); error = cb(&zone, i, data); if (error) @@ -145,6 +162,10 @@ int null_report_zones(struct gendisk *disk, sector_t sector, return nr_zones; } +/* + * This is called in the case of memory backing from null_process_cmd() + * with the target zone already locked. + */ size_t null_zone_valid_read_len(struct nullb *nullb, sector_t sector, unsigned int len) { @@ -176,10 +197,13 @@ static blk_status_t null_zone_write(struct nullb_cmd *cmd, sector_t sector, if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) return null_process_cmd(cmd, REQ_OP_WRITE, sector, nr_sectors); + null_lock_zone(dev, zno); + switch (zone->cond) { case BLK_ZONE_COND_FULL: /* Cannot write to a full zone */ - return BLK_STS_IOERR; + ret = BLK_STS_IOERR; + break; case BLK_ZONE_COND_EMPTY: case BLK_ZONE_COND_IMP_OPEN: case BLK_ZONE_COND_EXP_OPEN: @@ -197,68 +221,96 @@ static blk_status_t null_zone_write(struct nullb_cmd *cmd, sector_t sector, else cmd->rq->__sector = sector; } else if (sector != zone->wp) { - return BLK_STS_IOERR; + ret = BLK_STS_IOERR; + break; } - if (zone->wp + nr_sectors > zone->start + zone->capacity) - return BLK_STS_IOERR; + if (zone->wp + nr_sectors > zone->start + zone->capacity) { + ret = BLK_STS_IOERR; + break; + } if (zone->cond != BLK_ZONE_COND_EXP_OPEN) zone->cond = BLK_ZONE_COND_IMP_OPEN; ret = null_process_cmd(cmd, REQ_OP_WRITE, sector, nr_sectors); if (ret != BLK_STS_OK) - return ret; + break; zone->wp += nr_sectors; if (zone->wp == zone->start + zone->capacity) zone->cond = BLK_ZONE_COND_FULL; - return BLK_STS_OK; + ret = BLK_STS_OK; + break; default: /* Invalid zone condition */ - return BLK_STS_IOERR; + ret = BLK_STS_IOERR; } + + null_unlock_zone(dev, zno); + + return ret; } static blk_status_t null_zone_mgmt(struct nullb_cmd *cmd, enum req_opf op, sector_t sector) { struct nullb_device *dev = cmd->nq->dev; - unsigned int zone_no = null_zone_no(dev, sector); - struct blk_zone *zone = &dev->zones[zone_no]; + unsigned int zone_no; + struct blk_zone *zone; + blk_status_t ret = BLK_STS_OK; size_t i; - switch (op) { - case REQ_OP_ZONE_RESET_ALL: + if (op == REQ_OP_ZONE_RESET_ALL) { for (i = dev->zone_nr_conv; i < dev->nr_zones; i++) { + null_lock_zone(dev, i); zone = &dev->zones[i]; if (zone->cond != BLK_ZONE_COND_EMPTY) { zone->cond = BLK_ZONE_COND_EMPTY; zone->wp = zone->start; trace_nullb_zone_op(cmd, i, zone->cond); } + null_unlock_zone(dev, i); } return BLK_STS_OK; + } + + zone_no = null_zone_no(dev, sector); + zone = &dev->zones[zone_no]; + + null_lock_zone(dev, zone_no); + + switch (op) { case REQ_OP_ZONE_RESET: - if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) - return BLK_STS_IOERR; + if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) { + ret = BLK_STS_IOERR; + break; + } zone->cond = BLK_ZONE_COND_EMPTY; zone->wp = zone->start; break; case REQ_OP_ZONE_OPEN: - if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) - return BLK_STS_IOERR; - if (zone->cond == BLK_ZONE_COND_FULL) - return BLK_STS_IOERR; + if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) { + ret = BLK_STS_IOERR; + break; + } + if (zone->cond == BLK_ZONE_COND_FULL) { + ret = BLK_STS_IOERR; + break; + } zone->cond = BLK_ZONE_COND_EXP_OPEN; break; case REQ_OP_ZONE_CLOSE: - if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) - return BLK_STS_IOERR; - if (zone->cond == BLK_ZONE_COND_FULL) - return BLK_STS_IOERR; + if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) { + ret = BLK_STS_IOERR; + break; + } + if (zone->cond == BLK_ZONE_COND_FULL) { + ret = BLK_STS_IOERR; + break; + } if (zone->wp == zone->start) zone->cond = BLK_ZONE_COND_EMPTY; @@ -266,27 +318,35 @@ static blk_status_t null_zone_mgmt(struct nullb_cmd *cmd, enum req_opf op, zone->cond = BLK_ZONE_COND_CLOSED; break; case REQ_OP_ZONE_FINISH: - if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) - return BLK_STS_IOERR; + if (zone->type == BLK_ZONE_TYPE_CONVENTIONAL) { + ret = BLK_STS_IOERR; + break; + } zone->cond = BLK_ZONE_COND_FULL; zone->wp = zone->start + zone->len; + ret = BLK_STS_OK; break; default: - return BLK_STS_NOTSUPP; + ret = BLK_STS_NOTSUPP; + break; } - trace_nullb_zone_op(cmd, zone_no, zone->cond); - return BLK_STS_OK; + if (ret == BLK_STS_OK) + trace_nullb_zone_op(cmd, zone_no, zone->cond); + + null_unlock_zone(dev, zone_no); + + return ret; } blk_status_t null_process_zoned_cmd(struct nullb_cmd *cmd, enum req_opf op, sector_t sector, sector_t nr_sectors) { - blk_status_t sts; struct nullb_device *dev = cmd->nq->dev; + unsigned int zno = null_zone_no(dev, sector); + blk_status_t sts; - spin_lock_irq(&dev->zone_lock); switch (op) { case REQ_OP_WRITE: sts = null_zone_write(cmd, sector, nr_sectors, false); @@ -302,9 +362,10 @@ blk_status_t null_process_zoned_cmd(struct nullb_cmd *cmd, enum req_opf op, sts = null_zone_mgmt(cmd, op, sector); break; default: + null_lock_zone(dev, zno); sts = null_process_cmd(cmd, op, sector, nr_sectors); + null_unlock_zone(dev, zno); } - spin_unlock_irq(&dev->zone_lock); return sts; } -- 2.26.2 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 0/3] null_blk fixes for 5.9 stable 2020-11-04 11:27 ` [PATCH 0/3] null_blk fixes for 5.9 stable Damien Le Moal ` (2 preceding siblings ...) 2020-11-04 11:27 ` [PATCH 3/3] null_blk: Fix locking in zoned mode Damien Le Moal @ 2020-11-04 12:24 ` Greg Kroah-Hartman 3 siblings, 0 replies; 15+ messages in thread From: Greg Kroah-Hartman @ 2020-11-04 12:24 UTC (permalink / raw) To: Damien Le Moal; +Cc: stable, Jens Axboe, Kanchan Joshi On Wed, Nov 04, 2020 at 08:27:44PM +0900, Damien Le Moal wrote: > Greg, > > Here are three backported patches from this cycle for 5.9 stable. The > upstream patches are: > 1) 35bc10b2eafbb701064b94f283b77c54d3304842 > 2) f9c9104288da543cd64f186f9e2fba389f415630 > 3) aa1c09cb65e2ed17cb8e652bc7ec84e0af1229eb > > Thanks. > > Damien Le Moal (2): > null_blk: Fix zone reset all tracing > null_blk: Fix locking in zoned mode > > Kanchan Joshi (1): > null_blk: synchronization fix for zoned device > > drivers/block/null_blk.h | 1 + > drivers/block/null_blk_zoned.c | 157 ++++++++++++++++++++++++--------- > 2 files changed, 118 insertions(+), 40 deletions(-) All now queued up, thanks. greg k-h ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2020-11-04 12:23 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-03 14:24 FAILED: patch "[PATCH] null_blk: Fix zone reset all tracing" failed to apply to 5.9-stable tree gregkh 2020-11-03 23:37 ` Damien Le Moal 2020-11-04 5:29 ` [PATCH] null_blk: Fix zone reset all tracing Damien Le Moal 2020-11-04 9:10 ` Greg Kroah-Hartman 2020-11-04 9:15 ` Greg Kroah-Hartman 2020-11-04 9:21 ` Damien Le Moal 2020-11-04 9:27 ` Greg Kroah-Hartman 2020-11-04 9:31 ` Damien Le Moal 2020-11-04 9:51 ` Greg Kroah-Hartman 2020-11-04 9:54 ` Damien Le Moal 2020-11-04 11:27 ` [PATCH 0/3] null_blk fixes for 5.9 stable Damien Le Moal 2020-11-04 11:27 ` [PATCH 1/3] null_blk: synchronization fix for zoned device Damien Le Moal 2020-11-04 11:27 ` [PATCH 2/3] null_blk: Fix zone reset all tracing Damien Le Moal 2020-11-04 11:27 ` [PATCH 3/3] null_blk: Fix locking in zoned mode Damien Le Moal 2020-11-04 12:24 ` [PATCH 0/3] null_blk fixes for 5.9 stable Greg Kroah-Hartman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).