* [PATCH] virtio_console: break out of buf poll on remove @ 2021-10-05 7:04 ` Michael S. Tsirkin 0 siblings, 0 replies; 6+ messages in thread From: Michael S. Tsirkin @ 2021-10-05 7:04 UTC (permalink / raw) To: linux-kernel; +Cc: Amit Shah, Arnd Bergmann, Greg Kroah-Hartman, virtualization A common pattern for device reset is currently: vdev->config->reset(vdev); .. cleanup .. reset prevents new interrupts from arriving and waits for interrupt handlers to finish. However if - as is common - the handler queues a work request which is flushed during the cleanup stage, we have code adding buffers / trying to get buffers while device is reset. Not good. This was reproduced by running modprobe virtio_console modprobe -r virtio_console in a loop. Fixing this comprehensively needs some thought, and new APIs. Let's at least handle the specific case of virtio_console removal that was reported in the field. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1786239 Signed-off-by: Michael S. Tsirkin <mst@redhat.com> --- drivers/char/virtio_console.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c index 7eaf303a7a86..c852ce0b4d56 100644 --- a/drivers/char/virtio_console.c +++ b/drivers/char/virtio_console.c @@ -1956,6 +1956,12 @@ static void virtcons_remove(struct virtio_device *vdev) list_del(&portdev->list); spin_unlock_irq(&pdrvdata_lock); + /* Device is going away, exit any polling for buffers */ + virtio_break_device(vdev); + if (use_multiport(portdev)) + flush_work(&portdev->control_work); + else + flush_work(&portdev->config_work); /* Disable interrupts for vqs */ vdev->config->reset(vdev); /* Finish up work that's lined up */ -- MST ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH] virtio_console: break out of buf poll on remove @ 2021-10-05 7:04 ` Michael S. Tsirkin 0 siblings, 0 replies; 6+ messages in thread From: Michael S. Tsirkin @ 2021-10-05 7:04 UTC (permalink / raw) To: linux-kernel; +Cc: Greg Kroah-Hartman, virtualization, Arnd Bergmann, Amit Shah A common pattern for device reset is currently: vdev->config->reset(vdev); .. cleanup .. reset prevents new interrupts from arriving and waits for interrupt handlers to finish. However if - as is common - the handler queues a work request which is flushed during the cleanup stage, we have code adding buffers / trying to get buffers while device is reset. Not good. This was reproduced by running modprobe virtio_console modprobe -r virtio_console in a loop. Fixing this comprehensively needs some thought, and new APIs. Let's at least handle the specific case of virtio_console removal that was reported in the field. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1786239 Signed-off-by: Michael S. Tsirkin <mst@redhat.com> --- drivers/char/virtio_console.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c index 7eaf303a7a86..c852ce0b4d56 100644 --- a/drivers/char/virtio_console.c +++ b/drivers/char/virtio_console.c @@ -1956,6 +1956,12 @@ static void virtcons_remove(struct virtio_device *vdev) list_del(&portdev->list); spin_unlock_irq(&pdrvdata_lock); + /* Device is going away, exit any polling for buffers */ + virtio_break_device(vdev); + if (use_multiport(portdev)) + flush_work(&portdev->control_work); + else + flush_work(&portdev->config_work); /* Disable interrupts for vqs */ vdev->config->reset(vdev); /* Finish up work that's lined up */ -- MST _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_console: break out of buf poll on remove 2021-10-05 7:04 ` Michael S. Tsirkin @ 2021-10-05 13:33 ` Greg Kroah-Hartman -1 siblings, 0 replies; 6+ messages in thread From: Greg Kroah-Hartman @ 2021-10-05 13:33 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: linux-kernel, Amit Shah, Arnd Bergmann, virtualization On Tue, Oct 05, 2021 at 03:04:07AM -0400, Michael S. Tsirkin wrote: > A common pattern for device reset is currently: > vdev->config->reset(vdev); > .. cleanup .. > > reset prevents new interrupts from arriving and waits for interrupt > handlers to finish. > > However if - as is common - the handler queues a work request which is > flushed during the cleanup stage, we have code adding buffers / trying > to get buffers while device is reset. Not good. > > This was reproduced by running > modprobe virtio_console > modprobe -r virtio_console > in a loop. That's a pathological case that is not "in the field" except by people who want to abuse the system as root. And they can do much worse things than that. > Fixing this comprehensively needs some thought, and new APIs. > Let's at least handle the specific case of virtio_console > removal that was reported in the field. Let's fix this correctly, don't just hack it up now. > Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1786239 > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > --- > drivers/char/virtio_console.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c > index 7eaf303a7a86..c852ce0b4d56 100644 > --- a/drivers/char/virtio_console.c > +++ b/drivers/char/virtio_console.c > @@ -1956,6 +1956,12 @@ static void virtcons_remove(struct virtio_device *vdev) > list_del(&portdev->list); > spin_unlock_irq(&pdrvdata_lock); > > + /* Device is going away, exit any polling for buffers */ > + virtio_break_device(vdev); > + if (use_multiport(portdev)) > + flush_work(&portdev->control_work); > + else > + flush_work(&portdev->config_work); > /* Disable interrupts for vqs */ newline before comment? thanks, greg k-h ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_console: break out of buf poll on remove @ 2021-10-05 13:33 ` Greg Kroah-Hartman 0 siblings, 0 replies; 6+ messages in thread From: Greg Kroah-Hartman @ 2021-10-05 13:33 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: virtualization, linux-kernel, Arnd Bergmann, Amit Shah On Tue, Oct 05, 2021 at 03:04:07AM -0400, Michael S. Tsirkin wrote: > A common pattern for device reset is currently: > vdev->config->reset(vdev); > .. cleanup .. > > reset prevents new interrupts from arriving and waits for interrupt > handlers to finish. > > However if - as is common - the handler queues a work request which is > flushed during the cleanup stage, we have code adding buffers / trying > to get buffers while device is reset. Not good. > > This was reproduced by running > modprobe virtio_console > modprobe -r virtio_console > in a loop. That's a pathological case that is not "in the field" except by people who want to abuse the system as root. And they can do much worse things than that. > Fixing this comprehensively needs some thought, and new APIs. > Let's at least handle the specific case of virtio_console > removal that was reported in the field. Let's fix this correctly, don't just hack it up now. > Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1786239 > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > --- > drivers/char/virtio_console.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c > index 7eaf303a7a86..c852ce0b4d56 100644 > --- a/drivers/char/virtio_console.c > +++ b/drivers/char/virtio_console.c > @@ -1956,6 +1956,12 @@ static void virtcons_remove(struct virtio_device *vdev) > list_del(&portdev->list); > spin_unlock_irq(&pdrvdata_lock); > > + /* Device is going away, exit any polling for buffers */ > + virtio_break_device(vdev); > + if (use_multiport(portdev)) > + flush_work(&portdev->control_work); > + else > + flush_work(&portdev->config_work); > /* Disable interrupts for vqs */ newline before comment? thanks, greg k-h _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_console: break out of buf poll on remove 2021-10-05 13:33 ` Greg Kroah-Hartman @ 2022-01-14 21:48 ` Michael S. Tsirkin -1 siblings, 0 replies; 6+ messages in thread From: Michael S. Tsirkin @ 2022-01-14 21:48 UTC (permalink / raw) To: Greg Kroah-Hartman; +Cc: linux-kernel, Amit Shah, Arnd Bergmann, virtualization On Tue, Oct 05, 2021 at 03:33:42PM +0200, Greg Kroah-Hartman wrote: > On Tue, Oct 05, 2021 at 03:04:07AM -0400, Michael S. Tsirkin wrote: > > A common pattern for device reset is currently: > > vdev->config->reset(vdev); > > .. cleanup .. > > > > reset prevents new interrupts from arriving and waits for interrupt > > handlers to finish. > > > > However if - as is common - the handler queues a work request which is > > flushed during the cleanup stage, we have code adding buffers / trying > > to get buffers while device is reset. Not good. > > > > This was reproduced by running > > modprobe virtio_console > > modprobe -r virtio_console > > in a loop. > > That's a pathological case that is not "in the field" except by people > who want to abuse the system as root. And they can do much worse things > than that. > > > Fixing this comprehensively needs some thought, and new APIs. > > Let's at least handle the specific case of virtio_console > > removal that was reported in the field. > > Let's fix this correctly, don't just hack it up now. Well I poked at it some more, and things are not as bad as I thought. It's mostly just console and possibly virtio-mem. Well and virtio-bt has a completely borken cleanup that does not even bother to reset the device, but that's a separate issue, discussing it with the maintainer. So I wrote some patches to document the requirements better, added a wrapper for reset and generally cleaned the API up a bit, and added a patch for mem, but generally I no longer think we need a major API change. > > Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1786239 > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > > --- > > drivers/char/virtio_console.c | 6 ++++++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c > > index 7eaf303a7a86..c852ce0b4d56 100644 > > --- a/drivers/char/virtio_console.c > > +++ b/drivers/char/virtio_console.c > > @@ -1956,6 +1956,12 @@ static void virtcons_remove(struct virtio_device *vdev) > > list_del(&portdev->list); > > spin_unlock_irq(&pdrvdata_lock); > > > > + /* Device is going away, exit any polling for buffers */ > > + virtio_break_device(vdev); > > + if (use_multiport(portdev)) > > + flush_work(&portdev->control_work); > > + else > > + flush_work(&portdev->config_work); > > /* Disable interrupts for vqs */ > > newline before comment? sure > thanks, > > greg k-h ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_console: break out of buf poll on remove @ 2022-01-14 21:48 ` Michael S. Tsirkin 0 siblings, 0 replies; 6+ messages in thread From: Michael S. Tsirkin @ 2022-01-14 21:48 UTC (permalink / raw) To: Greg Kroah-Hartman; +Cc: virtualization, linux-kernel, Arnd Bergmann, Amit Shah On Tue, Oct 05, 2021 at 03:33:42PM +0200, Greg Kroah-Hartman wrote: > On Tue, Oct 05, 2021 at 03:04:07AM -0400, Michael S. Tsirkin wrote: > > A common pattern for device reset is currently: > > vdev->config->reset(vdev); > > .. cleanup .. > > > > reset prevents new interrupts from arriving and waits for interrupt > > handlers to finish. > > > > However if - as is common - the handler queues a work request which is > > flushed during the cleanup stage, we have code adding buffers / trying > > to get buffers while device is reset. Not good. > > > > This was reproduced by running > > modprobe virtio_console > > modprobe -r virtio_console > > in a loop. > > That's a pathological case that is not "in the field" except by people > who want to abuse the system as root. And they can do much worse things > than that. > > > Fixing this comprehensively needs some thought, and new APIs. > > Let's at least handle the specific case of virtio_console > > removal that was reported in the field. > > Let's fix this correctly, don't just hack it up now. Well I poked at it some more, and things are not as bad as I thought. It's mostly just console and possibly virtio-mem. Well and virtio-bt has a completely borken cleanup that does not even bother to reset the device, but that's a separate issue, discussing it with the maintainer. So I wrote some patches to document the requirements better, added a wrapper for reset and generally cleaned the API up a bit, and added a patch for mem, but generally I no longer think we need a major API change. > > Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1786239 > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > > --- > > drivers/char/virtio_console.c | 6 ++++++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c > > index 7eaf303a7a86..c852ce0b4d56 100644 > > --- a/drivers/char/virtio_console.c > > +++ b/drivers/char/virtio_console.c > > @@ -1956,6 +1956,12 @@ static void virtcons_remove(struct virtio_device *vdev) > > list_del(&portdev->list); > > spin_unlock_irq(&pdrvdata_lock); > > > > + /* Device is going away, exit any polling for buffers */ > > + virtio_break_device(vdev); > > + if (use_multiport(portdev)) > > + flush_work(&portdev->control_work); > > + else > > + flush_work(&portdev->config_work); > > /* Disable interrupts for vqs */ > > newline before comment? sure > thanks, > > greg k-h _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-14 21:49 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-10-05 7:04 [PATCH] virtio_console: break out of buf poll on remove Michael S. Tsirkin 2021-10-05 7:04 ` Michael S. Tsirkin 2021-10-05 13:33 ` Greg Kroah-Hartman 2021-10-05 13:33 ` Greg Kroah-Hartman 2022-01-14 21:48 ` Michael S. Tsirkin 2022-01-14 21:48 ` Michael S. Tsirkin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.