* [Qemu-devel] [PATCH 0/2] Monitor: make output buffer dynamic @ 2013-03-25 19:40 Luiz Capitulino 2013-03-25 19:40 ` [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() Luiz Capitulino ` (2 more replies) 0 siblings, 3 replies; 12+ messages in thread From: Luiz Capitulino @ 2013-03-25 19:40 UTC (permalink / raw) To: qemu-devel; +Cc: kraxel, fred.konrad Hi, This series fixes an easy to reproduce assertion in the Monitor code. The second patch contains all the relevant details. Gerd, I'd like a reviewed-by from you before merging this. Luiz Capitulino (2): qstring: add qobject_get_length() Monitor: Make output buffer dynamic include/qapi/qmp/qstring.h | 1 + monitor.c | 42 +++++++++++++++++++++++++----------------- qobject/qstring.c | 8 ++++++++ 3 files changed, 34 insertions(+), 17 deletions(-) -- 1.8.1.4 ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() 2013-03-25 19:40 [Qemu-devel] [PATCH 0/2] Monitor: make output buffer dynamic Luiz Capitulino @ 2013-03-25 19:40 ` Luiz Capitulino 2013-04-09 3:02 ` Hu Tao 2013-03-25 19:40 ` [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic Luiz Capitulino 2013-04-02 9:19 ` [Qemu-devel] [PATCH 0/2] Monitor: make " Gerd Hoffmann 2 siblings, 1 reply; 12+ messages in thread From: Luiz Capitulino @ 2013-03-25 19:40 UTC (permalink / raw) To: qemu-devel; +Cc: kraxel, fred.konrad Long overdue. Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> --- include/qapi/qmp/qstring.h | 1 + qobject/qstring.c | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h index 0e690f4..1bc3666 100644 --- a/include/qapi/qmp/qstring.h +++ b/include/qapi/qmp/qstring.h @@ -26,6 +26,7 @@ typedef struct QString { QString *qstring_new(void); QString *qstring_from_str(const char *str); QString *qstring_from_substr(const char *str, int start, int end); +size_t qstring_get_length(const QString *qstring); const char *qstring_get_str(const QString *qstring); void qstring_append_int(QString *qstring, int64_t value); void qstring_append(QString *qstring, const char *str); diff --git a/qobject/qstring.c b/qobject/qstring.c index 5f7376c..607b7a1 100644 --- a/qobject/qstring.c +++ b/qobject/qstring.c @@ -32,6 +32,14 @@ QString *qstring_new(void) } /** + * qstring_get_length(): Get the length of a QString + */ +size_t qstring_get_length(const QString *qstring) +{ + return qstring->length; +} + +/** * qstring_from_substr(): Create a new QString from a C string substring * * Return string reference -- 1.8.1.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() 2013-03-25 19:40 ` [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() Luiz Capitulino @ 2013-04-09 3:02 ` Hu Tao 2013-04-09 12:38 ` Luiz Capitulino 0 siblings, 1 reply; 12+ messages in thread From: Hu Tao @ 2013-04-09 3:02 UTC (permalink / raw) To: Luiz Capitulino; +Cc: fred.konrad, qemu-devel, kraxel 'add qobject_get_length()' in the subject line but you are actually adding qstring_get_length() On Mon, Mar 25, 2013 at 03:40:38PM -0400, Luiz Capitulino wrote: > Long overdue. > > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> > --- > include/qapi/qmp/qstring.h | 1 + > qobject/qstring.c | 8 ++++++++ > 2 files changed, 9 insertions(+) > > diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h > index 0e690f4..1bc3666 100644 > --- a/include/qapi/qmp/qstring.h > +++ b/include/qapi/qmp/qstring.h > @@ -26,6 +26,7 @@ typedef struct QString { > QString *qstring_new(void); > QString *qstring_from_str(const char *str); > QString *qstring_from_substr(const char *str, int start, int end); > +size_t qstring_get_length(const QString *qstring); > const char *qstring_get_str(const QString *qstring); > void qstring_append_int(QString *qstring, int64_t value); > void qstring_append(QString *qstring, const char *str); > diff --git a/qobject/qstring.c b/qobject/qstring.c > index 5f7376c..607b7a1 100644 > --- a/qobject/qstring.c > +++ b/qobject/qstring.c > @@ -32,6 +32,14 @@ QString *qstring_new(void) > } > > /** > + * qstring_get_length(): Get the length of a QString > + */ > +size_t qstring_get_length(const QString *qstring) > +{ > + return qstring->length; > +} > + > +/** > * qstring_from_substr(): Create a new QString from a C string substring > * > * Return string reference > -- > 1.8.1.4 ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() 2013-04-09 3:02 ` Hu Tao @ 2013-04-09 12:38 ` Luiz Capitulino 0 siblings, 0 replies; 12+ messages in thread From: Luiz Capitulino @ 2013-04-09 12:38 UTC (permalink / raw) To: Hu Tao; +Cc: fred.konrad, qemu-devel, kraxel On Tue, 9 Apr 2013 11:02:15 +0800 Hu Tao <hutao@cn.fujitsu.com> wrote: > 'add qobject_get_length()' in the subject line but you are actually > adding qstring_get_length() Thanks, but this was corrected in v2 and it's what got merged. > > On Mon, Mar 25, 2013 at 03:40:38PM -0400, Luiz Capitulino wrote: > > Long overdue. > > > > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> > > --- > > include/qapi/qmp/qstring.h | 1 + > > qobject/qstring.c | 8 ++++++++ > > 2 files changed, 9 insertions(+) > > > > diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h > > index 0e690f4..1bc3666 100644 > > --- a/include/qapi/qmp/qstring.h > > +++ b/include/qapi/qmp/qstring.h > > @@ -26,6 +26,7 @@ typedef struct QString { > > QString *qstring_new(void); > > QString *qstring_from_str(const char *str); > > QString *qstring_from_substr(const char *str, int start, int end); > > +size_t qstring_get_length(const QString *qstring); > > const char *qstring_get_str(const QString *qstring); > > void qstring_append_int(QString *qstring, int64_t value); > > void qstring_append(QString *qstring, const char *str); > > diff --git a/qobject/qstring.c b/qobject/qstring.c > > index 5f7376c..607b7a1 100644 > > --- a/qobject/qstring.c > > +++ b/qobject/qstring.c > > @@ -32,6 +32,14 @@ QString *qstring_new(void) > > } > > > > /** > > + * qstring_get_length(): Get the length of a QString > > + */ > > +size_t qstring_get_length(const QString *qstring) > > +{ > > + return qstring->length; > > +} > > + > > +/** > > * qstring_from_substr(): Create a new QString from a C string substring > > * > > * Return string reference > > -- > > 1.8.1.4 > ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic 2013-03-25 19:40 [Qemu-devel] [PATCH 0/2] Monitor: make output buffer dynamic Luiz Capitulino 2013-03-25 19:40 ` [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() Luiz Capitulino @ 2013-03-25 19:40 ` Luiz Capitulino 2013-03-25 21:07 ` Paolo Bonzini 2013-03-27 6:45 ` Wenchao Xia 2013-04-02 9:19 ` [Qemu-devel] [PATCH 0/2] Monitor: make " Gerd Hoffmann 2 siblings, 2 replies; 12+ messages in thread From: Luiz Capitulino @ 2013-03-25 19:40 UTC (permalink / raw) To: qemu-devel; +Cc: kraxel, fred.konrad Commit f628926bb423fa8a7e0b114511400ea9df38b76a changed monitor_flush() to retry on qemu_chr_fe_write() errors. However, the Monitor's output buffer can keep growing while the retry is not issued and this can cause the buffer to overflow. To reproduce this issue, just start qemu and type on the Monitor: (qemu) ? This will cause the assertion to trig. To fix this problem this commit makes the Monitor buffer dynamic, which means that it can grow as much as needed. Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> --- monitor.c | 42 +++++++++++++++++++++++++----------------- 1 file changed, 25 insertions(+), 17 deletions(-) diff --git a/monitor.c b/monitor.c index 2d9e887..c93dfe7 100644 --- a/monitor.c +++ b/monitor.c @@ -188,8 +188,7 @@ struct Monitor { int reset_seen; int flags; int suspend_cnt; - uint8_t outbuf[1024]; - int outbuf_index; + QString *outbuf; ReadLineState *rs; MonitorControl *mc; CPUArchState *mon_cpu; @@ -271,45 +270,52 @@ static gboolean monitor_unblocked(GIOChannel *chan, GIOCondition cond, void monitor_flush(Monitor *mon) { int rc; + size_t len; + const char *buf; + + buf = qstring_get_str(mon->outbuf); + len = qstring_get_length(mon->outbuf); - if (mon && mon->outbuf_index != 0 && !mon->mux_out) { - rc = qemu_chr_fe_write(mon->chr, mon->outbuf, mon->outbuf_index); - if (rc == mon->outbuf_index) { + if (mon && len && !mon->mux_out) { + rc = qemu_chr_fe_write(mon->chr, (const uint8_t *) buf, len); + if (rc == len) { /* all flushed */ - mon->outbuf_index = 0; + QDECREF(mon->outbuf); + mon->outbuf = qstring_new(); return; } if (rc > 0) { /* partinal write */ - memmove(mon->outbuf, mon->outbuf + rc, mon->outbuf_index - rc); - mon->outbuf_index -= rc; + QString *tmp = qstring_from_str(buf + rc); + QDECREF(mon->outbuf); + mon->outbuf = tmp; } qemu_chr_fe_add_watch(mon->chr, G_IO_OUT, monitor_unblocked, mon); } } -/* flush at every end of line or if the buffer is full */ +/* flush at every end of line */ static void monitor_puts(Monitor *mon, const char *str) { char c; for(;;) { - assert(mon->outbuf_index < sizeof(mon->outbuf) - 1); c = *str++; if (c == '\0') break; - if (c == '\n') - mon->outbuf[mon->outbuf_index++] = '\r'; - mon->outbuf[mon->outbuf_index++] = c; - if (mon->outbuf_index >= (sizeof(mon->outbuf) - 1) - || c == '\n') + if (c == '\n') { + qstring_append_chr(mon->outbuf, '\r'); + } + qstring_append_chr(mon->outbuf, c); + if (c == '\n') { monitor_flush(mon); + } } } void monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) { - char buf[4096]; + char *buf; if (!mon) return; @@ -318,8 +324,9 @@ void monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) return; } - vsnprintf(buf, sizeof(buf), fmt, ap); + buf = g_strdup_vprintf(fmt, ap); monitor_puts(mon, buf); + g_free(buf); } void monitor_printf(Monitor *mon, const char *fmt, ...) @@ -4732,6 +4739,7 @@ void monitor_init(CharDriverState *chr, int flags) } mon = g_malloc0(sizeof(*mon)); + mon->outbuf = qstring_new(); mon->chr = chr; mon->flags = flags; -- 1.8.1.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic 2013-03-25 19:40 ` [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic Luiz Capitulino @ 2013-03-25 21:07 ` Paolo Bonzini 2013-03-25 21:44 ` Luiz Capitulino 2013-03-27 6:45 ` Wenchao Xia 1 sibling, 1 reply; 12+ messages in thread From: Paolo Bonzini @ 2013-03-25 21:07 UTC (permalink / raw) To: Luiz Capitulino; +Cc: fred.konrad, qemu-devel, kraxel Il 25/03/2013 20:40, Luiz Capitulino ha scritto: > Commit f628926bb423fa8a7e0b114511400ea9df38b76a changed monitor_flush() > to retry on qemu_chr_fe_write() errors. However, the Monitor's output > buffer can keep growing while the retry is not issued and this can > cause the buffer to overflow. > > To reproduce this issue, just start qemu and type on the Monitor: > > (qemu) ? > > This will cause the assertion to trig. > > To fix this problem this commit makes the Monitor buffer dynamic, > which means that it can grow as much as needed. What about using a GString instead? Paolo > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> > --- > monitor.c | 42 +++++++++++++++++++++++++----------------- > 1 file changed, 25 insertions(+), 17 deletions(-) > > diff --git a/monitor.c b/monitor.c > index 2d9e887..c93dfe7 100644 > --- a/monitor.c > +++ b/monitor.c > @@ -188,8 +188,7 @@ struct Monitor { > int reset_seen; > int flags; > int suspend_cnt; > - uint8_t outbuf[1024]; > - int outbuf_index; > + QString *outbuf; > ReadLineState *rs; > MonitorControl *mc; > CPUArchState *mon_cpu; > @@ -271,45 +270,52 @@ static gboolean monitor_unblocked(GIOChannel *chan, GIOCondition cond, > void monitor_flush(Monitor *mon) > { > int rc; > + size_t len; > + const char *buf; > + > + buf = qstring_get_str(mon->outbuf); > + len = qstring_get_length(mon->outbuf); > > - if (mon && mon->outbuf_index != 0 && !mon->mux_out) { > - rc = qemu_chr_fe_write(mon->chr, mon->outbuf, mon->outbuf_index); > - if (rc == mon->outbuf_index) { > + if (mon && len && !mon->mux_out) { > + rc = qemu_chr_fe_write(mon->chr, (const uint8_t *) buf, len); > + if (rc == len) { > /* all flushed */ > - mon->outbuf_index = 0; > + QDECREF(mon->outbuf); > + mon->outbuf = qstring_new(); > return; > } > if (rc > 0) { > /* partinal write */ > - memmove(mon->outbuf, mon->outbuf + rc, mon->outbuf_index - rc); > - mon->outbuf_index -= rc; > + QString *tmp = qstring_from_str(buf + rc); > + QDECREF(mon->outbuf); > + mon->outbuf = tmp; > } > qemu_chr_fe_add_watch(mon->chr, G_IO_OUT, monitor_unblocked, mon); > } > } > > -/* flush at every end of line or if the buffer is full */ > +/* flush at every end of line */ > static void monitor_puts(Monitor *mon, const char *str) > { > char c; > > for(;;) { > - assert(mon->outbuf_index < sizeof(mon->outbuf) - 1); > c = *str++; > if (c == '\0') > break; > - if (c == '\n') > - mon->outbuf[mon->outbuf_index++] = '\r'; > - mon->outbuf[mon->outbuf_index++] = c; > - if (mon->outbuf_index >= (sizeof(mon->outbuf) - 1) > - || c == '\n') > + if (c == '\n') { > + qstring_append_chr(mon->outbuf, '\r'); > + } > + qstring_append_chr(mon->outbuf, c); > + if (c == '\n') { > monitor_flush(mon); > + } > } > } > > void monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) > { > - char buf[4096]; > + char *buf; > > if (!mon) > return; > @@ -318,8 +324,9 @@ void monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) > return; > } > > - vsnprintf(buf, sizeof(buf), fmt, ap); > + buf = g_strdup_vprintf(fmt, ap); > monitor_puts(mon, buf); > + g_free(buf); > } > > void monitor_printf(Monitor *mon, const char *fmt, ...) > @@ -4732,6 +4739,7 @@ void monitor_init(CharDriverState *chr, int flags) > } > > mon = g_malloc0(sizeof(*mon)); > + mon->outbuf = qstring_new(); > > mon->chr = chr; > mon->flags = flags; > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic 2013-03-25 21:07 ` Paolo Bonzini @ 2013-03-25 21:44 ` Luiz Capitulino 0 siblings, 0 replies; 12+ messages in thread From: Luiz Capitulino @ 2013-03-25 21:44 UTC (permalink / raw) To: Paolo Bonzini; +Cc: fred.konrad, qemu-devel, kraxel On Mon, 25 Mar 2013 22:07:36 +0100 Paolo Bonzini <pbonzini@redhat.com> wrote: > Il 25/03/2013 20:40, Luiz Capitulino ha scritto: > > Commit f628926bb423fa8a7e0b114511400ea9df38b76a changed monitor_flush() > > to retry on qemu_chr_fe_write() errors. However, the Monitor's output > > buffer can keep growing while the retry is not issued and this can > > cause the buffer to overflow. > > > > To reproduce this issue, just start qemu and type on the Monitor: > > > > (qemu) ? > > > > This will cause the assertion to trig. > > > > To fix this problem this commit makes the Monitor buffer dynamic, > > which means that it can grow as much as needed. > > What about using a GString instead? Makes no difference for me (although I don't enjoy mixing QString and GString in the same module), but if we do this with the goal of stopping QString proliferation then we should note somewhere that it shouldn't be used in new code anymore. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic 2013-03-25 19:40 ` [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic Luiz Capitulino 2013-03-25 21:07 ` Paolo Bonzini @ 2013-03-27 6:45 ` Wenchao Xia 2013-03-27 12:27 ` Luiz Capitulino 2013-04-02 9:18 ` Gerd Hoffmann 1 sibling, 2 replies; 12+ messages in thread From: Wenchao Xia @ 2013-03-27 6:45 UTC (permalink / raw) To: Luiz Capitulino; +Cc: fred.konrad, qemu-devel, kraxel Hi, Luiz Personally I hope reduce the dynamic allocated buffer which brings fragments and unexpected memory grow. Instead, how about sacrifice some time to wait output complete, since monitor is not time critical? in this case static buffer's size can decide how many work can be postponded. Following is my suggestion: --- a/monitor.c +++ b/monitor.c @@ -293,17 +293,28 @@ static void monitor_puts(Monitor *mon, const char *str) { char c; + /* if mux do not put in any thing to buffer */ + if (mon->mux_out) { + return; + } + for(;;) { - assert(mon->outbuf_index < sizeof(mon->outbuf) - 1); + if (mon->outbuf_index >= sizeof(mon->outbuf) - 1) { + /* when buffer is full, flush it and retry. If buffer is bigger, more + work can be postponed. */ + monitor_flush(mon); + usleep(1); + continue; + } c = *str++; if (c == '\0') break; if (c == '\n') mon->outbuf[mon->outbuf_index++] = '\r'; mon->outbuf[mon->outbuf_index++] = c; - if (mon->outbuf_index >= (sizeof(mon->outbuf) - 1) - || c == '\n') + if (c == '\n') { monitor_flush(mon); + } } } > Commit f628926bb423fa8a7e0b114511400ea9df38b76a changed monitor_flush() > to retry on qemu_chr_fe_write() errors. However, the Monitor's output > buffer can keep growing while the retry is not issued and this can > cause the buffer to overflow. > > To reproduce this issue, just start qemu and type on the Monitor: > > (qemu) ? > > This will cause the assertion to trig. > > To fix this problem this commit makes the Monitor buffer dynamic, > which means that it can grow as much as needed. > > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> > --- > monitor.c | 42 +++++++++++++++++++++++++----------------- > 1 file changed, 25 insertions(+), 17 deletions(-) > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic 2013-03-27 6:45 ` Wenchao Xia @ 2013-03-27 12:27 ` Luiz Capitulino 2013-03-28 3:21 ` Wenchao Xia 2013-04-02 9:18 ` Gerd Hoffmann 1 sibling, 1 reply; 12+ messages in thread From: Luiz Capitulino @ 2013-03-27 12:27 UTC (permalink / raw) To: Wenchao Xia; +Cc: fred.konrad, qemu-devel, kraxel On Wed, 27 Mar 2013 14:45:53 +0800 Wenchao Xia <xiawenc@linux.vnet.ibm.com> wrote: > Hi, Luiz > Personally I hope reduce the dynamic allocated buffer which brings > fragments and unexpected memory grow. Instead, how about sacrifice > some time to wait output complete, since monitor is not time critical? > in this case static buffer's size can decide how many work can be > postponded. Following is my suggestion: I'd be fine with it if you find a good way to postpone work, but sleeping on random timers is not a good way. Also, QEMU allocates fragments of memory in so many places that I doubt this is an valid argument. You're right that long QMP output can cause unexpected memory growth, but I expected this to be really rare and if this really bother us, we can ask monitor_puts() to flush at, say, every 4096 bytes. > > --- a/monitor.c > +++ b/monitor.c > @@ -293,17 +293,28 @@ static void monitor_puts(Monitor *mon, const char > *str) > { > char c; > > + /* if mux do not put in any thing to buffer */ > + if (mon->mux_out) { > + return; > + } > + > for(;;) { > - assert(mon->outbuf_index < sizeof(mon->outbuf) - 1); > + if (mon->outbuf_index >= sizeof(mon->outbuf) - 1) { > + /* when buffer is full, flush it and retry. If buffer is > bigger, more > + work can be postponed. */ > + monitor_flush(mon); > + usleep(1); > + continue; > + } > c = *str++; > if (c == '\0') > break; > if (c == '\n') > mon->outbuf[mon->outbuf_index++] = '\r'; > mon->outbuf[mon->outbuf_index++] = c; > - if (mon->outbuf_index >= (sizeof(mon->outbuf) - 1) > - || c == '\n') > + if (c == '\n') { > monitor_flush(mon); > + } > } > } > > > Commit f628926bb423fa8a7e0b114511400ea9df38b76a changed monitor_flush() > > to retry on qemu_chr_fe_write() errors. However, the Monitor's output > > buffer can keep growing while the retry is not issued and this can > > cause the buffer to overflow. > > > > To reproduce this issue, just start qemu and type on the Monitor: > > > > (qemu) ? > > > > This will cause the assertion to trig. > > > > To fix this problem this commit makes the Monitor buffer dynamic, > > which means that it can grow as much as needed. > > > > Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> > > --- > > monitor.c | 42 +++++++++++++++++++++++++----------------- > > 1 file changed, 25 insertions(+), 17 deletions(-) > > > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic 2013-03-27 12:27 ` Luiz Capitulino @ 2013-03-28 3:21 ` Wenchao Xia 0 siblings, 0 replies; 12+ messages in thread From: Wenchao Xia @ 2013-03-28 3:21 UTC (permalink / raw) To: Luiz Capitulino; +Cc: kraxel, qemu-devel, fred.konrad 于 2013-3-27 20:27, Luiz Capitulino 写道: > On Wed, 27 Mar 2013 14:45:53 +0800 > Wenchao Xia <xiawenc@linux.vnet.ibm.com> wrote: > >> Hi, Luiz >> Personally I hope reduce the dynamic allocated buffer which brings >> fragments and unexpected memory grow. Instead, how about sacrifice >> some time to wait output complete, since monitor is not time critical? >> in this case static buffer's size can decide how many work can be >> postponded. Following is my suggestion: > > I'd be fine with it if you find a good way to postpone work, but sleeping > on random timers is not a good way. > > Also, QEMU allocates fragments of memory in so many places that I doubt > this is an valid argument. You're right that long QMP output can cause > unexpected memory growth, but I expected this to be really rare and if > this really bother us, we can ask monitor_puts() to flush at, say, every > 4096 bytes. > If the point was async call, you are right it should not sleep. My thoughts is reducing dynamic allocation when possible. Two alternative are: make static buffer larger as 4096 bytes and drop the remain if buffer is full, or dynamic increase the buffer size with a fixed number when buffer overflow avoiding dynamic allocation every time. It is a bit overkill, so I am also OK with your solution. >> >> --- a/monitor.c >> +++ b/monitor.c >> @@ -293,17 +293,28 @@ static void monitor_puts(Monitor *mon, const char >> *str) >> { >> char c; >> >> + /* if mux do not put in any thing to buffer */ >> + if (mon->mux_out) { >> + return; >> + } >> + >> for(;;) { >> - assert(mon->outbuf_index < sizeof(mon->outbuf) - 1); >> + if (mon->outbuf_index >= sizeof(mon->outbuf) - 1) { >> + /* when buffer is full, flush it and retry. If buffer is >> bigger, more >> + work can be postponed. */ >> + monitor_flush(mon); >> + usleep(1); >> + continue; >> + } >> c = *str++; >> if (c == '\0') >> break; >> if (c == '\n') >> mon->outbuf[mon->outbuf_index++] = '\r'; >> mon->outbuf[mon->outbuf_index++] = c; >> - if (mon->outbuf_index >= (sizeof(mon->outbuf) - 1) >> - || c == '\n') >> + if (c == '\n') { >> monitor_flush(mon); >> + } >> } >> } >> >>> Commit f628926bb423fa8a7e0b114511400ea9df38b76a changed monitor_flush() >>> to retry on qemu_chr_fe_write() errors. However, the Monitor's output >>> buffer can keep growing while the retry is not issued and this can >>> cause the buffer to overflow. >>> >>> To reproduce this issue, just start qemu and type on the Monitor: >>> >>> (qemu) ? >>> >>> This will cause the assertion to trig. >>> >>> To fix this problem this commit makes the Monitor buffer dynamic, >>> which means that it can grow as much as needed. >>> >>> Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com> >>> --- >>> monitor.c | 42 +++++++++++++++++++++++++----------------- >>> 1 file changed, 25 insertions(+), 17 deletions(-) >>> >> > > -- Best Regards Wenchao Xia ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic 2013-03-27 6:45 ` Wenchao Xia 2013-03-27 12:27 ` Luiz Capitulino @ 2013-04-02 9:18 ` Gerd Hoffmann 1 sibling, 0 replies; 12+ messages in thread From: Gerd Hoffmann @ 2013-04-02 9:18 UTC (permalink / raw) To: Wenchao Xia; +Cc: fred.konrad, qemu-devel, Luiz Capitulino On 03/27/13 07:45, Wenchao Xia wrote: > Hi, Luiz > Personally I hope reduce the dynamic allocated buffer which brings > fragments and unexpected memory grow. It's a tradeoff. We can reduce the dynamic allocation, by simply reusing the qstring instead of allocation a new one (after complete/partial write). At the cost of wasting some memory. Given that qstrings grow expotentially I think the dynamic allocation overhead isn't that bad. > Instead, how about sacrifice > some time to wait output complete, since monitor is not time critical? The watch added recently will fire when there is space for writing. Right now it will write out the next chunk from the buffer. In theory we could change the monitor code to continue generating the response when there is more output space, so we don't have to buffer it in the first place. But I think the complexity simply isn't worth it for the memory savings we can gain. cheers, Gerd ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 0/2] Monitor: make output buffer dynamic 2013-03-25 19:40 [Qemu-devel] [PATCH 0/2] Monitor: make output buffer dynamic Luiz Capitulino 2013-03-25 19:40 ` [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() Luiz Capitulino 2013-03-25 19:40 ` [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic Luiz Capitulino @ 2013-04-02 9:19 ` Gerd Hoffmann 2 siblings, 0 replies; 12+ messages in thread From: Gerd Hoffmann @ 2013-04-02 9:19 UTC (permalink / raw) To: Luiz Capitulino; +Cc: qemu-devel, fred.konrad On 03/25/13 20:40, Luiz Capitulino wrote: > This series fixes an easy to reproduce assertion in the Monitor code. The > second patch contains all the relevant details. > > Gerd, I'd like a reviewed-by from you before merging this. Series looks good to me. Reviewed-by: Gerd Hoffmann <kraxel@redhat.com> cheers, Gerd ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2013-04-09 12:38 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-03-25 19:40 [Qemu-devel] [PATCH 0/2] Monitor: make output buffer dynamic Luiz Capitulino 2013-03-25 19:40 ` [Qemu-devel] [PATCH 1/2] qstring: add qobject_get_length() Luiz Capitulino 2013-04-09 3:02 ` Hu Tao 2013-04-09 12:38 ` Luiz Capitulino 2013-03-25 19:40 ` [Qemu-devel] [PATCH 2/2] Monitor: Make output buffer dynamic Luiz Capitulino 2013-03-25 21:07 ` Paolo Bonzini 2013-03-25 21:44 ` Luiz Capitulino 2013-03-27 6:45 ` Wenchao Xia 2013-03-27 12:27 ` Luiz Capitulino 2013-03-28 3:21 ` Wenchao Xia 2013-04-02 9:18 ` Gerd Hoffmann 2013-04-02 9:19 ` [Qemu-devel] [PATCH 0/2] Monitor: make " Gerd Hoffmann
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.