All of lore.kernel.org
 help / color / mirror / Atom feed
From: <jiang.biao2@zte.com.cn>
To: <peterx@redhat.com>
Cc: kvm@vger.kernel.org, mst@redhat.com, mtosatti@redhat.com,
	xiaoguangrong@tencent.com, qemu-devel@nongnu.org,
	guangrong.xiao@gmail.com, pbonzini@redhat.com
Subject: Re: [PATCH 3/8] migration: support todetectcompression and decompression errors
Date: Wed, 28 Mar 2018 12:08:19 +0800 (CST)	[thread overview]
Message-ID: <201803281208196040484@zte.com.cn> (raw)
In-Reply-To: <20180328030351.GU17789@xz-mi>

> 
> On Tue, Mar 27, 2018 at 10:35:29PM +0800, Xiao Guangrong wrote:
>
>> > > No, we can't make the assumption that "error _must_ be caused by page update".
>> > > No document/ABI about compress/decompress promised it. :)
>
> Indeed, I found no good documents about below errors that jiang.biao
> pointed out.
Hi, Peter
The description about the errors comes from here,
http://www.zlib.net/manual.html
And about the error codes returned by inflate(), they are described as,
** inflate() returns 
Z_OK if some progress has been made (more input processed or more output produced),
Z_STREAM_END if the end of the compressed data has been reached and all uncompressed output has been produced, 
Z_NEED_DICT if a preset dictionary is needed at this point, 
Z_DATA_ERROR if the input data was corrupted (input stream not conforming to the zlib format or incorrect check value, in which case strm->msg points to a string with a more specific error), 
Z_STREAM_ERROR if the stream structure was inconsistent (for example next_in or next_out was Z_NULL, or the state was inadvertently written over by the application), 
Z_MEM_ERROR if there was not enough memory, 
Z_BUF_ERROR if no progress was possible or if there was not enough room in the output buffer when Z_FINISH is used. ... 
**
According to the above description, the error caused by page update looks 
more like tend to return Z_DATA_ERROR, but I do not have env to verify that. :)
As I understand it, the real compress/decompress error cases other than that 
caused by page update should be rare, maybe the error code is enough to
distinguish those if we can verify the the error codes returned by page update
and other silent failures by test. If so, we can cut the cost of memcpy.  
If not, I agree with Guangrong's idea too. I never read the zlib code and all my
information comes from the manual, so if anything inaccurate, pls ignore my
option. :)

Regards,
Jiang

WARNING: multiple messages have this Message-ID (diff)
From: <jiang.biao2@zte.com.cn>
To: peterx@redhat.com
Cc: guangrong.xiao@gmail.com, kvm@vger.kernel.org, mst@redhat.com,
	mtosatti@redhat.com, xiaoguangrong@tencent.com,
	qemu-devel@nongnu.org, pbonzini@redhat.com
Subject: Re: [Qemu-devel] [PATCH 3/8] migration: support todetectcompression and decompression errors
Date: Wed, 28 Mar 2018 12:08:19 +0800 (CST)	[thread overview]
Message-ID: <201803281208196040484@zte.com.cn> (raw)
In-Reply-To: <20180328030351.GU17789@xz-mi>

> 
> On Tue, Mar 27, 2018 at 10:35:29PM +0800, Xiao Guangrong wrote:
>
>> > > No, we can't make the assumption that "error _must_ be caused by page update".
>> > > No document/ABI about compress/decompress promised it. :)
>
> Indeed, I found no good documents about below errors that jiang.biao
> pointed out.
Hi, Peter
The description about the errors comes from here,
http://www.zlib.net/manual.html
And about the error codes returned by inflate(), they are described as,
** inflate() returns 
Z_OK if some progress has been made (more input processed or more output produced),
Z_STREAM_END if the end of the compressed data has been reached and all uncompressed output has been produced, 
Z_NEED_DICT if a preset dictionary is needed at this point, 
Z_DATA_ERROR if the input data was corrupted (input stream not conforming to the zlib format or incorrect check value, in which case strm->msg points to a string with a more specific error), 
Z_STREAM_ERROR if the stream structure was inconsistent (for example next_in or next_out was Z_NULL, or the state was inadvertently written over by the application), 
Z_MEM_ERROR if there was not enough memory, 
Z_BUF_ERROR if no progress was possible or if there was not enough room in the output buffer when Z_FINISH is used. ... 
**
According to the above description, the error caused by page update looks 
more like tend to return Z_DATA_ERROR, but I do not have env to verify that. :)
As I understand it, the real compress/decompress error cases other than that 
caused by page update should be rare, maybe the error code is enough to
distinguish those if we can verify the the error codes returned by page update
and other silent failures by test. If so, we can cut the cost of memcpy.  
If not, I agree with Guangrong's idea too. I never read the zlib code and all my
information comes from the manual, so if anything inaccurate, pls ignore my
option. :)

Regards,
Jiang

  reply	other threads:[~2018-03-28  4:08 UTC|newest]

Thread overview: 126+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-03-13  7:57 [PATCH 0/8] migration: improve and cleanup compression guangrong.xiao
2018-03-13  7:57 ` [Qemu-devel] " guangrong.xiao
2018-03-13  7:57 ` [PATCH 1/8] migration: stop compressing page in migration thread guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 10:25   ` Dr. David Alan Gilbert
2018-03-15 10:25     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-16  8:05     ` Xiao Guangrong
2018-03-16  8:05       ` [Qemu-devel] " Xiao Guangrong
2018-03-19 12:11       ` Dr. David Alan Gilbert
2018-03-19 12:11         ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-21  8:19       ` Peter Xu
2018-03-21  8:19         ` [Qemu-devel] " Peter Xu
2018-03-22 11:38         ` Xiao Guangrong
2018-03-22 11:38           ` [Qemu-devel] " Xiao Guangrong
2018-03-26  9:02           ` Peter Xu
2018-03-26  9:02             ` [Qemu-devel] " Peter Xu
2018-03-26 15:43             ` Xiao Guangrong
2018-03-26 15:43               ` [Qemu-devel] " Xiao Guangrong
2018-03-27  7:33               ` Peter Xu
2018-03-27  7:33                 ` [Qemu-devel] " Peter Xu
2018-03-27 19:12               ` Dr. David Alan Gilbert
2018-03-27 19:12                 ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-28  3:01   ` Wang, Wei W
2018-03-28  3:01     ` [Qemu-devel] " Wang, Wei W
2018-03-27 15:24     ` Xiao Guangrong
2018-03-27 15:24       ` [Qemu-devel] " Xiao Guangrong
2018-03-28  7:30       ` Wei Wang
2018-03-28  7:30         ` [Qemu-devel] " Wei Wang
2018-03-28  7:37         ` Peter Xu
2018-03-28  7:37           ` [Qemu-devel] " Peter Xu
2018-03-28  8:30           ` Wei Wang
2018-03-28  8:30             ` [Qemu-devel] " Wei Wang
2018-03-13  7:57 ` [PATCH 2/8] migration: stop allocating and freeing memory frequently guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 11:03   ` Dr. David Alan Gilbert
2018-03-15 11:03     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-16  8:19     ` Xiao Guangrong
2018-03-16  8:19       ` [Qemu-devel] " Xiao Guangrong
2018-03-19 10:54       ` Dr. David Alan Gilbert
2018-03-19 10:54         ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-19 12:11         ` Xiao Guangrong
2018-03-19 12:11           ` [Qemu-devel] " Xiao Guangrong
2018-03-19  1:49   ` [PATCH 2/8] migration: stop allocating and freeingmemory frequently jiang.biao2
2018-03-19  1:49     ` [Qemu-devel] " jiang.biao2
2018-03-19  4:03     ` Xiao Guangrong
2018-03-19  4:03       ` [Qemu-devel] " Xiao Guangrong
2018-03-19  4:48       ` [PATCH 2/8] migration: stop allocating andfreeingmemory frequently jiang.biao2
2018-03-19  4:48         ` [Qemu-devel] " jiang.biao2
2018-03-21  9:06   ` [PATCH 2/8] migration: stop allocating and freeing memory frequently Peter Xu
2018-03-21  9:06     ` [Qemu-devel] " Peter Xu
2018-03-22 11:57     ` Xiao Guangrong
2018-03-22 11:57       ` [Qemu-devel] " Xiao Guangrong
2018-03-27  7:07       ` Peter Xu
2018-03-27  7:07         ` [Qemu-devel] " Peter Xu
2018-03-13  7:57 ` [PATCH 3/8] migration: support to detect compression and decompression errors guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 11:29   ` Dr. David Alan Gilbert
2018-03-15 11:29     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-16  8:25     ` Xiao Guangrong
2018-03-16  8:25       ` [Qemu-devel] " Xiao Guangrong
2018-03-19  7:56   ` [PATCH 3/8] migration: support to detect compressionand " jiang.biao2
2018-03-19  7:56     ` [Qemu-devel] " jiang.biao2
2018-03-19  8:01     ` Xiao Guangrong
2018-03-19  8:01       ` [Qemu-devel] " Xiao Guangrong
2018-03-21 10:00   ` [PATCH 3/8] migration: support to detect compression and " Peter Xu
2018-03-21 10:00     ` [Qemu-devel] " Peter Xu
2018-03-22 12:03     ` Xiao Guangrong
2018-03-22 12:03       ` [Qemu-devel] " Xiao Guangrong
2018-03-27  7:22       ` Peter Xu
2018-03-27  7:22         ` [Qemu-devel] " Peter Xu
2018-03-26 19:42         ` Xiao Guangrong
2018-03-26 19:42           ` [Qemu-devel] " Xiao Guangrong
2018-03-27 11:17           ` Peter Xu
2018-03-27 11:17             ` [Qemu-devel] " Peter Xu
2018-03-27  1:20             ` Xiao Guangrong
2018-03-27  1:20               ` [Qemu-devel] " Xiao Guangrong
2018-03-28  0:43               ` [PATCH 3/8] migration: support to detectcompression " jiang.biao2
2018-03-28  0:43                 ` [Qemu-devel] " jiang.biao2
2018-03-27 14:35                 ` Xiao Guangrong
2018-03-27 14:35                   ` [Qemu-devel] " Xiao Guangrong
2018-03-28  3:03                   ` Peter Xu
2018-03-28  3:03                     ` [Qemu-devel] " Peter Xu
2018-03-28  4:08                     ` jiang.biao2 [this message]
2018-03-28  4:08                       ` [Qemu-devel] [PATCH 3/8] migration: support todetectcompression " jiang.biao2
2018-03-28  4:20                       ` Peter Xu
2018-03-28  4:20                         ` [Qemu-devel] " Peter Xu
2018-03-27 18:44                         ` Xiao Guangrong
2018-03-27 18:44                           ` [Qemu-devel] " Xiao Guangrong
2018-03-28  8:07                           ` [PATCH 3/8] migration: support todetectcompressionand " jiang.biao2
2018-03-28  8:07                             ` [Qemu-devel] " jiang.biao2
2018-03-13  7:57 ` [PATCH 4/8] migration: introduce control_save_page() guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 11:37   ` Dr. David Alan Gilbert
2018-03-15 11:37     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-16  8:52     ` Xiao Guangrong
2018-03-16  8:52       ` [Qemu-devel] " Xiao Guangrong
2018-03-27  7:47     ` Peter Xu
2018-03-27  7:47       ` [Qemu-devel] " Peter Xu
2018-03-13  7:57 ` [PATCH 5/8] migration: move calling control_save_page to the common place guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 11:47   ` Dr. David Alan Gilbert
2018-03-15 11:47     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-16  8:59     ` Xiao Guangrong
2018-03-16  8:59       ` [Qemu-devel] " Xiao Guangrong
2018-03-19 13:15       ` Dr. David Alan Gilbert
2018-03-19 13:15         ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-27 12:35   ` Peter Xu
2018-03-27 12:35     ` [Qemu-devel] " Peter Xu
2018-03-13  7:57 ` [PATCH 6/8] migration: move calling save_zero_page " guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 12:27   ` Dr. David Alan Gilbert
2018-03-15 12:27     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-27 12:49   ` Peter Xu
2018-03-27 12:49     ` [Qemu-devel] " Peter Xu
2018-03-13  7:57 ` [PATCH 7/8] migration: introduce save_normal_page() guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 12:30   ` Dr. David Alan Gilbert
2018-03-15 12:30     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-27 12:54   ` Peter Xu
2018-03-27 12:54     ` [Qemu-devel] " Peter Xu
2018-03-13  7:57 ` [PATCH 8/8] migration: remove ram_save_compressed_page() guangrong.xiao
2018-03-13  7:57   ` [Qemu-devel] " guangrong.xiao
2018-03-15 12:32   ` Dr. David Alan Gilbert
2018-03-15 12:32     ` [Qemu-devel] " Dr. David Alan Gilbert
2018-03-27 12:56   ` Peter Xu
2018-03-27 12:56     ` [Qemu-devel] " Peter Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=201803281208196040484@zte.com.cn \
    --to=jiang.biao2@zte.com.cn \
    --cc=guangrong.xiao@gmail.com \
    --cc=kvm@vger.kernel.org \
    --cc=mst@redhat.com \
    --cc=mtosatti@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=peterx@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=xiaoguangrong@tencent.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.