RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled

RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled

[MaoXiaoyun]

[-- Attachment #1.1: Type: text/plain, Size: 1845 bytes --]


I think the 2.6.32.36 ext4 needs to backport this patch much likely.
Otherwise it will hit the problem I met.
 
http://patchwork.ozlabs.org/patch/79880/

> 
> Message: 3
> Date: Wed, 13 Jul 2011 20:33:41 +0800
> From: MaoXiaoyun <tinnycloud@hotmail.com>
> Subject: [Xen-devel] Window VM hit blue screen when dom0 uses ext4
> with extent enabled
> To: xen devel <xen-devel@lists.xensource.com>
> Message-ID: <BLU157-w24C4DDC1B2F561B849C853DA470@phx.gbl>
> Content-Type: text/plain; charset="gb2312"
> 
> 
> Hi:
> 
> We met quite strange blue screen problem when recently shift our dom0 fs from ext3 to
> ext4. We have some IO stress test, that is in a Win2003 HVM, a process perform file reading
> and writing after the VM startup. In our dom0 host, we start totally 10 HVMS to run the test.
> 
> Here is the test result
> 1) if it is ext3 in dom0, test is ok
> 2) if it is ext4 in dom0, entent feature is disabled, test is ok.
> 3) if it is ext4 in dom0, and extent feature is enable, HVMs will got blue screen one after another
> in 15 -30 minutes. And the blue screen code implys something wrong with the disk.
> (Such as KERNEL_STACK_INPAGE_ERROR, 
> STOP 0x00000077(0x00000185,0x00000185,0x00000000,0x001FD000))
> 
> Also serial port has some log report:
> (XEN) grant_table.c:578:d0 Iomem mapping not permitted ffffffffffffffff (domain 40)
> (XEN) grant_table.c:578:d0 Iomem mapping not permitted ffffffffffffffff (domain 40)
> 
> When blue screen, I don't see any abnormal log in messages. It's surprise me since
> wi/wo extent make such big difference. We've been run VMs in ext3 quite a long time 
> with no failure, I also learnt that extent is a important feature in ext4, couldn't be wrong 
> so easily. 
> 
> So what's problem could it be ?
> Any comments?
> Thanks.
> 
> BTW: we have kernel 2.6.32.36 + xen 4.0.1
> 
> 
 		 	   		  

[-- Attachment #1.2: Type: text/html, Size: 2913 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled

[Ian Campbell]

On Thu, 2011-07-14 at 06:44 +0100, MaoXiaoyun wrote:
> I think the 2.6.32.36 ext4 needs to backport this patch much likely.
> Otherwise it will hit the problem I met.
>  
> http://patchwork.ozlabs.org/patch/79880/

That version is still in state NEW but something appears to have been
committed upstream as e9e3bcecf44c04b9e6b505fd8e2eb9cea58fb94d:

    ext4: serialize unaligned asynchronous DIO
    
    ext4 has a data corruption case when doing non-block-aligned
    asynchronous direct IO into a sparse file, as demonstrated
    by xfstest 240.
    
    ...

Seems like a reasonable enough thing to backport to me (for what that's
worth). Although:
                            It is also quite a lot slower
    (14 min for package installs, vs. 8 min for well-aligned)
    but I'll take slow correctness over fast corruption any day.
    
    Mingming suggested that we can track outstanding
    conversions, and wait on those so that non-sparse
    files won't be affected, and I've implemented that here;
    unaligned AIO to nonsparse files won't take a perf hit.

Something to bear in mind if you are deploying anything based on sparse
files on ext4.

Ian.

RE: RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled

[MaoXiaoyun]

[-- Attachment #1.1: Type: text/plain, Size: 1792 bytes --]



 

> Subject: Re: [Xen-devel] RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled
> From: Ian.Campbell@citrix.com
> To: tinnycloud@hotmail.com
> CC: xen-devel@lists.xensource.com; jeremy@goop.org; konrad.wilk@oracle.com
> Date: Thu, 14 Jul 2011 09:02:06 +0100
> 
> On Thu, 2011-07-14 at 06:44 +0100, MaoXiaoyun wrote:
> > I think the 2.6.32.36 ext4 needs to backport this patch much likely.
> > Otherwise it will hit the problem I met.
> > 
> > http://patchwork.ozlabs.org/patch/79880/
> 
> That version is still in state NEW but something appears to have been
> committed upstream as e9e3bcecf44c04b9e6b505fd8e2eb9cea58fb94d:
> 
> ext4: serialize unaligned asynchronous DIO
> 
> ext4 has a data corruption case when doing non-block-aligned
> asynchronous direct IO into a sparse file, as demonstrated
> by xfstest 240.
> 
> ...
> 
> Seems like a reasonable enough thing to backport to me (for what that's
> worth). Although:
> It is also quite a lot slower
> (14 min for package installs, vs. 8 min for well-aligned)
> but I'll take slow correctness over fast corruption any day.
> 
> Mingming suggested that we can track outstanding
> conversions, and wait on those so that non-sparse
> files won't be affected, and I've implemented that here;
> unaligned AIO to nonsparse files won't take a perf hit.
> 
> Something to bear in mind if you are deploying anything based on sparse
> files on ext4.
> 
That's right. Since we use VHD as our base image. 
We are trying to backport this patch, but isn't easy for me.
Meanwhile, there are quite a lot ext4 patches in upstream, I'm afried 
some of them are also needed for stable ext4, well, not sure.
 
Could someone kindly backport e9e3bcecf44c04b9e6b505fd8e2eb9cea58fb94d
for me? 
 
Many thanks.

> Ian.
> 
> 
> 
 		 	   		  

[-- Attachment #1.2: Type: text/html, Size: 2445 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled

[Ian Campbell]

On Thu, 2011-07-14 at 09:41 +0100, MaoXiaoyun wrote:
> 
>  
> > Subject: Re: [Xen-devel] RE: Window VM hit blue screen when dom0
> uses ext4 with extent enabled
> > From: Ian.Campbell@citrix.com
> > To: tinnycloud@hotmail.com
> > CC: xen-devel@lists.xensource.com; jeremy@goop.org;
> konrad.wilk@oracle.com
> > Date: Thu, 14 Jul 2011 09:02:06 +0100
> > 
> > On Thu, 2011-07-14 at 06:44 +0100, MaoXiaoyun wrote:
> > > I think the 2.6.32.36 ext4 needs to backport this patch much
> likely.
> > > Otherwise it will hit the problem I met.
> > > 
> > > http://patchwork.ozlabs.org/patch/79880/
> > 
> > That version is still in state NEW but something appears to have
> been
> > committed upstream as e9e3bcecf44c04b9e6b505fd8e2eb9cea58fb94d:
> > 
> > ext4: serialize unaligned asynchronous DIO
> > 
> > ext4 has a data corruption case when doing non-block-aligned
> > asynchronous direct IO into a sparse file, as demonstrated
> > by xfstest 240.
> > 
> > ...
> > 
> > Seems like a reasonable enough thing to backport to me (for what
> that's
> > worth). Although:
> > It is also quite a lot slower
> > (14 min for package installs, vs. 8 min for well-aligned)
> > but I'll take slow correctness over fast corruption any day.
> > 
> > Mingming suggested that we can track outstanding
> > conversions, and wait on those so that non-sparse
> > files won't be affected, and I've implemented that here;
> > unaligned AIO to nonsparse files won't take a perf hit.
> > 
> > Something to bear in mind if you are deploying anything based on
> sparse
> > files on ext4.
> > 
> That's right. Since we use VHD as our base image. 
> We are trying to backport this patch, but isn't easy for me.
> Meanwhile, there are quite a lot ext4 patches in upstream, I'm afried 
> some of them are also needed for stable ext4, well, not sure.
>  
> Could someone kindly backport e9e3bcecf44c04b9e6b505fd8e2eb9cea58fb94d
> for me? 

I think would be a good idea to ping the ext4 developers about this and
suggest that this might be a candidate for an upstream stable backport.
If not then it would be good to know why not instead of blindly taking
it into our stable tree...

Ian.

>  
> Many thanks.
> 
> > Ian.
> > 
> > 
> > 
> 

RE: RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled

[MaoXiaoyun]

[-- Attachment #1.1: Type: text/plain, Size: 2746 bytes --]


Just a reminder 
Below patch is also needed.
 
http://thread.gmane.org/gmane.comp.file-systems.ext4/19659
 

> Subject: RE: [Xen-devel] RE: Window VM hit blue screen when dom0 uses ext4 with extent enabled
> From: Ian.Campbell@eu.citrix.com
> To: tinnycloud@hotmail.com
> CC: xen-devel@lists.xensource.com; jeremy@goop.org; konrad.wilk@oracle.com
> Date: Thu, 14 Jul 2011 10:14:22 +0100
> 
> On Thu, 2011-07-14 at 09:41 +0100, MaoXiaoyun wrote:
> > 
> > 
> > > Subject: Re: [Xen-devel] RE: Window VM hit blue screen when dom0
> > uses ext4 with extent enabled
> > > From: Ian.Campbell@citrix.com
> > > To: tinnycloud@hotmail.com
> > > CC: xen-devel@lists.xensource.com; jeremy@goop.org;
> > konrad.wilk@oracle.com
> > > Date: Thu, 14 Jul 2011 09:02:06 +0100
> > > 
> > > On Thu, 2011-07-14 at 06:44 +0100, MaoXiaoyun wrote:
> > > > I think the 2.6.32.36 ext4 needs to backport this patch much
> > likely.
> > > > Otherwise it will hit the problem I met.
> > > > 
> > > > http://patchwork.ozlabs.org/patch/79880/
> > > 
> > > That version is still in state NEW but something appears to have
> > been
> > > committed upstream as e9e3bcecf44c04b9e6b505fd8e2eb9cea58fb94d:
> > > 
> > > ext4: serialize unaligned asynchronous DIO
> > > 
> > > ext4 has a data corruption case when doing non-block-aligned
> > > asynchronous direct IO into a sparse file, as demonstrated
> > > by xfstest 240.
> > > 
> > > ...
> > > 
> > > Seems like a reasonable enough thing to backport to me (for what
> > that's
> > > worth). Although:
> > > It is also quite a lot slower
> > > (14 min for package installs, vs. 8 min for well-aligned)
> > > but I'll take slow correctness over fast corruption any day.
> > > 
> > > Mingming suggested that we can track outstanding
> > > conversions, and wait on those so that non-sparse
> > > files won't be affected, and I've implemented that here;
> > > unaligned AIO to nonsparse files won't take a perf hit.
> > > 
> > > Something to bear in mind if you are deploying anything based on
> > sparse
> > > files on ext4.
> > > 
> > That's right. Since we use VHD as our base image. 
> > We are trying to backport this patch, but isn't easy for me.
> > Meanwhile, there are quite a lot ext4 patches in upstream, I'm afried 
> > some of them are also needed for stable ext4, well, not sure.
> > 
> > Could someone kindly backport e9e3bcecf44c04b9e6b505fd8e2eb9cea58fb94d
> > for me? 
> 
> I think would be a good idea to ping the ext4 developers about this and
> suggest that this might be a candidate for an upstream stable backport.
> If not then it would be good to know why not instead of blindly taking
> it into our stable tree...
> 
> Ian.
> 
> > 
> > Many thanks.
> > 
> > > Ian.
> > > 
> > > 
> > > 
> > 
> 
> 
 		 	   		  

[-- Attachment #1.2: Type: text/html, Size: 3848 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel