Backporting the concurrent direct IO write fix to 3.4

Backporting the concurrent direct IO write fix to 3.4

[Kerin Millar]

Hi,

I recently became aware of commit 507630b2 (use shared ilock mode for
direct IO writes by default). I understand that this fixes a regression
which can have a notable impact upon MySQL performance. This is of
considerable interest to myself because my MySQL servers have to contend
with a really tough workload.

The patch applies cleanly to 3.4 stable but is it actually safe to use
it there? Or does it depend on other changes to XFS, such that I'd be
better off holding out for 3.5? Any advice would be greatly appreciated.

Cheers,

--Kerin

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Backporting the concurrent direct IO write fix to 3.4

[Christoph Hellwig]

On Tue, Jul 03, 2012 at 02:31:11PM +0100, Kerin Millar wrote:
> Hi,
> 
> I recently became aware of commit 507630b2 (use shared ilock mode for
> direct IO writes by default). I understand that this fixes a regression
> which can have a notable impact upon MySQL performance. This is of
> considerable interest to myself because my MySQL servers have to contend
> with a really tough workload.
> 
> The patch applies cleanly to 3.4 stable but is it actually safe to use
> it there? Or does it depend on other changes to XFS, such that I'd be
> better off holding out for 3.5? Any advice would be greatly appreciated.

You want the whole series starting at b4d05e3019692fc5a8c573fbce60de2d48c5b7a1
without the earlier patches there are potential problems when using
quotas.

On the one hand I'd love to see these patches go into 3.4-stable as it's
going to be a long term release, on the other hand they are fairly
invasive.  If you can give a headsup after passing xfstests and heavy
mysql testing including benchmarks it might make sense to consider them.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Backporting the concurrent direct IO write fix to 3.4

[Dave Chinner]

On Tue, Jul 03, 2012 at 12:09:49PM -0400, Christoph Hellwig wrote:
> On Tue, Jul 03, 2012 at 02:31:11PM +0100, Kerin Millar wrote:
> > Hi,
> > 
> > I recently became aware of commit 507630b2 (use shared ilock mode for
> > direct IO writes by default). I understand that this fixes a regression
> > which can have a notable impact upon MySQL performance. This is of
> > considerable interest to myself because my MySQL servers have to contend
> > with a really tough workload.
> > 
> > The patch applies cleanly to 3.4 stable but is it actually safe to use
> > it there? Or does it depend on other changes to XFS, such that I'd be
> > better off holding out for 3.5? Any advice would be greatly appreciated.
> 
> You want the whole series starting at b4d05e3019692fc5a8c573fbce60de2d48c5b7a1
> without the earlier patches there are potential problems when using
> quotas.
> 
> On the one hand I'd love to see these patches go into 3.4-stable as it's
> going to be a long term release,

Is it? So we will have long term support kernels for 3.0.x, 3.2.x
and now 3.4.x? 3.0.x is the long term stable kernel I'm targetting
for backports and there's no way I can really handle more than
that...

> on the other hand they are fairly
> invasive.  If you can give a headsup after passing xfstests and heavy
> mysql testing including benchmarks it might make sense to consider them.

I've got to port them back to 3.0, so I'm not sure it makes
sense to port them to 3.4.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Backporting the concurrent direct IO write fix to 3.4

[Igor M Podlesny]

On 4 July 2012 07:11, Dave Chinner <david@fromorbit.com> wrote:
[...]
>> On the one hand I'd love to see these patches go into 3.4-stable as it's
>> going to be a long term release,
>
> Is it? So we will have long term support kernels for 3.0.x, 3.2.x
> and now 3.4.x? 3.0.x is the long term stable kernel I'm targetting
> for backports and there's no way I can really handle more than
> that...

   That's quite a pity! In production there's often even 2.6.32 since
it's chosen as RHEL6's kernel.

--

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Backporting the concurrent direct IO write fix to 3.4

[Christoph Hellwig]

On Wed, Jul 04, 2012 at 09:11:22AM +1000, Dave Chinner wrote:
> Is it? So we will have long term support kernels for 3.0.x, 3.2.x
> and now 3.4.x? 3.0.x is the long term stable kernel I'm targetting
> for backports and there's no way I can really handle more than
> that...

I remember Greg announced 3.4 to be long term in addition to 3.0
recently.  3.2 only is the Debian release kernel, for which the
maintainers offered to maintain it as a formal stable release
on kernel.org

And yes, it's getting not only confusing but also way to much work
for subsystem maintainers.

I'm still happy to backport fixes to 3.0, but for more invasive work
it's getting a bit too old now.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Backporting the concurrent direct IO write fix to 3.4

[Dave Chinner]

On Wed, Jul 04, 2012 at 12:09:06PM +0800, Igor M Podlesny wrote:
> On 4 July 2012 07:11, Dave Chinner <david@fromorbit.com> wrote:
> [...]
> >> On the one hand I'd love to see these patches go into 3.4-stable as it's
> >> going to be a long term release,
> >
> > Is it? So we will have long term support kernels for 3.0.x, 3.2.x
> > and now 3.4.x? 3.0.x is the long term stable kernel I'm targetting
> > for backports and there's no way I can really handle more than
> > that...
> 
>    That's quite a pity! In production there's often even 2.6.32 since
> it's chosen as RHEL6's kernel.

RHEL6 is using a 2.6.32 kernel only in name. The XFS code in RHEL6
is only a few commits short of 3.0 in terms of functionality (i.e.
RHEL 6.2 has full delaylog functionality), and most other supported
subsystems and drivers are similarly advanced well past 2.6.32.
Hence I'm looking at 3.0.x for long term stable maintenance of XFS,
not 2.6.32.x...

On Wed, Jul 04, 2012 at 01:54:19AM -0400, Christoph Hellwig wrote:
> On Wed, Jul 04, 2012 at 09:11:22AM +1000, Dave Chinner wrote:
> > Is it? So we will have long term support kernels for 3.0.x, 3.2.x
> > and now 3.4.x? 3.0.x is the long term stable kernel I'm targetting
> > for backports and there's no way I can really handle more than
> > that...
> 
> I remember Greg announced 3.4 to be long term in addition to 3.0
> recently.

I missed that announcement - lost in the noise of lkml I guess...

> 3.2 only is the Debian release kernel, for which the
> maintainers offered to maintain it as a formal stable release
> on kernel.org

I know, but we're already seeing bug reports from people using that
kernel...

> And yes, it's getting not only confusing but also way to much work
> for subsystem maintainers.
> 
> I'm still happy to backport fixes to 3.0, but for more invasive work
> it's getting a bit too old now.

As RHEL6 and 3.0.x XFS code bases are almost the same I effectively
have to maintain a 3.0 tree and backport the necessary (invasive,
even) fixes to it.  It is comparitively little extra work for me to
test and push fixes to both, and I'll get much earlier and wider
deployment of the backports by pushing them to 3.0.x as well. It's a
no-brainer AFAIC.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs