Does Ext4 support parallel write similar to XFS?

Does Ext4 support parallel write similar to XFS?

[Amit Sahrawat]

In XFS we can write parallel (i.,e we can make use of allocation
groups for writing process). If the files are kept in individual
directories, there is a possibility that first the blocks for that
files be used from individual allocation groups. If I start ‘4’
writing process(cp 100MB_file /<dirnum>/) – after writing is finished
– if I check the bmap – it does shows that initial allocation was from
individual allocation groups.
Even though in Ext4 also we do have groups – but I am not able to get
behavior similar to XFS.
If I check the file extents – the extents are in mixed form, the
allocation pattern is also very fragmented.

Please share more on this. Also, if there is a possible exact test
case to check for parallel writes support.

Thanks & Regards,
Amit Sahrawat

Re: Does Ext4 support parallel write similar to XFS?

[Eric Sandeen]

On 1/24/12 11:19 PM, Amit Sahrawat wrote:
> In XFS we can write parallel (i.,e we can make use of allocation
> groups for writing process). If the files are kept in individual
> directories, there is a possibility that first the blocks for that
> files be used from individual allocation groups. If I start ‘4’
> writing process(cp 100MB_file /<dirnum>/) – after writing is finished
> – if I check the bmap – it does shows that initial allocation was from
> individual allocation groups.
> Even though in Ext4 also we do have groups – but I am not able to get
> behavior similar to XFS.
> If I check the file extents – the extents are in mixed form, the
> allocation pattern is also very fragmented.
> 
> Please share more on this. Also, if there is a possible exact test
> case to check for parallel writes support.

It seems that you are asking more about allocation policy than parallelism
in general?  With either filesystem, you could use preallocation to wind
up with more contiguous files when you write them in parallel, though
that requires some idea of the file size ahead of time.

ext4 doesn't have that exact dir::group heuristic that xfs uses,
but it does have other mechanisms and heuristics to try to get good
file and directory layout.

In general, ext4 tries to put new root directories into new groups, see
comments above find_group_orlov().  Other directories tend to stay near
their parent directory.  So it's really the roots of dir trees that
get spread across the disk in general.  New non-dir inodes also tend to
stay close to their parent.  (I think I have that all right ...)

The test you describe above does result in more contiguous allocation on
xfs than on ext4, though - a quick check on kernel 3.2 yielded 2-3
extents per file on ext4, 1 extent per file for xfs.

-Eric

> Thanks & Regards,
> Amit Sahrawat
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Does Ext4 support parallel write similar to XFS?

[Andreas Dilger]

On 2012-01-25, at 9:34, Eric Sandeen <sandeen@redhat.com> wrote:
> On 1/24/12 11:19 PM, Amit Sahrawat wrote:
>> In XFS we can write parallel (i.,e we can make use of allocation
>> groups for writing process). If the files are kept in individual
>> directories, there is a possibility that first the blocks for that
>> files be used from individual allocation groups. If I start ‘4’
>> writing process(cp 100MB_file /<dirnum>/) – after writing is finished
>> – if I check the bmap – it does shows that initial allocation was from
>> individual allocation groups.
>> Even though in Ext4 also we do have groups – but I am not able to get
>> behavior similar to XFS.
>> If I check the file extents – the extents are in mixed form, the
>> allocation pattern is also very fragmented.
>> 
>> Please share more on this. Also, if there is a possible exact test
>> case to check for parallel writes support.
> 
> It seems that you are asking more about allocation policy than parallelism
> in general?  With either filesystem, you could use preallocation to wind
> up with more contiguous files when you write them in parallel, though
> that requires some idea of the file size ahead of time.
> 
> ext4 doesn't have that exact dir::group heuristic that xfs uses,
> but it does have other mechanisms and heuristics to try to get good
> file and directory layout.
> 
> In general, ext4 tries to put new root directories into new groups, see
> comments above find_group_orlov().  Other directories tend to stay near
> their parent directory.  So it's really the roots of dir trees that
> get spread across the disk in general.  New non-dir inodes also tend to
> stay close to their parent.  (I think I have that all right ...)

Note that this policy of new subdirectory placement (and indirectly the placement of files created therein) can be tuned by userspace by setting the "topdir" flag via setattr on the parent directory. 

This will push new subdirs created in that dir to a new group, which would In turn allow the files to allocate from that group. It is intended to be set on directories like /home where files within the subdirs are unrelated.

Whether that is enough for your usage is unclear. 

> The test you describe above does result in more contiguous allocation on
> xfs than on ext4, though - a quick check on kernel 3.2 yielded 2-3
> extents per file on ext4, 1 extent per file for xfs.
> 
> -Eric
> 
>> Thanks & Regards,
>> Amit Sahrawat
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Does Ext4 support parallel write similar to XFS?

[Dave Chinner]

On Wed, Jan 25, 2012 at 10:34:54AM -0600, Eric Sandeen wrote:
> On 1/24/12 11:19 PM, Amit Sahrawat wrote:
> > In XFS we can write parallel (i.,e we can make use of allocation
> > groups for writing process). If the files are kept in individual
> > directories, there is a possibility that first the blocks for that
> > files be used from individual allocation groups. If I start ‘4’
> > writing process(cp 100MB_file /<dirnum>/) – after writing is finished
> > – if I check the bmap – it does shows that initial allocation was from
> > individual allocation groups.
> > Even though in Ext4 also we do have groups – but I am not able to get
> > behavior similar to XFS.
> > If I check the file extents – the extents are in mixed form, the
> > allocation pattern is also very fragmented.
> > 
> > Please share more on this. Also, if there is a possible exact test
> > case to check for parallel writes support.
> 
> It seems that you are asking more about allocation policy than parallelism
> in general?  With either filesystem, you could use preallocation to wind
> up with more contiguous files when you write them in parallel, though
> that requires some idea of the file size ahead of time.
> 
> ext4 doesn't have that exact dir::group heuristic that xfs uses,
> but it does have other mechanisms and heuristics to try to get good
> file and directory layout.

XFs has different allocation policies according to mount options
used.  inode32 (default for > 1TB), inode64 (default for <1TB) and
filestreams. Each will give you different layouts for the same test
depending on the size of your filesystem and the amount of free
space you have available in it.

If XFS does what you want, then use it. There is no good reason for
trying to make ext4 do everything XFS does because it simply can't.
Especially when it comes to allocation strategies and policies...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com