development APIs for used/free blocks information

development APIs for used/free blocks information

[Benjamin Wirth]

[-- Attachment #1.1: Type: text/plain, Size: 874 bytes --]

Hello,

I was looking for development APIs for XFS. The contents of xfsprogs-devel seem to only contain a few IOCTL commands, mostly file/inode based operations.

I need to query free/used blocks of an entire XFS filesystem and was hoping to find some user-space APIs for this. I have done the same task for EXT filesystem which seemed fairly easy as it provided simple open/query/close APIs. I know that XFS is a bit different as it manages free extents in two B+ trees per AG, but is this free/used block/extent information externalized somehow for developers to use? Am I not looking in the right place or these functionalities are simply not provided to application developers?

Any help/pointers would be greatly appreciated.

Regards,

Benjamin Wirth | Senior Software Developer
WinMagic Inc.
5600A Cancross Court
Mississauga, ON  |  L5R 3E9 |  Canada


[-- Attachment #1.2: Type: text/html, Size: 3649 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: development APIs for used/free blocks information

[Eric Sandeen]

On 4/26/16 3:51 PM, Benjamin Wirth wrote:
> Hello,
> 
> I was looking for development APIs for XFS. The contents of
> xfsprogs-devel seem to only contain a few IOCTL commands, mostly
> file/inode based operations.
> 
> I need to query free/used blocks of an entire XFS filesystem and was
> hoping to find some user-space APIs for this. I have done the same
> task for EXT filesystem which seemed fairly easy as it provided
> simple open/query/close APIs. I know that XFS is a bit different as
> it manages free extents in two B+ trees per AG, but is this free/used
> block/extent information externalized somehow for developers to use?
> Am I not looking in the right place or these functionalities are
> simply not provided to application developers?
> 
> Any help/pointers would be greatly appreciated.

It's not exported for application developers, no; it's not
generally something applications need, AFAIK.

You could look at how the freesp_f function works in xfs_db, I suppose.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: development APIs for used/free blocks information

[Dave Chinner]

On Tue, Apr 26, 2016 at 08:51:10PM +0000, Benjamin Wirth wrote:
> Hello,
> 
> I was looking for development APIs for XFS. The contents of
> xfsprogs-devel seem to only contain a few IOCTL commands, mostly
> file/inode based operations.
> 
> I need to query free/used blocks of an entire XFS filesystem and
> was hoping to find some user-space APIs for this.

$ man 2 statvfs

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: development APIs for used/free blocks information

[Eric Sandeen]

On 4/26/16 5:21 PM, Dave Chinner wrote:
> On Tue, Apr 26, 2016 at 08:51:10PM +0000, Benjamin Wirth wrote:
>> Hello,
>>
>> I was looking for development APIs for XFS. The contents of
>> xfsprogs-devel seem to only contain a few IOCTL commands, mostly
>> file/inode based operations.
>>
>> I need to query free/used blocks of an entire XFS filesystem and
>> was hoping to find some user-space APIs for this.
> 
> $ man 2 statvfs

I was assuming that he was interested in the actual used/free block
locations, not just the total count.  But maybe I was wrong!

-Eric

> Cheers,
> 
> Dave.
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: development APIs for used/free blocks information

[Dave Chinner]

On Tue, Apr 26, 2016 at 05:51:34PM -0500, Eric Sandeen wrote:
> 
> 
> On 4/26/16 5:21 PM, Dave Chinner wrote:
> > On Tue, Apr 26, 2016 at 08:51:10PM +0000, Benjamin Wirth wrote:
> >> Hello,
> >>
> >> I was looking for development APIs for XFS. The contents of
> >> xfsprogs-devel seem to only contain a few IOCTL commands, mostly
> >> file/inode based operations.
> >>
> >> I need to query free/used blocks of an entire XFS filesystem and
> >> was hoping to find some user-space APIs for this.
> > 
> > $ man 2 statvfs
> 
> I was assuming that he was interested in the actual used/free block
> locations, not just the total count.  But maybe I was wrong!

Maybe I'm wrong, but asking for free/used blocks is ambiguous to say
the least. If he wants free/used blocks, then we just need to
forward port this kernel branch:

https://git.kernel.org/cgit/linux/kernel/git/dgc/linux-xfs.git/log/?h=fiemapfs

And this xfsprogs branch:

https://git.kernel.org/cgit/fs/xfs/xfsprogs-dev.git/log/?h=spaceman

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

RE: development APIs for used/free blocks information

[Benjamin Wirth]

Hello Dave,

Thanks for your response.

And I apologize, reading back my email I wasn't clear about what I was looking for.

I want to be able to query which blocks are free or used, not just the counts. In EXT I was able to use ext2fs_test_block_bitmap() for that. XFS has a BTree with the free extents ordered by block number which I was hoping to be able to access from user-space, but couldn't find any APIs for it (or the headers containing the structure definitions) in the xfsprogs-devel package. Am I trying to do something impossible here?

Thanks,
Ben

-----Original Message-----
From: Dave Chinner [mailto:david@fromorbit.com] 
Sent: Tuesday, April 26, 2016 6:21 PM
To: Benjamin Wirth
Cc: xfs@oss.sgi.com
Subject: Re: development APIs for used/free blocks information

On Tue, Apr 26, 2016 at 08:51:10PM +0000, Benjamin Wirth wrote:
> Hello,
> 
> I was looking for development APIs for XFS. The contents of 
> xfsprogs-devel seem to only contain a few IOCTL commands, mostly 
> file/inode based operations.
> 
> I need to query free/used blocks of an entire XFS filesystem and was 
> hoping to find some user-space APIs for this.

$ man 2 statvfs

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: development APIs for used/free blocks information

[Dave Chinner]

On Wed, Apr 27, 2016 at 12:54:24PM +0000, Benjamin Wirth wrote:
> Hello Dave,
> 
> Thanks for your response.
> 
> And I apologize, reading back my email I wasn't clear about what I
> was looking for.
> 
> I want to be able to query which blocks are free or used, not just
> the counts. In EXT I was able to use ext2fs_test_block_bitmap()
> for that. XFS has a BTree with the free extents ordered by block
> number which I was hoping to be able to access from user-space,
> but couldn't find any APIs for it (or the headers containing the
> structure definitions) in the xfsprogs-devel package. Am I trying
> to do something impossible here?

There are prototypes that need a bit of polishing to complete:

http://oss.sgi.com/archives/xfs/2016-04/msg00633.html

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

RE: development APIs for used/free blocks information

[Benjamin Wirth]

Hi David,

I proceeded to parse the on-disk free blocks BTree for finding the free blocks information. Is there a way I could force a flush of BTree information to disk, so that in an online filesystem that is being changed I can also handle the changes in the BTree?

I was looking at XFS_FSOP_GOING_FLAGS_LOGFLUSH flag but it might do more then I need to do and actually shutdown the filesystem. Any other options? I don't mind about performance degradation so interested in any option.

Thanks,
Ben

-----Original Message-----
From: Dave Chinner [mailto:david@fromorbit.com] 
Sent: Wednesday, April 27, 2016 7:41 PM
To: Benjamin Wirth
Cc: xfs@oss.sgi.com
Subject: Re: development APIs for used/free blocks information

On Wed, Apr 27, 2016 at 12:54:24PM +0000, Benjamin Wirth wrote:
> Hello Dave,
> 
> Thanks for your response.
> 
> And I apologize, reading back my email I wasn't clear about what I was 
> looking for.
> 
> I want to be able to query which blocks are free or used, not just the 
> counts. In EXT I was able to use ext2fs_test_block_bitmap() for that. 
> XFS has a BTree with the free extents ordered by block number which I 
> was hoping to be able to access from user-space, but couldn't find any 
> APIs for it (or the headers containing the structure definitions) in 
> the xfsprogs-devel package. Am I trying to do something impossible 
> here?

There are prototypes that need a bit of polishing to complete:

http://oss.sgi.com/archives/xfs/2016-04/msg00633.html

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs