* Rust library for BTRFS
@ 2016-10-15 14:31 James Pharaoh
2016-10-15 18:41 ` Hans van Kranenburg
0 siblings, 1 reply; 3+ messages in thread
From: James Pharaoh @ 2016-10-15 14:31 UTC (permalink / raw)
To: linux-btrfs
Hi all,
I have created a BTRFS library for Rust - very incomplete so far, but
possibly of interest to someone.
https://gitlab.wellbehavedsoftware.com/well-behaved-software/rust-btrfs
I've basically reimplemented what I find in other tools because, as far
as I can tell, there is no C library for BTRFS at this point.
The currently supported functions are:
- Deduplication
- File system info
- Space info
James
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Rust library for BTRFS
2016-10-15 14:31 Rust library for BTRFS James Pharaoh
@ 2016-10-15 18:41 ` Hans van Kranenburg
2016-10-19 23:46 ` James Pharaoh
0 siblings, 1 reply; 3+ messages in thread
From: Hans van Kranenburg @ 2016-10-15 18:41 UTC (permalink / raw)
To: James Pharaoh, linux-btrfs
On 10/15/2016 04:31 PM, James Pharaoh wrote:
>
> I have created a BTRFS library for Rust - very incomplete so far, but
> possibly of interest to someone.
Nice work, I'm doing the same for python. Just picked up working on it
again, after summer: https://github.com/knorrie/python-btrfs
> https://gitlab.wellbehavedsoftware.com/well-behaved-software/rust-btrfs
>
> I've basically reimplemented what I find in other tools because, as far
> as I can tell, there is no C library for BTRFS at this point.
For code operating on an online filesystem, using the IOCTLs, you don't
really need a lot from btrfs-progs. You need to known about some structs
and throw them back and forth to the kernel. And not being afraid of
doing some reverse engineering helps. :o)
The btrfs-progs programs operate in two ways:
1. Using the IOCTLs, implementing a command/args input model and
converting the output back to text on the console.
2. Also directly doing low-level plumbing, reading and writing inside
unmounted btrfs filesystems. For this, there's a copy of a bunch of code
from the kernel, to be able to read/write a filesystem that is not
mounted from the running program, in a similar way that kernel does when
it's live. If you don't plan to write your own btrfs check --repair,
this is also not what you need to import as a library.
This might be interesting:
https://patchwork.kernel.org/patch/9356749/
But, I'm not the C expert here, this is what I know.
> The currently supported functions are:
>
> - Deduplication
> - File system info
> - Space info
Is there a specific use case you're building this for?
My primary trigger to start on it is best described as: "Tired of
parsing the output of btrfs <foo> <bar>? Try this!"
Also, if you're using IRC, #btrfs on freenode is a good place to hang out.
Have fun,
--
Hans van Kranenburg
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Rust library for BTRFS
2016-10-15 18:41 ` Hans van Kranenburg
@ 2016-10-19 23:46 ` James Pharaoh
0 siblings, 0 replies; 3+ messages in thread
From: James Pharaoh @ 2016-10-19 23:46 UTC (permalink / raw)
To: Hans van Kranenburg, linux-btrfs
On 15/10/16 19:41, Hans van Kranenburg wrote:
> On 10/15/2016 04:31 PM, James Pharaoh wrote:
>>
>> I've basically reimplemented what I find in other tools because, as far
>> as I can tell, there is no C library for BTRFS at this point.
>
> For code operating on an online filesystem, using the IOCTLs, you don't
> really need a lot from btrfs-progs. You need to known about some structs
> and throw them back and forth to the kernel. And not being afraid of
> doing some reverse engineering helps. :o)
Yeah I've managed fine so far. I feel like someone needs to split out
this library, really, but I'm not particularly interested into doing C
dev myself, except when it's necessary to interface with something.
> The btrfs-progs programs operate in two ways:
> 1. Using the IOCTLs, implementing a command/args input model and
> converting the output back to text on the console.
> 2. Also directly doing low-level plumbing, reading and writing inside
> unmounted btrfs filesystems. For this, there's a copy of a bunch of code
> from the kernel, to be able to read/write a filesystem that is not
> mounted from the running program, in a similar way that kernel does when
> it's live. If you don't plan to write your own btrfs check --repair,
> this is also not what you need to import as a library.
I have realised this as well. If I understand correctly, and this is one
of the things I find most interesting about BTRFS, is that it has a kind
of "microkernel" approach to managing it's on-disk data structures,
where a small, well-tested core of code manages some basic storage, and
advanced features are built on top of this.
As I understand it, the tools are able to simply read these data
structures directly from the disk, presumably after acquiring some kind
of "read lock" from the kernel, which is sufficient because I believe
BTFRS is COW at a low level, so with a read lock on the "committed root"
you are looking at, there is a guarantee all the data structures inside
it will be valid.
Any confirmation or comment on this would be appreciated, I am planning
to delve in further soon with my current efforts..
> This might be interesting:
> https://patchwork.kernel.org/patch/9356749/
Will take a look...
> But, I'm not the C expert here, this is what I know.
>
>> The currently supported functions are:
>>
>> - Deduplication
>> - File system info
>> - Space info
>
> Is there a specific use case you're building this for?
I am building a dense hosting platform, a kind of alternative-to-cloud
solution which acts/looks much more like traditional hosting, but with
immensely better backups and space efficiency.
I am also using zbackup as a way to store a LOT of similar builds of
entire containers (ie entire operating systems minus the kernel which is
shared), storing them efficiently, and deploying them efficiently.
> My primary trigger to start on it is best described as: "Tired of
> parsing the output of btrfs <foo> <bar>? Try this!"
>
> Also, if you're using IRC, #btrfs on freenode is a good place to hang out.
I have tried this but often don't get a response. I feel IRC is a bit
packed these days and people go for the easy answers and ignore the more
in depth ones...
Nevertheless, I've exchanged several emails with the author of ZBackup
and have a list of issues/feature requests/suggestions which I will be
adding to github and opening for discussion on the mailing list very soon.
> Have fun,
Always!
James
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-10-19 23:47 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-15 14:31 Rust library for BTRFS James Pharaoh
2016-10-15 18:41 ` Hans van Kranenburg
2016-10-19 23:46 ` James Pharaoh
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.