libblkid: Implementing basic Apple APFS support

libblkid: Implementing basic Apple APFS support

[Harry Mallon]

Hello all,

I am sort of floating around recently, messing with Fedora on a MacBook Pro=
 and trying to improve issues I find. Apple APFS has been the default on Ma=
cs for a couple of years now. I think it would be useful for users to be ab=
le to see/recognise their MacBook system partition in the various disk mana=
gement tools that depend on libblkid.

My initial patch was just adding enough to recognise partitions and mark th=
em as such. Support for recognition is in the upstream file utility so the =
magic values are known.

https://github.com/file/file/blob/a642587a9c9e2dd7feacdf513c3643ce26ad3c22/=
magic/Magdir/apple#L435

However, APFS is not really a filesystem type, but includes a volume manage=
r too. Perhaps recognition of the partition type is not complete enough for=
 upstream? I am looking at making an implementation in the partitions folde=
r rather than the superblocks folder (inspired by code in the reverse engin=
eered apfs-fuse driver) to enumerate and fill in the volumes listed but thi=
s is an order of magnitude more code to work with.

https://github.com/sgan81/apfs-fuse

Would a pure recognition-only apfs module be of interest?
Does anyone have any tips on writing a partition table reader version?
Would upstream be interested in this change at all given open source readin=
g/writing to the file system is likely a long way off (apfs-fuse exists, as=
 does a closed source Paragon implementation, both read only)?
Is libblkid the right place for this code or is there some other layer I sh=
ould be looking at?

Best,
Harry

Re: libblkid: Implementing basic Apple APFS support

[Karel Zak]

On Mon, Jun 18, 2018 at 09:52:43PM +0000, Harry Mallon wrote:
> However, APFS is not really a filesystem type, but includes a volume
> manager too. Perhaps recognition of the partition type is not
> complete enough for upstream?

Arbitrary Apple APFS detection will be improvement. So, detect magic
string and nothing else is good start.

> I am looking at making an
> implementation in the partitions folder rather than the superblocks
> folder (inspired by code in the reverse engineered apfs-fuse driver)
> to enumerate and fill in the volumes listed but this is an order of
> magnitude more code to work with.

I prefer libblkid/src/superblocks folder if we're not able to read
"slices" (partitions, root, etc) from the device. The same situation
is for example ZFS or BTRFS.

> Would a pure recognition-only apfs module be of interest?

Sure.

> Does anyone have any tips on writing a partition table reader version?

I have doubts about it as nothing about the internal logic is public
and it's crazy to provide write access or map internal APFS stuff to 
real Linux block devices if we're not sure how... sounds to dangerous.

> Is libblkid the right place for this code or is there some other layer I should be looking at?

libblkid is the right place for detection

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: libblkid: Implementing basic Apple APFS support

[Harry Mallon]

[-- Attachment #1: Type: text/plain, Size: 1798 bytes --]

Hi Karel,

Here is my basic 'recognition only' patch. I think you are right that anything else would be too complicated/liable for Apple to change something and break it.

Best,
Harry

On 19/06/2018, 16:23, "Karel Zak" <kzak@redhat.com> wrote:

    On Mon, Jun 18, 2018 at 09:52:43PM +0000, Harry Mallon wrote:
    > However, APFS is not really a filesystem type, but includes a volume
    > manager too. Perhaps recognition of the partition type is not
    > complete enough for upstream?
    
    Arbitrary Apple APFS detection will be improvement. So, detect magic
    string and nothing else is good start.
    
    > I am looking at making an
    > implementation in the partitions folder rather than the superblocks
    > folder (inspired by code in the reverse engineered apfs-fuse driver)
    > to enumerate and fill in the volumes listed but this is an order of
    > magnitude more code to work with.
    
    I prefer libblkid/src/superblocks folder if we're not able to read
    "slices" (partitions, root, etc) from the device. The same situation
    is for example ZFS or BTRFS.
    
    > Would a pure recognition-only apfs module be of interest?
    
    Sure.
    
    > Does anyone have any tips on writing a partition table reader version?
    
    I have doubts about it as nothing about the internal logic is public
    and it's crazy to provide write access or map internal APFS stuff to 
    real Linux block devices if we're not sure how... sounds to dangerous.
    
    > Is libblkid the right place for this code or is there some other layer I should be looking at?
    
    libblkid is the right place for detection
    
        Karel
    
    -- 
     Karel Zak  <kzak@redhat.com>
     http://karelzak.blogspot.com
    

[-- Attachment #2: 0001-libblkid-add-basic-Apple-APFS-recognition.patch --]
[-- Type: application/octet-stream, Size: 2462 bytes --]

From 494f227e291658987e04633a26ab4b5559e6ebe7 Mon Sep 17 00:00:00 2001
From: Harry Mallon <hjmallon@gmail.com>
Date: Sun, 17 Jun 2018 12:03:22 +0100
Subject: [PATCH] libblkid: add basic Apple APFS recognition

---
 libblkid/src/Makemodule.am             |  1 +
 libblkid/src/superblocks/apfs.c        | 21 +++++++++++++++++++++
 libblkid/src/superblocks/superblocks.c |  3 ++-
 libblkid/src/superblocks/superblocks.h |  1 +
 4 files changed, 25 insertions(+), 1 deletion(-)
 create mode 100644 libblkid/src/superblocks/apfs.c

diff --git a/libblkid/src/Makemodule.am b/libblkid/src/Makemodule.am
index ea0230702..4109561c4 100644
--- a/libblkid/src/Makemodule.am
+++ b/libblkid/src/Makemodule.am
@@ -44,6 +44,7 @@ libblkid_la_SOURCES = \
 	libblkid/src/partitions/unixware.c \
 	\
 	libblkid/src/superblocks/adaptec_raid.c \
+	libblkid/src/superblocks/apfs.c \
 	libblkid/src/superblocks/bcache.c \
 	libblkid/src/superblocks/befs.c \
 	libblkid/src/superblocks/bfs.c \
diff --git a/libblkid/src/superblocks/apfs.c b/libblkid/src/superblocks/apfs.c
new file mode 100644
index 000000000..5e8deb0a1
--- /dev/null
+++ b/libblkid/src/superblocks/apfs.c
@@ -0,0 +1,21 @@
+/*
+ * Copyright (C) 2018 Harry Mallon <hjmallon@gmail.com>
+ *
+ * This file may be redistributed under the terms of the
+ * GNU Lesser General Public License.
+ */
+
+#include "superblocks.h"
+
+
+const struct blkid_idinfo apfs_idinfo =
+{
+	.name		= "apfs",
+	.usage		= BLKID_USAGE_FILESYSTEM,
+	.magics		=
+	{
+		{ .magic = "NXSB", .len = 4, .sboff = 32 },
+		{ NULL }
+	}
+};
+
diff --git a/libblkid/src/superblocks/superblocks.c b/libblkid/src/superblocks/superblocks.c
index 6dfd2be64..e95c1e626 100644
--- a/libblkid/src/superblocks/superblocks.c
+++ b/libblkid/src/superblocks/superblocks.c
@@ -163,7 +163,8 @@ static const struct blkid_idinfo *idinfos[] =
 	&nilfs2_idinfo,
 	&exfat_idinfo,
 	&f2fs_idinfo,
-	&mpool_idinfo
+	&mpool_idinfo,
+	&apfs_idinfo
 };
 
 /*
diff --git a/libblkid/src/superblocks/superblocks.h b/libblkid/src/superblocks/superblocks.h
index d677f85bc..85858b9fc 100644
--- a/libblkid/src/superblocks/superblocks.h
+++ b/libblkid/src/superblocks/superblocks.h
@@ -82,6 +82,7 @@ extern const struct blkid_idinfo mpool_idinfo;
 extern const struct blkid_idinfo vdo_idinfo;
 extern const struct blkid_idinfo stratis_idinfo;
 extern const struct blkid_idinfo bitlocker_idinfo;
+extern const struct blkid_idinfo apfs_idinfo;
 
 /*
  * superblock functions
-- 
2.17.1