From: Jia Zhu <zhujia.zj@bytedance.com> To: linux-erofs@lists.ozlabs.org, xiang@kernel.org, chao@kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, yinxin.x@bytedance.com, jefflexu@linux.alibaba.com, huyue2@coolpad.com, Jia Zhu <zhujia.zj@bytedance.com> Subject: [PATCH V1 0/5] Introduce erofs shared domain Date: Fri, 2 Sep 2022 11:47:43 +0800 [thread overview] Message-ID: <20220902034748.60868-1-zhujia.zj@bytedance.com> (raw) Changes since RFC: 1. Fix the macro CONFIG_EROFS_FS_ONDEMAND misuse in erofs_fc_parse_param(). 2. Move the code in domain.c to fscache.c 3. Remove the definition of anon_inode_fs_type and just use erofs filesystem type to init pseudo mnt. 4. Use mutex lock for erofs_domain_list. 5. Unregister the cookies before kill_sb() to avoid inode busy when umount erofs. 6. Remove useless declaration of domain_get/put in internal.h [Kernel Patchset] =============== Git tree: https://github.com/userzj/linux.git zhujia/shared-domain-v1 Git web: https://github.com/userzj/linux/tree/zhujia/shared-domain-v1 [Background] ============ In ondemand read mode, we use individual volume to present an erofs mountpoint, cookies to present bootstrap and data blobs. In which case, since cookies can't be shared between fscache volumes, even if the data blobs between different mountpoints are exactly same, they can't be shared. [Introduction] ============== Here we introduce erofs shared domain to resolve above mentioned case. Several erofs filesystems can belong to one domain, and data blobs can be shared among these erofs filesystems of same domain. [Usage] Users could specify 'domain_id' mount option to create or join into a domain which reuses the same cookies(blobs). [Design] ======== 1. Use pseudo mnt to manage domain's lifecycle. 2. Use a linked list to maintain & traverse domains. 3. Use pseudo sb to create anonymous inode for recording cookie's info and manage cookies lifecycle. [Flow Path] =========== 1. User specify a new 'domain_id' in mount option. 1.1 Traverse domain list, compare domain_id with existing domain.[Miss] 1.2 Create a new domain(volume), add it to domain list. 1.3 Traverse pseudo sb's inode list, compare cookie name with existing cookies.[Miss] 1.4 Alloc new anonymous inodes and cookies. 2. User specify an existing 'domain_id' in mount option and the data blob is existed in domain. 2.1 Traverse domain list, compare domain_id with existing domain.[Hit] 2.2 Reuse the domain and increase its refcnt. 2.3 Traverse pseudo sb's inode list, compare cookie name with existing cookies.[Hit] 2.4 Reuse the cookie and increase its refcnt. [Test] ====== Git web: https://github.com/userzj/demand-read-cachefilesd/tree/shared-domain More test cases will be added to: https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=experimental-tests-fscache RFC: https://lore.kernel.org/all/YxAlO%2FDHDrIAafR2@B-P7TQMD6M-0146.local/ Jia Zhu (5): erofs: add 'domain_id' mount option for on-demand read sementics erofs: introduce fscache-based domain erofs: add 'domain_id' prefix when register sysfs erofs: remove duplicated unregister_cookie erofs: support fscache based shared domain fs/erofs/fscache.c | 169 +++++++++++++++++++++++++++++++++++++++++++- fs/erofs/internal.h | 33 ++++++++- fs/erofs/super.c | 94 ++++++++++++++++++------ fs/erofs/sysfs.c | 11 ++- 4 files changed, 281 insertions(+), 26 deletions(-) -- 2.20.1
WARNING: multiple messages have this Message-ID (diff)
From: Jia Zhu <zhujia.zj@bytedance.com> To: linux-erofs@lists.ozlabs.org, xiang@kernel.org, chao@kernel.org Cc: linux-kernel@vger.kernel.org, huyue2@coolpad.com, linux-fsdevel@vger.kernel.org, yinxin.x@bytedance.com Subject: [PATCH V1 0/5] Introduce erofs shared domain Date: Fri, 2 Sep 2022 11:47:43 +0800 [thread overview] Message-ID: <20220902034748.60868-1-zhujia.zj@bytedance.com> (raw) Changes since RFC: 1. Fix the macro CONFIG_EROFS_FS_ONDEMAND misuse in erofs_fc_parse_param(). 2. Move the code in domain.c to fscache.c 3. Remove the definition of anon_inode_fs_type and just use erofs filesystem type to init pseudo mnt. 4. Use mutex lock for erofs_domain_list. 5. Unregister the cookies before kill_sb() to avoid inode busy when umount erofs. 6. Remove useless declaration of domain_get/put in internal.h [Kernel Patchset] =============== Git tree: https://github.com/userzj/linux.git zhujia/shared-domain-v1 Git web: https://github.com/userzj/linux/tree/zhujia/shared-domain-v1 [Background] ============ In ondemand read mode, we use individual volume to present an erofs mountpoint, cookies to present bootstrap and data blobs. In which case, since cookies can't be shared between fscache volumes, even if the data blobs between different mountpoints are exactly same, they can't be shared. [Introduction] ============== Here we introduce erofs shared domain to resolve above mentioned case. Several erofs filesystems can belong to one domain, and data blobs can be shared among these erofs filesystems of same domain. [Usage] Users could specify 'domain_id' mount option to create or join into a domain which reuses the same cookies(blobs). [Design] ======== 1. Use pseudo mnt to manage domain's lifecycle. 2. Use a linked list to maintain & traverse domains. 3. Use pseudo sb to create anonymous inode for recording cookie's info and manage cookies lifecycle. [Flow Path] =========== 1. User specify a new 'domain_id' in mount option. 1.1 Traverse domain list, compare domain_id with existing domain.[Miss] 1.2 Create a new domain(volume), add it to domain list. 1.3 Traverse pseudo sb's inode list, compare cookie name with existing cookies.[Miss] 1.4 Alloc new anonymous inodes and cookies. 2. User specify an existing 'domain_id' in mount option and the data blob is existed in domain. 2.1 Traverse domain list, compare domain_id with existing domain.[Hit] 2.2 Reuse the domain and increase its refcnt. 2.3 Traverse pseudo sb's inode list, compare cookie name with existing cookies.[Hit] 2.4 Reuse the cookie and increase its refcnt. [Test] ====== Git web: https://github.com/userzj/demand-read-cachefilesd/tree/shared-domain More test cases will be added to: https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=experimental-tests-fscache RFC: https://lore.kernel.org/all/YxAlO%2FDHDrIAafR2@B-P7TQMD6M-0146.local/ Jia Zhu (5): erofs: add 'domain_id' mount option for on-demand read sementics erofs: introduce fscache-based domain erofs: add 'domain_id' prefix when register sysfs erofs: remove duplicated unregister_cookie erofs: support fscache based shared domain fs/erofs/fscache.c | 169 +++++++++++++++++++++++++++++++++++++++++++- fs/erofs/internal.h | 33 ++++++++- fs/erofs/super.c | 94 ++++++++++++++++++------ fs/erofs/sysfs.c | 11 ++- 4 files changed, 281 insertions(+), 26 deletions(-) -- 2.20.1
next reply other threads:[~2022-09-02 3:48 UTC|newest] Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-09-02 3:47 Jia Zhu [this message] 2022-09-02 3:47 ` [PATCH V1 0/5] Introduce erofs shared domain Jia Zhu 2022-09-02 3:47 ` [PATCH V1 1/5] erofs: add 'domain_id' mount option for on-demand read sementics Jia Zhu 2022-09-02 3:47 ` Jia Zhu 2022-09-02 3:47 ` [PATCH V1 2/5] erofs: introduce fscache-based domain Jia Zhu 2022-09-02 3:47 ` Jia Zhu 2022-09-02 3:47 ` [PATCH V1 3/5] erofs: add 'domain_id' prefix when register sysfs Jia Zhu 2022-09-02 3:47 ` Jia Zhu 2022-09-02 3:47 ` [PATCH V1 4/5] erofs: remove duplicated unregister_cookie Jia Zhu 2022-09-02 3:47 ` Jia Zhu 2022-09-02 3:47 ` [PATCH V1 5/5] erofs: support fscache based shared domain Jia Zhu 2022-09-02 3:47 ` Jia Zhu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220902034748.60868-1-zhujia.zj@bytedance.com \ --to=zhujia.zj@bytedance.com \ --cc=chao@kernel.org \ --cc=huyue2@coolpad.com \ --cc=jefflexu@linux.alibaba.com \ --cc=linux-erofs@lists.ozlabs.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=xiang@kernel.org \ --cc=yinxin.x@bytedance.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.