All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tejun Heo <tj@kernel.org>
To: axboe@kernel.dk
Cc: linux-kernel@vger.kernel.org, jack@suse.cz, hch@infradead.org,
	hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org,
	vgoyal@redhat.com, lizefan@huawei.com, cgroups@vger.kernel.org,
	linux-mm@kvack.org, mhocko@suse.cz, clm@fb.com,
	fengguang.wu@intel.com, david@fromorbit.com, gthelen@google.com,
	Tejun Heo <tj@kernel.org>
Subject: [PATCH 20/48] writeback: add {CONFIG|BDI_CAP|FS}_CGROUP_WRITEBACK
Date: Mon, 23 Mar 2015 00:54:31 -0400	[thread overview]
Message-ID: <1427086499-15657-21-git-send-email-tj@kernel.org> (raw)
In-Reply-To: <1427086499-15657-1-git-send-email-tj@kernel.org>

cgroup writeback requires support from both bdi and filesystem sides.
Add BDI_CAP_CGROUP_WRITEBACK and FS_CGROUP_WRITEBACK to indicate
support and enable BDI_CAP_CGROUP_WRITEBACK on block based bdi's by
default.  Also, define CONFIG_CGROUP_WRITEBACK which is enabled if
both MEMCG and BLK_CGROUP are enabled.

inode_cgwb_enabled() which determines whether a given inode's both bdi
and fs support cgroup writeback is added.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Jan Kara <jack@suse.cz>
---
 block/blk-core.c            |  2 +-
 include/linux/backing-dev.h | 32 +++++++++++++++++++++++++++++++-
 include/linux/fs.h          |  1 +
 init/Kconfig                |  5 +++++
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index fa1314e..c44018a 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -606,7 +606,7 @@ struct request_queue *blk_alloc_queue_node(gfp_t gfp_mask, int node_id)
 
 	q->backing_dev_info.ra_pages =
 			(VM_MAX_READAHEAD * 1024) / PAGE_CACHE_SIZE;
-	q->backing_dev_info.capabilities = 0;
+	q->backing_dev_info.capabilities = BDI_CAP_CGROUP_WRITEBACK;
 	q->backing_dev_info.name = "block";
 	q->node = node_id;
 
diff --git a/include/linux/backing-dev.h b/include/linux/backing-dev.h
index bfdaa18..6bb3123 100644
--- a/include/linux/backing-dev.h
+++ b/include/linux/backing-dev.h
@@ -134,12 +134,15 @@ int bdi_set_max_ratio(struct backing_dev_info *bdi, unsigned int max_ratio);
  * BDI_CAP_NO_WRITEBACK:   Don't write pages back
  * BDI_CAP_NO_ACCT_WB:     Don't automatically account writeback pages
  * BDI_CAP_STRICTLIMIT:    Keep number of dirty pages below bdi threshold.
+ *
+ * BDI_CAP_CGROUP_WRITEBACK: Supports cgroup-aware writeback.
  */
 #define BDI_CAP_NO_ACCT_DIRTY	0x00000001
 #define BDI_CAP_NO_WRITEBACK	0x00000002
 #define BDI_CAP_NO_ACCT_WB	0x00000004
 #define BDI_CAP_STABLE_WRITES	0x00000008
 #define BDI_CAP_STRICTLIMIT	0x00000010
+#define BDI_CAP_CGROUP_WRITEBACK 0x00000020
 
 #define BDI_CAP_NO_ACCT_AND_WRITEBACK \
 	(BDI_CAP_NO_WRITEBACK | BDI_CAP_NO_ACCT_DIRTY | BDI_CAP_NO_ACCT_WB)
@@ -229,4 +232,31 @@ static inline int bdi_sched_wait(void *word)
 	return 0;
 }
 
-#endif		/* _LINUX_BACKING_DEV_H */
+#ifdef CONFIG_CGROUP_WRITEBACK
+
+/**
+ * inode_cgwb_enabled - test whether cgroup writeback is enabled on an inode
+ * @inode: inode of interest
+ *
+ * cgroup writeback requires support from both the bdi and filesystem.
+ * Test whether @inode has both.
+ */
+static inline bool inode_cgwb_enabled(struct inode *inode)
+{
+	struct backing_dev_info *bdi = inode_to_bdi(inode);
+
+	return bdi_cap_account_dirty(bdi) &&
+		(bdi->capabilities & BDI_CAP_CGROUP_WRITEBACK) &&
+		(inode->i_sb->s_type->fs_flags & FS_CGROUP_WRITEBACK);
+}
+
+#else	/* CONFIG_CGROUP_WRITEBACK */
+
+static inline bool inode_cgwb_enabled(struct inode *inode)
+{
+	return false;
+}
+
+#endif	/* CONFIG_CGROUP_WRITEBACK */
+
+#endif	/* _LINUX_BACKING_DEV_H */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index ccf4b64..bc72737 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1862,6 +1862,7 @@ struct file_system_type {
 #define FS_HAS_SUBTYPE		4
 #define FS_USERNS_MOUNT		8	/* Can be mounted by userns root */
 #define FS_USERNS_DEV_MOUNT	16 /* A userns mount does not imply MNT_NODEV */
+#define FS_CGROUP_WRITEBACK	32	/* Supports cgroup-aware writeback */
 #define FS_RENAME_DOES_D_MOVE	32768	/* FS will handle d_move() during rename() internally. */
 	struct dentry *(*mount) (struct file_system_type *, int,
 		       const char *, void *);
diff --git a/init/Kconfig b/init/Kconfig
index f5dbc6d..9f17798 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1132,6 +1132,11 @@ config DEBUG_BLK_CGROUP
 	Enable some debugging help. Currently it exports additional stat
 	files in a cgroup which can be useful for debugging.
 
+config CGROUP_WRITEBACK
+	bool
+	depends on MEMCG && BLK_CGROUP
+	default y
+
 endif # CGROUPS
 
 config CHECKPOINT_RESTORE
-- 
2.1.0


WARNING: multiple messages have this Message-ID (diff)
From: Tejun Heo <tj@kernel.org>
To: axboe@kernel.dk
Cc: linux-kernel@vger.kernel.org, jack@suse.cz, hch@infradead.org,
	hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org,
	vgoyal@redhat.com, lizefan@huawei.com, cgroups@vger.kernel.org,
	linux-mm@kvack.org, mhocko@suse.cz, clm@fb.com,
	fengguang.wu@intel.com, david@fromorbit.com, gthelen@google.com,
	Tejun Heo <tj@kernel.org>
Subject: [PATCH 20/48] writeback: add {CONFIG|BDI_CAP|FS}_CGROUP_WRITEBACK
Date: Mon, 23 Mar 2015 00:54:31 -0400	[thread overview]
Message-ID: <1427086499-15657-21-git-send-email-tj@kernel.org> (raw)
In-Reply-To: <1427086499-15657-1-git-send-email-tj@kernel.org>

cgroup writeback requires support from both bdi and filesystem sides.
Add BDI_CAP_CGROUP_WRITEBACK and FS_CGROUP_WRITEBACK to indicate
support and enable BDI_CAP_CGROUP_WRITEBACK on block based bdi's by
default.  Also, define CONFIG_CGROUP_WRITEBACK which is enabled if
both MEMCG and BLK_CGROUP are enabled.

inode_cgwb_enabled() which determines whether a given inode's both bdi
and fs support cgroup writeback is added.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Jan Kara <jack@suse.cz>
---
 block/blk-core.c            |  2 +-
 include/linux/backing-dev.h | 32 +++++++++++++++++++++++++++++++-
 include/linux/fs.h          |  1 +
 init/Kconfig                |  5 +++++
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index fa1314e..c44018a 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -606,7 +606,7 @@ struct request_queue *blk_alloc_queue_node(gfp_t gfp_mask, int node_id)
 
 	q->backing_dev_info.ra_pages =
 			(VM_MAX_READAHEAD * 1024) / PAGE_CACHE_SIZE;
-	q->backing_dev_info.capabilities = 0;
+	q->backing_dev_info.capabilities = BDI_CAP_CGROUP_WRITEBACK;
 	q->backing_dev_info.name = "block";
 	q->node = node_id;
 
diff --git a/include/linux/backing-dev.h b/include/linux/backing-dev.h
index bfdaa18..6bb3123 100644
--- a/include/linux/backing-dev.h
+++ b/include/linux/backing-dev.h
@@ -134,12 +134,15 @@ int bdi_set_max_ratio(struct backing_dev_info *bdi, unsigned int max_ratio);
  * BDI_CAP_NO_WRITEBACK:   Don't write pages back
  * BDI_CAP_NO_ACCT_WB:     Don't automatically account writeback pages
  * BDI_CAP_STRICTLIMIT:    Keep number of dirty pages below bdi threshold.
+ *
+ * BDI_CAP_CGROUP_WRITEBACK: Supports cgroup-aware writeback.
  */
 #define BDI_CAP_NO_ACCT_DIRTY	0x00000001
 #define BDI_CAP_NO_WRITEBACK	0x00000002
 #define BDI_CAP_NO_ACCT_WB	0x00000004
 #define BDI_CAP_STABLE_WRITES	0x00000008
 #define BDI_CAP_STRICTLIMIT	0x00000010
+#define BDI_CAP_CGROUP_WRITEBACK 0x00000020
 
 #define BDI_CAP_NO_ACCT_AND_WRITEBACK \
 	(BDI_CAP_NO_WRITEBACK | BDI_CAP_NO_ACCT_DIRTY | BDI_CAP_NO_ACCT_WB)
@@ -229,4 +232,31 @@ static inline int bdi_sched_wait(void *word)
 	return 0;
 }
 
-#endif		/* _LINUX_BACKING_DEV_H */
+#ifdef CONFIG_CGROUP_WRITEBACK
+
+/**
+ * inode_cgwb_enabled - test whether cgroup writeback is enabled on an inode
+ * @inode: inode of interest
+ *
+ * cgroup writeback requires support from both the bdi and filesystem.
+ * Test whether @inode has both.
+ */
+static inline bool inode_cgwb_enabled(struct inode *inode)
+{
+	struct backing_dev_info *bdi = inode_to_bdi(inode);
+
+	return bdi_cap_account_dirty(bdi) &&
+		(bdi->capabilities & BDI_CAP_CGROUP_WRITEBACK) &&
+		(inode->i_sb->s_type->fs_flags & FS_CGROUP_WRITEBACK);
+}
+
+#else	/* CONFIG_CGROUP_WRITEBACK */
+
+static inline bool inode_cgwb_enabled(struct inode *inode)
+{
+	return false;
+}
+
+#endif	/* CONFIG_CGROUP_WRITEBACK */
+
+#endif	/* _LINUX_BACKING_DEV_H */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index ccf4b64..bc72737 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1862,6 +1862,7 @@ struct file_system_type {
 #define FS_HAS_SUBTYPE		4
 #define FS_USERNS_MOUNT		8	/* Can be mounted by userns root */
 #define FS_USERNS_DEV_MOUNT	16 /* A userns mount does not imply MNT_NODEV */
+#define FS_CGROUP_WRITEBACK	32	/* Supports cgroup-aware writeback */
 #define FS_RENAME_DOES_D_MOVE	32768	/* FS will handle d_move() during rename() internally. */
 	struct dentry *(*mount) (struct file_system_type *, int,
 		       const char *, void *);
diff --git a/init/Kconfig b/init/Kconfig
index f5dbc6d..9f17798 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1132,6 +1132,11 @@ config DEBUG_BLK_CGROUP
 	Enable some debugging help. Currently it exports additional stat
 	files in a cgroup which can be useful for debugging.
 
+config CGROUP_WRITEBACK
+	bool
+	depends on MEMCG && BLK_CGROUP
+	default y
+
 endif # CGROUPS
 
 config CHECKPOINT_RESTORE
-- 
2.1.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2015-03-23  5:15 UTC|newest]

Thread overview: 119+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-03-23  4:54 [PATCHSET 1/3 v2 block/for-4.1/core] writeback: cgroup writeback support Tejun Heo
2015-03-23  4:54 ` Tejun Heo
2015-03-23  4:54 ` [PATCH 01/48] memcg: add per cgroup dirty page accounting Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 02/48] blkcg: move block/blk-cgroup.h to include/linux/blk-cgroup.h Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 03/48] update !CONFIG_BLK_CGROUP dummies in include/linux/blk-cgroup.h Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 04/48] memcg: add mem_cgroup_root_css Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 05/48] blkcg: add blkcg_root_css Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 06/48] cgroup, block: implement task_get_css() and use it in bio_associate_current() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 07/48] blkcg: implement task_get_blkcg_css() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 08/48] blkcg: implement bio_associate_blkcg() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 09/48] memcg: implement mem_cgroup_css_from_page() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 10/48] writeback: move backing_dev_info->state into bdi_writeback Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 11/48] writeback: move backing_dev_info->bdi_stat[] " Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 12/48] writeback: move bandwidth related fields from backing_dev_info " Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 13/48] writeback: s/bdi/wb/ in mm/page-writeback.c Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 14/48] writeback: move backing_dev_info->wb_lock and ->worklist into bdi_writeback Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 15/48] writeback: reorganize mm/backing-dev.c Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 16/48] writeback: separate out include/linux/backing-dev-defs.h Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 17/48] bdi: make inode_to_bdi() inline Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 18/48] writeback: add @gfp to wb_init() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 19/48] bdi: separate out congested state into a separate struct Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` Tejun Heo [this message]
2015-03-23  4:54   ` [PATCH 20/48] writeback: add {CONFIG|BDI_CAP|FS}_CGROUP_WRITEBACK Tejun Heo
2015-03-23  4:54 ` [PATCH 21/48] writeback: make backing_dev_info host cgroup-specific bdi_writebacks Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-27 21:06   ` Vivek Goyal
2015-03-27 21:06     ` Vivek Goyal
2015-03-27 21:41     ` Tejun Heo
2015-03-27 21:41       ` Tejun Heo
2015-03-23  4:54 ` [PATCH 22/48] writeback, blkcg: associate each blkcg_gq with the corresponding bdi_writeback_congested Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 23/48] writeback: attribute stats to the matching per-cgroup bdi_writeback Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 24/48] writeback: let balance_dirty_pages() work on the matching cgroup bdi_writeback Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 25/48] writeback: make congestion functions per bdi_writeback Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 26/48] writeback, blkcg: restructure blk_{set|clear}_queue_congested() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 27/48] writeback, blkcg: propagate non-root blkcg congestion state Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 28/48] writeback: implement and use mapping_congested() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-27 18:06   ` Vivek Goyal
2015-03-27 18:06     ` Vivek Goyal
2015-03-27 18:06     ` Vivek Goyal
2015-03-27 21:46     ` Tejun Heo
2015-03-27 21:46       ` Tejun Heo
2015-03-23  4:54 ` [PATCH 29/48] writeback: implement WB_has_dirty_io wb_state flag Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 30/48] writeback: implement backing_dev_info->tot_write_bandwidth Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 31/48] writeback: make bdi_has_dirty_io() take multiple bdi_writeback's into account Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 32/48] writeback: don't issue wb_writeback_work if clean Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 33/48] writeback: make bdi->min/max_ratio handling cgroup writeback aware Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 34/48] writeback: implement bdi_for_each_wb() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 35/48] writeback: remove bdi_start_writeback() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 36/48] writeback: make laptop_mode_timer_fn() handle multiple bdi_writeback's Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 37/48] writeback: make writeback_in_progress() take bdi_writeback instead of backing_dev_info Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 38/48] writeback: make bdi_start_background_writeback() " Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 39/48] writeback: make wakeup_flusher_threads() handle multiple bdi_writeback's Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 40/48] writeback: add wb_writeback_work->auto_free Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 41/48] writeback: implement bdi_wait_for_completion() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 42/48] writeback: implement wb_wait_for_single_work() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 43/48] writeback: restructure try_writeback_inodes_sb[_nr]() Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 44/48] writeback: make writeback initiation functions handle multiple bdi_writeback's Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 45/48] writeback: dirty inodes against their matching cgroup bdi_writeback's Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 46/48] buffer, writeback: make __block_write_full_page() honor cgroup writeback Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 47/48] mpage: make __mpage_writepage() " Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-23  4:54 ` [PATCH 48/48] ext2: enable cgroup writeback support Tejun Heo
2015-03-23  4:54   ` Tejun Heo
2015-03-25 15:40 ` [PATCHSET 1/3 v2 block/for-4.1/core] writeback: " Vivek Goyal
2015-03-25 15:40   ` Vivek Goyal
2015-03-25 15:40   ` Vivek Goyal
2015-03-25 16:01   ` Tejun Heo
2015-03-25 16:01     ` Tejun Heo
2015-03-25 16:01     ` Tejun Heo
2015-03-31 15:26 ` Tejun Heo
2015-03-31 15:26   ` Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1427086499-15657-21-git-send-email-tj@kernel.org \
    --to=tj@kernel.org \
    --cc=axboe@kernel.dk \
    --cc=cgroups@vger.kernel.org \
    --cc=clm@fb.com \
    --cc=david@fromorbit.com \
    --cc=fengguang.wu@intel.com \
    --cc=gthelen@google.com \
    --cc=hannes@cmpxchg.org \
    --cc=hch@infradead.org \
    --cc=jack@suse.cz \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lizefan@huawei.com \
    --cc=mhocko@suse.cz \
    --cc=vgoyal@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.