All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vivek Goyal <vgoyal@redhat.com>
To: Jeff Moyer <jmoyer@redhat.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>,
	linux-kernel@vger.kernel.org,
	containers@lists.linux-foundation.org, dm-devel@redhat.com,
	jens.axboe@oracle.com, nauman@google.com, dpshah@google.com,
	lizf@cn.fujitsu.com, mikew@google.com, fchecconi@gmail.com,
	paolo.valente@unimore.it, ryov@valinux.co.jp,
	fernando@oss.ntt.co.jp, s-uchida@ap.jp.nec.com,
	taka@valinux.co.jp, guijianfeng@cn.fujitsu.com,
	dhaval@linux.vnet.ibm.com, righi.andrea@gmail.com,
	m-ikeda@ds.jp.nec.com, jbaron@redhat.com, agk@redhat.com,
	snitzer@redhat.com, akpm@linux-foundation.org,
	peterz@infradead.org
Subject: Re: [RFC] IO scheduler based io controller (V5)
Date: Mon, 22 Jun 2009 12:02:07 -0400	[thread overview]
Message-ID: <20090622160207.GC15600@redhat.com> (raw)
In-Reply-To: <x497hz4l1j9.fsf@segfault.boston.devel.redhat.com>

On Mon, Jun 22, 2009 at 11:40:42AM -0400, Jeff Moyer wrote:
> Vivek Goyal <vgoyal@redhat.com> writes:
> 
> > On Sun, Jun 21, 2009 at 08:51:16PM +0530, Balbir Singh wrote:
> >> * Vivek Goyal <vgoyal@redhat.com> [2009-06-19 16:37:18]:
> >> 
> >> > 
> >> > Hi All,
> >> > 
> >> > Here is the V5 of the IO controller patches generated on top of 2.6.30.
> >> [snip]
> >> 
> >> > Testing
> >> > =======
> >> >
> >> 
> >> [snip]
> >> 
> >> I've not been reading through the discussions in complete detail, but
> >> I see no reference to async reads or aio. In the case of aio, aio
> >> presumes the context of the user space process. Could you elaborate on
> >> any testing you've done with these cases? 
> >> 
> >
> > Hi Balbir,
> >
> > So far I had not done any testing with AIO. I have done some just now.
> > Here are the results.
> >
> > Test1 (AIO reads)
> > ================
> > Set up two fio, AIO read jobs in two cgroup with weight 1000 and 500
> > respectively. I am using cfq scheduler. Following are some lines from my test
> > script.
> >
> > ===================================================================
> > fio_args="--ioengine=libaio --rw=read --size=512M"
> 
> AIO doesn't make sense without O_DIRECT.
> 

Ok, here are the read results with --direct=1 for reads. In previous posting,
writes were already direct.

test1 statistics: time=8 16 20796   sectors=8 16 1049648
test2 statistics: time=8 16 10551   sectors=8 16 581160


Not sure why reads are so slow with --direct=1? In the previous test
(no direct IO), I had cleared the caches using
(echo 3 > /proc/sys/vm/drop_caches) so reads could not have come from page
cache?

Thanks
Vivek

WARNING: multiple messages have this Message-ID (diff)
From: Vivek Goyal <vgoyal@redhat.com>
To: Jeff Moyer <jmoyer@redhat.com>
Cc: dhaval@linux.vnet.ibm.com, snitzer@redhat.com,
	peterz@infradead.org, dm-devel@redhat.com, dpshah@google.com,
	jens.axboe@oracle.com, agk@redhat.com,
	Balbir Singh <balbir@linux.vnet.ibm.com>,
	paolo.valente@unimore.it, guijianfeng@cn.fujitsu.com,
	fernando@oss.ntt.co.jp, mikew@google.com, nauman@google.com,
	m-ikeda@ds.jp.nec.com, lizf@cn.fujitsu.com, fchecconi@gmail.com,
	akpm@linux-foundation.org, containers@lists.linux-foundation.org,
	linux-kernel@vger.kernel.org, s-uchida@ap.jp.nec.com,
	righi.andrea@gmail.com, jbaron@redhat.com
Subject: Re: [RFC] IO scheduler based io controller (V5)
Date: Mon, 22 Jun 2009 12:02:07 -0400	[thread overview]
Message-ID: <20090622160207.GC15600@redhat.com> (raw)
In-Reply-To: <x497hz4l1j9.fsf@segfault.boston.devel.redhat.com>

On Mon, Jun 22, 2009 at 11:40:42AM -0400, Jeff Moyer wrote:
> Vivek Goyal <vgoyal@redhat.com> writes:
> 
> > On Sun, Jun 21, 2009 at 08:51:16PM +0530, Balbir Singh wrote:
> >> * Vivek Goyal <vgoyal@redhat.com> [2009-06-19 16:37:18]:
> >> 
> >> > 
> >> > Hi All,
> >> > 
> >> > Here is the V5 of the IO controller patches generated on top of 2.6.30.
> >> [snip]
> >> 
> >> > Testing
> >> > =======
> >> >
> >> 
> >> [snip]
> >> 
> >> I've not been reading through the discussions in complete detail, but
> >> I see no reference to async reads or aio. In the case of aio, aio
> >> presumes the context of the user space process. Could you elaborate on
> >> any testing you've done with these cases? 
> >> 
> >
> > Hi Balbir,
> >
> > So far I had not done any testing with AIO. I have done some just now.
> > Here are the results.
> >
> > Test1 (AIO reads)
> > ================
> > Set up two fio, AIO read jobs in two cgroup with weight 1000 and 500
> > respectively. I am using cfq scheduler. Following are some lines from my test
> > script.
> >
> > ===================================================================
> > fio_args="--ioengine=libaio --rw=read --size=512M"
> 
> AIO doesn't make sense without O_DIRECT.
> 

Ok, here are the read results with --direct=1 for reads. In previous posting,
writes were already direct.

test1 statistics: time=8 16 20796   sectors=8 16 1049648
test2 statistics: time=8 16 10551   sectors=8 16 581160


Not sure why reads are so slow with --direct=1? In the previous test
(no direct IO), I had cleared the caches using
(echo 3 > /proc/sys/vm/drop_caches) so reads could not have come from page
cache?

Thanks
Vivek

  reply	other threads:[~2009-06-22 16:04 UTC|newest]

Thread overview: 176+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-06-19 20:37 [RFC] IO scheduler based io controller (V5) Vivek Goyal
2009-06-19 20:37 ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 01/20] io-controller: Documentation Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 02/20] io-controller: Common flat fair queuing code in elevaotor layer Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-22  8:46   ` Balbir Singh
2009-06-22  8:46     ` Balbir Singh
2009-06-22 12:43     ` Fabio Checconi
     [not found]       ` <20090622124313.GF28770-f9ZlEuEWxVeACYmtYXMKmw@public.gmane.org>
2009-06-23  2:43         ` Vivek Goyal
2009-06-23  2:43       ` Vivek Goyal
2009-06-23  2:43         ` Vivek Goyal
     [not found]         ` <20090623024337.GC3620-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-23  4:10           ` Fabio Checconi
2009-06-23  4:10             ` Fabio Checconi
2009-06-23  7:32             ` Balbir Singh
2009-06-23  7:32               ` Balbir Singh
     [not found]               ` <20090623073252.GJ8642-SINUvgVNF2CyUtPGxGje5AC/G2K4zDHf@public.gmane.org>
2009-06-23 13:42                 ` Fabio Checconi
2009-06-23 13:42               ` Fabio Checconi
     [not found]             ` <20090623041052.GS28770-f9ZlEuEWxVeACYmtYXMKmw@public.gmane.org>
2009-06-23  7:32               ` Balbir Singh
2009-06-23  2:05     ` Vivek Goyal
2009-06-23  2:05       ` Vivek Goyal
     [not found]       ` <20090623020515.GA3620-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-23  2:20         ` Jeff Moyer
2009-06-23  2:20           ` Jeff Moyer
     [not found]     ` <20090622084612.GD3728-SINUvgVNF2CyUtPGxGje5AC/G2K4zDHf@public.gmane.org>
2009-06-22 12:43       ` Fabio Checconi
2009-06-23  2:05       ` Vivek Goyal
2009-06-30  6:40   ` Gui Jianfeng
2009-07-01  1:28     ` Vivek Goyal
2009-07-01  1:28       ` Vivek Goyal
     [not found]     ` <4A49B364.5000508-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-07-01  1:28       ` Vivek Goyal
2009-07-01  9:24   ` Gui Jianfeng
     [not found]   ` <1245443858-8487-3-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-22  8:46     ` Balbir Singh
2009-06-30  6:40     ` Gui Jianfeng
2009-07-01  9:24     ` Gui Jianfeng
2009-06-19 20:37 ` [PATCH 03/20] io-controller: Charge for time slice based on average disk rate Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 04/20] io-controller: Modify cfq to make use of flat elevator fair queuing Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 05/20] io-controller: Common hierarchical fair queuing code in elevaotor layer Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-29  5:27   ` [PATCH] io-controller: optimization for iog deletion when elevator exiting Gui Jianfeng
     [not found]     ` <4A4850D3.3000700-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-29 14:06       ` Vivek Goyal
2009-06-29 14:06     ` Vivek Goyal
2009-06-29 14:06       ` Vivek Goyal
2009-06-30 17:14       ` Nauman Rafique
2009-06-30 17:14         ` Nauman Rafique
2009-07-01  1:34         ` Vivek Goyal
2009-07-01  1:34           ` Vivek Goyal
     [not found]         ` <e98e18940906301014n146e7146vb5a73c2f33c9e819-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2009-07-01  1:34           ` Vivek Goyal
     [not found]       ` <20090629140631.GA4622-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-30 17:14         ` Nauman Rafique
     [not found]   ` <1245443858-8487-6-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-29  5:27     ` Gui Jianfeng
2009-06-19 20:37 ` [PATCH 06/20] io-controller: cfq changes to use hierarchical fair queuing code in elevaotor layer Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 07/20] io-controller: Export disk time used and nr sectors dipatched through cgroups Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-23 12:10   ` Gui Jianfeng
2009-06-23 12:10     ` Gui Jianfeng
     [not found]     ` <4A40C64E.8040305-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-23 14:38       ` Vivek Goyal
2009-06-23 14:38     ` Vivek Goyal
2009-06-23 14:38       ` Vivek Goyal
     [not found]   ` <1245443858-8487-8-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-23 12:10     ` Gui Jianfeng
2009-06-19 20:37 ` [PATCH 08/20] io-controller: idle for sometime on sync queue before expiring it Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-30  7:49   ` [PATCH] io-controller: Don't expire an idle ioq if it's the only ioq in hierarchy Gui Jianfeng
     [not found]     ` <4A49C381.3040302-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-07-01  1:32       ` Vivek Goyal
2009-07-01  1:32     ` Vivek Goyal
2009-07-01  1:32       ` Vivek Goyal
     [not found]       ` <20090701013239.GB13958-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-07-01  1:40         ` Gui Jianfeng
2009-07-01  1:40       ` Gui Jianfeng
2009-07-01  1:40         ` Gui Jianfeng
     [not found]   ` <1245443858-8487-9-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-30  7:49     ` Gui Jianfeng
2009-06-19 20:37 ` [PATCH 09/20] io-controller: Separate out queue and data Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 10/20] io-conroller: Prepare elevator layer for single queue schedulers Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 11/20] io-controller: noop changes for hierarchical fair queuing Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 12/20] io-controller: deadline " Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 13/20] io-controller: anticipatory " Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
     [not found] ` <1245443858-8487-1-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-19 20:37   ` [PATCH 01/20] io-controller: Documentation Vivek Goyal
2009-06-19 20:37   ` [PATCH 02/20] io-controller: Common flat fair queuing code in elevaotor layer Vivek Goyal
2009-06-19 20:37   ` [PATCH 03/20] io-controller: Charge for time slice based on average disk rate Vivek Goyal
2009-06-19 20:37   ` [PATCH 04/20] io-controller: Modify cfq to make use of flat elevator fair queuing Vivek Goyal
2009-06-19 20:37   ` [PATCH 05/20] io-controller: Common hierarchical fair queuing code in elevaotor layer Vivek Goyal
2009-06-19 20:37   ` [PATCH 06/20] io-controller: cfq changes to use " Vivek Goyal
2009-06-19 20:37   ` [PATCH 07/20] io-controller: Export disk time used and nr sectors dipatched through cgroups Vivek Goyal
2009-06-19 20:37   ` [PATCH 08/20] io-controller: idle for sometime on sync queue before expiring it Vivek Goyal
2009-06-19 20:37   ` [PATCH 09/20] io-controller: Separate out queue and data Vivek Goyal
2009-06-19 20:37   ` [PATCH 10/20] io-conroller: Prepare elevator layer for single queue schedulers Vivek Goyal
2009-06-19 20:37   ` [PATCH 11/20] io-controller: noop changes for hierarchical fair queuing Vivek Goyal
2009-06-19 20:37   ` [PATCH 12/20] io-controller: deadline " Vivek Goyal
2009-06-19 20:37   ` [PATCH 13/20] io-controller: anticipatory " Vivek Goyal
2009-06-19 20:37   ` [PATCH 14/20] blkio_cgroup patches from Ryo to track async bios Vivek Goyal
2009-06-19 20:37   ` [PATCH 15/20] io-controller: map async requests to appropriate cgroup Vivek Goyal
2009-06-19 20:37   ` [PATCH 16/20] io-controller: Per cgroup request descriptor support Vivek Goyal
2009-06-19 20:37   ` [PATCH 17/20] io-controller: Per io group bdi congestion interface Vivek Goyal
2009-06-19 20:37   ` [PATCH 18/20] io-controller: Support per cgroup per device weights and io class Vivek Goyal
2009-06-19 20:37   ` [PATCH 19/20] io-controller: Debug hierarchical IO scheduling Vivek Goyal
2009-06-19 20:37   ` [PATCH 20/20] io-controller: experimental debug patch for async queue wait before expiry Vivek Goyal
2009-06-21 15:21   ` [RFC] IO scheduler based io controller (V5) Balbir Singh
2009-06-29 16:04   ` Vladislav Bolkhovitin
2009-06-19 20:37 ` [PATCH 14/20] blkio_cgroup patches from Ryo to track async bios Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 15/20] io-controller: map async requests to appropriate cgroup Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
     [not found]   ` <1245443858-8487-16-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-22  1:45     ` Gui Jianfeng
2009-06-22  1:45   ` Gui Jianfeng
     [not found]     ` <4A3EE245.7030409-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-22 15:39       ` Vivek Goyal
2009-06-22 15:39     ` Vivek Goyal
2009-06-22 15:39       ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 16/20] io-controller: Per cgroup request descriptor support Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 17/20] io-controller: Per io group bdi congestion interface Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 18/20] io-controller: Support per cgroup per device weights and io class Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-24 21:52   ` Paul Menage
2009-06-24 21:52     ` Paul Menage
2009-06-25 10:23     ` [PATCH] io-controller: do some changes of io.policy interface Gui Jianfeng
2009-06-25 10:23       ` Gui Jianfeng
     [not found]       ` <4A435038.60406-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-25 12:55         ` Vivek Goyal
2009-06-25 12:55       ` Vivek Goyal
2009-06-25 12:55         ` Vivek Goyal
2009-06-26  0:27         ` Gui Jianfeng
2009-06-26  0:27           ` Gui Jianfeng
     [not found]         ` <20090625125513.GA25439-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-26  0:27           ` Gui Jianfeng
2009-06-26  0:59           ` Gui Jianfeng
2009-06-26  0:59         ` Gui Jianfeng
     [not found]     ` <6599ad830906241452t76e64815s7d68a22a6e746a59-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2009-06-25 10:23       ` Gui Jianfeng
     [not found]   ` <1245443858-8487-19-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-24 21:52     ` [PATCH 18/20] io-controller: Support per cgroup per device weights and io class Paul Menage
2009-06-19 20:37 ` [PATCH 19/20] io-controller: Debug hierarchical IO scheduling Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
2009-06-19 20:37 ` [PATCH 20/20] io-controller: experimental debug patch for async queue wait before expiry Vivek Goyal
2009-06-19 20:37   ` Vivek Goyal
     [not found]   ` <1245443858-8487-21-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-22  7:44     ` [PATCH] io-controller: Preempt a non-rt queue if a rt ioq is present in ancestor or sibling groups Gui Jianfeng
2009-06-22  7:44   ` Gui Jianfeng
2009-06-22 17:21     ` Vivek Goyal
2009-06-22 17:21       ` Vivek Goyal
2009-06-23  6:44       ` Gui Jianfeng
2009-06-23 14:02         ` Vivek Goyal
2009-06-23 14:02           ` Vivek Goyal
2009-06-24  9:20           ` Gui Jianfeng
2009-06-26  8:13             ` [PATCH 1/2] io-controller: Prepare a rt ioq list in efqd to keep track of busy rt ioqs Gui Jianfeng
2009-06-26  8:13               ` Gui Jianfeng
     [not found]             ` <4A41EFE1.5050101-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-26  8:13               ` Gui Jianfeng
2009-06-26  8:13               ` [PATCH 2/2] io-controller: make rt preemption happen in the whole hierarchy Gui Jianfeng
2009-06-26  8:13             ` Gui Jianfeng
     [not found]               ` <4A44833F.8040308-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-26 12:39                 ` Vivek Goyal
2009-06-26 12:39               ` Vivek Goyal
2009-06-26 12:39                 ` Vivek Goyal
     [not found]           ` <20090623140250.GA4262-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-24  9:20             ` [PATCH] io-controller: Preempt a non-rt queue if a rt ioq is present in ancestor or sibling groups Gui Jianfeng
     [not found]         ` <4A4079B8.4020402-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-23 14:02           ` Vivek Goyal
     [not found]       ` <20090622172123.GE15600-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-23  6:44         ` Gui Jianfeng
     [not found]     ` <4A3F3648.7080007-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
2009-06-22 17:21       ` Vivek Goyal
2009-06-21 15:21 ` [RFC] IO scheduler based io controller (V5) Balbir Singh
2009-06-22 15:30   ` Vivek Goyal
2009-06-22 15:30     ` Vivek Goyal
2009-06-22 15:40     ` Jeff Moyer
2009-06-22 16:02       ` Vivek Goyal [this message]
2009-06-22 16:02         ` Vivek Goyal
     [not found]         ` <20090622160207.GC15600-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-22 16:06           ` Jeff Moyer
2009-06-22 16:06             ` Jeff Moyer
     [not found]             ` <x493a9sl0bx.fsf-RRHT56Q3PSP4kTEheFKJxxDDeQx5vsVwAInAS/Ez/D0@public.gmane.org>
2009-06-22 17:08               ` Vivek Goyal
2009-06-22 17:08             ` Vivek Goyal
2009-06-22 17:08               ` Vivek Goyal
     [not found]               ` <20090622170812.GD15600-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-23  6:52                 ` Balbir Singh
2009-06-23  6:52               ` Balbir Singh
     [not found]       ` <x497hz4l1j9.fsf-RRHT56Q3PSP4kTEheFKJxxDDeQx5vsVwAInAS/Ez/D0@public.gmane.org>
2009-06-22 16:02         ` Vivek Goyal
     [not found]     ` <20090622153030.GA15600-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2009-06-22 15:40       ` Jeff Moyer
     [not found]   ` <20090621152116.GC3728-SINUvgVNF2CyUtPGxGje5AC/G2K4zDHf@public.gmane.org>
2009-06-22 15:30     ` Vivek Goyal
2009-06-29 16:04 ` Vladislav Bolkhovitin
2009-06-29 17:23   ` Vivek Goyal
2009-06-29 17:23     ` Vivek Goyal
     [not found]   ` <4A48E601.2050203-d+Crzxg7Rs0@public.gmane.org>
2009-06-29 17:23     ` Vivek Goyal
2009-06-19 20:37 Vivek Goyal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20090622160207.GC15600@redhat.com \
    --to=vgoyal@redhat.com \
    --cc=agk@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=balbir@linux.vnet.ibm.com \
    --cc=containers@lists.linux-foundation.org \
    --cc=dhaval@linux.vnet.ibm.com \
    --cc=dm-devel@redhat.com \
    --cc=dpshah@google.com \
    --cc=fchecconi@gmail.com \
    --cc=fernando@oss.ntt.co.jp \
    --cc=guijianfeng@cn.fujitsu.com \
    --cc=jbaron@redhat.com \
    --cc=jens.axboe@oracle.com \
    --cc=jmoyer@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizf@cn.fujitsu.com \
    --cc=m-ikeda@ds.jp.nec.com \
    --cc=mikew@google.com \
    --cc=nauman@google.com \
    --cc=paolo.valente@unimore.it \
    --cc=peterz@infradead.org \
    --cc=righi.andrea@gmail.com \
    --cc=ryov@valinux.co.jp \
    --cc=s-uchida@ap.jp.nec.com \
    --cc=snitzer@redhat.com \
    --cc=taka@valinux.co.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.