* Poor performance with bcache write-back mode
@ 2013-05-16 21:51 Patrick Ng
[not found] ` <1368741106.62060.YahooMailNeo-QseONlJJzSFal3a5i+KC+JEhsgyP+Z75VpNB7YpNyf8@public.gmane.org>
[not found] ` <CA+gjuN7E2B+71BiCexm7HjDGzJMBuCdbGxhvD8YHU6-u6TyKkg@mail.gmail.com>
0 siblings, 2 replies; 6+ messages in thread
From: Patrick Ng @ 2013-05-16 21:51 UTC (permalink / raw)
To: linux-bcache-u79uwXL29TY76Z2rM5mHXA
Hi all,
I have installed Linux kernel 3.9.2 in my Fedora 18. When I ran FIO in bcache write-through mode, the performance was good, but running a similar test in bcache write-back mode, I got very poor performance.
Did I miss any options when setting the write-back mode?
Environment
===========
500G backing store at /dev/sdb1
150G SSD at /dev/sdc1
Write-through mode
==================
make-bcache -B /dev/sdb1 -C /dev/sdc1
echo /dev/sdb1 > /sys/fs/bcache/register
echo /dev/sdc1 > /sys/fs/bcache/register
mkfs.ext4 /dev/bcache0
mount /dev/bcache0 /mnt/bcache
Running FIO with random, 100% read, iosize=4096, queue depth=16
the result is approx. 65k IOPS
Write-backe mode (setup is almost the same as write-through except the extra step at the end "echo writeback ...")
================
make-bcache -B /dev/sdb1 -C /dev/sdc1
echo /dev/sdb1 > /sys/fs/bcache/register
echo /dev/sdc1 > /sys/fs/bcache/register
mkfs.ext4 /dev/bcache0
mount /dev/bcache0 /mnt/bcache
echo writeback > /sys/block/bcache0/bcache/cache_mode
Running FIO with random, 100% read, iosize=4096, queue depth=16
the result is approx. 12k IOPS
Running FIO with random, 80% read 20% write, iosize=4096, queue depth=16
the result is approx. 2k IOPS
Thanks,
Patrick
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Poor performance with bcache write-back mode
[not found] ` <1368741106.62060.YahooMailNeo-QseONlJJzSFal3a5i+KC+JEhsgyP+Z75VpNB7YpNyf8@public.gmane.org>
@ 2013-05-17 6:28 ` Robert Frånlund
2013-05-29 0:40 ` Kent Overstreet
1 sibling, 0 replies; 6+ messages in thread
From: Robert Frånlund @ 2013-05-17 6:28 UTC (permalink / raw)
To: mailinglist_bcache
Hi,
Have you aligned your partitions on the SSD to at-least 4 KiB
block-size, even better with 1 MiB?
Best regards,
Robert
2013/5/16 Patrick Ng <ppng-/E1597aS9LQAvxtiuMwx3w@public.gmane.org>:
> Hi all,
>
> I have installed Linux kernel 3.9.2 in my Fedora 18. When I ran FIO in bcache write-through mode, the performance was good, but running a similar test in bcache write-back mode, I got very poor performance.
> Did I miss any options when setting the write-back mode?
>
> Environment
> ===========
> 500G backing store at /dev/sdb1
> 150G SSD at /dev/sdc1
>
> Write-through mode
> ==================
> make-bcache -B /dev/sdb1 -C /dev/sdc1
> echo /dev/sdb1 > /sys/fs/bcache/register
> echo /dev/sdc1 > /sys/fs/bcache/register
> mkfs.ext4 /dev/bcache0
> mount /dev/bcache0 /mnt/bcache
>
>
> Running FIO with random, 100% read, iosize=4096, queue depth=16
> the result is approx. 65k IOPS
>
>
> Write-backe mode (setup is almost the same as write-through except the extra step at the end "echo writeback ...")
> ================
> make-bcache -B /dev/sdb1 -C /dev/sdc1
> echo /dev/sdb1 > /sys/fs/bcache/register
> echo /dev/sdc1 > /sys/fs/bcache/register
> mkfs.ext4 /dev/bcache0
> mount /dev/bcache0 /mnt/bcache
> echo writeback > /sys/block/bcache0/bcache/cache_mode
>
> Running FIO with random, 100% read, iosize=4096, queue depth=16
> the result is approx. 12k IOPS
>
> Running FIO with random, 80% read 20% write, iosize=4096, queue depth=16
> the result is approx. 2k IOPS
>
> Thanks,
> Patrick
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Poor performance with bcache write-back mode
[not found] ` <CA+gjuN7E2B+71BiCexm7HjDGzJMBuCdbGxhvD8YHU6-u6TyKkg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-05-17 20:13 ` Patrick Ng
2013-05-17 20:50 ` Gabriel de Perthuis
0 siblings, 1 reply; 6+ messages in thread
From: Patrick Ng @ 2013-05-17 20:13 UTC (permalink / raw)
To: Robert Frånlund; +Cc: linux-bcache-u79uwXL29TY76Z2rM5mHXA
Hi Robert,
I have checked that the cache partition (/dev/sdc1) starts at 2048, it should be properly aligned.
Similarly for the backing store partition (/dev/sdb1), it starts at 2048 at well.
Thanks,
Patrick
>________________________________
> From: Robert Frånlund <robert.franlund-Hou3YEnI4S0Iwnw1HX0dmQ@public.gmane.org>
>To: Patrick Ng <ppng-/E1597aS9LQAvxtiuMwx3w@public.gmane.org>
>Cc: "linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
>Sent: Friday, May 17, 2013 2:22 AM
>Subject: Re: Poor performance with bcache write-back mode
>
>
>
>Hi,
>
>
>Have you aligned your partitions on the SSD to at-least 4 KiB block-size, even better with 1 MiB?
>
>
>Best regards,
>Robert
>
>
>
>
>
>
>
>2013/5/16 Patrick Ng <ppng-/E1597aS9LQAvxtiuMwx3w@public.gmane.org>
>
>Hi all,
>>
>>I have installed Linux kernel 3.9.2 in my Fedora 18. When I ran FIO in bcache write-through mode, the performance was good, but running a similar test in bcache write-back mode, I got very poor performance.
>>Did I miss any options when setting the write-back mode?
>>
>>Environment
>>===========
>>500G backing store at /dev/sdb1
>>150G SSD at /dev/sdc1
>>
>>Write-through mode
>>==================
>>make-bcache -B /dev/sdb1 -C /dev/sdc1
>>echo /dev/sdb1 > /sys/fs/bcache/register
>>echo /dev/sdc1 > /sys/fs/bcache/register
>>mkfs.ext4 /dev/bcache0
>>mount /dev/bcache0 /mnt/bcache
>>
>>
>>Running FIO with random, 100% read, iosize=4096, queue depth=16
>>the result is approx. 65k IOPS
>>
>>
>>Write-backe mode (setup is almost the same as write-through except the extra step at the end "echo writeback ...")
>>================
>>make-bcache -B /dev/sdb1 -C /dev/sdc1
>>echo /dev/sdb1 > /sys/fs/bcache/register
>>echo /dev/sdc1 > /sys/fs/bcache/register
>>mkfs.ext4 /dev/bcache0
>>mount /dev/bcache0 /mnt/bcache
>>echo writeback > /sys/block/bcache0/bcache/cache_mode
>>
>>Running FIO with random, 100% read, iosize=4096, queue depth=16
>>the result is approx. 12k IOPS
>>
>>Running FIO with random, 80% read 20% write, iosize=4096, queue depth=16
>>the result is approx. 2k IOPS
>>
>>Thanks,
>>Patrick
>>--
>>To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>>the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Poor performance with bcache write-back mode
2013-05-17 20:13 ` Patrick Ng
@ 2013-05-17 20:50 ` Gabriel de Perthuis
0 siblings, 0 replies; 6+ messages in thread
From: Gabriel de Perthuis @ 2013-05-17 20:50 UTC (permalink / raw)
To: linux-bcache-u79uwXL29TY76Z2rM5mHXA
On Fri, 17 May 2013 13:13:04 -0700, Patrick Ng wrote:
> Hi Robert,
>
> I have checked that the cache partition (/dev/sdc1) starts at 2048, it should be properly aligned.
> Similarly for the backing store partition (/dev/sdb1), it starts at 2048 at well.
The bcache superblock shifts the start of your partition by 8k though.
I don't know if the upper layers compensate for that, but just in case,
you may want to add `--data_offset 2048` to the make-bcache command line
(1M alignment).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Poor performance with bcache write-back mode
[not found] ` <1368741106.62060.YahooMailNeo-QseONlJJzSFal3a5i+KC+JEhsgyP+Z75VpNB7YpNyf8@public.gmane.org>
2013-05-17 6:28 ` Robert Frånlund
@ 2013-05-29 0:40 ` Kent Overstreet
2013-06-03 21:44 ` Patrick Ng
1 sibling, 1 reply; 6+ messages in thread
From: Kent Overstreet @ 2013-05-29 0:40 UTC (permalink / raw)
To: Patrick Ng; +Cc: linux-bcache-u79uwXL29TY76Z2rM5mHXA
On Thu, May 16, 2013 at 02:51:46PM -0700, Patrick Ng wrote:
> Hi all,
>
> I have installed Linux kernel 3.9.2 in my Fedora 18. When I ran FIO in bcache write-through mode, the performance was good, but running a similar test in bcache write-back mode, I got very poor performance.
> Did I miss any options when setting the write-back mode?
>
> Environment
> ===========
> 500G backing store at /dev/sdb1
> 150G SSD at /dev/sdc1
>
> Write-through mode
> ==================
> make-bcache -B /dev/sdb1 -C /dev/sdc1
> echo /dev/sdb1 > /sys/fs/bcache/register
> echo /dev/sdc1 > /sys/fs/bcache/register
> mkfs.ext4 /dev/bcache0
> mount /dev/bcache0 /mnt/bcache
>
>
> Running FIO with random, 100% read, iosize=4096, queue depth=16
> the result is approx. 65k IOPS
>
>
> Write-backe mode (setup is almost the same as write-through except the extra step at the end "echo writeback ...")
> ================
> make-bcache -B /dev/sdb1 -C /dev/sdc1
> echo /dev/sdb1 > /sys/fs/bcache/register
> echo /dev/sdc1 > /sys/fs/bcache/register
> mkfs.ext4 /dev/bcache0
> mount /dev/bcache0 /mnt/bcache
> echo writeback > /sys/block/bcache0/bcache/cache_mode
>
> Running FIO with random, 100% read, iosize=4096, queue depth=16
> the result is approx. 12k IOPS
>
> Running FIO with random, 80% read 20% write, iosize=4096, queue depth=16
> the result is approx. 2k IOPS
You're probably getting bit by the congested bypass - try setting
congested_read_threshold_us and congested_write_threshold_us both to 0.
I wish there was a better solution to the congestion bypass stuff, it's
a hard problem...
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Poor performance with bcache write-back mode
2013-05-29 0:40 ` Kent Overstreet
@ 2013-06-03 21:44 ` Patrick Ng
0 siblings, 0 replies; 6+ messages in thread
From: Patrick Ng @ 2013-06-03 21:44 UTC (permalink / raw)
To: linux-bcache-u79uwXL29TY76Z2rM5mHXA
Kent Overstreet <koverstreet@...> writes:
>
> On Thu, May 16, 2013 at 02:51:46PM -0700, Patrick Ng wrote:
> > Hi all,
> >
> > I have installed Linux kernel 3.9.2 in my Fedora 18. When I ran FIO in
bcache write-through mode, the
> performance was good, but running a similar test in bcache write-back
mode, I got very poor performance.
> > Did I miss any options when setting the write-back mode?
> >
> > Environment
> > ===========
> > 500G backing store at /dev/sdb1
> > 150G SSD at /dev/sdc1
> >
> > Write-through mode
> > ==================
> > make-bcache -B /dev/sdb1 -C /dev/sdc1
> > echo /dev/sdb1 > /sys/fs/bcache/register
> > echo /dev/sdc1 > /sys/fs/bcache/register
> > mkfs.ext4 /dev/bcache0
> > mount /dev/bcache0 /mnt/bcache
> >
> >
> > Running FIO with random, 100% read, iosize=4096, queue depth=16
> > the result is approx. 65k IOPS
> >
> >
> > Write-backe mode (setup is almost the same as write-through except the
extra step at the end "echo
> writeback ...")
> > ================
> > make-bcache -B /dev/sdb1 -C /dev/sdc1
> > echo /dev/sdb1 > /sys/fs/bcache/register
> > echo /dev/sdc1 > /sys/fs/bcache/register
> > mkfs.ext4 /dev/bcache0
> > mount /dev/bcache0 /mnt/bcache
> > echo writeback > /sys/block/bcache0/bcache/cache_mode
> >
> > Running FIO with random, 100% read, iosize=4096, queue depth=16
> > the result is approx. 12k IOPS
> >
> > Running FIO with random, 80% read 20% write, iosize=4096, queue depth=16
> > the result is approx. 2k IOPS
>
> You're probably getting bit by the congested bypass - try setting
> congested_read_threshold_us and congested_write_threshold_us both to 0.
>
> I wish there was a better solution to the congestion bypass stuff, it's
> a hard problem...
>
Hi Kent, thank you for responding to my question.
I have tried setting both congested_read_threshold_us and
congested_write_threshold_us to zero.
The result of FIO random 100% read test in WriteBack mode has improved and
the IOPS is as good as in WriteThur mode.
The result of FIO random 80% read 20% write remain the same with no
improvement.
It appears that the option does not help on the WRITE?
Thanks,
Patrick
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-06-03 21:44 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-05-16 21:51 Poor performance with bcache write-back mode Patrick Ng
[not found] ` <1368741106.62060.YahooMailNeo-QseONlJJzSFal3a5i+KC+JEhsgyP+Z75VpNB7YpNyf8@public.gmane.org>
2013-05-17 6:28 ` Robert Frånlund
2013-05-29 0:40 ` Kent Overstreet
2013-06-03 21:44 ` Patrick Ng
[not found] ` <CA+gjuN7E2B+71BiCexm7HjDGzJMBuCdbGxhvD8YHU6-u6TyKkg@mail.gmail.com>
[not found] ` <CA+gjuN7E2B+71BiCexm7HjDGzJMBuCdbGxhvD8YHU6-u6TyKkg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-05-17 20:13 ` Patrick Ng
2013-05-17 20:50 ` Gabriel de Perthuis
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.