* Can't resgister cache
@ 2012-02-15 5:26 Franco
2012-02-15 6:24 ` Kent Overstreet
0 siblings, 1 reply; 5+ messages in thread
From: Franco @ 2012-02-15 5:26 UTC (permalink / raw)
To: linux-bcache-u79uwXL29TY76Z2rM5mHXA
Hi
I was in the process of testing linux-bcache but was seeing some strange
behaviour so I thought I would remount my cached filesystem. I then
noticed that the performance results started looking like the cache just
wasn't there, so I unregistered it:
bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
bcache: Caching disabled for sdc
but now when I try to register it again, I get bcache: error
opening /dev/sdb: device busy.
I haven't tried rebooting yet but probably will do after I've sent this
message.
These were the strange results I was getting that prompted me to unmount
and remount the filesystems.
Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD
for bcache.
Filesystem Size Used Avail Use% Mounted on
/dev/bcache0 1.9T 26G 1.8T 2% /data33
/dev/sdd 1.9T 23G 1.8T 2% /data34
Wrote an 8GB file to each filesystem, then tried reading it back.
Machine has 8GB RAM.
Speed for random reads from identical SSD = 181.766523 MB/Sec, random
read test done in the following order with measured speed in MB/Sec. The
random IO is actually just reading with a 4GB stride, wrapping around at
the end of the file.
/data33 49
/data34 114
/data33 114
/data33 and /data44 simultaneously 78 and 59 respectively
/data33 76
/data33 68
/data34 114
/data34 96
/data34 69
/data34 74
/data33 118
/data33 40
/data34 91
After remounting both filesystems:
/data34 148
/data33 120
/data34 120
/data33 82
After unregistering the cache
/data33 65
/data34 83
/data33 99
/data34 113
Cheers,
Franco
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Can't resgister cache
2012-02-15 5:26 Can't resgister cache Franco
@ 2012-02-15 6:24 ` Kent Overstreet
[not found] ` <CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 5+ messages in thread
From: Kent Overstreet @ 2012-02-15 6:24 UTC (permalink / raw)
To: Franco; +Cc: linux-bcache-u79uwXL29TY76Z2rM5mHXA
Not sure what's going on with not being able to register again - not a
lot of info to go off of. It was working last I checked in the
bcache-3.2-dev branch, it might be buggy in the older bcache branch.
From your numbers it looks like the 8 gb of data wasn't in the cache -
bcache bypasses sequential IO by default. You can flip it off by
echoing 0 to sequential_cutoff.
On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote:
> Hi
>
> I was in the process of testing linux-bcache but was seeing some strange
> behaviour so I thought I would remount my cached filesystem. I then
> noticed that the performance results started looking like the cache just
> wasn't there, so I unregistered it:
>
> bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
> bcache: Caching disabled for sdc
>
> but now when I try to register it again, I get bcache: error
> opening /dev/sdb: device busy.
>
> I haven't tried rebooting yet but probably will do after I've sent this
> message.
>
> These were the strange results I was getting that prompted me to unmount
> and remount the filesystems.
>
> Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD
> for bcache.
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/bcache0 1.9T 26G 1.8T 2% /data33
> /dev/sdd 1.9T 23G 1.8T 2% /data34
>
> Wrote an 8GB file to each filesystem, then tried reading it back.
> Machine has 8GB RAM.
>
> Speed for random reads from identical SSD = 181.766523 MB/Sec, random
> read test done in the following order with measured speed in MB/Sec. The
> random IO is actually just reading with a 4GB stride, wrapping around at
> the end of the file.
>
> /data33 49
> /data34 114
> /data33 114
> /data33 and /data44 simultaneously 78 and 59 respectively
> /data33 76
> /data33 68
> /data34 114
> /data34 96
> /data34 69
> /data34 74
> /data33 118
> /data33 40
> /data34 91
>
> After remounting both filesystems:
>
> /data34 148
> /data33 120
> /data34 120
> /data33 82
>
>
> After unregistering the cache
>
> /data33 65
> /data34 83
> /data33 99
> /data34 113
>
> Cheers,
> Franco
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Can't resgister cache
[not found] ` <CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2012-02-15 7:16 ` Franco
2012-02-16 1:09 ` Kent Overstreet
0 siblings, 1 reply; 5+ messages in thread
From: Franco @ 2012-02-15 7:16 UTC (permalink / raw)
To: Kent Overstreet; +Cc: linux-bcache-u79uwXL29TY76Z2rM5mHXA
I did a reboot and got it registered again.
bio: create slab <bio-1> at 1
bcache: invalidating existing data
bcache: registered cache device sdb
bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3
EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null)
First read went at 119 MB/Sec, second at 37MB/Sec
Here's what's in stats_hour
/sys/block/sdc/bcache/stats_hour/cache_bypass_hits
::::::::::::::
44
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_bypass_misses
::::::::::::::
81172
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hit_ratio
::::::::::::::
0
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hits
::::::::::::::
24
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_miss_collisions
::::::::::::::
14
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_misses
::::::::::::::
407422
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_readaheads
::::::::::::::
0
With echo 0 >/sys/block/sdc/bcache/sequential_cutoff
33 MB/Sec
On Tue, 2012-02-14 at 22:24 -0800, Kent Overstreet wrote:
> Not sure what's going on with not being able to register again - not a
> lot of info to go off of. It was working last I checked in the
> bcache-3.2-dev branch, it might be buggy in the older bcache branch.
>
> >From your numbers it looks like the 8 gb of data wasn't in the cache -
> bcache bypasses sequential IO by default. You can flip it off by
> echoing 0 to sequential_cutoff.
>
> On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote:
> > Hi
> >
> > I was in the process of testing linux-bcache but was seeing some strange
> > behaviour so I thought I would remount my cached filesystem. I then
> > noticed that the performance results started looking like the cache just
> > wasn't there, so I unregistered it:
> >
> > bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
> > bcache: Caching disabled for sdc
> >
> > but now when I try to register it again, I get bcache: error
> > opening /dev/sdb: device busy.
> >
> > I haven't tried rebooting yet but probably will do after I've sent this
> > message.
> >
> > These were the strange results I was getting that prompted me to unmount
> > and remount the filesystems.
> >
> > Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD
> > for bcache.
> >
> > Filesystem Size Used Avail Use% Mounted on
> > /dev/bcache0 1.9T 26G 1.8T 2% /data33
> > /dev/sdd 1.9T 23G 1.8T 2% /data34
> >
> > Wrote an 8GB file to each filesystem, then tried reading it back.
> > Machine has 8GB RAM.
> >
> > Speed for random reads from identical SSD = 181.766523 MB/Sec, random
> > read test done in the following order with measured speed in MB/Sec. The
> > random IO is actually just reading with a 4GB stride, wrapping around at
> > the end of the file.
> >
> > /data33 49
> > /data34 114
> > /data33 114
> > /data33 and /data44 simultaneously 78 and 59 respectively
> > /data33 76
> > /data33 68
> > /data34 114
> > /data34 96
> > /data34 69
> > /data34 74
> > /data33 118
> > /data33 40
> > /data34 91
> >
> > After remounting both filesystems:
> >
> > /data34 148
> > /data33 120
> > /data34 120
> > /data33 82
> >
> >
> > After unregistering the cache
> >
> > /data33 65
> > /data34 83
> > /data33 99
> > /data34 113
> >
> > Cheers,
> > Franco
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Can't resgister cache
2012-02-15 7:16 ` Franco
@ 2012-02-16 1:09 ` Kent Overstreet
[not found] ` <CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 5+ messages in thread
From: Kent Overstreet @ 2012-02-16 1:09 UTC (permalink / raw)
To: Franco; +Cc: Kent Overstreet, linux-bcache-u79uwXL29TY76Z2rM5mHXA
That sounds like your SSD is slow and bottlenecking you... What kind
of SSD is it?
Could be some strange plugging thing, maybe...
On Tue, Feb 14, 2012 at 11:16 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote:
>
> I did a reboot and got it registered again.
>
> bio: create slab <bio-1> at 1
> bcache: invalidating existing data
> bcache: registered cache device sdb
> bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3
> EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null)
>
>
> First read went at 119 MB/Sec, second at 37MB/Sec
>
> Here's what's in stats_hour
>
> /sys/block/sdc/bcache/stats_hour/cache_bypass_hits
> ::::::::::::::
> 44
> ::::::::::::::
> /sys/block/sdc/bcache/stats_hour/cache_bypass_misses
> ::::::::::::::
> 81172
> ::::::::::::::
> /sys/block/sdc/bcache/stats_hour/cache_hit_ratio
> ::::::::::::::
> 0
> ::::::::::::::
> /sys/block/sdc/bcache/stats_hour/cache_hits
> ::::::::::::::
> 24
> ::::::::::::::
> /sys/block/sdc/bcache/stats_hour/cache_miss_collisions
> ::::::::::::::
> 14
> ::::::::::::::
> /sys/block/sdc/bcache/stats_hour/cache_misses
> ::::::::::::::
> 407422
> ::::::::::::::
> /sys/block/sdc/bcache/stats_hour/cache_readaheads
> ::::::::::::::
> 0
>
> With echo 0 >/sys/block/sdc/bcache/sequential_cutoff
>
> 33 MB/Sec
>
>
> On Tue, 2012-02-14 at 22:24 -0800, Kent Overstreet wrote:
>> Not sure what's going on with not being able to register again - not a
>> lot of info to go off of. It was working last I checked in the
>> bcache-3.2-dev branch, it might be buggy in the older bcache branch.
>>
>> >From your numbers it looks like the 8 gb of data wasn't in the cache -
>> bcache bypasses sequential IO by default. You can flip it off by
>> echoing 0 to sequential_cutoff.
>>
>> On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote:
>> > Hi
>> >
>> > I was in the process of testing linux-bcache but was seeing some strange
>> > behaviour so I thought I would remount my cached filesystem. I then
>> > noticed that the performance results started looking like the cache just
>> > wasn't there, so I unregistered it:
>> >
>> > bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
>> > bcache: Caching disabled for sdc
>> >
>> > but now when I try to register it again, I get bcache: error
>> > opening /dev/sdb: device busy.
>> >
>> > I haven't tried rebooting yet but probably will do after I've sent this
>> > message.
>> >
>> > These were the strange results I was getting that prompted me to unmount
>> > and remount the filesystems.
>> >
>> > Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD
>> > for bcache.
>> >
>> > Filesystem Size Used Avail Use% Mounted on
>> > /dev/bcache0 1.9T 26G 1.8T 2% /data33
>> > /dev/sdd 1.9T 23G 1.8T 2% /data34
>> >
>> > Wrote an 8GB file to each filesystem, then tried reading it back.
>> > Machine has 8GB RAM.
>> >
>> > Speed for random reads from identical SSD = 181.766523 MB/Sec, random
>> > read test done in the following order with measured speed in MB/Sec. The
>> > random IO is actually just reading with a 4GB stride, wrapping around at
>> > the end of the file.
>> >
>> > /data33 49
>> > /data34 114
>> > /data33 114
>> > /data33 and /data44 simultaneously 78 and 59 respectively
>> > /data33 76
>> > /data33 68
>> > /data34 114
>> > /data34 96
>> > /data34 69
>> > /data34 74
>> > /data33 118
>> > /data33 40
>> > /data34 91
>> >
>> > After remounting both filesystems:
>> >
>> > /data34 148
>> > /data33 120
>> > /data34 120
>> > /data33 82
>> >
>> >
>> > After unregistering the cache
>> >
>> > /data33 65
>> > /data34 83
>> > /data33 99
>> > /data34 113
>> >
>> > Cheers,
>> > Franco
>> >
>> >
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Can't resgister cache
[not found] ` <CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2012-02-16 1:10 ` Kent Overstreet
0 siblings, 0 replies; 5+ messages in thread
From: Kent Overstreet @ 2012-02-16 1:10 UTC (permalink / raw)
To: Franco; +Cc: Kent Overstreet, linux-bcache-u79uwXL29TY76Z2rM5mHXA
Also, IO is still getting bypassed. If sequential_cutoff is 0, then
the only reason that'd happen is if the latency to the SSD is getting
too high...
On Wed, Feb 15, 2012 at 5:09 PM, Kent Overstreet
<kent.overstreet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> That sounds like your SSD is slow and bottlenecking you... What kind
> of SSD is it?
>
> Could be some strange plugging thing, maybe...
>
> On Tue, Feb 14, 2012 at 11:16 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote:
>>
>> I did a reboot and got it registered again.
>>
>> bio: create slab <bio-1> at 1
>> bcache: invalidating existing data
>> bcache: registered cache device sdb
>> bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3
>> EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null)
>>
>>
>> First read went at 119 MB/Sec, second at 37MB/Sec
>>
>> Here's what's in stats_hour
>>
>> /sys/block/sdc/bcache/stats_hour/cache_bypass_hits
>> ::::::::::::::
>> 44
>> ::::::::::::::
>> /sys/block/sdc/bcache/stats_hour/cache_bypass_misses
>> ::::::::::::::
>> 81172
>> ::::::::::::::
>> /sys/block/sdc/bcache/stats_hour/cache_hit_ratio
>> ::::::::::::::
>> 0
>> ::::::::::::::
>> /sys/block/sdc/bcache/stats_hour/cache_hits
>> ::::::::::::::
>> 24
>> ::::::::::::::
>> /sys/block/sdc/bcache/stats_hour/cache_miss_collisions
>> ::::::::::::::
>> 14
>> ::::::::::::::
>> /sys/block/sdc/bcache/stats_hour/cache_misses
>> ::::::::::::::
>> 407422
>> ::::::::::::::
>> /sys/block/sdc/bcache/stats_hour/cache_readaheads
>> ::::::::::::::
>> 0
>>
>> With echo 0 >/sys/block/sdc/bcache/sequential_cutoff
>>
>> 33 MB/Sec
>>
>>
>> On Tue, 2012-02-14 at 22:24 -0800, Kent Overstreet wrote:
>>> Not sure what's going on with not being able to register again - not a
>>> lot of info to go off of. It was working last I checked in the
>>> bcache-3.2-dev branch, it might be buggy in the older bcache branch.
>>>
>>> >From your numbers it looks like the 8 gb of data wasn't in the cache -
>>> bcache bypasses sequential IO by default. You can flip it off by
>>> echoing 0 to sequential_cutoff.
>>>
>>> On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote:
>>> > Hi
>>> >
>>> > I was in the process of testing linux-bcache but was seeing some strange
>>> > behaviour so I thought I would remount my cached filesystem. I then
>>> > noticed that the performance results started looking like the cache just
>>> > wasn't there, so I unregistered it:
>>> >
>>> > bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
>>> > bcache: Caching disabled for sdc
>>> >
>>> > but now when I try to register it again, I get bcache: error
>>> > opening /dev/sdb: device busy.
>>> >
>>> > I haven't tried rebooting yet but probably will do after I've sent this
>>> > message.
>>> >
>>> > These were the strange results I was getting that prompted me to unmount
>>> > and remount the filesystems.
>>> >
>>> > Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD
>>> > for bcache.
>>> >
>>> > Filesystem Size Used Avail Use% Mounted on
>>> > /dev/bcache0 1.9T 26G 1.8T 2% /data33
>>> > /dev/sdd 1.9T 23G 1.8T 2% /data34
>>> >
>>> > Wrote an 8GB file to each filesystem, then tried reading it back.
>>> > Machine has 8GB RAM.
>>> >
>>> > Speed for random reads from identical SSD = 181.766523 MB/Sec, random
>>> > read test done in the following order with measured speed in MB/Sec. The
>>> > random IO is actually just reading with a 4GB stride, wrapping around at
>>> > the end of the file.
>>> >
>>> > /data33 49
>>> > /data34 114
>>> > /data33 114
>>> > /data33 and /data44 simultaneously 78 and 59 respectively
>>> > /data33 76
>>> > /data33 68
>>> > /data34 114
>>> > /data34 96
>>> > /data34 69
>>> > /data34 74
>>> > /data33 118
>>> > /data33 40
>>> > /data34 91
>>> >
>>> > After remounting both filesystems:
>>> >
>>> > /data34 148
>>> > /data33 120
>>> > /data34 120
>>> > /data33 82
>>> >
>>> >
>>> > After unregistering the cache
>>> >
>>> > /data33 65
>>> > /data34 83
>>> > /data33 99
>>> > /data34 113
>>> >
>>> > Cheers,
>>> > Franco
>>> >
>>> >
>>> > --
>>> > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>>> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-02-16 1:10 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-02-15 5:26 Can't resgister cache Franco
2012-02-15 6:24 ` Kent Overstreet
[not found] ` <CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-02-15 7:16 ` Franco
2012-02-16 1:09 ` Kent Overstreet
[not found] ` <CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-02-16 1:10 ` Kent Overstreet
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).