can't attach caching device to backing device

can't attach caching device to backing device

[Justin Rush]

Using 3.1.0-rc9

I created a cache device on /dev/sdk1
# ./make-bcache -C -b512k -w4k --writeback /dev/sdk1

and a backing device on /dev/md127p1.
./make-bcache -B /dev/md127p1

# ./probe-bcache /dev/sdk1
5f42bdd1-4342-4ab6-8dda-855989a09604: UUID="" TYPE="bcache"

# ./probe-bcache /dev/md127p1
6bbac9a4-afbd-4d40-9562-cdd290622c5d: UUID="" TYPE="bcache"

I then registered them by:
# echo /dev/sdk1 >/sys/fs/bcache/register # echo /dev/md127p1 >/sys/fs/bcache/register

I have /dev/bcache0

When I try to attach the cache, I get this:
# echo "5f42bdd1-4342-4ab6-8dda-855989a09604" >/sys/block/md127/md127p1/bcache/attach
-bash: echo: write error: No such file or directory

I believe this problem is due to the UUIDs for the two devices, but this is just a guess.  The UUID for /dev/sdk1 and /dev/md127p1 doesn't appear to be registering properly as they don't show up in blkid.  I tried specifying the UUID to use with the -U option when creating the devices, but that didn't seem to help.  I do have the udev rules file from the tools in place

Does anyone know what is going on here and how to fix it?

Justin Rush
jarush-Ub+6UvMsOfg@public.gmane.org

Re: can't attach caching device to backing device

[Kent Overstreet]

When you tried to attach the cache, did the file you were echoing to
exist? That looks correct but I'm not sure what else would generate
that error.

For the UUIDs to show up in blkid, blkid itself would have to be
updated to know about the bcache superblock (IIRC it lives in
util-linux). I should probably ping the author and see if that can get
included.

If you have the udev rules and probe-bcache installed, the symlinks
/ought/ to show up when you reboot but I haven't tested that in ages.

I just got the code written so that it ought to be able to work on top
of md devices again, which means I should be able to run it at home
and I'll have to fix probe-bcache if it's broken then :)

On Mon, Nov 14, 2011 at 3:10 PM, Justin Rush <jarush-Ub+6UvMsOfg@public.gmane.org> wrote:
> Using 3.1.0-rc9
>
> I created a cache device on /dev/sdk1
> # ./make-bcache -C -b512k -w4k --writeback /dev/sdk1
>
> and a backing device on /dev/md127p1.
> ./make-bcache -B /dev/md127p1
>
> # ./probe-bcache /dev/sdk1
> 5f42bdd1-4342-4ab6-8dda-855989a09604: UUID="" TYPE="bcache"
>
> # ./probe-bcache /dev/md127p1
> 6bbac9a4-afbd-4d40-9562-cdd290622c5d: UUID="" TYPE="bcache"
>
> I then registered them by:
> # echo /dev/sdk1 >/sys/fs/bcache/register # echo /dev/md127p1 >/sys/fs/bcache/register
>
> I have /dev/bcache0
>
> When I try to attach the cache, I get this:
> # echo "5f42bdd1-4342-4ab6-8dda-855989a09604" >/sys/block/md127/md127p1/bcache/attach
> -bash: echo: write error: No such file or directory
>
> I believe this problem is due to the UUIDs for the two devices, but this is just a guess.  The UUID for /dev/sdk1 and /dev/md127p1 doesn't appear to be registering properly as they don't show up in blkid.  I tried specifying the UUID to use with the -U option when creating the devices, but that didn't seem to help.  I do have the udev rules file from the tools in place
>
> Does anyone know what is going on here and how to fix it?
>
> Justin Rush
> jarush-Ub+6UvMsOfg@public.gmane.org
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

RE: can't attach caching device to backing device

[Justin Rush]

Yes, the file does exist.

Should I try it with the individual devices in the array?

I don't suppose bcache can play with LVM?

- Justin

-----Original Message----- 
From: Kent Overstreet [kent.overstreet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]
Received: Sunday, 20 Nov 2011, 11:31pm
To: Justin Rush [jarush-Ub+6UvMsOfg@public.gmane.org]
CC: linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org]
Subject: Re: can't attach caching device to backing device
When you tried to attach the cache, did the file you were echoing to
exist? That looks correct but I'm not sure what else would generate
that error.

For the UUIDs to show up in blkid, blkid itself would have to be
updated to know about the bcache superblock (IIRC it lives in
util-linux). I should probably ping the author and see if that can get
included.

If you have the udev rules and probe-bcache installed, the symlinks
/ought/ to show up when you reboot but I haven't tested that in ages.

I just got the code written so that it ought to be able to work on top
of md devices again, which means I should be able to run it at home
and I'll have to fix probe-bcache if it's broken then :)

On Mon, Nov 14, 2011 at 3:10 PM, Justin Rush <jarush-Ub+6UvMsOfg@public.gmane.org> wrote:
> Using 3.1.0-rc9
>
> I created a cache device on /dev/sdk1
> # ./make-bcache -C -b512k -w4k --writeback /dev/sdk1
>
> and a backing device on /dev/md127p1.
> ./make-bcache -B /dev/md127p1
>
> # ./probe-bcache /dev/sdk1
> 5f42bdd1-4342-4ab6-8dda-855989a09604: UUID="" TYPE="bcache"
>
> # ./probe-bcache /dev/md127p1
> 6bbac9a4-afbd-4d40-9562-cdd290622c5d: UUID="" TYPE="bcache"
>
> I then registered them by:
> # echo /dev/sdk1 >/sys/fs/bcache/register # echo /dev/md127p1 >/sys/fs/bcache/register
>
> I have /dev/bcache0
>
> When I try to attach the cache, I get this:
> # echo "5f42bdd1-4342-4ab6-8dda-855989a09604" >/sys/block/md127/md127p1/bcache/attach
> -bash: echo: write error: No such file or directory
>
> I believe this problem is due to the UUIDs for the two devices, but this is just a guess.  The UUID for /dev/sdk1 and /dev/md127p1 doesn't appear to be registering properly as they don't show up in blkid.  I tried specifying the UUID to use with the -U option when creating the devices, but that didn't seem to help.  I do have the udev rules file from the tools in place
>
> Does anyone know what is going on here and how to fix it?
>
> Justin Rush
> jarush-Ub+6UvMsOfg@public.gmane.org
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: can't attach caching device to backing device

[Kent Overstreet]

On Mon, Nov 21, 2011 at 01:51:00PM +0000, Justin Rush wrote:
> Yes, the file does exist.

Hmm. Weird. I'll look at the code and see if I can think of anything.

> Should I try it with the individual devices in the array?

Worth a shot, though caching partitions is well tested. 

> I don't suppose bcache can play with LVM?

The version I just pushed should now.
> 
> - Justin
> 
> -----Original Message----- 
> From: Kent Overstreet [kent.overstreet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]
> Received: Sunday, 20 Nov 2011, 11:31pm
> To: Justin Rush [jarush-Ub+6UvMsOfg@public.gmane.org]
> CC: linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org]
> Subject: Re: can't attach caching device to backing device
> When you tried to attach the cache, did the file you were echoing to
> exist? That looks correct but I'm not sure what else would generate
> that error.
> 
> For the UUIDs to show up in blkid, blkid itself would have to be
> updated to know about the bcache superblock (IIRC it lives in
> util-linux). I should probably ping the author and see if that can get
> included.
> 
> If you have the udev rules and probe-bcache installed, the symlinks
> /ought/ to show up when you reboot but I haven't tested that in ages.
> 
> I just got the code written so that it ought to be able to work on top
> of md devices again, which means I should be able to run it at home
> and I'll have to fix probe-bcache if it's broken then :)
> 
> On Mon, Nov 14, 2011 at 3:10 PM, Justin Rush <jarush-Ub+6UvMsOfg@public.gmane.org> wrote:
> > Using 3.1.0-rc9
> >
> > I created a cache device on /dev/sdk1
> > # ./make-bcache -C -b512k -w4k --writeback /dev/sdk1
> >
> > and a backing device on /dev/md127p1.
> > ./make-bcache -B /dev/md127p1
> >
> > # ./probe-bcache /dev/sdk1
> > 5f42bdd1-4342-4ab6-8dda-855989a09604: UUID="" TYPE="bcache"
> >
> > # ./probe-bcache /dev/md127p1
> > 6bbac9a4-afbd-4d40-9562-cdd290622c5d: UUID="" TYPE="bcache"
> >
> > I then registered them by:
> > # echo /dev/sdk1 >/sys/fs/bcache/register # echo /dev/md127p1 >/sys/fs/bcache/register
> >
> > I have /dev/bcache0
> >
> > When I try to attach the cache, I get this:
> > # echo "5f42bdd1-4342-4ab6-8dda-855989a09604" >/sys/block/md127/md127p1/bcache/attach
> > -bash: echo: write error: No such file or directory
> >
> > I believe this problem is due to the UUIDs for the two devices, but this is just a guess.  The UUID for /dev/sdk1 and /dev/md127p1 doesn't appear to be registering properly as they don't show up in blkid.  I tried specifying the UUID to use with the -U option when creating the devices, but that didn't seem to help.  I do have the udev rules file from the tools in place
> >
> > Does anyone know what is going on here and how to fix it?
> >
> > Justin Rush
> > jarush-Ub+6UvMsOfg@public.gmane.org
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: can't attach caching device to backing device

[Justin Rush]

Great, I'll try the new code when I get home since the end goal is to use LVM on top of the raid devices and then present the LVs (or /dev/bcach*) to tcm/lio.

If there is any debug information I can give you, please let me know.

- Justin


-----Original Message-----
From: Kent Overstreet [mailto:koverstreet@google.com] 
Sent: Monday, November 21, 2011 7:54 AM
To: Justin Rush
Cc: kent.overstreet@gmail.com; linux-bcache@vger.kernel.org
Subject: Re: can't attach caching device to backing device

On Mon, Nov 21, 2011 at 01:51:00PM +0000, Justin Rush wrote:
> Yes, the file does exist.

Hmm. Weird. I'll look at the code and see if I can think of anything.

> Should I try it with the individual devices in the array?

Worth a shot, though caching partitions is well tested. 

> I don't suppose bcache can play with LVM?

The version I just pushed should now.
> 
> - Justin
> 
> -----Original Message-----
> From: Kent Overstreet [kent.overstreet@gmail.com]
> Received: Sunday, 20 Nov 2011, 11:31pm
> To: Justin Rush [jarush@epic.com]
> CC: linux-bcache@vger.kernel.org [linux-bcache@vger.kernel.org]
> Subject: Re: can't attach caching device to backing device When you 
> tried to attach the cache, did the file you were echoing to exist? 
> That looks correct but I'm not sure what else would generate that 
> error.
> 
> For the UUIDs to show up in blkid, blkid itself would have to be 
> updated to know about the bcache superblock (IIRC it lives in 
> util-linux). I should probably ping the author and see if that can get 
> included.
> 
> If you have the udev rules and probe-bcache installed, the symlinks 
> /ought/ to show up when you reboot but I haven't tested that in ages.
> 
> I just got the code written so that it ought to be able to work on top 
> of md devices again, which means I should be able to run it at home 
> and I'll have to fix probe-bcache if it's broken then :)
> 
> On Mon, Nov 14, 2011 at 3:10 PM, Justin Rush <jarush@epic.com> wrote:
> > Using 3.1.0-rc9
> >
> > I created a cache device on /dev/sdk1 # ./make-bcache -C -b512k -w4k 
> > --writeback /dev/sdk1
> >
> > and a backing device on /dev/md127p1.
> > ./make-bcache -B /dev/md127p1
> >
> > # ./probe-bcache /dev/sdk1
> > 5f42bdd1-4342-4ab6-8dda-855989a09604: UUID="" TYPE="bcache"
> >
> > # ./probe-bcache /dev/md127p1
> > 6bbac9a4-afbd-4d40-9562-cdd290622c5d: UUID="" TYPE="bcache"
> >
> > I then registered them by:
> > # echo /dev/sdk1 >/sys/fs/bcache/register # echo /dev/md127p1 
> > >/sys/fs/bcache/register
> >
> > I have /dev/bcache0
> >
> > When I try to attach the cache, I get this:
> > # echo "5f42bdd1-4342-4ab6-8dda-855989a09604" 
> > >/sys/block/md127/md127p1/bcache/attach
> > -bash: echo: write error: No such file or directory
> >
> > I believe this problem is due to the UUIDs for the two devices, but 
> > this is just a guess.  The UUID for /dev/sdk1 and /dev/md127p1 
> > doesn't appear to be registering properly as they don't show up in 
> > blkid.  I tried specifying the UUID to use with the -U option when 
> > creating the devices, but that didn't seem to help.  I do have the 
> > udev rules file from the tools in place
> >
> > Does anyone know what is going on here and how to fix it?
> >
> > Justin Rush
> > jarush@epic.com
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe 
> > linux-bcache" in the body of a message to majordomo@vger.kernel.org 
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe 
> linux-bcache" in the body of a message to majordomo@vger.kernel.org 
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: can't attach caching device to backing device

[Justin Rush]

I am running into the same issue with LVM, using the entire raid device, and using a single plain disk... I must be doing something wrong here.  I also tried deleting the writeback device's partition and just using the entire disk.

 ~ # pvcreate /dev/md127
  Physical volume "/dev/md127" successfully created

~ # vgcreate vg_500s_bcache /dev/md127
  Volume group "vg_500s_bcache" successfully created

~ # lvcreate -L50G -nlv_bcache_test vg_500s_bcache
  Logical volume "lv_bcache_test" created

 ~ # make-bcache -B /dev/vg_500s_bcache/lv_bcache_test
UUID:                   0847db15-4d6b-4271-a052-4338752a704f
Set UUID:               06fa14ab-cbbe-4e10-ac5c-f48f92ffea39
nbuckets:               102400
block_size:             8
bucket_size:            1024
nr_in_set:              1
nr_this_dev:            0
first_bucket:           1

 ~ # echo /dev/vg_500s_bcache/lv_bcache_test >/sys/fs/bcache/register
 ~ # echo /dev/sdk1 >/sys/fs/bcache/register

~ # echo "d5cadc1d-0fb6-4be5-bb32-f649c540c03d" >/sys/block/bcache0/bcache/attach
-bash: echo: write error: No such file or directory

Same issue with the device and no partitions:
~ # make-bcache -B /dev/md127
UUID:                   4c113d06-5724-45fa-900d-bead27233de9
Set UUID:               500a22f2-c35a-4031-b5f3-9bf865c941bf
nbuckets:               2861624
block_size:             8
bucket_size:            1024
nr_in_set:              1
nr_this_dev:            0
first_bucket:           1
~ # echo /dev/sdk1 >/sys/fs/bcache/register
 ~ # echo /dev/md127 >/sys/fs/bcache/register
 ~ # probe-bcache /dev/sdk1
d5cadc1d-0fb6-4be5-bb32-f649c540c03d: UUID="" TYPE="bcache"
 ~ # echo "d5cadc1d-0fb6-4be5-bb32-f649c540c03d" >/sys/block/md127/bcache/attach
-bash: echo: write error: No such file or directory

- Justin


-----Original Message-----
From: Justin Rush 
Sent: Monday, November 21, 2011 7:57 AM
To: 'Kent Overstreet'
Cc: kent.overstreet@gmail.com; linux-bcache@vger.kernel.org
Subject: RE: can't attach caching device to backing device

Great, I'll try the new code when I get home since the end goal is to use LVM on top of the raid devices and then present the LVs (or /dev/bcach*) to tcm/lio.

If there is any debug information I can give you, please let me know.

- Justin


-----Original Message-----
From: Kent Overstreet [mailto:koverstreet@google.com]
Sent: Monday, November 21, 2011 7:54 AM
To: Justin Rush
Cc: kent.overstreet@gmail.com; linux-bcache@vger.kernel.org
Subject: Re: can't attach caching device to backing device

On Mon, Nov 21, 2011 at 01:51:00PM +0000, Justin Rush wrote:
> Yes, the file does exist.

Hmm. Weird. I'll look at the code and see if I can think of anything.

> Should I try it with the individual devices in the array?

Worth a shot, though caching partitions is well tested. 

> I don't suppose bcache can play with LVM?

The version I just pushed should now.
> 
> - Justin
> 
> -----Original Message-----
> From: Kent Overstreet [kent.overstreet@gmail.com]
> Received: Sunday, 20 Nov 2011, 11:31pm
> To: Justin Rush [jarush@epic.com]
> CC: linux-bcache@vger.kernel.org [linux-bcache@vger.kernel.org]
> Subject: Re: can't attach caching device to backing device When you 
> tried to attach the cache, did the file you were echoing to exist?
> That looks correct but I'm not sure what else would generate that 
> error.
> 
> For the UUIDs to show up in blkid, blkid itself would have to be 
> updated to know about the bcache superblock (IIRC it lives in 
> util-linux). I should probably ping the author and see if that can get 
> included.
> 
> If you have the udev rules and probe-bcache installed, the symlinks 
> /ought/ to show up when you reboot but I haven't tested that in ages.
> 
> I just got the code written so that it ought to be able to work on top 
> of md devices again, which means I should be able to run it at home 
> and I'll have to fix probe-bcache if it's broken then :)
> 
> On Mon, Nov 14, 2011 at 3:10 PM, Justin Rush <jarush@epic.com> wrote:
> > Using 3.1.0-rc9
> >
> > I created a cache device on /dev/sdk1 # ./make-bcache -C -b512k -w4k 
> > --writeback /dev/sdk1
> >
> > and a backing device on /dev/md127p1.
> > ./make-bcache -B /dev/md127p1
> >
> > # ./probe-bcache /dev/sdk1
> > 5f42bdd1-4342-4ab6-8dda-855989a09604: UUID="" TYPE="bcache"
> >
> > # ./probe-bcache /dev/md127p1
> > 6bbac9a4-afbd-4d40-9562-cdd290622c5d: UUID="" TYPE="bcache"
> >
> > I then registered them by:
> > # echo /dev/sdk1 >/sys/fs/bcache/register # echo /dev/md127p1
> > >/sys/fs/bcache/register
> >
> > I have /dev/bcache0
> >
> > When I try to attach the cache, I get this:
> > # echo "5f42bdd1-4342-4ab6-8dda-855989a09604" 
> > >/sys/block/md127/md127p1/bcache/attach
> > -bash: echo: write error: No such file or directory
> >
> > I believe this problem is due to the UUIDs for the two devices, but 
> > this is just a guess.  The UUID for /dev/sdk1 and /dev/md127p1 
> > doesn't appear to be registering properly as they don't show up in 
> > blkid.  I tried specifying the UUID to use with the -U option when 
> > creating the devices, but that didn't seem to help.  I do have the 
> > udev rules file from the tools in place
> >
> > Does anyone know what is going on here and how to fix it?
> >
> > Justin Rush
> > jarush@epic.com
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe 
> > linux-bcache" in the body of a message to majordomo@vger.kernel.org 
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe 
> linux-bcache" in the body of a message to majordomo@vger.kernel.org 
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: can't attach caching device to backing device

[Brad Campbell]

On 22/11/11 05:31, Justin Rush wrote:
> I am running into the same issue with LVM, using the entire raid device, and using a single plain disk... I must be doing something wrong here.

You are.

>   ~ # pvcreate /dev/md127
>    Physical volume "/dev/md127" successfully created
>
> ~ # vgcreate vg_500s_bcache /dev/md127
>    Volume group "vg_500s_bcache" successfully created
>
> ~ # lvcreate -L50G -nlv_bcache_test vg_500s_bcache
>    Logical volume "lv_bcache_test" created
>
>   ~ # make-bcache -B /dev/vg_500s_bcache/lv_bcache_test
> UUID:                   0847db15-4d6b-4271-a052-4338752a704f
> Set UUID:               06fa14ab-cbbe-4e10-ac5c-f48f92ffea39
> nbuckets:               102400
> block_size:             8
> bucket_size:            1024
> nr_in_set:              1
> nr_this_dev:            0
> first_bucket:           1
>
>   ~ # echo /dev/vg_500s_bcache/lv_bcache_test>/sys/fs/bcache/register
>   ~ # echo /dev/sdk1>/sys/fs/bcache/register
>
> ~ # echo "d5cadc1d-0fb6-4be5-bb32-f649c540c03d">/sys/block/bcache0/bcache/attach
> -bash: echo: write error: No such file or directory


You are trying to attach a cache device directly to a backing device. 
That won't work.

When you register the devices, do :
ls /sys/fs/bcache/

You will see a directory name in there. That is the UUID of the 
"cache-set" you need to attach to your backing device. It is different 
from the cache device itself.

Regards,
Brad