Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Torbjørn]

Hi,

After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already 
exists" after reboot.
The fs is forced read only.
The error does not disappear after additional reboots.

mount|grep sda1
/dev/sda1 on / type btrfs (rw,noatime,compress=lzo,space_cache,subvol=@)
/dev/sda1 on /home type btrfs 
(rw,noatime,compress=lzo,space_cache,subvol=@home)
/dev/sda1 on /mnt/volumes type btrfs (rw,noatime,compress=lzo,space_cache)

Label: 'root'  uuid: 6a681300-faef-4195-a1b3-6699c7a5aa5e
     Total devices 2 FS bytes used 66.87GiB
     devid    1 size 298.09GiB used 124.03GiB path /dev/sda1
     devid    2 size 298.09GiB used 124.03GiB path /dev/sdd1

btrfs-progs v3.19

Any additional info I can provide? Tests you want me to run?

--
Torbjørn


[ 1851.856097] ------------[ cut here ]------------
[ 1851.856166] WARNING: CPU: 4 PID: 350 at fs/btrfs/super.c:260 
__btrfs_abort_transaction+0x54/0x130 [btrfs]()
[ 1851.856255] BTRFS: Transaction aborted (error -17)
[ 1851.856256] Modules linked in: vhost_net vhost macvtap macvlan 
ipmi_devintf ipt_REJECT nf_reject_ipv4 xt_CHECKSUM ip6table_filter 
ip6_tables ebtable_nat ebtables sch_prio bridge stp llc xt_state 
iptable_filter xt_CLASSIFY xt_tcpudp xt_DSCP iptable_mangle iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables dm_crypt netconsole configfs ppdev coretemp kvm_intel 
mac_hid lpc_ich video kvm serio_raw parport_pc ipmi_si tpm_infineon 
ipmi_msghandler aesni_intel aes_x86_64 glue_helper lrw gf128mul 
ablk_helper cryptd nfsd lp auth_rpcgss nfs_acl nfs lockd parport grace 
sunrpc fscache btrfs raid10 raid456 async_raid6_recov async_memcpy 
async_pq async_xor async_tx xor raid6_pq raid0 multipath linear raid1 
hid_generic usbhid hid psmouse ahci e1000e libahci ptp mpt2sas 
raid_class pps_core scsi_transport_sas
[ 1851.856977] CPU: 4 PID: 350 Comm: btrfs-transacti Not tainted 
4.0.0-rc5 #97
[ 1851.857057] Hardware name: To be filled by O.E.M. To be filled by 
O.E.M./P8B-X series, BIOS 2107 05/04/2012
[ 1851.857151]  ffffffffc02ad832 ffff88041795fbf8 ffffffff8176aaed 
0000000000000000
[ 1851.857248]  ffff88041795fc48 ffff88041795fc38 ffffffff8107a0ea 
0000000000000000
[ 1851.857336]  ffff88041a3d49a0 ffff880417cc4000 00000000ffffffef 
ffffffffc02ac0d0
[ 1851.857423] Call Trace:
[ 1851.857466]  [<ffffffff8176aaed>] dump_stack+0x45/0x57
[ 1851.857514]  [<ffffffff8107a0ea>] warn_slowpath_common+0x8a/0xc0
[ 1851.857565]  [<ffffffff8107a166>] warn_slowpath_fmt+0x46/0x50
[ 1851.857618]  [<ffffffffc01ffc84>] 
__btrfs_abort_transaction+0x54/0x130 [btrfs]
[ 1851.857704]  [<ffffffffc021d05c>] 
btrfs_run_delayed_refs.part.65+0x12c/0x2a0 [btrfs]
[ 1851.857792]  [<ffffffffc021f4d1>] ? btrfs_update_root+0xe1/0x270 [btrfs]
[ 1851.857849]  [<ffffffffc021d1e7>] btrfs_run_delayed_refs+0x17/0x20 
[btrfs]
[ 1851.857908]  [<ffffffffc02a51c6>] commit_cowonly_roots+0x27d/0x294 
[btrfs]
[ 1851.857967]  [<ffffffffc022dde1>] 
btrfs_commit_transaction+0x4c1/0xa50 [btrfs]
[ 1851.858053]  [<ffffffffc0229585>] transaction_kthread+0x1b5/0x240 [btrfs]
[ 1851.858110]  [<ffffffffc02293d0>] ? 
btrfs_cleanup_transaction+0x570/0x570 [btrfs]
[ 1851.858192]  [<ffffffff81098ea2>] kthread+0xd2/0xf0
[ 1851.858238]  [<ffffffff81098dd0>] ? kthread_create_on_node+0x180/0x180
[ 1851.858291]  [<ffffffff81772518>] ret_from_fork+0x58/0x90
[ 1851.858340]  [<ffffffff81098dd0>] ? kthread_create_on_node+0x180/0x180
[ 1851.858391] ---[ end trace 7c2776e855cf008f ]---
[ 1851.858437] BTRFS: error (device sda1) in 
btrfs_run_delayed_refs:2790: errno=-17 Object already exists
[ 1851.858524] BTRFS info (device sda1): forced readonly
[ 1851.858571] BTRFS warning (device sda1): Skipping commit of aborted 
transaction.
[ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: 
errno=-17 Object already exists

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Chris Mason]

On Mon, Mar 23, 2015 at 6:35 AM, Torbjørn <lists@skagestad.org> wrote:
> Hi,
> 
> After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already 
> exists" after reboot.
> The fs is forced read only.
> The error does not disappear after additional reboots.
> 
> mount|grep sda1
> /dev/sda1 on / type btrfs 
> (rw,noatime,compress=lzo,space_cache,subvol=@)
> /dev/sda1 on /home type btrfs 
> (rw,noatime,compress=lzo,space_cache,subvol=@home)
> /dev/sda1 on /mnt/volumes type btrfs 
> (rw,noatime,compress=lzo,space_cache)
> 
> Label: 'root'  uuid: 6a681300-faef-4195-a1b3-6699c7a5aa5e
>     Total devices 2 FS bytes used 66.87GiB
>     devid    1 size 298.09GiB used 124.03GiB path /dev/sda1
>     devid    2 size 298.09GiB used 124.03GiB path /dev/sdd1
> 
> btrfs-progs v3.19
> 
> Any additional info I can provide? Tests you want me to run?

Are you seeing this consistently with every boot?

If so, can you please revert this one:

commit ea526d18990018f224e5734748975bea1824545f
Author: Josef Bacik <jbacik@fb.com>
Date:   Fri Mar 13 16:40:45 2015 -0400

    Btrfs: fix ASSERT(list_empty(&cur_trans->dirty_bgs_list)

Anything special about your workload

-chris

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Filipe David Manana]

On Mon, Mar 23, 2015 at 10:35 AM, Torbjørn <lists@skagestad.org> wrote:
> Hi,
>
> After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already exists"
> after reboot.
> The fs is forced read only.
> The error does not disappear after additional reboots.

If you go back to 4.0.0-rc4, does the error still happens?

>
> mount|grep sda1
> /dev/sda1 on / type btrfs (rw,noatime,compress=lzo,space_cache,subvol=@)
> /dev/sda1 on /home type btrfs
> (rw,noatime,compress=lzo,space_cache,subvol=@home)
> /dev/sda1 on /mnt/volumes type btrfs (rw,noatime,compress=lzo,space_cache)
>
> Label: 'root'  uuid: 6a681300-faef-4195-a1b3-6699c7a5aa5e
>     Total devices 2 FS bytes used 66.87GiB
>     devid    1 size 298.09GiB used 124.03GiB path /dev/sda1
>     devid    2 size 298.09GiB used 124.03GiB path /dev/sdd1
>
> btrfs-progs v3.19
>
> Any additional info I can provide? Tests you want me to run?
>
> --
> Torbjørn
>
>
> [ 1851.856097] ------------[ cut here ]------------
> [ 1851.856166] WARNING: CPU: 4 PID: 350 at fs/btrfs/super.c:260
> __btrfs_abort_transaction+0x54/0x130 [btrfs]()
> [ 1851.856255] BTRFS: Transaction aborted (error -17)
> [ 1851.856256] Modules linked in: vhost_net vhost macvtap macvlan
> ipmi_devintf ipt_REJECT nf_reject_ipv4 xt_CHECKSUM ip6table_filter
> ip6_tables ebtable_nat ebtables sch_prio bridge stp llc xt_state
> iptable_filter xt_CLASSIFY xt_tcpudp xt_DSCP iptable_mangle iptable_nat
> nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables
> x_tables dm_crypt netconsole configfs ppdev coretemp kvm_intel mac_hid
> lpc_ich video kvm serio_raw parport_pc ipmi_si tpm_infineon ipmi_msghandler
> aesni_intel aes_x86_64 glue_helper lrw gf128mul ablk_helper cryptd nfsd lp
> auth_rpcgss nfs_acl nfs lockd parport grace sunrpc fscache btrfs raid10
> raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> raid6_pq raid0 multipath linear raid1 hid_generic usbhid hid psmouse ahci
> e1000e libahci ptp mpt2sas raid_class pps_core scsi_transport_sas
> [ 1851.856977] CPU: 4 PID: 350 Comm: btrfs-transacti Not tainted 4.0.0-rc5
> #97
> [ 1851.857057] Hardware name: To be filled by O.E.M. To be filled by
> O.E.M./P8B-X series, BIOS 2107 05/04/2012
> [ 1851.857151]  ffffffffc02ad832 ffff88041795fbf8 ffffffff8176aaed
> 0000000000000000
> [ 1851.857248]  ffff88041795fc48 ffff88041795fc38 ffffffff8107a0ea
> 0000000000000000
> [ 1851.857336]  ffff88041a3d49a0 ffff880417cc4000 00000000ffffffef
> ffffffffc02ac0d0
> [ 1851.857423] Call Trace:
> [ 1851.857466]  [<ffffffff8176aaed>] dump_stack+0x45/0x57
> [ 1851.857514]  [<ffffffff8107a0ea>] warn_slowpath_common+0x8a/0xc0
> [ 1851.857565]  [<ffffffff8107a166>] warn_slowpath_fmt+0x46/0x50
> [ 1851.857618]  [<ffffffffc01ffc84>] __btrfs_abort_transaction+0x54/0x130
> [btrfs]
> [ 1851.857704]  [<ffffffffc021d05c>]
> btrfs_run_delayed_refs.part.65+0x12c/0x2a0 [btrfs]
> [ 1851.857792]  [<ffffffffc021f4d1>] ? btrfs_update_root+0xe1/0x270 [btrfs]
> [ 1851.857849]  [<ffffffffc021d1e7>] btrfs_run_delayed_refs+0x17/0x20
> [btrfs]
> [ 1851.857908]  [<ffffffffc02a51c6>] commit_cowonly_roots+0x27d/0x294
> [btrfs]
> [ 1851.857967]  [<ffffffffc022dde1>] btrfs_commit_transaction+0x4c1/0xa50
> [btrfs]
> [ 1851.858053]  [<ffffffffc0229585>] transaction_kthread+0x1b5/0x240 [btrfs]
> [ 1851.858110]  [<ffffffffc02293d0>] ? btrfs_cleanup_transaction+0x570/0x570
> [btrfs]
> [ 1851.858192]  [<ffffffff81098ea2>] kthread+0xd2/0xf0
> [ 1851.858238]  [<ffffffff81098dd0>] ? kthread_create_on_node+0x180/0x180
> [ 1851.858291]  [<ffffffff81772518>] ret_from_fork+0x58/0x90
> [ 1851.858340]  [<ffffffff81098dd0>] ? kthread_create_on_node+0x180/0x180
> [ 1851.858391] ---[ end trace 7c2776e855cf008f ]---
> [ 1851.858437] BTRFS: error (device sda1) in btrfs_run_delayed_refs:2790:
> errno=-17 Object already exists
> [ 1851.858524] BTRFS info (device sda1): forced readonly
> [ 1851.858571] BTRFS warning (device sda1): Skipping commit of aborted
> transaction.
> [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686:
> errno=-17 Object already exists
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Torbjørn Skagestad]

On 23. mars 2015 13:32, Chris Mason wrote:
>
>
> On Mon, Mar 23, 2015 at 6:35 AM, Torbjørn <lists@skagestad.org> wrote:
>> Hi,
>>
>> After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already 
>> exists" after reboot.
>> The fs is forced read only.
>> The error does not disappear after additional reboots.
>>
>> mount|grep sda1
>> /dev/sda1 on / type btrfs (rw,noatime,compress=lzo,space_cache,subvol=@)
>> /dev/sda1 on /home type btrfs 
>> (rw,noatime,compress=lzo,space_cache,subvol=@home)
>> /dev/sda1 on /mnt/volumes type btrfs 
>> (rw,noatime,compress=lzo,space_cache)
>>
>> Label: 'root'  uuid: 6a681300-faef-4195-a1b3-6699c7a5aa5e
>>     Total devices 2 FS bytes used 66.87GiB
>>     devid    1 size 298.09GiB used 124.03GiB path /dev/sda1
>>     devid    2 size 298.09GiB used 124.03GiB path /dev/sdd1
>>
>> btrfs-progs v3.19
>>
>> Any additional info I can provide? Tests you want me to run?
>
> Are you seeing this consistently with every boot?
>
> If so, can you please revert this one:
>
> commit ea526d18990018f224e5734748975bea1824545f
> Author: Josef Bacik <jbacik@fb.com>
> Date:   Fri Mar 13 16:40:45 2015 -0400
>
>    Btrfs: fix ASSERT(list_empty(&cur_trans->dirty_bgs_list)
>
> Anything special about your workload
>
> -chris
>
I see it consistently with every reboot.
The fs is the root of a server. The fs itself has no special workload 
except being root.

I will try to boot with 4.0.0-rc4 as soon as I get physical access, and 
also revert ea526d18990018f224e5734748975bea1824545f.

Thanks.

--
Torbjørn

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Torbjørn Skagestad]

On 23. mars 2015 13:36, Filipe David Manana wrote:
> On Mon, Mar 23, 2015 at 10:35 AM, Torbjørn <lists@skagestad.org> wrote:
>> Hi,
>>
>> After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already exists"
>> after reboot.
>> The fs is forced read only.
>> The error does not disappear after additional reboots.
> If you go back to 4.0.0-rc4, does the error still happens?

As mentioned in the email to Chris:
I will try to boot with 4.0.0-rc4 as soon as I get physical access, and 
also revert ea526d18990018f224e5734748975bea1824545f.

Thanks.

--
Torbjørn

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Chris Mason]

On Mon, Mar 23, 2015 at 8:53 AM, Torbjørn Skagestad 
<torbjorn@itpas.no> wrote:
> On 23. mars 2015 13:36, Filipe David Manana wrote:
>> On Mon, Mar 23, 2015 at 10:35 AM, Torbjørn <lists@skagestad.org> 
>> wrote:
>>> Hi,
>>> 
>>> After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already 
>>> exists"
>>> after reboot.
>>> The fs is forced read only.
>>> The error does not disappear after additional reboots.
>> If you go back to 4.0.0-rc4, does the error still happens?
> 
> As mentioned in the email to Chris:
> I will try to boot with 4.0.0-rc4 as soon as I get physical access, 
> and also revert ea526d18990018f224e5734748975bea1824545f.

In order to revert ea526d1899, you need to revert this one first:

commit dcdf7f6ddba006f3482ebee73dfa6b75aec5f07b
Author: Josef Bacik <jbacik@fb.com>
Date:   Mon Mar 2 16:37:31 2015 -0500

    Btrfs: prepare block group cache before writing


This second commit relies on the first one, things won't work right 
with only one gone.

-chris

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Torbjørn]

On 23. mars 2015 14:47, Chris Mason wrote:
> On Mon, Mar 23, 2015 at 8:53 AM, Torbjørn Skagestad 
> <torbjorn@itpas.no> wrote:
>> On 23. mars 2015 13:36, Filipe David Manana wrote:
>>> On Mon, Mar 23, 2015 at 10:35 AM, Torbjørn <lists@skagestad.org> wrote:
>>>> Hi,
>>>>
>>>> After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already 
>>>> exists"
>>>> after reboot.
>>>> The fs is forced read only.
>>>> The error does not disappear after additional reboots.
>>> If you go back to 4.0.0-rc4, does the error still happens?
>>
>> As mentioned in the email to Chris:
>> I will try to boot with 4.0.0-rc4 as soon as I get physical access, 
>> and also revert ea526d18990018f224e5734748975bea1824545f.
>
> In order to revert ea526d1899, you need to revert this one first:
>
> commit dcdf7f6ddba006f3482ebee73dfa6b75aec5f07b
> Author: Josef Bacik <jbacik@fb.com>
> Date:   Mon Mar 2 16:37:31 2015 -0500
>
>    Btrfs: prepare block group cache before writing
>
>
> This second commit relies on the first one, things won't work right 
> with only one gone.
>
> -chris
>
>
>
Thanks,

I would most likely have missed that.

--
Torbjørn

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Torbjørn]

On 03/23/2015 02:50 PM, Torbjørn wrote:
> On 23. mars 2015 14:47, Chris Mason wrote:
>> On Mon, Mar 23, 2015 at 8:53 AM, Torbjørn Skagestad 
>> <torbjorn@itpas.no> wrote:
>>> On 23. mars 2015 13:36, Filipe David Manana wrote:
>>>> On Mon, Mar 23, 2015 at 10:35 AM, Torbjørn <lists@skagestad.org> 
>>>> wrote:
>>>>> Hi,
>>>>>
>>>>> After upgrading to 4.0.0-rc5 from 4.0.0-rc4 I see "Object already 
>>>>> exists"
>>>>> after reboot.
>>>>> The fs is forced read only.
>>>>> The error does not disappear after additional reboots.
>>>> If you go back to 4.0.0-rc4, does the error still happens?
>>>
>>> As mentioned in the email to Chris:
>>> I will try to boot with 4.0.0-rc4 as soon as I get physical access, 
>>> and also revert ea526d18990018f224e5734748975bea1824545f.
>>
>> In order to revert ea526d1899, you need to revert this one first:
>>
>> commit dcdf7f6ddba006f3482ebee73dfa6b75aec5f07b
>> Author: Josef Bacik <jbacik@fb.com>
>> Date:   Mon Mar 2 16:37:31 2015 -0500
>>
>>    Btrfs: prepare block group cache before writing
>>
>>
>> This second commit relies on the first one, things won't work right 
>> with only one gone.
>>
>> -chris
>>
>>
>>
> Thanks,
>
> I would most likely have missed that.
>
> -- 
> Torbjørn

Hi,

Reboot with rc-4 did not fix anything. The same "errno=-17 Object 
already exists" was still present.
Reboot with 3.19.0 failed and went straight to initramfs/busybox.
btrfs-zero-log from busybox allowed boot to 3.19.0
Reboot to 4.0.0-rc4 without problems.

touch tmp.file
sync
btrfs fi sync
No issues
Building a new rc-5 with reverts, also no issues.

git log --oneline |head -n 3
dc68208 Revert "Btrfs: fix ASSERT(list_empty(&cur_trans->dirty_bgs_list)"
f84bf90 Revert "Btrfs: prepare block group cache before writing"
bc465aa Linux 4.0-rc5

make clean;make -j9 deb-pkg

Reboot into 4.0.0-rc5 with reverts
Same warning and read only fs.

[Mon Mar 23 16:07:42 2015] Btrfs loaded
[Mon Mar 23 16:07:42 2015] BTRFS: device label root devid 2 transid 
1881030 /dev/sdd1
[Mon Mar 23 16:07:42 2015] BTRFS: device label root devid 1 transid 
1881030 /dev/sda1
[Mon Mar 23 16:07:42 2015] BTRFS info (device sda1): disk space caching 
is enabled
[Mon Mar 23 16:07:44 2015] init: ureadahead main process (373) 
terminated with status 5
[Mon Mar 23 16:07:45 2015] BTRFS info (device sda1): disk space caching 
is enabled
[Mon Mar 23 16:07:45 2015] BTRFS warning (device sda1): block group 
19356712960 has wrong amount of free space
[Mon Mar 23 16:07:45 2015] BTRFS warning (device sda1): failed to load 
free space cache for block group 19356712960, rebuild it now
[Mon Mar 23 16:07:45 2015] BTRFS warning (device sda1): block group 
20430454784 has wrong amount of free space
[Mon Mar 23 16:07:45 2015] BTRFS warning (device sda1): failed to load 
free space cache for block group 20430454784, rebuild it now
[Mon Mar 23 16:07:49 2015] ------------[ cut here ]------------
[Mon Mar 23 16:07:49 2015] WARNING: CPU: 2 PID: 207 at 
fs/btrfs/super.c:260 __btrfs_abort_transaction+0x54/0x130 [btrfs]()
[Mon Mar 23 16:07:49 2015] BTRFS: Transaction aborted (error -17)
[Mon Mar 23 16:07:49 2015] Modules linked in: btrfs raid10 raid456 
async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq 
raid0 multipath linear raid1 hid_generic usbhid hid psmouse e1000e ahci 
libahci ptp mpt2sas pps_core raid_class scsi_transport_sas
[Mon Mar 23 16:07:49 2015] CPU: 2 PID: 207 Comm: kworker/u16:7 Not 
tainted 4.0.0-rc5+ #98
[Mon Mar 23 16:07:49 2015] Hardware name: To be filled by O.E.M. To be 
filled by O.E.M./P8B-X series, BIOS 2107 05/04/2012
[Mon Mar 23 16:07:49 2015] Workqueue: btrfs-extent-refs 
btrfs_extent_refs_helper [btrfs]
[Mon Mar 23 16:07:49 2015]  ffffffffc0583832 ffff88041a3fbc28 
ffffffff8176aaed 0000000000003b5a
[Mon Mar 23 16:07:49 2015]  ffff88041a3fbc78 ffff88041a3fbc68 
ffffffff8107a0ea 0000000000000000
[Mon Mar 23 16:07:49 2015]  ffff88041c9a9210 ffff880418e17800 
00000000ffffffef ffffffffc05820d0
[Mon Mar 23 16:07:49 2015] Call Trace:
[Mon Mar 23 16:07:49 2015]  [<ffffffff8176aaed>] dump_stack+0x45/0x57
[Mon Mar 23 16:07:49 2015]  [<ffffffff8107a0ea>] 
warn_slowpath_common+0x8a/0xc0
[Mon Mar 23 16:07:49 2015]  [<ffffffff8107a166>] warn_slowpath_fmt+0x46/0x50
[Mon Mar 23 16:07:49 2015]  [<ffffffffc04d5c84>] 
__btrfs_abort_transaction+0x54/0x130 [btrfs]
[Mon Mar 23 16:07:49 2015]  [<ffffffffc04f2c0c>] 
btrfs_run_delayed_refs.part.65+0x12c/0x2a0 [btrfs]
[Mon Mar 23 16:07:49 2015]  [<ffffffffc04f2e28>] 
delayed_ref_async_start+0x88/0xa0 [btrfs]
[Mon Mar 23 16:07:49 2015]  [<ffffffffc0534182>] 
normal_work_helper+0xc2/0x2b0 [btrfs]
[Mon Mar 23 16:07:49 2015]  [<ffffffffc05345a2>] 
btrfs_extent_refs_helper+0x12/0x20 [btrfs]
[Mon Mar 23 16:07:49 2015]  [<ffffffff8109326f>] 
process_one_work+0x14f/0x420
[Mon Mar 23 16:07:49 2015]  [<ffffffff81093a28>] worker_thread+0x118/0x510
[Mon Mar 23 16:07:49 2015]  [<ffffffff81093910>] ? 
rescuer_thread+0x3d0/0x3d0
[Mon Mar 23 16:07:49 2015]  [<ffffffff81098ea2>] kthread+0xd2/0xf0
[Mon Mar 23 16:07:49 2015]  [<ffffffff81098dd0>] ? 
kthread_create_on_node+0x180/0x180
[Mon Mar 23 16:07:49 2015]  [<ffffffff81772518>] ret_from_fork+0x58/0x90
[Mon Mar 23 16:07:49 2015]  [<ffffffff81098dd0>] ? 
kthread_create_on_node+0x180/0x180
[Mon Mar 23 16:07:49 2015] ---[ end trace 4cc557b601889d6c ]---
[Mon Mar 23 16:07:49 2015] BTRFS: error (device sda1) in 
btrfs_run_delayed_refs:2790: errno=-17 Object already exists
[Mon Mar 23 16:07:49 2015] BTRFS info (device sda1): forced readonly

--
Torbjørn

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Torbjørn]

Hi,

Just a follow up on this report.

The file system in question is a raid1 across 2x320G old Western Digital 
WD3200KS.
I janked them out of the server to run a fsck on another computer (after 
a proper shutdown).

One of the disks did not get properly detected on the secondary computer.
Hopefully the fsck of the single disk is still of some value to you.

As you can see, there are several issues with the fs.
The system has occasionally had hard reboots.

The fs does not have any real value for me. Everything worth anything is 
backed up.
I'll keep the drive around in case it's of any value for some devs.

As noted before: this (corrupted) fs only get errors when booting into 
4.0-rc5. With 4.0-rc4 or earlier it works as if nothing is wrong.

The output of btrfs check follows

--
Torbjørn


btrfs check /dev/sde1
warning, device 2 is missing
warning devid 2 not found already
Checking filesystem on /dev/sde1
UUID: 6a681300-faef-4195-a1b3-6699c7a5aa5e
checking extents
checking free space cache
Error reading 77403848704, -1
failed to load free space cache for block group 19356712960
Error reading 32565755904, -1
failed to load free space cache for block group 20430454784
Error reading 32435732480, -1
failed to load free space cache for block group 23651680256
Error reading 84039802880, -1
failed to load free space cache for block group 26872905728
Error reading 84059484160, -1
failed to load free space cache for block group 30094131200
Error reading 84058771456, -1
failed to load free space cache for block group 31167873024
Error reading 79972204544, -1
failed to load free space cache for block group 32241614848
Error reading 77436489728, -1
failed to load free space cache for block group 33315356672
Error reading 82679889920, -1
failed to load free space cache for block group 34389098496
Error reading 82680545280, -1
failed to load free space cache for block group 35462840320
Error reading 84741390336, -1
failed to load free space cache for block group 36536582144
Error reading 84741652480, -1
failed to load free space cache for block group 37610323968
Error reading 33868480512, -1
failed to load free space cache for block group 38684065792
Error reading 82676482048, -1
failed to load free space cache for block group 39757807616
Error reading 82707415040, -1
failed to load free space cache for block group 40831549440
Error reading 82711347200, -1
failed to load free space cache for block group 41905291264
Error reading 84232962048, -1
failed to load free space cache for block group 42979033088
Error reading 84232437760, -1
failed to load free space cache for block group 44052774912
Error reading 82706640896, -1
failed to load free space cache for block group 45160071168
Error reading 82698240000, -1
failed to load free space cache for block group 46233812992
Error reading 82922373120, -1
failed to load free space cache for block group 47307554816
Error reading 82698764288, -1
failed to load free space cache for block group 48381296640
Error reading 82699550720, -1
failed to load free space cache for block group 49455038464
Error reading 82922635264, -1
failed to load free space cache for block group 50528780288
Error reading 82695487488, -1
failed to load free space cache for block group 51602522112
Error reading 84738768896, -1
failed to load free space cache for block group 52676263936
Error reading 84531548160, -1
failed to load free space cache for block group 53750005760
Error reading 82701123584, -1
failed to load free space cache for block group 54823747584
Error reading 82701647872, -1
failed to load free space cache for block group 55897489408
Error reading 82702221312, -1
failed to load free space cache for block group 56971231232
Error reading 84713619456, -1
failed to load free space cache for block group 58044973056
Error reading 79986659328, -1
failed to load free space cache for block group 59118714880
Error reading 82703613952, -1
failed to load free space cache for block group 62339940352
Error reading 82703876096, -1
failed to load free space cache for block group 67708649472
Error reading 82704814080, -1
failed to load free space cache for block group 69856133120
Error reading 82701385728, -1
failed to load free space cache for block group 70929874944
Error reading 84714405888, -1
failed to load free space cache for block group 73077358592
Error reading 82705076224, -1
failed to load free space cache for block group 74151100416
Error reading 82706366464, -1
failed to load free space cache for block group 75224842240
Error reading 82923159552, -1
failed to load free space cache for block group 76298584064
Error reading 82714259456, -1
failed to load free space cache for block group 77372325888
Error reading 82708226048, -1
failed to load free space cache for block group 78446067712
Error reading 82693390336, -1
failed to load free space cache for block group 79519809536
Error reading 82945351680, -1
failed to load free space cache for block group 80593551360
Error reading 82699812864, -1
failed to load free space cache for block group 81667293184
Error reading 82633158656, -1
failed to load free space cache for block group 82741035008
Error reading 82922897408, -1
failed to load free space cache for block group 83814776832
Error reading 84204675072, -1
failed to load free space cache for block group 88109744128
Error reading 84204937216, -1
failed to load free space cache for block group 89183485952
Error reading 84531810304, -1
failed to load free space cache for block group 90257227776
Error reading 84220116992, -1
failed to load free space cache for block group 91330969600
Error reading 33929195520, -1
failed to load free space cache for block group 92404711424
Error reading 84532072448, -1
failed to load free space cache for block group 93478453248
Error reading 84051034112, -1
failed to load free space cache for block group 95625936896
Error reading 84532334592, -1
failed to load free space cache for block group 96699678720
Error reading 84689895424, -1
failed to load free space cache for block group 97773420544
Error reading 84206112768, -1
failed to load free space cache for block group 98847162368
Error reading 84546416640, -1
failed to load free space cache for block group 102068387840
Error reading 84203134976, -1
failed to load free space cache for block group 104215871488
Error reading 82711609344, -1
failed to load free space cache for block group 105289613312
Error reading 82713858048, -1
failed to load free space cache for block group 106363355136
Error reading 84686471168, -1
failed to load free space cache for block group 107437096960
Error reading 84207403008, -1
failed to load free space cache for block group 111732064256
Error reading 33932611584, -1
failed to load free space cache for block group 112805806080
Error reading 82922110976, -1
failed to load free space cache for block group 113879547904
Error reading 84547198976, -1
failed to load free space cache for block group 114953289728
Error reading 84711768064, -1
failed to load free space cache for block group 116027031552
Error reading 82993995776, -1
failed to load free space cache for block group 122469482496
Error reading 33966653440, -1
failed to load free space cache for block group 123543224320
Error reading 83973832704, -1
failed to load free space cache for block group 126764449792
Error reading 82697977856, -1
failed to load free space cache for block group 127838191616
Error reading 126775787520, -1
failed to load free space cache for block group 128911933440
Error reading 126776049664, -1
failed to load free space cache for block group 129985675264
Error reading 126776311808, -1
failed to load free space cache for block group 131059417088
Error reading 126776573952, -1
failed to load free space cache for block group 132133158912
Error reading 126776836096, -1
failed to load free space cache for block group 133206900736
Error reading 126777098240, -1
failed to load free space cache for block group 134280642560
Error reading 126777360384, -1
failed to load free space cache for block group 135354384384
Error reading 126777622528, -1
failed to load free space cache for block group 136428126208
Error reading 126777884672, -1
failed to load free space cache for block group 137501868032
Error reading 126778146816, -1
failed to load free space cache for block group 138575609856
Error reading 126778408960, -1
failed to load free space cache for block group 139649351680
Error reading 126778671104, -1
failed to load free space cache for block group 140723093504
Error reading 126778933248, -1
failed to load free space cache for block group 141796835328
Error reading 126779195392, -1
failed to load free space cache for block group 142870577152
Error reading 126779457536, -1
failed to load free space cache for block group 143944318976
Error reading 126779719680, -1
failed to load free space cache for block group 145018060800
Error reading 126779981824, -1
failed to load free space cache for block group 146091802624
Error reading 126780243968, -1
failed to load free space cache for block group 147165544448
Error reading 126780506112, -1
failed to load free space cache for block group 148239286272
Error reading 126780768256, -1
failed to load free space cache for block group 149313028096
Error reading 126781030400, -1
failed to load free space cache for block group 150386769920
Error reading 126781292544, -1
failed to load free space cache for block group 151460511744
Error reading 126781554688, -1
failed to load free space cache for block group 152534253568
Error reading 126781816832, -1
failed to load free space cache for block group 153607995392
Error reading 126782078976, -1
failed to load free space cache for block group 154681737216
Error reading 126782341120, -1
failed to load free space cache for block group 155755479040
Error reading 126782603264, -1
failed to load free space cache for block group 156829220864
Error reading 126782865408, -1
failed to load free space cache for block group 157902962688
checking fs roots
root 615 inode 11074 errors 200, dir isize wrong
root 615 inode 11094 errors 200, dir isize wrong
root 615 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 642 inode 11074 errors 200, dir isize wrong
root 642 inode 11094 errors 200, dir isize wrong
root 643 inode 11074 errors 200, dir isize wrong
root 643 inode 11094 errors 200, dir isize wrong
root 644 inode 11074 errors 200, dir isize wrong
root 644 inode 11094 errors 200, dir isize wrong
root 645 inode 11074 errors 200, dir isize wrong
root 645 inode 11094 errors 200, dir isize wrong
root 646 inode 11074 errors 200, dir isize wrong
root 646 inode 11094 errors 200, dir isize wrong
root 647 inode 11074 errors 200, dir isize wrong
root 647 inode 11094 errors 200, dir isize wrong
root 648 inode 11074 errors 200, dir isize wrong
root 648 inode 11094 errors 200, dir isize wrong
root 654 inode 11074 errors 200, dir isize wrong
root 654 inode 11094 errors 200, dir isize wrong
root 656 inode 11074 errors 200, dir isize wrong
root 656 inode 11094 errors 200, dir isize wrong
root 657 inode 11074 errors 200, dir isize wrong
root 657 inode 11094 errors 200, dir isize wrong
root 658 inode 11074 errors 200, dir isize wrong
root 658 inode 11094 errors 200, dir isize wrong
root 659 inode 11074 errors 200, dir isize wrong
root 659 inode 11094 errors 200, dir isize wrong
root 660 inode 11074 errors 200, dir isize wrong
root 660 inode 11094 errors 200, dir isize wrong
root 663 inode 11074 errors 200, dir isize wrong
root 663 inode 11094 errors 200, dir isize wrong
root 665 inode 11074 errors 200, dir isize wrong
root 665 inode 11094 errors 200, dir isize wrong
root 668 inode 11074 errors 200, dir isize wrong
root 668 inode 11094 errors 200, dir isize wrong
root 670 inode 11074 errors 200, dir isize wrong
root 670 inode 11094 errors 200, dir isize wrong
root 672 inode 11074 errors 200, dir isize wrong
root 672 inode 11094 errors 200, dir isize wrong
root 674 inode 11074 errors 200, dir isize wrong
root 674 inode 11094 errors 200, dir isize wrong
root 676 inode 11074 errors 200, dir isize wrong
root 676 inode 11094 errors 200, dir isize wrong
root 679 inode 11074 errors 200, dir isize wrong
root 679 inode 11094 errors 200, dir isize wrong
root 679 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 682 inode 11074 errors 200, dir isize wrong
root 682 inode 11094 errors 200, dir isize wrong
root 682 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 685 inode 11074 errors 200, dir isize wrong
root 685 inode 11094 errors 200, dir isize wrong
root 685 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 687 inode 11074 errors 200, dir isize wrong
root 687 inode 11094 errors 200, dir isize wrong
root 687 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 689 inode 11074 errors 200, dir isize wrong
root 689 inode 11094 errors 200, dir isize wrong
root 689 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 691 inode 11074 errors 200, dir isize wrong
root 691 inode 11094 errors 200, dir isize wrong
root 691 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 693 inode 11074 errors 200, dir isize wrong
root 693 inode 11094 errors 200, dir isize wrong
root 693 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 695 inode 11074 errors 200, dir isize wrong
root 695 inode 11094 errors 200, dir isize wrong
root 695 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 697 inode 11074 errors 200, dir isize wrong
root 697 inode 11094 errors 200, dir isize wrong
root 697 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 699 inode 11074 errors 200, dir isize wrong
root 699 inode 11094 errors 200, dir isize wrong
root 699 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 701 inode 11074 errors 200, dir isize wrong
root 701 inode 11094 errors 200, dir isize wrong
root 701 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 733 inode 11074 errors 200, dir isize wrong
root 733 inode 11094 errors 200, dir isize wrong
root 733 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 735 inode 11074 errors 200, dir isize wrong
root 735 inode 11094 errors 200, dir isize wrong
root 735 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 737 inode 11074 errors 200, dir isize wrong
root 737 inode 11094 errors 200, dir isize wrong
root 737 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 739 inode 11074 errors 200, dir isize wrong
root 739 inode 11094 errors 200, dir isize wrong
root 739 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 741 inode 11074 errors 200, dir isize wrong
root 741 inode 11094 errors 200, dir isize wrong
root 741 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 743 inode 11074 errors 200, dir isize wrong
root 743 inode 11094 errors 200, dir isize wrong
root 743 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 745 inode 11074 errors 200, dir isize wrong
root 745 inode 11094 errors 200, dir isize wrong
root 745 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 747 inode 11074 errors 200, dir isize wrong
root 747 inode 11094 errors 200, dir isize wrong
root 747 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
root 748 inode 11074 errors 200, dir isize wrong
root 748 inode 11094 errors 200, dir isize wrong
root 748 inode 5297043 errors 2000, link count wrong
     unresolved ref dir 622479 index 2 namelen 32 name 
2d65589066f4dac8c3c5d06bf096dbf6 filetype 0 errors 3, no dir item, no 
dir index
found 72661483941 bytes used err is 1
total csum bytes: 69609632
total tree bytes: 1410691072
total fs tree bytes: 1158287360
total extent tree bytes: 152211456
btree space waste bytes: 334476308
file data blocks allocated: 137857265664
  referenced 92274180096
btrfs-progs v3.19.1

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Chris Mason]

On Mon, Mar 30, 2015 at 1:42 PM, Torbjørn <lists@skagestad.org> wrote:
> Hi,
> 
> Just a follow up on this report.
> 
> The file system in question is a raid1 across 2x320G old Western 
> Digital WD3200KS.
> I janked them out of the server to run a fsck on another computer 
> (after a proper shutdown).
> 
> One of the disks did not get properly detected on the secondary 
> computer.
> Hopefully the fsck of the single disk is still of some value to you.
> 
> As you can see, there are several issues with the fs.
> The system has occasionally had hard reboots.
> 
> The fs does not have any real value for me. Everything worth anything 
> is backed up.
> I'll keep the drive around in case it's of any value for some devs.
> 
> As noted before: this (corrupted) fs only get errors when booting 
> into 4.0-rc5. With 4.0-rc4 or earlier it works as if nothing is wrong.

This is really strange because we also have reports from v3.19 stable 
kernels, but none of the btrfs patches between rc4 and rc5 were tagged 
for stable.

Can I convince you to hammer a bit more on rc4?  I'd like to make sure 
it really was a regression introdcued in rc5.

-chris

Re: Linux 4.0.0-rc5 - [ 1851.858652] BTRFS: error (device sda1) in cleanup_transaction:1686: errno=-17 Object already exists

[Torbjørn]

On 03/31/2015 08:12 PM, Chris Mason wrote:
>
>
> On Mon, Mar 30, 2015 at 1:42 PM, Torbjørn <lists@skagestad.org> wrote:
>> Hi,
>>
>> Just a follow up on this report.
>>
>> The file system in question is a raid1 across 2x320G old Western 
>> Digital WD3200KS.
>> I janked them out of the server to run a fsck on another computer 
>> (after a proper shutdown).
>>
>> One of the disks did not get properly detected on the secondary 
>> computer.
>> Hopefully the fsck of the single disk is still of some value to you.
>>
>> As you can see, there are several issues with the fs.
>> The system has occasionally had hard reboots.
>>
>> The fs does not have any real value for me. Everything worth anything 
>> is backed up.
>> I'll keep the drive around in case it's of any value for some devs.
>>
>> As noted before: this (corrupted) fs only get errors when booting 
>> into 4.0-rc5. With 4.0-rc4 or earlier it works as if nothing is wrong.
>
> This is really strange because we also have reports from v3.19 stable 
> kernels, but none of the btrfs patches between rc4 and rc5 were tagged 
> for stable.
>
> Can I convince you to hammer a bit more on rc4?  I'd like to make sure 
> it really was a regression introdcued in rc5.
>
> -chris
>
>
>
>
Perhaps I was a bit unclear. The error is triggered when booting into 
rc5. If I reset and try to start up rc4 or earlier, it is still there. 
After zero-log I can boot into rc4.

I already did a fresh reinstall to alternative drives. I did not want to 
have the corrupted fs as root.

I'm not sure I can get the second disk in the raid1 to work. It was head 
crashing when trying to attach it for fsck. If I somehow get it back 
online I can do some more testing. Anything in particular?

--
Torbjørn