transaction_manager::new_block() couldn't allocate new block

transaction_manager::new_block() couldn't allocate new block

[Eric Wheeler]

Hello all, 

We are attempting to repair a thin meta volume and get the following 
error after it runs for a while:

~]# thin_repair -V
0.8.5

~]# thin_repair -i /dev/mapper/data-data--pool_tmeta -o /dev/data/tmeta-dest 
truncating metadata device to 4161600 4k blocks
terminate called after throwing an instance of 'std::runtime_error'
  what():  transaction_manager::new_block() couldn't allocate new block
Aborted (core dumped)

How I can troubleshoot this further?

I'm happy to try patches against thin_repair if you would like.  I'm also 
trying thin_dump/thin_restore, so we will see how that goes---but I 
thought you might want to know in case there is a bug in thin_repair that 
could be fixed while I have the metadata in this state.

--
Eric Wheeler

transaction_manager::new_block() couldn't allocate new block

[Nikhil Kshirsagar]

I recollect running into this one a few months ago on 0.8.5 , and i think
Joe fixed this in 0.8.5-1 , please try with the latest version of the pdata
tools.

On Sat, 19 Oct, 2019, 6:08 AM Eric Wheeler, <lvm-devel@lists.ewheeler.net>
wrote:

> Hello all,
>
> We are attempting to repair a thin meta volume and get the following
> error after it runs for a while:
>
> ~]# thin_repair -V
> 0.8.5
>
> ~]# thin_repair -i /dev/mapper/data-data--pool_tmeta -o
> /dev/data/tmeta-dest
> truncating metadata device to 4161600 4k blocks
> terminate called after throwing an instance of 'std::runtime_error'
>   what():  transaction_manager::new_block() couldn't allocate new block
> Aborted (core dumped)
>
> How I can troubleshoot this further?
>
> I'm happy to try patches against thin_repair if you would like.  I'm also
> trying thin_dump/thin_restore, so we will see how that goes---but I
> thought you might want to know in case there is a bug in thin_repair that
> could be fixed while I have the metadata in this state.
>
> --
> Eric Wheeler
>
> --
> lvm-devel mailing list
> lvm-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/lvm-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/lvm-devel/attachments/20191019/d7087621/attachment.htm>

transaction_manager::new_block() couldn't allocate new block

[Eric Wheeler]

On Sat, 19 Oct 2019, Eric Wheeler wrote:

> Hello all, 
> 
> We are attempting to repair a thin meta volume and get the following 
> error after it runs for a while:
> 
> ~]# thin_repair -V
> 0.8.5
> 
> ~]# thin_repair -i /dev/mapper/data-data--pool_tmeta -o /dev/data/tmeta-dest 
> truncating metadata device to 4161600 4k blocks
> terminate called after throwing an instance of 'std::runtime_error'
>   what():  transaction_manager::new_block() couldn't allocate new block
> Aborted (core dumped)
> 
> How I can troubleshoot this further?
> 
> I'm happy to try patches against thin_repair if you would like.  I'm also 
> trying thin_dump/thin_restore, so we will see how that goes

Same problem with thin_dump/restore:

~]# thin_dump --repair /dev/mapper/data-data--pool_tmeta -o /mnt/tmp/tmeta.xml
~]# thin_restore -i /mnt/tmp/tmeta.xml -o /dev/data/tmeta-dest
truncating metadata device to 4161600 4k blocks
Restoring: [====>                                             ] | 11%
transaction_manager::new_block() couldn't allocate new block

I added a print of dev_id in xml_format.cc:parse_device() so maybe I can 
skip the "bad" device, but I'm not really sure what the problem is so that 
might not help.

Suggestions and patches welcome.  I'll post the meta XML when its done 
bzipping (109GB) if that is helpful.

--
Eric Wheeler




> ---but I 
> thought you might want to know in case there is a bug in thin_repair that 
> could be fixed while I have the metadata in this state.
> 
> --
> Eric Wheeler
> 
> --
> lvm-devel mailing list
> lvm-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/lvm-devel
> 

transaction_manager::new_block() couldn't allocate new block

[Eric Wheeler]

On Sun, 20 Oct 2019, Eric Wheeler wrote:
> On Sat, 19 Oct 2019, Eric Wheeler wrote:
> 
> > Hello all, 
> > 
> > We are attempting to repair a thin meta volume and get the following 
> > error after it runs for a while:
> > 
> > ~]# thin_repair -V
> > 0.8.5
> > 
> > ~]# thin_repair -i /dev/mapper/data-data--pool_tmeta -o /dev/data/tmeta-dest 
> > truncating metadata device to 4161600 4k blocks
> > terminate called after throwing an instance of 'std::runtime_error'
> >   what():  transaction_manager::new_block() couldn't allocate new block
> > Aborted (core dumped)
> > 
> > How I can troubleshoot this further?
> > 
> > I'm happy to try patches against thin_repair if you would like.  I'm also 
> > trying thin_dump/thin_restore, so we will see how that goes
> 
> Same problem with thin_dump/restore:
> 
> ~]# thin_dump --repair /dev/mapper/data-data--pool_tmeta -o /mnt/tmp/tmeta.xml
> ~]# thin_restore -i /mnt/tmp/tmeta.xml -o /dev/data/tmeta-dest
> truncating metadata device to 4161600 4k blocks
> Restoring: [====>                                             ] | 11%
> transaction_manager::new_block() couldn't allocate new block
> 
> I added a print of dev_id in xml_format.cc:parse_device() so maybe I can 
> skip the "bad" device, but I'm not really sure what the problem is so that 
> might not help.
> 
> Suggestions and patches welcome.  I'll post the meta XML when its done 
> bzipping (109GB) if that is helpful.

bz# 1763895

Here is the meta XML: 
  https://www.duetsolution.com/static/tmeta.xml.bz2

If someone can reproduce this and propose a fix or patch I would greatly 
appreciate it!

--
Eric Wheeler


> 
> --
> Eric Wheeler
> 
> 
> 
> 
> > ---but I 
> > thought you might want to know in case there is a bug in thin_repair that 
> > could be fixed while I have the metadata in this state.
> > 
> > --
> > Eric Wheeler
> > 
> > --
> > lvm-devel mailing list
> > lvm-devel at redhat.com
> > https://www.redhat.com/mailman/listinfo/lvm-devel
> > 
> 
> --
> lvm-devel mailing list
> lvm-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/lvm-devel
> 
>