* [PATCH] sqlite_backup.pl: Create backup of DB using locks to avoid problems with queuerunner.
@ 2014-05-19 8:46 Birin Sanchez
2014-05-19 9:21 ` Birintxo Sánchez
0 siblings, 1 reply; 3+ messages in thread
From: Birin Sanchez @ 2014-05-19 8:46 UTC (permalink / raw)
To: ian.campbell; +Cc: Birin Sanchez, xen-devel
---
config/emesinae.conf | 3 +++
config/examples/test/emesinae.conf | 3 +++
scripts/sqlite_backup.pl | 39 ++++++++++++++++++++++++++++++++++++++
3 files changed, 45 insertions(+)
create mode 100755 scripts/sqlite_backup.pl
diff --git a/config/emesinae.conf b/config/emesinae.conf
index 0aa1b3f..dd58c94 100644
--- a/config/emesinae.conf
+++ b/config/emesinae.conf
@@ -76,3 +76,6 @@ $c{ControlBlacklistPath} = "/etc/emesinae/control.blacklist";
# Severity levels, in decending order of criticality
@{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
$c{DefaultSeverity} = "normal";
+
+# Backup configuration
+$c{BackupDir} = "/var/backups/";
diff --git a/config/examples/test/emesinae.conf b/config/examples/test/emesinae.conf
index 528dcb6..11e4fad 100644
--- a/config/examples/test/emesinae.conf
+++ b/config/examples/test/emesinae.conf
@@ -70,3 +70,6 @@ $c{ControlBlacklistPath} = "/srv/test/etc/control.blacklist";
# Severity levels, in decending order of criticality
@{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
$c{DefaultSeverity} = "normal";
+
+# Backup configuration
+$c{BackupDir} = "/srv/test/var/backups/";
diff --git a/scripts/sqlite_backup.pl b/scripts/sqlite_backup.pl
new file mode 100755
index 0000000..557bc59
--- /dev/null
+++ b/scripts/sqlite_backup.pl
@@ -0,0 +1,39 @@
+#!/usr/bin/perl
+#
+# Creates a backup of the current DB in $c{BackupDir}
+# using locks to avoid race conditions with queuerunner
+
+use strict;
+use warnings;
+
+use Emesinae::Common;
+use Emesinae::Paths;
+use File::Copy;
+use File::Basename;
+
+readconfig();
+
+chdir( $c{BackupDir} ) || die("chdir to dir $c{BackupDir}: $!");
+
+
+my $lock = subsyslock('queuerunner');
+
+my $BackupDB = $c{BackupDir} . basename($c{DB});
+my $BackupDBxz = $c{BackupDir} . basename($c{DB}) . ".xz";
+my $OldBackupDBxz = $c{BackupDir} . basename($c{DB}) . ".old.xz";
+
+if ( -f $BackupDBxz ) {
+ move($BackupDBxz, $OldBackupDBxz) or die ("move failed: $!");
+}
+
+my $dbh = DBI->connect("dbi:SQLite:dbname=$c{DB}","","");
+$dbh->sqlite_backup_to_file( $BackupDB );
+undef $dbh;
+
+my @args = ("xz", "$BackupDB");
+system(@args) == 0
+ or die ("xz compress failed: $?");
+
+subsysunlock($lock);
+
+exit(0)
--
1.8.3.2
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] sqlite_backup.pl: Create backup of DB using locks to avoid problems with queuerunner.
2014-05-19 8:46 [PATCH] sqlite_backup.pl: Create backup of DB using locks to avoid problems with queuerunner Birin Sanchez
@ 2014-05-19 9:21 ` Birintxo Sánchez
2014-05-19 10:51 ` Ian Campbell
0 siblings, 1 reply; 3+ messages in thread
From: Birintxo Sánchez @ 2014-05-19 9:21 UTC (permalink / raw)
To: Birin Sanchez; +Cc: Ian Campbell, xen-devel
Hi Ian,
I forgot to add the runes for the cronjob:
# SQLite backup for xen-devel-bugs DB. Runs everyday at 2:07 to get DB backup
# ready for offsite backup by Citrix (run at 3:30 am)
7 2 * * * /usr/bin/env
PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/
/srv/xen-devel-bugs/lib/emesinae/sqlite_backup.pl
Cheers,
Birin
On 19 May 2014 10:46, Birin Sanchez <birin.sanchez@citrix.com> wrote:
>
> ---
> config/emesinae.conf | 3 +++
> config/examples/test/emesinae.conf | 3 +++
> scripts/sqlite_backup.pl | 39 ++++++++++++++++++++++++++++++++++++++
> 3 files changed, 45 insertions(+)
> create mode 100755 scripts/sqlite_backup.pl
>
> diff --git a/config/emesinae.conf b/config/emesinae.conf
> index 0aa1b3f..dd58c94 100644
> --- a/config/emesinae.conf
> +++ b/config/emesinae.conf
> @@ -76,3 +76,6 @@ $c{ControlBlacklistPath} = "/etc/emesinae/control.blacklist";
> # Severity levels, in decending order of criticality
> @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
> $c{DefaultSeverity} = "normal";
> +
> +# Backup configuration
> +$c{BackupDir} = "/var/backups/";
> diff --git a/config/examples/test/emesinae.conf b/config/examples/test/emesinae.conf
> index 528dcb6..11e4fad 100644
> --- a/config/examples/test/emesinae.conf
> +++ b/config/examples/test/emesinae.conf
> @@ -70,3 +70,6 @@ $c{ControlBlacklistPath} = "/srv/test/etc/control.blacklist";
> # Severity levels, in decending order of criticality
> @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
> $c{DefaultSeverity} = "normal";
> +
> +# Backup configuration
> +$c{BackupDir} = "/srv/test/var/backups/";
> diff --git a/scripts/sqlite_backup.pl b/scripts/sqlite_backup.pl
> new file mode 100755
> index 0000000..557bc59
> --- /dev/null
> +++ b/scripts/sqlite_backup.pl
> @@ -0,0 +1,39 @@
> +#!/usr/bin/perl
> +#
> +# Creates a backup of the current DB in $c{BackupDir}
> +# using locks to avoid race conditions with queuerunner
> +
> +use strict;
> +use warnings;
> +
> +use Emesinae::Common;
> +use Emesinae::Paths;
> +use File::Copy;
> +use File::Basename;
> +
> +readconfig();
> +
> +chdir( $c{BackupDir} ) || die("chdir to dir $c{BackupDir}: $!");
> +
> +
> +my $lock = subsyslock('queuerunner');
> +
> +my $BackupDB = $c{BackupDir} . basename($c{DB});
> +my $BackupDBxz = $c{BackupDir} . basename($c{DB}) . ".xz";
> +my $OldBackupDBxz = $c{BackupDir} . basename($c{DB}) . ".old.xz";
> +
> +if ( -f $BackupDBxz ) {
> + move($BackupDBxz, $OldBackupDBxz) or die ("move failed: $!");
> +}
> +
> +my $dbh = DBI->connect("dbi:SQLite:dbname=$c{DB}","","");
> +$dbh->sqlite_backup_to_file( $BackupDB );
> +undef $dbh;
> +
> +my @args = ("xz", "$BackupDB");
> +system(@args) == 0
> + or die ("xz compress failed: $?");
> +
> +subsysunlock($lock);
> +
> +exit(0)
> --
> 1.8.3.2
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] sqlite_backup.pl: Create backup of DB using locks to avoid problems with queuerunner.
2014-05-19 9:21 ` Birintxo Sánchez
@ 2014-05-19 10:51 ` Ian Campbell
0 siblings, 0 replies; 3+ messages in thread
From: Ian Campbell @ 2014-05-19 10:51 UTC (permalink / raw)
To: Birintxo Sánchez; +Cc: Birin Sanchez, xen-devel
On Mon, 2014-05-19 at 11:21 +0200, Birintxo Sánchez wrote:
> Hi Ian,
>
> I forgot to add the runes for the cronjob:
>
> # SQLite backup for xen-devel-bugs DB. Runs everyday at 2:07 to get DB backup
> # ready for offsite backup by Citrix (run at 3:30 am)
> 7 2 * * * /usr/bin/env
> PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/
> /srv/xen-devel-bugs/lib/emesinae/sqlite_backup.pl
Thanks, I have applied your patch, modifying it to add the new script to
the top-level Makefile and with the following additional changeset on
top.
I also ran the backups a couple of times by hand and verified they
worked, the crontab has been updated.
Thanks!
Ian.
commit 0ade7d795d21ab4729bda6c07fe9f4f169bd7c5f
Author: Ian Campbell <ian.campbell@citrix.com>
Date: Mon May 19 11:45:42 2014 +0100
xen-bugs: Configure database backups
diff --git a/config/examples/xen-bugs.xenproject.org/README b/config/examples/xen-bugs.xenproject.org/README
index f49d934..cf989d0 100644
--- a/config/examples/xen-bugs.xenproject.org/README
+++ b/config/examples/xen-bugs.xenproject.org/README
@@ -15,10 +15,12 @@ $ make
# chmod 1777 /srv/xen-devel-bugs/var/lock
# mkdir /srv/xen-devel-bugs/var/raw /srv/xen-devel-bugs/var/run \
- /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming
+ /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming \
+ /srv/xen-devel-bugs/var/backups
# chown xen-devel-bugs:xen-devel-bugs \
/srv/xen-devel-bugs/var/raw /srv/xen-devel-bugs/var/run \
- /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming
+ /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming \
+ /srv/xen-devel-bugs/var/backups
# cp config/examples/xen-bugs.xenproject.org/emesinae.conf /srv/xen-devel-bugs/etc/
@@ -91,8 +93,8 @@ cron Configuration
==================
# mkdir /srv/xen-devel-bugs/var/log
-# touch /srv/xen-devel-bugs/var/log/queuerunner.log
-# chown xen-devel-bugs:xen-devel-bugs /srv/xen-devel-bugs/var/log/queuerunner.log
+# touch /srv/xen-devel-bugs/var/log/{backups,queuerunner}.log
+# chown xen-devel-bugs:xen-devel-bugs /srv/xen-devel-bugs/var/log/{backups,queuerunner}.log
# crontab -e -u xen-devel-bugs
8<-----------------------------
@@ -100,5 +102,9 @@ SHELL=/bin/sh
# Process queue every 15 minutes
*/15 * * * * /usr/bin/env PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/ /srv/xen-devel-bugs/lib/emesinae/queuerunner.pl >>/srv/xen-devel-bugs/var/log/queuerunner.log
+
+# Backup everyday at 2:07 ready for offsite backup by Citrix (run at 3:30 am)
+7 2 * * * /usr/bin/env PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/ /srv/xen-devel-bugs/lib/emesinae/sqlite_backup.pl >>/srv/xen-devel-bugs/var/log/backups.log
+
8<-----------------------------
diff --git a/config/examples/xen-bugs.xenproject.org/emesinae.conf b/config/examples/xen-bugs.xenproject.org/emesinae.conf
index bf6fcf8..beefa3e 100644
--- a/config/examples/xen-bugs.xenproject.org/emesinae.conf
+++ b/config/examples/xen-bugs.xenproject.org/emesinae.conf
@@ -72,3 +72,6 @@ $c{ControlBlacklistPath} = "/srv/xen-devel-bugs/etc/control.blacklist";
# Severity levels, in decending order of criticality
@{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
$c{DefaultSeverity} = "normal";
+
+# Backup configuration
+$c{BackupDir} = "/srv/xen-devel-bugs/var/backups/";
>
> Cheers,
>
> Birin
>
>
> On 19 May 2014 10:46, Birin Sanchez <birin.sanchez@citrix.com> wrote:
> >
> > ---
> > config/emesinae.conf | 3 +++
> > config/examples/test/emesinae.conf | 3 +++
> > scripts/sqlite_backup.pl | 39 ++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 45 insertions(+)
> > create mode 100755 scripts/sqlite_backup.pl
> >
> > diff --git a/config/emesinae.conf b/config/emesinae.conf
> > index 0aa1b3f..dd58c94 100644
> > --- a/config/emesinae.conf
> > +++ b/config/emesinae.conf
> > @@ -76,3 +76,6 @@ $c{ControlBlacklistPath} = "/etc/emesinae/control.blacklist";
> > # Severity levels, in decending order of criticality
> > @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
> > $c{DefaultSeverity} = "normal";
> > +
> > +# Backup configuration
> > +$c{BackupDir} = "/var/backups/";
> > diff --git a/config/examples/test/emesinae.conf b/config/examples/test/emesinae.conf
> > index 528dcb6..11e4fad 100644
> > --- a/config/examples/test/emesinae.conf
> > +++ b/config/examples/test/emesinae.conf
> > @@ -70,3 +70,6 @@ $c{ControlBlacklistPath} = "/srv/test/etc/control.blacklist";
> > # Severity levels, in decending order of criticality
> > @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
> > $c{DefaultSeverity} = "normal";
> > +
> > +# Backup configuration
> > +$c{BackupDir} = "/srv/test/var/backups/";
> > diff --git a/scripts/sqlite_backup.pl b/scripts/sqlite_backup.pl
> > new file mode 100755
> > index 0000000..557bc59
> > --- /dev/null
> > +++ b/scripts/sqlite_backup.pl
> > @@ -0,0 +1,39 @@
> > +#!/usr/bin/perl
> > +#
> > +# Creates a backup of the current DB in $c{BackupDir}
> > +# using locks to avoid race conditions with queuerunner
> > +
> > +use strict;
> > +use warnings;
> > +
> > +use Emesinae::Common;
> > +use Emesinae::Paths;
> > +use File::Copy;
> > +use File::Basename;
> > +
> > +readconfig();
> > +
> > +chdir( $c{BackupDir} ) || die("chdir to dir $c{BackupDir}: $!");
> > +
> > +
> > +my $lock = subsyslock('queuerunner');
> > +
> > +my $BackupDB = $c{BackupDir} . basename($c{DB});
> > +my $BackupDBxz = $c{BackupDir} . basename($c{DB}) . ".xz";
> > +my $OldBackupDBxz = $c{BackupDir} . basename($c{DB}) . ".old.xz";
> > +
> > +if ( -f $BackupDBxz ) {
> > + move($BackupDBxz, $OldBackupDBxz) or die ("move failed: $!");
> > +}
> > +
> > +my $dbh = DBI->connect("dbi:SQLite:dbname=$c{DB}","","");
> > +$dbh->sqlite_backup_to_file( $BackupDB );
> > +undef $dbh;
> > +
> > +my @args = ("xz", "$BackupDB");
> > +system(@args) == 0
> > + or die ("xz compress failed: $?");
> > +
> > +subsysunlock($lock);
> > +
> > +exit(0)
> > --
> > 1.8.3.2
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2014-05-19 10:51 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-05-19 8:46 [PATCH] sqlite_backup.pl: Create backup of DB using locks to avoid problems with queuerunner Birin Sanchez
2014-05-19 9:21 ` Birintxo Sánchez
2014-05-19 10:51 ` Ian Campbell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.