[PATCH/RFC] cvsserver: Make configuration way more flexible

[PATCH/RFC] cvsserver: Make configuration way more flexible

[Frank Lichtenheld]

Hi.

This patch series started with the thought that it is really cumbersome
to use the pserver access via git-cvsserver without giving the
nobody user write access to the .git directory itself (especially since
SQLite seems to insist on creating temporary files in the
same directory as the database itself on writes).

This problem itself is easily fixable with an one-line patch
to git-cvsserver that uses a gitcvs.dbdir configuration variable.

I tried to abstract the problem a bit more though and created
means to configure all aspects of the database backend in a
very flexible manner. I would glad about comments on wether
I made my solution overly complex on the way...

Most of the changes are tested intensively with test repositories,
exceptions are noted in the individual patches. More testing welcome
of course.

The documentation updates are not yet complete.

Gruesse,
	Frank Lichtenheld

 Documentation/git-cvsserver.txt |   12 ++++++
 git-cvsserver.perl              |   72 ++++++++++++++++++++++++++++------------
 2 files changed, 63 insertions(+), 21 deletions(-)

[PATCH 1/5] cvsserver: Introduce new state variable 'method'

[Frank Lichtenheld]

$state->{method} contains the CVS access method used,
either 'ext' or 'pserver'

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 git-cvsserver.perl |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 68aa752..e9d489b 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -91,7 +91,9 @@ $log->debug("Temporary directory is '$TEMP_DIR'");
 # if we are called with a pserver argument,
 # deal with the authentication cat before entering the
 # main loop
+$state->{method} = 'ext';
 if (@ARGV && $ARGV[0] eq 'pserver') {
+    $state->{method} = 'pserver';
     my $line = <STDIN>; chomp $line;
     unless( $line eq 'BEGIN AUTH REQUEST') {
        die "E Do not understand $line - expecting BEGIN AUTH REQUEST\n";
@@ -1026,7 +1028,7 @@ sub req_ci
 
     $log->info("req_ci : " . ( defined($data) ? $data : "[NULL]" ));
 
-    if ( @ARGV && $ARGV[0] eq 'pserver')
+    if ( $state->{method} eq 'pserver')
     {
         print "error 1 pserver access cannot commit\n";
         exit;
-- 
1.5.0.3

[PATCH 2/5] cvsserver: Handle three part keys in git config correctly

[Frank Lichtenheld]

This is intended to be used in the form gitcvs.<method>.<var>
but this patch doesn't introduce any users yet.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 git-cvsserver.perl |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index e9d489b..4edb796 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -183,8 +183,12 @@ sub req_Root
     }
     foreach my $line ( @gitvars )
     {
-        next unless ( $line =~ /^(.*?)\.(.*?)=(.*)$/ );
-        $cfg->{$1}{$2} = $3;
+        next unless ( $line =~ /^(.*?)\.(.*?)(?:\.(.*?))?=(.*)$/ );
+        unless ($3) {
+            $cfg->{$1}{$2} = $4;
+        } else {
+            $cfg->{$1}{$2}{$3} = $4;
+        }
     }
 
     unless ( defined ( $cfg->{gitcvs}{enabled} ) and $cfg->{gitcvs}{enabled} =~ /^\s*(1|true|yes)\s*$/i )
-- 
1.5.0.3

[PATCH 3/5] cvsserver: Allow to override the configuration per access method

[Frank Lichtenheld]

Allow to override the gitcvs.enabled and gitcvs.logfile configuration
variables for each access method (i.e. "ext" or "pserver") in the
form gitcvs.<method>.<var>

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 Documentation/git-cvsserver.txt |   12 ++++++++++++
 git-cvsserver.perl              |   10 +++++++---
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
index 85d0950..6904aad 100644
--- a/Documentation/git-cvsserver.txt
+++ b/Documentation/git-cvsserver.txt
@@ -68,6 +68,18 @@ Note: you need to ensure each user that is going to invoke git-cvsserver has
 write access to the log file and to the git repository. When offering anon
 access via pserver, this means that the nobody user should have write access
 to at least the sqlite database at the root of the repository.
+
+Both configuration variables can also be overriden for a specific method of
+access. Valid method names are "ext" (for SSH access) and "pserver". The
+following example configuration would disable pserver access while still
+allowing access over SSH.
+------
+   [gitcvs]
+        enabled=0
+
+   [gitcvs "ext"]
+        enabled=1
+------
 --
 3. On the client machine you need to set the following variables.
    CVSROOT should be set as per normal, but the directory should point at the
diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 4edb796..5d2b6f3 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -191,7 +191,10 @@ sub req_Root
         }
     }
 
-    unless ( defined ( $cfg->{gitcvs}{enabled} ) and $cfg->{gitcvs}{enabled} =~ /^\s*(1|true|yes)\s*$/i )
+    unless ( ($cfg->{gitcvs}{$state->{method}}{enabled}
+	      and $cfg->{gitcvs}{$state->{method}}{enabled} =~ /^\s*(1|true|yes)\s*$/i)
+	     or ($cfg->{gitcvs}{enabled}
+	      and $cfg->{gitcvs}{enabled} =~ /^\s*(1|true|yes)\s*$/i) )
     {
         print "E GITCVS emulation needs to be enabled on this repo\n";
         print "E the repo config file needs a [gitcvs] section added, and the parameter 'enabled' set to 1\n";
@@ -200,9 +203,10 @@ sub req_Root
         return 0;
     }
 
-    if ( defined ( $cfg->{gitcvs}{logfile} ) )
+    my $logfile = $cfg->{gitcvs}{$state->{method}}{logfile} || $cfg->{gitcvs}{logfile};
+    if ( $logfile )
     {
-        $log->setfile($cfg->{gitcvs}{logfile});
+        $log->setfile($logfile);
     } else {
         $log->nofile();
     }
-- 
1.5.0.3

[PATCH 4/5] cvsserver: Make the database backend configurable

[Frank Lichtenheld]

Make all the different parts of the database backend connection
configurable. This adds the following string configuration variables:
- gitcvs.dbdriver
- gitcvs.dbname
- gitcvs.dbuser
- gitcvs.dbpass
The default values emulate the current behavior exactly for
backwards compatibility.
All configuration variables can also be specified for a specific
access method (i.e. in the form gitcvs.<method>.<var>)

The dbdriver/dbuser/dbpass variables are added for completness.
No other backend than SQLite is tested yet.
The dbname variable on the other hand is useful with this backend
already (to not discriminate against other possible backends
it was not splitted in dbdir and dbfile).

Both dbname and dbuser support dynamic variable substitution where
the available variables are:
%m -- the CVS 'module' (i.e. GIT 'head') worked on
%a -- CVS access method used (i.e. 'ext' or 'pserver')
%u -- User name of the user invoking git-cvsserver
%G -- .git directory name
%g -- .git directory name, mangled to be used in a filename,
      currently this substitutes all chars except for [\w.-]
      with '_'

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 git-cvsserver.perl |   40 ++++++++++++++++++++++++++++++++++------
 1 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 5d2b6f3..6d10aa3 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -2141,19 +2141,33 @@ sub new
 
     bless $self, $class;
 
-    $self->{dbdir} = $config . "/";
-    die "Database dir '$self->{dbdir}' isn't a directory" unless ( defined($self->{dbdir}) and -d $self->{dbdir} );
-
     $self->{module} = $module;
-    $self->{file} = $self->{dbdir} . "/gitcvs.$module.sqlite";
-
     $self->{git_path} = $config . "/";
 
     $self->{log} = $log;
 
     die "Git repo '$self->{git_path}' doesn't exist" unless ( -d $self->{git_path} );
 
-    $self->{dbh} = DBI->connect("dbi:SQLite:dbname=" . $self->{file},"","");
+    $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
+        $cfg->{gitcvs}{dbdriver} || "dbi:SQLite";
+    $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
+        $cfg->{gitcvs}{dbname} || "%Ggitcvs.%m.sqlite";
+    $self->{dbuser} = $cfg->{gitcvs}{$state->{method}}{dbuser} ||
+        $cfg->{gitcvs}{dbuser} || "";
+    $self->{dbpass} = $cfg->{gitcvs}{$state->{method}}{dbpass} ||
+        $cfg->{gitcvs}{dbpass} || "";
+    my %mapping = ( m => $module,
+                    a => $state->{method},
+                    u => getlogin || getpwuid($<) || $<,
+                    G => $self->{git_path},
+                    g => mangle_dirname($self->{git_path}),
+                    );
+    $self->{dbname} =~ s/%([mauGg])/$mapping{$1}/eg;
+    $self->{dbuser} =~ s/%([mauGg])/$mapping{$1}/eg;
+
+    $self->{dbh} = DBI->connect("$self->{dbdriver}:dbname=$self->{dbname}",
+                                $self->{dbuser},
+                                $self->{dbpass});
 
     $self->{tables} = {};
     foreach my $table ( $self->{dbh}->tables )
@@ -2857,5 +2871,19 @@ sub safe_pipe_capture {
     return wantarray ? @output : join('',@output);
 }
 
+=head2 mangle_dirname
+
+create a string from a directory name that is suitable to use as
+part of a filename, mainly by converting all chars except \w.- to _
+
+=cut
+sub mangle_dirname {
+    my $dirname = shift;
+    return unless defined $dirname;
+
+    $dirname =~ s/[^\w.-]/_/g;
+
+    return $dirname;
+}
 
 1;
-- 
1.5.0.3

Re: [PATCH 4/5] cvsserver: Make the database backend configurable

[Martin Langhoff]

On 3/20/07, Frank Lichtenheld <frank@lichtenheld.de> wrote:
> Make all the different parts of the database backend connection
> configurable. This adds the following string configuration variables:

Nice. I guess the hard part of this is going to be creating DB schemas
that are reasonably portable. The SQL we use is as vanilla as it gets
;-)

> Both dbname and dbuser support dynamic variable substitution where
> the available variables are:
> %m -- the CVS 'module' (i.e. GIT 'head') worked on
> %a -- CVS access method used (i.e. 'ext' or 'pserver')
> %u -- User name of the user invoking git-cvsserver
> %G -- .git directory name
> %g -- .git directory name, mangled to be used in a filename,
>       currently this substitutes all chars except for [\w.-]
>       with '_'

It's missing from the POD though ;-)

Good to see patches coming to cvsserver -- I haven't been able to do
much on it lately, and my pet projects are pretty hard. If anyone
cares, they are:

 - mimic CVS branch support
 - allow skewing version numbers to match an existing repo

with those 2 in place, we'd have a means of applying a "vampire tap"
to an existing cvs server and take over without anyone noticing. But
tehy are both hard, hard hard.

cheers.


martin

Re: [PATCH 4/5] cvsserver: Make the database backend configurable

[Frank Lichtenheld]

On Tue, Mar 20, 2007 at 07:47:12AM +1200, Martin Langhoff wrote:
> On 3/20/07, Frank Lichtenheld <frank@lichtenheld.de> wrote:
> >Make all the different parts of the database backend connection
> >configurable. This adds the following string configuration variables:
> 
> Nice. I guess the hard part of this is going to be creating DB schemas
> that are reasonably portable. The SQL we use is as vanilla as it gets
> ;-)

I've now actually made a quick test to see how we do when using other
backends (with PostgreSQL 8.2, will also do one with MySQL later).

Some problems that I saw:

 - It would probably cool to be able to tell git-cvsserver that it
   should use only one database for all modules (i.e. git branches)
   This way one doesn't need to give the users database creation
   privileges. Of course pre-creating all databases possibly ever needed
   is possible but somewhat cumbersome.
 
 - DBI->tables seems to be a portability problem. e.g. with SQLite
   it returns "head", "commitmsgs", etc; with PostgreSQL it returns
   public.head, public.commitmsgs, etc. The output of MySQL might
   be different, too.

> >Both dbname and dbuser support dynamic variable substitution where
> >the available variables are:
> >%m -- the CVS 'module' (i.e. GIT 'head') worked on
> >%a -- CVS access method used (i.e. 'ext' or 'pserver')
> >%u -- User name of the user invoking git-cvsserver
> >%G -- .git directory name
> >%g -- .git directory name, mangled to be used in a filename,
> >      currently this substitutes all chars except for [\w.-]
> >      with '_'
> 
> It's missing from the POD though ;-)

You mean the asciidoc, right?

And yeah, I know. Writing English documentation is not actually one of my
preferred occupations :/ Will do it, though.

[...]
> with those 2 in place, we'd have a means of applying a "vampire tap"
> to an existing cvs server and take over without anyone noticing. But
> tehy are both hard, hard hard.

Indeed.

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

Re: [PATCH 4/5] cvsserver: Make the database backend configurable

[Frank Lichtenheld]

On Fri, Mar 23, 2007 at 04:17:58PM +0100, Frank Lichtenheld wrote:
> On Tue, Mar 20, 2007 at 07:47:12AM +1200, Martin Langhoff wrote:
> > On 3/20/07, Frank Lichtenheld <frank@lichtenheld.de> wrote:
> > >Make all the different parts of the database backend connection
> > >configurable. This adds the following string configuration variables:
> > 
> > Nice. I guess the hard part of this is going to be creating DB schemas
> > that are reasonably portable. The SQL we use is as vanilla as it gets
> > ;-)
> 
> I've now actually made a quick test to see how we do when using other
> backends (with PostgreSQL 8.2, will also do one with MySQL later).

Done the MySQL tests, too.

> Some problems that I saw:
> 
>  - It would probably cool to be able to tell git-cvsserver that it
>    should use only one database for all modules (i.e. git branches)
>    This way one doesn't need to give the users database creation
>    privileges. Of course pre-creating all databases possibly ever needed
>    is possible but somewhat cumbersome.
>  
>  - DBI->tables seems to be a portability problem. e.g. with SQLite
>    it returns "head", "commitmsgs", etc; with PostgreSQL it returns
>    public.head, public.commitmsgs, etc. The output of MySQL might
>    be different, too.

It is `head`, and `revision`. Fun ;)
Why no etc.? Because pretty much every other used SQL command (than these
two "create table") fails with syntax errors. Not that I actually expected
anything else from MySQL...

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

[PATCH 5/5] cvsserver: Abort if connect to database fails

[Frank Lichtenheld]

Currently all calls to the database backend make no
error checking or handling at all. At least abort
if the connection to the database failed since
there is really no way we could do anything useful
after that.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 git-cvsserver.perl |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 6d10aa3..941a91b 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -2168,6 +2168,7 @@ sub new
     $self->{dbh} = DBI->connect("$self->{dbdriver}:dbname=$self->{dbname}",
                                 $self->{dbuser},
                                 $self->{dbpass});
+    die "Error connecting to database\n" unless defined $self->{dbh};
 
     $self->{tables} = {};
     foreach my $table ( $self->{dbh}->tables )
-- 
1.5.0.3

[PATCH] cvsserver: Use DBI->table_info instead of DBI->tables

[Frank Lichtenheld]

DBI->table_info is portable across different DBD backends,
DBI->tables is not.

Limit the output to objects of type TABLE.
---
 git-cvsserver.perl |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

 Obviously to be applied on top of my previous
 patch series.
 
 With this patch I was able to use DBD::Pg as backend.
 It is not very comfortable because of the "one db for
 each module" problem, but at least it works.

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 941a91b..5532ae7 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -2171,10 +2171,8 @@ sub new
     die "Error connecting to database\n" unless defined $self->{dbh};
 
     $self->{tables} = {};
-    foreach my $table ( $self->{dbh}->tables )
+    foreach my $table ( keys %{$self->{dbh}->table_info(undef,undef,undef,'TABLE')->fetchall_hashref('TABLE_NAME')} )
     {
-        $table =~ s/^"//;
-        $table =~ s/"$//;
         $self->{tables}{$table} = 1;
     }
 
-- 
1.5.0.3

Re: [PATCH] cvsserver: Use DBI->table_info instead of DBI->tables

[Frank Lichtenheld]

On Sat, Mar 31, 2007 at 03:57:47PM +0200, Frank Lichtenheld wrote:
> DBI->table_info is portable across different DBD backends,
> DBI->tables is not.
> 
> Limit the output to objects of type TABLE.
> ---

I just noticed I forgot to add the
"Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>"

Does one resend the patch in such cases?

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

[PATCH 0/3] cvsserver: small corrections and bring documentation up to speed

[Frank Lichtenheld]

This patch series consists mainly of documentation updates for the
new features I introduced in my previous series. With all these
patches applied I would consider this topic in a shape so that it
could be included in an official release. It still contains
some experimental stuff but since the user has to enable that himself
and I put a "here be dragons" in the documentation I see no problem
with that.

Proofreading of patch 3 by someone that actually speaks English would
be greatly appreciated.

 Documentation/git-cvsserver.txt |   97 +++++++++++++++++++++++++++++++++++-----
 git-cvsserver.perl              |    7 +-
 2 files changed, 91 insertions(+), 13 deletions(-)

[PATCH 1/3] cvsserver: small corrections to asciidoc documentation

[Frank Lichtenheld]

Fix a typo: s/Not/Note/

Some formating fixes: Use ` ` syntax for all filenames and
' ' syntax for all commandline switches.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 Documentation/git-cvsserver.txt |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
index 6904aad..2cf8153 100644
--- a/Documentation/git-cvsserver.txt
+++ b/Documentation/git-cvsserver.txt
@@ -122,12 +122,12 @@ To get a checkout with the Eclipse CVS client:
 Protocol notes: If you are using anonymous access via pserver, just select that.
 Those using SSH access should choose the 'ext' protocol, and configure 'ext'
 access on the Preferences->Team->CVS->ExtConnection pane. Set CVS_SERVER to
-'git-cvsserver'. Not that password support is not good when using 'ext',
+'git-cvsserver'. Note that password support is not good when using 'ext',
 you will definitely want to have SSH keys setup.
 
 Alternatively, you can just use the non-standard extssh protocol that Eclipse
 offer. In that case CVS_SERVER is ignored, and you will have to replace
-the cvs utility on the server with git-cvsserver or manipulate your .bashrc
+the cvs utility on the server with git-cvsserver or manipulate your `.bashrc`
 so that calling 'cvs' effectively calls git-cvsserver.
 
 Clients known to work
@@ -146,9 +146,9 @@ checkout, diff, status, update, log, add, remove, commit.
 Legacy monitoring operations are not supported (edit, watch and related).
 Exports and tagging (tags and branches) are not supported at this stage.
 
-The server should set the -k mode to binary when relevant, however,
+The server should set the '-k' mode to binary when relevant, however,
 this is not really implemented yet. For now, you can force the server
-to set `-kb` for all files by setting the `gitcvs.allbinary` config
+to set '-kb' for all files by setting the `gitcvs.allbinary` config
 variable. In proper GIT tradition, the contents of the files are
 always respected. No keyword expansion or newline munging is supported.
 
-- 
1.5.1

[PATCH 2/3] cvsserver: Corrections to the database backend configuration

[Frank Lichtenheld]

Don't include the scheme name in gitcvs.dbdriver, it is
always 'dbi' anyway.

Don't allow ':' in driver names nor ';' in database names for
sanity reasons.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 git-cvsserver.perl |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

 I wasn't sure whether I should send this as a new patch or as a new version of my
 older one?

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 5532ae7..7fe7949 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -2149,7 +2149,7 @@ sub new
     die "Git repo '$self->{git_path}' doesn't exist" unless ( -d $self->{git_path} );
 
     $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
-        $cfg->{gitcvs}{dbdriver} || "dbi:SQLite";
+        $cfg->{gitcvs}{dbdriver} || "SQLite";
     $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
         $cfg->{gitcvs}{dbname} || "%Ggitcvs.%m.sqlite";
     $self->{dbuser} = $cfg->{gitcvs}{$state->{method}}{dbuser} ||
@@ -2165,7 +2165,9 @@ sub new
     $self->{dbname} =~ s/%([mauGg])/$mapping{$1}/eg;
     $self->{dbuser} =~ s/%([mauGg])/$mapping{$1}/eg;
 
-    $self->{dbh} = DBI->connect("$self->{dbdriver}:dbname=$self->{dbname}",
+    die "Invalid char ':' in dbdriver" if $self->{dbdriver} =~ /:/;
+    die "Invalid char ';' in dbname" if $self->{dbname} =~ /;/;
+    $self->{dbh} = DBI->connect("dbi:$self->{dbdriver}:dbname=$self->{dbname}",
                                 $self->{dbuser},
                                 $self->{dbpass});
     die "Error connecting to database\n" unless defined $self->{dbh};
-- 
1.5.1

[PATCH 3/3] cvsserver: Add asciidoc documentation for new database backend configuration

[Frank Lichtenheld]

Documents the new configuration variables and the variable
substitution mechanism.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 Documentation/git-cvsserver.txt |   87 +++++++++++++++++++++++++++++++++++++--
 1 files changed, 83 insertions(+), 4 deletions(-)

 Proofreading would be greatly appreciated.

diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
index 2cf8153..efc7fa6 100644
--- a/Documentation/git-cvsserver.txt
+++ b/Documentation/git-cvsserver.txt
@@ -65,11 +65,12 @@ env variable, you can rename git-cvsserver to cvs.
 
 ------
 Note: you need to ensure each user that is going to invoke git-cvsserver has
-write access to the log file and to the git repository. When offering anon
-access via pserver, this means that the nobody user should have write access
-to at least the sqlite database at the root of the repository.
+write access to the log file and to the database (see
+<<dbbackend,Database Backend>>. If you want to offer write access over
+SSH, the users of course also need write access to the git repository itself.
 
-Both configuration variables can also be overriden for a specific method of
+[[configaccessmethod]]
+All configuration variables can also be overriden for a specific method of
 access. Valid method names are "ext" (for SSH access) and "pserver". The
 following example configuration would disable pserver access while still
 allowing access over SSH.
@@ -105,6 +106,84 @@ Example:
      cvs co -d project-master master
 ------
 
+[[dbbackend]]
+Database Backend
+----------------
+
+git-cvsserver uses one database per git head (i.e. CVS module) to
+store information about the repository for faster access. The
+database doesn't contain any persitent data and can be completly
+regenerated from the git repository at any time. The database
+needs to be updated (i.e. written to) after every commit. That
+means that even if you offer only read access (e.g. by using
+the pserver method), git-cvsserver should have write access to
+the database to work reliably (otherwise you need to make sure
+that the database if up-to-date all the time git-cvsserver is run).
+
+By default it uses SQLite databases in the git directory, named
+`gitcvs.<module_name>.sqlite`. Note that the SQLite backend creates
+temporary files in the same directory as the database file on
+write so it might not be enough to grant the users using
+git-cvsserver write access to the database file without granting
+them also write access to the directory.
+
+You can configure the database backend with the following
+configuration variables:
+
+Configuring database backend
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+git-cvsserver uses the Perl DBI module. Please also read
+its documentation if changing these variables, especially
+about `DBI->connect()`.
+
+gitcvs.dbname::
+	Database name. The exact meaning depends on the
+	used database driver, for SQLite this is a filename.
+	Supports variable substitution (see below). May
+	not contain semicolons (`;`).
+	Default: '%Ggitcvs.%m.sqlite'
+
+gitcvs.dbdriver::
+	Used DBI driver. You can specify any available driver
+	for this here, but it might not work. cvsserver is tested
+	with 'DBD::SQLite', reported to work with
+	'DBD::Pg', and reported *not* to work with 'DBD::mysql'.
+	Please regard this as an experimental feature. May not
+	contain double colons (`:`).
+	Default: 'SQLite'
+
+gitcvs.dbuser::
+	Database user. Only useful if setting `dbdriver`, since
+	SQLite has no concept of database users. Supports variable
+	substitution (see below).
+
+gitcvs.dbpass::
+	Database password.  Only useful if setting `dbdriver`, since
+	SQLite has no concept of database passwords.
+
+All variables can also be set per access method, see <<configaccessmethod,above>>.
+
+Variable substitution
+^^^^^^^^^^^^^^^^^^^^^
+In `dbdriver` and `dbuser` you can use the following variables:
+
+%G::
+	git directory name
+%g::
+	git directory name, where all characters except for
+	alpha-numeric ones, `.`, and `-` are replaced with
+	`_` (this should make it easier to use the directory
+	name in a filename if wanted)
+%m::
+	CVS module/git head name
+%a::
+	access method (one of "ext" or "pserver")
+%u::
+	Name of the user running git-cvsserver.
+	If no name can be determined, the
+	numeric uid is used.
+
 Eclipse CVS Client Notes
 ------------------------
 
-- 
1.5.1

Re: [PATCH 3/3] cvsserver: Add asciidoc documentation for new database backend configuration

[Junio C Hamano]

Frank Lichtenheld <frank@lichtenheld.de> writes:

> +[[dbbackend]]
> +Database Backend
> +----------------
> +
> +git-cvsserver uses one database per git head (i.e. CVS module) to

Probably "git branch" is easier to understand than "git head".

> +store information about the repository for faster access. The
> +database doesn't contain any persitent data and can be completly
> +regenerated from the git repository at any time. The database
> +needs to be updated (i.e. written to) after every commit. That
> +means that even if you offer only read access (e.g. by using
> +the pserver method), git-cvsserver should have write access to
> +the database to work reliably (otherwise you need to make sure
> +that the database if up-to-date all the time git-cvsserver is run).

This rationale for db update is a bit hard to understand.
Immediately saying that you need a database update "after every
commit", you say that read-only access still need it.

I think the situation where database update is needed is when a
commit on the branch that has not been given to any cvs client
is given out for the first time, and when somebody adds new
commits from git side, the cvsserver session that serves the
commit for the first time needs to record the branch
information.

Re: [PATCH 3/3] cvsserver: Add asciidoc documentation for new database backend configuration

[Frank Lichtenheld]

On Sun, Apr 08, 2007 at 01:44:50AM -0700, Junio C Hamano wrote:
> Frank Lichtenheld <frank@lichtenheld.de> writes:
> 
> > +[[dbbackend]]
> > +Database Backend
> > +----------------
> > +
> > +git-cvsserver uses one database per git head (i.e. CVS module) to
> 
> Probably "git branch" is easier to understand than "git head".

Hmm, should I replace "git head" with "git branch" everywhere
in the document or does it make sense sometimes to use "head"?
Because there are already several places in the current documentation
where head is used.

> > +store information about the repository for faster access. The
> > +database doesn't contain any persitent data and can be completly
> > +regenerated from the git repository at any time. The database
> > +needs to be updated (i.e. written to) after every commit. That
> > +means that even if you offer only read access (e.g. by using
> > +the pserver method), git-cvsserver should have write access to
> > +the database to work reliably (otherwise you need to make sure
> > +that the database if up-to-date all the time git-cvsserver is run).
> 
> This rationale for db update is a bit hard to understand.
> Immediately saying that you need a database update "after every
> commit", you say that read-only access still need it.
> 
> I think the situation where database update is needed is when a
> commit on the branch that has not been given to any cvs client
> is given out for the first time, and when somebody adds new
> commits from git side, the cvsserver session that serves the
> commit for the first time needs to record the branch
> information.

That's what I meant, yes ;)
Will try to reword it to actually say it, too.

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

[PATCH (amend)] cvsserver: Add asciidoc documentation for new database backend configuration

[Frank Lichtenheld]

Documents the new configuration variables and the variable
substitution mechanism.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 Documentation/git-cvsserver.txt |   93 +++++++++++++++++++++++++++++++++++++--
 1 files changed, 89 insertions(+), 4 deletions(-)

 Reworded the section about git-cvsserver needing to update the
 database.

diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
index 2cf8153..9fbaf75 100644
--- a/Documentation/git-cvsserver.txt
+++ b/Documentation/git-cvsserver.txt
@@ -65,11 +65,12 @@ env variable, you can rename git-cvsserver to cvs.
 
 ------
 Note: you need to ensure each user that is going to invoke git-cvsserver has
-write access to the log file and to the git repository. When offering anon
-access via pserver, this means that the nobody user should have write access
-to at least the sqlite database at the root of the repository.
+write access to the log file and to the database (see
+<<dbbackend,Database Backend>>. If you want to offer write access over
+SSH, the users of course also need write access to the git repository itself.
 
-Both configuration variables can also be overriden for a specific method of
+[[configaccessmethod]]
+All configuration variables can also be overriden for a specific method of
 access. Valid method names are "ext" (for SSH access) and "pserver". The
 following example configuration would disable pserver access while still
 allowing access over SSH.
@@ -105,6 +106,90 @@ Example:
      cvs co -d project-master master
 ------
 
+[[dbbackend]]
+Database Backend
+----------------
+
+git-cvsserver uses one database per git head (i.e. CVS module) to
+store information about the repository for faster access. The
+database doesn't contain any persitent data and can be completly
+regenerated from the git repository at any time. The database
+needs to be updated (i.e. written to) after every commit.
+
+If the commit is done directly by using git (as opposed to
+using git-cvsserver) the update will need to happen on the
+next repository access by git-cvsserver, independent of
+access method and requested operation.
+
+That means that even if you offer only read access (e.g. by using
+the pserver method), git-cvsserver should have write access to
+the database to work reliably (otherwise you need to make sure
+that the database if up-to-date all the time git-cvsserver is run).
+
+By default it uses SQLite databases in the git directory, named
+`gitcvs.<module_name>.sqlite`. Note that the SQLite backend creates
+temporary files in the same directory as the database file on
+write so it might not be enough to grant the users using
+git-cvsserver write access to the database file without granting
+them write access to the directory, too.
+
+You can configure the database backend with the following
+configuration variables:
+
+Configuring database backend
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+git-cvsserver uses the Perl DBI module. Please also read
+its documentation if changing these variables, especially
+about `DBI->connect()`.
+
+gitcvs.dbname::
+	Database name. The exact meaning depends on the
+	used database driver, for SQLite this is a filename.
+	Supports variable substitution (see below). May
+	not contain semicolons (`;`).
+	Default: '%Ggitcvs.%m.sqlite'
+
+gitcvs.dbdriver::
+	Used DBI driver. You can specify any available driver
+	for this here, but it might not work. cvsserver is tested
+	with 'DBD::SQLite', reported to work with
+	'DBD::Pg', and reported *not* to work with 'DBD::mysql'.
+	Please regard this as an experimental feature. May not
+	contain double colons (`:`).
+	Default: 'SQLite'
+
+gitcvs.dbuser::
+	Database user. Only useful if setting `dbdriver`, since
+	SQLite has no concept of database users. Supports variable
+	substitution (see below).
+
+gitcvs.dbpass::
+	Database password.  Only useful if setting `dbdriver`, since
+	SQLite has no concept of database passwords.
+
+All variables can also be set per access method, see <<configaccessmethod,above>>.
+
+Variable substitution
+^^^^^^^^^^^^^^^^^^^^^
+In `dbdriver` and `dbuser` you can use the following variables:
+
+%G::
+	git directory name
+%g::
+	git directory name, where all characters except for
+	alpha-numeric ones, `.`, and `-` are replaced with
+	`_` (this should make it easier to use the directory
+	name in a filename if wanted)
+%m::
+	CVS module/git head name
+%a::
+	access method (one of "ext" or "pserver")
+%u::
+	Name of the user running git-cvsserver.
+	If no name can be determined, the
+	numeric uid is used.
+
 Eclipse CVS Client Notes
 ------------------------
 
-- 
1.5.1

Re: [PATCH (amend)] cvsserver: Add asciidoc documentation for new database backend configuration

[Jakub Narebski]

[Cc: Frank Lichtenheld <frank@lichtenheld.de>, git@vger.kernel.org]

Frank Lichtenheld wrote:

> Documents the new configuration variables and the variable
> substitution mechanism.
> 
> Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
> ---
>  Documentation/git-cvsserver.txt |   93 +++++++++++++++++++++++++++++++++++++--
>  1 files changed, 89 insertions(+), 4 deletions(-)

Could you also update Documentation/config.txt? TIA.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

[PATCH] config.txt: Add gitcvs.db* variables

[Frank Lichtenheld]

Adds documentation for gitcvs.{dbname,dbdriver,dbuser,dbpass}
Texts are mostly taken from git-cvsserver.txt whith some
adaptions so that they make more sense out of the context
of the original man page.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 Documentation/config.txt |   27 +++++++++++++++++++++++++++
 1 files changed, 27 insertions(+), 0 deletions(-)

 Note that this one is made on top of the cvsserver topic
 branch and my fixes to config.txt (i.e. the description
 for gitcvs.allbinary is in the context of the patch).
 Just saying.

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 62168e6..ad2c1f5 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -425,6 +425,33 @@ gitcvs.allbinary::
 	fact that there is no way yet to set single files to mode '-kb'.
 	See gitlink:git-cvsserver[1].
 
+gitcvs.dbname::
+	Database used by git-cvsserver to cache revision information
+	derived from the git repository. The exact meaning depends on the
+	used database driver, for SQLite (which is the default driver) this
+	is a filename. Supports variable substitution (see
+	gitlink:git-cvsserver[1] for details). May not contain semicolons (`;`).
+	Default: '%Ggitcvs.%m.sqlite'
+
+gitcvs.dbdriver::
+	Used Perl DBI driver. You can specify any available driver	
+        for this here, but it might not work. git-cvsserver is tested
+	with 'DBD::SQLite', reported to work with 'DBD::Pg', and
+	reported *not* to work with 'DBD::mysql'. Experimental feature.
+	May not contain double colons (`:`). Default: 'SQLite'.
+	See gitlink:git-cvsserver[1].
+
+gitcvs.dbuser, gitcvs.dbpass::
+	Database user and password. Only useful if setting 'gitcvs.dbdriver',
+	since SQLite has no concept of database users and/or passwords.
+	'gitcvs.dbuser' supports variable substitution (see
+	gitlink:git-cvsserver[1] for details).
+
+All gitcvs variables except for 'gitcvs.allbinary' can also specifed
+as 'gitcvs.<access_method>.<varname>' (where 'access_method' is one
+of "ext" and "pserver") to make them apply only for the given access
+method.
+
 http.sslVerify::
 	Whether to verify the SSL certificate when fetching or pushing
 	over HTTPS. Can be overridden by the 'GIT_SSL_NO_VERIFY' environment
-- 
1.5.1.1

[PATCH] cvsserver: Document the GIT branches -> CVS modules mapping more prominently

[Frank Lichtenheld]

Add a note about the branches -> modules mapping to LIMITATIONS because
I really think it should be noted there and not just at the end of
the installation step-by-step HOWTO.

I used "git branches" there and changed "heads" to "branches" in
my section about database configuration. I'm reluctant to replace
all occourences of "head" with "branch" though because you always
have to say "git branch" because CVS also has the concept of
branches. You can say "head" though, because there is no such
concept in CVS. In all the existing occourences of head other than
the one I changed I think "head" flows better in the text.

Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
---
 Documentation/git-cvsserver.txt |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

 Caused by me thinking about Junio's comment.
 Better patches welcome but my motivation to try to write clear
 and concise sentences in English is depleted for now,
 so no critical comments without patches, plz :)

diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
index 9fbaf75..3615ce5 100644
--- a/Documentation/git-cvsserver.txt
+++ b/Documentation/git-cvsserver.txt
@@ -31,6 +31,10 @@ over pserver for anonymous CVS access.
 
 CVS clients cannot tag, branch or perform GIT merges.
 
+git-cvsserver maps GIT branches to CVS modules. This is very different
+from what most CVS users would expect since in CVS modules usually represent
+one or more directories.
+
 INSTALLATION
 ------------
 
@@ -110,7 +114,7 @@ Example:
 Database Backend
 ----------------
 
-git-cvsserver uses one database per git head (i.e. CVS module) to
+git-cvsserver uses one database per git branch (i.e. CVS module) to
 store information about the repository for faster access. The
 database doesn't contain any persitent data and can be completly
 regenerated from the git repository at any time. The database
-- 
1.5.1

Re: [PATCH] cvsserver: Document the GIT branches -> CVS modules mapping more prominently

[Junio C Hamano]

Frank Lichtenheld <frank@lichtenheld.de> writes:

> Add a note about the branches -> modules mapping to LIMITATIONS because
> I really think it should be noted there and not just at the end of
> the installation step-by-step HOWTO.
>
> I used "git branches" there and changed "heads" to "branches" in
> my section about database configuration. I'm reluctant to replace
> all occourences of "head" with "branch" though because you always
> have to say "git branch" because CVS also has the concept of
> branches. You can say "head" though, because there is no such
> concept in CVS. In all the existing occourences of head other than
> the one I changed I think "head" flows better in the text.

Ah, I tend to agree.  Mind if I applied only the first hunk then?

>
> Signed-off-by: Frank Lichtenheld <frank@lichtenheld.de>
> ---
>  Documentation/git-cvsserver.txt |    6 +++++-
>  1 files changed, 5 insertions(+), 1 deletions(-)
>
>  Caused by me thinking about Junio's comment.
>  Better patches welcome but my motivation to try to write clear
>  and concise sentences in English is depleted for now,
>  so no critical comments without patches, plz :)
>
> diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
> index 9fbaf75..3615ce5 100644
> --- a/Documentation/git-cvsserver.txt
> +++ b/Documentation/git-cvsserver.txt
> @@ -31,6 +31,10 @@ over pserver for anonymous CVS access.
>  
>  CVS clients cannot tag, branch or perform GIT merges.
>  
> +git-cvsserver maps GIT branches to CVS modules. This is very different
> +from what most CVS users would expect since in CVS modules usually represent
> +one or more directories.
> +
>  INSTALLATION
>  ------------
>  
> @@ -110,7 +114,7 @@ Example:
>  Database Backend
>  ----------------
>  
> -git-cvsserver uses one database per git head (i.e. CVS module) to
> +git-cvsserver uses one database per git branch (i.e. CVS module) to
>  store information about the repository for faster access. The
>  database doesn't contain any persitent data and can be completly
>  regenerated from the git repository at any time. The database
> -- 
> 1.5.1

Re: [PATCH] cvsserver: Document the GIT branches -> CVS modules mapping more prominently

[Frank Lichtenheld]

On Thu, Apr 12, 2007 at 03:05:51PM -0700, Junio C Hamano wrote:
> Frank Lichtenheld <frank@lichtenheld.de> writes:
> 
> > Add a note about the branches -> modules mapping to LIMITATIONS because
> > I really think it should be noted there and not just at the end of
> > the installation step-by-step HOWTO.
> >
> > I used "git branches" there and changed "heads" to "branches" in
> > my section about database configuration. I'm reluctant to replace
> > all occourences of "head" with "branch" though because you always
> > have to say "git branch" because CVS also has the concept of
> > branches. You can say "head" though, because there is no such
> > concept in CVS. In all the existing occourences of head other than
> > the one I changed I think "head" flows better in the text.
> 
> Ah, I tend to agree.  Mind if I applied only the first hunk then?

Not at all. Go ahead.

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/