* [JGit-io-RFC-PATCH v2 1/4] Introduce a new module for IO SPI of JGit
@ 2009-10-11 14:07 imyousuf
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation imyousuf
0 siblings, 1 reply; 8+ messages in thread
From: imyousuf @ 2009-10-11 14:07 UTC (permalink / raw)
To: git; +Cc: spearce, egit-dev, Imran M Yousuf
From: Imran M Yousuf <imyousuf@smartitengineering.com>
The reason for having a separate module is quite obvious, the IO SPI
providers need not have to depend on the whole JGit but rather only on
the SPI itself. As of the initial implementation the SPI will also contain
the default implementation - local file system provider.
Extract maven POM file to a common file to encapsulate the common setting
and configuration to be used across all maven modules for JGit.
Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
jgit-maven/jgit-io/pom.xml | 65 +++++++++
jgit-maven/jgit/pom.xml | 287 ++++++++++-------------------------------
jgit-maven/{jgit => }/pom.xml | 31 +----
3 files changed, 143 insertions(+), 240 deletions(-)
create mode 100644 jgit-maven/jgit-io/pom.xml
rewrite jgit-maven/jgit/pom.xml (68%)
rename jgit-maven/{jgit => }/pom.xml (90%)
diff --git a/jgit-maven/jgit-io/pom.xml b/jgit-maven/jgit-io/pom.xml
new file mode 100644
index 0000000..f197f58
--- /dev/null
+++ b/jgit-maven/jgit-io/pom.xml
@@ -0,0 +1,65 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+All rights reserved.
+Redistribution and use in source and binary forms, with or
+without modification, are permitted provided that the following
+conditions are met:
+
+Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+
+Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following
+disclaimer in the documentation and/or other materials provided
+with the distribution.
+
+Neither the name of the Eclipse Foundation, Inc. nor the
+names of its contributors may be used to endorse or promote
+products derived from this software without specific prior
+written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.eclipse</groupId>
+ <artifactId>jgit-parent</artifactId>
+ <version>0.6.0-SNAPSHOT</version>
+ </parent>
+ <groupId>org.eclipse</groupId>
+ <artifactId>jgit-io</artifactId>
+ <packaging>jar</packaging>
+ <name>jgit-io</name>
+ <build>
+ <sourceDirectory>../../org.eclipse.jgit.io/src/</sourceDirectory>
+ <testResources>
+ <testResource>
+ <directory>../../org.eclipse.jgit.test/tst-rsrc/</directory>
+ </testResource>
+ </testResources>
+ <testSourceDirectory>../../org.eclipse.jgit.io/tst/</testSourceDirectory>
+ </build>
+ <dependencies>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+</project>
diff --git a/jgit-maven/jgit/pom.xml b/jgit-maven/jgit/pom.xml
dissimilarity index 68%
index d8772ad..9eb5758 100644
--- a/jgit-maven/jgit/pom.xml
+++ b/jgit-maven/jgit/pom.xml
@@ -1,216 +1,71 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
-All rights reserved.
-Redistribution and use in source and binary forms, with or
-without modification, are permitted provided that the following
-conditions are met:
-
-Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
-
-Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following
-disclaimer in the documentation and/or other materials provided
-with the distribution.
-
-Neither the name of the Eclipse Foundation, Inc. nor the
-names of its contributors may be used to endorse or promote
-products derived from this software without specific prior
-written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
-CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
-INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
-OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
-CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
-NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
-STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
- <modelVersion>4.0.0</modelVersion>
- <groupId>org.eclipse</groupId>
- <artifactId>jgit</artifactId>
- <packaging>jar</packaging>
- <version>0.6.0-SNAPSHOT</version>
- <name>jgit</name>
- <url>http://repo.or.cz/w/jgit.git</url>
- <mailingLists>
- <mailingList>
- <name>GIT Mailing List</name>
- <post>git@vger.kernel.org</post>
- <archive>http://marc.info/?l=git</archive>
- </mailingList>
- </mailingLists>
- <description>Pure Java implementation of Git</description>
- <developers>
- <developer>
- <name>Shawn O. Pearce</name>
- <email>spearce@spearce.org</email>
- <roles>
- <role>Maintainer</role>
- </roles>
- </developer>
- <developer>
- <name>Robin Rosenberg</name>
- <email>robin.rosenberg@dewire.com</email>
- <roles>
- <role>Maintainer</role>
- </roles>
- </developer>
- <developer>
- <name>Dave Watson</name>
- <email>dwatson@mimvista.com</email>
- <roles>
- <role>Developer</role>
- </roles>
- </developer>
- <developer>
- <name>Roger C. Soares</name>
- <email>rogersoares@intelinet.com.br</email>
- <roles>
- <role>Developer</role>
- </roles>
- </developer>
- <developer>
- <name>Marek Zawirski</name>
- <email>marek.zawirski@gmail.com</email>
- <roles>
- <role>Developer</role>
- </roles>
- </developer>
- <developer>
- <name>Charles O'Farrell</name>
- <email>charleso@charleso.org</email>
- <roles>
- <role>Contributor</role>
- </roles>
- </developer>
- <developer>
- <name>Imran M Yousuf</name>
- <email>imyousuf@smartitengineering.com</email>
- <organization>Smart IT Engineering</organization>
- <roles>
- <role>Contributor</role>
- </roles>
- </developer>
- </developers>
- <licenses>
- <license>
- <name>Eclipse Distribution License (New BSD License)</name>
- <comments>
- All rights reserved.
- Redistribution and use in source and binary forms, with or
- without modification, are permitted provided that the following
- conditions are met:
-
- Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
-
- Redistributions in binary form must reproduce the above
- copyright notice, this list of conditions and the following
- disclaimer in the documentation and/or other materials provided
- with the distribution.
-
- Neither the name of the Eclipse Foundation, Inc. nor the
- names of its contributors may be used to endorse or promote
- products derived from this software without specific prior
- written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
- CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
- INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
- OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
- CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
- NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
- CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
- ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- </comments>
- </license>
- </licenses>
- <build>
- <sourceDirectory>../../org.eclipse.jgit/src/</sourceDirectory>
- <testResources>
- <testResource>
- <directory>../../org.eclipse.jgit.test/tst-rsrc/</directory>
- </testResource>
- </testResources>
- <testSourceDirectory>../../org.eclipse.jgit.test/tst/</testSourceDirectory>
- <plugins>
- <plugin>
- <artifactId>maven-compiler-plugin</artifactId>
- <version>2.0.2</version>
- <configuration>
- <source>1.5</source>
- <target>1.5</target>
- <encoding>UTF-8</encoding>
- </configuration>
- </plugin>
- <plugin>
- <artifactId>maven-surefire-plugin</artifactId>
- <version>2.4.2</version>
- <configuration>
- <includes>
- <include>**/*Test.java</include>
- <include>**/*TestCase.java</include>
- <include>**/T000*.java</include>
- </includes>
- </configuration>
- </plugin>
- <plugin>
- <artifactId>maven-source-plugin</artifactId>
- <executions>
- <execution>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <artifactId>maven-javadoc-plugin</artifactId>
- <executions>
- <execution>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
- <dependencies>
- <dependency>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- <version>3.8.1</version>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>com.jcraft</groupId>
- <artifactId>jsch</artifactId>
- <version>0.1.41</version>
- <scope>compile</scope>
- </dependency>
- </dependencies>
- <distributionManagement>
- <snapshotRepository>
- <id>jgit-maven-snapshot-repository</id>
- <name>JGit Maven Snapshot Repository</name>
- <url>dav:https://egit.googlecode.com/svn/maven/snapshot-repository/</url>
- <uniqueVersion>true</uniqueVersion>
- </snapshotRepository>
- </distributionManagement>
-</project>
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+All rights reserved.
+Redistribution and use in source and binary forms, with or
+without modification, are permitted provided that the following
+conditions are met:
+
+Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+
+Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following
+disclaimer in the documentation and/or other materials provided
+with the distribution.
+
+Neither the name of the Eclipse Foundation, Inc. nor the
+names of its contributors may be used to endorse or promote
+products derived from this software without specific prior
+written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.eclipse</groupId>
+ <artifactId>jgit-parent</artifactId>
+ <version>0.6.0-SNAPSHOT</version>
+ </parent>
+ <groupId>org.eclipse</groupId>
+ <artifactId>jgit</artifactId>
+ <packaging>jar</packaging>
+ <name>jgit</name>
+ <build>
+ <sourceDirectory>../../org.eclipse.jgit/src/</sourceDirectory>
+ <testResources>
+ <testResource>
+ <directory>../../org.eclipse.jgit.test/tst-rsrc/</directory>
+ </testResource>
+ </testResources>
+ <testSourceDirectory>../../org.eclipse.jgit.test/tst/</testSourceDirectory>
+ </build>
+ <dependencies>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.jcraft</groupId>
+ <artifactId>jsch</artifactId>
+ <version>0.1.41</version>
+ <scope>compile</scope>
+ </dependency>
+ </dependencies>
+</project>
diff --git a/jgit-maven/jgit/pom.xml b/jgit-maven/pom.xml
similarity index 90%
rename from jgit-maven/jgit/pom.xml
rename to jgit-maven/pom.xml
index d8772ad..bac9629 100644
--- a/jgit-maven/jgit/pom.xml
+++ b/jgit-maven/pom.xml
@@ -37,10 +37,10 @@ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.eclipse</groupId>
- <artifactId>jgit</artifactId>
- <packaging>jar</packaging>
+ <artifactId>jgit-parent</artifactId>
+ <packaging>pom</packaging>
<version>0.6.0-SNAPSHOT</version>
- <name>jgit</name>
+ <name>JGit</name>
<url>http://repo.or.cz/w/jgit.git</url>
<mailingLists>
<mailingList>
@@ -141,13 +141,6 @@ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
</license>
</licenses>
<build>
- <sourceDirectory>../../org.eclipse.jgit/src/</sourceDirectory>
- <testResources>
- <testResource>
- <directory>../../org.eclipse.jgit.test/tst-rsrc/</directory>
- </testResource>
- </testResources>
- <testSourceDirectory>../../org.eclipse.jgit.test/tst/</testSourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
@@ -191,20 +184,6 @@ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
</plugin>
</plugins>
</build>
- <dependencies>
- <dependency>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- <version>3.8.1</version>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>com.jcraft</groupId>
- <artifactId>jsch</artifactId>
- <version>0.1.41</version>
- <scope>compile</scope>
- </dependency>
- </dependencies>
<distributionManagement>
<snapshotRepository>
<id>jgit-maven-snapshot-repository</id>
@@ -213,4 +192,8 @@ ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
<uniqueVersion>true</uniqueVersion>
</snapshotRepository>
</distributionManagement>
+ <modules>
+ <module>jgit</module>
+ <module>jgit-io</module>
+ </modules>
</project>
--
1.6.2.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation
2009-10-11 14:07 [JGit-io-RFC-PATCH v2 1/4] Introduce a new module for IO SPI of JGit imyousuf
@ 2009-10-11 14:07 ` imyousuf
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 3/4] Incorporate current FileSystem Util implementations to the SPI imyousuf
2009-10-12 14:57 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation Shawn O. Pearce
0 siblings, 2 replies; 8+ messages in thread
From: imyousuf @ 2009-10-11 14:07 UTC (permalink / raw)
To: git; +Cc: spearce, egit-dev, Imran M Yousuf
From: Imran M Yousuf <imyousuf@smartitengineering.com>
The SPI mainly focus's in providing an API to JGit to be able to perform
similar operations to that of java.io.File. All direct I/O is based on the
java.io.Input/OutputStream classes.
Different JGit IO SPI provider is designed to be URI scheme based and thus
the default implementation is that of "file" scheme. SPI provider will be
integrated by their respective users in a manner similar to that of JDBC
driver registration. There is a SystemStorageManager that has similar
registration capabilities and the system storage providers should be
registered with the manager in one of the provided ways.
This SPI is based on the initial requirements for switching to it and thus
this SPI will change as required during full migration to it.
Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
.../src/org/eclipse/jgit/io/Entry.java | 148 +++++++++++++
.../src/org/eclipse/jgit/io/StorageSystem.java | 57 +++---
.../org/eclipse/jgit/io/StorageSystemManager.java | 154 ++++++++++++++
.../eclipse/jgit/io/localfs/LocalFileEntry.java | 219 ++++++++++++++++++++
.../eclipse/jgit/io/localfs/LocalFileSystem.java | 51 +++--
5 files changed, 577 insertions(+), 52 deletions(-)
create mode 100644 org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
copy org.eclipse.jgit.test/tst/org/eclipse/jgit/util/JGitTestUtil.java => org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java (62%)
create mode 100644 org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystemManager.java
create mode 100644 org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
copy org.eclipse.jgit.pgm/src/org/eclipse/jgit/pgm/Init.java => org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java (65%)
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
new file mode 100644
index 0000000..cd69172
--- /dev/null
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
@@ -0,0 +1,148 @@
+/*
+ * Copyright (C) 2009, Imran M Yousuf <imyousuf@smartitengineering.com>
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+package org.eclipse.jgit.io;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+
+/**
+ * Represents each entry in a storage system. For example, in a local filesystem
+ * storage it would represent {@link java.io.File}. Here the storage system
+ * mainly refers to where repository meta data such as git objects, ref logs,
+ * packs are stored; for example a '.git' directory and all its contents in a
+ * clone repo would correspond to an entry.
+ * @author Imran M Yousuf (imyousuf at smartitengineering.com)
+ * @since 0.6
+ */
+public interface Entry {
+
+ /**
+ * Retrieves the name of the entry
+ * @return Name of the entry
+ */
+ public String getName();
+
+ /**
+ * Retrieves the absolute path of the entry.
+ * @return Absoluth path
+ */
+ public String getAbsolutePath();
+
+ /**
+ * Retrieves whether the entry represents a directory or not
+ * @return True if represents a directory else false
+ */
+ public boolean isDirectory();
+
+ /**
+ * Signifies whether the entry is a new one or being read from the
+ * persistent storage
+ * @return True if being read form storage else false
+ */
+ public boolean isExists();
+
+ /**
+ * Make directories upto the entry represented by this instance, provided
+ * that this instance itself is a directory.
+ * @return True if directories were created.
+ */
+ public boolean mkdirs();
+
+ /**
+ * Retrieves the URI of this entry. URI in this case acts as a primary key
+ * to identify an entry.
+ * @return URI to identify this entry instance
+ */
+ public URI getURI();
+
+ /**
+ * Retrieves the length of the entry if its predictable.
+ * @return < 0 if the length is unpredictable else the length of the entry's
+ * content
+ */
+ public long length();
+
+ /**
+ * Retrieves the InputStream for reading the content of the entry
+ * @return Input stream to read entry content
+ * @throws IOException If no such file exists or there is any other error
+ */
+ public InputStream getInputStream()
+ throws IOException;
+
+ /**
+ * Retrieves the OutputStream for writing content into the entry. It can be
+ * opened to either overwrite it or append to it.
+ * @param overwrite False if to write in append mode else true
+ * @return Output stream to write content to
+ * @throws IOException If no such file exists in append mode or there is any
+ * error in retrieving it.
+ */
+ public OutputStream getOutputStream(boolean overwrite)
+ throws IOException;
+
+ /**
+ * Retrieve all the child entries of this entries if its a directory.
+ * @return If not a directory then a empty array else array of sub-entries.
+ */
+ public Entry[] getChildren();
+
+ /**
+ * Retrieve a specific child of an entry. It will basically match
+ * {@link Entry#getName() name} of the children to find and that too only
+ * the direct children.
+ * @param name Name of the child to find
+ * @return If child is not found then NULL or else the child specified by
+ * the name
+ */
+ public Entry getChild(String name);
+
+ /**
+ * Retrieve the parent entry of the current entry.
+ * @return NULL if no parent or else the direct parent of the current entry
+ */
+ public Entry getParent();
+
+ /**
+ * Retrieve the storage system this entry either is from or will be
+ * persisted to.
+ * @return Storage system of the entry, will never be NULL.
+ */
+ public StorageSystem getStorageSystem();
+}
diff --git a/org.eclipse.jgit.test/tst/org/eclipse/jgit/util/JGitTestUtil.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
similarity index 62%
copy from org.eclipse.jgit.test/tst/org/eclipse/jgit/util/JGitTestUtil.java
copy to org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
index b259185..9f45cb3 100644
--- a/org.eclipse.jgit.test/tst/org/eclipse/jgit/util/JGitTestUtil.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2008, Imran M Yousuf <imyousuf@smartitengineering.com>
+ * Copyright (C) 2009, Imran M Yousuf <imyousuf@smartitengineering.com>
*
* All rights reserved.
*
@@ -34,38 +34,35 @@
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
+package org.eclipse.jgit.io;
-package org.eclipse.jgit.util;
+import java.net.URI;
-import java.io.File;
-import java.net.URISyntaxException;
-import java.net.URL;
-
-public abstract class JGitTestUtil {
- public static final String CLASSPATH_TO_RESOURCES = "org/eclipse/jgit/test/resources/";
+/**
+ * SPI providing access to the underlying storage system. Each provider is
+ * differentiated using their {@link StorageSystem#getURIProtocol() URI Protocol}.
+ * @author Imran M Yousuf (imyousuf at smartitengineering.com)
+ * @since 0.6
+ */
+public interface StorageSystem {
- private JGitTestUtil() {
- throw new UnsupportedOperationException();
- }
+ /**
+ * Returns the supported scheme of this storage system.
+ * @return Scheme supported by this storage system.
+ * @see {@link http://tr.im/BiQ0 URI Scheme}
+ */
+ public String getURIScheme();
- public static File getTestResourceFile(final String fileName) {
- if (fileName == null || fileName.length() <= 0) {
- return null;
- }
- final URL url = cl().getResource(CLASSPATH_TO_RESOURCES + fileName);
- if (url == null) {
- // If URL is null then try to load it as it was being
- // loaded previously
- return new File("tst", fileName);
- }
- try {
- return new File(url.toURI());
- } catch(URISyntaxException e) {
- return new File(url.getPath());
- }
- }
+ /**
+ * Retrieve an entry using its URI
+ * @param uri URI to retrieve the entry for.
+ * @return Entry representing the URI
+ */
+ public Entry getEntry(URI uri);
- private static ClassLoader cl() {
- return JGitTestUtil.class.getClassLoader();
- }
+ /**
+ * Retrieve the current working directory from the file system if any.
+ * @return Entry for current working directory.
+ */
+ public Entry getWorkingDirectory();
}
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystemManager.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystemManager.java
new file mode 100644
index 0000000..999122e
--- /dev/null
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystemManager.java
@@ -0,0 +1,154 @@
+/*
+ * Copyright (C) 2009, Imran M Yousuf <imyousuf@smartitengineering.com>
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+package org.eclipse.jgit.io;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Hashtable;
+import java.util.Map;
+import org.eclipse.jgit.io.localfs.LocalFileSystem;
+
+/**
+ * Manager that registers different storage system for different {@link URI}
+ * schemes and serves users with {@link Entry} or {@link StorageSystem} as URI
+ * schemes request.
+ * @author Imran M Yousuf (imyousuf at smartitengineering.com)
+ * @since 0.6
+ */
+public class StorageSystemManager {
+
+ /**
+ * Map to act as registrar for all registered schemes
+ */
+ private static Map<String, StorageSystem> storageSystemMap;
+
+ /**
+ * Initialize the registrar and and load the instance for default storage,
+ * i.e. the local file system.
+ */
+ static {
+ storageSystemMap = new Hashtable<String, StorageSystem>();
+ try {
+ register(LocalFileSystem.class);
+ }
+ catch (Exception ex) {
+ ex.printStackTrace();
+ }
+ }
+
+ /**
+ * Retrieves the storage system for a given scheme from the registrar.
+ * @param scheme Scheme to retrieve; it should never be null
+ * @return Storage system representing the scheme.
+ * @throws IOException If scheme is null or scheme does not exist.
+ */
+ public static StorageSystem getStorageSystem(String scheme)
+ throws IOException {
+ if (scheme != null && storageSystemMap.containsKey(scheme)) {
+ return storageSystemMap.get(scheme);
+ }
+ else {
+ throw new IOException("Scheme ( " + scheme +
+ " ) not registered with manager!");
+ }
+ }
+
+ /**
+ * Load an storage system entry using its respective URI.
+ * @param uri URI to retrieve the storage system entity for
+ * @return The entry in the storage system for the requested URI
+ * @throws IOException If scheme does not exist.
+ * @throws NullPointerException If uri is null
+ */
+ public static Entry getEntry(URI uri)
+ throws IOException,
+ NullPointerException {
+ String scheme = uri.getScheme();
+ return getStorageSystem(scheme).getEntry(uri);
+ }
+
+ /**
+ * Registers a {@link StorageSystem} implementaiton against its schema into
+ * the registrar using the implementions fully qualified class name and its
+ * non-args constructor.
+ * @param storageSystemClassName The class names representation is string.
+ * @throws ClassNotFoundException If no class is found with the name specified
+ * @throws InstantiationException If there is any exception during
+ * initialization
+ * @throws IllegalAccessException If the class dpes not have a public
+ * non-args constructor
+ * @throws ClassCastException If the class does implement {@link StorageSystem}
+ */
+ public static void register(String storageSystemClassName)
+ throws ClassNotFoundException,
+ InstantiationException,
+ IllegalAccessException,
+ ClassCastException {
+ register((Class<? extends StorageSystem>) Class.forName(
+ storageSystemClassName));
+ }
+
+ /**
+ * Registers a {@link StorageSystem} implementaiton against its schema into
+ * the registrar using the implemention class name and its non-args constructor.
+ * @param storageSystemClassName The class names representation is string.
+ * @throws InstantiationException If there is any exception during
+ * initialization
+ * @throws IllegalAccessException If the class dpes not have a public
+ * non-args constructor
+ */
+ public static void register(
+ Class<? extends StorageSystem> storageSystemClass)
+ throws InstantiationException,
+ IllegalAccessException {
+ if (storageSystemClass != null) {
+ StorageSystem system = storageSystemClass.newInstance();
+ register(system);
+ }
+ }
+
+ /**
+ * Registers a {@link StorageSystem} instance into the reigstrar. It would be
+ * particularly useful if system does not have a non-args constructor.
+ * @param system System to register in the manager
+ */
+ public static void register(StorageSystem system) {
+ if (system != null) {
+ storageSystemMap.put(system.getURIScheme(), system);
+ }
+ }
+}
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
new file mode 100644
index 0000000..99df831
--- /dev/null
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
@@ -0,0 +1,219 @@
+/*
+ * Copyright (C) 2009, Imran M Yousuf <imyousuf@smartitengineering.com>
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+package org.eclipse.jgit.io.localfs;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import org.eclipse.jgit.io.Entry;
+import org.eclipse.jgit.io.StorageSystem;
+import org.eclipse.jgit.io.StorageSystemManager;
+
+/**
+ * Entry implementation for local file system. This class should not be
+ * initialized directly unless its a {@link LocalFileSystem}. SPI users should
+ * use {@link StorageSystemManager#getEntry(java.net.URI)} to get the first
+ * {@link Entry} and then traverse from there onwards.
+ * @author Imran M Yousuf (imyousuf at smartitengineering.com)
+ * @since 0.6
+ */
+public class LocalFileEntry
+ implements Entry {
+
+ private File localFile;
+ private StorageSystem storageSystem;
+
+ /**
+ * Contructs an entry based of on the local file system storage and a file
+ * that will be represented by this entry.
+ * @param localFile File represented by this entry
+ * @param storageSystem Storage system of the entry
+ * @throws IllegalArgumentException If either argument is NULL
+ */
+ protected LocalFileEntry(File localFile,
+ StorageSystem storageSystem)
+ throws IllegalArgumentException {
+ setLocalFile(localFile);
+ setStorageSystem(storageSystem);
+ }
+
+ /**
+ * Sets the storage system instance for this entry.
+ * @param storageSystem Storage system
+ * @throws IllegalArgumentException IF storageSystem is null
+ */
+ protected void setStorageSystem(StorageSystem storageSystem)
+ throws IllegalArgumentException {
+ if (storageSystem == null) {
+ throw new IllegalArgumentException("Storage system can't be NULL!");
+ }
+ this.storageSystem = storageSystem;
+ }
+
+ /**
+ * Retrieves the file being adapted by this entry.
+ * @return File being adapted
+ */
+ public File getLocalFile() {
+ return localFile;
+ }
+
+ /**
+ * Sets the file which is to be used as adapt from this instance by this
+ * implementation
+ * @param localFile Local file being adapted by this instance
+ * @throws IllegalArgumentException If localFile is null
+ */
+ protected void setLocalFile(File localFile)
+ throws IllegalArgumentException {
+ if (localFile == null) {
+ throw new IllegalArgumentException(
+ "Local file to be set can't be NULL");
+ }
+ this.localFile = localFile;
+ }
+
+ public String getName() {
+ return getLocalFile().getName();
+ }
+
+ public String getAbsolutePath() {
+ return getLocalFile().getAbsolutePath();
+ }
+
+ public boolean isDirectory() {
+ return getLocalFile().isDirectory();
+ }
+
+ public boolean isExists() {
+ return getLocalFile().exists();
+ }
+
+ public boolean mkdirs() {
+ return getLocalFile().mkdirs();
+ }
+
+ public URI getURI() {
+ return getLocalFile().toURI();
+ }
+
+ public InputStream getInputStream()
+ throws IOException {
+ if (getLocalFile().exists()) {
+ try {
+ return new FileInputStream(getLocalFile());
+ }
+ catch (FileNotFoundException ex) {
+ throw ex;
+ }
+ }
+ else {
+ throw new FileNotFoundException("File does not exists!");
+ }
+ }
+
+ public OutputStream getOutputStream(boolean overwrite)
+ throws IOException {
+ try {
+ if (!isExists()) {
+
+ return new FileOutputStream(getLocalFile());
+ }
+ else {
+ if (overwrite) {
+ return new FileOutputStream(getLocalFile());
+ }
+ else {
+ return new FileOutputStream(getLocalFile(), true);
+ }
+ }
+ }
+ catch (FileNotFoundException ex) {
+ throw ex;
+ }
+ }
+
+ public Entry[] getChildren() {
+ File[] children = getLocalFile().listFiles();
+ if (children == null || children.length == 0) {
+ return new Entry[0];
+ }
+ else {
+ Entry[] entries = new Entry[children.length];
+ for (int i = 0; i < children.length; ++i) {
+ entries[i] = getStorageSystem().getEntry(children[i].toURI());
+ }
+ return entries;
+ }
+ }
+
+ public Entry getChild(String name) {
+ if (name == null || name.length() == 0) {
+ return null;
+ }
+ Entry[] children = getChildren();
+ for (Entry entry : children) {
+ if (name.equals(entry.getName())) {
+ return entry;
+ }
+ }
+ return null;
+ }
+
+ public Entry getParent() {
+ File parent = getLocalFile().getParentFile();
+ if (parent == null) {
+ return null;
+ }
+ else {
+ return getStorageSystem().getEntry(parent.toURI());
+ }
+ }
+
+ public StorageSystem getStorageSystem() {
+ return storageSystem;
+ }
+
+ public long length() {
+ return getLocalFile().length();
+ }
+}
diff --git a/org.eclipse.jgit.pgm/src/org/eclipse/jgit/pgm/Init.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java
similarity index 65%
copy from org.eclipse.jgit.pgm/src/org/eclipse/jgit/pgm/Init.java
copy to org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java
index 49aa418..d0cb536 100644
--- a/org.eclipse.jgit.pgm/src/org/eclipse/jgit/pgm/Init.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2008, Google Inc.
+ * Copyright (C) 2009, Imran M Yousuf <imyousuf@smartitengineering.com>
*
* All rights reserved.
*
@@ -34,31 +34,38 @@
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
-
-package org.eclipse.jgit.pgm;
+package org.eclipse.jgit.io.localfs;
import java.io.File;
+import java.net.URI;
+import org.eclipse.jgit.io.Entry;
+import org.eclipse.jgit.io.StorageSystem;
+
+/**
+ * Implementation of storage system for the local file system.
+ * @author Imran M Yousuf (imyousuf at smartitengineering.com)
+ * @since 0.6
+ */
+public class LocalFileSystem
+ implements StorageSystem {
-import org.kohsuke.args4j.Option;
-import org.eclipse.jgit.lib.Repository;
+ public static final String PROTOCOL_FILE = "file";
-@Command(common = true, usage = "Create an empty git repository")
-class Init extends TextBuiltin {
- @Option(name = "--bare", usage = "Create a bare repository")
- private boolean bare;
+ public String getURIScheme() {
+ return PROTOCOL_FILE;
+ }
- @Override
- protected final boolean requiresRepository() {
- return false;
- }
+ public Entry getEntry(URI uri) {
+ if (uri == null) {
+ return null;
+ }
+ else {
+ return new LocalFileEntry(new File(uri), this);
+ }
+ }
- @Override
- protected void run() throws Exception {
- if (gitdir == null)
- gitdir = new File(bare ? "." : ".git");
- db = new Repository(gitdir);
- db.create(bare);
- out.println("Initialized empty Git repository in "
- + gitdir.getAbsolutePath());
- }
+ public Entry getWorkingDirectory() {
+ String curDir = System.getProperty("user.dir");
+ return new LocalFileEntry(new File(curDir), this);
+ }
}
--
1.6.2.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [JGit-io-RFC-PATCH v2 3/4] Incorporate current FileSystem Util implementations to the SPI
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation imyousuf
@ 2009-10-11 14:07 ` imyousuf
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 4/4] Add locking capability to the IO SPI based on Java Concurrency Lock API imyousuf
2009-10-12 14:57 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation Shawn O. Pearce
1 sibling, 1 reply; 8+ messages in thread
From: imyousuf @ 2009-10-11 14:07 UTC (permalink / raw)
To: git; +Cc: spearce, egit-dev, Imran M Yousuf
From: Imran M Yousuf <imyousuf@smartitengineering.com>
Operations such as setting executable bits if supported and resolving
relative path is incorporated with this change, as a result now it should
be possible to completely replace the currently being used util FS and
java.io.File.
Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
.../src/org/eclipse/jgit/io/Entry.java | 35 ++++-
.../src/org/eclipse/jgit/io/StorageSystem.java | 16 ++
.../eclipse/jgit/io/localfs/LocalFileEntry.java | 52 ++++++-
.../eclipse/jgit/io/localfs/LocalFileSystem.java | 159 ++++++++++++++++++++
4 files changed, 257 insertions(+), 5 deletions(-)
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
index cd69172..5256815 100644
--- a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
@@ -78,6 +78,24 @@
public boolean isExists();
/**
+ * Does this operating system and JRE support the execute flag on entries?
+ *
+ * @return true if this implementation can provide reasonably accurate
+ * executable bit information; false otherwise.
+ */
+ public boolean isExecutableSupported();
+
+ /**
+ * Determine if the entry is executable (or not).
+ * <p>
+ * Not all platforms and JREs support executable flags on entries. If the
+ * feature is unsupported this method will always return false.
+ *
+ * @return true if the entry is believed to be executable by the user.
+ */
+ public boolean isExecutable();
+
+ /**
* Make directories upto the entry represented by this instance, provided
* that this instance itself is a directory.
* @return True if directories were created.
@@ -85,6 +103,19 @@
public boolean mkdirs();
/**
+ * Set an entry to be executable by the user.
+ * <p>
+ * Not all platforms and JREs support executable flags on entries. If the
+ * feature is unsupported this method will always return false and no
+ * changes will be made to the entry specified.
+ *
+ * @param executable
+ * true to enable execution; false to disable it.
+ * @return true if the change succeeded; false otherwise.
+ */
+ public boolean setExecutable(boolean executable);
+
+ /**
* Retrieves the URI of this entry. URI in this case acts as a primary key
* to identify an entry.
* @return URI to identify this entry instance
@@ -101,7 +132,7 @@
/**
* Retrieves the InputStream for reading the content of the entry
* @return Input stream to read entry content
- * @throws IOException If no such file exists or there is any other error
+ * @throws IOException If no such entry exists or there is any other error
*/
public InputStream getInputStream()
throws IOException;
@@ -111,7 +142,7 @@ public InputStream getInputStream()
* opened to either overwrite it or append to it.
* @param overwrite False if to write in append mode else true
* @return Output stream to write content to
- * @throws IOException If no such file exists in append mode or there is any
+ * @throws IOException If no such entry exists in append mode or there is any
* error in retrieving it.
*/
public OutputStream getOutputStream(boolean overwrite)
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
index 9f45cb3..15af614 100644
--- a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
@@ -65,4 +65,20 @@
* @return Entry for current working directory.
*/
public Entry getWorkingDirectory();
+
+ /**
+ * Retrieve the home directory of the current user
+ * @return Home directory
+ */
+ public Entry getHomeDirectory();
+
+ /**
+ * Resolve relative path with respect to a path and return the absolute
+ * entry representing the relative path.
+ * @param entry The point of reference for the relative path
+ * @param path The relative path
+ * @return The absolute entry representing the relative path entry
+ */
+ public Entry resolve(Entry entry,
+ String path);
}
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
index 99df831..4ef3076 100644
--- a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
@@ -43,6 +43,7 @@
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
+import java.lang.reflect.InvocationTargetException;
import java.net.URI;
import org.eclipse.jgit.io.Entry;
import org.eclipse.jgit.io.StorageSystem;
@@ -60,7 +61,7 @@
implements Entry {
private File localFile;
- private StorageSystem storageSystem;
+ private LocalFileSystem storageSystem;
/**
* Contructs an entry based of on the local file system storage and a file
@@ -70,7 +71,7 @@
* @throws IllegalArgumentException If either argument is NULL
*/
protected LocalFileEntry(File localFile,
- StorageSystem storageSystem)
+ LocalFileSystem storageSystem)
throws IllegalArgumentException {
setLocalFile(localFile);
setStorageSystem(storageSystem);
@@ -81,7 +82,7 @@ protected LocalFileEntry(File localFile,
* @param storageSystem Storage system
* @throws IllegalArgumentException IF storageSystem is null
*/
- protected void setStorageSystem(StorageSystem storageSystem)
+ protected void setStorageSystem(LocalFileSystem storageSystem)
throws IllegalArgumentException {
if (storageSystem == null) {
throw new IllegalArgumentException("Storage system can't be NULL!");
@@ -216,4 +217,49 @@ public StorageSystem getStorageSystem() {
public long length() {
return getLocalFile().length();
}
+
+ public boolean isExecutableSupported() {
+ return LocalFileSystem.platform.isExecutableSupproted();
+ }
+
+ public boolean isExecutable() {
+ if (LocalFileSystem.platform.isExecutableSupproted()) {
+ try {
+ final Object r = LocalFileSystem.canExecute.invoke(
+ getLocalFile(),
+ (Object[]) null);
+ return ((Boolean) r).booleanValue();
+ }
+ catch (IllegalArgumentException e) {
+ throw new Error(e);
+ }
+ catch (IllegalAccessException e) {
+ throw new Error(e);
+ }
+ catch (InvocationTargetException e) {
+ throw new Error(e);
+ }
+ }
+ else {
+ return false;
+ }
+ }
+
+ public boolean setExecutable(boolean executable) {
+ try {
+ final Object r;
+ r = LocalFileSystem.setExecute.invoke(getLocalFile(), new Object[] {
+ Boolean.valueOf(executable)});
+ return ((Boolean) r).booleanValue();
+ }
+ catch (IllegalArgumentException e) {
+ throw new Error(e);
+ }
+ catch (IllegalAccessException e) {
+ throw new Error(e);
+ }
+ catch (InvocationTargetException e) {
+ throw new Error(e);
+ }
+ }
}
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java
index d0cb536..7cef2d8 100644
--- a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileSystem.java
@@ -36,8 +36,14 @@
*/
package org.eclipse.jgit.io.localfs;
+import java.io.BufferedReader;
import java.io.File;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.lang.reflect.Method;
import java.net.URI;
+import java.security.AccessController;
+import java.security.PrivilegedAction;
import org.eclipse.jgit.io.Entry;
import org.eclipse.jgit.io.StorageSystem;
@@ -50,6 +56,14 @@
implements StorageSystem {
public static final String PROTOCOL_FILE = "file";
+ public static final Platform platform;
+ public static Method canExecute;
+ public static Method setExecute;
+ public static String cygpath;
+
+ static {
+ platform = Platform.detect();
+ }
public String getURIScheme() {
return PROTOCOL_FILE;
@@ -68,4 +82,149 @@ public Entry getWorkingDirectory() {
String curDir = System.getProperty("user.dir");
return new LocalFileEntry(new File(curDir), this);
}
+
+ public Entry resolve(Entry entry,
+ String path) {
+ if (!(entry instanceof LocalFileEntry)) {
+ return null;
+ }
+ LocalFileEntry fileEntry = (LocalFileEntry) entry;
+ File localFile = fileEntry.getLocalFile();
+ if (platform.equals(Platform.WIN32_CYGWIN)) {
+ try {
+ final Process p;
+
+ p = Runtime.getRuntime().exec(
+ new String[] {cygpath, "--windows", "--absolute", path},
+ null, localFile);
+ p.getOutputStream().close();
+
+ final BufferedReader lineRead = new BufferedReader(
+ new InputStreamReader(p.getInputStream(), "UTF-8"));
+ String r = null;
+ try {
+ r = lineRead.readLine();
+ }
+ finally {
+ lineRead.close();
+ }
+
+ for (;;) {
+ try {
+ if (p.waitFor() == 0 && r != null && r.length() > 0) {
+ return new LocalFileEntry(new File(r), this);
+ }
+ break;
+ }
+ catch (InterruptedException ie) {
+ // Stop bothering me, I have a zombie to reap.
+ }
+ }
+ }
+ catch (IOException ioe) {
+ // Fall through and use the default return.
+ }
+
+ }
+ final File abspn = new File(path);
+ if (abspn.isAbsolute()) {
+ return new LocalFileEntry(abspn, this);
+ }
+ return new LocalFileEntry(new File(localFile, path), this);
+ }
+
+ public Entry getHomeDirectory() {
+ if (platform.equals(Platform.WIN32_CYGWIN)) {
+ final String home = AccessController.doPrivileged(new PrivilegedAction<String>() {
+
+ public String run() {
+ return System.getenv("HOME");
+ }
+ });
+ if (!(home == null || home.length() == 0)) {
+ return resolve(new LocalFileEntry(new File("."), this), home);
+ }
+ }
+ final String home = AccessController.doPrivileged(new PrivilegedAction<String>() {
+
+ public String run() {
+ return System.getProperty("user.home");
+ }
+ });
+ if (home == null || home.length() == 0) {
+ return null;
+ }
+ return new LocalFileEntry(new File(home).getAbsoluteFile(), this);
+ }
+
+ public enum Platform {
+
+ WIN32_CYGWIN(false),
+ WIN32(false),
+ POSIX_JAVA5(false),
+ POSIX_JAVA6(true);
+ private boolean executableSupproted;
+
+ private Platform(boolean executableSupproted) {
+ this.executableSupproted = executableSupproted;
+ }
+
+ public boolean isExecutableSupproted() {
+ return executableSupproted;
+ }
+
+ public static Platform detect() {
+ final String osDotName = AccessController.doPrivileged(new PrivilegedAction<String>() {
+
+ public String run() {
+ return System.getProperty("os.name");
+ }
+ });
+ if (osDotName != null &&
+ osDotName.toLowerCase().indexOf("windows") != -1) {
+ final String path = AccessController.doPrivileged(new PrivilegedAction<String>() {
+
+ public String run() {
+ return System.getProperty("java.library.path");
+ }
+ });
+ if (path == null) {
+ return WIN32;
+ }
+ for (final String p : path.split(";")) {
+ final File e = new File(p, "cygpath.exe");
+ if (e.isFile()) {
+ cygpath = e.getAbsolutePath();
+ return WIN32_CYGWIN;
+ }
+ }
+ return WIN32;
+ }
+ else {
+ canExecute = needMethod(File.class, "canExecute");
+ setExecute = needMethod(File.class, "setExecutable",
+ Boolean.TYPE);
+ if (canExecute != null && setExecute != null) {
+ return POSIX_JAVA6;
+ }
+ else {
+ return POSIX_JAVA5;
+ }
+ }
+ }
+
+ private static Method needMethod(final Class<?> on,
+ final String name,
+ final Class<?>... args) {
+ try {
+ return on.getMethod(name, args);
+ }
+ catch (SecurityException e) {
+ return null;
+ }
+ catch (NoSuchMethodException e) {
+ return null;
+ }
+ }
+ }
}
--
1.6.2.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [JGit-io-RFC-PATCH v2 4/4] Add locking capability to the IO SPI based on Java Concurrency Lock API
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 3/4] Incorporate current FileSystem Util implementations to the SPI imyousuf
@ 2009-10-11 14:07 ` imyousuf
0 siblings, 0 replies; 8+ messages in thread
From: imyousuf @ 2009-10-11 14:07 UTC (permalink / raw)
To: git; +Cc: spearce, egit-dev, Imran M Yousuf
From: Imran M Yousuf <imyousuf@smartitengineering.com>
The lock implementation for local FS has 2 layer for locking, first JVM
level using the Java Concurrency API and second level is in local FS and
its Git specific *.lock files.
This change basically replaces the LockFile implementation in JGit Lib.
Signed-off-by: Imran M Yousuf <imyousuf@smartitengineering.com>
---
.../src/org/eclipse/jgit/io/Entry.java | 44 ++++-
.../eclipse/jgit/io/localfs/LocalFileEntry.java | 177 +++++++++++++++++-
.../org/eclipse/jgit/io/lock/AbstractLockable.java | 199 ++++++++++++++++++++
.../{StorageSystem.java => lock/LockManager.java} | 96 ++++++----
.../src/org/eclipse/jgit/io/lock/Lockable.java | 26 ++--
5 files changed, 486 insertions(+), 56 deletions(-)
create mode 100644 org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/AbstractLockable.java
copy org.eclipse.jgit.io/src/org/eclipse/jgit/io/{StorageSystem.java => lock/LockManager.java} (53%)
copy org.eclipse.jgit/src/org/eclipse/jgit/lib/RepositoryAdapter.java => org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/Lockable.java (77%)
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
index 5256815..4c264db 100644
--- a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/Entry.java
@@ -36,6 +36,7 @@
*/
package org.eclipse.jgit.io;
+import org.eclipse.jgit.io.lock.Lockable;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
@@ -50,7 +51,8 @@
* @author Imran M Yousuf (imyousuf at smartitengineering.com)
* @since 0.6
*/
-public interface Entry {
+public interface Entry
+ extends Lockable {
/**
* Retrieves the name of the entry
@@ -138,12 +140,26 @@ public InputStream getInputStream()
throws IOException;
/**
- * Retrieves the OutputStream for writing content into the entry. It can be
- * opened to either overwrite it or append to it.
+ * Retrieves a locked channeled output stream. When the output stream is closed
+ * the channel is released automatically.
+ * @param overwrite False if to write in append mode else true
+ * @param lock Whether to attain lock or not
+ * @return Output stream to write content to
+ * @throws IOException If no such entry exists in append mode or there is any
+ * error in retrieving it or retrieving the lock.
+ */
+ public OutputStream getOutputStream(boolean overwrite,
+ boolean lock)
+ throws IOException;
+
+ /**
+ * Behaves in as if {@link Entry#getOutputStream(boolean, boolean)} is called
+ * with lock param as false.
* @param overwrite False if to write in append mode else true
* @return Output stream to write content to
* @throws IOException If no such entry exists in append mode or there is any
* error in retrieving it.
+ * @see Entry#getOutputStream(boolean, boolean)
*/
public OutputStream getOutputStream(boolean overwrite)
throws IOException;
@@ -171,6 +187,28 @@ public OutputStream getOutputStream(boolean overwrite)
public Entry getParent();
/**
+ * Create this entry in the underlying system storage.
+ * @return True if created else false
+ * @throws IOException If any I/O during creation
+ */
+ public boolean createNew()
+ throws IOException;
+
+ /**
+ * Delete current entry
+ * @return True if deleted successfully else false
+ * @throws IOException If any error during writing
+ */
+ public boolean delete()
+ throws IOException;
+
+ /**
+ * Retrieve the entry for lock file
+ * @return Null if lock is attained else null
+ */
+ public Entry getLockEntry();
+
+ /**
* Retrieve the storage system this entry either is from or will be
* persisted to.
* @return Storage system of the entry, will never be NULL.
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
index 4ef3076..f6c84c1 100644
--- a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/localfs/LocalFileEntry.java
@@ -45,9 +45,13 @@
import java.io.OutputStream;
import java.lang.reflect.InvocationTargetException;
import java.net.URI;
+import java.nio.channels.FileLock;
+import java.nio.channels.OverlappingFileLockException;
+import java.util.concurrent.TimeUnit;
import org.eclipse.jgit.io.Entry;
import org.eclipse.jgit.io.StorageSystem;
import org.eclipse.jgit.io.StorageSystemManager;
+import org.eclipse.jgit.io.lock.AbstractLockable;
/**
* Entry implementation for local file system. This class should not be
@@ -58,10 +62,15 @@
* @since 0.6
*/
public class LocalFileEntry
+ extends AbstractLockable
implements Entry {
+ public static final String LOCK_FILE_EXT = ".lock";
+ public static final long DEFAULT_WAIT_TIME_MS = 200;
private File localFile;
private LocalFileSystem storageSystem;
+ private File localLockFile;
+ private String lockFileName;
/**
* Contructs an entry based of on the local file system storage and a file
@@ -152,25 +161,86 @@ public InputStream getInputStream()
}
}
- public OutputStream getOutputStream(boolean overwrite)
+ public OutputStream getOutputStream(boolean overwrite,
+ boolean lock)
throws IOException {
+ final FileOutputStream fileOutputStream;
try {
if (!isExists()) {
-
- return new FileOutputStream(getLocalFile());
+ fileOutputStream = new FileOutputStream(getLocalFile());
}
else {
if (overwrite) {
- return new FileOutputStream(getLocalFile());
+ fileOutputStream = new FileOutputStream(getLocalFile());
}
else {
- return new FileOutputStream(getLocalFile(), true);
+ fileOutputStream = new FileOutputStream(getLocalFile(), true);
+ }
+ }
+ if (lock) {
+ final FileLock fileLock;
+ try {
+ fileLock = fileOutputStream.getChannel().lock();
+ }
+ catch (IOException ex) {
+ fileOutputStream.close();
+ throw ex;
+ }
+ catch (Exception ex) {
+ fileOutputStream.close();
+ throw new IOException(ex);
+ }
+ if (fileLock == null) {
+ fileOutputStream.close();
+ throw new OverlappingFileLockException();
}
+ return new OutputStream() {
+
+ @Override
+ public void write(int b)
+ throws IOException {
+ fileOutputStream.write(b);
+ }
+
+ @Override
+ public void close()
+ throws IOException {
+ fileLock.release();
+ fileOutputStream.close();
+ }
+
+ @Override
+ public void flush()
+ throws IOException {
+ fileOutputStream.flush();
+ }
+
+ @Override
+ public void write(byte[] b)
+ throws IOException {
+ fileOutputStream.write(b);
+ }
+
+ @Override
+ public void write(byte[] b,
+ int off,
+ int len)
+ throws IOException {
+ fileOutputStream.write(b, off, len);
+ }
+ };
}
+ return fileOutputStream;
}
catch (FileNotFoundException ex) {
throw ex;
}
+
+ }
+
+ public OutputStream getOutputStream(boolean overwrite)
+ throws IOException {
+ return getOutputStream(overwrite, false);
}
public Entry[] getChildren() {
@@ -262,4 +332,101 @@ public boolean setExecutable(boolean executable) {
throw new Error(e);
}
}
+
+ public boolean createNew()
+ throws IOException {
+ return getLocalFile().createNewFile();
+ }
+
+ public boolean delete()
+ throws IOException {
+ return getLocalFile().delete();
+ }
+
+ public Entry getLockEntry() {
+ if (localLockFile == null) {
+ return null;
+ }
+ else {
+ return getStorageSystem().getEntry(localLockFile.toURI());
+ }
+ }
+
+ @Override
+ protected void performLock()
+ throws Exception {
+ boolean lock = performTryLock(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
+ if (!lock) {
+ throw new IOException("Could not attain lock!");
+ }
+ }
+
+ @Override
+ protected boolean performTryLock(long time,
+ TimeUnit unit) {
+ final long milliSecTime = TimeUnit.MILLISECONDS.convert(time, unit);
+ long waitLeft = milliSecTime;
+ boolean tryAgain = true;
+ do {
+ long nextWaitDuration = Math.min(waitLeft, DEFAULT_WAIT_TIME_MS);
+ waitLeft = waitLeft - nextWaitDuration;
+ boolean success = performTryLock();
+ if (success) {
+ return success;
+ }
+ else {
+ if (nextWaitDuration > 0) {
+ try {
+ Thread.sleep(nextWaitDuration);
+ }
+ catch (InterruptedException ex) {
+ }
+ }
+ else {
+ tryAgain = false;
+ }
+ }
+ }
+ while (tryAgain);
+ return false;
+ }
+
+ @Override
+ protected boolean performTryLock() {
+ StringBuilder lockFileNameBuilder = new StringBuilder();
+ if (this.lockFileName == null) {
+ lockFileNameBuilder.append(getLocalFile().getName());
+ lockFileNameBuilder.append(LOCK_FILE_EXT);
+ this.lockFileName = lockFileNameBuilder.toString();
+ }
+ final File parent = getLocalFile().getParentFile();
+ if (parent != null) {
+ parent.mkdirs();
+ }
+ localLockFile = new File(parent, this.lockFileName);
+ if (localLockFile.exists()) {
+ localLockFile = null;
+ return false;
+ }
+ else {
+ boolean createNewFile;
+ try {
+ createNewFile = localLockFile.createNewFile();
+ }
+ catch (IOException ex) {
+ createNewFile = false;
+ }
+ if (!createNewFile) {
+ localLockFile = null;
+ }
+ return createNewFile;
+ }
+ }
+
+ @Override
+ protected void performUnlock() {
+ if (localLockFile != null) {
+ localLockFile.delete();
+ }
+ }
}
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/AbstractLockable.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/AbstractLockable.java
new file mode 100644
index 0000000..1e95494
--- /dev/null
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/AbstractLockable.java
@@ -0,0 +1,199 @@
+/*
+ * Copyright (C) 2009, Imran M Yousuf <imyousuf@smartitengineering.com>
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+package org.eclipse.jgit.io.lock;
+
+import java.net.URI;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.Condition;
+import java.util.concurrent.locks.ReentrantLock;
+
+/**
+ * Abstract implementation of lockable
+ * @author Imran M Yousuf (imyousuf at smartitengineering.com)
+ * @since 0.6
+ */
+public abstract class AbstractLockable
+ implements Lockable {
+
+ private ReentrantLock lock;
+ private boolean attainedCompleteLock;
+
+ protected AbstractLockable() {
+ }
+
+ @Override
+ protected void finalize()
+ throws Throwable {
+ LockManager.getInstance().unregister(getURI());
+ super.finalize();
+ }
+
+ /**
+ * Retrieve the URI of this instance.
+ * @return The URI to identify this instance. Should never be NULL
+ */
+ public abstract URI getURI();
+
+ protected boolean isInternalLockHeldOnly() {
+ return getLock().isHeldByCurrentThread();
+ }
+
+ public boolean isHeldByCurrentThread() {
+ return isInternalLockHeldOnly() && attainedCompleteLock;
+ }
+
+ public void lock() {
+ getLock().lock();
+ childLock();
+ }
+
+ public void lockInterruptibly()
+ throws InterruptedException {
+ getLock().lockInterruptibly();
+ childLock();
+ }
+
+ public Condition newCondition() {
+ return getLock().newCondition();
+ }
+
+ public boolean tryLock() {
+ if (isHeldByCurrentThread()) {
+ return true;
+ }
+ final boolean tryLock = getLock().tryLock();
+ if (tryLock) {
+ final boolean performTryLock = performTryLock();
+ if (!performTryLock) {
+ unlock();
+ }
+ else {
+ attainedCompleteLock = true;
+ }
+ }
+ return tryLock;
+ }
+
+ public boolean tryLock(long time,
+ TimeUnit unit)
+ throws InterruptedException {
+ if (isHeldByCurrentThread()) {
+ return true;
+ }
+ final boolean tryLock;
+ long currentTime = System.currentTimeMillis();
+ tryLock = getLock().tryLock(time, unit);
+ long duration = unit.convert(System.currentTimeMillis() - currentTime,
+ TimeUnit.MILLISECONDS);
+ if (tryLock) {
+ final boolean performTryLock = performTryLock(duration, unit);
+ if (!performTryLock) {
+ unlock();
+ }
+ else {
+ attainedCompleteLock = true;
+ }
+ return performTryLock;
+ }
+ return tryLock;
+ }
+
+ public void unlock() {
+ if (isInternalLockHeldOnly()) {
+ if (attainedCompleteLock) {
+ performUnlock();
+ }
+ attainedCompleteLock = false;
+ getLock().unlock();
+ }
+ }
+
+ /**
+ * Retrieves the {@link ReentrantLock lock} based on this instances URI. It is
+ * to be noted that all instances of this URI will share the same lock.
+ * @return The lock for this lockable instance
+ */
+ protected ReentrantLock getLock() {
+ if (lock == null) {
+ lock = LockManager.getInstance().register(getURI());
+ }
+ return lock;
+ }
+
+ /**
+ * Performs additional lock operations if required by children. It will wait
+ * until it can avail for lock, but it will not wait for ever and will then
+ * throw and exception.
+ * @throws Exception If lock could be attained
+ */
+ protected abstract void performLock()
+ throws Exception;
+
+ /**
+ * Performs additional lock operations if required by children, but it will
+ * not wait for lock until beyond the time unit specified.
+ * @param time Number to wait for lock
+ * @param unit Unit of the time number
+ * @return True if lock was attained else false
+ */
+ protected abstract boolean performTryLock(long time,
+ TimeUnit unit);
+
+ /**
+ * Performs additional lock operations if required by children, but it will
+ * not wait for lock at all.
+ * @return True if lock was attained else false
+ */
+ protected abstract boolean performTryLock();
+
+ /**
+ * Performs additional unlock operations if required by children.
+ */
+ protected abstract void performUnlock();
+
+ private void childLock() {
+ try {
+ performLock();
+ attainedCompleteLock = true;
+ }
+ catch (Exception ex) {
+ unlock();
+ attainedCompleteLock = false;
+ throw new RuntimeException("Could not attain lock!", ex);
+ }
+ }
+}
diff --git a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/LockManager.java
similarity index 53%
copy from org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
copy to org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/LockManager.java
index 15af614..dfdcf21 100644
--- a/org.eclipse.jgit.io/src/org/eclipse/jgit/io/StorageSystem.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/LockManager.java
@@ -34,51 +34,77 @@
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
-package org.eclipse.jgit.io;
+package org.eclipse.jgit.io.lock;
import java.net.URI;
+import java.util.Hashtable;
+import java.util.Map;
+import java.util.concurrent.locks.ReentrantLock;
/**
- * SPI providing access to the underlying storage system. Each provider is
- * differentiated using their {@link StorageSystem#getURIProtocol() URI Protocol}.
+ * Manages reentrant lock per URI
* @author Imran M Yousuf (imyousuf at smartitengineering.com)
* @since 0.6
*/
-public interface StorageSystem {
+public final class LockManager {
- /**
- * Returns the supported scheme of this storage system.
- * @return Scheme supported by this storage system.
- * @see {@link http://tr.im/BiQ0 URI Scheme}
- */
- public String getURIScheme();
+ private static LockManager lockManager;
- /**
- * Retrieve an entry using its URI
- * @param uri URI to retrieve the entry for.
- * @return Entry representing the URI
- */
- public Entry getEntry(URI uri);
+ public static LockManager getInstance() {
+ if (lockManager == null) {
+ lockManager = new LockManager();
+ }
+ return lockManager;
+ }
+ private final Map<URI, LockProvider> locks;
- /**
- * Retrieve the current working directory from the file system if any.
- * @return Entry for current working directory.
- */
- public Entry getWorkingDirectory();
+ private LockManager() {
+ locks = new Hashtable<URI, LockProvider>();
+ }
- /**
- * Retrieve the home directory of the current user
- * @return Home directory
- */
- public Entry getHomeDirectory();
+ public synchronized ReentrantLock register(URI key) {
+ if (!locks.containsKey(key)) {
+ locks.put(key, new LockProvider(
+ new ReentrantLock()));
+ }
+ return locks.get(key).get();
+ }
- /**
- * Resolve relative path with respect to a path and return the absolute
- * entry representing the relative path.
- * @param entry The point of reference for the relative path
- * @param path The relative path
- * @return The absolute entry representing the relative path entry
- */
- public Entry resolve(Entry entry,
- String path);
+ public synchronized void unregister(URI key) {
+ if (locks.containsKey(key)) {
+ LockProvider provider = locks.get(key);
+ if (provider != null) {
+ provider.decreateCount();
+ if (provider.getRegisterCount() < 1) {
+ locks.remove(key);
+ }
+ }
+ }
+ }
+
+ private static class LockProvider {
+
+ private int registerCount = 0;
+ private ReentrantLock lock;
+
+ public LockProvider(ReentrantLock lock) {
+ this.lock = lock;
+ }
+
+ public ReentrantLock get() {
+ registerCount += 1;
+ return lock;
+ }
+
+ public void decreateCount() {
+ if (lock.isHeldByCurrentThread()) {
+ lock.unlock();
+ }
+ registerCount -= 1;
+ }
+
+ public int getRegisterCount() {
+ return registerCount;
+ }
+ }
}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/lib/RepositoryAdapter.java b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/Lockable.java
similarity index 77%
copy from org.eclipse.jgit/src/org/eclipse/jgit/lib/RepositoryAdapter.java
copy to org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/Lockable.java
index 008fef8..d2bb039 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/lib/RepositoryAdapter.java
+++ b/org.eclipse.jgit.io/src/org/eclipse/jgit/io/lock/Lockable.java
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2008, Robin Rosenberg <robin.rosenberg@dewire.com>
+ * Copyright (C) 2009, Imran M Yousuf <imyousuf@smartitengineering.com>
*
* All rights reserved.
*
@@ -34,21 +34,21 @@
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
+package org.eclipse.jgit.io.lock;
-package org.eclipse.jgit.lib;
+import java.util.concurrent.locks.Lock;
/**
- * A default {@link RepositoryListener} that does nothing except invoke an
- * optional general method for any repository change.
+ * For objects wanting to make themselves lockable.
+ * @author Imran M Yousuf (imyousuf at smartitengineering.com)
+ * @since 0.6
*/
-public class RepositoryAdapter implements RepositoryListener {
-
- public void indexChanged(final IndexChangedEvent e) {
- // Empty
- }
-
- public void refsChanged(final RefsChangedEvent e) {
- // Empty
- }
+public interface Lockable
+ extends Lock {
+ /**
+ * Retrieves whether the current thread owns the object lock or not.
+ * @return
+ */
+ public boolean isHeldByCurrentThread();
}
--
1.6.2.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation imyousuf
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 3/4] Incorporate current FileSystem Util implementations to the SPI imyousuf
@ 2009-10-12 14:57 ` Shawn O. Pearce
2009-10-13 1:30 ` Imran M Yousuf
1 sibling, 1 reply; 8+ messages in thread
From: Shawn O. Pearce @ 2009-10-12 14:57 UTC (permalink / raw)
To: imyousuf; +Cc: git, egit-dev, Imran M Yousuf
imyousuf@gmail.com wrote:
> The SPI mainly focus's in providing an API to JGit to be able to perform
> similar operations to that of java.io.File. All direct I/O is based on the
> java.io.Input/OutputStream classes.
>
> Different JGit IO SPI provider is designed to be URI scheme based and thus
> the default implementation is that of "file" scheme. SPI provider will be
> integrated by their respective users in a manner similar to that of JDBC
> driver registration. There is a SystemStorageManager that has similar
> registration capabilities and the system storage providers should be
> registered with the manager in one of the provided ways.
I think this may be a bit in the wrong direction for what we are
trying to accomplish.
A number of people really want to map Git onto what is essentially
Google's BigTable schema. Aside from Google's own BigTable product
(which I want to use Git on at work, because it would vastly simplfiy
my system administration duties at $DAYJOB) there is Cassandra and
Hadoop HBase which implement the same schema semantics.
None of those systems implement file streams, they implement cell
storage in a non-transactional system with a semi-dynamic schema.
Some people have built transactional semantics on top of these
storage layers, e.g. Google AppEngine provides multiple row
transactions through some magic sauce layered on top of BigTable.
I'm sure people will build similar tools on top of Cassandra
and HBase.
Where I'm trying to go with this is that things that are stored
in files on the filesystem in traditional Git wouldn't normally be
mapped into "byte streams" in a BigTable-ish system, or even the
JDBC-ish system you were describing.
For .git/config we might want to map config variable names into
keys in the table, with values stored in cells. This makes it
easier to query or edit the data.
Fortunately, "Config" is abstract enough that we could subclass
it with a CassandraConfig and simply use that instance when on a
based Cassandra storage system. No file streams required. Ditto
for a JdbcConfig.
For RefDatabase, we'd want to do the same and avoid the concept of
packed-refs altogether. Each Ref should go into its own row in a
Cassandra storage system, and essentially act as a loose object.
Ditto with JDBC.
We'd probably never need to read-or-write the info/refs or
objects/info/packs listings.
And I think that's everything that a bare repository needs, aside
from ObjectDatabase, which is already mostly abstract anyway.
--
Shawn.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation
2009-10-12 14:57 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation Shawn O. Pearce
@ 2009-10-13 1:30 ` Imran M Yousuf
2009-10-13 15:15 ` Shawn O. Pearce
0 siblings, 1 reply; 8+ messages in thread
From: Imran M Yousuf @ 2009-10-13 1:30 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git, egit-dev, Imran M Yousuf
On Mon, Oct 12, 2009 at 9:57 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
> imyousuf@gmail.com wrote:
>> The SPI mainly focus's in providing an API to JGit to be able to perform
>> similar operations to that of java.io.File. All direct I/O is based on the
>> java.io.Input/OutputStream classes.
>>
>> Different JGit IO SPI provider is designed to be URI scheme based and thus
>> the default implementation is that of "file" scheme. SPI provider will be
<snip />
> I think this may be a bit in the wrong direction for what we are
> trying to accomplish.
>
> A number of people really want to map Git onto what is essentially
> Google's BigTable schema. Aside from Google's own BigTable product
> (which I want to use Git on at work, because it would vastly simplfiy
> my system administration duties at $DAYJOB) there is Cassandra and
> Hadoop HBase which implement the same schema semantics.
>
> None of those systems implement file streams, they implement cell
> storage in a non-transactional system with a semi-dynamic schema.
>
> Some people have built transactional semantics on top of these
> storage layers, e.g. Google AppEngine provides multiple row
> transactions through some magic sauce layered on top of BigTable.
> I'm sure people will build similar tools on top of Cassandra
> and HBase.
>
> Where I'm trying to go with this is that things that are stored
> in files on the filesystem in traditional Git wouldn't normally be
> mapped into "byte streams" in a BigTable-ish system, or even the
> JDBC-ish system you were describing.
>
> For .git/config we might want to map config variable names into
> keys in the table, with values stored in cells. This makes it
> easier to query or edit the data.
>
> Fortunately, "Config" is abstract enough that we could subclass
> it with a CassandraConfig and simply use that instance when on a
> based Cassandra storage system. No file streams required. Ditto
> for a JdbcConfig.
>
Firstly, I am sorry but I am not intelligent enough to perceive, how
do the user decide which instance of Config to use? I personally think
that there is no API to achieve what you just mentioned :(; i.e. the
user will have know CassandraConfig directly. Secondly, I instead was
thinking of porting JGit for that matter to any system supporting
streams (not any specific sub-class of them), such HBase/BigTable or
HDFS anything.... Thirdly, I think we actually have several task in
hand and I would state them as -
1. First introduce the I/O API such that it completely replaces java.io.File
2. Secondly segregate persistence of for config (or config like
objects) and introduce a SPI for them for smarter storage.
I am not thinking of storing "only" the bare content of a git
repository, but I intent to be able to also store the versioned
contents it self as well. If we choose the above 2 steps I mentioned I
am of the opinion that we will be able
to achieve both our ideas. In addition I hope that if one day Git
itself introduces a similar I/O API then Git can also support the I/O
SPI implementations JGit will.
Waiting eagerly to learn what you think :).
Best regards,
Imran
> For RefDatabase, we'd want to do the same and avoid the concept of
> packed-refs altogether. Each Ref should go into its own row in a
> Cassandra storage system, and essentially act as a loose object.
> Ditto with JDBC.
>
> We'd probably never need to read-or-write the info/refs or
> objects/info/packs listings.
>
> And I think that's everything that a bare repository needs, aside
> from ObjectDatabase, which is already mostly abstract anyway.
>
> --
> Shawn.
>
--
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Blog: http://imyousuf-tech.blogs.smartitengineering.com/
Mobile: +880-1711402557
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation
2009-10-13 1:30 ` Imran M Yousuf
@ 2009-10-13 15:15 ` Shawn O. Pearce
2009-10-13 15:59 ` Imran M Yousuf
0 siblings, 1 reply; 8+ messages in thread
From: Shawn O. Pearce @ 2009-10-13 15:15 UTC (permalink / raw)
To: Imran M Yousuf; +Cc: git, egit-dev, Imran M Yousuf
Imran M Yousuf <imyousuf@gmail.com> wrote:
> Firstly, I am sorry but I am not intelligent enough to perceive, how
> do the user decide which instance of Config to use? I personally think
> that there is no API to achieve what you just mentioned :(; i.e. the
> user will have know CassandraConfig directly.
Yes. Well, almost.
The user will have to know that s/he wants a CassandraRepository or
a JdbcRepository in order to obtain the abstract Repository handle.
Each of these will need different configuration, possibly data which
is too complex to simply cram into a URL string, so I was expecting
the application would construct the concrete Repository class and
configure it with the proper arguments required for contact with
the underlying storage.
Since the Repository wants several things associated with it, each
concrete Repository class knows what concrete Config, ObjectDatabase
and RefDatabase it should create. Those concrete classes know how
to read a repository stored on that medium.
> Secondly, I instead was
> thinking of porting JGit for that matter to any system supporting
> streams (not any specific sub-class of them), such HBase/BigTable or
> HDFS anything.... Thirdly, I think we actually have several task in
> hand and I would state them as -
>
> 1. First introduce the I/O API such that it completely replaces java.io.File
> 2. Secondly segregate persistence of for config (or config like
> objects) and introduce a SPI for them for smarter storage.
Supporting streams on an arbitrary backend is difficult. DHTs like
BigTable/Cassandra aren't very good at providing streams, they tend
to have a limit on how big a row can be. They tend to have very
slow read latencies, but can return a small block of consecutive
rows in one reply.
I want to talk about the DHT backend more with Scott Chacon at the
GitTogether, but I have this feeling that just laying a pack file
into a stream in a DHT is going to perform very poorly.
Likewise JDBC has similar performance problems, you can only store
so much in a row before performance of the RDBMS drops off sharply.
You can get a handful of rows in a single reply pretty efficiently,
but each query takes longer than you'd like. Yes, there is often
a BLOB type that allows large file storage, but different RDBMS
support these differently and have different performance when it
comes to accessing the BLOB types. Some don't support random access,
some do. Even if they do support random access read, writing a large
2 GiB repository's pack file after repacking it would take ages.
Once you get outside of the pack file, *everything* else git stores
is either a loose object, or very tiny text files (aka refs, their
logs, config). The loose object case should be handled by the same
thing that handles the bulk of the object store, loose objects are
a trivial thing compared to packed objects.
The refs, the ref logs, and the config are all structured text.
If you lay a Git repository down into a database of some sort,
I think its reasonable to expect that the schema for these items
in that database permits query and update using relatively native
primitives in that database. E.g. if you put this in SQL I would
expect a schema like:
CREATE TABLE refs (
repository_id INT NOT NULL
,name VARCHAR(255) NOT NULL
,id CHAR(40)
,target VARCHAR(255)
,PRIMARY KEY (repository_id, name)
,CHECK (id IS NOT NULL OR target IS NOT NULL));
CREATE TABLE reflogs (
repository_id INT NOT NULL
,name VARCHAR(255) NOT NULL
,old_id CHAR(40) NOT NULL
,new_id CHAR(40) NOT NULL
,committer_name VARCHAR(255)
,committer_email VARCHAR(255)
,committer_date TIMESTAMP NOT NULL
,message VARCHAR(255)
,PRIMARY KEY (repository_id, name, committer_date));
CREATE TABLE config (
repository_id INT NOT NULL
,section VARCHAR(255) NOT NULL
,group VARCHAR(255)
,name VARCHAR(255) NOT NULL
,value VARCHAR(255)
,PRIMARY KEY(repository_id, section, group, name, value))
This makes it easier to manipulate settings, you can use direct
SQL UPDATE to modify the configuration, or SELECT to scan through
reflogs. Etc.
If we just threw everything as streams into the database this
would be a lot more difficult to work with through the database's
own native query and update interface. You'd lose alot of the
benefits of using a database, but still be paying the massive price
in performance.
> I am not thinking of storing "only" the bare content of a git
> repository, but I intent to be able to also store the versioned
> contents it self as well.
When I say "bare repository" I only mean a repository without a
working directory. It still holds the complete revision history.
If you wanted a git repository on Cassandra but wanted to actually
have a working directory checkout, you'd need the local filesystem
for the checkout and .git/index, but could otherwise keep the objects
and refs in Cassandra. Its nuts... but in theory one could do it.
--
Shawn.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation
2009-10-13 15:15 ` Shawn O. Pearce
@ 2009-10-13 15:59 ` Imran M Yousuf
0 siblings, 0 replies; 8+ messages in thread
From: Imran M Yousuf @ 2009-10-13 15:59 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git, egit-dev, Imran M Yousuf
Hi Shawn,
Firstly thanks for the reply, my comments are inline.
On Tue, Oct 13, 2009 at 10:15 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
> Imran M Yousuf <imyousuf@gmail.com> wrote:
>> Firstly, I am sorry but I am not intelligent enough to perceive, how
>> do the user decide which instance of Config to use? I personally think
>> that there is no API to achieve what you just mentioned :(; i.e. the
>> user will have know CassandraConfig directly.
>
> Yes. Well, almost.
>
> The user will have to know that s/he wants a CassandraRepository or
> a JdbcRepository in order to obtain the abstract Repository handle.
> Each of these will need different configuration, possibly data which
> is too complex to simply cram into a URL string, so I was expecting
> the application would construct the concrete Repository class and
> configure it with the proper arguments required for contact with
> the underlying storage.
>
> Since the Repository wants several things associated with it, each
> concrete Repository class knows what concrete Config, ObjectDatabase
> and RefDatabase it should create. Those concrete classes know how
> to read a repository stored on that medium.
>
Hmm, when trying to come up with an API, where I essentially wanted a
to abstract all records, I noticed that everything uses java.io.File
and I never actually thought in this line.
Well, I was thinking of in terms of URI as I think Git and in turn
JGit (some how feels so) follows REST and from my understanding of git
storage (which very well could be incorrect) URI could be a perfect
match. Question of how a implementation requiring configuration be fed
to the SPI manager is simple before JGit is used the instance it self
is registered to the manager. So e.g. for JDBC the URI could very well
look like - jdbc://etc/X11/ for a repo path and the JDBC implementor
will already know the connection config specs etc. so there is no need
to cramp in all info in the URL.
>> Secondly, I instead was
>> thinking of porting JGit for that matter to any system supporting
>> streams (not any specific sub-class of them), such HBase/BigTable or
>> HDFS anything.... Thirdly, I think we actually have several task in
>> hand and I would state them as -
>>
>> 1. First introduce the I/O API such that it completely replaces java.io.File
>> 2. Secondly segregate persistence of for config (or config like
>> objects) and introduce a SPI for them for smarter storage.
>
> Supporting streams on an arbitrary backend is difficult. DHTs like
> BigTable/Cassandra aren't very good at providing streams, they tend
> to have a limit on how big a row can be. They tend to have very
> slow read latencies, but can return a small block of consecutive
> rows in one reply.
>
> I want to talk about the DHT backend more with Scott Chacon at the
> GitTogether, but I have this feeling that just laying a pack file
> into a stream in a DHT is going to perform very poorly.
>
> Likewise JDBC has similar performance problems, you can only store
> so much in a row before performance of the RDBMS drops off sharply.
> You can get a handful of rows in a single reply pretty efficiently,
> but each query takes longer than you'd like. Yes, there is often
> a BLOB type that allows large file storage, but different RDBMS
> support these differently and have different performance when it
> comes to accessing the BLOB types. Some don't support random access,
> some do. Even if they do support random access read, writing a large
> 2 GiB repository's pack file after repacking it would take ages.
>
> Once you get outside of the pack file, *everything* else git stores
> is either a loose object, or very tiny text files (aka refs, their
> logs, config). The loose object case should be handled by the same
> thing that handles the bulk of the object store, loose objects are
> a trivial thing compared to packed objects.
>
> The refs, the ref logs, and the config are all structured text.
> If you lay a Git repository down into a database of some sort,
> I think its reasonable to expect that the schema for these items
> in that database permits query and update using relatively native
> primitives in that database. E.g. if you put this in SQL I would
> expect a schema like:
>
> CREATE TABLE refs (
> repository_id INT NOT NULL
> ,name VARCHAR(255) NOT NULL
> ,id CHAR(40)
> ,target VARCHAR(255)
> ,PRIMARY KEY (repository_id, name)
> ,CHECK (id IS NOT NULL OR target IS NOT NULL));
>
> CREATE TABLE reflogs (
> repository_id INT NOT NULL
> ,name VARCHAR(255) NOT NULL
> ,old_id CHAR(40) NOT NULL
> ,new_id CHAR(40) NOT NULL
> ,committer_name VARCHAR(255)
> ,committer_email VARCHAR(255)
> ,committer_date TIMESTAMP NOT NULL
> ,message VARCHAR(255)
> ,PRIMARY KEY (repository_id, name, committer_date));
>
> CREATE TABLE config (
> repository_id INT NOT NULL
> ,section VARCHAR(255) NOT NULL
> ,group VARCHAR(255)
> ,name VARCHAR(255) NOT NULL
> ,value VARCHAR(255)
> ,PRIMARY KEY(repository_id, section, group, name, value))
>
> This makes it easier to manipulate settings, you can use direct
> SQL UPDATE to modify the configuration, or SELECT to scan through
> reflogs. Etc.
>
> If we just threw everything as streams into the database this
> would be a lot more difficult to work with through the database's
> own native query and update interface. You'd lose alot of the
> benefits of using a database, but still be paying the massive price
> in performance.
>
To be honest I understood your point the last time you mentioned it
:). I agree with performance part fully and I too have doubts, that is
why while you were mentioning HBase, I was HDFS :) and I was actually
thinking of in terms of FS. But after your elaboration, it just makes
more sense for the changes -
* Firstly we decouple from a particular FS and have our own I/O, for
packed and loose objects, so that we can easily retain the current
behavior.
* Then we first rework the key objects, e.g. Refs, Configs etc. to
segregate their persistence, that is introduce their persistence layer
which should friendly enough for native operations for platforms such
as RDBMS or HBase or BigTable. Using packs will depend of setup and
repositories, but certain implementations
* We implement this SPI to support different persistence platforms then :).
The thing is I first want to make JGit independent of java.io.File :),
thats was my motto to start with, but you showed me a path beyond that
:) and that is free from java.io.File and optimized persistence API
:). I want to have them both and let the implementor choose how to
implement it :).
>> I am not thinking of storing "only" the bare content of a git
>> repository, but I intent to be able to also store the versioned
>> contents it self as well.
>
> When I say "bare repository" I only mean a repository without a
> working directory. It still holds the complete revision history.
> If you wanted a git repository on Cassandra but wanted to actually
> have a working directory checkout, you'd need the local filesystem
> for the checkout and .git/index, but could otherwise keep the objects
> and refs in Cassandra. Its nuts... but in theory one could do it.
>
My requirement also involves needing to check it out on HDFS :), that
is why I was mentioning it, but it could be a different topic other
than that of JGit.
Eagerly waiting for a reply.
Thank you,
--
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Blog: http://imyousuf-tech.blogs.smartitengineering.com/
Mobile: +880-1711402557
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2009-10-13 16:05 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-10-11 14:07 [JGit-io-RFC-PATCH v2 1/4] Introduce a new module for IO SPI of JGit imyousuf
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation imyousuf
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 3/4] Incorporate current FileSystem Util implementations to the SPI imyousuf
2009-10-11 14:07 ` [JGit-io-RFC-PATCH v2 4/4] Add locking capability to the IO SPI based on Java Concurrency Lock API imyousuf
2009-10-12 14:57 ` [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation Shawn O. Pearce
2009-10-13 1:30 ` Imran M Yousuf
2009-10-13 15:15 ` Shawn O. Pearce
2009-10-13 15:59 ` Imran M Yousuf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).