[PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[Nguyễn Thái Ngọc Duy]

add_submodule_odb() can be used to import objects from another
repository temporarily. After this point we don't know which objects
are ours, which are external. If we create an object that refers to an
external object, next time git runs, it may find a hole in the object
graph because the external repository may not be imported. The same
goes for pointing a ref to an external SHA-1.

To protect ourselves, once add_submodule_odb() is used:

 - trees, tags and commits cannot be created
 - refs cannot be updated

In certain cases that submodule code knows that it's safe to write, it
can turn the readonly flag off.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 I think this is a good safety check. It catches at least a case in
 t7405.3. I did not investigate further though.

 cache.h      | 1 +
 refs.c       | 2 ++
 sha1_file.c  | 2 ++
 submodule.c  | 7 +++++++
 5 files changed, 16 insertions(+)

diff --git a/cache.h b/cache.h
index c257953..772d229 100644
--- a/cache.h
+++ b/cache.h
@@ -753,6 +753,7 @@ extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
+extern int git_repo_readonly();
 
 /* global flag to enable extra checks when accessing packed objects */
 extern int do_check_packed_object_crc;
diff --git a/refs.c b/refs.c
index 541fec2..22b13f4 100644
--- a/refs.c
+++ b/refs.c
@@ -1711,6 +1711,8 @@ struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha
 struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1, int flags)
 {
+	if (git_repo_readonly())
+		die("repository in read-only mode, cannot update refs");
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		return NULL;
 	return lock_ref_sha1_basic(refname, old_sha1, flags, NULL);
diff --git a/sha1_file.c b/sha1_file.c
index 40b2329..b9e8b59 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -2575,6 +2575,8 @@ int write_sha1_file(const void *buf, unsigned long len, const char *type, unsign
 	char hdr[32];
 	int hdrlen;
 
+	if (git_repo_readonly() && strcmp(type, "blob"))
+		die("repository in read-only mode, cannot update object database");
 	/* Normally if we have it in the pack then we do not bother writing
 	 * it out into .git/objects/??/?{38} file.
 	 */
diff --git a/submodule.c b/submodule.c
index 2f55436..5eba597 100644
--- a/submodule.c
+++ b/submodule.c
@@ -19,6 +19,7 @@ static struct string_list changed_submodule_paths;
 static int initialized_fetch_ref_tips;
 static struct sha1_array ref_tips_before_fetch;
 static struct sha1_array ref_tips_after_fetch;
+static int readonly;
 
 /*
  * The following flag is set if the .gitmodules file is unmerged. We then
@@ -30,6 +31,11 @@ static struct sha1_array ref_tips_after_fetch;
  */
 static int gitmodules_is_unmerged;
 
+int git_repo_readonly()
+{
+	return readonly;
+}
+
 static int add_submodule_odb(const char *path)
 {
 	struct strbuf objects_directory = STRBUF_INIT;
@@ -67,6 +73,7 @@ static int add_submodule_odb(const char *path)
 	/* add possible alternates from the submodule */
 	read_info_alternates(objects_directory.buf, 0);
 	prepare_alt_odb();
+	readonly = 1;
 done:
 	strbuf_release(&objects_directory);
 	return ret;
-- 
1.8.0.rc2.23.g1fb49df

Re: [PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[Duy Nguyen]

On Wed, Jan 23, 2013 at 8:34 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> add_submodule_odb() can be used to import objects from another
> repository temporarily. After this point we don't know which objects
> are ours, which are external. If we create an object that refers to an
> external object, next time git runs, it may find a hole in the object
> graph because the external repository may not be imported. The same
> goes for pointing a ref to an external SHA-1.
>
> To protect ourselves, once add_submodule_odb() is used:
>
>  - trees, tags and commits cannot be created
>  - refs cannot be updated

.. and soon after I pressed send, I realized I missed at least two
other places write should not be allowed:

 - index
 - reflog

But the general idea is probably more important than implementation
details at this stage.
-- 
Duy

Re: [PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> add_submodule_odb() can be used to import objects from another
> repository temporarily. After this point we don't know which objects
> are ours, which are external. If we create an object that refers to an
> external object, next time git runs, it may find a hole in the object
> graph because the external repository may not be imported. The same
> goes for pointing a ref to an external SHA-1.
>
> To protect ourselves, once add_submodule_odb() is used:
>
>  - trees, tags and commits cannot be created
>  - refs cannot be updated
>
> In certain cases that submodule code knows that it's safe to write, it
> can turn the readonly flag off.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  I think this is a good safety check.

Two step implementation to bring "read-only" support into a testable
shape and then flip that bit in add_submdule_odb() would be a
sensible approach.

I however have this suspicion that this will become a losing battle
and we would be better off getting rid of add_submodule_odb();
instead operations that work across repositories will be done as a
subprocess, which will get us back closer to one of the original
design goals of submodule support to have a clear separation between
the superproject and its submodules.

Re: [PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[Jens Lehmann]

Am 23.01.2013 18:01, schrieb Junio C Hamano:
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
> 
>> add_submodule_odb() can be used to import objects from another
>> repository temporarily. After this point we don't know which objects
>> are ours, which are external. If we create an object that refers to an
>> external object, next time git runs, it may find a hole in the object
>> graph because the external repository may not be imported. The same
>> goes for pointing a ref to an external SHA-1.
>>
>> To protect ourselves, once add_submodule_odb() is used:
>>
>>  - trees, tags and commits cannot be created
>>  - refs cannot be updated
>>
>> In certain cases that submodule code knows that it's safe to write, it
>> can turn the readonly flag off.
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>>  I think this is a good safety check.
> 
> Two step implementation to bring "read-only" support into a testable
> shape and then flip that bit in add_submdule_odb() would be a
> sensible approach.

I agree this is a worthwhile change so nobody accidentally screws
things up.

>>  It catches at least a case in
>>  t7405.3. I did not investigate further though.

This is a false positive. The merge algorithm picked a fast-forward
in a submodule as a proper merge result and records that in a
gitlink. But as Duy pointed out this could be easily fixed by
turning the readonly flag off in that case.

> I however have this suspicion that this will become a losing battle
> and we would be better off getting rid of add_submodule_odb();
> instead operations that work across repositories will be done as a
> subprocess, which will get us back closer to one of the original
> design goals of submodule support to have a clear separation between
> the superproject and its submodules.

Please don't. While I agree with your goal, I'd be unhappy to do
that because of the performance drop (especially on fork-challenged
operating systems).

Re: [PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> This is a false positive. The merge algorithm picked a fast-forward
> in a submodule as a proper merge result and records that in a
> gitlink. But as Duy pointed out this could be easily fixed by
> turning the readonly flag off in that case.

I see that as "easily circumvented and not an effective protection",
though.

In theory, adding a gitlink to the index, removing a gitlink to the
index and modifying an existing gitlink in the index to another
gitlink in the index and writing the resulting in-core index out to
the on-disk index should be allowed, even after objects from the
submodule object database have contaminated our in-core object pool,
as long as you do not run cache_tree_update().  I am not sure if that
single loophole would be sufficient, though.

Re: [PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[Duy Nguyen]

On Thu, Jan 24, 2013 at 12:01 AM, Junio C Hamano <gitster@pobox.com> wrote:
> I however have this suspicion that this will become a losing battle
> and we would be better off getting rid of add_submodule_odb();
> instead operations that work across repositories will be done as a
> subprocess, which will get us back closer to one of the original
> design goals of submodule support to have a clear separation between
> the superproject and its submodules.

It does not have to be subprocess. Thomas Rast did some work on
support multithread access to object db by basically replicating all
datastructure per thread. If that work is complete, we have something
like "odb container" that could be used to access objects from another
repository and it won't contaminate the original odb. The same thing
can be done for ref and index access.
-- 
Duy

Re: [PATCH/RFC] Revoke write access to refs and odb after importing another repo's odb

[Duy Nguyen]

On Thu, Jan 24, 2013 at 4:06 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
>
>> This is a false positive. The merge algorithm picked a fast-forward
>> in a submodule as a proper merge result and records that in a
>> gitlink. But as Duy pointed out this could be easily fixed by
>> turning the readonly flag off in that case.
>
> I see that as "easily circumvented and not an effective protection",
> though.
>
> In theory, adding a gitlink to the index, removing a gitlink to the
> index and modifying an existing gitlink in the index to another
> gitlink in the index and writing the resulting in-core index out to
> the on-disk index should be allowed, even after objects from the
> submodule object database have contaminated our in-core object pool,
> as long as you do not run cache_tree_update().  I am not sure if that
> single loophole would be sufficient, though.

The problem is we don't know which entries are updated in index. We
don't keep track of them. And I think in the unpack-trees case, we
scape the whole index then copy over, making it look like the whole
index is updated (even with the same content). One way to check this
is verify the source of all non-gitlink entries in index before
writing to disk (only when readonly flag is on, of course).
sha1_object_info_extended() should help (or be extended to do the
job). Hmm.. if we do this, we could also verify if new sha-1 objects
do not refer to an external source, if so allow them to be created.
-- 
Duy