[PATCH 0/8] update-ref: add support for update-symref option

[PATCH 0/8] update-ref: add support for update-symref option

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'git-update-ref(1)' command allows transactional reference updates.
But currently only supports regular reference updates. Meaning, if one
wants to update HEAD (symbolic ref) in a transaction, there is no tool
to do so.

One option to obtain transactional updates for the HEAD ref is to
manually create the HEAD.lock file and commit. This is intrusive, where
the user needs to mimic internal git behavior. Also, this only works
when using the files backend.

At GitLab, we've been using the manual process till date, to allow users
to set and change their default branch. But with the introduction of
reftables as a reference backend, this becomes a necessity to be solved
within git.

This patch series goes about introducing 'update-symref' command for
'git-update-ref(1)'. This command allows the user to create symref in a
transaction similar to the 'update' command of 'git-update-ref(1)'. This
command also works well with the existing 'no-deref' option.

We don't create a dedicated 'create-symref' since 'update-symref' can
also create symrefs. We don't create 'delete-symref', since the 'delete'
command can also delete symrefs.

Karthik Nayak (8):
  files-backend: extract out `create_symref_lock`
  reftable-backend: extract out `write_symref_with_log`
  reftable-backend: move `write_symref_with_log` up
  refs: accept symref in `ref_transaction_add_update`
  refs/files-backend: add support for symref updates
  refs/reftable-backend: add support for symref updates
  refs: add 'update-symref' command to 'update-ref'
  refs: support symrefs in 'reference-transaction' hook

 Documentation/git-update-ref.txt |  11 ++-
 Documentation/githooks.txt       |  13 ++-
 branch.c                         |   2 +-
 builtin/fast-import.c            |   6 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   2 +-
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   2 +-
 builtin/update-ref.c             |  63 +++++++++++---
 refs.c                           |  37 ++++++---
 refs.h                           |   9 +-
 refs/files-backend.c             |  95 ++++++++++++++++-----
 refs/refs-internal.h             |  12 ++-
 refs/reftable-backend.c          | 136 ++++++++++++++++++-------------
 sequencer.c                      |   6 +-
 t/t0600-reffiles-backend.sh      |  30 +++++++
 t/t1400-update-ref.sh            | 127 +++++++++++++++++++++++++++++
 t/t1416-ref-transaction-hooks.sh |  27 ++++++
 walker.c                         |   2 +-
 19 files changed, 464 insertions(+), 120 deletions(-)

-- 
2.43.GIT

[PATCH 1/8] files-backend: extract out `create_symref_lock`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Split this into two individual functions `create_and_commit_symref` and
`create_symref_locked`. This way we can create the symref lock and
commit it at different times. This will be used to provide symref
support in `git-update-ref(1)`.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 40 +++++++++++++++++++++++++++-------------
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index a098d14ea0..3f0f9521cb 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1920,26 +1920,39 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target)
 {
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	/* no error check; commit_ref will check ferror */
+	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
 	return 0;
 }
 
@@ -1960,7 +1973,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

Re: [PATCH 1/8] files-backend: extract out `create_symref_lock`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3625 bytes --]

On Sat, Mar 30, 2024 at 11:46:16PM +0100, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The function `create_symref_locked` creates a symref by creating a
> '<symref>.lock' file and then committing the symref lock, which creates
> the final symref.
> 
> Split this into two individual functions `create_and_commit_symref` and
> `create_symref_locked`. This way we can create the symref lock and
> commit it at different times. This will be used to provide symref
> support in `git-update-ref(1)`.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  refs/files-backend.c | 40 +++++++++++++++++++++++++++-------------
>  1 file changed, 27 insertions(+), 13 deletions(-)
> 
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index a098d14ea0..3f0f9521cb 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -1920,26 +1920,39 @@ static void update_symref_reflog(struct files_ref_store *refs,
>  	}
>  }
>  
> -static int create_symref_locked(struct files_ref_store *refs,
> -				struct ref_lock *lock, const char *refname,
> -				const char *target, const char *logmsg)
> +static int create_symref_lock(struct files_ref_store *refs,
> +			      struct ref_lock *lock, const char *refname,
> +			      const char *target)
>  {
> +	if (!fdopen_lock_file(&lock->lk, "w"))
> +		return error("unable to fdopen %s: %s",
> +			     get_lock_file_path(&lock->lk), strerror(errno));
> +
> +	/* no error check; commit_ref will check ferror */
> +	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
> +	return 0;
> +}
> +
> +static int create_and_commit_symref(struct files_ref_store *refs,
> +				    struct ref_lock *lock, const char *refname,
> +				    const char *target, const char *logmsg)
> +{
> +	int ret;
> +
>  	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
>  		update_symref_reflog(refs, lock, refname, target, logmsg);
>  		return 0;
>  	}
>  
> -	if (!fdopen_lock_file(&lock->lk, "w"))
> -		return error("unable to fdopen %s: %s",
> -			     get_lock_file_path(&lock->lk), strerror(errno));
> +	ret = create_symref_lock(refs, lock, refname, target);
> +	if (!ret) {
> +		update_symref_reflog(refs, lock, refname, target, logmsg);

I feel like the resulting code here is a bit hard to read because the
successful path is now nested into the condition. This does not really
conform to our typical coding style. Exiting early in case the function
returns an error would be easier to read.

> -	update_symref_reflog(refs, lock, refname, target, logmsg);
> +		if (commit_ref(lock) < 0)
> +			return error("unable to write symref for %s: %s", refname,
> +				     strerror(errno));
> +	}
>  
> -	/* no error check; commit_ref will check ferror */
> -	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
> -	if (commit_ref(lock) < 0)
> -		return error("unable to write symref for %s: %s", refname,
> -			     strerror(errno));
>  	return 0;

Also, is it correct to `return 0` here in case `create_symref_lock()`
returns an error? If so it certainly requires an in-code comment to
explain what is going on. If this is a bug I feel like we have
identified a test gap that we might want to plug.

Patrick

>  }
>  
> @@ -1960,7 +1973,8 @@ static int files_create_symref(struct ref_store *ref_store,
>  		return -1;
>  	}
>  
> -	ret = create_symref_locked(refs, lock, refname, target, logmsg);
> +	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
> +
>  	unlock_ref(lock);
>  	return ret;
>  }
> -- 
> 2.43.GIT
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/8] files-backend: extract out `create_symref_lock`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]

Patrick Steinhardt <ps@pks.im> writes:
>> +static int create_and_commit_symref(struct files_ref_store *refs,
>> +				    struct ref_lock *lock, const char *refname,
>> +				    const char *target, const char *logmsg)
>> +{
>> +	int ret;
>> +
>>  	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
>>  		update_symref_reflog(refs, lock, refname, target, logmsg);
>>  		return 0;
>>  	}
>>
>> -	if (!fdopen_lock_file(&lock->lk, "w"))
>> -		return error("unable to fdopen %s: %s",
>> -			     get_lock_file_path(&lock->lk), strerror(errno));
>> +	ret = create_symref_lock(refs, lock, refname, target);
>> +	if (!ret) {
>> +		update_symref_reflog(refs, lock, refname, target, logmsg);
>
> I feel like the resulting code here is a bit hard to read because the
> successful path is now nested into the condition. This does not really
> conform to our typical coding style. Exiting early in case the function
> returns an error would be easier to read.

Agreed, will modify this to exit early.

>> -	update_symref_reflog(refs, lock, refname, target, logmsg);
>> +		if (commit_ref(lock) < 0)
>> +			return error("unable to write symref for %s: %s", refname,
>> +				     strerror(errno));
>> +	}
>>
>> -	/* no error check; commit_ref will check ferror */
>> -	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
>> -	if (commit_ref(lock) < 0)
>> -		return error("unable to write symref for %s: %s", refname,
>> -			     strerror(errno));
>>  	return 0;
>
> Also, is it correct to `return 0` here in case `create_symref_lock()`
> returns an error? If so it certainly requires an in-code comment to
> explain what is going on. If this is a bug I feel like we have
> identified a test gap that we might want to plug.
>

It's wrong, we should definitely be catching and returning that error.
Regarding fixing the test gap, it is kinda hard to do so, since this
is capturing a filesystem error. It would be easier to do so in the
upcoming commits where I could possibly do:

1. Start transaction
2. Add symref creation to transaction
3. Before preparing the transaction, manually create the lock file.
4. This should hit this error.

I'll add something in corresponding commit.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH 2/8] reftable-backend: extract out `write_symref_with_log`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `write_create_symref_table`, creates a
`reftable_ref_record` for a symref and adds it to the writer. Then it
also creates a log entry for the symref. It does all of this while also
obtaining and using a new update index.

We extract out `write_symref_with_log` from this to provide the
functionality of creating a symref without making changes to the update
index. This will be used to add `update-symref` option to the
`git-update-ref` command.

Rename the `create` field to `arg` while we're here, as `create` is a
bit misleading.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/reftable-backend.c | 41 +++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index e206d5a073..282a08e3cb 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1222,23 +1222,22 @@ struct write_create_symref_arg {
 	const char *logmsg;
 };
 
-static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
+static int write_symref_with_log(struct reftable_writer *writer,
+				 struct write_create_symref_arg *arg,
+				 uint64_t update_index)
 {
-	struct write_create_symref_arg *create = cb_data;
-	uint64_t ts = reftable_stack_next_update_index(create->stack);
 	struct reftable_ref_record ref = {
-		.refname = (char *)create->refname,
+		.refname = (char *)arg->refname,
 		.value_type = REFTABLE_REF_SYMREF,
-		.value.symref = (char *)create->target,
-		.update_index = ts,
+		.value.symref = (char *)arg->target,
+		.update_index = update_index,
 	};
+
 	struct reftable_log_record log = {0};
 	struct object_id new_oid;
 	struct object_id old_oid;
 	int ret;
 
-	reftable_writer_set_limits(writer, ts, ts);
-
 	ret = reftable_writer_add_ref(writer, &ref);
 	if (ret)
 		return ret;
@@ -1251,25 +1250,35 @@ static int write_create_symref_table(struct reftable_writer *writer, void *cb_da
 	 * not resolve for new repositories this ordering will ensure that this
 	 * never happens.
 	 */
-	if (!create->logmsg ||
-	    !refs_resolve_ref_unsafe(&create->refs->base, create->target,
+	if (!arg->logmsg ||
+	    !refs_resolve_ref_unsafe(&arg->refs->base, arg->target,
 				     RESOLVE_REF_READING, &new_oid, NULL) ||
-	    !should_write_log(&create->refs->base, create->refname))
+	    !should_write_log(&arg->refs->base, arg->refname))
 		return 0;
 
 	fill_reftable_log_record(&log);
-	log.refname = xstrdup(create->refname);
-	log.update_index = ts;
-	log.value.update.message = xstrndup(create->logmsg,
-					    create->refs->write_options.block_size / 2);
+	log.refname = xstrdup(arg->refname);
+	log.update_index = update_index;
+	log.value.update.message = xstrndup(arg->logmsg,
+					    arg->refs->write_options.block_size / 2);
 	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
-	if (refs_resolve_ref_unsafe(&create->refs->base, create->refname,
+	if (refs_resolve_ref_unsafe(&arg->refs->base, arg->refname,
 				    RESOLVE_REF_READING, &old_oid, NULL))
 		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
 
 	ret = reftable_writer_add_log(writer, &log);
 	reftable_log_record_release(&log);
 	return ret;
+
+}
+
+static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
+{
+	struct write_create_symref_arg *arg = cb_data;
+	uint64_t ts = reftable_stack_next_update_index(arg->stack);
+	reftable_writer_set_limits(writer, ts, ts);
+
+	return write_symref_with_log(writer, arg, ts);
 }
 
 static int reftable_be_create_symref(struct ref_store *ref_store,
-- 
2.43.GIT

Re: [PATCH 2/8] reftable-backend: extract out `write_symref_with_log`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 4069 bytes --]

On Sat, Mar 30, 2024 at 11:46:17PM +0100, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The function `write_create_symref_table`, creates a
> `reftable_ref_record` for a symref and adds it to the writer. Then it
> also creates a log entry for the symref. It does all of this while also
> obtaining and using a new update index.
> 
> We extract out `write_symref_with_log` from this to provide the
> functionality of creating a symref without making changes to the update
> index. This will be used to add `update-symref` option to the
> `git-update-ref` command.
> 
> Rename the `create` field to `arg` while we're here, as `create` is a
> bit misleading.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  refs/reftable-backend.c | 41 +++++++++++++++++++++++++----------------
>  1 file changed, 25 insertions(+), 16 deletions(-)
> 
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index e206d5a073..282a08e3cb 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -1222,23 +1222,22 @@ struct write_create_symref_arg {
>  	const char *logmsg;
>  };
>  
> -static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
> +static int write_symref_with_log(struct reftable_writer *writer,
> +				 struct write_create_symref_arg *arg,
> +				 uint64_t update_index)
>  {
> -	struct write_create_symref_arg *create = cb_data;
> -	uint64_t ts = reftable_stack_next_update_index(create->stack);
>  	struct reftable_ref_record ref = {
> -		.refname = (char *)create->refname,
> +		.refname = (char *)arg->refname,
>  		.value_type = REFTABLE_REF_SYMREF,
> -		.value.symref = (char *)create->target,
> -		.update_index = ts,
> +		.value.symref = (char *)arg->target,
> +		.update_index = update_index,
>  	};
> +

Nit: let's remove this superfluous newline.

Patrick

>  	struct reftable_log_record log = {0};
>  	struct object_id new_oid;
>  	struct object_id old_oid;
>  	int ret;
>  
> -	reftable_writer_set_limits(writer, ts, ts);
> -
>  	ret = reftable_writer_add_ref(writer, &ref);
>  	if (ret)
>  		return ret;
> @@ -1251,25 +1250,35 @@ static int write_create_symref_table(struct reftable_writer *writer, void *cb_da
>  	 * not resolve for new repositories this ordering will ensure that this
>  	 * never happens.
>  	 */
> -	if (!create->logmsg ||
> -	    !refs_resolve_ref_unsafe(&create->refs->base, create->target,
> +	if (!arg->logmsg ||
> +	    !refs_resolve_ref_unsafe(&arg->refs->base, arg->target,
>  				     RESOLVE_REF_READING, &new_oid, NULL) ||
> -	    !should_write_log(&create->refs->base, create->refname))
> +	    !should_write_log(&arg->refs->base, arg->refname))
>  		return 0;
>  
>  	fill_reftable_log_record(&log);
> -	log.refname = xstrdup(create->refname);
> -	log.update_index = ts;
> -	log.value.update.message = xstrndup(create->logmsg,
> -					    create->refs->write_options.block_size / 2);
> +	log.refname = xstrdup(arg->refname);
> +	log.update_index = update_index;
> +	log.value.update.message = xstrndup(arg->logmsg,
> +					    arg->refs->write_options.block_size / 2);
>  	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
> -	if (refs_resolve_ref_unsafe(&create->refs->base, create->refname,
> +	if (refs_resolve_ref_unsafe(&arg->refs->base, arg->refname,
>  				    RESOLVE_REF_READING, &old_oid, NULL))
>  		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
>  
>  	ret = reftable_writer_add_log(writer, &log);
>  	reftable_log_record_release(&log);
>  	return ret;
> +
> +}
> +
> +static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
> +{
> +	struct write_create_symref_arg *arg = cb_data;
> +	uint64_t ts = reftable_stack_next_update_index(arg->stack);
> +	reftable_writer_set_limits(writer, ts, ts);
> +
> +	return write_symref_with_log(writer, arg, ts);
>  }
>  
>  static int reftable_be_create_symref(struct ref_store *ref_store,
> -- 
> 2.43.GIT
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 3/8] reftable-backend: move `write_symref_with_log` up

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `write_symref_with_log` code will be consequently used to provide
symref creation functionality in transactions. To do this, we move the
declaration up so it can be used accordingly.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/reftable-backend.c | 116 ++++++++++++++++++++--------------------
 1 file changed, 58 insertions(+), 58 deletions(-)

diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 282a08e3cb..9b53d42541 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -588,6 +588,64 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+struct write_create_symref_arg {
+	struct reftable_ref_store *refs;
+	struct reftable_stack *stack;
+	const char *refname;
+	const char *target;
+	const char *logmsg;
+};
+
+static int write_symref_with_log(struct reftable_writer *writer,
+				 struct write_create_symref_arg *arg,
+				 uint64_t update_index)
+{
+	struct reftable_ref_record ref = {
+		.refname = (char *)arg->refname,
+		.value_type = REFTABLE_REF_SYMREF,
+		.value.symref = (char *)arg->target,
+		.update_index = update_index,
+	};
+
+	struct reftable_log_record log = {0};
+	struct object_id new_oid;
+	struct object_id old_oid;
+	int ret;
+
+	ret = reftable_writer_add_ref(writer, &ref);
+	if (ret)
+		return ret;
+
+	/*
+	 * Note that it is important to try and resolve the reference before we
+	 * write the log entry. This is because `should_write_log()` will munge
+	 * `core.logAllRefUpdates`, which is undesirable when we create a new
+	 * repository because it would be written into the config. As HEAD will
+	 * not resolve for new repositories this ordering will ensure that this
+	 * never happens.
+	 */
+	if (!arg->logmsg ||
+	    !refs_resolve_ref_unsafe(&arg->refs->base, arg->target,
+				     RESOLVE_REF_READING, &new_oid, NULL) ||
+	    !should_write_log(&arg->refs->base, arg->refname))
+		return 0;
+
+	fill_reftable_log_record(&log);
+	log.refname = xstrdup(arg->refname);
+	log.update_index = update_index;
+	log.value.update.message = xstrndup(arg->logmsg,
+					    arg->refs->write_options.block_size / 2);
+	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
+	if (refs_resolve_ref_unsafe(&arg->refs->base, arg->refname,
+				    RESOLVE_REF_READING, &old_oid, NULL))
+		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
+
+	ret = reftable_writer_add_log(writer, &log);
+	reftable_log_record_release(&log);
+	return ret;
+
+}
+
 struct reftable_transaction_update {
 	struct ref_update *update;
 	struct object_id current_oid;
@@ -1214,64 +1272,6 @@ static int reftable_be_pack_refs(struct ref_store *ref_store,
 	return ret;
 }
 
-struct write_create_symref_arg {
-	struct reftable_ref_store *refs;
-	struct reftable_stack *stack;
-	const char *refname;
-	const char *target;
-	const char *logmsg;
-};
-
-static int write_symref_with_log(struct reftable_writer *writer,
-				 struct write_create_symref_arg *arg,
-				 uint64_t update_index)
-{
-	struct reftable_ref_record ref = {
-		.refname = (char *)arg->refname,
-		.value_type = REFTABLE_REF_SYMREF,
-		.value.symref = (char *)arg->target,
-		.update_index = update_index,
-	};
-
-	struct reftable_log_record log = {0};
-	struct object_id new_oid;
-	struct object_id old_oid;
-	int ret;
-
-	ret = reftable_writer_add_ref(writer, &ref);
-	if (ret)
-		return ret;
-
-	/*
-	 * Note that it is important to try and resolve the reference before we
-	 * write the log entry. This is because `should_write_log()` will munge
-	 * `core.logAllRefUpdates`, which is undesirable when we create a new
-	 * repository because it would be written into the config. As HEAD will
-	 * not resolve for new repositories this ordering will ensure that this
-	 * never happens.
-	 */
-	if (!arg->logmsg ||
-	    !refs_resolve_ref_unsafe(&arg->refs->base, arg->target,
-				     RESOLVE_REF_READING, &new_oid, NULL) ||
-	    !should_write_log(&arg->refs->base, arg->refname))
-		return 0;
-
-	fill_reftable_log_record(&log);
-	log.refname = xstrdup(arg->refname);
-	log.update_index = update_index;
-	log.value.update.message = xstrndup(arg->logmsg,
-					    arg->refs->write_options.block_size / 2);
-	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
-	if (refs_resolve_ref_unsafe(&arg->refs->base, arg->refname,
-				    RESOLVE_REF_READING, &old_oid, NULL))
-		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
-
-	ret = reftable_writer_add_log(writer, &log);
-	reftable_log_record_release(&log);
-	return ret;
-
-}
-
 static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
 {
 	struct write_create_symref_arg *arg = cb_data;
-- 
2.43.GIT

[PATCH 4/8] refs: accept symref in `ref_transaction_add_update`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `ref_transaction_add_update` function obtains ref information and
flags to create a `ref_update` and add it to the transaction at hand.

To extend symref support in transactions, we need to also accept the
symref and process it. While we handle the processing in upcoming
commits. In this commit, let's add the paramater to the function.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  6 +++---
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  2 +-
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  2 +-
 builtin/update-ref.c    |  2 +-
 refs.c                  | 16 ++++++++--------
 refs.h                  |  9 +++++++--
 refs/files-backend.c    | 14 +++++++-------
 refs/refs-internal.h    | 12 +++++++++---
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  6 +++---
 walker.c                |  2 +-
 14 files changed, 46 insertions(+), 35 deletions(-)

diff --git a/branch.c b/branch.c
index 621019fcf4..266dcb6a89 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					0, msg, NULL, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 71a195ca22..96c3fd5a30 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1648,7 +1648,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   0, msg, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1688,8 +1688,8 @@ static void dump_tags(void)
 		strbuf_reset(&ref_name);
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
-		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+		if (ref_transaction_update(transaction, ref_name.buf, &t->oid,
+					   NULL, 0, msg, NULL, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 46a793411a..5a8a58b6fa 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -667,7 +667,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     0, msg, NULL, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 56d8a77ed7..5318bc4b58 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1596,7 +1596,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 					   namespaced_name,
 					   new_oid, old_oid,
 					   0, "push",
-					   &err)) {
+					   NULL, &err)) {
 			rp_error("%s", err.buf);
 			ret = "failed to update ref";
 		} else {
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..b3281758d0 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   0, NULL, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 19a7e06bf4..8de32535de 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -624,7 +624,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
-				   reflog_msg.buf, &err) ||
+				   reflog_msg.buf, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		if (path)
 			fprintf(stderr,
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 61338a01ec..3807cf4106 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -205,7 +205,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
 				   update_flags | create_reflog_flag,
-				   msg, &err))
+				   msg, NULL, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
diff --git a/refs.c b/refs.c
index 55d2e0b2cb..69b89a1aa2 100644
--- a/refs.c
+++ b/refs.c
@@ -1228,7 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
-		const char *msg)
+		const char *msg, const char *symref)
 {
 	struct ref_update *update;
 
@@ -1254,7 +1254,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
 			   unsigned int flags, const char *msg,
-			   struct strbuf *err)
+			   const char *symref, struct strbuf *err)
 {
 	assert(err);
 
@@ -1280,7 +1280,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, msg, symref);
 	return 0;
 }
 
@@ -1295,7 +1295,7 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), flags, msg, NULL, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1308,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      flags, msg, NULL, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1320,8 +1320,8 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 	if (!old_oid)
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
-				      NULL, old_oid,
-				      flags, NULL, err);
+				      NULL, old_oid, flags,
+				      NULL, NULL, err);
 }
 
 int refs_update_ref(struct ref_store *refs, const char *msg,
@@ -1336,7 +1336,7 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
 	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+				   NULL, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index 298caf6c61..073653d1a4 100644
--- a/refs.h
+++ b/refs.h
@@ -694,13 +694,18 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  */
 #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
 
+/*
+ * Used to denote a symbolic reference update.
+ */
+#define REF_UPDATE_SYMREF (1 << 12)
+
 /*
  * Bitmask of all of the flags that are allowed to be passed in to
  * ref_transaction_update() and friends:
  */
 #define REF_TRANSACTION_UPDATE_ALLOWED_FLAGS                                  \
 	(REF_NO_DEREF | REF_FORCE_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION | \
-	 REF_SKIP_REFNAME_VERIFICATION)
+	 REF_SKIP_REFNAME_VERIFICATION | REF_UPDATE_SYMREF)
 
 /*
  * Add a reference update to transaction. `new_oid` is the value that
@@ -721,7 +726,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
 			   unsigned int flags, const char *msg,
-			   struct strbuf *err);
+			   const char *symref, struct strbuf *err);
 
 /*
  * Add a reference creation to transaction. new_oid is the value that
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3f0f9521cb..4dbe73c106 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,8 +1292,8 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
-					   REF_NO_DEREF, NULL, &err))
+					   iter->oid, NULL, REF_NO_DEREF,
+					   NULL, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
 
@@ -2323,7 +2323,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			update->msg, NULL);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2386,7 +2386,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			update->msg, NULL);
 
 	new_update->parent_update = update;
 
@@ -2777,7 +2777,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL);
 		}
 	}
 
@@ -3062,7 +3062,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..3fccf784d4 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -134,6 +134,12 @@ struct ref_update {
 	unsigned int type;
 	char *msg;
 
+	/*
+	 * If (flags & REF_UPDATE_SYMREF), we update the reference to be a
+	 * symbolic reference and the value is taken from this field.
+	 */
+	char *symref_target;
+
 	/*
 	 * If this ref_update was split off of a symref update via
 	 * split_symref_update(), then this member points at that
@@ -164,8 +170,8 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 /*
  * Add a ref_update with the specified properties to transaction, and
  * return a pointer to the new object. This function does not verify
- * that refname is well-formed. new_oid and old_oid are only
- * dereferenced if the REF_HAVE_NEW and REF_HAVE_OLD bits,
+ * that refname is well-formed. new_oid, old_oid, symref are only
+ * dereferenced if the REF_HAVE_NEW, REF_HAVE_OLD and REF_UPDATE_SYMREF bits,
  * respectively, are set in flags.
  */
 struct ref_update *ref_transaction_add_update(
@@ -173,7 +179,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
-		const char *msg);
+		const char *msg, const char *symref);
 
 /*
  * Transaction states.
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 9b53d42541..92f2803e90 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -884,7 +884,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, u->msg, NULL);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -963,7 +963,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, u->msg, NULL);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 4e14fa6541..668ea683ee 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -614,7 +614,7 @@ static int fast_forward_to(struct repository *r,
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
 				   null_oid() : from,
-				   0, sb.buf, &err) ||
+				   0, sb.buf, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1232,7 +1232,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   0, sb.buf, NULL, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3750,7 +3750,7 @@ static int do_label(struct repository *r, const char *name, int len)
 		error(_("could not read HEAD"));
 		ret = -1;
 	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+					  NULL, 0, msg.buf, NULL, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index 65002a7220..33eef703ce 100644
--- a/walker.c
+++ b/walker.c
@@ -326,7 +326,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		if (ref_transaction_update(transaction, refname.buf,
 					   oids + i, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
-					   &err)) {
+					   NULL, &err)) {
 			error("%s", err.buf);
 			goto done;
 		}
-- 
2.43.GIT

[PATCH 5/8] refs/files-backend: add support for symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Add support for transactional symbolic reference updates in the files
backend. This also adheres to the config of using symlinks for symbolic
references.

While this commit is setting up the files-backend to support symrefs in
transaction's. It will only be used in a consequent commit, when we wire
up the `update-symref` option for `git-update-ref`.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 45 +++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 42 insertions(+), 3 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4dbe73c106..6b4cc80843 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2323,7 +2323,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg, NULL);
+			update->msg, update->symref_target);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2386,7 +2386,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg, NULL);
+			update->msg, update->symref_target);
 
 	new_update->parent_update = update;
 
@@ -2396,7 +2396,7 @@ static int split_symref_update(struct ref_update *update,
 	 * done when new_update is processed.
 	 */
 	update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
-	update->flags &= ~REF_HAVE_OLD;
+	update->flags &= ~(REF_HAVE_OLD|REF_UPDATE_SYMREF);
 
 	/*
 	 * Add the referent. This insertion is O(N) in the transaction
@@ -2567,6 +2567,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
+	if (update->flags & REF_UPDATE_SYMREF) {
+		if (create_symref_lock(refs, lock, update->refname, update->symref_target)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		if (update->flags & REF_UPDATE_SYMREF)
+			update->flags |= REF_NEEDS_COMMIT;
+	}
+
 	if ((update->flags & REF_HAVE_NEW) &&
 	    !(update->flags & REF_DELETING) &&
 	    !(update->flags & REF_LOG_ONLY)) {
@@ -2862,6 +2883,14 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
+			if (update->flags & REF_UPDATE_SYMREF) {
+				if (!refs_resolve_ref_unsafe(&refs->base, update->symref_target,
+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
+					strbuf_addf(err, "refname %s not found", update->symref_target);
+					goto cleanup;
+				}
+			}
+
 			if (files_log_ref_write(refs,
 						lock->ref_name,
 						&lock->old_oid,
@@ -2879,6 +2908,16 @@ static int files_transaction_finish(struct ref_store *ref_store,
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next updated. If not, we try and create a regular symref.
+		 */
+		if (update->flags & REF_UPDATE_SYMREF && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->symref_target))
+				continue;
+
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
-- 
2.43.GIT

Re: [PATCH 5/8] refs/files-backend: add support for symref updates

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 4794 bytes --]

On Sat, Mar 30, 2024 at 11:46:20PM +0100, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> Add support for transactional symbolic reference updates in the files
> backend. This also adheres to the config of using symlinks for symbolic
> references.
> 
> While this commit is setting up the files-backend to support symrefs in
> transaction's. It will only be used in a consequent commit, when we wire
> up the `update-symref` option for `git-update-ref`.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  refs/files-backend.c | 45 +++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 42 insertions(+), 3 deletions(-)
> 
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 4dbe73c106..6b4cc80843 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2323,7 +2323,7 @@ static int split_head_update(struct ref_update *update,
>  			transaction, "HEAD",
>  			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
>  			&update->new_oid, &update->old_oid,
> -			update->msg, NULL);
> +			update->msg, update->symref_target);
>  
>  	/*
>  	 * Add "HEAD". This insertion is O(N) in the transaction
> @@ -2386,7 +2386,7 @@ static int split_symref_update(struct ref_update *update,
>  	new_update = ref_transaction_add_update(
>  			transaction, referent, new_flags,
>  			&update->new_oid, &update->old_oid,
> -			update->msg, NULL);
> +			update->msg, update->symref_target);
>  
>  	new_update->parent_update = update;
>  
> @@ -2396,7 +2396,7 @@ static int split_symref_update(struct ref_update *update,
>  	 * done when new_update is processed.
>  	 */
>  	update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
> -	update->flags &= ~REF_HAVE_OLD;
> +	update->flags &= ~(REF_HAVE_OLD|REF_UPDATE_SYMREF);
>  
>  	/*
>  	 * Add the referent. This insertion is O(N) in the transaction
> @@ -2567,6 +2567,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>  		}
>  	}
>  
> +	if (update->flags & REF_UPDATE_SYMREF) {
> +		if (create_symref_lock(refs, lock, update->refname, update->symref_target)) {
> +			ret = TRANSACTION_GENERIC_ERROR;
> +			goto out;
> +		}
> +
> +		if (close_ref_gently(lock)) {
> +			strbuf_addf(err, "couldn't close '%s.lock'",
> +				    update->refname);
> +			ret = TRANSACTION_GENERIC_ERROR;
> +			goto out;
> +		}
> +
> +		/*
> +		 * Once we have created the symref lock, the commit
> +		 * phase of the transaction only needs to commit the lock.
> +		 */
> +		if (update->flags & REF_UPDATE_SYMREF)
> +			update->flags |= REF_NEEDS_COMMIT;

As far as I can see the `update->flags` aren't ever modified in this
block, which already is guarded via `update->flags = REF_UPDATE_SYMREF`.
This condition should thus be superfluous, and we can instead set
`REF_NEEDS_COMMIT` unconditionally here.

> +	}
> +
>  	if ((update->flags & REF_HAVE_NEW) &&
>  	    !(update->flags & REF_DELETING) &&
>  	    !(update->flags & REF_LOG_ONLY)) {
> @@ -2862,6 +2883,14 @@ static int files_transaction_finish(struct ref_store *ref_store,
>  
>  		if (update->flags & REF_NEEDS_COMMIT ||
>  		    update->flags & REF_LOG_ONLY) {
> +			if (update->flags & REF_UPDATE_SYMREF) {
> +				if (!refs_resolve_ref_unsafe(&refs->base, update->symref_target,
> +							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
> +					strbuf_addf(err, "refname %s not found", update->symref_target);
> +					goto cleanup;
> +				}
> +			}

So we try to resolve the symref target here so that we can provide a
proper new object ID for the reflog entry. What happens though when the
caller tries to create a dangling symref where the target ref does not
exist? Wouldn't we raise an error and abort in that case?

I think we should handle this case gracefully and set the new object ID
to the zero OID. Which is also what happens when deleting the target of
e.g. the `HEAD` symref.

>  			if (files_log_ref_write(refs,
>  						lock->ref_name,
>  						&lock->old_oid,
> @@ -2879,6 +2908,16 @@ static int files_transaction_finish(struct ref_store *ref_store,
>  				goto cleanup;
>  			}
>  		}
> +
> +		/*
> +		 * We try creating a symlink, if that succeeds we continue to the
> +		 * next updated. If not, we try and create a regular symref.
> +		 */
> +		if (update->flags & REF_UPDATE_SYMREF && prefer_symlink_refs)
> +			if (!create_ref_symlink(lock, update->symref_target))
> +				continue;
> +
> +

There's a superfluous newline here.

>  		if (update->flags & REF_NEEDS_COMMIT) {
>  			clear_loose_ref_cache(refs);
>  			if (commit_ref(lock)) {

What is the purpose of `clear_loose_ref_cache()` here, and do we need to
call it when updating symrefs, too?

Patrick

> -- 
> 2.43.GIT
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 6/8] refs/reftable-backend: add support for symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Add support for transactional symbolic reference updates in the reftable
backend.

While this commit is setting up the reftable-backend to support symrefs
in transaction's. It will only be used in a consequent commit, when we
wire up the `update-symref` option for `git-update-ref`.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/reftable-backend.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 92f2803e90..35f2e8e050 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -884,7 +884,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg, NULL);
+					&u->new_oid, &u->old_oid, u->msg, u->symref_target);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -909,9 +909,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 
 			/*
 			 * There is no need to write the reference deletion
-			 * when the reference in question doesn't exist.
+			 * when the reference in question doesn't exist except
+			 * when we want to create new symrefs.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			if ((u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) ||
+			    u->flags & REF_UPDATE_SYMREF) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -963,7 +965,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg, NULL);
+						&u->new_oid, &u->old_oid, u->msg, u->symref_target);
 				new_update->parent_update = u;
 
 				/*
@@ -1026,6 +1028,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 */
 		if ((u->type & REF_ISSYMREF) ||
 		    (u->flags & REF_LOG_ONLY) ||
+		    (u->flags & REF_UPDATE_SYMREF) ||
 		    (u->flags & REF_HAVE_NEW && !oideq(&current_oid, &u->new_oid))) {
 			ret = queue_transaction_update(refs, tx_data, u,
 						       &current_oid, err);
@@ -1187,6 +1190,18 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			ret = reftable_writer_add_ref(writer, &ref);
 			if (ret < 0)
 				goto done;
+		} else if (u->flags & REF_UPDATE_SYMREF) {
+			struct write_create_symref_arg create  = {
+				.refs = arg->refs,
+				.stack = arg->stack,
+				.refname = u->refname,
+				.target = u->symref_target,
+				.logmsg = u->msg,
+			};
+
+			write_symref_with_log(writer, &create, ts);
+			if (ret < 0)
+				goto done;
 		}
 	}
 
-- 
2.43.GIT

Re: [PATCH 6/8] refs/reftable-backend: add support for symref updates

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3312 bytes --]

On Sat, Mar 30, 2024 at 11:46:21PM +0100, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> Add support for transactional symbolic reference updates in the reftable
> backend.
> 
> While this commit is setting up the reftable-backend to support symrefs
> in transaction's. It will only be used in a consequent commit, when we
> wire up the `update-symref` option for `git-update-ref`.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  refs/reftable-backend.c | 23 +++++++++++++++++++----
>  1 file changed, 19 insertions(+), 4 deletions(-)
> 
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index 92f2803e90..35f2e8e050 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -884,7 +884,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>  			new_update = ref_transaction_add_update(
>  					transaction, "HEAD",
>  					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
> -					&u->new_oid, &u->old_oid, u->msg, NULL);
> +					&u->new_oid, &u->old_oid, u->msg, u->symref_target);
>  			string_list_insert(&affected_refnames, new_update->refname);
>  		}
>  
> @@ -909,9 +909,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>  
>  			/*
>  			 * There is no need to write the reference deletion
> -			 * when the reference in question doesn't exist.
> +			 * when the reference in question doesn't exist except
> +			 * when we want to create new symrefs.

Nit: it's not really an exception because it's not a reference deletion
in the first place when we write a symref.

Patrick

>  			 */
> -			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
> +			if ((u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) ||
> +			    u->flags & REF_UPDATE_SYMREF) {
>  				 ret = queue_transaction_update(refs, tx_data, u,
>  								&current_oid, err);
>  				 if (ret)
> @@ -963,7 +965,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>  				 */
>  				new_update = ref_transaction_add_update(
>  						transaction, referent.buf, new_flags,
> -						&u->new_oid, &u->old_oid, u->msg, NULL);
> +						&u->new_oid, &u->old_oid, u->msg, u->symref_target);
>  				new_update->parent_update = u;
>  
>  				/*
> @@ -1026,6 +1028,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>  		 */
>  		if ((u->type & REF_ISSYMREF) ||
>  		    (u->flags & REF_LOG_ONLY) ||
> +		    (u->flags & REF_UPDATE_SYMREF) ||
>  		    (u->flags & REF_HAVE_NEW && !oideq(&current_oid, &u->new_oid))) {
>  			ret = queue_transaction_update(refs, tx_data, u,
>  						       &current_oid, err);
> @@ -1187,6 +1190,18 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
>  			ret = reftable_writer_add_ref(writer, &ref);
>  			if (ret < 0)
>  				goto done;
> +		} else if (u->flags & REF_UPDATE_SYMREF) {
> +			struct write_create_symref_arg create  = {
> +				.refs = arg->refs,
> +				.stack = arg->stack,
> +				.refname = u->refname,
> +				.target = u->symref_target,
> +				.logmsg = u->msg,
> +			};
> +
> +			write_symref_with_log(writer, &create, ts);
> +			if (ret < 0)
> +				goto done;
>  		}
>  	}
>  
> -- 
> 2.43.GIT
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'git-update-ref(1)' command allows transactional reference updates.
But currently only supports regular reference updates. Meaning, if one
wants to update HEAD (symbolic ref) in a transaction, there is no tool
to do so.

One option to obtain transactional updates for the HEAD ref is to
manually create the HEAD.lock file and commit. This is intrusive, where
the user needs to mimic internal git behavior. Also, this only works
when using the files backend.

To allow users to update the HEAD ref in a transaction, we introduce
'update-symref' command for 'git-update-ref(1)'. This command allows the
user to create symref in a transaction similar to the 'update' command
of 'git-update-ref(1)'. This command also works well with the existing
'no-deref' option.

The option can also be used to create new symrefs too. This means we
don't need a dedicated `create-symref` option. This is also because we
don't verify the old symref value when updating a symref. So in this
case update and create hold the same meaning.

The regular `delete` option can also be used to delete symrefs. So we
don't add a dedicated `delete-symref` option.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |  11 ++-
 builtin/update-ref.c             |  61 ++++++++++++---
 refs.c                           |  10 +++
 t/t0600-reffiles-backend.sh      |  30 ++++++++
 t/t1400-update-ref.sh            | 127 +++++++++++++++++++++++++++++++
 5 files changed, 229 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 0561808cca..2ea8bc8167 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <newvalue> LF
 	delete SP <ref> [SP <oldvalue>] LF
 	verify SP <ref> [SP <oldvalue>] LF
+	update-symref SP <ref> SP <newvalue> LF
 	option SP <opt> LF
 	start LF
 	prepare LF
@@ -86,6 +87,7 @@ quoting:
 	create SP <ref> NUL <newvalue> NUL
 	delete SP <ref> NUL [<oldvalue>] NUL
 	verify SP <ref> NUL [<oldvalue>] NUL
+	update-symref NUL <ref> NUL <newvalue> NUL
 	option SP <opt> NUL
 	start NUL
 	prepare NUL
@@ -111,12 +113,19 @@ create::
 
 delete::
 	Delete <ref> after verifying it exists with <oldvalue>, if
-	given.  If given, <oldvalue> may not be zero.
+	given.  If given, <oldvalue> may not be zero. Can also delete
+	symrefs.
 
 verify::
 	Verify <ref> against <oldvalue> but do not change it.  If
 	<oldvalue> is zero or missing, the ref must not exist.
 
+update-symref::
+	Update <ref> as a symbolic reference to point to the given
+	reference <newvalue>. By default, <ref> will be recursively
+	de-referenced, unless the `no-deref` option is used. Can also
+	be used to create new symrefs.
+
 option::
 	Modify the behavior of the next command naming a <ref>.
 	The only valid option is `no-deref` to avoid dereferencing
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 3807cf4106..357daf31b8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -213,6 +213,48 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+static void parse_cmd_update_symref(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *symref;
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("update-symref: missing <ref>");
+
+	if (line_termination) {
+		/* Without -z, consume SP and use next argument */
+		if (!*next || *next == line_termination)
+			die("update-symref %s: missing <newvalue>", refname);
+		if (*next != ' ')
+			die("update-symref %s: expected SP but got: %s",
+			    refname, next);
+	} else {
+		/* With -z, read the next NUL-terminated line */
+		if (*next)
+			die("update-symref %s: missing <newvalue>", refname);
+	}
+	next++;
+
+	symref = parse_refname(&next);
+	if (!symref)
+		die("update-symref %s: missing <newvalue>", refname);
+
+	if (*next != line_termination)
+		die("update-symref %s: extra input: %s", refname, next);
+
+	if (ref_transaction_update(transaction, refname, NULL, NULL,
+				   update_flags | create_reflog_flag | REF_UPDATE_SYMREF,
+				   msg, symref, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(symref);
+	free(refname);
+	strbuf_release(&err);
+}
+
 static void parse_cmd_create(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -379,15 +421,16 @@ static const struct parse_cmd {
 	unsigned args;
 	enum update_refs_state state;
 } command[] = {
-	{ "update",  parse_cmd_update,  3, UPDATE_REFS_OPEN },
-	{ "create",  parse_cmd_create,  2, UPDATE_REFS_OPEN },
-	{ "delete",  parse_cmd_delete,  2, UPDATE_REFS_OPEN },
-	{ "verify",  parse_cmd_verify,  2, UPDATE_REFS_OPEN },
-	{ "option",  parse_cmd_option,  1, UPDATE_REFS_OPEN },
-	{ "start",   parse_cmd_start,   0, UPDATE_REFS_STARTED },
-	{ "prepare", parse_cmd_prepare, 0, UPDATE_REFS_PREPARED },
-	{ "abort",   parse_cmd_abort,   0, UPDATE_REFS_CLOSED },
-	{ "commit",  parse_cmd_commit,  0, UPDATE_REFS_CLOSED },
+	{ "update",        parse_cmd_update,        3, UPDATE_REFS_OPEN },
+	{ "update-symref", parse_cmd_update_symref, 2, UPDATE_REFS_OPEN },
+	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
+	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
+	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
+	{ "start",         parse_cmd_start,         0, UPDATE_REFS_STARTED },
+	{ "prepare",       parse_cmd_prepare,       0, UPDATE_REFS_PREPARED },
+	{ "abort",         parse_cmd_abort,         0, UPDATE_REFS_CLOSED },
+	{ "commit",        parse_cmd_commit,        0, UPDATE_REFS_CLOSED },
 };
 
 static void update_refs_stdin(void)
diff --git a/refs.c b/refs.c
index 69b89a1aa2..706dcd6deb 100644
--- a/refs.c
+++ b/refs.c
@@ -1216,6 +1216,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
 	}
 
 	for (i = 0; i < transaction->nr; i++) {
+		free(transaction->updates[i]->symref_target);
 		free(transaction->updates[i]->msg);
 		free(transaction->updates[i]);
 	}
@@ -1235,6 +1236,9 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if ((flags & (REF_HAVE_NEW | REF_UPDATE_SYMREF)) == (REF_HAVE_NEW | REF_UPDATE_SYMREF))
+		BUG("cannot create regular ref and symref at once");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1245,6 +1249,8 @@ struct ref_update *ref_transaction_add_update(
 		oidcpy(&update->new_oid, new_oid);
 	if (flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);
+	if (flags & REF_UPDATE_SYMREF)
+		update->symref_target = xstrdup(symref);
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -2337,6 +2343,10 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
 
+		// Reference transaction does not support symbolic updates.
+		if (update->flags & REF_UPDATE_SYMREF)
+			continue;
+
 		strbuf_reset(&buf);
 		strbuf_addf(&buf, "%s %s %s\n",
 			    oid_to_hex(&update->old_oid),
diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh
index 64214340e7..6d334cb477 100755
--- a/t/t0600-reffiles-backend.sh
+++ b/t/t0600-reffiles-backend.sh
@@ -472,4 +472,34 @@ test_expect_success POSIXPERM 'git reflog expire honors core.sharedRepository' '
 	esac
 '
 
+test_expect_success SYMLINKS 'symref transaction supports symlinks' '
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs true &&
+	cat >stdin <<-EOF &&
+	start
+	update-symref TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_symlink .git/TESTSYMREFONE &&
+	test "$(test_readlink .git/TESTSYMREFONE)" = refs/heads/new
+'
+
+test_expect_success 'symref transaction supports false symlink config' '
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs false &&
+	cat >stdin <<-EOF &&
+	start
+	update-symref TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_file .git/TESTSYMREFONE &&
+	git symbolic-ref TESTSYMREFONE >actual &&
+	echo refs/heads/new >expect &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 6ebc3ef945..2a6036471b 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -868,6 +868,105 @@ test_expect_success 'stdin delete symref works flag --no-deref' '
 	test_cmp expect actual
 '
 
+test_expect_success 'stdin update-symref creates symref with --no-deref' '
+	# ensure that the symref does not already exist 
+	test_must_fail git symbolic-ref --no-recurse refs/heads/symref &&
+	cat >stdin <<-EOF &&
+	update-symref refs/heads/symref $b
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	echo $b >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$Z $(git rev-parse $b)" actual
+'
+
+test_expect_success 'stdin update-symref updates symref with --no-deref' '
+	# ensure that the symref already exists
+	git symbolic-ref --no-recurse refs/heads/symref &&
+	cat >stdin <<-EOF &&
+	update-symref refs/heads/symref $a
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$(git rev-parse $b) $(git rev-parse $a)" actual
+'
+
+test_expect_success 'stdin update-symref noop update with --no-deref' '
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	echo $a >expect &&
+	test_cmp expect actual &&
+	cat >stdin <<-EOF &&
+	update-symref refs/heads/symref $a
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success 'stdin update-symref regular ref with --no-deref' '
+	git update-ref refs/heads/regularref $a &&
+	cat >stdin <<-EOF &&
+	update-symref refs/heads/regularref $a
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success 'stdin update-symref creates symref' '
+	# delete the ref since it already exists from previous tests
+	git update-ref --no-deref -d refs/heads/symref &&
+	cat >stdin <<-EOF &&
+	update-symref refs/heads/symref $b
+	EOF
+	git update-ref --stdin <stdin &&
+	echo $b >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$Z $(git rev-parse $b)" actual
+'
+
+test_expect_success 'stdin update-symref updates symref' '
+	git update-ref refs/heads/symref2 $b &&
+	git symbolic-ref --no-recurse refs/heads/symref refs/heads/symref2 &&
+	cat >stdin <<-EOF &&
+	update-symref refs/heads/symref $a
+	EOF
+	git update-ref --stdin <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref2 >actual &&
+	test_cmp expect actual &&
+	echo refs/heads/symref2 >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$(git rev-parse $b) $(git rev-parse $b)" actual
+'
+
+test_expect_success 'stdin update-symref regular ref' '
+	git update-ref --no-deref refs/heads/regularref $a &&
+	cat >stdin <<-EOF &&
+	update-symref refs/heads/regularref $a
+	EOF
+	git update-ref --stdin <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
 test_expect_success 'stdin delete ref works with right old value' '
 	echo "delete $b $m~1" >stdin &&
 	git update-ref --stdin <stdin &&
@@ -1641,4 +1740,32 @@ test_expect_success PIPE 'transaction flushes status updates' '
 	test_cmp expected actual
 '
 
+test_expect_success 'transaction can commit symref update' '
+	git symbolic-ref TESTSYMREFONE $a &&
+	cat >stdin <<-EOF &&
+	start
+	update-symref TESTSYMREFONE refs/heads/branch
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	echo refs/heads/branch >expect &&
+	git symbolic-ref TESTSYMREFONE >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'transaction can abort symref update' '
+	git symbolic-ref TESTSYMREFONE $a &&
+	cat >stdin <<-EOF &&
+	start
+	update-symref TESTSYMREFONE refs/heads/branch
+	prepare
+	abort
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	echo $a >expect &&
+	git symbolic-ref TESTSYMREFONE >actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index 0561808cca..2ea8bc8167 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:

In pre-context, there is this entry.

	update SP <ref> SP <newvalue> [SP <oldvalue>] LF

Unrelated to this patch, we probably should say <new-value> and
<old-value> to follow the (recent) documentation guideline to write
multiple words concatenated with hyphen.

If we are updating these newvalue and oldvalue anyway, we may want
to update them to <new-oid> and <old-oid>, as the existing commands
on refs are about object names, and what we are adding here are not.

I would prefer to see such an update of existing documentation as a
separate preliminary clean-up patch, so that we can cleanly add the
"update-symref" that takes <new-symref-target> on top of a
cleaned-up base.

The <newvalue> ...

>  	create SP <ref> SP <newvalue> LF
>  	delete SP <ref> [SP <oldvalue>] LF
>  	verify SP <ref> [SP <oldvalue>] LF
> +	update-symref SP <ref> SP <newvalue> LF

... we are giving to "update-symref" is not an object name, but a
refname (i.e. the target of the symref at <ref> points at), so
"newvalue" -> "new-ref" or something is needed.

The semantics of this new command is not quite in line with the
existing commands.  Existing "update" and "delete" allow you to
optionally specify <old-oid> to make sure you do not overwrite
somebody else's update in the meantime.  Your "update-symref" lacks
such safety completely, which I doubt is a good idea in the context
of adding it to an existing set that has such safety features.  We
should at least offer the same "optional" safety.  That makes the
syntax more like

	update-symref SP <ref> SP <new-ref> [SP <old-ref>] LF

probably.

Existing "update" command can be used to create (by having "zero"
<old-oid>) and to delete (by having "zero" <new-oid>), but we still
have "create" and "delete" separately.  Given that we are teaching
the command to also work with symrefs by adding command(s) for
symrefs to an existing vocabulary, we should offer "create-symref"
and "delete-symref" for consistency between refs and symrefs.

It probably is a good idea to allow "update-symref" to also create
and delete a symref in similar ways with the existing "update"
allows creation and deletion of a ref.  An "zero" <old-ref> may be
an update that can happen only when the <ref> does not exist, i.e.,
creation, and an "zero" <new-ref> may be an update that makes <ref>
disappear.

"zero" value in the context of refs is a 0{40} object name (side
note: we have an explicit mention of 40 in the doc, which needs to
be updated eventually, probably outside this series).  But in the
new context of symrefs, a safe "zero" value needs to be picked
carefully.  An empty string may not work well syntactically
especially in the `-z` format, because you cannot tell if

	update-symref NUL HEAD NUL refs/heads/main NUL NUL

wants to say that <old-ref> is an empty string, or if it is missing.
As a refname cannot have a path component that begins with a dot,
a usable "zero" value for <new-ref> and <old-ref> may be a string
like ".missing", ".detached", and perhaps ".detached-f78d7...f2d4".

To summarize:

	update-symref SP HEAD SP refs/heads/main LF

forcibly sets HEAD to point at refs/heads/main without regard to the
current state of HEAD.

	update-symref SP HEAD SP .missing LF

forcibly removes HEAD symref without regard to the current state of
HEAD.

	update-symref SP HEAD SP refs/heads/main SP .missing LF

creates HEAD symref to point at the 'main' branch but fails if HEAD
already exists.

	update-symref SP HEAD SP refs/heads/main SP .detached LF
	update-symref SP HEAD SP refs/heads/main SP .detached-f78d7...f2d4 LF

creates HEAD symref to point at the 'main' branch but fails if HEAD
is not detached (or detached at the specified commit).

	update-symref SP HEAD SP refs/heads/main SP refs/heads/next LF

creates HEAD symref to point at the 'main' branch but fails if HEAD
is not pointing at the 'next' branch.

Hmm?

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Chris Torek]

On Sun, Mar 31, 2024 at 3:09 PM Junio C Hamano <gitster@pobox.com> wrote:
> ... we are giving to "update-symref" is not an object name, but a
> refname (i.e. the target of the symref at <ref> points at), so
> "newvalue" -> "new-ref" or something is needed.

I kept this line just to say, I agree on all of the above ...

> Existing "update" command can be used to create (by having "zero"
> <old-oid>) and to delete (by having "zero" <new-oid>), but we still
> have "create" and "delete" separately.  Given that we are teaching
> the command to also work with symrefs by adding command(s) for
> symrefs to an existing vocabulary, we should offer "create-symref"
> and "delete-symref" for consistency between refs and symrefs.

.. and on this, but:

> It probably is a good idea to allow "update-symref" to also create
> and delete a symref in similar ways with the existing "update"
> allows creation and deletion of a ref.  An "zero" <old-ref> may be
> an update that can happen only when the <ref> does not exist, i.e.,
> creation, and an "zero" <new-ref> may be an update that makes <ref>
> disappear.
>
> "zero" value in the context of refs is a 0{40} object name (side
> note: we have an explicit mention of 40 in the doc, which needs to
> be updated eventually, probably outside this series).  But in the
> new context of symrefs, a safe "zero" value needs to be picked
> carefully.  An empty string may not work well syntactically
> especially in the `-z` format, because you cannot tell if
>
>         update-symref NUL HEAD NUL refs/heads/main NUL NUL
>
> wants to say that <old-ref> is an empty string, or if it is missing.

For these reasons, I'd suggest that the all-zero hash be officially
deprecated in favor of create/delete and of course create-symref
and delete-symref. Of course, compatibility requires some sort
of support for the old system for some time.  As to whether that
means something like the suggestion of ".missing" etc, I have no
particular opinion -- but since the symref options are new, they
would not *need* symmetric options, if we just say that "update-symref"
cannot create or delete a symref.

Meanwhile, for testing purposes I was curious as to what happens
if you ask `git update-ref` to delete an existing symref, so after
creating a test repository:

$ git branch
* main
  symref -> main
$ git update-ref --stdin
delete refs/heads/symref
$ git branch

Whoops, this doesn't look good...

Restoring the branch name (I had saved the hash ID Just In Case):

$ echo d88ee82e6a5c29c95f712030f5efc9d43116ae79 > .git/refs/heads/main

brings things back, after which this works properly:

$ git branch -d symref
Deleted branch symref (was refs/heads/main).
$ git branch
* main

If this (deleting the target of the symref when using "delete")
is a bug, and I think it is, that's a separate topic of course...

Chris

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Junio C Hamano]

Chris Torek <chris.torek@gmail.com> writes:

> For these reasons, I'd suggest that the all-zero hash be officially
> deprecated in favor of create/delete and of course create-symref
> and delete-symref. Of course, compatibility requires some sort
> of support for the old system for some time.  As to whether that
> means something like the suggestion of ".missing" etc, I have no
> particular opinion -- but since the symref options are new, they
> would not *need* symmetric options, if we just say that "update-symref"
> cannot create or delete a symref.

I love that simplicity.

> If this (deleting the target of the symref when using "delete")
> is a bug, and I think it is, that's a separate topic of course...

"git blame" may find those who can give us answers, but my gut
feeling is that they weren't thinking too deeply about what should
happen, and the code does what it happens to do.

When you update, you'd want the update go through a symref (i.e.,
update-ref HEAD would update your current branch unless you are
detached), so they must have been aware of the fact that they need
to deal with symrefs.

But when you delete a symref, what do you want that deletion affect?
I do not have a good answer offhand, and they probably didn't even
think of the need to deal with symrefs, as the most common symref
you see, HEAD, is not something you'd ever want to delete anyway.

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Chris Torek <chris.torek@gmail.com> writes:
>
>> For these reasons, I'd suggest that the all-zero hash be officially
>> deprecated in favor of create/delete and of course create-symref
>> and delete-symref. Of course, compatibility requires some sort
>> of support for the old system for some time.  As to whether that
>> means something like the suggestion of ".missing" etc, I have no
>> particular opinion -- but since the symref options are new, they
>> would not *need* symmetric options, if we just say that "update-symref"
>> cannot create or delete a symref.
>
> I love that simplicity.

Having said that, the loose "update that can create or delete" may
actually be used by applications that do not care about overwriting
competing operation, so I am not sure if we can easily deprecate
that mode of operation.  Saying "update refs/heads/next to point at
this object" and have it created if it does not exist may be handy
for some loosely written applications.

So perhaps we can say "update with a concrete <old-oid> will ensure
that the <ref> poitns at <old-oid> before proceeding, but update
with 0{40} as <old-oid> to ensure creation is deprecated.  update
with 0{40} as <new-oid> as deletion is also deprecated.  Use create
and delete for these two deprecated operation modes".

This assumes that create and delete currently ensures that what is
asked to be created does not exist, and what is asked to be deleted
does exist, before the operation.  If we are loosely doing these two
operations, then we cannot easily deprecate the checking-update,
without breaking existing users.

As {create,update,delete,verify}-symref do not exist yet, we can
start with the right semantics from day one.  "update-symref" will
accept a <old-ref> only to ensure that the symref is pointing to
that ref and there is no "zero" value based creation/deletion
validation offered via "update-symref".  "create-symref" will error
out if the ref asked to be created already exists, "delete-symref"
will error out if the ref asked to be deleted does not exist.

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2664 bytes --]

On Sun, Mar 31, 2024 at 06:31:14PM -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Chris Torek <chris.torek@gmail.com> writes:
> >
> >> For these reasons, I'd suggest that the all-zero hash be officially
> >> deprecated in favor of create/delete and of course create-symref
> >> and delete-symref. Of course, compatibility requires some sort
> >> of support for the old system for some time.  As to whether that
> >> means something like the suggestion of ".missing" etc, I have no
> >> particular opinion -- but since the symref options are new, they
> >> would not *need* symmetric options, if we just say that "update-symref"
> >> cannot create or delete a symref.
> >
> > I love that simplicity.
> 
> Having said that, the loose "update that can create or delete" may
> actually be used by applications that do not care about overwriting
> competing operation, so I am not sure if we can easily deprecate
> that mode of operation.  Saying "update refs/heads/next to point at
> this object" and have it created if it does not exist may be handy
> for some loosely written applications.

I wouldn't say "loosely written here". I certainly know that we do use
these implicit modes in Gitaly, and we have conciously chosen them
because they have been supported by Git all along. It simply makes our
lifes easier when we don't have to special-case creations and deletions
in any way.

So I'd really not want those to go away or become deprecated.

Patrick

> So perhaps we can say "update with a concrete <old-oid> will ensure
> that the <ref> poitns at <old-oid> before proceeding, but update
> with 0{40} as <old-oid> to ensure creation is deprecated.  update
> with 0{40} as <new-oid> as deletion is also deprecated.  Use create
> and delete for these two deprecated operation modes".
> 
> This assumes that create and delete currently ensures that what is
> asked to be created does not exist, and what is asked to be deleted
> does exist, before the operation.  If we are loosely doing these two
> operations, then we cannot easily deprecate the checking-update,
> without breaking existing users.
> 
> As {create,update,delete,verify}-symref do not exist yet, we can
> start with the right semantics from day one.  "update-symref" will
> accept a <old-ref> only to ensure that the symref is pointing to
> that ref and there is no "zero" value based creation/deletion
> validation offered via "update-symref".  "create-symref" will error
> out if the ref asked to be created already exists, "delete-symref"
> will error out if the ref asked to be deleted does not exist.
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> because they have been supported by Git all along. It simply makes our
> lifes easier when we don't have to special-case creations and deletions
> in any way.
>
> So I'd really not want those to go away or become deprecated.

That is a good input.

Do you have anything to add as a counter-proposal?  The "I do not
care what was there before" update mode does make it necessary to
have a "zero" value for symrefs that can be distinguishable from
not having a value at all.

Thanks.

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1864 bytes --]

On Tue, Apr 02, 2024 at 09:40:41AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > because they have been supported by Git all along. It simply makes our
> > lifes easier when we don't have to special-case creations and deletions
> > in any way.
> >
> > So I'd really not want those to go away or become deprecated.
> 
> That is a good input.
> 
> Do you have anything to add as a counter-proposal?  The "I do not
> care what was there before" update mode does make it necessary to
> have a "zero" value for symrefs that can be distinguishable from
> not having a value at all.
> 
> Thanks.

Sorry for taking this long to answer your question.

I might have missed it while scanning through this thread, but why
exactly is the zero OID not a good enough placeholder here to say that
the ref must not exist? A symref cannot point to a ref named after the
zero OID anyway.

In my opinion, "update-symref" with an old-value must be able to accept
both object IDs and symrefs as old value. Like this it would be possible
to update a proper ref to a symref in a race-free way. So you can say:

    git update-ref SYMREF refs/heads/main 19981daefd7c147444462739375462b49412ce33

To update "SYRMEF" to "refs/heads/main", but only in case it currently
is a proper ref that points to 19981daefd7c147444462739375462b49412ce33.
Similarly...

    git update-ref SYMREF refs/heads/main refs/heads/master

would update "SYMREF" to "refs/heads/main", but only if it currently
points to the symref "refs/heads/master". And by extension I think that
the zero OID should retain its established meaning of "This ref must not
exist":

    git update-ref SYMREF refs/heads/main 0000000000000000000000000000000000000000

This would only update "SYMREF" to "refs/heads/main" if it does not yet
exist.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2569 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Tue, Apr 02, 2024 at 09:40:41AM -0700, Junio C Hamano wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>>
>> > because they have been supported by Git all along. It simply makes our
>> > lifes easier when we don't have to special-case creations and deletions
>> > in any way.
>> >
>> > So I'd really not want those to go away or become deprecated.
>>
>> That is a good input.
>>
>> Do you have anything to add as a counter-proposal?  The "I do not
>> care what was there before" update mode does make it necessary to
>> have a "zero" value for symrefs that can be distinguishable from
>> not having a value at all.
>>
>> Thanks.
>
> Sorry for taking this long to answer your question.
>
> I might have missed it while scanning through this thread, but why
> exactly is the zero OID not a good enough placeholder here to say that
> the ref must not exist? A symref cannot point to a ref named after the
> zero OID anyway.
>
> In my opinion, "update-symref" with an old-value must be able to accept
> both object IDs and symrefs as old value. Like this it would be possible
> to update a proper ref to a symref in a race-free way. So you can say:
>
>     git update-ref SYMREF refs/heads/main 19981daefd7c147444462739375462b49412ce33
>
> To update "SYRMEF" to "refs/heads/main", but only in case it currently
> is a proper ref that points to 19981daefd7c147444462739375462b49412ce33.
> Similarly...
>
>     git update-ref SYMREF refs/heads/main refs/heads/master
>
> would update "SYMREF" to "refs/heads/main", but only if it currently
> points to the symref "refs/heads/master". And by extension I think that
> the zero OID should retain its established meaning of "This ref must not
> exist":
>
>     git update-ref SYMREF refs/heads/main 0000000000000000000000000000000000000000
>
> This would only update "SYMREF" to "refs/heads/main" if it does not yet
> exist.
>

This is definitely nicer experience for the user. From looking at the
other commands, {verify, create, delete} I can only see this applying to
`symref-update`. Making the syntax for update something like:

    symref-update SP <ref> SP <new-ref> [SP (<old-oid> | <old-rev>)] LF

I think this makes sense to me, will incorporate this and send the next
version in the next few days.

On a side-note: This would also mean that we should somehow support
moving from symrefs to a regular ref via a transaction, so that means we
should allow

    update SP <ref> SP <new-oid> [SP (<old-oid> | <old-rev>)] LF

too, but I'm not going to tackle that in my patches.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3073 bytes --]

On Tue, Apr 09, 2024 at 09:15:59AM -0700, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > On Tue, Apr 02, 2024 at 09:40:41AM -0700, Junio C Hamano wrote:
> >> Patrick Steinhardt <ps@pks.im> writes:
> >>
> >> > because they have been supported by Git all along. It simply makes our
> >> > lifes easier when we don't have to special-case creations and deletions
> >> > in any way.
> >> >
> >> > So I'd really not want those to go away or become deprecated.
> >>
> >> That is a good input.
> >>
> >> Do you have anything to add as a counter-proposal?  The "I do not
> >> care what was there before" update mode does make it necessary to
> >> have a "zero" value for symrefs that can be distinguishable from
> >> not having a value at all.
> >>
> >> Thanks.
> >
> > Sorry for taking this long to answer your question.
> >
> > I might have missed it while scanning through this thread, but why
> > exactly is the zero OID not a good enough placeholder here to say that
> > the ref must not exist? A symref cannot point to a ref named after the
> > zero OID anyway.
> >
> > In my opinion, "update-symref" with an old-value must be able to accept
> > both object IDs and symrefs as old value. Like this it would be possible
> > to update a proper ref to a symref in a race-free way. So you can say:
> >
> >     git update-ref SYMREF refs/heads/main 19981daefd7c147444462739375462b49412ce33
> >
> > To update "SYRMEF" to "refs/heads/main", but only in case it currently
> > is a proper ref that points to 19981daefd7c147444462739375462b49412ce33.
> > Similarly...
> >
> >     git update-ref SYMREF refs/heads/main refs/heads/master
> >
> > would update "SYMREF" to "refs/heads/main", but only if it currently
> > points to the symref "refs/heads/master". And by extension I think that
> > the zero OID should retain its established meaning of "This ref must not
> > exist":
> >
> >     git update-ref SYMREF refs/heads/main 0000000000000000000000000000000000000000
> >
> > This would only update "SYMREF" to "refs/heads/main" if it does not yet
> > exist.
> >
> 
> This is definitely nicer experience for the user. From looking at the
> other commands, {verify, create, delete} I can only see this applying to
> `symref-update`. Making the syntax for update something like:
> 
>     symref-update SP <ref> SP <new-ref> [SP (<old-oid> | <old-rev>)] LF
> 
> I think this makes sense to me, will incorporate this and send the next
> version in the next few days.
> 
> On a side-note: This would also mean that we should somehow support
> moving from symrefs to a regular ref via a transaction, so that means we
> should allow
> 
>     update SP <ref> SP <new-oid> [SP (<old-oid> | <old-rev>)] LF
> 
> too, but I'm not going to tackle that in my patches.

Yes, I think that would be a sensible idea, even though we have to be
careful with backwards compatibility here. In any case, I think it makes
sense to not extend the scope of your patch series and leave this for
the future.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

>> > I might have missed it while scanning through this thread, but why
>> > exactly is the zero OID not a good enough placeholder here to say that
>> > the ref must not exist? A symref cannot point to a ref named after the
>> > zero OID anyway.
>
>> > In my opinion, "update-symref" with an old-value must be able to accept
>> > both object IDs and symrefs as old value. Like this it would be possible
>> > to update a proper ref to a symref in a race-free way. So you can say:
>> >
>> >     git update-ref SYMREF refs/heads/main 19981daefd7c147444462739375462b49412ce33
>
>> > To update "SYRMEF" to "refs/heads/main", but only in case it currently
>> > is a proper ref that points to 19981daefd7c147444462739375462b49412ce33.
>> > Similarly...
>> >
>> >     git update-ref SYMREF refs/heads/main refs/heads/master
>
>> > would update "SYMREF" to "refs/heads/main", but only if it currently
>> > points to the symref "refs/heads/master".

I think that would work well.  We need to explicitly forbid a file
$GIT_DIR/[0-9a-f]{40} to be used as a pseudoref, which I think that
is an improvement.  I do not know how the transition to move to a
world with a stricter rule would look like, though.

>> > And by extension I think that
>> > the zero OID should retain its established meaning of "This ref must not
>> > exist":
>> >
>> >     git update-ref SYMREF refs/heads/main 0000000000000000000000000000000000000000
>> >
>> > This would only update "SYMREF" to "refs/heads/main" if it does not yet
>> > exist.

We express "SYMREF must currently be a symref" by naming an old-ref,
and we say "SYMREF can currently be either a ref or symref, but it
must point at this object" by naming an old-oid.  The looseness of
the latter (i.e. we cannot express "Before making HEAD point at ref
X, ensure that it is detached at OID") looks a bit disturbing, but
otherwise looks good.  I offhand do not know how expressive we would
want the "old" requirement to be.

>> On a side-note: This would also mean that we should somehow support
>> moving from symrefs to a regular ref via a transaction, so that means we
>> should allow
>> 
>>     update SP <ref> SP <new-oid> [SP (<old-oid> | <old-rev>)] LF
>> 
>> too, but I'm not going to tackle that in my patches.
>
> Yes, I think that would be a sensible idea, even though we have to be
> careful with backwards compatibility here. In any case, I think it makes
> sense to not extend the scope of your patch series and leave this for
> the future.

Likewise, we may want to be able to express "Before making HEAD
detached at commit X, ensure that HEAD points at ref Y that points
at commit Z".  IOW, the "old" part might have to be not

	[SP (<old-oid> | <old-ref>)]

but

	[SP <old-oid> SP <old-ref>]

to specify both, perhaps?

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3533 bytes --]

On Wed, Apr 10, 2024 at 09:06:45AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> >> > I might have missed it while scanning through this thread, but why
> >> > exactly is the zero OID not a good enough placeholder here to say that
> >> > the ref must not exist? A symref cannot point to a ref named after the
> >> > zero OID anyway.
> >
> >> > In my opinion, "update-symref" with an old-value must be able to accept
> >> > both object IDs and symrefs as old value. Like this it would be possible
> >> > to update a proper ref to a symref in a race-free way. So you can say:
> >> >
> >> >     git update-ref SYMREF refs/heads/main 19981daefd7c147444462739375462b49412ce33
> >
> >> > To update "SYRMEF" to "refs/heads/main", but only in case it currently
> >> > is a proper ref that points to 19981daefd7c147444462739375462b49412ce33.
> >> > Similarly...
> >> >
> >> >     git update-ref SYMREF refs/heads/main refs/heads/master
> >
> >> > would update "SYMREF" to "refs/heads/main", but only if it currently
> >> > points to the symref "refs/heads/master".
> 
> I think that would work well.  We need to explicitly forbid a file
> $GIT_DIR/[0-9a-f]{40} to be used as a pseudoref, which I think that
> is an improvement.  I do not know how the transition to move to a
> world with a stricter rule would look like, though.

I thought that Git already refuses such refnames anyway. Otherwise it
would be impossible to distinguish a ref called [0-9a-f]{40} from the
actual object hash in much of our tooling. I certainly know that GitLab
does refuse such refnames, and thought that GitHub does, too.

But turns out that at least git-update-ref(1) is happy to write such
refs:

```
$ git update-ref 1111111111111111111111111111111111111111 HEAD
$ cat .git/1111111111111111111111111111111111111111
cf6ba211cd2fce88f5d22d9f036029d502565509
```

What does it resolve to?

```
$ git rev-parse --verify 1111111111111111111111111111111111111111
warning: refname '1111111111111111111111111111111111111111' is ambiguous.
Git normally never creates a ref that ends with 40 hex characters
because it will be ignored when you just specify 40-hex. These refs
may be created by mistake. For example,

  git switch -c $br $(git rev-parse ...)

where "$br" is somehow empty and a 40-hex ref is created. Please
examine these refs and maybe delete them. Turn this message off by
running "git config advice.objectNameWarning false"
1111111111111111111111111111111111111111
```

It resolves to 1111111111111111111111111111111111111111 because it's a
valid object ID already. And what if we try to peel it?

```
$ git rev-parse --verify 1111111111111111111111111111111111111111^{commit}
warning: refname '1111111111111111111111111111111111111111' is ambiguous.
Git normally never creates a ref that ends with 40 hex characters
because it will be ignored when you just specify 40-hex. These refs
may be created by mistake. For example,

  git switch -c $br $(git rev-parse ...)

where "$br" is somehow empty and a 40-hex ref is created. Please
examine these refs and maybe delete them. Turn this message off by
running "git config advice.objectNameWarning false"
fatal: Needed a single revision
```

Doesn't work.

So these refs essentially cannot be accessed anyway. Restricting
git-update-ref(1) such that it cannot write them in the first place thus
shouldn't be breaking much, I'd claim, and feels like a strict
improvement overall.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1096 bytes --]

Chris Torek <chris.torek@gmail.com> writes:

> Meanwhile, for testing purposes I was curious as to what happens
> if you ask `git update-ref` to delete an existing symref, so after
> creating a test repository:
>
> $ git branch
> * main
>   symref -> main
> $ git update-ref --stdin
> delete refs/heads/symref
> $ git branch
>
> Whoops, this doesn't look good...
>

This is expected though. Remember that `git-update-ref(1)` by default
does dereferencing of symlinks. So in your case, 'refs/heads/symref' is
dereferenced to 'refs/heads/main' and when a delete command is issued,
'main' is deleted.

Try the same with the `git update-ref --no-deref --stdin` instead.

> Restoring the branch name (I had saved the hash ID Just In Case):
>
> $ echo d88ee82e6a5c29c95f712030f5efc9d43116ae79 > .git/refs/heads/main
>
> brings things back, after which this works properly:
>
> $ git branch -d symref
> Deleted branch symref (was refs/heads/main).
> $ git branch
> * main
>

Here you switch to using 'git-branch(1)' and that doesn't dereference by
default. So there is a difference in the two attempts.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 4760 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
>> index 0561808cca..2ea8bc8167 100644
>> --- a/Documentation/git-update-ref.txt
>> +++ b/Documentation/git-update-ref.txt
>> @@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
>
> In pre-context, there is this entry.
>
> 	update SP <ref> SP <newvalue> [SP <oldvalue>] LF
>
> Unrelated to this patch, we probably should say <new-value> and
> <old-value> to follow the (recent) documentation guideline to write
> multiple words concatenated with hyphen.
>
> If we are updating these newvalue and oldvalue anyway, we may want
> to update them to <new-oid> and <old-oid>, as the existing commands
> on refs are about object names, and what we are adding here are not.
>
> I would prefer to see such an update of existing documentation as a
> separate preliminary clean-up patch, so that we can cleanly add the
> "update-symref" that takes <new-symref-target> on top of a
> cleaned-up base.
>
> The <newvalue> ...
>

I agree with your synopsis here, I'll send in a patch for this,
independent of these changes as a precursor, while we discuss the final
design of this series.

>> Chris Torek <chris.torek@gmail.com> writes:
>>
>>> For these reasons, I'd suggest that the all-zero hash be officially
>>> deprecated in favor of create/delete and of course create-symref
>>> and delete-symref. Of course, compatibility requires some sort
>>> of support for the old system for some time.  As to whether that
>>> means something like the suggestion of ".missing" etc, I have no
>>> particular opinion -- but since the symref options are new, they
>>> would not *need* symmetric options, if we just say that "update-symref"
>>> cannot create or delete a symref.
>>
>> I love that simplicity.
>
> Having said that, the loose "update that can create or delete" may
> actually be used by applications that do not care about overwriting
> competing operation, so I am not sure if we can easily deprecate
> that mode of operation.  Saying "update refs/heads/next to point at
> this object" and have it created if it does not exist may be handy
> for some loosely written applications.
>
> So perhaps we can say "update with a concrete <old-oid> will ensure
> that the <ref> poitns at <old-oid> before proceeding, but update
> with 0{40} as <old-oid> to ensure creation is deprecated.  update
> with 0{40} as <new-oid> as deletion is also deprecated.  Use create
> and delete for these two deprecated operation modes".
>
> This assumes that create and delete currently ensures that what is
> asked to be created does not exist, and what is asked to be deleted
> does exist, before the operation.  If we are loosely doing these two
> operations, then we cannot easily deprecate the checking-update,
> without breaking existing users.
>
> As {create,update,delete,verify}-symref do not exist yet, we can
> start with the right semantics from day one.  "update-symref" will
> accept a <old-ref> only to ensure that the symref is pointing to
> that ref and there is no "zero" value based creation/deletion
> validation offered via "update-symref".  "create-symref" will error
> out if the ref asked to be created already exists, "delete-symref"
> will error out if the ref asked to be deleted does not exist.

This seems very reasonable to me. This ensures that each of the commands
do not spill over to the other and most importantly we don't have to
deal with "zero" values.

But this still means we need to think of the best output for the
reference transaction hook (following commit).

My current implementation of:
   <symref-target> SP <ref-name> LF
Should be changed to:
   <old-ref> SP <new-ref> LF

But this means, for creation of symrefs <old-ref> needs to be "zero"
value. Also there is no way for clients to differentiate between regular
refs and symrefs here. I wonder if it makes sense to do something like:

   symref SP <old-ref> SP <new-ref> LF

Where symref is a fixed string at the start, used as a differentiator.
This however still would leave us to deal with "zero" values for
creation and deletion.

Perhaps the best way here to actually be a lot more verbose and have the
hook output the following:

   symref-create SP <new-ref> LF
   symref-delete SP <old-ref> LF
   symref-update SP <old-ref> SP <new-ref> LF
   symref-update-forced <new-ref> LF

I lean towards the latter, because its also much easier to extend in the
future and we don't have to deal with the "zero" values.

I'm against the "zero" values mostly cause there is no nice way to
describe a zero value for a ref unlike OIDs, which is inherently baked
into the type of data.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

>> So perhaps we can say "update with a concrete <old-oid> will ensure
>> that the <ref> poitns at <old-oid> before proceeding, but update
>> with 0{40} as <old-oid> to ensure creation is deprecated.  update
>> with 0{40} as <new-oid> as deletion is also deprecated.  Use create
>> and delete for these two deprecated operation modes".
>>
>> This assumes that create and delete currently ensures that what is
>> asked to be created does not exist, and what is asked to be deleted
>> does exist, before the operation.  If we are loosely doing these two
>> operations, then we cannot easily deprecate the checking-update,
>> without breaking existing users.

Note that I did not (and do not) know if "create" and "delete" have
such checks; I expected somebody (other than me) to check before
going forward.

> But this still means we need to think of the best output for the
> reference transaction hook (following commit).
>
> My current implementation of:
>    <symref-target> SP <ref-name> LF
> Should be changed to:
>    <old-ref> SP <new-ref> LF
>
> But this means, for creation of symrefs <old-ref> needs to be "zero"
> value. Also there is no way for clients to differentiate between regular
> refs and symrefs here. I wonder if it makes sense to do something like:
>
>    symref SP <old-ref> SP <new-ref> LF

What do clients see for a regular ref?  "<old-oid> SP <new-oid> LF"?
That would be OK, as "symref" cannot be an object name, I guess?

> Where symref is a fixed string at the start, used as a differentiator.
> This however still would leave us to deal with "zero" values for
> creation and deletion.

Are these two <old-ref> and <new-ref> values optional in the context
of your discussion?  The review comment was on input from the end-user
that made it optional to validate the precondition, but this is what
you produce as a result---if these are not optional, then an empty
string can be a reasonable "zero" value.  I am confused...

> Perhaps the best way here to actually be a lot more verbose and have the
> hook output the following:
>
>    symref-create SP <new-ref> LF
>    symref-delete SP <old-ref> LF
>    symref-update SP <old-ref> SP <new-ref> LF
>    symref-update-forced <new-ref> LF

It is unfortunate that the input to the hook for a normal reference
update uses syntax different from the "--stdin" input format, i.e.,

    <old-oid> SP <new-oid> SP <ref> LF

but it is way too late to change it now.  So to be consistent,

    symref-create SP <new-ref> SP <ref> LF
    symref-delete SP <old-ref> SP <ref> LF
    symref-update SP <old-ref> SP <new-ref> SP <ref> LF

would be the three operations.

But this is not an end-user input that tells Git "I do not care
about precondition, I did not even bother to learn the current state
to give you as <old-something>, just force it".  The input to hook
is what we tell the hook what we are planning to do (so that it can
decline), and we do not need the ability to say "I do not know what
the current state is".  So I do not think you need any "zero" value
in the input to the reference-transaction hook.  And I do not see a
need for the "symref-update-forced" variant, either.

By the way, if we were to use symref-{create,delete,update} here,
wouldn't it make more sense to name the command on the "--stdin"
side the same, i.e., not "update-symref" but "symref-update"?

What I suspect that needs more thought is what should happen when
you request via "--stdin" to create, update, or delete a symref,
but <ref> is a regular ref, e.g., "symref-delete <ref>".  For
"symref-create <ref> <new-ref>", we would fail if <ref> exists,
whether it is a symref or a normal ref, so that is OK.  For
"symref-delete <ref> <old-ref>", we would fail if <ref> is *not*
a symref to <old-ref>, so the case where <ref> is a normal ref
is already covered. 

Should we support "symref-update <ref> <new-ref> <old-oid>" that
makes <ref> a symref that points at <new-ref>, but ensures that
<ref> before the operation is a normal ref that points at <old-oid>?

Or should "symref-update" work only on <ref> that is an existing
symref?

I think I am OK if the answer was "You can only give a precondition
in the form of <old-ref>, which means you can only turn an existing
symref to point at a different ref with precondition protection. If
you want to turn a normal ref into a symref, you have to force it by
not having a precondition, or delete the ref and then (re)create it
as a symref".  But we need to decide the semantics and document it.

Thanks.

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

>> But this still means we need to think of the best output for the
>> reference transaction hook (following commit).
>> ...

One thing I missed.  We are not currently reporting symref updates
to these hooks.  Are they prepared to take such extra input?  If not,
are they going to barf when they see "symref-update" while expecting
to see <old-oid>?

We may need to make it possible for Git to tell which variant of the
hook script it was given somehow (the easiest cop-out is to introduce
ref-transaction-hook-v2 as a separate hook, and use it if exists, or
fall back to the reference-transaction hook, and report symref updates
only when we are using v2, but there may be better approaches).

> But this is not an end-user input that tells Git "I do not care
> about precondition, I did not even bother to learn the current state
> to give you as <old-something>, just force it".  The input to hook
> is what we tell the hook what we are planning to do (so that it can
> decline), and we do not need the ability to say "I do not know what
> the current state is".  So I do not think you need any "zero" value
> in the input to the reference-transaction hook.  And I do not see a
> need for the "symref-update-forced" variant, either.

I misspoke here.  We do need "zero" value to indicate that "this
update is a creation event" and "this update is a deletion event".
What I meant to say is that there is no need to make the "zero"
value distinguishable from a "missing optional" value, which was a
problem on the "--stdin" side with "-z" format, where each command
is in a format with fixed number of parameters, unlike the textual
format, where a missing optional argument can be expressed by
omitting SP before the value and the value itself and it can be
differentiated from an empty string as an optional value that is not
missing.

Thanks.

Re: [PATCH 7/8] refs: add 'update-symref' command to 'update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 9130 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>>> So perhaps we can say "update with a concrete <old-oid> will ensure
>>> that the <ref> poitns at <old-oid> before proceeding, but update
>>> with 0{40} as <old-oid> to ensure creation is deprecated.  update
>>> with 0{40} as <new-oid> as deletion is also deprecated.  Use create
>>> and delete for these two deprecated operation modes".
>>>
>>> This assumes that create and delete currently ensures that what is
>>> asked to be created does not exist, and what is asked to be deleted
>>> does exist, before the operation.  If we are loosely doing these two
>>> operations, then we cannot easily deprecate the checking-update,
>>> without breaking existing users.
>
> Note that I did not (and do not) know if "create" and "delete" have
> such checks; I expected somebody (other than me) to check before
> going forward.
>

create does:

$ git update-ref --stdin
create refs/heads/a 53eb33454ce4f3db4d8c79e9c15640c2dffc4a37
fatal: cannot lock ref 'refs/heads/a': reference already exists

delete doesn't:

$ git update-ref --stdin
delete refs/heads/b
% echo $?
0
$ ls .git/refs/heads/
a  master

>> But this still means we need to think of the best output for the
>> reference transaction hook (following commit).
>>
>> My current implementation of:
>>    <symref-target> SP <ref-name> LF
>> Should be changed to:
>>    <old-ref> SP <new-ref> LF
>>
>> But this means, for creation of symrefs <old-ref> needs to be "zero"
>> value. Also there is no way for clients to differentiate between regular
>> refs and symrefs here. I wonder if it makes sense to do something like:
>>
>>    symref SP <old-ref> SP <new-ref> LF
>
> What do clients see for a regular ref?  "<old-oid> SP <new-oid> LF"?
> That would be OK, as "symref" cannot be an object name, I guess?
>

<old-value> SP <new-value> SP <ref-name> LF

Yeah, 'symref' should work.

>> Where symref is a fixed string at the start, used as a differentiator.
>> This however still would leave us to deal with "zero" values for
>> creation and deletion.
>
> Are these two <old-ref> and <new-ref> values optional in the context
> of your discussion?  The review comment was on input from the end-user
> that made it optional to validate the precondition, but this is what
> you produce as a result---if these are not optional, then an empty
> string can be a reasonable "zero" value.  I am confused...
>> Perhaps the best way here to actually be a lot more verbose and have the
>> hook output the following:
>>
>>    symref-create SP <new-ref> LF
>>    symref-delete SP <old-ref> LF
>>    symref-update SP <old-ref> SP <new-ref> LF
>>    symref-update-forced <new-ref> LF
>
> It is unfortunate that the input to the hook for a normal reference
> update uses syntax different from the "--stdin" input format, i.e.,
>
>     <old-oid> SP <new-oid> SP <ref> LF
>
> but it is way too late to change it now.  So to be consistent,
>
>     symref-create SP <new-ref> SP <ref> LF
>     symref-delete SP <old-ref> SP <ref> LF
>     symref-update SP <old-ref> SP <new-ref> SP <ref> LF
>
> would be the three operations.
>
> But this is not an end-user input that tells Git "I do not care
> about precondition, I did not even bother to learn the current state
> to give you as <old-something>, just force it".  The input to hook
> is what we tell the hook what we are planning to do (so that it can
> decline), and we do not need the ability to say "I do not know what
> the current state is".  So I do not think you need any "zero" value
> in the input to the reference-transaction hook.  And I do not see a
> need for the "symref-update-forced" variant, either.

... also from your latest email ...

>> But this is not an end-user input that tells Git "I do not care
>> about precondition, I did not even bother to learn the current state
>> to give you as <old-something>, just force it".  The input to hook
>> is what we tell the hook what we are planning to do (so that it can
>> decline), and we do not need the ability to say "I do not know what
>> the current state is".  So I do not think you need any "zero" value
>> in the input to the reference-transaction hook.  And I do not see a
>> need for the "symref-update-forced" variant, either.
>
> I misspoke here.  We do need "zero" value to indicate that "this
> update is a creation event" and "this update is a deletion event".
> What I meant to say is that there is no need to make the "zero"
> value distinguishable from a "missing optional" value, which was a
> problem on the "--stdin" side with "-z" format, where each command
> is in a format with fixed number of parameters, unlike the textual
> format, where a missing optional argument can be expressed by
> omitting SP before the value and the value itself and it can be
> differentiated from an empty string as an optional value that is not
> missing.
>
> Thanks.

Yup. There is a slight subtlety here though, currently with the
reference-transaction hook:

    When force updating the reference regardless of its current value or
    when the reference is to be created anew, <old-value> is the
    all-zeroes object name. To distinguish these cases, you can inspect
    the current value of <ref-name> via git rev-parse.

I'll keep the same behavior with the symref updates.

> By the way, if we were to use symref-{create,delete,update} here,
> wouldn't it make more sense to name the command on the "--stdin"
> side the same, i.e., not "update-symref" but "symref-update"?

If we were to use symref-*, definitely.

> What I suspect that needs more thought is what should happen when
> you request via "--stdin" to create, update, or delete a symref,
> but <ref> is a regular ref, e.g., "symref-delete <ref>".  For
> "symref-create <ref> <new-ref>", we would fail if <ref> exists,
> whether it is a symref or a normal ref, so that is OK.  For
> "symref-delete <ref> <old-ref>", we would fail if <ref> is *not*
> a symref to <old-ref>, so the case where <ref> is a normal ref
> is already covered.
>
> Should we support "symref-update <ref> <new-ref> <old-oid>" that
> makes <ref> a symref that points at <new-ref>, but ensures that
> <ref> before the operation is a normal ref that points at <old-oid>?
>
> Or should "symref-update" work only on <ref> that is an existing
> symref?
>
> I think I am OK if the answer was "You can only give a precondition
> in the form of <old-ref>, which means you can only turn an existing
> symref to point at a different ref with precondition protection. If
> you want to turn a normal ref into a symref, you have to force it by
> not having a precondition, or delete the ref and then (re)create it
> as a symref".  But we need to decide the semantics and document it.

I would think that doing what you mentioned in the last para to be the
way to go, unless someone thinks otherwise. This allows the ugly mess of
parsing an value as a ref and then as a oid and provides some structure
to what the input cases are.

One more case I'd add is that the <ref> argument for "symref-delete"
should be optional, similar to having forced update, we'd also want to
support forced deletion.

> Junio C Hamano <gitster@pobox.com> writes:
>
>>> But this still means we need to think of the best output for the
>>> reference transaction hook (following commit).
>>> ...
>
> One thing I missed.  We are not currently reporting symref updates
> to these hooks.  Are they prepared to take such extra input?  If not,
> are they going to barf when they see "symref-update" while expecting
> to see <old-oid>?
>
> We may need to make it possible for Git to tell which variant of the
> hook script it was given somehow (the easiest cop-out is to introduce
> ref-transaction-hook-v2 as a separate hook, and use it if exists, or
> fall back to the reference-transaction hook, and report symref updates
> only when we are using v2, but there may be better approaches).
>

Well I was hoping this is okay since reference-transaction and
update-ref never supported symrefs. So after this change:
1. If a user never updates the hook to support symrefs, but doesn't use
symref feature of update-ref, they would be fine.
2. If a user starts using symref features of update-ref, they would see
that reference transaction needs to be updated too.

This especially since the hook's documentation always claimed that
symref support might be added in the future.

   The hook does not cover symbolic references (but that may change in
   the future).

---

In summary the plan going forward from my side would be to:

Implement the following in update-ref:

    symref-create SP <ref> SP <new-ref> LF
    symref-update SP <ref> SP <new-ref> [SP <old-ref>] LF
    symref-delete SP <ref> [SP <old-ref>] LF
    symref-verify SP <ref> [SP <old-ref>] LF

Also on the reference transaction hook, we'll be adding the following
new inputs to the hook:

    symref-create SP <new-ref> SP <ref> LF
    symref-delete SP <old-ref> SP <ref> LF
    symref-update SP <old-ref> SP <new-ref> SP <ref> LF

This either will be added to the existing hook or we would support a new
hook (v2).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH 8/8] refs: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In the previous commit, we added support for the
`update-symref` command in `git-update-ref`. While it allowed us to now
create symbolic refs via `git-update-ref`, it didn't activate the
'reference-transaction' hook.

Let's activate the hook for symbolic reference updates too. This takes
the form `<symref-target> SP <ref-name> LF`, which deviates from the
form for regular updates since this only contains two fields.

While this seems to be backward incompatible, it is okay, since the only
way the `reference-transaction` hook outputs this syntax is when
`git-update-ref` is used with `update-symref` command. The command was
only introduced in the previous commit and hence only users of this
command will face this incompatibility.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/githooks.txt       | 13 +++++++++++--
 refs.c                           | 17 +++++++++--------
 t/t1416-ref-transaction-hooks.sh | 27 +++++++++++++++++++++++++++
 3 files changed, 47 insertions(+), 10 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 37f91d5b50..ae9f02974d 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -485,8 +485,7 @@ reference-transaction
 
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
-committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+committed or aborted and may thus get called multiple times.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -513,6 +512,16 @@ to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For each symbolic reference update that was added to the transaction,
+the hook receives on standard input a line of the format:
+
+  <symref-target> SP <ref-name> LF
+
+where `<symref-target>` is the target of the symbolic reference update
+passed into the reference transaction, `<ref-name>` is the full name of
+the ref being updated. To distinguish from the regular updates, we can
+note that there are only two fields.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index 706dcd6deb..d0929c5684 100644
--- a/refs.c
+++ b/refs.c
@@ -2342,16 +2342,17 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		strbuf_reset(&buf);
 
-		// Reference transaction does not support symbolic updates.
 		if (update->flags & REF_UPDATE_SYMREF)
-			continue;
-
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+			strbuf_addf(&buf, "%s %s\n",
+				    update->symref_target,
+				    update->refname);
+		else
+			strbuf_addf(&buf, "%s %s %s\n",
+				    oid_to_hex(&update->old_oid),
+				    oid_to_hex(&update->new_oid),
+				    update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..fac5d5fc6d 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -83,6 +83,33 @@ test_expect_success 'hook gets all queued updates in committed state' '
 	test_cmp expect actual
 '
 
+test_expect_success 'hook gets all queued symref updates' '
+	test_when_finished "rm actual" &&
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+	cat >expect <<-EOF &&
+		prepared
+		refs/heads/test TESTSYMREF
+		refs/heads/test refs/heads/symref
+		committed
+		refs/heads/test TESTSYMREF
+		refs/heads/test refs/heads/symref
+	EOF
+	git update-ref --no-deref --stdin <<-EOF &&
+		start
+		update-symref TESTSYMREF refs/heads/test
+		update-symref refs/heads/symref refs/heads/test
+		prepare
+		commit
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'hook gets all queued updates in aborted state' '
 	test_when_finished "rm actual" &&
 	git reset --hard PRE &&
-- 
2.43.GIT

Re: [PATCH 8/8] refs: support symrefs in 'reference-transaction' hook

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 5702 bytes --]

On Sat, Mar 30, 2024 at 11:46:23PM +0100, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The 'reference-transaction' hook runs whenever a reference update is
> made to the system. In the previous commit, we added support for the
> `update-symref` command in `git-update-ref`. While it allowed us to now
> create symbolic refs via `git-update-ref`, it didn't activate the
> 'reference-transaction' hook.
> 
> Let's activate the hook for symbolic reference updates too. This takes
> the form `<symref-target> SP <ref-name> LF`, which deviates from the
> form for regular updates since this only contains two fields.
> 
> While this seems to be backward incompatible, it is okay, since the only
> way the `reference-transaction` hook outputs this syntax is when
> `git-update-ref` is used with `update-symref` command. The command was
> only introduced in the previous commit and hence only users of this
> command will face this incompatibility.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  Documentation/githooks.txt       | 13 +++++++++++--
>  refs.c                           | 17 +++++++++--------
>  t/t1416-ref-transaction-hooks.sh | 27 +++++++++++++++++++++++++++
>  3 files changed, 47 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
> index 37f91d5b50..ae9f02974d 100644
> --- a/Documentation/githooks.txt
> +++ b/Documentation/githooks.txt
> @@ -485,8 +485,7 @@ reference-transaction
>  
>  This hook is invoked by any Git command that performs reference
>  updates. It executes whenever a reference transaction is prepared,
> -committed or aborted and may thus get called multiple times. The hook
> -does not cover symbolic references (but that may change in the future).
> +committed or aborted and may thus get called multiple times.
>  
>  The hook takes exactly one argument, which is the current state the
>  given reference transaction is in:
> @@ -513,6 +512,16 @@ to be created anew, `<old-value>` is the all-zeroes object name. To
>  distinguish these cases, you can inspect the current value of
>  `<ref-name>` via `git rev-parse`.
>  
> +For each symbolic reference update that was added to the transaction,
> +the hook receives on standard input a line of the format:
> +
> +  <symref-target> SP <ref-name> LF

I was wondering whether we want the format to be a bit more explicit.
The proposed format works alright because refnames must not contain any
spaces, and thus it can be disambiguated from normal ref updates. But
it's rather easy to miss for authors of this hook.

I don't really know of a better format though. We could of course prefix
things with "symref:" or something like this, but that might not be a
good idea either. In hindsight it would've been clever to have a
specific prefix for every single ref update in the reference-transaction
hook. But well, here we are.

Patrick

> +where `<symref-target>` is the target of the symbolic reference update
> +passed into the reference transaction, `<ref-name>` is the full name of
> +the ref being updated. To distinguish from the regular updates, we can
> +note that there are only two fields.
> +
>  The exit status of the hook is ignored for any state except for the
>  "prepared" state. In the "prepared" state, a non-zero exit status will
>  cause the transaction to be aborted. The hook will not be called with
> diff --git a/refs.c b/refs.c
> index 706dcd6deb..d0929c5684 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2342,16 +2342,17 @@ static int run_transaction_hook(struct ref_transaction *transaction,
>  
>  	for (i = 0; i < transaction->nr; i++) {
>  		struct ref_update *update = transaction->updates[i];
> +		strbuf_reset(&buf);
>  
> -		// Reference transaction does not support symbolic updates.
>  		if (update->flags & REF_UPDATE_SYMREF)
> -			continue;
> -
> -		strbuf_reset(&buf);
> -		strbuf_addf(&buf, "%s %s %s\n",
> -			    oid_to_hex(&update->old_oid),
> -			    oid_to_hex(&update->new_oid),
> -			    update->refname);
> +			strbuf_addf(&buf, "%s %s\n",
> +				    update->symref_target,
> +				    update->refname);
> +		else
> +			strbuf_addf(&buf, "%s %s %s\n",
> +				    oid_to_hex(&update->old_oid),
> +				    oid_to_hex(&update->new_oid),
> +				    update->refname);
>  
>  		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
>  			if (errno != EPIPE) {
> diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
> index 2092488090..fac5d5fc6d 100755
> --- a/t/t1416-ref-transaction-hooks.sh
> +++ b/t/t1416-ref-transaction-hooks.sh
> @@ -83,6 +83,33 @@ test_expect_success 'hook gets all queued updates in committed state' '
>  	test_cmp expect actual
>  '
>  
> +test_expect_success 'hook gets all queued symref updates' '
> +	test_when_finished "rm actual" &&
> +	test_hook reference-transaction <<-\EOF &&
> +		echo "$*" >>actual
> +		while read -r line
> +		do
> +			printf "%s\n" "$line"
> +		done >>actual
> +	EOF
> +	cat >expect <<-EOF &&
> +		prepared
> +		refs/heads/test TESTSYMREF
> +		refs/heads/test refs/heads/symref
> +		committed
> +		refs/heads/test TESTSYMREF
> +		refs/heads/test refs/heads/symref
> +	EOF
> +	git update-ref --no-deref --stdin <<-EOF &&
> +		start
> +		update-symref TESTSYMREF refs/heads/test
> +		update-symref refs/heads/symref refs/heads/test
> +		prepare
> +		commit
> +	EOF
> +	test_cmp expect actual
> +'
> +
>  test_expect_success 'hook gets all queued updates in aborted state' '
>  	test_when_finished "rm actual" &&
>  	git reset --hard PRE &&
> -- 
> 2.43.GIT
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 0/7] update-ref: add symref oriented commands

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'git-update-ref(1)' command allows transactional reference updates.
But currently only supports regular reference updates. Meaning, if one
wants to update HEAD (symbolic ref) in a transaction, there is no tool
to do so.

One option to obtain transactional updates for the HEAD ref is to
manually create the HEAD.lock file and commit. This is intrusive, where
the user needs to mimic internal git behavior. Also, this only works
when using the files backend.

At GitLab, we've been using the manual process till date, to allow users
to set and change their default branch. But with the introduction of
reftables as a reference backend, this becomes a necessity to be solved
within git.

This patch series goes about introducing a set of commands
symref-{create,verify,delete,update} to work with symrefs complimenting
the existing commands for the regular refs within 'git-update-ref(1)'.

The 'symref-verify' command can be used to verify if a symref exists and
its existing value.

The 'symref-create' command can be used to create a new symref.

The 'symref-delete' command can be used to delete an existing symref while
optionally checking its existing value.

The 'symref-update' command can be used to update a symref, create a symref,
delete a symref or even convert an existing regular ref to a symref. Wherein
like the regular 'update' command, the zero OID can be used to create/delete
a symref.

V1 of the patch series can be found here:
https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com/

I'm not adding a range diff here, cause I redid the whole series, things which
have changed:
1. The earlier series simply propagated a 'symref_target' and only supported the
'symref-update' command, without checks for existing values and such. Now we
support the entire fleet of commands with support for checking old_values.
2. The flow is now changedc to send an old_ref, new_ref pair in supplement to
the existing old_oid, new_oid pair to the reference backends. This allows the
backends to simply do a combination of changes based on what values are set.
This allows us to do symref-update's where we change a regular ref to a symref
while also validating its old OID.
3. I added a lot more tests to cover reflog checks and also '-z' input.
4. The entered <old-ref> and <new-ref> values are checked within update-ref, to
ensure we don't create dangling refs. This could be extended in the future with
a flag, maybe "REF_ALLOW_DANGLING_SYMREF" and a corresponding option within
update-ref.
5. Removed some commits where reftable backend code was reused for symref creation.
This actually caused issues since the reused code also created a reflog along with
the symref but we should defer reflog creations only after all ref creations have
taken place. There is a bit of DRY here, but I think overall its still much cleaner.

Thanks all for the review and discussion in the previous version. Patrick, I did
incorporate the changes you suggested, but I just noticed that I didn't reply to
your emails.

Karthik Nayak (7):
  refs: accept symref values in `ref_transaction[_add]_update`
  update-ref: add support for symref-verify
  update-ref: add support for symref-delete
  files-backend: extract out `create_symref_lock`
  update-ref: add support for symref-create
  update-ref: add support for symref-update
  refs: support symrefs in 'reference-transaction' hook

 Documentation/git-update-ref.txt |  25 +++
 Documentation/githooks.txt       |  13 +-
 branch.c                         |   2 +-
 builtin/clone.c                  |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   4 +-
 builtin/receive-pack.c           |   4 +-
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             | 202 +++++++++++++++++--
 refs.c                           |  74 +++++--
 refs.h                           |  15 +-
 refs/files-backend.c             | 143 +++++++++++---
 refs/refs-internal.h             |  21 ++
 refs/reftable-backend.c          |  51 ++++-
 sequencer.c                      |   9 +-
 t/t0600-reffiles-backend.sh      |  32 ++++
 t/t1400-update-ref.sh            | 320 ++++++++++++++++++++++++++++++-
 t/t1416-ref-transaction-hooks.sh |  41 ++++
 walker.c                         |   2 +-
 20 files changed, 886 insertions(+), 82 deletions(-)

-- 
2.43.GIT

[PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `ref_transaction[_add]_update` functions obtain ref information and
flags to create a `ref_update` and add it to the transaction at hand.

To extend symref support in transactions, we need to also accept the
old and new ref values and process it. In this commit, let's add the
required paramaters to the function and modify all call sites.

The two paramaters added are `new_ref` and `old_ref`. The `new_ref` is
used to denote what the reference should point to when the transaction
is applied. Some functions allow this parameter to be NULL, meaning that
the reference is not changed, or `""`, meaning that the reference should
be deleted.

The `old_ref` denotes the value of that the reference must have before
the update. Some functions allow this parameter to be NULL, meaning that
the old value of the reference is not checked, or `""`, meaning that the
reference must not exist before the update. A copy of this value is made
in the transaction.

The handling logic of these parameters will be added in consequent
commits as we implement symref-{create, update, delete, verify}.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 15 ++++++++++-----
 refs.h                  |  9 ++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 55 insertions(+), 24 deletions(-)

diff --git a/branch.c b/branch.c
index 621019fcf4..3ebcfdca65 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 782bda007c..6a0b39de32 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5857d860db..66840b7c5b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 56d8a77ed7..ebea1747cb 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..7690687b0e 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 9a33cb50b4..40a65fdebc 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index e46afbc46d..21fdbf6ac8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
diff --git a/refs.c b/refs.c
index 55d2e0b2cb..967c81167e 100644
--- a/refs.c
+++ b/refs.c
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_ref, const char *old_ref,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1253,6 +1254,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_ref, const char *old_ref,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1282,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_ref, old_ref, msg);
 	return 0;
 }
 
@@ -1295,7 +1297,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1311,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1325,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1340,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index d278775e08..645fe9fdb8 100644
--- a/refs.h
+++ b/refs.h
@@ -696,13 +696,19 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  */
 #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
 
+/*
+ * The reference update is considered to be done on a symbolic reference. This
+ * ensures that we verify, delete, create and update the ref correspondingly.
+ */
+#define REF_SYMREF_UPDATE (1 << 12)
+
 /*
  * Bitmask of all of the flags that are allowed to be passed in to
  * ref_transaction_update() and friends:
  */
 #define REF_TRANSACTION_UPDATE_ALLOWED_FLAGS                                  \
 	(REF_NO_DEREF | REF_FORCE_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION | \
-	 REF_SKIP_REFNAME_VERIFICATION)
+	 REF_SKIP_REFNAME_VERIFICATION | REF_SYMREF_UPDATE )
 
 /*
  * Add a reference update to transaction. `new_oid` is the value that
@@ -722,6 +728,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_ref, const char *old_ref,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index a098d14ea0..e4d0aa3d41 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..4c5fe02687 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If (flags & REF_SYMREF_UPDATE), set the reference to this
+	 * value (or delete it, if `new_ref` is an empty string).
+	 */
+	const char *new_ref;
+
+	/*
+	 * If (type & REF_SYMREF_UPDATE), check that the reference
+	 * previously had this value (or didn't previously exist,
+	 * if `old_ref` is an empty string).
+	 */
+	const char *old_ref;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_ref, const char *old_ref,
 		const char *msg);
 
 /*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 1cda48c504..6104471199 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 2c19846385..af1b25692b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -616,7 +616,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1248,7 +1248,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3764,8 +3764,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index c0fd632d92..1b3df43906 100644
--- a/walker.c
+++ b/walker.c
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
-- 
2.43.GIT

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Christian Couder]

On Fri, Apr 12, 2024 at 11:59 AM Karthik Nayak <karthik.188@gmail.com> wrote:
>
> From: Karthik Nayak <karthik.188@gmail.com>
>
> The `ref_transaction[_add]_update` functions obtain ref information and
> flags to create a `ref_update` and add it to the transaction at hand.
>
> To extend symref support in transactions, we need to also accept the
> old and new ref values and process it. In this commit, let's add the
> required paramaters to the function and modify all call sites.

s/paramaters/parameters/

> The two paramaters added are `new_ref` and `old_ref`. The `new_ref` is

s/paramaters/parameters/

> used to denote what the reference should point to when the transaction
> is applied. Some functions allow this parameter to be NULL, meaning that
> the reference is not changed, or `""`, meaning that the reference should
> be deleted.

> The `old_ref` denotes the value of that the reference must have before

s/the value of that the reference/the value the reference/

> the update. Some functions allow this parameter to be NULL, meaning that
> the old value of the reference is not checked, or `""`, meaning that the
> reference must not exist before the update. A copy of this value is made
> in the transaction.
>
> The handling logic of these parameters will be added in consequent
> commits as we implement symref-{create, update, delete, verify}.
>
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>

> diff --git a/refs.h b/refs.h
> index d278775e08..645fe9fdb8 100644
> --- a/refs.h
> +++ b/refs.h

There is the following big comment in this file:

/*
 * Reference transaction updates
 *
 * The following four functions add a reference check or update to a
 * ref_transaction.  They have some common similar parameters:
 *
 *     transaction -- a pointer to an open ref_transaction, obtained
 *         from ref_transaction_begin().
 *
 *     refname -- the name of the reference to be affected.
 *
 *     new_oid -- the object ID that should be set to be the new value
 *         of the reference. Some functions allow this parameter to be
 *         NULL, meaning that the reference is not changed, or
 *         null_oid, meaning that the reference should be deleted. A
 *         copy of this value is made in the transaction.
 *
 *     old_oid -- the object ID that the reference must have before
 *         the update. Some functions allow this parameter to be NULL,
 *         meaning that the old value of the reference is not checked,
 *         or null_oid, meaning that the reference must not exist
 *         before the update. A copy of this value is made in the
 *         transaction.
 *
 *     flags -- flags affecting the update, passed to
 *         update_ref_lock(). Possible flags: REF_NO_DEREF,
 *         REF_FORCE_CREATE_REFLOG. See those constants for more
 *         information.
 *
 *     msg -- a message describing the change (for the reflog).
 *
 *     err -- a strbuf for receiving a description of any error that
 *         might have occurred.
 *
 * The functions make internal copies of refname and msg, so the
 * caller retains ownership of these parameters.
 *
 * The functions return 0 on success and non-zero on failure. A
 * failure means that the transaction as a whole has failed and needs
 * to be rolled back.
 */

I would expect the patch to update this comment.

> @@ -722,6 +728,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>                            const char *refname,
>                            const struct object_id *new_oid,
>                            const struct object_id *old_oid,
> +                          const char *new_ref, const char *old_ref,
>                            unsigned int flags, const char *msg,
>                            struct strbuf *err);

The patch might also want to update the comment just above the
ref_transaction_update() declaration as it is changing what the
function can (or will be able to) do.

> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 56641aa57a..4c5fe02687 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -124,6 +124,19 @@ struct ref_update {
>          */
>         struct object_id old_oid;
>
> +       /*
> +        * If (flags & REF_SYMREF_UPDATE), set the reference to this
> +        * value (or delete it, if `new_ref` is an empty string).

What if this value is NULL?

> +        */
> +       const char *new_ref;
> +
> +       /*
> +        * If (type & REF_SYMREF_UPDATE), check that the reference
> +        * previously had this value (or didn't previously exist,
> +        * if `old_ref` is an empty string).

What if this value is NULL?


> +        */
> +       const char *old_ref;
> +
>         /*
>          * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
>          * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 5373 bytes --]

Christian Couder <christian.couder@gmail.com> writes:

> On Fri, Apr 12, 2024 at 11:59 AM Karthik Nayak <karthik.188@gmail.com> wrote:
>>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The `ref_transaction[_add]_update` functions obtain ref information and
>> flags to create a `ref_update` and add it to the transaction at hand.
>>
>> To extend symref support in transactions, we need to also accept the
>> old and new ref values and process it. In this commit, let's add the
>> required paramaters to the function and modify all call sites.
>
> s/paramaters/parameters/
>
>> The two paramaters added are `new_ref` and `old_ref`. The `new_ref` is
>
> s/paramaters/parameters/
>

Thanks, will fix both.

>> used to denote what the reference should point to when the transaction
>> is applied. Some functions allow this parameter to be NULL, meaning that
>> the reference is not changed, or `""`, meaning that the reference should
>> be deleted.
>
>> The `old_ref` denotes the value of that the reference must have before
>
> s/the value of that the reference/the value the reference/
>

Will change.

>> the update. Some functions allow this parameter to be NULL, meaning that
>> the old value of the reference is not checked, or `""`, meaning that the
>> reference must not exist before the update. A copy of this value is made
>> in the transaction.
>>
>> The handling logic of these parameters will be added in consequent
>> commits as we implement symref-{create, update, delete, verify}.
>>
>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>
>> diff --git a/refs.h b/refs.h
>> index d278775e08..645fe9fdb8 100644
>> --- a/refs.h
>> +++ b/refs.h
>
> There is the following big comment in this file:
>
> /*
>  * Reference transaction updates
>  *
>  * The following four functions add a reference check or update to a
>  * ref_transaction.  They have some common similar parameters:
>  *
>  *     transaction -- a pointer to an open ref_transaction, obtained
>  *         from ref_transaction_begin().
>  *
>  *     refname -- the name of the reference to be affected.
>  *
>  *     new_oid -- the object ID that should be set to be the new value
>  *         of the reference. Some functions allow this parameter to be
>  *         NULL, meaning that the reference is not changed, or
>  *         null_oid, meaning that the reference should be deleted. A
>  *         copy of this value is made in the transaction.
>  *
>  *     old_oid -- the object ID that the reference must have before
>  *         the update. Some functions allow this parameter to be NULL,
>  *         meaning that the old value of the reference is not checked,
>  *         or null_oid, meaning that the reference must not exist
>  *         before the update. A copy of this value is made in the
>  *         transaction.
>  *
>  *     flags -- flags affecting the update, passed to
>  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
>  *         REF_FORCE_CREATE_REFLOG. See those constants for more
>  *         information.
>  *
>  *     msg -- a message describing the change (for the reflog).
>  *
>  *     err -- a strbuf for receiving a description of any error that
>  *         might have occurred.
>  *
>  * The functions make internal copies of refname and msg, so the
>  * caller retains ownership of these parameters.
>  *
>  * The functions return 0 on success and non-zero on failure. A
>  * failure means that the transaction as a whole has failed and needs
>  * to be rolled back.
>  */
>
> I would expect the patch to update this comment.
>

Since this patch doesn't use this value, I think its best to modify
this in each patch as we start using it, I'll do that.

>> @@ -722,6 +728,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>>                            const char *refname,
>>                            const struct object_id *new_oid,
>>                            const struct object_id *old_oid,
>> +                          const char *new_ref, const char *old_ref,
>>                            unsigned int flags, const char *msg,
>>                            struct strbuf *err);
>
> The patch might also want to update the comment just above the
> ref_transaction_update() declaration as it is changing what the
> function can (or will be able to) do.
>

Agreed, same as above, will modify in each patch. Also will add a
comment in the commit.

>> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
>> index 56641aa57a..4c5fe02687 100644
>> --- a/refs/refs-internal.h
>> +++ b/refs/refs-internal.h
>> @@ -124,6 +124,19 @@ struct ref_update {
>>          */
>>         struct object_id old_oid;
>>
>> +       /*
>> +        * If (flags & REF_SYMREF_UPDATE), set the reference to this
>> +        * value (or delete it, if `new_ref` is an empty string).
>
> What if this value is NULL?
>
>> +        */
>> +       const char *new_ref;
>> +
>> +       /*
>> +        * If (type & REF_SYMREF_UPDATE), check that the reference
>> +        * previously had this value (or didn't previously exist,
>> +        * if `old_ref` is an empty string).
>
> What if this value is NULL?
>

Well we ignore NULL values, but I see that the documentation is lacking,
will update.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Phillip Wood]

Hi Karthik

I agree with Christian's comments on this patch, I've got a couple of 
additional comments below

On 12/04/2024 10:59, Karthik Nayak wrote:
> diff --git a/refs.c b/refs.c
> index 55d2e0b2cb..967c81167e 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
>   		const char *refname, unsigned int flags,
>   		const struct object_id *new_oid,
>   		const struct object_id *old_oid,
> +		const char *new_ref, const char *old_ref,
>   		const char *msg)
>   {
>   	struct ref_update *update;
> @@ -1253,6 +1254,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>   			   const char *refname,
>   			   const struct object_id *new_oid,
>   			   const struct object_id *old_oid,
> +			   const char *new_ref, const char *old_ref,
>   			   unsigned int flags, const char *msg,
>   			   struct strbuf *err)
>   {

Adding these two new parameters is quite disruptive as all the existing 
callers have to be updated. It makes it easy for callers to misuse this 
function for example by providing old_oid and old_ref (I'm assuming that 
is an error but it is hard to know for sure without any documentation). 
It also makes the calling code harder to read because there are so many 
parameters it is hard to keep track of exactly what is being passed. An 
alternative strategy would be to add a new function that takes a struct 
instead of lots of individual parameters. That would make the calling 
code more readable as it would be clear which struct members are being 
set (see reset.h for an example of this). The approach of adding a 
struct is still prone to setting the wrong combination of options so 
either way it would be helpful to add some assertions to detect mistakes

	if (old_oid && old_ref)
		BUG("Only one of old_oid and old_ref should be non NULL");
	if (new_oid && new_ref)
		BUG("Only one of new_oid and new_ref should be non NULL");


> diff --git a/refs.h b/refs.h
> index d278775e08..645fe9fdb8 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -696,13 +696,19 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>    */
>   #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
>   
> +/*
> + * The reference update is considered to be done on a symbolic reference. This
> + * ensures that we verify, delete, create and update the ref correspondingly.
> + */
> +#define REF_SYMREF_UPDATE (1 << 12)

I'm confused as to why we need this as I assumed that we could use the 
presence of old_ref/new_ref to determine that the caller wants to update 
symbolic ref. Having this flag means that there are more possibilities 
to misuse the new API setting this flag but providing NULL for old_ref 
and new_ref.

Best Wishes

Phillip

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1165 bytes --]

On Thu, Apr 18, 2024 at 04:08:00PM +0100, Phillip Wood wrote:
[snip]
> > diff --git a/refs.h b/refs.h
> > index d278775e08..645fe9fdb8 100644
> > --- a/refs.h
> > +++ b/refs.h
> > @@ -696,13 +696,19 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
> >    */
> >   #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
> > +/*
> > + * The reference update is considered to be done on a symbolic reference. This
> > + * ensures that we verify, delete, create and update the ref correspondingly.
> > + */
> > +#define REF_SYMREF_UPDATE (1 << 12)
> 
> I'm confused as to why we need this as I assumed that we could use the
> presence of old_ref/new_ref to determine that the caller wants to update
> symbolic ref. Having this flag means that there are more possibilities to
> misuse the new API setting this flag but providing NULL for old_ref and
> new_ref.

In my opinion the same comment applies to `REF_HAVE_NEW` and
`REF_HAVE_OLD`, which I found to be redundant, as well. Those may make
sense in the internals when the object IDs are stored as non-pointers,
but queueing ref updates only accepts pointers anyway.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 4777 bytes --]

Hey Phillip,

Phillip Wood <phillip.wood123@gmail.com> writes:

> Hi Karthik
>
> I agree with Christian's comments on this patch, I've got a couple of
> additional comments below
>
> On 12/04/2024 10:59, Karthik Nayak wrote:
>> diff --git a/refs.c b/refs.c
>> index 55d2e0b2cb..967c81167e 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
>>   		const char *refname, unsigned int flags,
>>   		const struct object_id *new_oid,
>>   		const struct object_id *old_oid,
>> +		const char *new_ref, const char *old_ref,
>>   		const char *msg)
>>   {
>>   	struct ref_update *update;
>> @@ -1253,6 +1254,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>>   			   const char *refname,
>>   			   const struct object_id *new_oid,
>>   			   const struct object_id *old_oid,
>> +			   const char *new_ref, const char *old_ref,
>>   			   unsigned int flags, const char *msg,
>>   			   struct strbuf *err)
>>   {
>
> Adding these two new parameters is quite disruptive as all the existing
> callers have to be updated. It makes it easy for callers to misuse this
> function for example by providing old_oid and old_ref (I'm assuming that
> is an error but it is hard to know for sure without any documentation).



> It also makes the calling code harder to read because there are so many
> parameters it is hard to keep track of exactly what is being passed. An
> alternative strategy would be to add a new function that takes a struct
> instead of lots of individual parameters. That would make the calling
> code more readable as it would be clear which struct members are being
> set (see reset.h for an example of this). The approach of adding a
> struct is still prone to setting the wrong combination of options so

We do already have refs-internal.h:ref_update and this struct would be
the best candidate for what you're saying. I even thought of exposing
this struct and using it. I think I realized that it's a lot more work
to do this, because there are checks and cleanups which are built around
this struct. So exposing and using it would mean we refactor a bunch of
that code. Which while necessary I believe should be out of this series.
I'd actually be happy to do it right after we can agree that this is a
good direction.

> either way it would be helpful to add some assertions to detect mistakes
>
> 	if (old_oid && old_ref)
> 		BUG("Only one of old_oid and old_ref should be non NULL");
> 	if (new_oid && new_ref)
> 		BUG("Only one of new_oid and new_ref should be non NULL");
>

I have slightly modified it to:

 	if (old_oid && !is_null_oid(old_oid) && old_ref)
 		BUG("Only one of old_oid and old_ref should be non NULL");
 	if (new_oid && !is_null_oid(new_oid) && new_ref)
 		BUG("Only one of new_oid and new_ref should be non NULL");

But I agree, this is needed and have added it.

>> diff --git a/refs.h b/refs.h
>> index d278775e08..645fe9fdb8 100644
>> --- a/refs.h
>> +++ b/refs.h
>> @@ -696,13 +696,19 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>>    */
>>   #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
>>
>> +/*
>> + * The reference update is considered to be done on a symbolic reference. This
>> + * ensures that we verify, delete, create and update the ref correspondingly.
>> + */
>> +#define REF_SYMREF_UPDATE (1 << 12)
>
> I'm confused as to why we need this as I assumed that we could use the
> presence of old_ref/new_ref to determine that the caller wants to update
> symbolic ref. Having this flag means that there are more possibilities
> to misuse the new API setting this flag but providing NULL for old_ref
> and new_ref.
>

I think I started with this flag but as the direction of the series
changed and I moved using zero_oid values for deletion or for using the
verify command, this is not really needed anymore. I just tried removing
all the code around the flags and fixing up things and all the tests
still pass. Thanks for brining this up.

Patrick Steinhardt <ps@pks.im> writes:
>> I'm confused as to why we need this as I assumed that we could use the
>> presence of old_ref/new_ref to determine that the caller wants to update
>> symbolic ref. Having this flag means that there are more possibilities to
>> misuse the new API setting this flag but providing NULL for old_ref and
>> new_ref.
>
> In my opinion the same comment applies to `REF_HAVE_NEW` and
> `REF_HAVE_OLD`, which I found to be redundant, as well. Those may make
> sense in the internals when the object IDs are stored as non-pointers,
> but queueing ref updates only accepts pointers anyway.
>

Yeah like you mentioned, since we're dealing with pointers, checking the
if its set is enough indication, which doesn't work with the static OID
values.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[phillip.wood123]

Hi Karthik

Sorry for the slow response on this

On 19/04/2024 16:47, Karthik Nayak wrote:
>> either way it would be helpful to add some assertions to detect mistakes
>>
>> 	if (old_oid && old_ref)
>> 		BUG("Only one of old_oid and old_ref should be non NULL");
>> 	if (new_oid && new_ref)
>> 		BUG("Only one of new_oid and new_ref should be non NULL");
>>
> 
> I have slightly modified it to:
> 
>   	if (old_oid && !is_null_oid(old_oid) && old_ref)
>   		BUG("Only one of old_oid and old_ref should be non NULL");
>   	if (new_oid && !is_null_oid(new_oid) && new_ref)
>   		BUG("Only one of new_oid and new_ref should be non NULL");
> 
> But I agree, this is needed and have added it.

I'm confused as to why we want to allow old_ref in conjunction with 
old_oid being the null oid. Reading the documentation a null oid means 
"this ref must not exist" and NULL means "I don't care about the current 
state of the ref" so NULL and the null oid are not equivalent.

Best Wishes

Phillip

> 
>>> diff --git a/refs.h b/refs.h
>>> index d278775e08..645fe9fdb8 100644
>>> --- a/refs.h
>>> +++ b/refs.h
>>> @@ -696,13 +696,19 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>>>     */
>>>    #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
>>>
>>> +/*
>>> + * The reference update is considered to be done on a symbolic reference. This
>>> + * ensures that we verify, delete, create and update the ref correspondingly.
>>> + */
>>> +#define REF_SYMREF_UPDATE (1 << 12)
>>
>> I'm confused as to why we need this as I assumed that we could use the
>> presence of old_ref/new_ref to determine that the caller wants to update
>> symbolic ref. Having this flag means that there are more possibilities
>> to misuse the new API setting this flag but providing NULL for old_ref
>> and new_ref.
>>
> 
> I think I started with this flag but as the direction of the series
> changed and I moved using zero_oid values for deletion or for using the
> verify command, this is not really needed anymore. I just tried removing
> all the code around the flags and fixing up things and all the tests
> still pass. Thanks for brining this up.
> 
> Patrick Steinhardt <ps@pks.im> writes:
>>> I'm confused as to why we need this as I assumed that we could use the
>>> presence of old_ref/new_ref to determine that the caller wants to update
>>> symbolic ref. Having this flag means that there are more possibilities to
>>> misuse the new API setting this flag but providing NULL for old_ref and
>>> new_ref.
>>
>> In my opinion the same comment applies to `REF_HAVE_NEW` and
>> `REF_HAVE_OLD`, which I found to be redundant, as well. Those may make
>> sense in the internals when the object IDs are stored as non-pointers,
>> but queueing ref updates only accepts pointers anyway.
>>
> 
> Yeah like you mentioned, since we're dealing with pointers, checking the
> if its set is enough indication, which doesn't work with the static OID
> values.

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]

On Fri, Apr 12, 2024 at 11:59:02AM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The `ref_transaction[_add]_update` functions obtain ref information and
> flags to create a `ref_update` and add it to the transaction at hand.
> 
> To extend symref support in transactions, we need to also accept the
> old and new ref values and process it. In this commit, let's add the
> required paramaters to the function and modify all call sites.
> 
> The two paramaters added are `new_ref` and `old_ref`. The `new_ref` is

Would `new_target` and `old_target` be easier to understand? `new_ref`
and `old_ref` to me sound as if they might also apply to the ref itself,
for example when doing a rename.

[snip]
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 56641aa57a..4c5fe02687 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -124,6 +124,19 @@ struct ref_update {
>  	 */
>  	struct object_id old_oid;
>  
> +	/*
> +	 * If (flags & REF_SYMREF_UPDATE), set the reference to this
> +	 * value (or delete it, if `new_ref` is an empty string).
> +	 */
> +	const char *new_ref;
> +
> +	/*
> +	 * If (type & REF_SYMREF_UPDATE), check that the reference
> +	 * previously had this value (or didn't previously exist,
> +	 * if `old_ref` is an empty string).
> +	 */
> +	const char *old_ref;

I think one important bit of information here would be how to handle the
update from a plain ref to a symref or vice versa. Would I set both
`REF_SYMREF_UPDATE` and `REF_HAVE_NEW`/`REF_HAVE_OLD`?

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2196 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Fri, Apr 12, 2024 at 11:59:02AM +0200, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The `ref_transaction[_add]_update` functions obtain ref information and
>> flags to create a `ref_update` and add it to the transaction at hand.
>>
>> To extend symref support in transactions, we need to also accept the
>> old and new ref values and process it. In this commit, let's add the
>> required paramaters to the function and modify all call sites.
>>
>> The two paramaters added are `new_ref` and `old_ref`. The `new_ref` is
>
> Would `new_target` and `old_target` be easier to understand? `new_ref`
> and `old_ref` to me sound as if they might also apply to the ref itself,
> for example when doing a rename.
>

I guess that makes sense, I'll add it.

> [snip]
>> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
>> index 56641aa57a..4c5fe02687 100644
>> --- a/refs/refs-internal.h
>> +++ b/refs/refs-internal.h
>> @@ -124,6 +124,19 @@ struct ref_update {
>>  	 */
>>  	struct object_id old_oid;
>>
>> +	/*
>> +	 * If (flags & REF_SYMREF_UPDATE), set the reference to this
>> +	 * value (or delete it, if `new_ref` is an empty string).
>> +	 */
>> +	const char *new_ref;
>> +
>> +	/*
>> +	 * If (type & REF_SYMREF_UPDATE), check that the reference
>> +	 * previously had this value (or didn't previously exist,
>> +	 * if `old_ref` is an empty string).
>> +	 */
>> +	const char *old_ref;
>
> I think one important bit of information here would be how to handle the
> update from a plain ref to a symref or vice versa. Would I set both
> `REF_SYMREF_UPDATE` and `REF_HAVE_NEW`/`REF_HAVE_OLD`?

I'll now remove `REF_SYMREF_UPDATE` and add documentation around the
usage on `new_target` and `old_target`, where if either of them are set,
they take precedence over `old_oid` and `new_oid`. The `new_target` will
ensure that the ref is now a symbolic ref which points to the
`new_target` value. While the `old_target` will treat the ref as a
symbolic ref and check its old value.

`REF_HAVE_NEW`/`REF_HAVE_OLD` should however never be set by users of
ref.c, this is set internally. See REF_TRANSACTION_UPDATE_ALLOWED_FLAGS.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1975 bytes --]

On Fri, Apr 19, 2024 at 01:09:39PM -0500, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > On Fri, Apr 12, 2024 at 11:59:02AM +0200, Karthik Nayak wrote:
> >> From: Karthik Nayak <karthik.188@gmail.com>
> > [snip]
> >> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> >> index 56641aa57a..4c5fe02687 100644
> >> --- a/refs/refs-internal.h
> >> +++ b/refs/refs-internal.h
> >> @@ -124,6 +124,19 @@ struct ref_update {
> >>  	 */
> >>  	struct object_id old_oid;
> >>
> >> +	/*
> >> +	 * If (flags & REF_SYMREF_UPDATE), set the reference to this
> >> +	 * value (or delete it, if `new_ref` is an empty string).
> >> +	 */
> >> +	const char *new_ref;
> >> +
> >> +	/*
> >> +	 * If (type & REF_SYMREF_UPDATE), check that the reference
> >> +	 * previously had this value (or didn't previously exist,
> >> +	 * if `old_ref` is an empty string).
> >> +	 */
> >> +	const char *old_ref;
> >
> > I think one important bit of information here would be how to handle the
> > update from a plain ref to a symref or vice versa. Would I set both
> > `REF_SYMREF_UPDATE` and `REF_HAVE_NEW`/`REF_HAVE_OLD`?
> 
> I'll now remove `REF_SYMREF_UPDATE` and add documentation around the
> usage on `new_target` and `old_target`, where if either of them are set,
> they take precedence over `old_oid` and `new_oid`. The `new_target` will
> ensure that the ref is now a symbolic ref which points to the
> `new_target` value. While the `old_target` will treat the ref as a
> symbolic ref and check its old value.
> 
> `REF_HAVE_NEW`/`REF_HAVE_OLD` should however never be set by users of
> ref.c, this is set internally. See REF_TRANSACTION_UPDATE_ALLOWED_FLAGS.

Should they really take precedence, or should it be forbidden to pass
both old target and old object ID or new target and new object ID,
respectively? I'd rather claim the latter, and that should be detected
such that there is no bad surprises here.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2231 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Fri, Apr 19, 2024 at 01:09:39PM -0500, Karthik Nayak wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>>
>> > On Fri, Apr 12, 2024 at 11:59:02AM +0200, Karthik Nayak wrote:
>> >> From: Karthik Nayak <karthik.188@gmail.com>
>> > [snip]
>> >> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
>> >> index 56641aa57a..4c5fe02687 100644
>> >> --- a/refs/refs-internal.h
>> >> +++ b/refs/refs-internal.h
>> >> @@ -124,6 +124,19 @@ struct ref_update {
>> >>  	 */
>> >>  	struct object_id old_oid;
>> >>
>> >> +	/*
>> >> +	 * If (flags & REF_SYMREF_UPDATE), set the reference to this
>> >> +	 * value (or delete it, if `new_ref` is an empty string).
>> >> +	 */
>> >> +	const char *new_ref;
>> >> +
>> >> +	/*
>> >> +	 * If (type & REF_SYMREF_UPDATE), check that the reference
>> >> +	 * previously had this value (or didn't previously exist,
>> >> +	 * if `old_ref` is an empty string).
>> >> +	 */
>> >> +	const char *old_ref;
>> >
>> > I think one important bit of information here would be how to handle the
>> > update from a plain ref to a symref or vice versa. Would I set both
>> > `REF_SYMREF_UPDATE` and `REF_HAVE_NEW`/`REF_HAVE_OLD`?
>>
>> I'll now remove `REF_SYMREF_UPDATE` and add documentation around the
>> usage on `new_target` and `old_target`, where if either of them are set,
>> they take precedence over `old_oid` and `new_oid`. The `new_target` will
>> ensure that the ref is now a symbolic ref which points to the
>> `new_target` value. While the `old_target` will treat the ref as a
>> symbolic ref and check its old value.
>>
>> `REF_HAVE_NEW`/`REF_HAVE_OLD` should however never be set by users of
>> ref.c, this is set internally. See REF_TRANSACTION_UPDATE_ALLOWED_FLAGS.
>
> Should they really take precedence, or should it be forbidden to pass
> both old target and old object ID or new target and new object ID,
> respectively? I'd rather claim the latter, and that should be detected
> such that there is no bad surprises here.
>
> Patrick

I think after that email, I agreed to Phillip's suggestion [1] and now
I've added an explicit check for this.

[1]: https://lore.kernel.org/git/CAOLa=ZSwtOQXwbgregzKMtVA30wUCH8R=8D7u1+KGnsGEDD1oA@mail.gmail.com/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v2 2/7] update-ref: add support for symref-verify

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

In the previous commit, we added the required base for adding symref
support in transactions provided by the 'git-update-ref(1)'. This commit
introduces the 'symref-verify' command which is similar to the existing
'verify' command for regular refs.

The 'symref-verify' command allows users to verify if a provided <ref>
contains the provided <old-ref> without changing the <ref>. If <old-ref>
is not provided, the command will verify that the <ref> doesn't exist.
Since we're checking for symbolic refs, this command will only work with
the 'no-deref' mode. This is because any dereferenced symbolic ref will
point to an object and not a ref and the regular 'verify' command can be
used in such situations.

This commit adds all required helper functions required to also
introduce the other symref commands, namely create, delete, and update.
We also add tests to test the command in both the regular stdin mode and
also with the '-z' flag.

When the user doesn't provide a <old-ref> we need to check that the
provided <ref> doesn't exist. And to do this, we take over the existing
understanding that <old-oid> when set to its zero value, it refers to
the ref not existing. While this seems like a mix of contexts between
using <*-ref> and <*-oid>, this actually works really well, especially
considering the fact that we want to eventually also introduce

    symref-update SP <ref> SP <new-ref> [SP (<old-oid> | <old-rev>)] LF

and here, we'd allow the user to update a regular <ref> to a symref and
use <old-oid> to check the <ref>'s oid. This can be extrapolated to the
user using this to create a symref when provided a zero <old-oid>. Which
will work given how we're setting it up.

We also disable the reference-transaction hook for symref-updates which
will be tackled in its own commit.

Add required tests for 'symref-verify' while also adding reflog checks for
the pre-existing 'verify' tests.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |  7 +++
 builtin/update-ref.c             | 82 ++++++++++++++++++++++++++++----
 refs.c                           | 37 +++++++++++---
 refs.h                           |  1 +
 refs/files-backend.c             | 46 +++++++++++++++++-
 refs/refs-internal.h             |  7 +++
 refs/reftable-backend.c          | 23 ++++++++-
 t/t1400-update-ref.sh            | 80 ++++++++++++++++++++++++++++++-
 8 files changed, 262 insertions(+), 21 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 374a2ebd2b..749aaa7892 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-verify SP <ref> [SP <old-ref>] LF
 	option SP <opt> LF
 	start LF
 	prepare LF
@@ -86,6 +87,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-verify SP <ref> [NUL <old-ref>] NUL
 	option SP <opt> NUL
 	start NUL
 	prepare NUL
@@ -117,6 +119,11 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-verify::
+	Verify symbolic <ref> against <old-ref> but do not change it.
+	If <old-ref> is missing, the ref must not exist.  Can only be
+	used in `no-deref` mode.
+
 option::
 	Modify the behavior of the next command naming a <ref>.
 	The only valid option is `no-deref` to avoid dereferencing
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 21fdbf6ac8..4ae6bdcb12 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -76,6 +76,30 @@ static char *parse_refname(const char **next)
 	return strbuf_detach(&ref, NULL);
 }
 
+
+
+/*
+ * Wrapper around parse_refname which skips the next delimiter.
+ */
+static char *parse_next_refname(const char **next)
+{
+        if (line_termination) {
+                /* Without -z, consume SP and use next argument */
+                if (!**next || **next == line_termination)
+                        return NULL;
+                if (**next != ' ')
+                        die("expected SP but got: %s", *next);
+        } else {
+                /* With -z, read the next NUL-terminated line */
+                if (**next)
+                        return NULL;
+        }
+        /* Skip the delimiter */
+        (*next)++;
+
+        return parse_refname(next);
+}
+
 /*
  * The value being parsed is <old-oid> (as opposed to <new-oid>; the
  * difference affects which error messages are generated):
@@ -297,11 +321,48 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
 		die("verify %s: extra input: %s", refname, next);
 
 	if (ref_transaction_verify(transaction, refname, &old_oid,
-				   update_flags, &err))
+				   NULL, update_flags, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	strbuf_release(&err);
+}
+
+static void parse_cmd_symref_verify(struct ref_transaction *transaction,
+                                    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	struct object_id old_oid;
+	char *refname, *old_ref;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-verify: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-verify: missing <ref>");
+
+	/*
+	 * old_ref is optional, but we want to differentiate between
+	 * a NULL and zero value.
+	 */
+	old_ref = parse_next_refname(&next);
+	if (!old_ref)
+		old_oid = *null_oid();
+	else if (read_ref(old_ref, NULL))
+		die("symref-verify %s: invalid <old-ref>", refname);
+
+	if (*next != line_termination)
+		die("symref-verify %s: extra input: %s", refname, next);
+
+	if (ref_transaction_verify(transaction, refname, old_ref ? NULL : &old_oid,
+				   old_ref, update_flags | REF_SYMREF_UPDATE, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
 	free(refname);
+	free(old_ref);
 	strbuf_release(&err);
 }
 
@@ -380,15 +441,16 @@ static const struct parse_cmd {
 	unsigned args;
 	enum update_refs_state state;
 } command[] = {
-	{ "update",  parse_cmd_update,  3, UPDATE_REFS_OPEN },
-	{ "create",  parse_cmd_create,  2, UPDATE_REFS_OPEN },
-	{ "delete",  parse_cmd_delete,  2, UPDATE_REFS_OPEN },
-	{ "verify",  parse_cmd_verify,  2, UPDATE_REFS_OPEN },
-	{ "option",  parse_cmd_option,  1, UPDATE_REFS_OPEN },
-	{ "start",   parse_cmd_start,   0, UPDATE_REFS_STARTED },
-	{ "prepare", parse_cmd_prepare, 0, UPDATE_REFS_PREPARED },
-	{ "abort",   parse_cmd_abort,   0, UPDATE_REFS_CLOSED },
-	{ "commit",  parse_cmd_commit,  0, UPDATE_REFS_CLOSED },
+	{ "update",        parse_cmd_update,        3, UPDATE_REFS_OPEN },
+	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
+	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
+	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
+	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
+	{ "start",         parse_cmd_start,         0, UPDATE_REFS_STARTED },
+	{ "prepare",       parse_cmd_prepare,       0, UPDATE_REFS_PREPARED },
+	{ "abort",         parse_cmd_abort,         0, UPDATE_REFS_CLOSED },
+	{ "commit",        parse_cmd_commit,        0, UPDATE_REFS_CLOSED },
 };
 
 static void update_refs_stdin(void)
diff --git a/refs.c b/refs.c
index 967c81167e..124b294c9f 100644
--- a/refs.c
+++ b/refs.c
@@ -19,6 +19,7 @@
 #include "object-store-ll.h"
 #include "object.h"
 #include "path.h"
+#include "string.h"
 #include "tag.h"
 #include "submodule.h"
 #include "worktree.h"
@@ -29,6 +30,7 @@
 #include "date.h"
 #include "commit.h"
 #include "wildmatch.h"
+#include "wrapper.h"
 
 /*
  * List of all available backends
@@ -1217,6 +1219,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((void *)transaction->updates[i]->old_ref);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1242,10 +1245,19 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
-		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
-		oidcpy(&update->old_oid, old_oid);
+	/*
+	 * The ref values are to be considered over the oid values when we're
+	 * doing symref operations.
+	 */
+	if (update->flags & REF_SYMREF_UPDATE) {
+		if (old_ref)
+			update->old_ref = xstrdup(old_ref);
+	} else {
+		if (flags & REF_HAVE_NEW)
+			oidcpy(&update->new_oid, new_oid);
+		if (flags & REF_HAVE_OLD)
+			oidcpy(&update->old_oid, old_oid);
+	}
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1280,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_ref ? REF_HAVE_NEW : 0) | (old_ref ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_ref, old_ref, msg);
@@ -1318,14 +1331,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_ref,
 			   unsigned int flags,
 			   struct strbuf *err)
 {
-	if (!old_oid)
+	if (flags & REF_SYMREF_UPDATE && !old_ref && !old_oid)
+		BUG("verify called with old_ref set to NULL");
+	if (!(flags & REF_SYMREF_UPDATE) && !old_oid)
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
-				      NULL, NULL,
+				      NULL, old_ref,
 				      flags, NULL, err);
 }
 
@@ -2342,6 +2358,9 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
 
+		if (update->flags & REF_SYMREF_UPDATE)
+			continue;
+
 		strbuf_reset(&buf);
 		strbuf_addf(&buf, "%s %s %s\n",
 			    oid_to_hex(&update->old_oid),
@@ -2795,3 +2814,9 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int null_new_value(struct ref_update *update) {
+	if (update->flags & REF_SYMREF_UPDATE && update->new_ref)
+		return 0;
+	return is_null_oid(&update->new_oid);
+}
diff --git a/refs.h b/refs.h
index 645fe9fdb8..a988e672ff 100644
--- a/refs.h
+++ b/refs.h
@@ -772,6 +772,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_ref,
 			   unsigned int flags,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index e4d0aa3d41..8421530bde 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2411,6 +2411,37 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+/*
+ * Check whether the REF_HAVE_OLD and old_ref values stored in update
+ * are consistent with ref, which is the symbolic reference's current
+ * value. If everything is OK, return 0; otherwise, write an error
+ * message to err and return -1.
+ */
+static int check_old_ref(struct ref_update *update, char *ref,
+			 struct strbuf *err)
+{
+	if (!(update->flags & REF_HAVE_OLD) ||
+	    !strcmp(update->old_ref, ref))
+		return 0;
+
+	if (!strcmp(update->old_ref, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference already exists",
+			    original_update_refname(update));
+	else if (!strcmp(ref, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_ref);
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    ref, update->old_ref);
+
+	return -1;
+}
+
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2464,8 +2495,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 			       struct strbuf *err)
 {
 	struct strbuf referent = STRBUF_INIT;
-	int mustexist = (update->flags & REF_HAVE_OLD) &&
-		!is_null_oid(&update->old_oid);
+	int mustexist = (update->flags & REF_HAVE_OLD) && !is_null_oid(&update->old_oid);
 	int ret = 0;
 	struct ref_lock *lock;
 
@@ -2514,6 +2544,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
+			}
+
+			/*
+			 * For symref verification, we need to check the referent value
+			 * rather than the oid. If we're dealing with regular refs or we're
+			 * verifying a dereferenced symref, we then check the oid.
+			 */
+			if (update->flags & REF_SYMREF_UPDATE && update->old_ref) {
+				if (check_old_ref(update, referent.buf, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}
 			} else if (check_old_oid(update, &lock->old_oid, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto out;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 4c5fe02687..21c6b940d8 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -749,4 +749,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 6104471199..7a03922c7b 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -938,7 +938,28 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if ((u->flags & REF_HAVE_OLD) &&
+		    (u->flags & REF_SYMREF_UPDATE) &&
+		    u->old_ref) {
+			if   (strcmp(referent.buf, u->old_ref)) {
+				if (!strcmp(u->old_ref, ""))
+					strbuf_addf(err, "cannot lock ref '%s': "
+						    "reference already exists",
+						    original_update_refname(u));
+				else if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "cannot lock ref '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_ref);
+				else
+					strbuf_addf(err, "cannot lock ref '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_ref);
+				ret = -1;
+				goto done;
+			}
+		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 					    "reference already exists"),
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index ec3443cc87..d8ffda4096 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -890,17 +890,23 @@ test_expect_success 'stdin update/create/verify combination works' '
 '
 
 test_expect_success 'stdin verify succeeds for correct value' '
+	test-tool ref-store main for-each-reflog-ent $m >before &&
 	git rev-parse $m >expect &&
 	echo "verify $m $m" >stdin &&
 	git update-ref --stdin <stdin &&
 	git rev-parse $m >actual &&
-	test_cmp expect actual
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent $m >after &&
+	test_cmp before after
 '
 
 test_expect_success 'stdin verify succeeds for missing reference' '
+	test-tool ref-store main for-each-reflog-ent $m >before &&
 	echo "verify refs/heads/missing $Z" >stdin &&
 	git update-ref --stdin <stdin &&
-	test_must_fail git rev-parse --verify -q refs/heads/missing
+	test_must_fail git rev-parse --verify -q refs/heads/missing &&
+	test-tool ref-store main for-each-reflog-ent $m >after &&
+	test_cmp before after
 '
 
 test_expect_success 'stdin verify treats no value as missing' '
@@ -1641,4 +1647,74 @@ test_expect_success PIPE 'transaction flushes status updates' '
 	test_cmp expected actual
 '
 
+create_stdin_buf ()
+{
+	if test "$1" = "-z"
+	then
+		shift
+		printf "$F" "$@" >stdin
+	else
+		echo "$@" >stdin
+	fi
+}
+
+for type in "" "-z"
+do
+
+test_expect_success "stdin ${type} symref-verify fails without --no-deref" '
+	git symbolic-ref refs/heads/symref $a &&
+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+	grep "fatal: symref-verify: cannot operate with deref mode" err
+'
+
+test_expect_success "stdin ${type} symref-verify fails with too many arguments" '
+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" "$a" &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err  &&
+	if test "$type" = "-z"
+	then
+		grep "fatal: unknown command: $a" err
+	else
+		grep "fatal: symref-verify refs/heads/symref: extra input:  $a" err
+	fi
+'
+
+test_expect_success "stdin ${type} symref-verify succeeds for correct value" '
+	git symbolic-ref refs/heads/symref >expect &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
+	test_cmp before after
+'
+
+test_expect_success "stdin ${type} symref-verify succeeds for missing reference" '
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
+	create_stdin_buf ${type} "symref-verify refs/heads/missing" &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	test_must_fail git rev-parse --verify -q refs/heads/missing &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
+	test_cmp before after
+'
+
+test_expect_success "stdin ${type} symref-verify fails for wrong value" '
+	git symbolic-ref refs/heads/symref >expect &&
+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$b" &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-verify fails for mistaken null value" '
+	git symbolic-ref refs/heads/symref >expect &&
+	create_stdin_buf ${type} "symref-verify refs/heads/symref" &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual
+'
+
+done
+
 test_done
-- 
2.43.GIT

Re: [PATCH v2 2/7] update-ref: add support for symref-verify

[Christian Couder]

On Fri, Apr 12, 2024 at 11:59 AM Karthik Nayak <karthik.188@gmail.com> wrote:
>
> From: Karthik Nayak <karthik.188@gmail.com>
>
> In the previous commit, we added the required base for adding symref
> support in transactions provided by the 'git-update-ref(1)'. This commit

s/by the 'git-update-ref(1)'/by 'git-update-ref(1)'/

> introduces the 'symref-verify' command which is similar to the existing
> 'verify' command for regular refs.
>
> The 'symref-verify' command allows users to verify if a provided <ref>
> contains the provided <old-ref> without changing the <ref>.

What if <old-ref> looks like an oid? Will it work if <ref> is a
regular ref that actually contains this oid?

> If <old-ref>
> is not provided, the command will verify that the <ref> doesn't exist.
> Since we're checking for symbolic refs,

So if <ref> isn't a symbolic ref, it will fail? I guess the answer is
yes, but I think it would be better to be clear about this.

> this command will only work with
> the 'no-deref' mode. This is because any dereferenced symbolic ref will
> point to an object and not a ref and the regular 'verify' command can be
> used in such situations.
>
> This commit adds all required helper functions required to also
> introduce the other symref commands, namely create, delete, and update.

Are these helper functions actually used in this commit? If yes, it
would be nice to say it explicitly. If not, why is it a good idea to
add them now?

Also I think we prefer imperative wordings like "Add all required..."
over wordings like "This commit adds all required..."

> We also add tests to test the command in both the regular stdin mode and
> also with the '-z' flag.

> When the user doesn't provide a <old-ref> we need to check that the
> provided <ref> doesn't exist.

That the provided <ref> doesn't exist or that it isn't a symref?

> And to do this, we take over the existing
> understanding that <old-oid> when set to its zero value, it refers to
> the ref not existing. While this seems like a mix of contexts between
> using <*-ref> and <*-oid>, this actually works really well, especially
> considering the fact that we want to eventually also introduce
>
>     symref-update SP <ref> SP <new-ref> [SP (<old-oid> | <old-rev>)] LF
>
> and here, we'd allow the user to update a regular <ref> to a symref and
> use <old-oid> to check the <ref>'s oid.

This means that the ref actually exists but isn't a symref.

So if I understand correctly, for now it will work only if the ref
doesn't exist, but in the future we can make it work also if the ref
exists but isn't a symref.

> This can be extrapolated to the
> user using this to create a symref when provided a zero <old-oid>. Which
> will work given how we're setting it up.
>
> We also disable the reference-transaction hook for symref-updates which
> will be tackled in its own commit.

Why do we need to disable it?

> Add required tests for 'symref-verify' while also adding reflog checks for
> the pre-existing 'verify' tests.
>
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>

> +symref-verify::
> +       Verify symbolic <ref> against <old-ref> but do not change it.
> +       If <old-ref> is missing, the ref must not exist.

"must not exist" means that we will need to change this when we make
it work if the ref exists but isn't a symref. Ok.

>  Can only be
> +       used in `no-deref` mode.
> +


> +       /*
> +        * old_ref is optional, but we want to differentiate between
> +        * a NULL and zero value.
> +        */
> +       old_ref = parse_next_refname(&next);
> +       if (!old_ref)
> +               old_oid = *null_oid();

So for now we always overwrite old_oid when old_ref is not provided.
So the ref should not exist and the command will fail if it exists as
a regular ref. Ok.

> +       else if (read_ref(old_ref, NULL))
> +               die("symref-verify %s: invalid <old-ref>", refname);

So I guess we die() if old_ref is the empty string.

It's kind of strange as in the previous patch there was:

> +        * If (type & REF_SYMREF_UPDATE), check that the reference
> +        * previously had this value (or didn't previously exist,
> +        * if `old_ref` is an empty string).

So it said the empty string meant the old_ref didn't previously exist,
but now it's actually NULL that means the old_ref didn't previously
exist.

> +       if (*next != line_termination)
> +               die("symref-verify %s: extra input: %s", refname, next);
> +
> +       if (ref_transaction_verify(transaction, refname, old_ref ? NULL : &old_oid,
> +                                  old_ref, update_flags | REF_SYMREF_UPDATE, &err))
>                 die("%s", err.buf);
>
>         update_flags = default_flags;
>         free(refname);
> +       free(old_ref);
>         strbuf_release(&err);
>  }

Re: [PATCH v2 2/7] update-ref: add support for symref-verify

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 5626 bytes --]

Christian Couder <christian.couder@gmail.com> writes:

> On Fri, Apr 12, 2024 at 11:59 AM Karthik Nayak <karthik.188@gmail.com> wrote:
>>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> In the previous commit, we added the required base for adding symref
>> support in transactions provided by the 'git-update-ref(1)'. This commit
>
> s/by the 'git-update-ref(1)'/by 'git-update-ref(1)'/
>

will change.

>> introduces the 'symref-verify' command which is similar to the existing
>> 'verify' command for regular refs.
>>
>> The 'symref-verify' command allows users to verify if a provided <ref>
>> contains the provided <old-ref> without changing the <ref>.
>
> What if <old-ref> looks like an oid? Will it work if <ref> is a
> regular ref that actually contains this oid?
>

This would fail, since we parse and check the ref entered.

>> If <old-ref>
>> is not provided, the command will verify that the <ref> doesn't exist.
>> Since we're checking for symbolic refs,
>
> So if <ref> isn't a symbolic ref, it will fail? I guess the answer is
> yes, but I think it would be better to be clear about this.

Will try and clarify more in the commit message.

>> this command will only work with
>> the 'no-deref' mode. This is because any dereferenced symbolic ref will
>> point to an object and not a ref and the regular 'verify' command can be
>> used in such situations.
>>
>> This commit adds all required helper functions required to also
>> introduce the other symref commands, namely create, delete, and update.
>
> Are these helper functions actually used in this commit? If yes, it
> would be nice to say it explicitly. If not, why is it a good idea to
> add them now?
>
> Also I think we prefer imperative wordings like "Add all required..."
> over wordings like "This commit adds all required..."
>

Will clarify and make it imperative.

>> We also add tests to test the command in both the regular stdin mode and
>> also with the '-z' flag.
>
>> When the user doesn't provide a <old-ref> we need to check that the
>> provided <ref> doesn't exist.
>
> That the provided <ref> doesn't exist or that it isn't a symref?

That it doesn't exist. I don't know if it makes sense to make it 'that
it isn't a symref'.

>> And to do this, we take over the existing
>> understanding that <old-oid> when set to its zero value, it refers to
>> the ref not existing. While this seems like a mix of contexts between
>> using <*-ref> and <*-oid>, this actually works really well, especially
>> considering the fact that we want to eventually also introduce
>>
>>     symref-update SP <ref> SP <new-ref> [SP (<old-oid> | <old-rev>)] LF
>>
>> and here, we'd allow the user to update a regular <ref> to a symref and
>> use <old-oid> to check the <ref>'s oid.
>
> This means that the ref actually exists but isn't a symref.
>
> So if I understand correctly, for now it will work only if the ref
> doesn't exist, but in the future we can make it work also if the ref
> exists but isn't a symref.

This comment specifically talks about the `symref-update` command which
is in an upcoming commit, in that commit, we do allow users to update a
regular ref to a symref, while also checking the old_oid value of that
ref.

>> This can be extrapolated to the
>> user using this to create a symref when provided a zero <old-oid>. Which
>> will work given how we're setting it up.
>>
>> We also disable the reference-transaction hook for symref-updates which
>> will be tackled in its own commit.
>
> Why do we need to disable it?
>

Historically, reference transaction didn't support symrefs, so we're not
actually changing functionality here. While we can fix that in this
commit and add tests, it makes more sense to tackle it in its own
commit, which we do at the end.

>> Add required tests for 'symref-verify' while also adding reflog checks for
>> the pre-existing 'verify' tests.
>>
>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>
>> +symref-verify::
>> +       Verify symbolic <ref> against <old-ref> but do not change it.
>> +       If <old-ref> is missing, the ref must not exist.
>
> "must not exist" means that we will need to change this when we make
> it work if the ref exists but isn't a symref. Ok.
>
>>  Can only be
>> +       used in `no-deref` mode.
>> +
>
>
>> +       /*
>> +        * old_ref is optional, but we want to differentiate between
>> +        * a NULL and zero value.
>> +        */
>> +       old_ref = parse_next_refname(&next);
>> +       if (!old_ref)
>> +               old_oid = *null_oid();
>
> So for now we always overwrite old_oid when old_ref is not provided.
> So the ref should not exist and the command will fail if it exists as
> a regular ref. Ok.
>
>> +       else if (read_ref(old_ref, NULL))
>> +               die("symref-verify %s: invalid <old-ref>", refname);
>
> So I guess we die() if old_ref is the empty string.
>
> It's kind of strange as in the previous patch there was:
>
>> +        * If (type & REF_SYMREF_UPDATE), check that the reference
>> +        * previously had this value (or didn't previously exist,
>> +        * if `old_ref` is an empty string).
>
> So it said the empty string meant the old_ref didn't previously exist,
> but now it's actually NULL that means the old_ref didn't previously
> exist.
>

Good catch, I initially did start with the empty string idea, but
switched to using zero_oid, since it made a lot more sense, especially
because now it sets interoperability. I will fix that comment.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 2/7] update-ref: add support for symref-verify

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 21521 bytes --]

On Fri, Apr 12, 2024 at 11:59:03AM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> In the previous commit, we added the required base for adding symref
> support in transactions provided by the 'git-update-ref(1)'. This commit
> introduces the 'symref-verify' command which is similar to the existing
> 'verify' command for regular refs.
> 
> The 'symref-verify' command allows users to verify if a provided <ref>
> contains the provided <old-ref> without changing the <ref>. If <old-ref>
> is not provided, the command will verify that the <ref> doesn't exist.
> Since we're checking for symbolic refs, this command will only work with
> the 'no-deref' mode. This is because any dereferenced symbolic ref will
> point to an object and not a ref and the regular 'verify' command can be
> used in such situations.
> 
> This commit adds all required helper functions required to also
> introduce the other symref commands, namely create, delete, and update.
> We also add tests to test the command in both the regular stdin mode and
> also with the '-z' flag.
> 
> When the user doesn't provide a <old-ref> we need to check that the
> provided <ref> doesn't exist. And to do this, we take over the existing
> understanding that <old-oid> when set to its zero value, it refers to
> the ref not existing. While this seems like a mix of contexts between
> using <*-ref> and <*-oid>, this actually works really well, especially
> considering the fact that we want to eventually also introduce
> 
>     symref-update SP <ref> SP <new-ref> [SP (<old-oid> | <old-rev>)] LF
> 
> and here, we'd allow the user to update a regular <ref> to a symref and
> use <old-oid> to check the <ref>'s oid. This can be extrapolated to the
> user using this to create a symref when provided a zero <old-oid>. Which
> will work given how we're setting it up.
> 
> We also disable the reference-transaction hook for symref-updates which
> will be tackled in its own commit.
> 
> Add required tests for 'symref-verify' while also adding reflog checks for
> the pre-existing 'verify' tests.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  Documentation/git-update-ref.txt |  7 +++
>  builtin/update-ref.c             | 82 ++++++++++++++++++++++++++++----
>  refs.c                           | 37 +++++++++++---
>  refs.h                           |  1 +
>  refs/files-backend.c             | 46 +++++++++++++++++-
>  refs/refs-internal.h             |  7 +++
>  refs/reftable-backend.c          | 23 ++++++++-
>  t/t1400-update-ref.sh            | 80 ++++++++++++++++++++++++++++++-
>  8 files changed, 262 insertions(+), 21 deletions(-)
> 
> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index 374a2ebd2b..749aaa7892 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
>  	create SP <ref> SP <new-oid> LF
>  	delete SP <ref> [SP <old-oid>] LF
>  	verify SP <ref> [SP <old-oid>] LF
> +	symref-verify SP <ref> [SP <old-ref>] LF
>  	option SP <opt> LF
>  	start LF
>  	prepare LF
> @@ -86,6 +87,7 @@ quoting:
>  	create SP <ref> NUL <new-oid> NUL
>  	delete SP <ref> NUL [<old-oid>] NUL
>  	verify SP <ref> NUL [<old-oid>] NUL
> +	symref-verify SP <ref> [NUL <old-ref>] NUL
>  	option SP <opt> NUL
>  	start NUL
>  	prepare NUL
> @@ -117,6 +119,11 @@ verify::
>  	Verify <ref> against <old-oid> but do not change it.  If
>  	<old-oid> is zero or missing, the ref must not exist.
>  
> +symref-verify::
> +	Verify symbolic <ref> against <old-ref> but do not change it.
> +	If <old-ref> is missing, the ref must not exist.  Can only be
> +	used in `no-deref` mode.

Should this say "is zero or missing", like the comment for "verify"
does?

[snip]
> @@ -297,11 +321,48 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
>  		die("verify %s: extra input: %s", refname, next);
>  
>  	if (ref_transaction_verify(transaction, refname, &old_oid,
> -				   update_flags, &err))
> +				   NULL, update_flags, &err))
> +		die("%s", err.buf);
> +
> +	update_flags = default_flags;
> +	free(refname);
> +	strbuf_release(&err);
> +}
> +
> +static void parse_cmd_symref_verify(struct ref_transaction *transaction,
> +                                    const char *next, const char *end)
> +{
> +	struct strbuf err = STRBUF_INIT;
> +	struct object_id old_oid;
> +	char *refname, *old_ref;
> +
> +	if (!(update_flags & REF_NO_DEREF))
> +		die("symref-verify: cannot operate with deref mode");

This feels quite restrictive to me. Wouldn't it be preferable to simply
ignore `REF_NO_DEREF` here? It basically means that this command can't
ever be used in a normal `git update-ref --stdin` session.

> +	refname = parse_refname(&next);
> +	if (!refname)
> +		die("symref-verify: missing <ref>");
> +
> +	/*
> +	 * old_ref is optional, but we want to differentiate between
> +	 * a NULL and zero value.
> +	 */
> +	old_ref = parse_next_refname(&next);
> +	if (!old_ref)
> +		old_oid = *null_oid();
> +	else if (read_ref(old_ref, NULL))
> +		die("symref-verify %s: invalid <old-ref>", refname);
> +
> +	if (*next != line_termination)
> +		die("symref-verify %s: extra input: %s", refname, next);
> +
> +	if (ref_transaction_verify(transaction, refname, old_ref ? NULL : &old_oid,
> +				   old_ref, update_flags | REF_SYMREF_UPDATE, &err))
>  		die("%s", err.buf);
>  
>  	update_flags = default_flags;
>  	free(refname);
> +	free(old_ref);
>  	strbuf_release(&err);
>  }
>  
> @@ -380,15 +441,16 @@ static const struct parse_cmd {
>  	unsigned args;
>  	enum update_refs_state state;
>  } command[] = {
> -	{ "update",  parse_cmd_update,  3, UPDATE_REFS_OPEN },
> -	{ "create",  parse_cmd_create,  2, UPDATE_REFS_OPEN },
> -	{ "delete",  parse_cmd_delete,  2, UPDATE_REFS_OPEN },
> -	{ "verify",  parse_cmd_verify,  2, UPDATE_REFS_OPEN },
> -	{ "option",  parse_cmd_option,  1, UPDATE_REFS_OPEN },
> -	{ "start",   parse_cmd_start,   0, UPDATE_REFS_STARTED },
> -	{ "prepare", parse_cmd_prepare, 0, UPDATE_REFS_PREPARED },
> -	{ "abort",   parse_cmd_abort,   0, UPDATE_REFS_CLOSED },
> -	{ "commit",  parse_cmd_commit,  0, UPDATE_REFS_CLOSED },
> +	{ "update",        parse_cmd_update,        3, UPDATE_REFS_OPEN },
> +	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
> +	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
> +	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
> +	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
> +	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
> +	{ "start",         parse_cmd_start,         0, UPDATE_REFS_STARTED },
> +	{ "prepare",       parse_cmd_prepare,       0, UPDATE_REFS_PREPARED },
> +	{ "abort",         parse_cmd_abort,         0, UPDATE_REFS_CLOSED },
> +	{ "commit",        parse_cmd_commit,        0, UPDATE_REFS_CLOSED },
>  };
>  
>  static void update_refs_stdin(void)
> diff --git a/refs.c b/refs.c
> index 967c81167e..124b294c9f 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -19,6 +19,7 @@
>  #include "object-store-ll.h"
>  #include "object.h"
>  #include "path.h"
> +#include "string.h"
>  #include "tag.h"
>  #include "submodule.h"
>  #include "worktree.h"
> @@ -29,6 +30,7 @@
>  #include "date.h"
>  #include "commit.h"
>  #include "wildmatch.h"
> +#include "wrapper.h"
>  
>  /*
>   * List of all available backends
> @@ -1217,6 +1219,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
>  
>  	for (i = 0; i < transaction->nr; i++) {
>  		free(transaction->updates[i]->msg);
> +		free((void *)transaction->updates[i]->old_ref);
>  		free(transaction->updates[i]);
>  	}
>  	free(transaction->updates);
> @@ -1242,10 +1245,19 @@ struct ref_update *ref_transaction_add_update(
>  
>  	update->flags = flags;
>  
> -	if (flags & REF_HAVE_NEW)
> -		oidcpy(&update->new_oid, new_oid);
> -	if (flags & REF_HAVE_OLD)
> -		oidcpy(&update->old_oid, old_oid);
> +	/*
> +	 * The ref values are to be considered over the oid values when we're
> +	 * doing symref operations.
> +	 */

I feel like this is a statement that should be backed up by a deeper
explanation of why that is. I'm still wondering here why we cannot
assert that the old value is an object ID when I want to update it to a
symref, or alternatively why it would even be possible to have both
`REF_SYMREF_UPDATE` and a set of other, incompatible fields set. It
feels like this should be a `BUG()` instead if this is supposedly an
unsupported configuration rather than silently ignoring it.

In any case, I feel like it would be easier to reason about if this was
introduced together with the actual user. As far as I can see this code
shouldn't ever be hit for "verify-symref", right? Currently, the reader
is forced to figure out what is and isn't related to the new command.

> +	if (update->flags & REF_SYMREF_UPDATE) {
> +		if (old_ref)
> +			update->old_ref = xstrdup(old_ref);
> +	} else {
> +		if (flags & REF_HAVE_NEW)
> +			oidcpy(&update->new_oid, new_oid);
> +		if (flags & REF_HAVE_OLD)
> +			oidcpy(&update->old_oid, old_oid);
> +	}
>  	update->msg = normalize_reflog_message(msg);
>  	return update;
>  }
> @@ -1280,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>  	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
>  
>  	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
> +	flags |= (new_ref ? REF_HAVE_NEW : 0) | (old_ref ? REF_HAVE_OLD : 0);
>  
>  	ref_transaction_add_update(transaction, refname, flags,
>  				   new_oid, old_oid, new_ref, old_ref, msg);
> @@ -1318,14 +1331,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
>  int ref_transaction_verify(struct ref_transaction *transaction,
>  			   const char *refname,
>  			   const struct object_id *old_oid,
> +			   const char *old_ref,
>  			   unsigned int flags,
>  			   struct strbuf *err)
>  {
> -	if (!old_oid)
> +	if (flags & REF_SYMREF_UPDATE && !old_ref && !old_oid)
> +		BUG("verify called with old_ref set to NULL");
> +	if (!(flags & REF_SYMREF_UPDATE) && !old_oid)
>  		BUG("verify called with old_oid set to NULL");
>  	return ref_transaction_update(transaction, refname,
>  				      NULL, old_oid,
> -				      NULL, NULL,
> +				      NULL, old_ref,
>  				      flags, NULL, err);
>  }
>  
> @@ -2342,6 +2358,9 @@ static int run_transaction_hook(struct ref_transaction *transaction,
>  	for (i = 0; i < transaction->nr; i++) {
>  		struct ref_update *update = transaction->updates[i];
>  
> +		if (update->flags & REF_SYMREF_UPDATE)
> +			continue;
> +
>  		strbuf_reset(&buf);
>  		strbuf_addf(&buf, "%s %s %s\n",
>  			    oid_to_hex(&update->old_oid),
> @@ -2795,3 +2814,9 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
>  {
>  	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
>  }
> +
> +int null_new_value(struct ref_update *update) {
> +	if (update->flags & REF_SYMREF_UPDATE && update->new_ref)
> +		return 0;
> +	return is_null_oid(&update->new_oid);
> +}
> diff --git a/refs.h b/refs.h
> index 645fe9fdb8..a988e672ff 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -772,6 +772,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
>  int ref_transaction_verify(struct ref_transaction *transaction,
>  			   const char *refname,
>  			   const struct object_id *old_oid,
> +			   const char *old_ref,
>  			   unsigned int flags,
>  			   struct strbuf *err);
>  
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index e4d0aa3d41..8421530bde 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2411,6 +2411,37 @@ static const char *original_update_refname(struct ref_update *update)
>  	return update->refname;
>  }
>  
> +/*
> + * Check whether the REF_HAVE_OLD and old_ref values stored in update
> + * are consistent with ref, which is the symbolic reference's current
> + * value. If everything is OK, return 0; otherwise, write an error
> + * message to err and return -1.
> + */
> +static int check_old_ref(struct ref_update *update, char *ref,
> +			 struct strbuf *err)
> +{
> +	if (!(update->flags & REF_HAVE_OLD) ||
> +	    !strcmp(update->old_ref, ref))
> +		return 0;
> +
> +	if (!strcmp(update->old_ref, ""))
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "reference already exists",
> +			    original_update_refname(update));
> +	else if (!strcmp(ref, ""))
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "reference is missing but expected %s",
> +			    original_update_refname(update),
> +			    update->old_ref);
> +	else
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "is at %s but expected %s",
> +			    original_update_refname(update),
> +			    ref, update->old_ref);
> +
> +	return -1;
> +}
> +
>  /*
>   * Check whether the REF_HAVE_OLD and old_oid values stored in update
>   * are consistent with oid, which is the reference's current value. If
> @@ -2464,8 +2495,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>  			       struct strbuf *err)
>  {
>  	struct strbuf referent = STRBUF_INIT;
> -	int mustexist = (update->flags & REF_HAVE_OLD) &&
> -		!is_null_oid(&update->old_oid);
> +	int mustexist = (update->flags & REF_HAVE_OLD) && !is_null_oid(&update->old_oid);

This change is a no-op, right? If so, let's rather drop it.

>  	int ret = 0;
>  	struct ref_lock *lock;
>  
> @@ -2514,6 +2544,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>  					ret = TRANSACTION_GENERIC_ERROR;
>  					goto out;
>  				}
> +			}
> +
> +			/*
> +			 * For symref verification, we need to check the referent value
> +			 * rather than the oid. If we're dealing with regular refs or we're
> +			 * verifying a dereferenced symref, we then check the oid.
> +			 */
> +			if (update->flags & REF_SYMREF_UPDATE && update->old_ref) {
> +				if (check_old_ref(update, referent.buf, err)) {
> +					ret = TRANSACTION_GENERIC_ERROR;
> +					goto out;
> +				}
>  			} else if (check_old_oid(update, &lock->old_oid, err)) {
>  				ret = TRANSACTION_GENERIC_ERROR;
>  				goto out;
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 4c5fe02687..21c6b940d8 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -749,4 +749,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
>   */
>  struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
>  
> +/*
> + * Helper function to check if the new value is null, this
> + * takes into consideration that the update could be a regular
> + * ref or a symbolic ref.
> + */
> +int null_new_value(struct ref_update *update);

When adding it to the header we should probably prefix this to avoid
name collisions. `ref_update_is_null_new_value()` might be a mouth full,
but feels preferable to me.

>  #endif /* REFS_REFS_INTERNAL_H */
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index 6104471199..7a03922c7b 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -938,7 +938,28 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>  		 * individual refs. But the error messages match what the files
>  		 * backend returns, which keeps our tests happy.
>  		 */
> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
> +		if ((u->flags & REF_HAVE_OLD) &&
> +		    (u->flags & REF_SYMREF_UPDATE) &&
> +		    u->old_ref) {
> +			if   (strcmp(referent.buf, u->old_ref)) {

s/   / /

> +				if (!strcmp(u->old_ref, ""))
> +					strbuf_addf(err, "cannot lock ref '%s': "
> +						    "reference already exists",
> +						    original_update_refname(u));
> +				else if (!strcmp(referent.buf, ""))
> +					strbuf_addf(err, "cannot lock ref '%s': "
> +						    "reference is missing but expected %s",
> +						    original_update_refname(u),
> +						    u->old_ref);
> +				else
> +					strbuf_addf(err, "cannot lock ref '%s': "
> +						    "is at %s but expected %s",
> +						    original_update_refname(u),
> +						    referent.buf, u->old_ref);

I'd use better-matching error messages here. I know that we talk about
"cannot lock ref" in the next branch, as well. But the only reason we
did this is to have the same error messages as the "files" backend.
Semantically, those errors don't make much sense as the "reftable"
backend never locks specific refs, but only the complete stack.

> +				ret = -1;
> +				goto done;
> +			}
> +		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>  			if (is_null_oid(&u->old_oid))
>  				strbuf_addf(err, _("cannot lock ref '%s': "
>  					    "reference already exists"),
> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
> index ec3443cc87..d8ffda4096 100755
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -890,17 +890,23 @@ test_expect_success 'stdin update/create/verify combination works' '
>  '
>  
>  test_expect_success 'stdin verify succeeds for correct value' '
> +	test-tool ref-store main for-each-reflog-ent $m >before &&
>  	git rev-parse $m >expect &&
>  	echo "verify $m $m" >stdin &&
>  	git update-ref --stdin <stdin &&
>  	git rev-parse $m >actual &&
> -	test_cmp expect actual
> +	test_cmp expect actual &&
> +	test-tool ref-store main for-each-reflog-ent $m >after &&
> +	test_cmp before after
>  '
>  
>  test_expect_success 'stdin verify succeeds for missing reference' '
> +	test-tool ref-store main for-each-reflog-ent $m >before &&
>  	echo "verify refs/heads/missing $Z" >stdin &&
>  	git update-ref --stdin <stdin &&
> -	test_must_fail git rev-parse --verify -q refs/heads/missing
> +	test_must_fail git rev-parse --verify -q refs/heads/missing &&
> +	test-tool ref-store main for-each-reflog-ent $m >after &&
> +	test_cmp before after
>  '

The updated tests merely assert that the refs didn't change, right?

>  test_expect_success 'stdin verify treats no value as missing' '
> @@ -1641,4 +1647,74 @@ test_expect_success PIPE 'transaction flushes status updates' '
>  	test_cmp expected actual
>  '
>  
> +create_stdin_buf ()
> +{

The curly brace should go on the same line as the function name.

> +	if test "$1" = "-z"
> +	then
> +		shift
> +		printf "$F" "$@" >stdin
> +	else
> +		echo "$@" >stdin
> +	fi
> +}
> +
> +for type in "" "-z"
> +do

We should probably indent all of the tests to make it easier to see that
they run in a loop.

Patrick

> +test_expect_success "stdin ${type} symref-verify fails without --no-deref" '
> +	git symbolic-ref refs/heads/symref $a &&
> +	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
> +	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
> +	grep "fatal: symref-verify: cannot operate with deref mode" err
> +'
> +
> +test_expect_success "stdin ${type} symref-verify fails with too many arguments" '
> +	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" "$a" &&
> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err  &&
> +	if test "$type" = "-z"
> +	then
> +		grep "fatal: unknown command: $a" err
> +	else
> +		grep "fatal: symref-verify refs/heads/symref: extra input:  $a" err
> +	fi
> +'
> +
> +test_expect_success "stdin ${type} symref-verify succeeds for correct value" '
> +	git symbolic-ref refs/heads/symref >expect &&
> +	test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
> +	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
> +	git update-ref --stdin ${type} --no-deref <stdin &&
> +	git symbolic-ref refs/heads/symref >actual &&
> +	test_cmp expect actual &&
> +	test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
> +	test_cmp before after
> +'
> +
> +test_expect_success "stdin ${type} symref-verify succeeds for missing reference" '
> +	test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
> +	create_stdin_buf ${type} "symref-verify refs/heads/missing" &&
> +	git update-ref --stdin ${type} --no-deref <stdin &&
> +	test_must_fail git rev-parse --verify -q refs/heads/missing &&
> +	test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
> +	test_cmp before after
> +'
> +
> +test_expect_success "stdin ${type} symref-verify fails for wrong value" '
> +	git symbolic-ref refs/heads/symref >expect &&
> +	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$b" &&
> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
> +	git symbolic-ref refs/heads/symref >actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success "stdin ${type} symref-verify fails for mistaken null value" '
> +	git symbolic-ref refs/heads/symref >expect &&
> +	create_stdin_buf ${type} "symref-verify refs/heads/symref" &&
> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
> +	git symbolic-ref refs/heads/symref >actual &&
> +	test_cmp expect actual
> +'
> +
> +done
> +
>  test_done
> -- 
> 2.43.GIT
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 2/7] update-ref: add support for symref-verify

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 9081 bytes --]

Patrick Steinhardt <ps@pks.im> writes:
>> +symref-verify::
>> +	Verify symbolic <ref> against <old-ref> but do not change it.
>> +	If <old-ref> is missing, the ref must not exist.  Can only be
>> +	used in `no-deref` mode.
>
> Should this say "is zero or missing", like the comment for "verify"
> does?
>

We don't allow users to enter OID here, we do convert it to zero OID
internally. But the user input is expected to be old_ref or nothing.

> [snip]
>> @@ -297,11 +321,48 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
>>  		die("verify %s: extra input: %s", refname, next);
>>
>>  	if (ref_transaction_verify(transaction, refname, &old_oid,
>> -				   update_flags, &err))
>> +				   NULL, update_flags, &err))
>> +		die("%s", err.buf);
>> +
>> +	update_flags = default_flags;
>> +	free(refname);
>> +	strbuf_release(&err);
>> +}
>> +
>> +static void parse_cmd_symref_verify(struct ref_transaction *transaction,
>> +                                    const char *next, const char *end)
>> +{
>> +	struct strbuf err = STRBUF_INIT;
>> +	struct object_id old_oid;
>> +	char *refname, *old_ref;
>> +
>> +	if (!(update_flags & REF_NO_DEREF))
>> +		die("symref-verify: cannot operate with deref mode");
>
> This feels quite restrictive to me. Wouldn't it be preferable to simply
> ignore `REF_NO_DEREF` here? It basically means that this command can't
> ever be used in a normal `git update-ref --stdin` session.
>

We do support 'option' with the '--stdin' flag. So technically a user
should be able to do.

   $ git update-ref --stdin
   no-deref
   symref-verify refs/heads/symref refs/heads/master
   update-ref refs/heads/branch 0b3b55ad0e593ead604f80fe3f621239b34cce7e

I guess we could make it implicit, but I thought it's better to keep it
explicit so the user knows that there is no dereferencing taking place
here, eventhough the default option is to dereference.

[snip]
>>
>>  	update->flags = flags;
>>
>> -	if (flags & REF_HAVE_NEW)
>> -		oidcpy(&update->new_oid, new_oid);
>> -	if (flags & REF_HAVE_OLD)
>> -		oidcpy(&update->old_oid, old_oid);
>> +	/*
>> +	 * The ref values are to be considered over the oid values when we're
>> +	 * doing symref operations.
>> +	 */
>
> I feel like this is a statement that should be backed up by a deeper
> explanation of why that is. I'm still wondering here why we cannot
> assert that the old value is an object ID when I want to update it to a
> symref, or alternatively why it would even be possible to have both
> `REF_SYMREF_UPDATE` and a set of other, incompatible fields set. It
> feels like this should be a `BUG()` instead if this is supposedly an
> unsupported configuration rather than silently ignoring it.
>
> In any case, I feel like it would be easier to reason about if this was
> introduced together with the actual user. As far as I can see this code
> shouldn't ever be hit for "verify-symref", right? Currently, the reader
> is forced to figure out what is and isn't related to the new command.
>

I've changed this now to no longer have this condition and also added
'BUG' for cases where both old_{ref,target} and new_{ref,target} exist.

[snip]
>> @@ -2464,8 +2495,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>>  			       struct strbuf *err)
>>  {
>>  	struct strbuf referent = STRBUF_INIT;
>> -	int mustexist = (update->flags & REF_HAVE_OLD) &&
>> -		!is_null_oid(&update->old_oid);
>> +	int mustexist = (update->flags & REF_HAVE_OLD) && !is_null_oid(&update->old_oid);
>
> This change is a no-op, right? If so, let's rather drop it.
>

Yeah, will do.

>>  	int ret = 0;
>>  	struct ref_lock *lock;
>>
>> @@ -2514,6 +2544,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>>  					ret = TRANSACTION_GENERIC_ERROR;
>>  					goto out;
>>  				}
>> +			}
>> +
>> +			/*
>> +			 * For symref verification, we need to check the referent value
>> +			 * rather than the oid. If we're dealing with regular refs or we're
>> +			 * verifying a dereferenced symref, we then check the oid.
>> +			 */
>> +			if (update->flags & REF_SYMREF_UPDATE && update->old_ref) {
>> +				if (check_old_ref(update, referent.buf, err)) {
>> +					ret = TRANSACTION_GENERIC_ERROR;
>> +					goto out;
>> +				}
>>  			} else if (check_old_oid(update, &lock->old_oid, err)) {
>>  				ret = TRANSACTION_GENERIC_ERROR;
>>  				goto out;
>> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
>> index 4c5fe02687..21c6b940d8 100644
>> --- a/refs/refs-internal.h
>> +++ b/refs/refs-internal.h
>> @@ -749,4 +749,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
>>   */
>>  struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
>>
>> +/*
>> + * Helper function to check if the new value is null, this
>> + * takes into consideration that the update could be a regular
>> + * ref or a symbolic ref.
>> + */
>> +int null_new_value(struct ref_update *update);
>
> When adding it to the header we should probably prefix this to avoid
> name collisions. `ref_update_is_null_new_value()` might be a mouth full,
> but feels preferable to me.
>

Makes sense.

>>  #endif /* REFS_REFS_INTERNAL_H */
>> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
>> index 6104471199..7a03922c7b 100644
>> --- a/refs/reftable-backend.c
>> +++ b/refs/reftable-backend.c
>> @@ -938,7 +938,28 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>>  		 * individual refs. But the error messages match what the files
>>  		 * backend returns, which keeps our tests happy.
>>  		 */
>> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>> +		if ((u->flags & REF_HAVE_OLD) &&
>> +		    (u->flags & REF_SYMREF_UPDATE) &&
>> +		    u->old_ref) {
>> +			if   (strcmp(referent.buf, u->old_ref)) {
>
> s/   / /
>
>> +				if (!strcmp(u->old_ref, ""))
>> +					strbuf_addf(err, "cannot lock ref '%s': "
>> +						    "reference already exists",
>> +						    original_update_refname(u));
>> +				else if (!strcmp(referent.buf, ""))
>> +					strbuf_addf(err, "cannot lock ref '%s': "
>> +						    "reference is missing but expected %s",
>> +						    original_update_refname(u),
>> +						    u->old_ref);
>> +				else
>> +					strbuf_addf(err, "cannot lock ref '%s': "
>> +						    "is at %s but expected %s",
>> +						    original_update_refname(u),
>> +						    referent.buf, u->old_ref);
>
> I'd use better-matching error messages here. I know that we talk about
> "cannot lock ref" in the next branch, as well. But the only reason we
> did this is to have the same error messages as the "files" backend.
> Semantically, those errors don't make much sense as the "reftable"
> backend never locks specific refs, but only the complete stack.
>

Fair enough, will change.

>> +				ret = -1;
>> +				goto done;
>> +			}
>> +		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>>  			if (is_null_oid(&u->old_oid))
>>  				strbuf_addf(err, _("cannot lock ref '%s': "
>>  					    "reference already exists"),
>> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
>> index ec3443cc87..d8ffda4096 100755
>> --- a/t/t1400-update-ref.sh
>> +++ b/t/t1400-update-ref.sh
>> @@ -890,17 +890,23 @@ test_expect_success 'stdin update/create/verify combination works' '
>>  '
>>
>>  test_expect_success 'stdin verify succeeds for correct value' '
>> +	test-tool ref-store main for-each-reflog-ent $m >before &&
>>  	git rev-parse $m >expect &&
>>  	echo "verify $m $m" >stdin &&
>>  	git update-ref --stdin <stdin &&
>>  	git rev-parse $m >actual &&
>> -	test_cmp expect actual
>> +	test_cmp expect actual &&
>> +	test-tool ref-store main for-each-reflog-ent $m >after &&
>> +	test_cmp before after
>>  '
>>
>>  test_expect_success 'stdin verify succeeds for missing reference' '
>> +	test-tool ref-store main for-each-reflog-ent $m >before &&
>>  	echo "verify refs/heads/missing $Z" >stdin &&
>>  	git update-ref --stdin <stdin &&
>> -	test_must_fail git rev-parse --verify -q refs/heads/missing
>> +	test_must_fail git rev-parse --verify -q refs/heads/missing &&
>> +	test-tool ref-store main for-each-reflog-ent $m >after &&
>> +	test_cmp before after
>>  '
>
> The updated tests merely assert that the refs didn't change, right?
>

Yes, also that we didn't add anything unexpected to the reflog.


>>  test_expect_success 'stdin verify treats no value as missing' '
>> @@ -1641,4 +1647,74 @@ test_expect_success PIPE 'transaction flushes status updates' '
>>  	test_cmp expected actual
>>  '
>>
>> +create_stdin_buf ()
>> +{
>
> The curly brace should go on the same line as the function name.
>

Right, will change.

>> +	if test "$1" = "-z"
>> +	then
>> +		shift
>> +		printf "$F" "$@" >stdin
>> +	else
>> +		echo "$@" >stdin
>> +	fi
>> +}
>> +
>> +for type in "" "-z"
>> +do
>
> We should probably indent all of the tests to make it easier to see that
> they run in a loop.
>
> Patrick
>

I was a bit confused about this, I saw smaller tests with loops
indented, while some larger ones not indented. I think its better to do
so too, let me do that.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v2 3/7] update-ref: add support for symref-delete

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Similar to the previous commit, add 'symref-delete' to allow deletions
of symbolic refs in a transaction via the 'git-update-ref' command. The
'symref-delete' command can when given with an <old-ref>, deletes the
provided <ref> only when it points to <old-ref>.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |  6 +++++
 builtin/fetch.c                  |  2 +-
 builtin/receive-pack.c           |  3 ++-
 builtin/update-ref.c             | 34 +++++++++++++++++++++++++++-
 refs.c                           | 16 +++++++++----
 refs.h                           |  4 +++-
 refs/files-backend.c             |  2 +-
 refs/reftable-backend.c          |  2 +-
 t/t1400-update-ref.sh            | 39 ++++++++++++++++++++++++++++++++
 9 files changed, 97 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 749aaa7892..ef22a1a2f4 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-delete SP <ref> [SP <old-ref>] LF
 	symref-verify SP <ref> [SP <old-ref>] LF
 	option SP <opt> LF
 	start LF
@@ -87,6 +88,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-delete SP <ref> [NUL <old-ref>] NUL
 	symref-verify SP <ref> [NUL <old-ref>] NUL
 	option SP <opt> NUL
 	start NUL
@@ -119,6 +121,10 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-delete::
+	Delete <ref> after verifying it exists with <old-ref>, if
+	given.
+
 symref-verify::
 	Verify symbolic <ref> against <old-ref> but do not change it.
 	If <old-ref> is missing, the ref must not exist.  Can only be
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 66840b7c5b..d02592efca 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1383,7 +1383,7 @@ static int prune_refs(struct display_state *display_state,
 		if (transaction) {
 			for (ref = stale_refs; ref; ref = ref->next) {
 				result = ref_transaction_delete(transaction, ref->name, NULL, 0,
-								"fetch: prune", &err);
+								NULL, "fetch: prune", &err);
 				if (result)
 					goto cleanup;
 			}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ebea1747cb..6b728baaac 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1576,7 +1576,8 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_delete(transaction,
 					   namespaced_name,
 					   old_oid,
-					   0, "push", &err)) {
+					   0, NULL,
+					   "push", &err)) {
 			rp_error("%s", err.buf);
 			ret = "failed to delete";
 		} else {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 4ae6bdcb12..3be9ae0d00 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -294,7 +294,7 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 
 	if (ref_transaction_delete(transaction, refname,
 				   have_old ? &old_oid : NULL,
-				   update_flags, msg, &err))
+				   update_flags, NULL, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
@@ -302,6 +302,37 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+static void parse_cmd_symref_delete(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *old_ref;
+
+	if (!(update_flags & REF_NO_DEREF))
+                die("symref-delete: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-delete: missing <ref>");
+
+        old_ref = parse_next_refname(&next);
+	if (old_ref && read_ref(old_ref, NULL))
+		die("symref-delete %s: invalid <old-ref>", refname);
+
+	if (*next != line_termination)
+		die("symref-delete %s: extra input: %s", refname, next);
+
+	if (ref_transaction_delete(transaction, refname, NULL,
+				   update_flags | REF_SYMREF_UPDATE,
+				   old_ref, msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(old_ref);
+	strbuf_release(&err);
+}
+
 static void parse_cmd_verify(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -445,6 +476,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
 	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
 	{ "start",         parse_cmd_start,         0, UPDATE_REFS_STARTED },
diff --git a/refs.c b/refs.c
index 124b294c9f..6d98d9652d 100644
--- a/refs.c
+++ b/refs.c
@@ -981,7 +981,7 @@ int refs_delete_ref(struct ref_store *refs, const char *msg,
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, refname, old_oid,
-				   flags, msg, &err) ||
+				   flags, NULL, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ref_transaction_free(transaction);
@@ -1220,6 +1220,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
 		free((void *)transaction->updates[i]->old_ref);
+		free((void *)transaction->updates[i]->new_ref);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1252,6 +1253,8 @@ struct ref_update *ref_transaction_add_update(
 	if (update->flags & REF_SYMREF_UPDATE) {
 		if (old_ref)
 			update->old_ref = xstrdup(old_ref);
+		if (new_ref)
+			update->new_ref = xstrdup(new_ref);
 	} else {
 		if (flags & REF_HAVE_NEW)
 			oidcpy(&update->new_oid, new_oid);
@@ -1317,14 +1320,17 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_ref,
+			   const char *msg,
 			   struct strbuf *err)
 {
-	if (old_oid && is_null_oid(old_oid))
+	if (!(flags & REF_SYMREF_UPDATE) && old_oid &&
+	    is_null_oid(old_oid))
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      NULL, NULL, flags,
+				      NULL, old_ref, flags,
 				      msg, err);
 }
 
@@ -2748,7 +2754,7 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg,
 
 	for_each_string_list_item(item, refnames) {
 		ret = ref_transaction_delete(transaction, item->string,
-					     NULL, flags, msg, &err);
+					     NULL, flags, 0, msg, &err);
 		if (ret) {
 			warning(_("could not delete reference %s: %s"),
 				item->string, err.buf);
diff --git a/refs.h b/refs.h
index a988e672ff..60e6a21a31 100644
--- a/refs.h
+++ b/refs.h
@@ -758,7 +758,9 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_ref,
+			   const char *msg,
 			   struct strbuf *err);
 
 /*
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 8421530bde..f74ea308b5 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2501,7 +2501,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 7a03922c7b..935bf407df 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1122,7 +1122,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index d8ffda4096..cf01c5d867 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1715,6 +1715,45 @@ test_expect_success "stdin ${type} symref-verify fails for mistaken null value"
 	test_cmp expect actual
 '
 
+test_expect_success "stdin ${type} symref-delete fails without --no-deref" '
+	git symbolic-ref refs/heads/symref $a &&
+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+	grep "fatal: symref-delete: cannot operate with deref mode" err
+'
+
+test_expect_success "stdin ${type} fails symref-delete with no ref" '
+	create_stdin_buf ${type} "symref-delete " &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	grep "fatal: symref-delete: missing <ref>" err
+'
+
+test_expect_success "stdin ${type} fails symref-delete with too many arguments" '
+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" "$a" &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	if test "$type" = "-z"
+	then
+		grep "fatal: unknown command: $a" err
+	else
+		grep "fatal: symref-delete refs/heads/symref: extra input:  $a" err
+	fi
+'
+
+test_expect_success "stdin ${type} symref-delete ref fails with wrong old value" '
+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$m" &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	grep "fatal: cannot lock ref '"'"'refs/heads/symref'"'"'" err &&
+	git symbolic-ref refs/heads/symref >expect &&
+	echo $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-delete ref works with right old value" '
+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
 done
 
 test_done
-- 
2.43.GIT

Re: [PATCH v2 3/7] update-ref: add support for symref-delete

[Christian Couder]

On Fri, Apr 12, 2024 at 11:59 AM Karthik Nayak <karthik.188@gmail.com> wrote:
>
> From: Karthik Nayak <karthik.188@gmail.com>
>
> Similar to the previous commit, add 'symref-delete' to allow deletions
> of symbolic refs in a transaction via the 'git-update-ref' command. The
> 'symref-delete' command can when given with an <old-ref>, deletes the
> provided <ref> only when it points to <old-ref>.

I have a similar question as with the previous patch about what
happens if <old-ref> looks like an oid and <ref> is a regular ref
pointing to it.

> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>

> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index 749aaa7892..ef22a1a2f4 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
>         create SP <ref> SP <new-oid> LF
>         delete SP <ref> [SP <old-oid>] LF
>         verify SP <ref> [SP <old-oid>] LF
> +       symref-delete SP <ref> [SP <old-ref>] LF
>         symref-verify SP <ref> [SP <old-ref>] LF
>         option SP <opt> LF
>         start LF
> @@ -87,6 +88,7 @@ quoting:
>         create SP <ref> NUL <new-oid> NUL
>         delete SP <ref> NUL [<old-oid>] NUL
>         verify SP <ref> NUL [<old-oid>] NUL
> +       symref-delete SP <ref> [NUL <old-ref>] NUL
>         symref-verify SP <ref> [NUL <old-ref>] NUL
>         option SP <opt> NUL
>         start NUL

Also I wonder if there is a test where <old-ref> is an empty string, so where:

    symref-delete SP <ref> SP LF

or:

    symref-delete SP <ref> NUL NUL

are used?

Re: [PATCH v2 3/7] update-ref: add support for symref-delete

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

Christian Couder <christian.couder@gmail.com> writes:

> On Fri, Apr 12, 2024 at 11:59 AM Karthik Nayak <karthik.188@gmail.com> wrote:
>>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> Similar to the previous commit, add 'symref-delete' to allow deletions
>> of symbolic refs in a transaction via the 'git-update-ref' command. The
>> 'symref-delete' command can when given with an <old-ref>, deletes the
>> provided <ref> only when it points to <old-ref>.
>
> I have a similar question as with the previous patch about what
> happens if <old-ref> looks like an oid and <ref> is a regular ref
> pointing to it.
>

We parse refs passed as <old-ref> and this would fail.

>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>
>> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
>> index 749aaa7892..ef22a1a2f4 100644
>> --- a/Documentation/git-update-ref.txt
>> +++ b/Documentation/git-update-ref.txt
>> @@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
>>         create SP <ref> SP <new-oid> LF
>>         delete SP <ref> [SP <old-oid>] LF
>>         verify SP <ref> [SP <old-oid>] LF
>> +       symref-delete SP <ref> [SP <old-ref>] LF
>>         symref-verify SP <ref> [SP <old-ref>] LF
>>         option SP <opt> LF
>>         start LF
>> @@ -87,6 +88,7 @@ quoting:
>>         create SP <ref> NUL <new-oid> NUL
>>         delete SP <ref> NUL [<old-oid>] NUL
>>         verify SP <ref> NUL [<old-oid>] NUL
>> +       symref-delete SP <ref> [NUL <old-ref>] NUL
>>         symref-verify SP <ref> [NUL <old-ref>] NUL
>>         option SP <opt> NUL
>>         start NUL
>
> Also I wonder if there is a test where <old-ref> is an empty string, so where:
>
>     symref-delete SP <ref> SP LF
>
> or:
>
>     symref-delete SP <ref> NUL NUL
>
> are used?

I didn't add such tests, will add.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 3/7] update-ref: add support for symref-delete

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3011 bytes --]

On Fri, Apr 12, 2024 at 11:59:04AM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
[snip]
> @@ -302,6 +302,37 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
>  	strbuf_release(&err);
>  }
>  
> +static void parse_cmd_symref_delete(struct ref_transaction *transaction,
> +				    const char *next, const char *end)
> +{
> +	struct strbuf err = STRBUF_INIT;
> +	char *refname, *old_ref;
> +
> +	if (!(update_flags & REF_NO_DEREF))
> +                die("symref-delete: cannot operate with deref mode");

Again, I'm a bit on the fence regarding this restriction. I feel like it
ought to be possible to delete both plain and symbolic refs in a single
git-update-ref(1) command.

> +	refname = parse_refname(&next);
> +	if (!refname)
> +		die("symref-delete: missing <ref>");
> +
> +        old_ref = parse_next_refname(&next);

This line is indented with spaces and not tabs.

[snip]
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -1715,6 +1715,45 @@ test_expect_success "stdin ${type} symref-verify fails for mistaken null value"
>  	test_cmp expect actual
>  '
>  
> +test_expect_success "stdin ${type} symref-delete fails without --no-deref" '
> +	git symbolic-ref refs/heads/symref $a &&
> +	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
> +	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
> +	grep "fatal: symref-delete: cannot operate with deref mode" err
> +'
> +
> +test_expect_success "stdin ${type} fails symref-delete with no ref" '
> +	create_stdin_buf ${type} "symref-delete " &&
> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
> +	grep "fatal: symref-delete: missing <ref>" err
> +'
> +
> +test_expect_success "stdin ${type} fails symref-delete with too many arguments" '
> +	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" "$a" &&
> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
> +	if test "$type" = "-z"
> +	then
> +		grep "fatal: unknown command: $a" err
> +	else
> +		grep "fatal: symref-delete refs/heads/symref: extra input:  $a" err
> +	fi
> +'
> +
> +test_expect_success "stdin ${type} symref-delete ref fails with wrong old value" '
> +	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$m" &&
> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
> +	grep "fatal: cannot lock ref '"'"'refs/heads/symref'"'"'" err &&

You can use "${SQ}" to insert single quotes.

Patrick

> +	git symbolic-ref refs/heads/symref >expect &&
> +	echo $a >actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success "stdin ${type} symref-delete ref works with right old value" '
> +	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
> +	git update-ref --stdin ${type} --no-deref <stdin &&
> +	test_must_fail git rev-parse --verify -q $b
> +'
> +
>  done
>  
>  test_done
> -- 
> 2.43.GIT
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 3/7] update-ref: add support for symref-delete

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2772 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Fri, Apr 12, 2024 at 11:59:04AM +0200, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
> [snip]
>> @@ -302,6 +302,37 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
>>  	strbuf_release(&err);
>>  }
>>
>> +static void parse_cmd_symref_delete(struct ref_transaction *transaction,
>> +				    const char *next, const char *end)
>> +{
>> +	struct strbuf err = STRBUF_INIT;
>> +	char *refname, *old_ref;
>> +
>> +	if (!(update_flags & REF_NO_DEREF))
>> +                die("symref-delete: cannot operate with deref mode");
>
> Again, I'm a bit on the fence regarding this restriction. I feel like it
> ought to be possible to delete both plain and symbolic refs in a single
> git-update-ref(1) command.
>

Yup this is still possible since we have the 'no-deref' option.

>> +	refname = parse_refname(&next);
>> +	if (!refname)
>> +		die("symref-delete: missing <ref>");
>> +
>> +        old_ref = parse_next_refname(&next);
>
> This line is indented with spaces and not tabs.
>

There was a bunch of this, I'll have them all fixed.

> [snip]
>> --- a/t/t1400-update-ref.sh
>> +++ b/t/t1400-update-ref.sh
>> @@ -1715,6 +1715,45 @@ test_expect_success "stdin ${type} symref-verify fails for mistaken null value"
>>  	test_cmp expect actual
>>  '
>>
>> +test_expect_success "stdin ${type} symref-delete fails without --no-deref" '
>> +	git symbolic-ref refs/heads/symref $a &&
>> +	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
>> +	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
>> +	grep "fatal: symref-delete: cannot operate with deref mode" err
>> +'
>> +
>> +test_expect_success "stdin ${type} fails symref-delete with no ref" '
>> +	create_stdin_buf ${type} "symref-delete " &&
>> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
>> +	grep "fatal: symref-delete: missing <ref>" err
>> +'
>> +
>> +test_expect_success "stdin ${type} fails symref-delete with too many arguments" '
>> +	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" "$a" &&
>> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
>> +	if test "$type" = "-z"
>> +	then
>> +		grep "fatal: unknown command: $a" err
>> +	else
>> +		grep "fatal: symref-delete refs/heads/symref: extra input:  $a" err
>> +	fi
>> +'
>> +
>> +test_expect_success "stdin ${type} symref-delete ref fails with wrong old value" '
>> +	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$m" &&
>> +	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
>> +	grep "fatal: cannot lock ref '"'"'refs/heads/symref'"'"'" err &&
>
> You can use "${SQ}" to insert single quotes.
>
> Patrick
>

Neat, this is much better, thanks!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v2 4/7] files-backend: extract out `create_symref_lock`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Split this into two individual functions `create_and_commit_symref` and
`create_symref_locked`. This way we can create the symref lock and
commit it at different times. This will be used to provide symref
support in `git-update-ref(1)`.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 40 +++++++++++++++++++++++++++-------------
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index f74ea308b5..7c894ebe65 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1920,26 +1920,39 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target)
 {
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	/* no error check; commit_ref will check ferror */
+	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
 	return 0;
 }
 
@@ -1960,7 +1973,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

[PATCH v2 5/7] update-ref: add support for symref-create

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Add 'symref-create' to allow creation of symbolic refs in a transaction
via the 'git-update-ref' command. The 'symref-create' command takes in a
<new-ref>, which the created <ref> will point to.

We also support the 'core.prefersymlinkrefs', wherein if the flag is set
and the filesystem supports symlinks, we create the symbolic ref as a
symlink.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |  6 ++++
 builtin/clone.c                  |  2 +-
 builtin/update-ref.c             | 36 +++++++++++++++++++-
 refs.c                           |  9 +++--
 refs.h                           |  1 +
 refs/files-backend.c             | 38 +++++++++++++++++++++
 refs/reftable-backend.c          | 19 +++++++++--
 t/t0600-reffiles-backend.sh      | 32 ++++++++++++++++++
 t/t1400-update-ref.sh            | 58 ++++++++++++++++++++++++++++++++
 9 files changed, 194 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index ef22a1a2f4..a5b1f42728 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-create SP <ref> SP <new-ref> LF
 	symref-delete SP <ref> [SP <old-ref>] LF
 	symref-verify SP <ref> [SP <old-ref>] LF
 	option SP <opt> LF
@@ -88,6 +89,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-create SP <ref> NUL <new-ref> NUL
 	symref-delete SP <ref> [NUL <old-ref>] NUL
 	symref-verify SP <ref> [NUL <old-ref>] NUL
 	option SP <opt> NUL
@@ -121,6 +123,10 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-create::
+	Create symbolic ref <ref> with <new-ref> after verifying
+	it does not exist.  Can only be used in `no-deref` mode.
+
 symref-delete::
 	Delete <ref> after verifying it exists with <old-ref>, if
 	given.
diff --git a/builtin/clone.c b/builtin/clone.c
index 74ec14542e..c0eed8e795 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -547,7 +547,7 @@ static void write_remote_refs(const struct ref *local_refs)
 		if (!r->peer_ref)
 			continue;
 		if (ref_transaction_create(t, r->peer_ref->name, &r->old_oid,
-					   0, NULL, &err))
+					   NULL, 0, NULL, &err))
 			die("%s", err.buf);
 	}
 
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 3be9ae0d00..24556a28a8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -258,7 +258,7 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	if (*next != line_termination)
 		die("create %s: extra input: %s", refname, next);
 
-	if (ref_transaction_create(transaction, refname, &new_oid,
+	if (ref_transaction_create(transaction, refname, &new_oid, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
@@ -268,6 +268,39 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+static void parse_cmd_symref_create(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *new_ref;
+
+	if (!(update_flags & REF_NO_DEREF))
+                die("symref-create: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-create: missing <ref>");
+
+	new_ref = parse_next_refname(&next);
+	if (!new_ref)
+		die("symref-create %s: missing <new-ref>", refname);
+	if (read_ref(new_ref, NULL))
+		die("symref-create %s: invalid <new-ref>", refname);
+
+	if (*next != line_termination)
+		die("symref-create %s: extra input: %s", refname, next);
+
+	if (ref_transaction_create(transaction, refname, NULL, new_ref,
+				   update_flags | create_reflog_flag |
+				   REF_SYMREF_UPDATE, msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(new_ref);
+	strbuf_release(&err);
+}
+
 static void parse_cmd_delete(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -476,6 +509,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
 	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
 	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
diff --git a/refs.c b/refs.c
index 6d98d9652d..e62c0f4aca 100644
--- a/refs.c
+++ b/refs.c
@@ -1305,15 +1305,20 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_ref,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
-	if (!new_oid || is_null_oid(new_oid)) {
+	if ((flags & REF_SYMREF_UPDATE) && !new_ref) {
+		strbuf_addf(err, "'%s' has a no new ref", refname);
+		return 1;
+	}
+	if (!(flags & REF_SYMREF_UPDATE) && (!new_oid || is_null_oid(new_oid))) {
 		strbuf_addf(err, "'%s' has a null OID", refname);
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), NULL, NULL, flags,
+				      null_oid(), new_ref, NULL, flags,
 				      msg, err);
 }
 
diff --git a/refs.h b/refs.h
index 60e6a21a31..c01a517e40 100644
--- a/refs.h
+++ b/refs.h
@@ -744,6 +744,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_ref,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7c894ebe65..59d438878a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2609,6 +2609,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
+	if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
+		if (create_symref_lock(refs, lock, update->refname, update->new_ref)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		update->flags |= REF_NEEDS_COMMIT;
+	}
+
+
 	if ((update->flags & REF_HAVE_NEW) &&
 	    !(update->flags & REF_DELETING) &&
 	    !(update->flags & REF_LOG_ONLY)) {
@@ -2904,6 +2925,14 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
+			if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
+				/* for dangling symrefs we gracefully set the oid to zero */
+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_ref,
+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
+					update->new_oid = *null_oid();
+				}
+			}
+
 			if (files_log_ref_write(refs,
 						lock->ref_name,
 						&lock->old_oid,
@@ -2921,6 +2950,15 @@ static int files_transaction_finish(struct ref_store *ref_store,
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next updated. If not, we try and create a regular symref.
+		 */
+		if (update->flags & REF_SYMREF_UPDATE && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->new_ref))
+				continue;
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 935bf407df..6d42838e15 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -856,7 +856,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if (u->flags & REF_HAVE_NEW && !null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -1064,7 +1064,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1122,7 +1122,20 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && null_new_value(u)) {
+		if (u->flags & REF_SYMREF_UPDATE &&
+		    u->flags & REF_HAVE_NEW &&
+		    !null_new_value(u)) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_ref,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if (u->flags & REF_HAVE_NEW && null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh
index 64214340e7..c5061c26cf 100755
--- a/t/t0600-reffiles-backend.sh
+++ b/t/t0600-reffiles-backend.sh
@@ -472,4 +472,36 @@ test_expect_success POSIXPERM 'git reflog expire honors core.sharedRepository' '
 	esac
 '
 
+test_expect_success SYMLINKS 'symref transaction supports symlinks' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs true &&
+	cat >stdin <<-EOF &&
+	start
+	symref-create TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_symlink .git/TESTSYMREFONE &&
+	test "$(test_readlink .git/TESTSYMREFONE)" = refs/heads/new
+'
+
+test_expect_success 'symref transaction supports false symlink config' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs false &&
+	cat >stdin <<-EOF &&
+	start
+	symref-create TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_file .git/TESTSYMREFONE &&
+	git symbolic-ref TESTSYMREFONE >actual &&
+	echo refs/heads/new >expect &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index cf01c5d867..f4e63fae6e 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1754,6 +1754,64 @@ test_expect_success "stdin ${type} symref-delete ref works with right old value"
 	test_must_fail git rev-parse --verify -q $b
 '
 
+test_expect_success "stdin ${type} symref-create fails without --no-deref" '
+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" &&
+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+	grep "fatal: symref-create: cannot operate with deref mode" err
+'
+
+test_expect_success "stdin ${type} fails symref-create with no ref" '
+	create_stdin_buf ${type} "symref-create " >stdin &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	grep "fatal: symref-create: missing <ref>" err
+'
+
+test_expect_success "stdin ${type} fails symref-create with no new value" '
+	create_stdin_buf ${type} "symref-create refs/heads/symref" >stdin &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	grep "fatal: symref-create refs/heads/symref: missing <new-ref>" err
+'
+
+test_expect_success "stdin ${type} fails symref-create with too many arguments" '
+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" "$a" >stdin &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	if test "$type" = "-z"
+	then
+		grep "fatal: unknown command: $a" err
+	else
+		grep "fatal: symref-create refs/heads/symref: extra input:  $a" err
+	fi
+'
+
+test_expect_success "stdin ${type} symref-create ref works" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	git symbolic-ref refs/heads/symref >expect &&
+	echo $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-create does not create reflogs by default" '
+	test_when_finished "git symbolic-ref -d refs/symref" &&
+	create_stdin_buf ${type} "symref-create refs/symref" "$a" >stdin &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	git symbolic-ref refs/symref >expect &&
+	echo $a >actual &&
+	test_cmp expect actual &&
+	test_must_fail git reflog exists refs/symref
+'
+
+test_expect_success "stdin ${type} symref-create reflogs with --create-reflog" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
+	git update-ref --create-reflog --stdin ${type} --no-deref <stdin &&
+	git symbolic-ref refs/heads/symref >expect &&
+	echo $a >actual &&
+	test_cmp expect actual &&
+	git reflog exists refs/heads/symref
+'
+
 done
 
 test_done
-- 
2.43.GIT

Re: [PATCH v2 5/7] update-ref: add support for symref-create

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 5343 bytes --]

On Fri, Apr 12, 2024 at 11:59:06AM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
[snip]
> @@ -268,6 +268,39 @@ static void parse_cmd_create(struct ref_transaction *transaction,
>  	strbuf_release(&err);
>  }
>  
> +static void parse_cmd_symref_create(struct ref_transaction *transaction,
> +				    const char *next, const char *end)
> +{
> +	struct strbuf err = STRBUF_INIT;
> +	char *refname, *new_ref;
> +
> +	if (!(update_flags & REF_NO_DEREF))
> +                die("symref-create: cannot operate with deref mode");
> +
> +	refname = parse_refname(&next);
> +	if (!refname)
> +		die("symref-create: missing <ref>");
> +
> +	new_ref = parse_next_refname(&next);
> +	if (!new_ref)
> +		die("symref-create %s: missing <new-ref>", refname);
> +	if (read_ref(new_ref, NULL))
> +		die("symref-create %s: invalid <new-ref>", refname);

This restricts the creation of dangling symrefs, right? I think we might
want to lift this restriction, in which case we can eventually get rid
of the `create_symref` callback in ref backends completely.

> +	if (*next != line_termination)
> +		die("symref-create %s: extra input: %s", refname, next);
> +
> +	if (ref_transaction_create(transaction, refname, NULL, new_ref,
> +				   update_flags | create_reflog_flag |
> +				   REF_SYMREF_UPDATE, msg, &err))
> +		die("%s", err.buf);
> +
> +	update_flags = default_flags;
> +	free(refname);
> +	free(new_ref);
> +	strbuf_release(&err);
> +}
> +
>  static void parse_cmd_delete(struct ref_transaction *transaction,
>  			     const char *next, const char *end)
>  {
> @@ -476,6 +509,7 @@ static const struct parse_cmd {
>  	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
>  	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
>  	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
> +	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
>  	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
>  	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
>  	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
> diff --git a/refs.c b/refs.c
> index 6d98d9652d..e62c0f4aca 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1305,15 +1305,20 @@ int ref_transaction_update(struct ref_transaction *transaction,
>  int ref_transaction_create(struct ref_transaction *transaction,
>  			   const char *refname,
>  			   const struct object_id *new_oid,
> +			   const char *new_ref,
>  			   unsigned int flags, const char *msg,
>  			   struct strbuf *err)
>  {
> -	if (!new_oid || is_null_oid(new_oid)) {
> +	if ((flags & REF_SYMREF_UPDATE) && !new_ref) {
> +		strbuf_addf(err, "'%s' has a no new ref", refname);
> +		return 1;
> +	}
> +	if (!(flags & REF_SYMREF_UPDATE) && (!new_oid || is_null_oid(new_oid))) {
>  		strbuf_addf(err, "'%s' has a null OID", refname);
>  		return 1;
>  	}
>  	return ref_transaction_update(transaction, refname, new_oid,
> -				      null_oid(), NULL, NULL, flags,
> +				      null_oid(), new_ref, NULL, flags,
>  				      msg, err);
>  }
>  
> diff --git a/refs.h b/refs.h
> index 60e6a21a31..c01a517e40 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -744,6 +744,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>  int ref_transaction_create(struct ref_transaction *transaction,
>  			   const char *refname,
>  			   const struct object_id *new_oid,
> +			   const char *new_ref,
>  			   unsigned int flags, const char *msg,
>  			   struct strbuf *err);
>  
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 7c894ebe65..59d438878a 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2609,6 +2609,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>  		}
>  	}
>  
> +	if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {

Let's add braces around `(updaet->flags & REF_SYMREF_UPDATE)` to make
this easier to read.

> +		if (create_symref_lock(refs, lock, update->refname, update->new_ref)) {
> +			ret = TRANSACTION_GENERIC_ERROR;
> +			goto out;
> +		}
> +
> +		if (close_ref_gently(lock)) {
> +			strbuf_addf(err, "couldn't close '%s.lock'",
> +				    update->refname);
> +			ret = TRANSACTION_GENERIC_ERROR;
> +			goto out;
> +		}
> +
> +		/*
> +		 * Once we have created the symref lock, the commit
> +		 * phase of the transaction only needs to commit the lock.
> +		 */
> +		update->flags |= REF_NEEDS_COMMIT;
> +	}
> +
> +
>  	if ((update->flags & REF_HAVE_NEW) &&
>  	    !(update->flags & REF_DELETING) &&
>  	    !(update->flags & REF_LOG_ONLY)) {
> @@ -2904,6 +2925,14 @@ static int files_transaction_finish(struct ref_store *ref_store,
>  
>  		if (update->flags & REF_NEEDS_COMMIT ||
>  		    update->flags & REF_LOG_ONLY) {
> +			if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {

Here, as well.

> +				/* for dangling symrefs we gracefully set the oid to zero */
> +				if (!refs_resolve_ref_unsafe(&refs->base, update->new_ref,
> +							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
> +					update->new_oid = *null_oid();
> +				}

Can this actually happenn right now? I thought that the `read_ref()`
further up forbids this case.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 5/7] update-ref: add support for symref-create

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

>> +	if (!new_ref)
>> +		die("symref-create %s: missing <new-ref>", refname);
>> +	if (read_ref(new_ref, NULL))
>> +		die("symref-create %s: invalid <new-ref>", refname);
>
> This restricts the creation of dangling symrefs, right? I think we might
> want to lift this restriction, in which case we can eventually get rid
> of the `create_symref` callback in ref backends completely.

True.

>> @@ -2609,6 +2609,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>>  		}
>>  	}
>>  
>> +	if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
>
> Let's add braces around `(updaet->flags & REF_SYMREF_UPDATE)` to make
> this easier to read.

Yup, the language may not require the extra pair of parentheses, but
human eyes are helped by them.

Thanks, again and as always, for carefully reading.

Re: [PATCH v2 5/7] update-ref: add support for symref-create

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 5924 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Fri, Apr 12, 2024 at 11:59:06AM +0200, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
> [snip]
>> @@ -268,6 +268,39 @@ static void parse_cmd_create(struct ref_transaction *transaction,
>>  	strbuf_release(&err);
>>  }
>>
>> +static void parse_cmd_symref_create(struct ref_transaction *transaction,
>> +				    const char *next, const char *end)
>> +{
>> +	struct strbuf err = STRBUF_INIT;
>> +	char *refname, *new_ref;
>> +
>> +	if (!(update_flags & REF_NO_DEREF))
>> +                die("symref-create: cannot operate with deref mode");
>> +
>> +	refname = parse_refname(&next);
>> +	if (!refname)
>> +		die("symref-create: missing <ref>");
>> +
>> +	new_ref = parse_next_refname(&next);
>> +	if (!new_ref)
>> +		die("symref-create %s: missing <new-ref>", refname);
>> +	if (read_ref(new_ref, NULL))
>> +		die("symref-create %s: invalid <new-ref>", refname);
>
> This restricts the creation of dangling symrefs, right? I think we might
> want to lift this restriction, in which case we can eventually get rid
> of the `create_symref` callback in ref backends completely.
>

Yes it does. I thought it'd be more consistent with what update-ref does
with regular ref updates. We verify that the object exists. Also this
could be an added option 'allow-dangling'.

I'm not sure I understand what you mean 'the `create_symref` callback in
ref backends completely.'.

>> +	if (*next != line_termination)
>> +		die("symref-create %s: extra input: %s", refname, next);
>> +
>> +	if (ref_transaction_create(transaction, refname, NULL, new_ref,
>> +				   update_flags | create_reflog_flag |
>> +				   REF_SYMREF_UPDATE, msg, &err))
>> +		die("%s", err.buf);
>> +
>> +	update_flags = default_flags;
>> +	free(refname);
>> +	free(new_ref);
>> +	strbuf_release(&err);
>> +}
>> +
>>  static void parse_cmd_delete(struct ref_transaction *transaction,
>>  			     const char *next, const char *end)
>>  {
>> @@ -476,6 +509,7 @@ static const struct parse_cmd {
>>  	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
>>  	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
>>  	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
>> +	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
>>  	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
>>  	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
>>  	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
>> diff --git a/refs.c b/refs.c
>> index 6d98d9652d..e62c0f4aca 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1305,15 +1305,20 @@ int ref_transaction_update(struct ref_transaction *transaction,
>>  int ref_transaction_create(struct ref_transaction *transaction,
>>  			   const char *refname,
>>  			   const struct object_id *new_oid,
>> +			   const char *new_ref,
>>  			   unsigned int flags, const char *msg,
>>  			   struct strbuf *err)
>>  {
>> -	if (!new_oid || is_null_oid(new_oid)) {
>> +	if ((flags & REF_SYMREF_UPDATE) && !new_ref) {
>> +		strbuf_addf(err, "'%s' has a no new ref", refname);
>> +		return 1;
>> +	}
>> +	if (!(flags & REF_SYMREF_UPDATE) && (!new_oid || is_null_oid(new_oid))) {
>>  		strbuf_addf(err, "'%s' has a null OID", refname);
>>  		return 1;
>>  	}
>>  	return ref_transaction_update(transaction, refname, new_oid,
>> -				      null_oid(), NULL, NULL, flags,
>> +				      null_oid(), new_ref, NULL, flags,
>>  				      msg, err);
>>  }
>>
>> diff --git a/refs.h b/refs.h
>> index 60e6a21a31..c01a517e40 100644
>> --- a/refs.h
>> +++ b/refs.h
>> @@ -744,6 +744,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>>  int ref_transaction_create(struct ref_transaction *transaction,
>>  			   const char *refname,
>>  			   const struct object_id *new_oid,
>> +			   const char *new_ref,
>>  			   unsigned int flags, const char *msg,
>>  			   struct strbuf *err);
>>
>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index 7c894ebe65..59d438878a 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -2609,6 +2609,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>>  		}
>>  	}
>>
>> +	if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
>
> Let's add braces around `(updaet->flags & REF_SYMREF_UPDATE)` to make
> this easier to read.
>

This should now be cleaned up as we removed the flag entirely.

>> +		if (create_symref_lock(refs, lock, update->refname, update->new_ref)) {
>> +			ret = TRANSACTION_GENERIC_ERROR;
>> +			goto out;
>> +		}
>> +
>> +		if (close_ref_gently(lock)) {
>> +			strbuf_addf(err, "couldn't close '%s.lock'",
>> +				    update->refname);
>> +			ret = TRANSACTION_GENERIC_ERROR;
>> +			goto out;
>> +		}
>> +
>> +		/*
>> +		 * Once we have created the symref lock, the commit
>> +		 * phase of the transaction only needs to commit the lock.
>> +		 */
>> +		update->flags |= REF_NEEDS_COMMIT;
>> +	}
>> +
>> +
>>  	if ((update->flags & REF_HAVE_NEW) &&
>>  	    !(update->flags & REF_DELETING) &&
>>  	    !(update->flags & REF_LOG_ONLY)) {
>> @@ -2904,6 +2925,14 @@ static int files_transaction_finish(struct ref_store *ref_store,
>>
>>  		if (update->flags & REF_NEEDS_COMMIT ||
>>  		    update->flags & REF_LOG_ONLY) {
>> +			if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
>
> Here, as well.
>

Same here.

>> +				/* for dangling symrefs we gracefully set the oid to zero */
>> +				if (!refs_resolve_ref_unsafe(&refs->base, update->new_ref,
>> +							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
>> +					update->new_oid = *null_oid();
>> +				}
>
> Can this actually happenn right now? I thought that the `read_ref()`
> further up forbids this case.
>
> Patrick

With update-ref, it won't happen anymore, because as you mentioned, we
use `read_ref()`. I thought it was still worthwhile to have. But I guess
its cleaner to remove this.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 5/7] update-ref: add support for symref-create

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]

Karthik Nayak <karthik.188@gmail.

[snip]
>>> +				/* for dangling symrefs we gracefully set the oid to zero */
>>> +				if (!refs_resolve_ref_unsafe(&refs->base, update->new_ref,
>>> +							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
>>> +					update->new_oid = *null_oid();
>>> +				}
>>
>> Can this actually happenn right now? I thought that the `read_ref()`
>> further up forbids this case.
>>
>> Patrick
>
> With update-ref, it won't happen anymore, because as you mentioned, we
> use `read_ref()`. I thought it was still worthwhile to have. But I guess
> its cleaner to remove this.

Well, just realized that this needs to stay because we also want the
resolved OID for the reflogs. Will modify the comment though.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 5/7] update-ref: add support for symref-create

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 7462 bytes --]

On Sun, Apr 21, 2024 at 08:50:07AM -0400, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > On Fri, Apr 12, 2024 at 11:59:06AM +0200, Karthik Nayak wrote:
> >> From: Karthik Nayak <karthik.188@gmail.com>
> > [snip]
> >> @@ -268,6 +268,39 @@ static void parse_cmd_create(struct ref_transaction *transaction,
> >>  	strbuf_release(&err);
> >>  }
> >>
> >> +static void parse_cmd_symref_create(struct ref_transaction *transaction,
> >> +				    const char *next, const char *end)
> >> +{
> >> +	struct strbuf err = STRBUF_INIT;
> >> +	char *refname, *new_ref;
> >> +
> >> +	if (!(update_flags & REF_NO_DEREF))
> >> +                die("symref-create: cannot operate with deref mode");
> >> +
> >> +	refname = parse_refname(&next);
> >> +	if (!refname)
> >> +		die("symref-create: missing <ref>");
> >> +
> >> +	new_ref = parse_next_refname(&next);
> >> +	if (!new_ref)
> >> +		die("symref-create %s: missing <new-ref>", refname);
> >> +	if (read_ref(new_ref, NULL))
> >> +		die("symref-create %s: invalid <new-ref>", refname);
> >
> > This restricts the creation of dangling symrefs, right? I think we might
> > want to lift this restriction, in which case we can eventually get rid
> > of the `create_symref` callback in ref backends completely.
> >
> 
> Yes it does. I thought it'd be more consistent with what update-ref does
> with regular ref updates. We verify that the object exists. Also this
> could be an added option 'allow-dangling'.
> 
> I'm not sure I understand what you mean 'the `create_symref` callback in
> ref backends completely.'.

Well, once normal reference transactions learn to update symrefs we can
do the following:

    diff --git a/refs/refs-internal.h b/refs/refs-internal.h
    index 56641aa57a..2302311282 100644
    --- a/refs/refs-internal.h
    +++ b/refs/refs-internal.h
    @@ -676,7 +676,6 @@ struct ref_storage_be {
        ref_transaction_commit_fn *initial_transaction_commit;
     
        pack_refs_fn *pack_refs;
    -	create_symref_fn *create_symref;
        rename_ref_fn *rename_ref;
        copy_ref_fn *copy_ref;

There would be no need to specifically have this as a separate callback
anymore as we can now provide a generic wrapper `refs_create_symref()`
(in pseudo-code):

```
int refs_create_symref(struct ref_store *refs, const char *refname,
                       const char *target)
{
    tx = ref_store_transaction_begin(refs);
    ref_transaction_create_symref(tx, refname, target);
    ref_transaction_commit(tx);
}
```

Patrick

> >> +	if (*next != line_termination)
> >> +		die("symref-create %s: extra input: %s", refname, next);
> >> +
> >> +	if (ref_transaction_create(transaction, refname, NULL, new_ref,
> >> +				   update_flags | create_reflog_flag |
> >> +				   REF_SYMREF_UPDATE, msg, &err))
> >> +		die("%s", err.buf);
> >> +
> >> +	update_flags = default_flags;
> >> +	free(refname);
> >> +	free(new_ref);
> >> +	strbuf_release(&err);
> >> +}
> >> +
> >>  static void parse_cmd_delete(struct ref_transaction *transaction,
> >>  			     const char *next, const char *end)
> >>  {
> >> @@ -476,6 +509,7 @@ static const struct parse_cmd {
> >>  	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
> >>  	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
> >>  	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
> >> +	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
> >>  	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
> >>  	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
> >>  	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
> >> diff --git a/refs.c b/refs.c
> >> index 6d98d9652d..e62c0f4aca 100644
> >> --- a/refs.c
> >> +++ b/refs.c
> >> @@ -1305,15 +1305,20 @@ int ref_transaction_update(struct ref_transaction *transaction,
> >>  int ref_transaction_create(struct ref_transaction *transaction,
> >>  			   const char *refname,
> >>  			   const struct object_id *new_oid,
> >> +			   const char *new_ref,
> >>  			   unsigned int flags, const char *msg,
> >>  			   struct strbuf *err)
> >>  {
> >> -	if (!new_oid || is_null_oid(new_oid)) {
> >> +	if ((flags & REF_SYMREF_UPDATE) && !new_ref) {
> >> +		strbuf_addf(err, "'%s' has a no new ref", refname);
> >> +		return 1;
> >> +	}
> >> +	if (!(flags & REF_SYMREF_UPDATE) && (!new_oid || is_null_oid(new_oid))) {
> >>  		strbuf_addf(err, "'%s' has a null OID", refname);
> >>  		return 1;
> >>  	}
> >>  	return ref_transaction_update(transaction, refname, new_oid,
> >> -				      null_oid(), NULL, NULL, flags,
> >> +				      null_oid(), new_ref, NULL, flags,
> >>  				      msg, err);
> >>  }
> >>
> >> diff --git a/refs.h b/refs.h
> >> index 60e6a21a31..c01a517e40 100644
> >> --- a/refs.h
> >> +++ b/refs.h
> >> @@ -744,6 +744,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
> >>  int ref_transaction_create(struct ref_transaction *transaction,
> >>  			   const char *refname,
> >>  			   const struct object_id *new_oid,
> >> +			   const char *new_ref,
> >>  			   unsigned int flags, const char *msg,
> >>  			   struct strbuf *err);
> >>
> >> diff --git a/refs/files-backend.c b/refs/files-backend.c
> >> index 7c894ebe65..59d438878a 100644
> >> --- a/refs/files-backend.c
> >> +++ b/refs/files-backend.c
> >> @@ -2609,6 +2609,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
> >>  		}
> >>  	}
> >>
> >> +	if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
> >
> > Let's add braces around `(updaet->flags & REF_SYMREF_UPDATE)` to make
> > this easier to read.
> >
> 
> This should now be cleaned up as we removed the flag entirely.
> 
> >> +		if (create_symref_lock(refs, lock, update->refname, update->new_ref)) {
> >> +			ret = TRANSACTION_GENERIC_ERROR;
> >> +			goto out;
> >> +		}
> >> +
> >> +		if (close_ref_gently(lock)) {
> >> +			strbuf_addf(err, "couldn't close '%s.lock'",
> >> +				    update->refname);
> >> +			ret = TRANSACTION_GENERIC_ERROR;
> >> +			goto out;
> >> +		}
> >> +
> >> +		/*
> >> +		 * Once we have created the symref lock, the commit
> >> +		 * phase of the transaction only needs to commit the lock.
> >> +		 */
> >> +		update->flags |= REF_NEEDS_COMMIT;
> >> +	}
> >> +
> >> +
> >>  	if ((update->flags & REF_HAVE_NEW) &&
> >>  	    !(update->flags & REF_DELETING) &&
> >>  	    !(update->flags & REF_LOG_ONLY)) {
> >> @@ -2904,6 +2925,14 @@ static int files_transaction_finish(struct ref_store *ref_store,
> >>
> >>  		if (update->flags & REF_NEEDS_COMMIT ||
> >>  		    update->flags & REF_LOG_ONLY) {
> >> +			if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
> >
> > Here, as well.
> >
> 
> Same here.
> 
> >> +				/* for dangling symrefs we gracefully set the oid to zero */
> >> +				if (!refs_resolve_ref_unsafe(&refs->base, update->new_ref,
> >> +							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
> >> +					update->new_oid = *null_oid();
> >> +				}
> >
> > Can this actually happenn right now? I thought that the `read_ref()`
> > further up forbids this case.
> >
> > Patrick
> 
> With update-ref, it won't happen anymore, because as you mentioned, we
> use `read_ref()`. I thought it was still worthwhile to have. But I guess
> its cleaner to remove this.



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 5/7] update-ref: add support for symref-create

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2759 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Sun, Apr 21, 2024 at 08:50:07AM -0400, Karthik Nayak wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>>
>> > On Fri, Apr 12, 2024 at 11:59:06AM +0200, Karthik Nayak wrote:
>> >> From: Karthik Nayak <karthik.188@gmail.com>
>> > [snip]
>> >> @@ -268,6 +268,39 @@ static void parse_cmd_create(struct ref_transaction *transaction,
>> >>  	strbuf_release(&err);
>> >>  }
>> >>
>> >> +static void parse_cmd_symref_create(struct ref_transaction *transaction,
>> >> +				    const char *next, const char *end)
>> >> +{
>> >> +	struct strbuf err = STRBUF_INIT;
>> >> +	char *refname, *new_ref;
>> >> +
>> >> +	if (!(update_flags & REF_NO_DEREF))
>> >> +                die("symref-create: cannot operate with deref mode");
>> >> +
>> >> +	refname = parse_refname(&next);
>> >> +	if (!refname)
>> >> +		die("symref-create: missing <ref>");
>> >> +
>> >> +	new_ref = parse_next_refname(&next);
>> >> +	if (!new_ref)
>> >> +		die("symref-create %s: missing <new-ref>", refname);
>> >> +	if (read_ref(new_ref, NULL))
>> >> +		die("symref-create %s: invalid <new-ref>", refname);
>> >
>> > This restricts the creation of dangling symrefs, right? I think we might
>> > want to lift this restriction, in which case we can eventually get rid
>> > of the `create_symref` callback in ref backends completely.
>> >
>>
>> Yes it does. I thought it'd be more consistent with what update-ref does
>> with regular ref updates. We verify that the object exists. Also this
>> could be an added option 'allow-dangling'.
>>
>> I'm not sure I understand what you mean 'the `create_symref` callback in
>> ref backends completely.'.
>
> Well, once normal reference transactions learn to update symrefs we can
> do the following:
>
>     diff --git a/refs/refs-internal.h b/refs/refs-internal.h
>     index 56641aa57a..2302311282 100644
>     --- a/refs/refs-internal.h
>     +++ b/refs/refs-internal.h
>     @@ -676,7 +676,6 @@ struct ref_storage_be {
>         ref_transaction_commit_fn *initial_transaction_commit;
>
>         pack_refs_fn *pack_refs;
>     -	create_symref_fn *create_symref;
>         rename_ref_fn *rename_ref;
>         copy_ref_fn *copy_ref;
>
> There would be no need to specifically have this as a separate callback
> anymore as we can now provide a generic wrapper `refs_create_symref()`
> (in pseudo-code):
>
> ```
> int refs_create_symref(struct ref_store *refs, const char *refname,
>                        const char *target)
> {
>     tx = ref_store_transaction_begin(refs);
>     ref_transaction_create_symref(tx, refname, target);
>     ref_transaction_commit(tx);
> }
> ```
>
> Patrick
>

Indeed, this is pretty nice side-effect. Thanks for explaining, I'll
remove the explicit check and allow dangling refs.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v2 6/7] update-ref: add support for symref-update

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Add 'symref-update' to allow updates of symbolic refs in a transaction
via the 'git-update-ref' command. The 'symref-update' command takes in a
<new-ref>, which the <ref> will be updated to. If the <ref> doesn't
exist it will be created.

It also optionally takes either an <old-ref> or <old-oid>. If the
<old-ref> is provided, it checks to see if the <ref> ponints to the
<old-ref> before the update. If <old-oid> is provided it checks <ref> to
ensure that it is a regular ref and <old-oid> is the OID before the
update. This by extension also means that this when a zero <old-oid> is
provided, it ensures that the ref didn't exist before.

This command will also support deref mode, to ensure that we can update
dereferenced regular refs to symrefs.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |   6 ++
 builtin/update-ref.c             |  49 +++++++++++
 refs.c                           |  24 ++----
 refs/files-backend.c             |  15 ++--
 refs/reftable-backend.c          |   7 +-
 t/t1400-update-ref.sh            | 143 +++++++++++++++++++++++++++++++
 6 files changed, 220 insertions(+), 24 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index a5b1f42728..9710c9bc78 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-update SP <ref> SP <new-ref> [SP (<old-ref> | <old-oid>)] LF
 	symref-create SP <ref> SP <new-ref> LF
 	symref-delete SP <ref> [SP <old-ref>] LF
 	symref-verify SP <ref> [SP <old-ref>] LF
@@ -89,6 +90,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-update SP <ref> NUL <new-ref> [NUL (<old-ref> | <old-oid>)] NUL
 	symref-create SP <ref> NUL <new-ref> NUL
 	symref-delete SP <ref> [NUL <old-ref>] NUL
 	symref-verify SP <ref> [NUL <old-ref>] NUL
@@ -123,6 +125,10 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-update::
+	Set <ref> to <new-ref> after verifying <old-ref> or <old-oid>,
+	if given. Can be used to delete or create symrefs too.
+
 symref-create::
 	Create symbolic ref <ref> with <new-ref> after verifying
 	it does not exist.  Can only be used in `no-deref` mode.
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 24556a28a8..809c1c7a76 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -238,6 +238,54 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+static void parse_cmd_symref_update(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *new_ref, *old_ref;
+	struct object_id old_oid;
+	int have_old = 0;
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-update: missing <ref>");
+
+	new_ref = parse_next_refname(&next);
+	if (!new_ref)
+		die("symref-update %s: missing <new-ref>", refname);
+	if (read_ref(new_ref, NULL))
+		die("symref-update %s: invalid <new-ref>", refname);
+
+	old_ref = parse_next_refname(&next);
+	/*
+	 * Since the user can also send in an old-oid, we try to parse
+	 * it as such too.
+	 */
+	if (old_ref && read_ref(old_ref, NULL)) {
+		if (!repo_get_oid(the_repository, old_ref, &old_oid)) {
+			old_ref = NULL;
+			have_old = 1;
+		} else
+			die("symref-update %s: invalid <old-ref> or <old-oid>", refname);
+	}
+
+	if (*next != line_termination)
+		die("symref-update %s: extra input: %s", refname, next);
+
+	update_flags |= create_reflog_flag | REF_SYMREF_UPDATE;
+	if (ref_transaction_update(transaction, refname, NULL,
+				   have_old ? &old_oid : NULL,
+				   new_ref, old_ref, update_flags,
+				   msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(old_ref);
+	free(new_ref);
+	strbuf_release(&err);
+}
+
 static void parse_cmd_create(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -509,6 +557,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-update", parse_cmd_symref_update, 3, UPDATE_REFS_OPEN },
 	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
 	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
diff --git a/refs.c b/refs.c
index e62c0f4aca..31c09c3317 100644
--- a/refs.c
+++ b/refs.c
@@ -1246,21 +1246,15 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	/*
-	 * The ref values are to be considered over the oid values when we're
-	 * doing symref operations.
-	 */
-	if (update->flags & REF_SYMREF_UPDATE) {
-		if (old_ref)
-			update->old_ref = xstrdup(old_ref);
-		if (new_ref)
-			update->new_ref = xstrdup(new_ref);
-	} else {
-		if (flags & REF_HAVE_NEW)
-			oidcpy(&update->new_oid, new_oid);
-		if (flags & REF_HAVE_OLD)
-			oidcpy(&update->old_oid, old_oid);
-	}
+	if (old_ref)
+		update->old_ref = xstrdup(old_ref);
+	if (new_ref)
+		update->new_ref = xstrdup(new_ref);
+	if (new_oid && flags & REF_HAVE_NEW)
+		oidcpy(&update->new_oid, new_oid);
+	if (old_oid && flags & REF_HAVE_OLD)
+		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 59d438878a..fb9886484c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2386,7 +2386,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_ref, update->old_ref, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2609,7 +2609,9 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
+	if (update->flags & REF_SYMREF_UPDATE &&
+	    !(update->flags & REF_LOG_ONLY) &&
+	    update->new_ref) {
 		if (create_symref_lock(refs, lock, update->refname, update->new_ref)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
@@ -2627,12 +2629,9 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		 * phase of the transaction only needs to commit the lock.
 		 */
 		update->flags |= REF_NEEDS_COMMIT;
-	}
-
-
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 6d42838e15..4bc6a369c7 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+						&u->new_oid, &u->old_oid, u->new_ref, u->old_ref, u->msg);
 				new_update->parent_update = u;
 
 				/*
@@ -1106,6 +1106,11 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
 
+			if (u->flags & REF_SYMREF_UPDATE && u->new_ref)
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_ref,
+				     RESOLVE_REF_READING, &u->new_oid, NULL))
+					goto done;
+
 			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
 			log = &logs[logs_nr++];
 			memset(log, 0, sizeof(*log));
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index f4e63fae6e..eeb3ee7952 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1360,6 +1360,7 @@ test_expect_success 'fails with duplicate HEAD update' '
 '
 
 test_expect_success 'fails with duplicate ref update via symref' '
+	test_when_finished "git symbolic-ref -d refs/heads/symref2" &&
 	git branch target2 $A &&
 	git symbolic-ref refs/heads/symref2 refs/heads/target2 &&
 	cat >stdin <<-EOF &&
@@ -1812,6 +1813,148 @@ test_expect_success "stdin ${type} symref-create reflogs with --create-reflog" '
 	git reflog exists refs/heads/symref
 '
 
+test_expect_success "stdin ${type} fails symref-update with no ref" '
+	create_stdin_buf ${type} "symref-update " >stdin &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	grep "fatal: symref-update: missing <ref>" err
+'
+
+test_expect_success "stdin ${type} fails symref-update with no new value" '
+	create_stdin_buf ${type} "symref-update refs/heads/symref" >stdin &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	grep "fatal: symref-update refs/heads/symref: missing <new-ref>" err
+'
+
+test_expect_success "stdin ${type} fails symref-update with too many arguments" '
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "$a" "$a" >stdin &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	if test "$type" = "-z"
+	then
+		grep "fatal: unknown command: $a" err
+	else
+		grep "fatal: symref-update refs/heads/symref: extra input:  $a" err
+	fi
+'
+
+test_expect_success "stdin ${type} symref-update ref creates with zero old value" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "$Z" >stdin &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	echo $a >expect &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-update ref creates with empty old value" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" >stdin &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	echo $a >expect &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-update ref fails with wrong old value" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	git symbolic-ref refs/heads/symref $a &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$m" "$b" >stdin &&
+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+	grep "fatal: symref-update refs/heads/symref: invalid <old-ref> or <old-oid>" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success "stdin ${type} symref-update ref works with right old value" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	git symbolic-ref refs/heads/symref $a &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$m" "$a" >stdin &&
+	git update-ref --stdin ${type} --no-deref <stdin &&
+	echo $m >expect &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-update creates symref (with deref)" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" >stdin &&
+	git update-ref --stdin ${type} <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$Z $(git rev-parse $a)" actual
+'
+
+test_expect_success "stdin ${type} symref-update updates symref (with deref)" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	test_when_finished "git update-ref -d --no-deref refs/heads/symref2" &&
+	git update-ref refs/heads/symref2 $a &&
+	git symbolic-ref --no-recurse refs/heads/symref refs/heads/symref2 &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" >stdin &&
+	git update-ref --stdin ${type} <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref2 >actual &&
+	test_cmp expect actual &&
+	echo refs/heads/symref2 >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success "stdin ${type} symref-update regular ref" '
+	test_when_finished "git symbolic-ref -d --no-recurse refs/heads/regularref" &&
+	git update-ref --no-deref refs/heads/regularref $a &&
+	create_stdin_buf ${type} "symref-update refs/heads/regularref" "$a" >stdin &&
+	git update-ref --stdin ${type} <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success "stdin ${type} symref-update regular ref with correct old-oid" '
+	test_when_finished "git symbolic-ref -d --no-recurse refs/heads/regularref" &&
+	git update-ref --no-deref refs/heads/regularref $a &&
+	create_stdin_buf ${type} "symref-update refs/heads/regularref" "$a" "$(git rev-parse $a)" >stdin &&
+	git update-ref --stdin ${type} <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success "stdin ${type} symref-update regular ref fails with wrong old-oid" '
+	test_when_finished "git update-ref -d refs/heads/regularref" &&
+	git update-ref --no-deref refs/heads/regularref $a &&
+	create_stdin_buf ${type} "symref-update refs/heads/regularref" "$a" "$(git rev-parse refs/heads/target2)" >stdin &&
+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+	echo $(git rev-parse $a) >expect &&
+	git rev-parse refs/heads/regularref >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-update with zero old-oid" '
+	test_when_finished "git symbolic-ref -d --no-recurse refs/heads/symref" &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "$Z" >stdin &&
+	git update-ref --stdin ${type} <stdin 2>err &&
+	echo $a >expect &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success "stdin ${type} symref-update ref with zero old-oid" '
+	test_when_finished "git symbolic-ref -d --no-recurse refs/heads/symref" &&
+	git symbolic-ref refs/heads/symref refs/heads/target2 &&
+	create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "$Z" >stdin &&
+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+	grep "fatal: cannot lock ref '"'"'refs/heads/symref'"'"': reference already exists" err &&
+	echo refs/heads/target2 >expect &&
+	git symbolic-ref refs/heads/symref >actual &&
+	test_cmp expect actual
+'
+
 done
 
 test_done
-- 
2.43.GIT

Re: [PATCH v2 6/7] update-ref: add support for symref-update

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 4333 bytes --]

On Fri, Apr 12, 2024 at 11:59:07AM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> Add 'symref-update' to allow updates of symbolic refs in a transaction
> via the 'git-update-ref' command. The 'symref-update' command takes in a
> <new-ref>, which the <ref> will be updated to. If the <ref> doesn't
> exist it will be created.
> 
> It also optionally takes either an <old-ref> or <old-oid>. If the
> <old-ref> is provided, it checks to see if the <ref> ponints to the
> <old-ref> before the update. If <old-oid> is provided it checks <ref> to
> ensure that it is a regular ref and <old-oid> is the OID before the
> update. This by extension also means that this when a zero <old-oid> is
> provided, it ensures that the ref didn't exist before.
> 
> This command will also support deref mode, to ensure that we can update
> dereferenced regular refs to symrefs.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>  Documentation/git-update-ref.txt |   6 ++
>  builtin/update-ref.c             |  49 +++++++++++
>  refs.c                           |  24 ++----
>  refs/files-backend.c             |  15 ++--
>  refs/reftable-backend.c          |   7 +-
>  t/t1400-update-ref.sh            | 143 +++++++++++++++++++++++++++++++
>  6 files changed, 220 insertions(+), 24 deletions(-)
> 
> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index a5b1f42728..9710c9bc78 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
>  	create SP <ref> SP <new-oid> LF
>  	delete SP <ref> [SP <old-oid>] LF
>  	verify SP <ref> [SP <old-oid>] LF
> +	symref-update SP <ref> SP <new-ref> [SP (<old-ref> | <old-oid>)] LF
>  	symref-create SP <ref> SP <new-ref> LF
>  	symref-delete SP <ref> [SP <old-ref>] LF
>  	symref-verify SP <ref> [SP <old-ref>] LF
> @@ -89,6 +90,7 @@ quoting:
>  	create SP <ref> NUL <new-oid> NUL
>  	delete SP <ref> NUL [<old-oid>] NUL
>  	verify SP <ref> NUL [<old-oid>] NUL
> +	symref-update SP <ref> NUL <new-ref> [NUL (<old-ref> | <old-oid>)] NUL
>  	symref-create SP <ref> NUL <new-ref> NUL
>  	symref-delete SP <ref> [NUL <old-ref>] NUL
>  	symref-verify SP <ref> [NUL <old-ref>] NUL
> @@ -123,6 +125,10 @@ verify::
>  	Verify <ref> against <old-oid> but do not change it.  If
>  	<old-oid> is zero or missing, the ref must not exist.
>  
> +symref-update::
> +	Set <ref> to <new-ref> after verifying <old-ref> or <old-oid>,
> +	if given. Can be used to delete or create symrefs too.
> +
>  symref-create::
>  	Create symbolic ref <ref> with <new-ref> after verifying
>  	it does not exist.  Can only be used in `no-deref` mode.
> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
> index 24556a28a8..809c1c7a76 100644
> --- a/builtin/update-ref.c
> +++ b/builtin/update-ref.c
> @@ -238,6 +238,54 @@ static void parse_cmd_update(struct ref_transaction *transaction,
>  	strbuf_release(&err);
>  }
>  
> +static void parse_cmd_symref_update(struct ref_transaction *transaction,
> +				    const char *next, const char *end)
> +{
> +	struct strbuf err = STRBUF_INIT;
> +	char *refname, *new_ref, *old_ref;
> +	struct object_id old_oid;
> +	int have_old = 0;
> +
> +	refname = parse_refname(&next);
> +	if (!refname)
> +		die("symref-update: missing <ref>");
> +
> +	new_ref = parse_next_refname(&next);
> +	if (!new_ref)
> +		die("symref-update %s: missing <new-ref>", refname);
> +	if (read_ref(new_ref, NULL))
> +		die("symref-update %s: invalid <new-ref>", refname);
> +
> +	old_ref = parse_next_refname(&next);
> +	/*
> +	 * Since the user can also send in an old-oid, we try to parse
> +	 * it as such too.
> +	 */
> +	if (old_ref && read_ref(old_ref, NULL)) {
> +		if (!repo_get_oid(the_repository, old_ref, &old_oid)) {
> +			old_ref = NULL;
> +			have_old = 1;
> +		} else
> +			die("symref-update %s: invalid <old-ref> or <old-oid>", refname);
> +	}

So we first try to parse it as a ref, and then as an object ID? Wouldn't
it preferable to try it the other way round and first check whether it
is a valid object ID? That would likely be cheaper, even though it may
be premature optimization.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 6/7] update-ref: add support for symref-update

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

Patrick Steinhardt <ps@pks.im> writes:
[snip]
>> +	/*
>> +	 * Since the user can also send in an old-oid, we try to parse
>> +	 * it as such too.
>> +	 */
>> +	if (old_ref && read_ref(old_ref, NULL)) {
>> +		if (!repo_get_oid(the_repository, old_ref, &old_oid)) {
>> +			old_ref = NULL;
>> +			have_old = 1;
>> +		} else
>> +			die("symref-update %s: invalid <old-ref> or <old-oid>", refname);
>> +	}
>
> So we first try to parse it as a ref, and then as an object ID? Wouldn't
> it preferable to try it the other way round and first check whether it
> is a valid object ID? That would likely be cheaper, even though it may
> be premature optimization.
>
> Patrick

I think the issue is `repo_get_oid` would also parse a refname to an
OID. Whereas we want to first check and keep refnames and only if it
isn't a refname, we'd want to parse it as an OID.

Also, why do you say it is cheaper?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 6/7] update-ref: add support for symref-update

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2683 bytes --]

On Sun, Apr 21, 2024 at 03:00:11PM -0400, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> [snip]
> >> +	/*
> >> +	 * Since the user can also send in an old-oid, we try to parse
> >> +	 * it as such too.
> >> +	 */
> >> +	if (old_ref && read_ref(old_ref, NULL)) {
> >> +		if (!repo_get_oid(the_repository, old_ref, &old_oid)) {
> >> +			old_ref = NULL;
> >> +			have_old = 1;
> >> +		} else
> >> +			die("symref-update %s: invalid <old-ref> or <old-oid>", refname);
> >> +	}
> >
> > So we first try to parse it as a ref, and then as an object ID? Wouldn't
> > it preferable to try it the other way round and first check whether it
> > is a valid object ID? That would likely be cheaper, even though it may
> > be premature optimization.
> >
> > Patrick
> 
> I think the issue is `repo_get_oid` would also parse a refname to an
> OID. Whereas we want to first check and keep refnames and only if it
> isn't a refname, we'd want to parse it as an OID.

Okay. The question is whether this matches precedence rules that we have
in other places. Namely, whether a reference name that looks like an
object ID overrides an object with the same name. Testing it:

```
$ rm -rf repo/
$ git init --ref-format=files repo
Initialized empty Git repository in /tmp/repo/.git/
$ cd repo/
$ git commit --allow-empty --message first
[main (root-commit) 09293d8] first
$ git commit --allow-empty --message second
[main 1588e76] second

$ git update-ref $(git rev-parse HEAD~) HEAD
$ cat .git/09293d82c434cdc1f7f286cf7b90cf35a6e57c43
1588e76ce7ef1ab25ee6f846a7b5d7032f83a69e

$ git rev-parse 09293d82c434cdc1f7f286cf7b90cf35a6e57c43
warning: refname '09293d82c434cdc1f7f286cf7b90cf35a6e57c43' is ambiguous.
Git normally never creates a ref that ends with 40 hex characters
because it will be ignored when you just specify 40-hex. These refs
may be created by mistake. For example,

  git switch -c $br $(git rev-parse ...)

where "$br" is somehow empty and a 40-hex ref is created. Please
examine these refs and maybe delete them. Turn this message off by
running "git config advice.objectNameWarning false"
09293d82c434cdc1f7f286cf7b90cf35a6e57c43
```

So the object ID has precedence over the reference with the same name.
Unless I'm mistaken, your proposed order would reverse that, wouldn't
it?

> Also, why do you say it is cheaper?

Checking whether a string can be parsed as an object ID should be faster
than having to ask the ref backend whether it knows any reference with
that name. So it should be fast for `repo_get_oid()` to bail out in case
the provided string doesn't look like an object ID.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 6/7] update-ref: add support for symref-update

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 4397 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Sun, Apr 21, 2024 at 03:00:11PM -0400, Karthik Nayak wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>> [snip]
>> >> +	/*
>> >> +	 * Since the user can also send in an old-oid, we try to parse
>> >> +	 * it as such too.
>> >> +	 */
>> >> +	if (old_ref && read_ref(old_ref, NULL)) {
>> >> +		if (!repo_get_oid(the_repository, old_ref, &old_oid)) {
>> >> +			old_ref = NULL;
>> >> +			have_old = 1;
>> >> +		} else
>> >> +			die("symref-update %s: invalid <old-ref> or <old-oid>", refname);
>> >> +	}
>> >
>> > So we first try to parse it as a ref, and then as an object ID? Wouldn't
>> > it preferable to try it the other way round and first check whether it
>> > is a valid object ID? That would likely be cheaper, even though it may
>> > be premature optimization.
>> >
>> > Patrick
>>
>> I think the issue is `repo_get_oid` would also parse a refname to an
>> OID. Whereas we want to first check and keep refnames and only if it
>> isn't a refname, we'd want to parse it as an OID.
>
> Okay. The question is whether this matches precedence rules that we have
> in other places. Namely, whether a reference name that looks like an
> object ID overrides an object with the same name. Testing it:
>
> ```
> $ rm -rf repo/
> $ git init --ref-format=files repo
> Initialized empty Git repository in /tmp/repo/.git/
> $ cd repo/
> $ git commit --allow-empty --message first
> [main (root-commit) 09293d8] first
> $ git commit --allow-empty --message second
> [main 1588e76] second
>
> $ git update-ref $(git rev-parse HEAD~) HEAD
> $ cat .git/09293d82c434cdc1f7f286cf7b90cf35a6e57c43
> 1588e76ce7ef1ab25ee6f846a7b5d7032f83a69e
>
> $ git rev-parse 09293d82c434cdc1f7f286cf7b90cf35a6e57c43
> warning: refname '09293d82c434cdc1f7f286cf7b90cf35a6e57c43' is ambiguous.
> Git normally never creates a ref that ends with 40 hex characters
> because it will be ignored when you just specify 40-hex. These refs
> may be created by mistake. For example,
>
>   git switch -c $br $(git rev-parse ...)
>
> where "$br" is somehow empty and a 40-hex ref is created. Please
> examine these refs and maybe delete them. Turn this message off by
> running "git config advice.objectNameWarning false"
> 09293d82c434cdc1f7f286cf7b90cf35a6e57c43
> ```
>
> So the object ID has precedence over the reference with the same name.
> Unless I'm mistaken, your proposed order would reverse that, wouldn't
> it?
>

I wasn't talking about a reference being mistaken for a object ID,
rather I was talking about how a reference will be `rev-parse`'d into an
object ID.

So instead if we did something like:

```
if (old_target) {
	if (!repo_get_oid(the_repository, old_target, &old_oid)) {
		old_target = NULL;
		have_old = 1;
	} else if (read_ref(old_target, NULL)) {
	} else {
		die("symref-update %s: invalid <old-target> or <old-oid>", refname);
	}
}
```

The problem is that now:

$ git init repo && cd repo
$ git commit --allow-empty -m"c1"
[master (root-commit) af416de] c1
$ git commit --allow-empty -m"c2"
[master 52e95b2] c2
$ git symbolic-ref refs/heads/symref refs/heads/master
$ git branch b1 master~1
$ git branch b2 master
$ cat .git/refs/heads/symref
ref: refs/heads/master
$ git update-ref --no-deref --stdin
symref-update refs/heads/symref refs/heads/b1 refs/heads/b2
$ cat .git/refs/heads/symref
ref: refs/heads/b1

This shouldn't have worked because symref was pointing to master, but
this did work, because `refs/heads/b2` was rev-parse'd to '52e95b2'
which is also what master is at.

The issue is that we call 'repo_get_oid', which takes in a commitish and
this could even be a reference. But ideally in 'symref' commands we want
to first treat refnames as refnames and not have them parsed as an OID.

Since, I'm not moving to 'refs:<target>' with the existing commands.
This is no longer an issue.

>> Also, why do you say it is cheaper?
>
> Checking whether a string can be parsed as an object ID should be faster
> than having to ask the ref backend whether it knows any reference with
> that name. So it should be fast for `repo_get_oid()` to bail out in case
> the provided string doesn't look like an object ID.
>
> Patrick

Yes. But for a valid refname, the `repo_get_oid()` is querying the
reference backend similar to `read_ref()`. Also `read_ref` also does
initial checks with `check_refname_format` which I think doesn't check
the ref backend.

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v2 7/7] refs: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In the previous commits, we added support for
various symref commands in `git-update-ref`. While it allowed us to now
manipulate symbolic refs via `git-update-ref`, it didn't activate the
'reference-transaction' hook.

Let's activate the hook for symbolic reference updates too. There is no
new format described for this and we stick to the existing format of:
    <old-value> SP <new-value> SP <ref-name> LF
but now, <old-value> and <new-value> could also denote references
instead of objects.

While this seems to be backward incompatible, it is okay, since the only
way the `reference-transaction` hook has refs in its output is when
`git-update-ref` is used with `update-symref` command. Also the
documentation for reference-transaction hook always stated that support
for symbolic references may be added in the future.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---

We did initially discuss the possibility of having {symref-<command>} as
the prefix [1]. But this information is not propagated through the system
and we don't know which command is actually registering at the hook. We
could use flags for propagating this, but this simply complicates things.

I then thought of adding 'symref' prefix to the symref updates. But I
realized that that is no different from not having the symref prefix, because
even 'symref' prefix entries received by the hook as input could contain
either object IDs or refnames, like updating a regular ref to a symref. So
the hook needs to dynamically read inputs anyways. So might as well keep the
current syntax.

[1]: https://lore.kernel.org/git/CAOLa=ZTLv39b4Q=AAUA39tXKgOSuu54xk3-r9OUenzxR-6qcag@mail.gmail.com/

 Documentation/githooks.txt       | 13 ++++++----
 refs.c                           | 17 ++++++++-----
 t/t1416-ref-transaction-hooks.sh | 41 ++++++++++++++++++++++++++++++++
 3 files changed, 60 insertions(+), 11 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ee9b92c90d..1db5ab02d6 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also cover symbolic references.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,19 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index 31c09c3317..010f426def 100644
--- a/refs.c
+++ b/refs.c
@@ -2362,15 +2362,20 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		const char *new_value, *old_value;
 
-		if (update->flags & REF_SYMREF_UPDATE)
-			continue;
+		new_value = oid_to_hex(&update->new_oid);
+		old_value = oid_to_hex(&update->old_oid);
+
+		if (update->flags & REF_SYMREF_UPDATE) {
+			if (update->flags & REF_HAVE_NEW && !null_new_value(update))
+				new_value = update->new_ref;
+			if (update->flags & REF_HAVE_OLD && update->old_ref)
+				old_value = update->old_ref;
+		}
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+		strbuf_addf(&buf, "%s %s %s\n", old_value, new_value, update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..104e2c5da4 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -108,6 +108,10 @@ test_expect_success 'hook gets all queued updates in aborted state' '
 	test_cmp expect actual
 '
 
+# This test doesn't add a check for 'symref-delete' since there is a
+# variation between the ref backends WRT 'delete'. In the files backend,
+# 'delete' also triggers an additional transaction update on the
+# packed-refs backend, which constitutes additional reflog entries.
 test_expect_success 'interleaving hook calls succeed' '
 	test_when_finished "rm -r target-repo.git" &&
 
@@ -134,4 +138,41 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook gets all queued symref updates' '
+	test_when_finished "rm actual" &&
+
+	git update-ref refs/heads/branch $POST_OID &&
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+	git symbolic-ref refs/heads/symrefu refs/heads/main &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	cat >expect <<-EOF &&
+		prepared
+		refs/heads/main $ZERO_OID refs/heads/symref
+		$ZERO_OID refs/heads/main refs/heads/symrefc
+		refs/heads/main refs/heads/branch refs/heads/symrefu
+		committed
+		refs/heads/main $ZERO_OID refs/heads/symref
+		$ZERO_OID refs/heads/main refs/heads/symrefc
+		refs/heads/main refs/heads/branch refs/heads/symrefu
+	EOF
+
+	git update-ref --no-deref --stdin <<-EOF &&
+		start
+		symref-verify refs/heads/symref refs/heads/main
+		symref-create refs/heads/symrefc refs/heads/main
+		symref-update refs/heads/symrefu refs/heads/branch refs/heads/main
+		prepare
+		commit
+	EOF
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

Re: [PATCH v2 0/7] update-ref: add symref oriented commands

[Junio C Hamano]

"make refs.sp" should have told you that you should not write NULL
as just 0.  I'll queue a SQUASH??? fix-up on top before merging it
into 'seen' for today's integration.

diff --git c/refs.c w/refs.c
index 010f426def..d578a2823b 100644
--- c/refs.c
+++ w/refs.c
@@ -2758,7 +2758,7 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg,
 
 	for_each_string_list_item(item, refnames) {
 		ret = ref_transaction_delete(transaction, item->string,
-					     NULL, flags, 0, msg, &err);
+					     NULL, flags, NULL, msg, &err);
 		if (ret) {
 			warning(_("could not delete reference %s: %s"),
 				item->string, err.buf);




Also there are quite many whitespace breakages.

$ git am -s ./+kn7-v2-update-ref-symref
Applying: refs: accept symref values in `ref_transaction[_add]_update`
Applying: update-ref: add support for symref-verify
.git/rebase-apply/patch:59: indent with spaces.
        if (line_termination) {
.git/rebase-apply/patch:60: indent with spaces.
                /* Without -z, consume SP and use next argument */
.git/rebase-apply/patch:61: indent with spaces.
                if (!**next || **next == line_termination)
.git/rebase-apply/patch:62: indent with spaces.
                        return NULL;
.git/rebase-apply/patch:63: indent with spaces.
                if (**next != ' ')
warning: squelched 10 whitespace errors
warning: 15 lines applied after fixing whitespace errors.
Applying: update-ref: add support for symref-delete
.git/rebase-apply/patch:95: indent with spaces.
                die("symref-delete: cannot operate with deref mode");
.git/rebase-apply/patch:101: indent with spaces.
        old_ref = parse_next_refname(&next);
warning: 2 lines applied after fixing whitespace errors.
Applying: files-backend: extract out `create_symref_lock`
Applying: update-ref: add support for symref-create
.git/rebase-apply/patch:81: indent with spaces.
                die("symref-create: cannot operate with deref mode");
warning: 1 line applied after fixing whitespace errors.
Applying: update-ref: add support for symref-update
Applying: refs: support symrefs in 'reference-transaction' hook

Re: [PATCH v2 0/7] update-ref: add symref oriented commands

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 6347 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> "make refs.sp" should have told you that you should not write NULL
> as just 0.  I'll queue a SQUASH??? fix-up on top before merging it
> into 'seen' for today's integration.
>
> diff --git c/refs.c w/refs.c
> index 010f426def..d578a2823b 100644
> --- c/refs.c
> +++ w/refs.c
> @@ -2758,7 +2758,7 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg,
>
>  	for_each_string_list_item(item, refnames) {
>  		ret = ref_transaction_delete(transaction, item->string,
> -					     NULL, flags, 0, msg, &err);
> +					     NULL, flags, NULL, msg, &err);
>  		if (ret) {
>  			warning(_("could not delete reference %s: %s"),
>  				item->string, err.buf);
>
>

Thanks Junio, I'll apply it locally too.

>
>
> Also there are quite many whitespace breakages.
>
> $ git am -s ./+kn7-v2-update-ref-symref
> Applying: refs: accept symref values in `ref_transaction[_add]_update`
> Applying: update-ref: add support for symref-verify
> .git/rebase-apply/patch:59: indent with spaces.
>         if (line_termination) {
> .git/rebase-apply/patch:60: indent with spaces.
>                 /* Without -z, consume SP and use next argument */
> .git/rebase-apply/patch:61: indent with spaces.
>                 if (!**next || **next == line_termination)
> .git/rebase-apply/patch:62: indent with spaces.
>                         return NULL;
> .git/rebase-apply/patch:63: indent with spaces.
>                 if (**next != ' ')
> warning: squelched 10 whitespace errors
> warning: 15 lines applied after fixing whitespace errors.
> Applying: update-ref: add support for symref-delete
> .git/rebase-apply/patch:95: indent with spaces.
>                 die("symref-delete: cannot operate with deref mode");
> .git/rebase-apply/patch:101: indent with spaces.
>         old_ref = parse_next_refname(&next);
> warning: 2 lines applied after fixing whitespace errors.
> Applying: files-backend: extract out `create_symref_lock`
> Applying: update-ref: add support for symref-create
> .git/rebase-apply/patch:81: indent with spaces.
>                 die("symref-create: cannot operate with deref mode");
> warning: 1 line applied after fixing whitespace errors.
> Applying: update-ref: add support for symref-update
> Applying: refs: support symrefs in 'reference-transaction' hook

Sigh. Even with editorconfig, this seems to be problematic for me on
Emacs. I'll just start applying locally from now before sending it.

As such I've applied this locally and will refrain from sending a new
version until a review happens.

1:  3269d0e91e = 1:  3269d0e91e refs: accept symref values in
`ref_transaction[_add]_update`
2:  a8cb0e0a1d ! 2:  2293b514a8 update-ref: add support for symref-verify
    @@ builtin/update-ref.c: static char *parse_refname(const char **next)
      	return strbuf_detach(&ref, NULL);
      }

    -+
    -+
     +/*
     + * Wrapper around parse_refname which skips the next delimiter.
     + */
     +static char *parse_next_refname(const char **next)
     +{
    -+        if (line_termination) {
    -+                /* Without -z, consume SP and use next argument */
    -+                if (!**next || **next == line_termination)
    -+                        return NULL;
    -+                if (**next != ' ')
    -+                        die("expected SP but got: %s", *next);
    -+        } else {
    -+                /* With -z, read the next NUL-terminated line */
    -+                if (**next)
    -+                        return NULL;
    -+        }
    -+        /* Skip the delimiter */
    -+        (*next)++;
    ++	if (line_termination) {
    ++		/* Without -z, consume SP and use next argument */
    ++		if (!**next || **next == line_termination)
    ++			return NULL;
    ++		if (**next != ' ')
    ++			die("expected SP but got: %s", *next);
    ++	} else {
    ++		/* With -z, read the next NUL-terminated line */
    ++		if (**next)
    ++			return NULL;
    ++	}
    ++	/* Skip the delimiter */
    ++	(*next)++;
     +
    -+        return parse_refname(next);
    ++	return parse_refname(next);
     +}
     +
      /*
    @@ builtin/update-ref.c: static void parse_cmd_verify(struct
ref_transaction *trans
     +}
     +
     +static void parse_cmd_symref_verify(struct ref_transaction *transaction,
    -+                                    const char *next, const char *end)
    ++				    const char *next, const char *end)
     +{
     +	struct strbuf err = STRBUF_INIT;
     +	struct object_id old_oid;
3:  37c3e006da ! 3:  62d05e7cb7 update-ref: add support for symref-delete
    @@ builtin/update-ref.c: static void parse_cmd_delete(struct
ref_transaction *trans
     +	char *refname, *old_ref;
     +
     +	if (!(update_flags & REF_NO_DEREF))
    -+                die("symref-delete: cannot operate with deref mode");
    ++		die("symref-delete: cannot operate with deref mode");
     +
     +	refname = parse_refname(&next);
     +	if (!refname)
     +		die("symref-delete: missing <ref>");
     +
    -+        old_ref = parse_next_refname(&next);
    ++	old_ref = parse_next_refname(&next);
     +	if (old_ref && read_ref(old_ref, NULL))
     +		die("symref-delete %s: invalid <old-ref>", refname);
     +
    @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg,
      	for_each_string_list_item(item, refnames) {
      		ret = ref_transaction_delete(transaction, item->string,
     -					     NULL, flags, msg, &err);
    -+					     NULL, flags, 0, msg, &err);
    ++					     NULL, flags, NULL, msg, &err);
      		if (ret) {
      			warning(_("could not delete reference %s: %s"),
      				item->string, err.buf);
4:  53fdb408ef = 4:  1bd39a38dc files-backend: extract out `create_symref_lock`
5:  8fa0151f94 ! 5:  732d31b43d update-ref: add support for symref-create
    @@ builtin/update-ref.c: static void parse_cmd_create(struct
ref_transaction *trans
     +	char *refname, *new_ref;
     +
     +	if (!(update_flags & REF_NO_DEREF))
    -+                die("symref-create: cannot operate with deref mode");
    ++		die("symref-create: cannot operate with deref mode");
     +
     +	refname = parse_refname(&next);
     +	if (!refname)
6:  714492ede3 = 6:  eb3d239b4b update-ref: add support for symref-update
7:  c483104562 = 7:  3cdfb9e528 refs: support symrefs in
'reference-transaction' hook

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 0/7] update-ref: add symref oriented commands

[Christian Couder]

On Fri, Apr 12, 2024 at 12:11 PM Karthik Nayak <karthik.188@gmail.com> wrote:

> The 'symref-update' command can be used to update a symref, create a symref,
> delete a symref or even convert an existing regular ref to a symref. Wherein
> like the regular 'update' command, the zero OID can be used to create/delete
> a symref.

Is it possible to also convert a symref to a regular ref, like when
HEAD becomes detached?

> 2. The flow is now changedc to send an old_ref, new_ref pair in supplement to

s/changedc/changed/

> the existing old_oid, new_oid pair to the reference backends. This allows the
> backends to simply do a combination of changes based on what values are set.

I had a number of comments and questions when reading the first few
patches in this series. I also took a very quick look at the rest of
the series, but I think answering my questions about the first few
patches would make reading the rest of the series easier, so I will
take a deeper look later.

Thanks.

Re: [PATCH v2 0/7] update-ref: add symref oriented commands

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

Christian Couder <christian.couder@gmail.com> writes:

> On Fri, Apr 12, 2024 at 12:11 PM Karthik Nayak <karthik.188@gmail.com> wrote:
>
>> The 'symref-update' command can be used to update a symref, create a symref,
>> delete a symref or even convert an existing regular ref to a symref. Wherein
>> like the regular 'update' command, the zero OID can be used to create/delete
>> a symref.
>
> Is it possible to also convert a symref to a regular ref, like when
> HEAD becomes detached?
>

Currently not, that could be a follow up, since that would involve
updating the regular commands. A quick way to think about it is that
this series adds commands whose end product is a symref. Since what
you're talking about is the inverse, it makes sense to add it to the
existing commands.

>> 2. The flow is now changedc to send an old_ref, new_ref pair in supplement to
>
> s/changedc/changed/
>
>> the existing old_oid, new_oid pair to the reference backends. This allows the
>> backends to simply do a combination of changes based on what values are set.
>
> I had a number of comments and questions when reading the first few
> patches in this series. I also took a very quick look at the rest of
> the series, but I think answering my questions about the first few
> patches would make reading the rest of the series easier, so I will
> take a deeper look later.
>
> Thanks.

Thanks for the review. Really appreciate it.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v2 0/7] update-ref: add symref oriented commands

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2211 bytes --]

On Fri, Apr 12, 2024 at 11:59:01AM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The 'git-update-ref(1)' command allows transactional reference updates.
> But currently only supports regular reference updates. Meaning, if one
> wants to update HEAD (symbolic ref) in a transaction, there is no tool
> to do so.
> 
> One option to obtain transactional updates for the HEAD ref is to
> manually create the HEAD.lock file and commit. This is intrusive, where
> the user needs to mimic internal git behavior. Also, this only works
> when using the files backend.
> 
> At GitLab, we've been using the manual process till date, to allow users
> to set and change their default branch. But with the introduction of
> reftables as a reference backend, this becomes a necessity to be solved
> within git.
> 
> This patch series goes about introducing a set of commands
> symref-{create,verify,delete,update} to work with symrefs complimenting
> the existing commands for the regular refs within 'git-update-ref(1)'.

One more thought crossed my mind this night: is it even necessary to
introduce new commands for this? As far as I can see, it should be
feasible to introduce symref support for all existing commands without
breaking backwards compatibility. This can be achieved by using a prefix
that cannot ever be an object ID, like for example "ref:".

Thus, all of the following should work (with 1234 being an OID and 0000
being the null OID):

    update HEAD ref:refs/heads/main ref:refs/heads/master
    update HEAD ref:refs/heads/main 1234
    update HEAD ref:refs/heads/main 0000
    update HEAD 1234 ref:refs/heads/main
    update HEAD 0000 ref:refs/heads/main
    create HEAD ref:refs/heads/main
    delete HEAD ref:refs/heads/main
    verify HEAD ref:refs/heads/mains

Parsing is unambiguous because we can use `starts_with("ref:")` for an
otherwise invalid object ID. Furthermore, because refs cannot have
spaces, we also don't have an issue with the SP separator.

I have a hunch that this variant might lead to less code duplication,
lead to less confusing behaviour and also makes for an easier user
interface.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 0/7] update-ref: add symref oriented commands

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2696 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Fri, Apr 12, 2024 at 11:59:01AM +0200, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The 'git-update-ref(1)' command allows transactional reference updates.
>> But currently only supports regular reference updates. Meaning, if one
>> wants to update HEAD (symbolic ref) in a transaction, there is no tool
>> to do so.
>>
>> One option to obtain transactional updates for the HEAD ref is to
>> manually create the HEAD.lock file and commit. This is intrusive, where
>> the user needs to mimic internal git behavior. Also, this only works
>> when using the files backend.
>>
>> At GitLab, we've been using the manual process till date, to allow users
>> to set and change their default branch. But with the introduction of
>> reftables as a reference backend, this becomes a necessity to be solved
>> within git.
>>
>> This patch series goes about introducing a set of commands
>> symref-{create,verify,delete,update} to work with symrefs complimenting
>> the existing commands for the regular refs within 'git-update-ref(1)'.
>
> One more thought crossed my mind this night: is it even necessary to
> introduce new commands for this? As far as I can see, it should be
> feasible to introduce symref support for all existing commands without
> breaking backwards compatibility. This can be achieved by using a prefix
> that cannot ever be an object ID, like for example "ref:".
>

I couldn't think of any reason that we can't do this. I think it would
work. I'll try to patch something with my existing tests and see if
breaks something.

> Thus, all of the following should work (with 1234 being an OID and 0000
> being the null OID):
>
>     update HEAD ref:refs/heads/main ref:refs/heads/master
>     update HEAD ref:refs/heads/main 1234
>     update HEAD ref:refs/heads/main 0000
>     update HEAD 1234 ref:refs/heads/main
>     update HEAD 0000 ref:refs/heads/main
>     create HEAD ref:refs/heads/main
>     delete HEAD ref:refs/heads/main
>     verify HEAD ref:refs/heads/mains
>
> Parsing is unambiguous because we can use `starts_with("ref:")` for an
> otherwise invalid object ID. Furthermore, because refs cannot have
> spaces, we also don't have an issue with the SP separator.
>
> I have a hunch that this variant might lead to less code duplication,
> lead to less confusing behaviour and also makes for an easier user
> interface.

I'm certain that the duplication in update-ref.c will go away with this.
Regarding other code changes, I think they'll stay the same.

But I do agree, this seems much nicer in terms of UX and also allows us
to introduce `symref => regular ref` conversion in this series itself.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'git-update-ref(1)' command allows transactional reference updates.
But currently only supports regular reference updates. Meaning, if one
wants to update HEAD (symbolic ref) in a transaction, there is no tool
to do so.

One option to obtain transactional updates for the HEAD ref is to
manually create the HEAD.lock file and commit. This is intrusive, where
the user needs to mimic internal git behavior. Also, this only works
when using the files backend.

At GitLab, we've been using the manual process till date, to allow users
to set and change their default branch. But with the introduction of
reftables as a reference backend, this becomes a necessity to be solved
within git.

The patch series adds symref support to the existing commands {verify,
create, delete, update} within 'git-update-ref'. This is done by parsing
inputs with the 'ref:' prefix as symref targets. This updates our current
commands to:

update SP <ref> SP (<new-oid> | ref:<new-target>) [SP (<old-oid> | ref:<old-target>)] LF
create SP <ref> SP (<new-oid> | ref:<new-target>) LF
delete SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
verify SP <ref> [SP (<old-oid> | ref:<old-target>)] LF

Wherein, when ref:<new-target> is provided, the update ensures that
the <ref> is a symbolic ref which targets <new-target>. When
ref:<old-target> is provided, we ensure that <ref> is a symbolic ref
which targets <old-target> before the update.

With this it is possible to:
1. Create, verify, delete, and update symrefs
2. Create dangling symrefs
3. Update regular refs to become symrefs
4. Update symrefs to become regular refs

V1 of the patch series can be found here:
https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com/
V2 of the patch series can be found here:
https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com/

Changes from v2:
1. We no longer have separate commands for symrefs, instead the regular
commands learn to parse 'ref:<target>' as symref targets. This reduces
the code in this series. Thanks Patrick for the suggestion.
2. Apart from supporting regular refs => symrefs. We also support the
inverse now.
3. Also allow creation of dangling refs.
4. Bunch of cleanups
   - whitespace issues in the previous patch series
   - uneeded header includes
   - uneeded tests
5. Added more documentation and better error messages, especially in reftables.
6. Better assertions around the input data.

Thanks all for the reviews on the previous iteration. Appreciate the support!

Range diff against v2:

1:  3269d0e91e ! 1:  4e49d54dcc refs: accept symref values in `ref_transaction[_add]_update`
    @@ Commit message
         flags to create a `ref_update` and add it to the transaction at hand.
     
         To extend symref support in transactions, we need to also accept the
    -    old and new ref values and process it. In this commit, let's add the
    -    required paramaters to the function and modify all call sites.
    +    old and new ref targets and process it. In this commit, let's add the
    +    required parameters to the function and modify all call sites.
     
    -    The two paramaters added are `new_ref` and `old_ref`. The `new_ref` is
    -    used to denote what the reference should point to when the transaction
    -    is applied. Some functions allow this parameter to be NULL, meaning that
    -    the reference is not changed, or `""`, meaning that the reference should
    -    be deleted.
    +    The two parameters added are `new_target` and `old_target`. The
    +    `new_target` is used to denote what the reference should point to when
    +    the transaction is applied. Some functions allow this parameter to be
    +    NULL, meaning that the reference is not changed.
     
    -    The `old_ref` denotes the value of that the reference must have before
    -    the update. Some functions allow this parameter to be NULL, meaning that
    -    the old value of the reference is not checked, or `""`, meaning that the
    -    reference must not exist before the update. A copy of this value is made
    +    The `old_target` denotes the value the reference must have before the
    +    update. Some functions allow this parameter to be NULL, meaning that the
    +    old value of the reference is not checked. A copy of this value is made
         in the transaction.
     
         The handling logic of these parameters will be added in consequent
    -    commits as we implement symref-{create, update, delete, verify}.
    +    commits as we add symref support to the existing 'git-update-ref'
    +    commands.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      		const char *refname, unsigned int flags,
      		const struct object_id *new_oid,
      		const struct object_id *old_oid,
    -+		const char *new_ref, const char *old_ref,
    ++		const char *new_target, const char *old_target,
      		const char *msg)
      {
      	struct ref_update *update;
    +@@ refs.c: struct ref_update *ref_transaction_add_update(
    + 	if (transaction->state != REF_TRANSACTION_OPEN)
    + 		BUG("update called for transaction that is not open");
    + 
    ++	if (old_oid && !is_null_oid(old_oid) && old_target)
    ++		BUG("Only one of old_oid and old_target should be non NULL");
    ++	if (new_oid && !is_null_oid(new_oid) && new_target)
    ++		BUG("Only one of new_oid and new_target should be non NULL");
    ++
    + 	FLEX_ALLOC_STR(update, refname, refname);
    + 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
    + 	transaction->updates[transaction->nr++] = update;
     @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *new_oid,
      			   const struct object_id *old_oid,
    -+			   const char *new_ref, const char *old_ref,
    ++			   const char *new_target,
    ++			   const char *old_target,
      			   unsigned int flags, const char *msg,
      			   struct strbuf *err)
      {
    @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      
      	ref_transaction_add_update(transaction, refname, flags,
     -				   new_oid, old_oid, msg);
    -+				   new_oid, old_oid, new_ref, old_ref, msg);
    ++				   new_oid, old_oid, new_target,
    ++				   old_target, msg);
      	return 0;
      }
      
    @@ refs.c: int refs_update_ref(struct ref_store *refs, const char *msg,
     
      ## refs.h ##
     @@ refs.h: struct ref_transaction *ref_transaction_begin(struct strbuf *err);
    -  */
    - #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
    - 
    -+/*
    -+ * The reference update is considered to be done on a symbolic reference. This
    -+ * ensures that we verify, delete, create and update the ref correspondingly.
    -+ */
    -+#define REF_SYMREF_UPDATE (1 << 12)
    -+
    - /*
    -  * Bitmask of all of the flags that are allowed to be passed in to
    -  * ref_transaction_update() and friends:
    -  */
    - #define REF_TRANSACTION_UPDATE_ALLOWED_FLAGS                                  \
    - 	(REF_NO_DEREF | REF_FORCE_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION | \
    --	 REF_SKIP_REFNAME_VERIFICATION)
    -+	 REF_SKIP_REFNAME_VERIFICATION | REF_SYMREF_UPDATE )
    - 
    - /*
    -  * Add a reference update to transaction. `new_oid` is the value that
    +  *         before the update. A copy of this value is made in the
    +  *         transaction.
    +  *
    ++ *     new_target -- the target reference that the reference will be
    ++ *         update to point to. This takes precedence over new_oid when
    ++ *         set. If the reference is a regular reference, it will be
    ++ *         converted to a symbolic reference.
    ++ *
    ++ *     old_target -- the reference that the reference must be pointing to.
    ++ *         Will only be taken into account when the reference is a symbolic
    ++ *         reference.
    ++ *
    +  *     flags -- flags affecting the update, passed to
    +  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
    +  *         REF_FORCE_CREATE_REFLOG. See those constants for more
    +@@ refs.h: struct ref_transaction *ref_transaction_begin(struct strbuf *err);
    +  * beforehand. The old value is checked after the lock is taken to
    +  * prevent races. If the old value doesn't agree with old_oid, the
    +  * whole transaction fails. If old_oid is NULL, then the previous
    +- * value is not checked.
    ++ * value is not checked. If `old_target` is not NULL, treat the reference
    ++ * as a symbolic ref and validate that its target before the update is
    ++ * `old_target`. If the `new_target` is not NULL, then the reference
    ++ * will be updated to a symbolic ref which targets `new_target`.
    ++ * Together, these allow us to update between regular refs and symrefs.
    +  *
    +  * See the above comment "Reference transaction updates" for more
    +  * information.
     @@ refs.h: int ref_transaction_update(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *new_oid,
      			   const struct object_id *old_oid,
    -+			   const char *new_ref, const char *old_ref,
    ++			   const char *new_target,
    ++			   const char *old_target,
      			   unsigned int flags, const char *msg,
      			   struct strbuf *err);
      
    @@ refs/refs-internal.h: struct ref_update {
      	struct object_id old_oid;
      
     +	/*
    -+	 * If (flags & REF_SYMREF_UPDATE), set the reference to this
    -+	 * value (or delete it, if `new_ref` is an empty string).
    ++	 * If set, point the reference to this value. This can also be
    ++	 * used to convert regular references to become symbolic refs.
     +	 */
    -+	const char *new_ref;
    ++	const char *new_target;
     +
     +	/*
    -+	 * If (type & REF_SYMREF_UPDATE), check that the reference
    -+	 * previously had this value (or didn't previously exist,
    -+	 * if `old_ref` is an empty string).
    ++	 * If set and the reference is a symbolic ref, check that the
    ++	 * reference previously pointed to this value.
     +	 */
    -+	const char *old_ref;
    ++	const char *old_target;
     +
      	/*
      	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
    @@ refs/refs-internal.h: struct ref_update *ref_transaction_add_update(
      		const char *refname, unsigned int flags,
      		const struct object_id *new_oid,
      		const struct object_id *old_oid,
    -+		const char *new_ref, const char *old_ref,
    ++		const char *new_target, const char *old_target,
      		const char *msg);
      
      /*
2:  a8cb0e0a1d < -:  ---------- update-ref: add support for symref-verify
3:  37c3e006da < -:  ---------- update-ref: add support for symref-delete
-:  ---------- > 2:  37b7aadca4 update-ref: support parsing ref targets in `parse_next_oid`
4:  53fdb408ef = 3:  9cb7817f94 files-backend: extract out `create_symref_lock`
5:  8fa0151f94 < -:  ---------- update-ref: add support for symref-create
6:  714492ede3 < -:  ---------- update-ref: add support for symref-update
-:  ---------- > 4:  c7f43f6058 update-ref: support symrefs in the verify command
-:  ---------- > 5:  4016d6ca98 update-ref: support symrefs in the delete command
-:  ---------- > 6:  9f19e82f00 update-ref: support symrefs in the create command
-:  ---------- > 7:  132dbfcc5f update-ref: support symrefs in the update command
7:  c483104562 ! 8:  1b709f995b refs: support symrefs in 'reference-transaction' hook
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    refs: support symrefs in 'reference-transaction' hook
    +    ref: support symrefs in 'reference-transaction' hook
     
         The 'reference-transaction' hook runs whenever a reference update is
    -    made to the system. In the previous commits, we added support for
    -    various symref commands in `git-update-ref`. While it allowed us to now
    +    made to the system. In the previous commits, we added symref support for
    +    various commands in `git-update-ref`. While it allowed us to now
         manipulate symbolic refs via `git-update-ref`, it didn't activate the
         'reference-transaction' hook.
     
    @@ Commit message
         new format described for this and we stick to the existing format of:
             <old-value> SP <new-value> SP <ref-name> LF
         but now, <old-value> and <new-value> could also denote references
    -    instead of objects.
    +    instead of objects, where the format is similar to that in
    +    'git-update-ref', i.e. 'ref:<ref-target>'.
     
         While this seems to be backward incompatible, it is okay, since the only
         way the `reference-transaction` hook has refs in its output is when
    -    `git-update-ref` is used with `update-symref` command. Also the
    -    documentation for reference-transaction hook always stated that support
    -    for symbolic references may be added in the future.
    +    `git-update-ref` is used to manipulate symrefs. Also the documentation
    +    for reference-transaction hook always stated that support for symbolic
    +    references may be added in the future.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ Documentation/githooks.txt: given reference transaction is in:
      `<ref-name>` via `git rev-parse`.
      
     +For symbolic reference updates the `<old_value>` and `<new-value>`
    -+fields could denote references instead of objects.
    ++fields could denote references instead of objects, denoted via the
    ++`ref:<ref-target>` format.
     +
      The exit status of the hook is ignored for any state except for the
      "prepared" state. In the "prepared" state, a non-zero exit status will
    @@ refs.c: static int run_transaction_hook(struct ref_transaction *transaction,
      
      	for (i = 0; i < transaction->nr; i++) {
      		struct ref_update *update = transaction->updates[i];
    -+		const char *new_value, *old_value;
    ++		strbuf_reset(&buf);
      
    --		if (update->flags & REF_SYMREF_UPDATE)
    +-		/*
    +-		 * Skip reference transaction for symbolic refs.
    +-		 */
    +-		if (update->new_target || update->old_target)
     -			continue;
    -+		new_value = oid_to_hex(&update->new_oid);
    -+		old_value = oid_to_hex(&update->old_oid);
    -+
    -+		if (update->flags & REF_SYMREF_UPDATE) {
    -+			if (update->flags & REF_HAVE_NEW && !null_new_value(update))
    -+				new_value = update->new_ref;
    -+			if (update->flags & REF_HAVE_OLD && update->old_ref)
    -+				old_value = update->old_ref;
    -+		}
    ++		if (update->flags & REF_HAVE_OLD && update->old_target)
    ++			strbuf_addf(&buf, "ref:%s ", update->old_target);
    ++		else
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
      
    - 		strbuf_reset(&buf);
    +-		strbuf_reset(&buf);
     -		strbuf_addf(&buf, "%s %s %s\n",
     -			    oid_to_hex(&update->old_oid),
     -			    oid_to_hex(&update->new_oid),
     -			    update->refname);
    -+		strbuf_addf(&buf, "%s %s %s\n", old_value, new_value, update->refname);
    ++		if (update->flags & REF_HAVE_NEW && update->new_target)
    ++			strbuf_addf(&buf, "ref:%s ", update->new_target);
    ++		else
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
    ++
    ++		strbuf_addf(&buf, "%s\n", update->refname);
      
      		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
      			if (errno != EPIPE) {
     
      ## t/t1416-ref-transaction-hooks.sh ##
    -@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'hook gets all queued updates in aborted state' '
    - 	test_cmp expect actual
    +@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
    + 	test_cmp expect target-repo.git/actual
      '
      
    -+# This test doesn't add a check for 'symref-delete' since there is a
    ++# This test doesn't add a check for symref 'delete' since there is a
     +# variation between the ref backends WRT 'delete'. In the files backend,
     +# 'delete' also triggers an additional transaction update on the
     +# packed-refs backend, which constitutes additional reflog entries.
    - test_expect_success 'interleaving hook calls succeed' '
    - 	test_when_finished "rm -r target-repo.git" &&
    - 
    -@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
    - 	test_cmp expect target-repo.git/actual
    - '
    - 
     +test_expect_success 'hook gets all queued symref updates' '
     +	test_when_finished "rm actual" &&
     +
    @@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls s
     +
     +	cat >expect <<-EOF &&
     +		prepared
    -+		refs/heads/main $ZERO_OID refs/heads/symref
    -+		$ZERO_OID refs/heads/main refs/heads/symrefc
    -+		refs/heads/main refs/heads/branch refs/heads/symrefu
    ++		ref:refs/heads/main $ZERO_OID refs/heads/symref
    ++		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
    ++		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
     +		committed
    -+		refs/heads/main $ZERO_OID refs/heads/symref
    -+		$ZERO_OID refs/heads/main refs/heads/symrefc
    -+		refs/heads/main refs/heads/branch refs/heads/symrefu
    ++		ref:refs/heads/main $ZERO_OID refs/heads/symref
    ++		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
    ++		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
     +	EOF
     +
     +	git update-ref --no-deref --stdin <<-EOF &&
     +		start
    -+		symref-verify refs/heads/symref refs/heads/main
    -+		symref-create refs/heads/symrefc refs/heads/main
    -+		symref-update refs/heads/symrefu refs/heads/branch refs/heads/main
    ++		verify refs/heads/symref ref:refs/heads/main
    ++		create refs/heads/symrefc ref:refs/heads/main
    ++		update refs/heads/symrefu ref:refs/heads/branch ref:refs/heads/main
     +		prepare
     +		commit
     +	EOF


Karthik Nayak (8):
  refs: accept symref values in `ref_transaction[_add]_update`
  update-ref: support parsing ref targets in `parse_next_oid`
  files-backend: extract out `create_symref_lock`
  update-ref: support symrefs in the verify command
  update-ref: support symrefs in the delete command
  update-ref: support symrefs in the create command
  update-ref: support symrefs in the update command
  ref: support symrefs in 'reference-transaction' hook

 Documentation/git-update-ref.txt |  41 ++--
 Documentation/githooks.txt       |  14 +-
 branch.c                         |   2 +-
 builtin/clone.c                  |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   4 +-
 builtin/receive-pack.c           |   4 +-
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             | 106 ++++++++--
 refs.c                           |  78 +++++--
 refs.h                           |  23 +-
 refs/files-backend.c             | 141 +++++++++++--
 refs/refs-internal.h             |  20 ++
 refs/reftable-backend.c          |  49 ++++-
 sequencer.c                      |   9 +-
 t/t0600-reffiles-backend.sh      |  32 +++
 t/t1400-update-ref.sh            | 346 ++++++++++++++++++++++++++++++-
 t/t1416-ref-transaction-hooks.sh |  41 ++++
 walker.c                         |   2 +-
 20 files changed, 817 insertions(+), 105 deletions(-)

-- 
2.43.GIT

[PATCH v3 1/8] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `ref_transaction[_add]_update` functions obtain ref information and
flags to create a `ref_update` and add it to the transaction at hand.

To extend symref support in transactions, we need to also accept the
old and new ref targets and process it. In this commit, let's add the
required parameters to the function and modify all call sites.

The two parameters added are `new_target` and `old_target`. The
`new_target` is used to denote what the reference should point to when
the transaction is applied. Some functions allow this parameter to be
NULL, meaning that the reference is not changed.

The `old_target` denotes the value the reference must have before the
update. Some functions allow this parameter to be NULL, meaning that the
old value of the reference is not checked. A copy of this value is made
in the transaction.

The handling logic of these parameters will be added in consequent
commits as we add symref support to the existing 'git-update-ref'
commands.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 17 ++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 13 +++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 69 insertions(+), 24 deletions(-)

diff --git a/branch.c b/branch.c
index e4a738fc7b..48af4c3ceb 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index dc5a9d32dd..297dfb91a1 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5857d860db..66840b7c5b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e8d7df14b6..b150ef39a8 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..7690687b0e 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 9a33cb50b4..40a65fdebc 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index e46afbc46d..21fdbf6ac8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
diff --git a/refs.c b/refs.c
index 55d2e0b2cb..060a31616d 100644
--- a/refs.c
+++ b/refs.c
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && !is_null_oid(old_oid) && old_target)
+		BUG("Only one of old_oid and old_target should be non NULL");
+	if (new_oid && !is_null_oid(new_oid) && new_target)
+		BUG("Only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index d278775e08..c792e13a64 100644
--- a/refs.h
+++ b/refs.h
@@ -648,6 +648,15 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         update to point to. This takes precedence over new_oid when
+ *         set. If the reference is a regular reference, it will be
+ *         converted to a symbolic reference.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Will only be taken into account when the reference is a symbolic
+ *         reference.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +722,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +735,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index a098d14ea0..e4d0aa3d41 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..3040d4797c 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -124,6 +124,18 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set and the reference is a symbolic ref, check that the
+	 * reference previously pointed to this value.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +185,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 1cda48c504..6104471199 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 2c19846385..af1b25692b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -616,7 +616,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1248,7 +1248,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3764,8 +3764,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index c0fd632d92..1b3df43906 100644
--- a/walker.c
+++ b/walker.c
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
-- 
2.43.GIT

[PATCH v3 2/8] update-ref: support parsing ref targets in `parse_next_oid`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `parse_next_oid` is used for parsing the next oid present in the
input buffer. Extend this function to also parse refnames in the form
`ref:<ref_target>`. This will be used in the upcoming commits to add
symref support to the existing update-ref commands.

Since `parse_next_oid` now also parses refs apart from oids, we rename
the function to a more apt name `parse_next_arg`.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 builtin/update-ref.c | 58 ++++++++++++++++++++++++++++++++------------
 1 file changed, 42 insertions(+), 16 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 21fdbf6ac8..98ec356394 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -88,6 +88,11 @@ static char *parse_refname(const char **next)
  */
 #define PARSE_SHA1_ALLOW_EMPTY 0x02
 
+/*
+ * Parse refname targets using the ref:<ref_target> format.
+ */
+#define PARSE_REFNAME_TARGETS 0x04
+
 /*
  * Parse an argument separator followed by the next argument, if any.
  * If there is an argument, convert it to a SHA-1, write it to sha1,
@@ -95,10 +100,13 @@ static char *parse_refname(const char **next)
  * return 0.  If there is no argument at all (not even the empty
  * string), return 1 and leave *next unchanged.  If the value is
  * provided but cannot be converted to a SHA-1, die.  flags can
- * include PARSE_SHA1_OLD and/or PARSE_SHA1_ALLOW_EMPTY.
+ * include PARSE_SHA1_OLD and/or PARSE_SHA1_ALLOW_EMPTY and/or
+ * PARSE_REFNAME_TARGETS. When PARSE_REFNAME_TARGETS is set, parse
+ * the argument as `ref:<refname>` and store the refname into
+ * the target strbuf.
  */
-static int parse_next_oid(const char **next, const char *end,
-			  struct object_id *oid,
+static int parse_next_arg(const char **next, const char *end,
+			  struct object_id *oid, struct strbuf *target,
 			  const char *command, const char *refname,
 			  int flags)
 {
@@ -118,8 +126,17 @@ static int parse_next_oid(const char **next, const char *end,
 		(*next)++;
 		*next = parse_arg(*next, &arg);
 		if (arg.len) {
-			if (repo_get_oid(the_repository, arg.buf, oid))
-				goto invalid;
+			if (repo_get_oid(the_repository, arg.buf, oid)) {
+				const char *value;
+				if (flags & PARSE_REFNAME_TARGETS &&
+				    skip_prefix(arg.buf, "ref:", &value)) {
+					if (check_refname_format(value, REFNAME_ALLOW_ONELEVEL))
+						die("invalid ref format: %s", value);
+					strbuf_addstr(target, value);
+				} else {
+					goto invalid;
+				}
+			}
 		} else {
 			/* Without -z, an empty value means all zeros: */
 			oidclr(oid);
@@ -136,8 +153,17 @@ static int parse_next_oid(const char **next, const char *end,
 		*next += arg.len;
 
 		if (arg.len) {
-			if (repo_get_oid(the_repository, arg.buf, oid))
-				goto invalid;
+			if (repo_get_oid(the_repository, arg.buf, oid)) {
+				const char *value;
+				if (flags & PARSE_REFNAME_TARGETS &&
+				    skip_prefix(arg.buf, "ref:", &value)) {
+					if (check_refname_format(value, REFNAME_ALLOW_ONELEVEL))
+						die("invalid ref format: %s", value);
+					strbuf_addstr(target, value);
+				} else {
+					goto invalid;
+				}
+			}
 		} else if (flags & PARSE_SHA1_ALLOW_EMPTY) {
 			/* With -z, treat an empty value as all zeros: */
 			warning("%s %s: missing <new-oid>, treating as zero",
@@ -192,12 +218,12 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	if (!refname)
 		die("update: missing <ref>");
 
-	if (parse_next_oid(&next, end, &new_oid, "update", refname,
-			   PARSE_SHA1_ALLOW_EMPTY))
+	if (parse_next_arg(&next, end, &new_oid, NULL,
+			   "update", refname, PARSE_SHA1_ALLOW_EMPTY))
 		die("update %s: missing <new-oid>", refname);
 
-	have_old = !parse_next_oid(&next, end, &old_oid, "update", refname,
-				   PARSE_SHA1_OLD);
+	have_old = !parse_next_arg(&next, end, &old_oid, NULL,
+				   "update", refname, PARSE_SHA1_OLD);
 
 	if (*next != line_termination)
 		die("update %s: extra input: %s", refname, next);
@@ -225,7 +251,7 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	if (!refname)
 		die("create: missing <ref>");
 
-	if (parse_next_oid(&next, end, &new_oid, "create", refname, 0))
+	if (parse_next_arg(&next, end, &new_oid, NULL, "create", refname, 0))
 		die("create %s: missing <new-oid>", refname);
 
 	if (is_null_oid(&new_oid))
@@ -256,8 +282,8 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 	if (!refname)
 		die("delete: missing <ref>");
 
-	if (parse_next_oid(&next, end, &old_oid, "delete", refname,
-			   PARSE_SHA1_OLD)) {
+	if (parse_next_arg(&next, end, &old_oid, NULL,
+			   "delete", refname, PARSE_SHA1_OLD)) {
 		have_old = 0;
 	} else {
 		if (is_null_oid(&old_oid))
@@ -289,8 +315,8 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
 	if (!refname)
 		die("verify: missing <ref>");
 
-	if (parse_next_oid(&next, end, &old_oid, "verify", refname,
-			   PARSE_SHA1_OLD))
+	if (parse_next_arg(&next, end, &old_oid, NULL,
+			   "verify", refname, PARSE_SHA1_OLD))
 		oidclr(&old_oid);
 
 	if (*next != line_termination)
-- 
2.43.GIT

[PATCH v3 3/8] files-backend: extract out `create_symref_lock`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Split this into two individual functions `create_and_commit_symref` and
`create_symref_locked`. This way we can create the symref lock and
commit it at different times. This will be used to provide symref
support in `git-update-ref(1)`.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 40 +++++++++++++++++++++++++++-------------
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index e4d0aa3d41..2420dac2aa 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1920,26 +1920,39 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target)
 {
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	/* no error check; commit_ref will check ferror */
+	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
 	return 0;
 }
 
@@ -1960,7 +1973,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

[PATCH v3 4/8] update-ref: support symrefs in the verify command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

In the previous commits, we added the required base for adding symref
support to the transaction commands provided by 'git-update-ref(1)'.
Using them, extend the 'verify' command to support symrefs.

The 'verify' command allows users to verify if a provided symbolic
reference `<ref>` contains the provided `<old-oid>` without changing the
`<ref>`. Now we alternatively allow users to provide a
`ref:<old-target>` instead to verify if a symref targets the provided
target. Since we're checking for symbolic refs, this will only work with
the 'no-deref' mode. This is because any dereferenced symbolic ref will
point to an object and not a ref.

Add and use `null_new_value`, a helper function which is used to check
if there is a new_value in a reference update. The new value could
either be a symref target `new_target` or a OID `new_oid`. We also add
tests to test the command in both the regular stdin mode and also with
the '-z' flag.

We also disable the reference-transaction hook for symref-updates which
will be tackled in its own commit.

Add required tests for symref support in 'verify' while also adding
reflog checks for the pre-existing 'verify' tests.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt | 13 ++++--
 builtin/update-ref.c             | 14 ++++--
 refs.c                           | 30 ++++++++++--
 refs.h                           |  1 +
 refs/files-backend.c             | 43 +++++++++++++++++
 refs/refs-internal.h             |  7 +++
 refs/reftable-backend.c          | 21 ++++++++-
 t/t1400-update-ref.sh            | 79 +++++++++++++++++++++++++++++++-
 8 files changed, 194 insertions(+), 14 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 374a2ebd2b..9f8c059944 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -64,7 +64,7 @@ performs all modifications together.  Specify commands of the form:
 	update SP <ref> SP <new-oid> [SP <old-oid>] LF
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
-	verify SP <ref> [SP <old-oid>] LF
+	verify SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
 	option SP <opt> LF
 	start LF
 	prepare LF
@@ -85,7 +85,7 @@ quoting:
 	update SP <ref> NUL <new-oid> NUL [<old-oid>] NUL
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
-	verify SP <ref> NUL [<old-oid>] NUL
+	verify SP <ref> NUL [(<old-oid> | ref:<old-target>)] NUL
 	option SP <opt> NUL
 	start NUL
 	prepare NUL
@@ -95,6 +95,12 @@ quoting:
 In this format, use 40 "0" to specify a zero value, and use the empty
 string to specify a missing value.
 
+For commands which support it, substituting the <old-oid> value with
+ref:<old-target> will ensure that the <ref> targets the specified
+old-target before the update.  Similarly, substituting the <new-oid>
+with ref:<new-target> will ensure that the <ref> is a symbolic ref
+targeting the new-target after the update.
+
 In either format, values can be specified in any form that Git
 recognizes as an object name.  Commands in any other format or a
 repeated <ref> produce an error.  Command meanings are:
@@ -115,7 +121,8 @@ delete::
 
 verify::
 	Verify <ref> against <old-oid> but do not change it.  If
-	<old-oid> is zero or missing, the ref must not exist.
+	<old-oid> is zero or missing, the ref must not exist. For
+	verifying symbolic refs, provide ref:<old-target>.
 
 option::
 	Modify the behavior of the next command naming a <ref>.
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 98ec356394..246167e835 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -308,6 +308,7 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
 	struct strbuf err = STRBUF_INIT;
+	struct strbuf old_target = STRBUF_INIT;
 	char *refname;
 	struct object_id old_oid;
 
@@ -315,20 +316,27 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
 	if (!refname)
 		die("verify: missing <ref>");
 
-	if (parse_next_arg(&next, end, &old_oid, NULL,
-			   "verify", refname, PARSE_SHA1_OLD))
+	if (parse_next_arg(&next, end, &old_oid, &old_target,
+			   "verify", refname,
+			   PARSE_SHA1_OLD | PARSE_REFNAME_TARGETS))
 		oidclr(&old_oid);
 
+	if (old_target.len && !(update_flags & REF_NO_DEREF))
+		die("verify %s: cannot operate on symrefs in deref mode", refname);
+
 	if (*next != line_termination)
 		die("verify %s: extra input: %s", refname, next);
 
-	if (ref_transaction_verify(transaction, refname, &old_oid,
+	if (ref_transaction_verify(transaction, refname,
+				   old_target.len ? NULL : &old_oid,
+				   old_target.len ? old_target.buf : NULL,
 				   update_flags, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
 	free(refname);
 	strbuf_release(&err);
+	strbuf_release(&old_target);
 }
 
 static void report_ok(const char *command)
diff --git a/refs.c b/refs.c
index 060a31616d..0e1013b5ab 100644
--- a/refs.c
+++ b/refs.c
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((void *)transaction->updates[i]->old_target);
+		free((void *)transaction->updates[i]->new_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,9 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);
+	if (new_oid && flags & REF_HAVE_NEW)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if (old_oid && flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);
 	update->msg = normalize_reflog_message(msg);
 	return update;
@@ -1286,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -1325,14 +1332,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err)
 {
-	if (!old_oid)
-		BUG("verify called with old_oid set to NULL");
+	if (!old_target && !old_oid)
+		BUG("verify called with old_oid and old_target set to NULL");
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("verify cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
-				      NULL, NULL,
+				      NULL, old_target,
 				      flags, NULL, err);
 }
 
@@ -2349,6 +2359,12 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
 
+		/*
+		 * Skip reference transaction for symbolic refs.
+		 */
+		if (update->new_target || update->old_target)
+			continue;
+
 		strbuf_reset(&buf);
 		strbuf_addf(&buf, "%s %s %s\n",
 			    oid_to_hex(&update->old_oid),
@@ -2802,3 +2818,7 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int ref_update_is_null_new_value(struct ref_update *update) {
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
diff --git a/refs.h b/refs.h
index c792e13a64..27b9aeaf54 100644
--- a/refs.h
+++ b/refs.h
@@ -780,6 +780,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 2420dac2aa..53197fa3af 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2425,6 +2425,37 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+/*
+ * Check whether the REF_HAVE_OLD and old_target values stored in
+ * update are consistent with ref, which is the symbolic reference's
+ * current value. If everything is OK, return 0; otherwise, write an
+ * error message to err and return -1.
+ */
+static int check_old_target(struct ref_update *update, char *ref,
+			    struct strbuf *err)
+{
+	if (!(update->flags & REF_HAVE_OLD) ||
+	    !strcmp(update->old_target, ref))
+		return 0;
+
+	if (!strcmp(update->old_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference already exists",
+			    original_update_refname(update));
+	else if (!strcmp(ref, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    ref, update->old_target);
+
+	return -1;
+}
+
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2528,6 +2559,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
+			}
+
+			/*
+			 * For symref verification, we need to check the reference value
+			 * rather than the oid. If we're dealing with regular refs or we're
+			 * verifying a dereferenced symref, we then check the oid.
+			 */
+			if (update->old_target) {
+				if (check_old_target(update, referent.buf, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}
 			} else if (check_old_oid(update, &lock->old_oid, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto out;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3040d4797c..23e65f65e8 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -748,4 +748,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_is_null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 6104471199..a2474245aa 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -938,7 +938,26 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {
+				if (!strcmp(u->old_target, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "provided target is empty",
+						    original_update_refname(u));
+				else if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
+				ret = -1;
+				goto done;
+			}
+		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 					    "reference already exists"),
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index ec3443cc87..1f2b63755a 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -890,17 +890,23 @@ test_expect_success 'stdin update/create/verify combination works' '
 '
 
 test_expect_success 'stdin verify succeeds for correct value' '
+	test-tool ref-store main for-each-reflog-ent $m >before &&
 	git rev-parse $m >expect &&
 	echo "verify $m $m" >stdin &&
 	git update-ref --stdin <stdin &&
 	git rev-parse $m >actual &&
-	test_cmp expect actual
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent $m >after &&
+	test_cmp before after
 '
 
 test_expect_success 'stdin verify succeeds for missing reference' '
+	test-tool ref-store main for-each-reflog-ent $m >before &&
 	echo "verify refs/heads/missing $Z" >stdin &&
 	git update-ref --stdin <stdin &&
-	test_must_fail git rev-parse --verify -q refs/heads/missing
+	test_must_fail git rev-parse --verify -q refs/heads/missing &&
+	test-tool ref-store main for-each-reflog-ent $m >after &&
+	test_cmp before after
 '
 
 test_expect_success 'stdin verify treats no value as missing' '
@@ -1641,4 +1647,73 @@ test_expect_success PIPE 'transaction flushes status updates' '
 	test_cmp expected actual
 '
 
+create_stdin_buf () {
+	if test "$1" = "-z"
+	then
+		shift
+		printf "$F" "$@" >stdin
+	else
+		echo "$@" >stdin
+	fi
+}
+
+for type in "" "-z"
+do
+
+	test_expect_success "stdin ${type} verify symref fails without --no-deref" '
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "verify refs/heads/symref" "ref:$a" &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: verify refs/heads/symref: cannot operate on symrefs in deref mode" err
+	'
+
+	test_expect_success "stdin ${type} verify symref fails with too many arguments" '
+		create_stdin_buf ${type} "verify refs/heads/symref" "ref:$a" "ref:$a" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err  &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: ref:$a" err
+		else
+			grep "fatal: verify refs/heads/symref: extra input:  ref:$a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} verify symref succeeds for correct value" '
+		git symbolic-ref refs/heads/symref >expect &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
+		create_stdin_buf ${type} "verify refs/heads/symref" "ref:$a" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
+		test_cmp before after
+	'
+
+	test_expect_success "stdin ${type} verify symref succeeds for missing reference" '
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
+		create_stdin_buf ${type} "verify refs/heads/missing" "$Z" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		test_must_fail git rev-parse --verify -q refs/heads/missing &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
+		test_cmp before after
+	'
+
+	test_expect_success "stdin ${type} verify symref fails for wrong value" '
+		git symbolic-ref refs/heads/symref >expect &&
+		create_stdin_buf ${type} "verify refs/heads/symref" "$b" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} verify symref fails for mistaken null value" '
+		git symbolic-ref refs/heads/symref >expect &&
+		create_stdin_buf ${type} "verify refs/heads/symref" "$Z" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+done
+
 test_done
-- 
2.43.GIT

[PATCH v3 5/8] update-ref: support symrefs in the delete command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'delete' command in 'git-update-ref' allows users to delete `<ref>`
after verifying it exists with `<old-oid>`, if given. Extend this command
to alternatively take in `ref:<old-target>` which is used to verify if
the symbolic ref targets the provided `<old-target>` before deletion.
This will only work when used with the 'no-deref' mode as it doesn't
make sense to deref a symref during deletion.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt | 10 +++++---
 builtin/fetch.c                  |  2 +-
 builtin/receive-pack.c           |  3 ++-
 builtin/update-ref.c             | 18 +++++++++----
 refs.c                           | 12 ++++++---
 refs.h                           |  4 ++-
 refs/files-backend.c             |  2 +-
 refs/reftable-backend.c          |  2 +-
 t/t1400-update-ref.sh            | 44 ++++++++++++++++++++++++++++++++
 9 files changed, 79 insertions(+), 18 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 9f8c059944..f28b026cd7 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -63,7 +63,7 @@ performs all modifications together.  Specify commands of the form:
 
 	update SP <ref> SP <new-oid> [SP <old-oid>] LF
 	create SP <ref> SP <new-oid> LF
-	delete SP <ref> [SP <old-oid>] LF
+	delete SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
 	verify SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
 	option SP <opt> LF
 	start LF
@@ -84,7 +84,7 @@ quoting:
 
 	update SP <ref> NUL <new-oid> NUL [<old-oid>] NUL
 	create SP <ref> NUL <new-oid> NUL
-	delete SP <ref> NUL [<old-oid>] NUL
+	delete SP <ref> NUL [(<old-oid> | ref:<old-target>)] NUL
 	verify SP <ref> NUL [(<old-oid> | ref:<old-target>)] NUL
 	option SP <opt> NUL
 	start NUL
@@ -116,8 +116,10 @@ create::
 	exist.  The given <new-oid> may not be zero.
 
 delete::
-	Delete <ref> after verifying it exists with <old-oid>, if
-	given.  If given, <old-oid> may not be zero.
+	Delete <ref> after verifying it exists with <old-oid>, if given.
+	If given, <old-oid> may not be zero.  If instead, ref:<old-target>
+	is provided, verify that the symbolic ref <ref> targets
+	<old-target> before deleting it.
 
 verify::
 	Verify <ref> against <old-oid> but do not change it.  If
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 66840b7c5b..d02592efca 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1383,7 +1383,7 @@ static int prune_refs(struct display_state *display_state,
 		if (transaction) {
 			for (ref = stale_refs; ref; ref = ref->next) {
 				result = ref_transaction_delete(transaction, ref->name, NULL, 0,
-								"fetch: prune", &err);
+								NULL, "fetch: prune", &err);
 				if (result)
 					goto cleanup;
 			}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b150ef39a8..9a4667d57d 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1576,7 +1576,8 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_delete(transaction,
 					   namespaced_name,
 					   old_oid,
-					   0, "push", &err)) {
+					   0, NULL,
+					   "push", &err)) {
 			rp_error("%s", err.buf);
 			ret = "failed to delete";
 		} else {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 246167e835..cee7a5ebc0 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -274,6 +274,7 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
 	struct strbuf err = STRBUF_INIT;
+	struct strbuf old_target = STRBUF_INIT;
 	char *refname;
 	struct object_id old_oid;
 	int have_old;
@@ -282,26 +283,33 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 	if (!refname)
 		die("delete: missing <ref>");
 
-	if (parse_next_arg(&next, end, &old_oid, NULL,
-			   "delete", refname, PARSE_SHA1_OLD)) {
+	if (parse_next_arg(&next, end, &old_oid, &old_target,
+			   "delete", refname, PARSE_SHA1_OLD |
+			   PARSE_REFNAME_TARGETS)) {
 		have_old = 0;
 	} else {
-		if (is_null_oid(&old_oid))
+		if (!old_target.len && is_null_oid(&old_oid))
 			die("delete %s: zero <old-oid>", refname);
-		have_old = 1;
+		have_old = 1 && !old_target.len;
 	}
 
+	if (old_target.len && !(update_flags & REF_NO_DEREF))
+		die("delete %s: cannot operate on symrefs in deref mode", refname);
+
 	if (*next != line_termination)
 		die("delete %s: extra input: %s", refname, next);
 
 	if (ref_transaction_delete(transaction, refname,
 				   have_old ? &old_oid : NULL,
-				   update_flags, msg, &err))
+				   update_flags,
+				   old_target.len ? old_target.buf : NULL,
+				   msg, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
 	free(refname);
 	strbuf_release(&err);
+	strbuf_release(&old_target);
 }
 
 static void parse_cmd_verify(struct ref_transaction *transaction,
diff --git a/refs.c b/refs.c
index 0e1013b5ab..6b7c46bfd8 100644
--- a/refs.c
+++ b/refs.c
@@ -979,7 +979,7 @@ int refs_delete_ref(struct ref_store *refs, const char *msg,
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, refname, old_oid,
-				   flags, msg, &err) ||
+				   flags, NULL, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ref_transaction_free(transaction);
@@ -1318,14 +1318,18 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_target,
+			   const char *msg,
 			   struct strbuf *err)
 {
 	if (old_oid && is_null_oid(old_oid))
 		BUG("delete called with old_oid set to zeros");
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("delete cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      NULL, NULL, flags,
+				      NULL, old_target, flags,
 				      msg, err);
 }
 
@@ -2752,7 +2756,7 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg,
 
 	for_each_string_list_item(item, refnames) {
 		ret = ref_transaction_delete(transaction, item->string,
-					     NULL, flags, msg, &err);
+					     NULL, flags, NULL, msg, &err);
 		if (ret) {
 			warning(_("could not delete reference %s: %s"),
 				item->string, err.buf);
diff --git a/refs.h b/refs.h
index 27b9aeaf54..4be4930f04 100644
--- a/refs.h
+++ b/refs.h
@@ -766,7 +766,9 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_target,
+			   const char *msg,
 			   struct strbuf *err);
 
 /*
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 53197fa3af..fc5037fe5a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2516,7 +2516,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && ref_update_is_null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index a2474245aa..2b2cbca8c0 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1120,7 +1120,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 1f2b63755a..cd1ad0d2ec 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1714,6 +1714,50 @@ do
 		test_cmp expect actual
 	'
 
+	test_expect_success "stdin ${type} delete symref fails without --no-deref" '
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "delete refs/heads/symref" "ref:$a" &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: delete refs/heads/symref: cannot operate on symrefs in deref mode" err
+	'
+
+	test_expect_success "stdin ${type} delete symref fails with no ref" '
+		create_stdin_buf ${type} "delete " &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		grep "fatal: delete: missing <ref>" err
+	'
+
+	test_expect_success "stdin ${type} delete symref fails with too many arguments" '
+		create_stdin_buf ${type} "delete refs/heads/symref" "ref:$a" "ref:$a" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: ref:$a" err
+		else
+			grep "fatal: delete refs/heads/symref: extra input:  ref:$a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} delete symref fails with wrong old value" '
+		create_stdin_buf ${type} "delete refs/heads/symref" "ref:$m" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test_have_prereq REFTABLE
+		then
+			grep "fatal: verifying symref target: ${SQ}refs/heads/symref${SQ}: is at $a but expected refs/heads/main" err
+		else
+			grep "fatal: cannot lock ref ${SQ}refs/heads/symref${SQ}" err
+		fi &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} delete symref works with right old value" '
+		create_stdin_buf ${type} "delete refs/heads/symref" "ref:$a" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		test_must_fail git rev-parse --verify -q $b
+	'
+
 done
 
 test_done
-- 
2.43.GIT

[PATCH v3 6/8] update-ref: support symrefs in the create command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'create' commands in 'git-update-ref' allows users to create `<ref>`
with `<new-oid>` after verifying it does not exist. Extend this command
to alternatively take in `ref:<new-target>` which is used to create a
symref with `<new-target>` as its target.

Also, support the 'core.prefersymlinkrefs' config, wherein if the config
is set and the filesystem supports symlinks, we create the symbolic ref
as a symlink. We fallback to creating a regular symref if creating the
symlink is unsuccessful.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |  8 +++--
 builtin/clone.c                  |  2 +-
 builtin/update-ref.c             | 14 ++++++--
 refs.c                           |  9 ++++--
 refs.h                           |  1 +
 refs/files-backend.c             | 42 ++++++++++++++++++++++++
 refs/reftable-backend.c          | 23 +++++++++++--
 t/t0600-reffiles-backend.sh      | 32 +++++++++++++++++++
 t/t1400-update-ref.sh            | 55 ++++++++++++++++++++++++++++++++
 9 files changed, 173 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index f28b026cd7..1202769178 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -62,7 +62,7 @@ With `--stdin`, update-ref reads instructions from standard input and
 performs all modifications together.  Specify commands of the form:
 
 	update SP <ref> SP <new-oid> [SP <old-oid>] LF
-	create SP <ref> SP <new-oid> LF
+	create SP <ref> SP (<new-oid> | ref:<new-target>) LF
 	delete SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
 	verify SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
 	option SP <opt> LF
@@ -83,7 +83,7 @@ Alternatively, use `-z` to specify in NUL-terminated format, without
 quoting:
 
 	update SP <ref> NUL <new-oid> NUL [<old-oid>] NUL
-	create SP <ref> NUL <new-oid> NUL
+	create SP <ref> NUL (<new-oid> | ref:<new-target>) NUL
 	delete SP <ref> NUL [(<old-oid> | ref:<old-target>)] NUL
 	verify SP <ref> NUL [(<old-oid> | ref:<old-target>)] NUL
 	option SP <opt> NUL
@@ -113,7 +113,9 @@ update::
 
 create::
 	Create <ref> with <new-oid> after verifying it does not
-	exist.  The given <new-oid> may not be zero.
+	exist.  The given <new-oid> may not be zero.  If instead
+	ref:<new-target> is provided, a symbolic ref is created
+	which targets <new-target>.
 
 delete::
 	Delete <ref> after verifying it exists with <old-oid>, if given.
diff --git a/builtin/clone.c b/builtin/clone.c
index 74ec14542e..c0eed8e795 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -547,7 +547,7 @@ static void write_remote_refs(const struct ref *local_refs)
 		if (!r->peer_ref)
 			continue;
 		if (ref_transaction_create(t, r->peer_ref->name, &r->old_oid,
-					   0, NULL, &err))
+					   NULL, 0, NULL, &err))
 			die("%s", err.buf);
 	}
 
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index cee7a5ebc0..afab706cd7 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -244,6 +244,7 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
 	struct strbuf err = STRBUF_INIT;
+	struct strbuf new_target = STRBUF_INIT;
 	char *refname;
 	struct object_id new_oid;
 
@@ -251,16 +252,22 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	if (!refname)
 		die("create: missing <ref>");
 
-	if (parse_next_arg(&next, end, &new_oid, NULL, "create", refname, 0))
+	if (parse_next_arg(&next, end, &new_oid, &new_target,
+			   "create", refname, PARSE_REFNAME_TARGETS))
 		die("create %s: missing <new-oid>", refname);
 
-	if (is_null_oid(&new_oid))
+	if (!new_target.len && is_null_oid(&new_oid))
 		die("create %s: zero <new-oid>", refname);
 
+	if (new_target.len && !(update_flags & REF_NO_DEREF))
+		die("create %s: cannot create symrefs in deref mode", refname);
+
 	if (*next != line_termination)
 		die("create %s: extra input: %s", refname, next);
 
-	if (ref_transaction_create(transaction, refname, &new_oid,
+	if (ref_transaction_create(transaction, refname,
+				   new_target.len ? NULL : &new_oid ,
+				   new_target.len ? new_target.buf : NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
@@ -268,6 +275,7 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	update_flags = default_flags;
 	free(refname);
 	strbuf_release(&err);
+	strbuf_release(&new_target);
 }
 
 static void parse_cmd_delete(struct ref_transaction *transaction,
diff --git a/refs.c b/refs.c
index 6b7c46bfd8..42cb4126a7 100644
--- a/refs.c
+++ b/refs.c
@@ -1303,15 +1303,18 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
-	if (!new_oid || is_null_oid(new_oid)) {
-		strbuf_addf(err, "'%s' has a null OID", refname);
+	if ((!new_oid || is_null_oid(new_oid)) && !new_target) {
+		strbuf_addf(err, "'%s' has a null OID or no new target", refname);
 		return 1;
 	}
+	if (new_target && !(flags & REF_NO_DEREF))
+		BUG("create cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), NULL, NULL, flags,
+				      null_oid(), new_target, NULL, flags,
 				      msg, err);
 }
 
diff --git a/refs.h b/refs.h
index 4be4930f04..bde8606213 100644
--- a/refs.h
+++ b/refs.h
@@ -752,6 +752,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index fc5037fe5a..f5e271a442 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2610,6 +2610,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
+	if (update->new_target) {
+		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		update->flags |= REF_NEEDS_COMMIT;
+	}
+
+
 	if ((update->flags & REF_HAVE_NEW) &&
 	    !(update->flags & REF_DELETING) &&
 	    !(update->flags & REF_LOG_ONLY)) {
@@ -2905,6 +2926,18 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
+			if (update->new_target) {
+				/*
+				 * We want to get the resolved OID for the target, to ensure
+				 * that the correct value is added to the reflog.
+				 */
+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
+					/* for dangling symrefs we gracefully set the oid to zero */
+					update->new_oid = *null_oid();
+				}
+			}
+
 			if (files_log_ref_write(refs,
 						lock->ref_name,
 						&lock->old_oid,
@@ -2922,6 +2955,15 @@ static int files_transaction_finish(struct ref_store *ref_store,
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next updated. If not, we try and create a regular symref.
+		 */
+		if (update->new_target && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->new_target))
+				continue;
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 2b2cbca8c0..e203c697f2 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -856,7 +856,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if (u->flags & REF_HAVE_NEW && !ref_update_is_null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -1062,7 +1062,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && ref_update_is_null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1104,6 +1104,12 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
 
+			if (u->new_target)
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
+							     RESOLVE_REF_READING, &u->new_oid, NULL))
+					/* for dangling symrefs we gracefully set the oid to zero */
+					u->new_oid = *null_oid();
+
 			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
 			log = &logs[logs_nr++];
 			memset(log, 0, sizeof(*log));
@@ -1120,7 +1126,18 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
+		if (u->flags & REF_HAVE_NEW && u->new_target) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_target,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh
index 64214340e7..1291242940 100755
--- a/t/t0600-reffiles-backend.sh
+++ b/t/t0600-reffiles-backend.sh
@@ -472,4 +472,36 @@ test_expect_success POSIXPERM 'git reflog expire honors core.sharedRepository' '
 	esac
 '
 
+test_expect_success SYMLINKS 'symref transaction supports symlinks' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs true &&
+	cat >stdin <<-EOF &&
+	start
+	create TESTSYMREFONE ref:refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_symlink .git/TESTSYMREFONE &&
+	test "$(test_readlink .git/TESTSYMREFONE)" = refs/heads/new
+'
+
+test_expect_success 'symref transaction supports false symlink config' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs false &&
+	cat >stdin <<-EOF &&
+	start
+	create TESTSYMREFONE ref:refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_file .git/TESTSYMREFONE &&
+	git symbolic-ref TESTSYMREFONE >actual &&
+	echo refs/heads/new >expect &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index cd1ad0d2ec..e85d08ce5c 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1758,6 +1758,61 @@ do
 		test_must_fail git rev-parse --verify -q $b
 	'
 
+	test_expect_success "stdin ${type} create symref fails without --no-deref" '
+		create_stdin_buf ${type} "create refs/heads/symref" "ref:$a" &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: create refs/heads/symref: cannot create symrefs in deref mode" err
+	'
+
+	test_expect_success "stdin ${type} create symref fails with too many arguments" '
+		create_stdin_buf ${type} "create refs/heads/symref" "ref:$a" "ref:$a" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: ref:$a" err
+		else
+			grep "fatal: create refs/heads/symref: extra input:  ref:$a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} create symref ref works" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "create refs/heads/symref" "ref:$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} create dangling symref ref works" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "create refs/heads/symref" "ref:refs/heads/unkown" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo refs/heads/unkown >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} create symref does not create reflogs by default" '
+		test_when_finished "git symbolic-ref -d refs/symref" &&
+		create_stdin_buf ${type} "create refs/symref" "ref:$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual &&
+		test_must_fail git reflog exists refs/symref
+	'
+
+	test_expect_success "stdin ${type} create symref reflogs with --create-reflog" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "create refs/heads/symref" "ref:$a" >stdin &&
+		git update-ref --create-reflog --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual &&
+		git reflog exists refs/heads/symref
+	'
+
 done
 
 test_done
-- 
2.43.GIT

[PATCH v3 7/8] update-ref: support symrefs in the update command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'update' command in 'git-update-ref' allows users to set `<ref>` to
`<new-oid>` after verifying `<old-oid>`, if given. Extend this command
to alternatively take in `ref:<new-target>` which is used to update to/a
symref with `<new-target>` as its target. And take in `ref:<old-target>`
which is used to verify that the symref points to `<old-target>` before
the update.

With this the 'update' command can also now be used to:
- create symrefs
- change a regular ref to a symref
- change a symref to a regular ref

This command will also support deref mode, to ensure that we can update
dereferenced regular refs to symrefs.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |  10 +-
 builtin/update-ref.c             |  21 ++--
 refs/files-backend.c             |  14 ++-
 refs/reftable-backend.c          |   3 +-
 t/t1400-update-ref.sh            | 168 +++++++++++++++++++++++++++++++
 5 files changed, 198 insertions(+), 18 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 1202769178..79e29fead6 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -61,7 +61,7 @@ still contains <old-oid>.
 With `--stdin`, update-ref reads instructions from standard input and
 performs all modifications together.  Specify commands of the form:
 
-	update SP <ref> SP <new-oid> [SP <old-oid>] LF
+	update SP <ref> SP (<new-oid> | ref:<new-target>) [SP (<old-oid> | ref:<old-target>)] LF
 	create SP <ref> SP (<new-oid> | ref:<new-target>) LF
 	delete SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
 	verify SP <ref> [SP (<old-oid> | ref:<old-target>)] LF
@@ -82,7 +82,7 @@ specify a missing value, omit the value and its preceding SP entirely.
 Alternatively, use `-z` to specify in NUL-terminated format, without
 quoting:
 
-	update SP <ref> NUL <new-oid> NUL [<old-oid>] NUL
+	update SP <ref> NUL (<new-oid> | ref:<new-target>) NUL [(<old-oid> | ref:<old-target>)] NUL
 	create SP <ref> NUL (<new-oid> | ref:<new-target>) NUL
 	delete SP <ref> NUL [(<old-oid> | ref:<old-target>)] NUL
 	verify SP <ref> NUL [(<old-oid> | ref:<old-target>)] NUL
@@ -109,7 +109,11 @@ update::
 	Set <ref> to <new-oid> after verifying <old-oid>, if given.
 	Specify a zero <new-oid> to ensure the ref does not exist
 	after the update and/or a zero <old-oid> to make sure the
-	ref does not exist before the update.
+	ref does not exist before the update.  If ref:<old-target>
+	is provided, we verify that the <ref> is an existing symbolic
+	ref which targets <old-target>.  If ref:<new-target> is given,
+	the update ensures <ref> is a symbolic ref which targets
+	<new-target>.
 
 create::
 	Create <ref> with <new-oid> after verifying it does not
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index afab706cd7..175579148f 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -210,6 +210,8 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
 	struct strbuf err = STRBUF_INIT;
+	struct strbuf new_target = STRBUF_INIT;
+	struct strbuf old_target = STRBUF_INIT;
 	char *refname;
 	struct object_id new_oid, old_oid;
 	int have_old;
@@ -218,19 +220,24 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	if (!refname)
 		die("update: missing <ref>");
 
-	if (parse_next_arg(&next, end, &new_oid, NULL,
-			   "update", refname, PARSE_SHA1_ALLOW_EMPTY))
+	if (parse_next_arg(&next, end, &new_oid,
+			   &new_target, "update", refname,
+			   PARSE_SHA1_ALLOW_EMPTY | PARSE_REFNAME_TARGETS))
 		die("update %s: missing <new-oid>", refname);
 
-	have_old = !parse_next_arg(&next, end, &old_oid, NULL,
-				   "update", refname, PARSE_SHA1_OLD);
+	have_old = !parse_next_arg(&next, end, &old_oid,
+				   &old_target, "update", refname,
+				   PARSE_SHA1_OLD | PARSE_REFNAME_TARGETS);
+	have_old = have_old & !old_target.len;
 
 	if (*next != line_termination)
 		die("update %s: extra input: %s", refname, next);
 
 	if (ref_transaction_update(transaction, refname,
-				   &new_oid, have_old ? &old_oid : NULL,
-				   NULL, NULL,
+				   new_target.len ? NULL : &new_oid,
+				   have_old ? &old_oid : NULL,
+				   new_target.len ? new_target.buf : NULL,
+				   old_target.len ? old_target.buf : NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
@@ -238,6 +245,8 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	update_flags = default_flags;
 	free(refname);
 	strbuf_release(&err);
+	strbuf_release(&old_target);
+	strbuf_release(&new_target);
 }
 
 static void parse_cmd_create(struct ref_transaction *transaction,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f5e271a442..59d1ab3eeb 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2386,7 +2386,8 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target, update->old_target,
+			update->msg);
 
 	new_update->parent_update = update;
 
@@ -2610,7 +2611,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if (update->new_target) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
 		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
@@ -2628,12 +2629,9 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		 * phase of the transaction only needs to commit the lock.
 		 */
 		update->flags |= REF_NEEDS_COMMIT;
-	}
-
-
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index e203c697f2..a00f55802a 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -908,7 +908,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+						&u->new_oid, &u->old_oid, u->new_target,
+						u->old_target, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index e85d08ce5c..5b2d23da37 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1360,6 +1360,7 @@ test_expect_success 'fails with duplicate HEAD update' '
 '
 
 test_expect_success 'fails with duplicate ref update via symref' '
+	test_when_finished "git symbolic-ref -d refs/heads/symref2" &&
 	git branch target2 $A &&
 	git symbolic-ref refs/heads/symref2 refs/heads/target2 &&
 	cat >stdin <<-EOF &&
@@ -1813,6 +1814,173 @@ do
 		git reflog exists refs/heads/symref
 	'
 
+	test_expect_success "stdin ${type} update symref fails with too many arguments" '
+		create_stdin_buf ${type} "update refs/heads/symref" "ref:$a" "ref:$a" "ref:$a" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: ref:$a" err
+		else
+			grep "fatal: update refs/heads/symref: extra input:  ref:$a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} update creates symref with zero old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "update refs/heads/symref" "ref:$a" "$Z" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $a >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} update creates symref with empty old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "update refs/heads/symref" "ref:$a" "" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $a >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} update symref fails with wrong old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "update refs/heads/symref" "ref:$m" "ref:$b" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test_have_prereq REFTABLE
+		then
+			grep "fatal: verifying symref target: ${SQ}refs/heads/symref${SQ}: is at $a but expected $b" err
+		else
+			grep "fatal: cannot lock ref ${SQ}refs/heads/symref${SQ}: is at $a but expected $b" err
+		fi &&
+		test_must_fail git rev-parse --verify -q $c
+	'
+
+	test_expect_success "stdin ${type} update symref works with right old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "update refs/heads/symref" "ref:$m" "ref:$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $m >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} update creates symref (with deref)" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "update refs/heads/symref" "ref:$a" "" >stdin &&
+		git update-ref --stdin ${type} <stdin &&
+		echo $a >expect &&
+		git symbolic-ref --no-recurse refs/heads/symref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+		grep "$Z $(git rev-parse $a)" actual
+	'
+
+	test_expect_success "stdin ${type} update regular ref to symref with correct old-oid" '
+		test_when_finished "git symbolic-ref -d --no-recurse refs/heads/regularref" &&
+		git update-ref --no-deref refs/heads/regularref $a &&
+		create_stdin_buf ${type} "update refs/heads/regularref" "ref:$a" "$(git rev-parse $a)" >stdin &&
+		git update-ref --stdin ${type} <stdin &&
+		echo $a >expect &&
+		git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+		grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+	'
+
+	test_expect_success "stdin ${type} update regular ref to symref fails with wrong old-oid" '
+		test_when_finished "git update-ref -d refs/heads/regularref" &&
+		git update-ref --no-deref refs/heads/regularref $a &&
+		create_stdin_buf ${type} "update refs/heads/regularref" "ref:$a" "$(git rev-parse refs/heads/target2)" >stdin &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		echo $(git rev-parse $a) >expect &&
+		git rev-parse refs/heads/regularref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} update existing symref with zero old-oid" '
+		test_when_finished "git symbolic-ref -d --no-recurse refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref refs/heads/target2 &&
+		create_stdin_buf ${type} "update refs/heads/symref" "ref:$a" "$Z" >stdin &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: cannot lock ref ${SQ}refs/heads/symref${SQ}: reference already exists" err &&
+		echo refs/heads/target2 >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} update existing symref to regular ref" '
+		test_when_finished "git update-ref -d refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref refs/heads/target2 &&
+		create_stdin_buf ${type} "update refs/heads/symref" "$(git rev-parse $a)" "ref:refs/heads/target2" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $(git rev-parse $a) >expect &&
+		git rev-parse refs/heads/symref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+		grep "$(git rev-parse refs/heads/target2) $(git rev-parse $a)" actual
+	'
+
 done
 
+test_expect_success "stdin update symref (with deref)" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	test_when_finished "git update-ref -d --no-deref refs/heads/symref2" &&
+	git update-ref refs/heads/symref2 $a &&
+	git symbolic-ref --no-recurse refs/heads/symref refs/heads/symref2 &&
+	echo "update refs/heads/symref" "ref:$a" >stdin &&
+	git update-ref --stdin <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref2 >actual &&
+	test_cmp expect actual &&
+	echo refs/heads/symref2 >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success "stdin -z update symref (with deref)" '
+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+	test_when_finished "git update-ref -d --no-deref refs/heads/symref2" &&
+	git update-ref refs/heads/symref2 $a &&
+	git symbolic-ref --no-recurse refs/heads/symref refs/heads/symref2 &&
+	printf "$F" "update refs/heads/symref" "ref:$a" "" >stdin &&
+	git update-ref --stdin -z <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref2 >actual &&
+	test_cmp expect actual &&
+	echo refs/heads/symref2 >expect &&
+	git symbolic-ref --no-recurse refs/heads/symref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success "stdin update regular ref to symref" '
+	test_when_finished "git symbolic-ref -d --no-recurse refs/heads/regularref" &&
+	git update-ref --no-deref refs/heads/regularref $a &&
+	echo "update refs/heads/regularref" "ref:$a" >stdin &&
+	git update-ref --stdin <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
+test_expect_success "stdin -z update regular ref to symref" '
+	test_when_finished "git symbolic-ref -d --no-recurse refs/heads/regularref" &&
+	git update-ref --no-deref refs/heads/regularref $a &&
+	printf "$F" "update refs/heads/regularref" "ref:$a" "" >stdin &&
+	git update-ref --stdin -z <stdin &&
+	echo $a >expect &&
+	git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+	grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+'
+
 test_done
-- 
2.43.GIT

[PATCH v3 8/8] ref: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In the previous commits, we added symref support for
various commands in `git-update-ref`. While it allowed us to now
manipulate symbolic refs via `git-update-ref`, it didn't activate the
'reference-transaction' hook.

Let's activate the hook for symbolic reference updates too. There is no
new format described for this and we stick to the existing format of:
    <old-value> SP <new-value> SP <ref-name> LF
but now, <old-value> and <new-value> could also denote references
instead of objects, where the format is similar to that in
'git-update-ref', i.e. 'ref:<ref-target>'.

While this seems to be backward incompatible, it is okay, since the only
way the `reference-transaction` hook has refs in its output is when
`git-update-ref` is used to manipulate symrefs. Also the documentation
for reference-transaction hook always stated that support for symbolic
references may be added in the future.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/githooks.txt       | 14 +++++++----
 refs.c                           | 21 ++++++++--------
 t/t1416-ref-transaction-hooks.sh | 41 ++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 15 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ee9b92c90d..0bf8ca87a6 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also cover symbolic references.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects, denoted via the
+`ref:<ref-target>` format.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index 42cb4126a7..9a510744a7 100644
--- a/refs.c
+++ b/refs.c
@@ -2365,18 +2365,19 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		strbuf_reset(&buf);
 
-		/*
-		 * Skip reference transaction for symbolic refs.
-		 */
-		if (update->new_target || update->old_target)
-			continue;
+		if (update->flags & REF_HAVE_OLD && update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
 
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+		if (update->flags & REF_HAVE_NEW && update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..abd4777819 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -134,4 +134,45 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+# This test doesn't add a check for symref 'delete' since there is a
+# variation between the ref backends WRT 'delete'. In the files backend,
+# 'delete' also triggers an additional transaction update on the
+# packed-refs backend, which constitutes additional reflog entries.
+test_expect_success 'hook gets all queued symref updates' '
+	test_when_finished "rm actual" &&
+
+	git update-ref refs/heads/branch $POST_OID &&
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+	git symbolic-ref refs/heads/symrefu refs/heads/main &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	cat >expect <<-EOF &&
+		prepared
+		ref:refs/heads/main $ZERO_OID refs/heads/symref
+		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
+		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
+		committed
+		ref:refs/heads/main $ZERO_OID refs/heads/symref
+		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
+		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
+	EOF
+
+	git update-ref --no-deref --stdin <<-EOF &&
+		start
+		verify refs/heads/symref ref:refs/heads/main
+		create refs/heads/symrefc ref:refs/heads/main
+		update refs/heads/symrefu ref:refs/heads/branch ref:refs/heads/main
+		prepare
+		commit
+	EOF
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Jeff King]

On Tue, Apr 23, 2024 at 11:28:10PM +0200, Karthik Nayak wrote:

> Changes from v2:
> 1. We no longer have separate commands for symrefs, instead the regular
> commands learn to parse 'ref:<target>' as symref targets. This reduces
> the code in this series. Thanks Patrick for the suggestion.

Hmm. I can see how this makes things a lot simpler, but it introduces an
ambiguity, since you can pass full ref expressions to "update-ref" (like
"ref:foo" to find the "foo" entry in ref^{tree}). I see that you only
kick in the symref "ref:" handling if the regular oid lookup failed, so
there's no backwards-compatibility issue (anything that used to work
will still take precedence, and the new code only runs when the old code
would have reported an error).

But I wonder if it would let somebody cause mischief in a repository
they can push to, but which may get updates from other sources. For
example, imagine a forge like GitLab runs the equivalent of:

  echo "create refs/whatever ref:refs/heads/main" |
  git update-ref --stdin

as part of some system process. Now if I push up "refs/heads/ref" that
contains the path "refs/heads/main" in its tree, that will take
precedence, causing the system process to do something it did not
expect.

I think you'd have to pile on a lot of assumptions to get any kind of
security problem. Something like:

 1. The system has a hidden ref namespace like refs/gitlab that normal
    remote push/fetch users are not allowed to read/write to.

 2. The system tries to make a symlink within that namespace. Say,
    "refs/gitlab/metadata/HEAD" to point to
    "refs/gitlab/metadata/branches/main" or something.

 3. The user pushes up "refs/heads/ref" with a tree that contains
    "refs/gitlab/metadata/branches/main". Now when (2) happens, the
    hidden ref points to user-controlled data.

That's pretty convoluted. But we can avoid it entirely if there's no
ambiguity in the protocol at all.

-Peff

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> That's pretty convoluted. But we can avoid it entirely if there's no
> ambiguity in the protocol at all.

;-).

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 4177 bytes --]

Jeff King <peff@peff.net> writes:

> On Tue, Apr 23, 2024 at 11:28:10PM +0200, Karthik Nayak wrote:
>
>> Changes from v2:
>> 1. We no longer have separate commands for symrefs, instead the regular
>> commands learn to parse 'ref:<target>' as symref targets. This reduces
>> the code in this series. Thanks Patrick for the suggestion.
>
> Hmm. I can see how this makes things a lot simpler, but it introduces an
> ambiguity, since you can pass full ref expressions to "update-ref" (like
> "ref:foo" to find the "foo" entry in ref^{tree}). I see that you only
> kick in the symref "ref:" handling if the regular oid lookup failed, so
> there's no backwards-compatibility issue (anything that used to work
> will still take precedence, and the new code only runs when the old code
> would have reported an error).
>
> But I wonder if it would let somebody cause mischief in a repository
> they can push to, but which may get updates from other sources. For
> example, imagine a forge like GitLab runs the equivalent of:
>
>   echo "create refs/whatever ref:refs/heads/main" |
>   git update-ref --stdin
>
> as part of some system process. Now if I push up "refs/heads/ref" that
> contains the path "refs/heads/main" in its tree, that will take
> precedence, causing the system process to do something it did not
> expect.
>
> I think you'd have to pile on a lot of assumptions to get any kind of
> security problem. Something like:
>
>  1. The system has a hidden ref namespace like refs/gitlab that normal
>     remote push/fetch users are not allowed to read/write to.
>
>  2. The system tries to make a symlink within that namespace. Say,
>     "refs/gitlab/metadata/HEAD" to point to
>     "refs/gitlab/metadata/branches/main" or something.
>
>  3. The user pushes up "refs/heads/ref" with a tree that contains
>     "refs/gitlab/metadata/branches/main". Now when (2) happens, the
>     hidden ref points to user-controlled data.
>
> That's pretty convoluted. But we can avoid it entirely if there's no
> ambiguity in the protocol at all.
>
> -Peff


Thanks Peff, that is indeed something I totally missed thinking about.

This also brings light onto the previous versions we were considering:

    symref-update SP <ref> SP <new-target> [SP (<old-target> | <old-oid>)] LF

There is also some ambiguity here which we missed, especially when we
support dangling refs. If we're updating a dangling ref <ref>, and we
provide an old value. Then there is uncertainty around whether the
provided value is actually a <old-target> or if it's an <old-oid>.

For non dangling ref symref, we first parse it as an <old-target> and
since the <old-target> would exist, we can move on.

So I see two ways to go about this,

1. In the symref-update function, we need to parse and see if <ref> is a
regular ref or a symref, if it is symref, we simply set the provided old
value as <old-target>, if not, we set it as <old-oid>. This seems clunky
because we'll be parsing the ref and trying to understand its type in
'update-ref.c', before the actual update.

2. We change the syntax to something like

    symref-update SP <ref> SP <new-ref> [SP (ref <old-target> | oid
<old-oid>)] LF

this would remove any ambiguity since the user specifies the data type
they're providing.

Overall, I think it is best to discard this version and I will push v4
with the older schematics of introducing new commands.

I'm currently considering going ahead with the [2], but will wait for a
day or two to consider other opinions.

Also on a sidenote, it's worth considering that with the direction of
[2], we could also extrapolate to introduce {verify, update, create,
delete} v2, which support both symrefs and regular refs. But require
explicit types from the user:

    update-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
[(oid <old-oid> | ref <old-target>)] NUL
	create-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
	delete-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL
	verify-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL

This is similar to the v3 patches I've currently sent out, in that it
would also allow cross operations between regular refs and symrefs.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

On Wed, Apr 24, 2024 at 09:25:27AM -0700, Karthik Nayak wrote:
> Jeff King <peff@peff.net> writes:
> > On Tue, Apr 23, 2024 at 11:28:10PM +0200, Karthik Nayak wrote:
[snip]
> Also on a sidenote, it's worth considering that with the direction of
> [2], we could also extrapolate to introduce {verify, update, create,
> delete} v2, which support both symrefs and regular refs. But require
> explicit types from the user:
> 
>     update-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
> [(oid <old-oid> | ref <old-target>)] NUL
> 	create-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
> 	delete-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL
> 	verify-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL
> 
> This is similar to the v3 patches I've currently sent out, in that it
> would also allow cross operations between regular refs and symrefs.

One could put this new syntax behind a feature flag to avoid the "-v2"
suffixes, e.g. `git update-ref --with-symrefs`. But I'm not sure whether
this would be beneficial.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Wed, Apr 24, 2024 at 09:25:27AM -0700, Karthik Nayak wrote:
>> Jeff King <peff@peff.net> writes:
>> > On Tue, Apr 23, 2024 at 11:28:10PM +0200, Karthik Nayak wrote:
> [snip]
>> Also on a sidenote, it's worth considering that with the direction of
>> [2], we could also extrapolate to introduce {verify, update, create,
>> delete} v2, which support both symrefs and regular refs. But require
>> explicit types from the user:
>>
>>     update-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
>> [(oid <old-oid> | ref <old-target>)] NUL
>> 	create-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
>> 	delete-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL
>> 	verify-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL
>>
>> This is similar to the v3 patches I've currently sent out, in that it
>> would also allow cross operations between regular refs and symrefs.
>
> One could put this new syntax behind a feature flag to avoid the "-v2"
> suffixes, e.g. `git update-ref --with-symrefs`. But I'm not sure whether
> this would be beneficial.
>

Yeah, me neither. I mean it doesn't provide any new functionality. The
only usecase we're missing currently (with old + symref-* commands), is
that we have no way to convert a symref to a regular ref while checking
the old_value. This could however simply be solved by introducing a new
`symref-convert` command.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> 2. We change the syntax to something like
>
>     symref-update SP <ref> SP <new-ref> [SP (ref <old-target> | oid
> <old-oid>)] LF
>
> this would remove any ambiguity since the user specifies the data type
> they're providing.

Yup.  Being explicit helps, especially if you are only dealing with
programs that do not complain "that's too many keystrokes" like
pesky humans ;-).

When the topic's overall title is to add support for symbolic refs
to the update-ref command, a natural expectation is that in a far
enough future everything that can be done with "git symbolic-ref"
can be done with "git update-ref" and we can, if we wanted to,
depreate "git symbolic-ref".  Is that really what is going on here?

IOW, we should add support for operation modes other than "--stdin"
as well, shouldn't we?

Thanks.

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> 2. We change the syntax to something like
>>
>>     symref-update SP <ref> SP <new-ref> [SP (ref <old-target> | oid
>> <old-oid>)] LF
>>
>> this would remove any ambiguity since the user specifies the data type
>> they're providing.
>
> Yup.  Being explicit helps, especially if you are only dealing with
> programs that do not complain "that's too many keystrokes" like
> pesky humans ;-).
>
> When the topic's overall title is to add support for symbolic refs
> to the update-ref command, a natural expectation is that in a far
> enough future everything that can be done with "git symbolic-ref"
> can be done with "git update-ref" and we can, if we wanted to,
> depreate "git symbolic-ref".  Is that really what is going on here?
>
> IOW, we should add support for operation modes other than "--stdin"
> as well, shouldn't we?
>
> Thanks.

That's a good point, I was thinking of something like this too, but
didn't want to bloat this series. Would it make sense to add this
functionality in a follow up series with the cleanup that Patrick
mentioned [1] too?

[1]: https://lore.kernel.org/git/ZidXrdi7hXdAnDhy@tanuki/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

>> IOW, we should add support for operation modes other than "--stdin"
>> as well, shouldn't we?
>>
>> Thanks.
>
> That's a good point, I was thinking of something like this too, but
> didn't want to bloat this series. Would it make sense to add this
> functionality in a follow up series with the cleanup that Patrick
> mentioned [1] too?

Going multi-step is usually a preferred approach but we have to be
sure that we do not have to retract the way earlier steps gave our
users when we move to later steps.  "Earlier we said that your
commands should look like this, but now you have to give your
commands differently" is absolutely what we want to avoid.

So, "in this iteration, you can use this enhanced feature only via
the "--stdin" mode, but we promise we will make the same available
on the command line" is perfectly fine.  You may need to retitle the
topic to clarify the scope of each iteration, though.

Thansk.

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1214 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>>> IOW, we should add support for operation modes other than "--stdin"
>>> as well, shouldn't we?
>>>
>>> Thanks.
>>
>> That's a good point, I was thinking of something like this too, but
>> didn't want to bloat this series. Would it make sense to add this
>> functionality in a follow up series with the cleanup that Patrick
>> mentioned [1] too?
>
> Going multi-step is usually a preferred approach but we have to be
> sure that we do not have to retract the way earlier steps gave our
> users when we move to later steps.  "Earlier we said that your
> commands should look like this, but now you have to give your
> commands differently" is absolutely what we want to avoid.
>
> So, "in this iteration, you can use this enhanced feature only via
> the "--stdin" mode, but we promise we will make the same available
> on the command line" is perfectly fine.  You may need to retitle the
> topic to clarify the scope of each iteration, though.
>
> Thansk.

Yeah, will do that. Possibly also some of the commits too, where we
could mention that this is being added to the `--stdin` mode and can be
extended further on.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Jeff King]

On Wed, Apr 24, 2024 at 09:25:27AM -0700, Karthik Nayak wrote:

> This also brings light onto the previous versions we were considering:
> 
>     symref-update SP <ref> SP <new-target> [SP (<old-target> | <old-oid>)] LF
> 
> There is also some ambiguity here which we missed, especially when we
> support dangling refs. If we're updating a dangling ref <ref>, and we
> provide an old value. Then there is uncertainty around whether the
> provided value is actually a <old-target> or if it's an <old-oid>.
> 
> For non dangling ref symref, we first parse it as an <old-target> and
> since the <old-target> would exist, we can move on.
> 
> So I see two ways to go about this,
> 
> 1. In the symref-update function, we need to parse and see if <ref> is a
> regular ref or a symref, if it is symref, we simply set the provided old
> value as <old-target>, if not, we set it as <old-oid>. This seems clunky
> because we'll be parsing the ref and trying to understand its type in
> 'update-ref.c', before the actual update.

I think this avoids the "mischief" case I mentioned because it is about
looking at what is in <ref> currently, not spooky action from a
possibly-unrelated "refs/heads/ref". But in general, I think it is a
good thing if we can tell what the caller is asking for based only on
the syntax of the request, without taking into account repository state.
It just makes things easier to reason about.

> 2. We change the syntax to something like
> 
>     symref-update SP <ref> SP <new-ref> [SP (ref <old-target> | oid
> <old-oid>)] LF
> 
> this would remove any ambiguity since the user specifies the data type
> they're providing.

Yeah, I was going to suggest that it could be resolved with any syntax
that would not be a valid oid name. Certainly check-ref-format places
some restrictions there (e.g., no "^") but name resolution relies on
that, too (so foo^{tree} is a valid name). Probably something like
"^foo" is unambiguous, but it's ugly and hard to explain. ;)

But I think your "ref <old-target>" solves that. Resolved names can have
spaces in them, but only after a "^" (e.g., "foo^{/some regex}") or ":"
(e.g., "foo:path with spaces"). So seeing just "ref" by itself, followed
by a space, I think is unambiguous. And it looks pretty.

It gets a little tricky when the field delimiter is also space, and the
item in question is not the final field. See below.

> Also on a sidenote, it's worth considering that with the direction of
> [2], we could also extrapolate to introduce {verify, update, create,
> delete} v2, which support both symrefs and regular refs. But require
> explicit types from the user:
> 
>     update-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
> [(oid <old-oid> | ref <old-target>)] NUL
> 	create-v2 SP <ref> NUL (oid <new-oid> | ref <new-target>) NUL
> 	delete-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL
> 	verify-v2 SP <ref> NUL [(oid <old-oid> | ref <old-target>)] NUL
> 
> This is similar to the v3 patches I've currently sent out, in that it
> would also allow cross operations between regular refs and symrefs.

So I _think_ that "<oid> | ref <symref-target>" is unambiguous. In which
case you could just have:

  update SP <ref> NUL (<new-oid> | ref <new-target>) NUL [(<old-oid> | ref <old-target>)]

which is backwards-compatible.

With the NUL separator it's easy to parse, because "ref " with a
trailing space always means a ref, even in the "new" field. But with
spaces instead, it gets weird. If you have:

  update refs/heads/foo refs refs/heads/bar

it can either be creating a symref to "bar" (with no "old" specifier) or
it could be pointing it at the resolved-name "refs", which the old value
coming from "bar".

I guess one option would be to only allow "ref" syntax in "-z" mode, but
that is probably getting to be a bit weird and hard to explain.

-Peff

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Junio C Hamano]

You'd want something like this squashed into an appropriate step to
avoid breaking "make sparse".

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 175579148f..1cdafc33f3 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -228,7 +228,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	have_old = !parse_next_arg(&next, end, &old_oid,
 				   &old_target, "update", refname,
 				   PARSE_SHA1_OLD | PARSE_REFNAME_TARGETS);
-	have_old = have_old & !old_target.len;
+	have_old = have_old && !old_target.len;
 
 	if (*next != line_termination)
 		die("update %s: extra input: %s", refname, next);

Re: [PATCH v3 0/8] refs: add symref support to 'git-update-ref'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> You'd want something like this squashed into an appropriate step to
> avoid breaking "make sparse".
>
> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
> index 175579148f..1cdafc33f3 100644
> --- a/builtin/update-ref.c
> +++ b/builtin/update-ref.c
> @@ -228,7 +228,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
>  	have_old = !parse_next_arg(&next, end, &old_oid,
>  				   &old_target, "update", refname,
>  				   PARSE_SHA1_OLD | PARSE_REFNAME_TARGETS);
> -	have_old = have_old & !old_target.len;
> +	have_old = have_old && !old_target.len;
>
>  	if (*next != line_termination)
>  		die("update %s: extra input: %s", refname, next);

Thanks, will check and squash this in.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v4 0/7] add symref-* commands to 'git-update-ref --stdin'

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'git-update-ref(1)' command allows transactional reference updates.
But currently only supports regular reference updates. Meaning, if one
wants to update HEAD (symbolic ref) in a transaction, there is no tool
to do so.

One option to obtain transactional updates for the HEAD ref is to
manually create the HEAD.lock file and commit. This is intrusive, where
the user needs to mimic internal git behavior. Also, this only works
when using the files backend.

At GitLab, we've been using the manual process till date, to allow users
to set and change their default branch. But with the introduction of
reftables as a reference backend, this becomes a necessity to be solved
within git.

This patch series goes about introducing a set of commands
symref-{create,verify,delete,update} to work with symrefs complimenting
the existing commands for the regular refs in the '--stdin' mode of
'git-update-ref'.

The 'symref-verify' command can be used to verify if a symref exists and
its existing value.

The 'symref-create' command can be used to create a new symref.

The 'symref-delete' command can be used to delete an existing symref while
optionally checking its existing value.

The 'symref-update' command can be used to update a symref, create a symref,
delete a symref or even convert an existing regular ref to a symref. Wherein
like the regular 'update' command, the zero OID can be used to create/delete
a symref.

While this series adds the commands and the required ground work, it only
is accessile within the '--stdin' mode of 'git-update-ref'. However, it makes
it easy to extend it further to the command line too, which will be present
in a follow up series.

Previous versions:
V1: https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com/
V2: https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com/
V3: https://lore.kernel.org/git/20240423212818.574123-1-knayak@gitlab.com/

V3 took a different approach of incorporating changes into the existing commands, 
of 'git-update-ref --stdin' but we realized that there was some ambiguity in how
these commands are parsed [1]. In that sense it makes more sense to compare this
version with v2 instead.

Changes over v2 are:

- Rename (old|new)_ref to (old|new)_target, to avoid confusion.
- Better assertions around the input data.
- Removing the REF_SYMREF_UPDATE flag.
- Filled in missing/new documentation.
- For symref-update, realized that there was some ambiguity on how the
  old_oid and old_target was parsed, and now the command requires the
  user to explicitly input the data type.
- Support dangling refs in all operations.
- More test cases around empty values.
- Removed unecessary header includes.
- Fixed whitespace issues with the previous series.
- Other review comments.


[1]: https://lore.kernel.org/git/20240423220308.GC1172807@coredump.intra.peff.net/

Range diff (against v2):

1:  3269d0e91e ! 1:  4a56e3ede4 refs: accept symref values in `ref_transaction[_add]_update`
    @@ Commit message
         flags to create a `ref_update` and add it to the transaction at hand.
     
         To extend symref support in transactions, we need to also accept the
    -    old and new ref values and process it. In this commit, let's add the
    -    required paramaters to the function and modify all call sites.
    +    old and new ref targets and process it. In this commit, let's add the
    +    required parameters to the function and modify all call sites.
     
    -    The two paramaters added are `new_ref` and `old_ref`. The `new_ref` is
    -    used to denote what the reference should point to when the transaction
    -    is applied. Some functions allow this parameter to be NULL, meaning that
    -    the reference is not changed, or `""`, meaning that the reference should
    -    be deleted.
    +    The two parameters added are `new_target` and `old_target`. The
    +    `new_target` is used to denote what the reference should point to when
    +    the transaction is applied.
     
    -    The `old_ref` denotes the value of that the reference must have before
    -    the update. Some functions allow this parameter to be NULL, meaning that
    -    the old value of the reference is not checked, or `""`, meaning that the
    -    reference must not exist before the update. A copy of this value is made
    -    in the transaction.
    +    The `old_target` denotes the value the reference must have before the
    +    update. Some functions allow this parameter to be NULL, meaning that the
    +    old value of the reference is not checked.
     
         The handling logic of these parameters will be added in consequent
    -    commits as we implement symref-{create, update, delete, verify}.
    +    commits as we add symref commands to the '--stdin' mode of
    +    'git-update-ref'.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      		const char *refname, unsigned int flags,
      		const struct object_id *new_oid,
      		const struct object_id *old_oid,
    -+		const char *new_ref, const char *old_ref,
    ++		const char *new_target, const char *old_target,
      		const char *msg)
      {
      	struct ref_update *update;
    +@@ refs.c: struct ref_update *ref_transaction_add_update(
    + 	if (transaction->state != REF_TRANSACTION_OPEN)
    + 		BUG("update called for transaction that is not open");
    + 
    ++	if (old_oid && !is_null_oid(old_oid) && old_target)
    ++		BUG("Only one of old_oid and old_target should be non NULL");
    ++	if (new_oid && !is_null_oid(new_oid) && new_target)
    ++		BUG("Only one of new_oid and new_target should be non NULL");
    ++
    + 	FLEX_ALLOC_STR(update, refname, refname);
    + 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
    + 	transaction->updates[transaction->nr++] = update;
     @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *new_oid,
      			   const struct object_id *old_oid,
    -+			   const char *new_ref, const char *old_ref,
    ++			   const char *new_target,
    ++			   const char *old_target,
      			   unsigned int flags, const char *msg,
      			   struct strbuf *err)
      {
    @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      
      	ref_transaction_add_update(transaction, refname, flags,
     -				   new_oid, old_oid, msg);
    -+				   new_oid, old_oid, new_ref, old_ref, msg);
    ++				   new_oid, old_oid, new_target,
    ++				   old_target, msg);
      	return 0;
      }
      
    @@ refs.c: int refs_update_ref(struct ref_store *refs, const char *msg,
     
      ## refs.h ##
     @@ refs.h: struct ref_transaction *ref_transaction_begin(struct strbuf *err);
    -  */
    - #define REF_SKIP_REFNAME_VERIFICATION (1 << 11)
    - 
    -+/*
    -+ * The reference update is considered to be done on a symbolic reference. This
    -+ * ensures that we verify, delete, create and update the ref correspondingly.
    -+ */
    -+#define REF_SYMREF_UPDATE (1 << 12)
    -+
    - /*
    -  * Bitmask of all of the flags that are allowed to be passed in to
    -  * ref_transaction_update() and friends:
    -  */
    - #define REF_TRANSACTION_UPDATE_ALLOWED_FLAGS                                  \
    - 	(REF_NO_DEREF | REF_FORCE_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION | \
    --	 REF_SKIP_REFNAME_VERIFICATION)
    -+	 REF_SKIP_REFNAME_VERIFICATION | REF_SYMREF_UPDATE )
    - 
    - /*
    -  * Add a reference update to transaction. `new_oid` is the value that
    +  *         before the update. A copy of this value is made in the
    +  *         transaction.
    +  *
    ++ *     new_target -- the target reference that the reference will be
    ++ *         update to point to. This takes precedence over new_oid when
    ++ *         set. If the reference is a regular reference, it will be
    ++ *         converted to a symbolic reference.
    ++ *
    ++ *     old_target -- the reference that the reference must be pointing to.
    ++ *         Will only be taken into account when the reference is a symbolic
    ++ *         reference.
    ++ *
    +  *     flags -- flags affecting the update, passed to
    +  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
    +  *         REF_FORCE_CREATE_REFLOG. See those constants for more
    +@@ refs.h: struct ref_transaction *ref_transaction_begin(struct strbuf *err);
    +  * beforehand. The old value is checked after the lock is taken to
    +  * prevent races. If the old value doesn't agree with old_oid, the
    +  * whole transaction fails. If old_oid is NULL, then the previous
    +- * value is not checked.
    ++ * value is not checked. If `old_target` is not NULL, treat the reference
    ++ * as a symbolic ref and validate that its target before the update is
    ++ * `old_target`. If the `new_target` is not NULL, then the reference
    ++ * will be updated to a symbolic ref which targets `new_target`.
    ++ * Together, these allow us to update between regular refs and symrefs.
    +  *
    +  * See the above comment "Reference transaction updates" for more
    +  * information.
     @@ refs.h: int ref_transaction_update(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *new_oid,
      			   const struct object_id *old_oid,
    -+			   const char *new_ref, const char *old_ref,
    ++			   const char *new_target,
    ++			   const char *old_target,
      			   unsigned int flags, const char *msg,
      			   struct strbuf *err);
      
    @@ refs/refs-internal.h: struct ref_update {
      	struct object_id old_oid;
      
     +	/*
    -+	 * If (flags & REF_SYMREF_UPDATE), set the reference to this
    -+	 * value (or delete it, if `new_ref` is an empty string).
    ++	 * If set, point the reference to this value. This can also be
    ++	 * used to convert regular references to become symbolic refs.
     +	 */
    -+	const char *new_ref;
    ++	const char *new_target;
     +
     +	/*
    -+	 * If (type & REF_SYMREF_UPDATE), check that the reference
    -+	 * previously had this value (or didn't previously exist,
    -+	 * if `old_ref` is an empty string).
    ++	 * If set and the reference is a symbolic ref, check that the
    ++	 * reference previously pointed to this value.
     +	 */
    -+	const char *old_ref;
    ++	const char *old_target;
     +
      	/*
      	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
    @@ refs/refs-internal.h: struct ref_update *ref_transaction_add_update(
      		const char *refname, unsigned int flags,
      		const struct object_id *new_oid,
      		const struct object_id *old_oid,
    -+		const char *new_ref, const char *old_ref,
    ++		const char *new_target, const char *old_target,
      		const char *msg);
      
      /*
4:  53fdb408ef = 2:  496bf14f28 files-backend: extract out `create_symref_lock`
2:  a8cb0e0a1d ! 3:  6337859cbb update-ref: add support for symref-verify
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    update-ref: add support for symref-verify
    +    update-ref: add support for 'symref-verify' command
     
    -    In the previous commit, we added the required base for adding symref
    -    support in transactions provided by the 'git-update-ref(1)'. This commit
    -    introduces the 'symref-verify' command which is similar to the existing
    -    'verify' command for regular refs.
    +    In the previous commits, we added the required base for adding symref
    +    commands to the '--stdin' mode provided by 'git-update-ref(1)'. Using
    +    them, add a new 'symref-verify' command to verify symrefs.
     
         The 'symref-verify' command allows users to verify if a provided <ref>
    -    contains the provided <old-ref> without changing the <ref>. If <old-ref>
    -    is not provided, the command will verify that the <ref> doesn't exist.
    -    Since we're checking for symbolic refs, this command will only work with
    -    the 'no-deref' mode. This is because any dereferenced symbolic ref will
    -    point to an object and not a ref and the regular 'verify' command can be
    -    used in such situations.
    +    contains the provided <old-target> without changing the <ref>. If
    +    <old-target> is not provided, the command will verify that the <ref>
    +    doesn't exist. Since we're checking for symbolic refs, this command will
    +    only work with the 'no-deref' mode. This is because any dereferenced
    +    symbolic ref will point to an object and not a ref and the regular
    +    'verify' command can be used in such situations.
     
    -    This commit adds all required helper functions required to also
    -    introduce the other symref commands, namely create, delete, and update.
    +    Add and use `ref_update_is_null_new_value`, a helper function which is
    +    used to check if there is a new_value in a reference update. The new
    +    value could either be a symref target `new_target` or a OID `new_oid`.
         We also add tests to test the command in both the regular stdin mode and
         also with the '-z' flag.
     
    -    When the user doesn't provide a <old-ref> we need to check that the
    -    provided <ref> doesn't exist. And to do this, we take over the existing
    -    understanding that <old-oid> when set to its zero value, it refers to
    -    the ref not existing. While this seems like a mix of contexts between
    -    using <*-ref> and <*-oid>, this actually works really well, especially
    -    considering the fact that we want to eventually also introduce
    -
    -        symref-update SP <ref> SP <new-ref> [SP (<old-oid> | <old-rev>)] LF
    -
    -    and here, we'd allow the user to update a regular <ref> to a symref and
    -    use <old-oid> to check the <ref>'s oid. This can be extrapolated to the
    -    user using this to create a symref when provided a zero <old-oid>. Which
    -    will work given how we're setting it up.
    -
         We also disable the reference-transaction hook for symref-updates which
         will be tackled in its own commit.
     
    -    Add required tests for 'symref-verify' while also adding reflog checks for
    -    the pre-existing 'verify' tests.
    +    Add required tests for symref support in 'verify' while also adding
    +    reflog checks for the pre-existing 'verify' tests.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ Documentation/git-update-ref.txt: performs all modifications together.  Specify
      	create SP <ref> SP <new-oid> LF
      	delete SP <ref> [SP <old-oid>] LF
      	verify SP <ref> [SP <old-oid>] LF
    -+	symref-verify SP <ref> [SP <old-ref>] LF
    ++	symref-verify SP <ref> [SP <old-target>] LF
      	option SP <opt> LF
      	start LF
      	prepare LF
    @@ Documentation/git-update-ref.txt: quoting:
      	create SP <ref> NUL <new-oid> NUL
      	delete SP <ref> NUL [<old-oid>] NUL
      	verify SP <ref> NUL [<old-oid>] NUL
    -+	symref-verify SP <ref> [NUL <old-ref>] NUL
    ++	symref-verify SP <ref> [NUL <old-target>] NUL
      	option SP <opt> NUL
      	start NUL
      	prepare NUL
    @@ Documentation/git-update-ref.txt: verify::
      	<old-oid> is zero or missing, the ref must not exist.
      
     +symref-verify::
    -+	Verify symbolic <ref> against <old-ref> but do not change it.
    -+	If <old-ref> is missing, the ref must not exist.  Can only be
    ++	Verify symbolic <ref> against <old-target> but do not change it.
    ++	If <old-target> is missing, the ref must not exist.  Can only be
     +	used in `no-deref` mode.
     +
      option::
    @@ builtin/update-ref.c: static char *parse_refname(const char **next)
      	return strbuf_detach(&ref, NULL);
      }
      
    -+
    -+
     +/*
     + * Wrapper around parse_refname which skips the next delimiter.
     + */
     +static char *parse_next_refname(const char **next)
     +{
    -+        if (line_termination) {
    -+                /* Without -z, consume SP and use next argument */
    -+                if (!**next || **next == line_termination)
    -+                        return NULL;
    -+                if (**next != ' ')
    -+                        die("expected SP but got: %s", *next);
    -+        } else {
    -+                /* With -z, read the next NUL-terminated line */
    -+                if (**next)
    -+                        return NULL;
    -+        }
    -+        /* Skip the delimiter */
    -+        (*next)++;
    ++	if (line_termination) {
    ++		/* Without -z, consume SP and use next argument */
    ++		if (!**next || **next == line_termination)
    ++			return NULL;
    ++		if (**next != ' ')
    ++			die("expected SP but got: %s", *next);
    ++	} else {
    ++		/* With -z, read the next NUL-terminated line */
    ++		if (**next)
    ++			return NULL;
    ++	}
    ++	/* Skip the delimiter */
    ++	(*next)++;
     +
    -+        return parse_refname(next);
    ++	return parse_refname(next);
     +}
    ++
     +
      /*
       * The value being parsed is <old-oid> (as opposed to <new-oid>; the
    @@ builtin/update-ref.c: static void parse_cmd_verify(struct ref_transaction *trans
     +}
     +
     +static void parse_cmd_symref_verify(struct ref_transaction *transaction,
    -+                                    const char *next, const char *end)
    ++				    const char *next, const char *end)
     +{
     +	struct strbuf err = STRBUF_INIT;
     +	struct object_id old_oid;
    -+	char *refname, *old_ref;
    ++	char *refname, *old_target;
     +
     +	if (!(update_flags & REF_NO_DEREF))
     +		die("symref-verify: cannot operate with deref mode");
    @@ builtin/update-ref.c: static void parse_cmd_verify(struct ref_transaction *trans
     +	 * old_ref is optional, but we want to differentiate between
     +	 * a NULL and zero value.
     +	 */
    -+	old_ref = parse_next_refname(&next);
    -+	if (!old_ref)
    ++	old_target = parse_next_refname(&next);
    ++	if (!old_target)
     +		old_oid = *null_oid();
    -+	else if (read_ref(old_ref, NULL))
    -+		die("symref-verify %s: invalid <old-ref>", refname);
     +
     +	if (*next != line_termination)
     +		die("symref-verify %s: extra input: %s", refname, next);
     +
    -+	if (ref_transaction_verify(transaction, refname, old_ref ? NULL : &old_oid,
    -+				   old_ref, update_flags | REF_SYMREF_UPDATE, &err))
    ++	if (ref_transaction_verify(transaction, refname,
    ++				   old_target ? NULL : &old_oid,
    ++				   old_target, update_flags, &err))
      		die("%s", err.buf);
      
      	update_flags = default_flags;
      	free(refname);
    -+	free(old_ref);
    ++	free(old_target);
      	strbuf_release(&err);
      }
      
    @@ builtin/update-ref.c: static const struct parse_cmd {
      static void update_refs_stdin(void)
     
      ## refs.c ##
    -@@
    - #include "object-store-ll.h"
    - #include "object.h"
    - #include "path.h"
    -+#include "string.h"
    - #include "tag.h"
    - #include "submodule.h"
    - #include "worktree.h"
    -@@
    - #include "date.h"
    - #include "commit.h"
    - #include "wildmatch.h"
    -+#include "wrapper.h"
    - 
    - /*
    -  * List of all available backends
     @@ refs.c: void ref_transaction_free(struct ref_transaction *transaction)
      
      	for (i = 0; i < transaction->nr; i++) {
      		free(transaction->updates[i]->msg);
    -+		free((void *)transaction->updates[i]->old_ref);
    ++		free((void *)transaction->updates[i]->old_target);
    ++		free((void *)transaction->updates[i]->new_target);
      		free(transaction->updates[i]);
      	}
      	free(transaction->updates);
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      	update->flags = flags;
      
     -	if (flags & REF_HAVE_NEW)
    --		oidcpy(&update->new_oid, new_oid);
    ++	if (new_target)
    ++		update->new_target = xstrdup(new_target);
    ++	if (old_target)
    ++		update->old_target = xstrdup(old_target);
    ++	if (new_oid && flags & REF_HAVE_NEW)
    + 		oidcpy(&update->new_oid, new_oid);
     -	if (flags & REF_HAVE_OLD)
    --		oidcpy(&update->old_oid, old_oid);
    -+	/*
    -+	 * The ref values are to be considered over the oid values when we're
    -+	 * doing symref operations.
    -+	 */
    -+	if (update->flags & REF_SYMREF_UPDATE) {
    -+		if (old_ref)
    -+			update->old_ref = xstrdup(old_ref);
    -+	} else {
    -+		if (flags & REF_HAVE_NEW)
    -+			oidcpy(&update->new_oid, new_oid);
    -+		if (flags & REF_HAVE_OLD)
    -+			oidcpy(&update->old_oid, old_oid);
    -+	}
    ++	if (old_oid && flags & REF_HAVE_OLD)
    + 		oidcpy(&update->old_oid, old_oid);
      	update->msg = normalize_reflog_message(msg);
      	return update;
    - }
     @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
      
      	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
    -+	flags |= (new_ref ? REF_HAVE_NEW : 0) | (old_ref ? REF_HAVE_OLD : 0);
    ++	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
      
      	ref_transaction_add_update(transaction, refname, flags,
    - 				   new_oid, old_oid, new_ref, old_ref, msg);
    + 				   new_oid, old_oid, new_target,
     @@ refs.c: int ref_transaction_delete(struct ref_transaction *transaction,
      int ref_transaction_verify(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *old_oid,
    -+			   const char *old_ref,
    ++			   const char *old_target,
      			   unsigned int flags,
      			   struct strbuf *err)
      {
     -	if (!old_oid)
    -+	if (flags & REF_SYMREF_UPDATE && !old_ref && !old_oid)
    -+		BUG("verify called with old_ref set to NULL");
    -+	if (!(flags & REF_SYMREF_UPDATE) && !old_oid)
    - 		BUG("verify called with old_oid set to NULL");
    +-		BUG("verify called with old_oid set to NULL");
    ++	if (!old_target && !old_oid)
    ++		BUG("verify called with old_oid and old_target set to NULL");
    ++	if (old_target && !(flags & REF_NO_DEREF))
    ++		BUG("verify cannot operate on symrefs with deref mode");
      	return ref_transaction_update(transaction, refname,
      				      NULL, old_oid,
     -				      NULL, NULL,
    -+				      NULL, old_ref,
    ++				      NULL, old_target,
      				      flags, NULL, err);
      }
      
    @@ refs.c: static int run_transaction_hook(struct ref_transaction *transaction,
      	for (i = 0; i < transaction->nr; i++) {
      		struct ref_update *update = transaction->updates[i];
      
    -+		if (update->flags & REF_SYMREF_UPDATE)
    ++		/*
    ++		 * Skip reference transaction for symbolic refs.
    ++		 */
    ++		if (update->new_target || update->old_target)
     +			continue;
     +
      		strbuf_reset(&buf);
    @@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char
      	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
      }
     +
    -+int null_new_value(struct ref_update *update) {
    -+	if (update->flags & REF_SYMREF_UPDATE && update->new_ref)
    -+		return 0;
    -+	return is_null_oid(&update->new_oid);
    ++int ref_update_is_null_new_value(struct ref_update *update) {
    ++	return !update->new_target && is_null_oid(&update->new_oid);
     +}
     
      ## refs.h ##
    @@ refs.h: int ref_transaction_delete(struct ref_transaction *transaction,
      int ref_transaction_verify(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *old_oid,
    -+			   const char *old_ref,
    ++			   const char *old_target,
      			   unsigned int flags,
      			   struct strbuf *err);
      
    @@ refs/files-backend.c: static const char *original_update_refname(struct ref_upda
      }
      
     +/*
    -+ * Check whether the REF_HAVE_OLD and old_ref values stored in update
    -+ * are consistent with ref, which is the symbolic reference's current
    -+ * value. If everything is OK, return 0; otherwise, write an error
    -+ * message to err and return -1.
    ++ * Check whether the REF_HAVE_OLD and old_target values stored in
    ++ * update are consistent with ref, which is the symbolic reference's
    ++ * current value. If everything is OK, return 0; otherwise, write an
    ++ * error message to err and return -1.
     + */
    -+static int check_old_ref(struct ref_update *update, char *ref,
    -+			 struct strbuf *err)
    ++static int check_old_target(struct ref_update *update, char *ref,
    ++			    struct strbuf *err)
     +{
     +	if (!(update->flags & REF_HAVE_OLD) ||
    -+	    !strcmp(update->old_ref, ref))
    ++	    !strcmp(update->old_target, ref))
     +		return 0;
     +
    -+	if (!strcmp(update->old_ref, ""))
    ++	if (!strcmp(update->old_target, ""))
     +		strbuf_addf(err, "cannot lock ref '%s': "
     +			    "reference already exists",
     +			    original_update_refname(update));
    @@ refs/files-backend.c: static const char *original_update_refname(struct ref_upda
     +		strbuf_addf(err, "cannot lock ref '%s': "
     +			    "reference is missing but expected %s",
     +			    original_update_refname(update),
    -+			    update->old_ref);
    ++			    update->old_target);
     +	else
     +		strbuf_addf(err, "cannot lock ref '%s': "
     +			    "is at %s but expected %s",
     +			    original_update_refname(update),
    -+			    ref, update->old_ref);
    ++			    ref, update->old_target);
     +
     +	return -1;
     +}
    @@ refs/files-backend.c: static const char *original_update_refname(struct ref_upda
      /*
       * Check whether the REF_HAVE_OLD and old_oid values stored in update
       * are consistent with oid, which is the reference's current value. If
    -@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
    - 			       struct strbuf *err)
    - {
    - 	struct strbuf referent = STRBUF_INIT;
    --	int mustexist = (update->flags & REF_HAVE_OLD) &&
    --		!is_null_oid(&update->old_oid);
    -+	int mustexist = (update->flags & REF_HAVE_OLD) && !is_null_oid(&update->old_oid);
    - 	int ret = 0;
    - 	struct ref_lock *lock;
    - 
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
      					ret = TRANSACTION_GENERIC_ERROR;
      					goto out;
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +			}
     +
     +			/*
    -+			 * For symref verification, we need to check the referent value
    ++			 * For symref verification, we need to check the reference value
     +			 * rather than the oid. If we're dealing with regular refs or we're
     +			 * verifying a dereferenced symref, we then check the oid.
     +			 */
    -+			if (update->flags & REF_SYMREF_UPDATE && update->old_ref) {
    -+				if (check_old_ref(update, referent.buf, err)) {
    ++			if (update->old_target) {
    ++				if (check_old_target(update, referent.buf, err)) {
     +					ret = TRANSACTION_GENERIC_ERROR;
     +					goto out;
     +				}
    @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
     + * takes into consideration that the update could be a regular
     + * ref or a symbolic ref.
     + */
    -+int null_new_value(struct ref_update *update);
    ++int ref_update_is_null_new_value(struct ref_update *update);
     +
      #endif /* REFS_REFS_INTERNAL_H */
     
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      		 * backend returns, which keeps our tests happy.
      		 */
     -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
    -+		if ((u->flags & REF_HAVE_OLD) &&
    -+		    (u->flags & REF_SYMREF_UPDATE) &&
    -+		    u->old_ref) {
    -+			if   (strcmp(referent.buf, u->old_ref)) {
    -+				if (!strcmp(u->old_ref, ""))
    -+					strbuf_addf(err, "cannot lock ref '%s': "
    -+						    "reference already exists",
    ++		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
    ++			if (strcmp(referent.buf, u->old_target)) {
    ++				if (!strcmp(u->old_target, ""))
    ++					strbuf_addf(err, "verifying symref target: '%s': "
    ++						    "provided target is empty",
     +						    original_update_refname(u));
     +				else if (!strcmp(referent.buf, ""))
    -+					strbuf_addf(err, "cannot lock ref '%s': "
    ++					strbuf_addf(err, "verifying symref target: '%s': "
     +						    "reference is missing but expected %s",
     +						    original_update_refname(u),
    -+						    u->old_ref);
    ++						    u->old_target);
     +				else
    -+					strbuf_addf(err, "cannot lock ref '%s': "
    ++					strbuf_addf(err, "verifying symref target: '%s': "
     +						    "is at %s but expected %s",
     +						    original_update_refname(u),
    -+						    referent.buf, u->old_ref);
    ++						    referent.buf, u->old_target);
     +				ret = -1;
     +				goto done;
     +			}
    @@ t/t1400-update-ref.sh: test_expect_success PIPE 'transaction flushes status upda
      	test_cmp expected actual
      '
      
    -+create_stdin_buf ()
    -+{
    ++create_stdin_buf () {
     +	if test "$1" = "-z"
     +	then
     +		shift
    @@ t/t1400-update-ref.sh: test_expect_success PIPE 'transaction flushes status upda
     +for type in "" "-z"
     +do
     +
    -+test_expect_success "stdin ${type} symref-verify fails without --no-deref" '
    -+	git symbolic-ref refs/heads/symref $a &&
    -+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
    -+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
    -+	grep "fatal: symref-verify: cannot operate with deref mode" err
    -+'
    ++	test_expect_success "stdin ${type} symref-verify fails without --no-deref" '
    ++		git symbolic-ref refs/heads/symref $a &&
    ++		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
    ++		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
    ++		grep "fatal: symref-verify: cannot operate with deref mode" err
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-verify fails with too many arguments" '
    -+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" "$a" &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err  &&
    -+	if test "$type" = "-z"
    -+	then
    -+		grep "fatal: unknown command: $a" err
    -+	else
    -+		grep "fatal: symref-verify refs/heads/symref: extra input:  $a" err
    -+	fi
    -+'
    ++	test_expect_success "stdin ${type} symref-verify fails with too many arguments" '
    ++		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" "$a" &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err  &&
    ++		if test "$type" = "-z"
    ++		then
    ++			grep "fatal: unknown command: $a" err
    ++		else
    ++			grep "fatal: symref-verify refs/heads/symref: extra input:  $a" err
    ++		fi
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-verify succeeds for correct value" '
    -+	git symbolic-ref refs/heads/symref >expect &&
    -+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
    -+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
    -+	git update-ref --stdin ${type} --no-deref <stdin &&
    -+	git symbolic-ref refs/heads/symref >actual &&
    -+	test_cmp expect actual &&
    -+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
    -+	test_cmp before after
    -+'
    ++	test_expect_success "stdin ${type} symref-verify succeeds for correct value" '
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
    ++		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		git symbolic-ref refs/heads/symref >actual &&
    ++		test_cmp expect actual &&
    ++		test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
    ++		test_cmp before after
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-verify succeeds for missing reference" '
    -+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
    -+	create_stdin_buf ${type} "symref-verify refs/heads/missing" &&
    -+	git update-ref --stdin ${type} --no-deref <stdin &&
    -+	test_must_fail git rev-parse --verify -q refs/heads/missing &&
    -+	test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
    -+	test_cmp before after
    -+'
    ++	test_expect_success "stdin ${type} symref-verify no value is treated as zero value" '
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		create_stdin_buf ${type} "symref-verify refs/heads/symref" "" &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
    ++	'
    ++
    ++	test_expect_success "stdin ${type} symref-verify succeeds for dangling reference" '
    ++		test_when_finished "git symbolic-ref -d refs/heads/symref2" &&
    ++		test_must_fail git symbolic-ref refs/heads/nonexistent &&
    ++		git symbolic-ref refs/heads/symref2 refs/heads/nonexistent &&
    ++		create_stdin_buf ${type} "symref-verify refs/heads/symref2" "refs/heads/nonexistent" &&
    ++		git update-ref --stdin ${type} --no-deref <stdin
    ++	'
    ++
    ++	test_expect_success "stdin ${type} symref-verify succeeds for missing reference" '
    ++		test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
    ++		create_stdin_buf ${type} "symref-verify refs/heads/missing" "$Z" &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		test_must_fail git rev-parse --verify -q refs/heads/missing &&
    ++		test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
    ++		test_cmp before after
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-verify fails for wrong value" '
    -+	git symbolic-ref refs/heads/symref >expect &&
    -+	create_stdin_buf ${type} "symref-verify refs/heads/symref" "$b" &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
    -+	git symbolic-ref refs/heads/symref >actual &&
    -+	test_cmp expect actual
    -+'
    ++	test_expect_success "stdin ${type} symref-verify fails for wrong value" '
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$b" &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
    ++		git symbolic-ref refs/heads/symref >actual &&
    ++		test_cmp expect actual
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-verify fails for mistaken null value" '
    -+	git symbolic-ref refs/heads/symref >expect &&
    -+	create_stdin_buf ${type} "symref-verify refs/heads/symref" &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
    -+	git symbolic-ref refs/heads/symref >actual &&
    -+	test_cmp expect actual
    -+'
    ++	test_expect_success "stdin ${type} symref-verify fails for mistaken null value" '
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$Z" &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
    ++		git symbolic-ref refs/heads/symref >actual &&
    ++		test_cmp expect actual
    ++	'
     +
     +done
     +
3:  37c3e006da ! 4:  e611cb5a8c update-ref: add support for symref-delete
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    update-ref: add support for symref-delete
    +    update-ref: add support for 'symref-delete' command
     
    -    Similar to the previous commit, add 'symref-delete' to allow deletions
    -    of symbolic refs in a transaction via the 'git-update-ref' command. The
    -    'symref-delete' command can when given with an <old-ref>, deletes the
    -    provided <ref> only when it points to <old-ref>.
    +    Add a new command 'symref-delete' to allow deletions of symbolic refs in
    +    a transaction via the '--stdin' mode of the 'git-update-ref' command.
    +    The 'symref-delete' command can, when given an <old-target>, delete the
    +    provided <ref> only when it points to <old-target>. This will only work
    +    when used with the 'no-deref' mode as it doesn't make sense to deref a
    +    symref during deletion.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ Documentation/git-update-ref.txt: performs all modifications together.  Specify
      	create SP <ref> SP <new-oid> LF
      	delete SP <ref> [SP <old-oid>] LF
      	verify SP <ref> [SP <old-oid>] LF
    -+	symref-delete SP <ref> [SP <old-ref>] LF
    - 	symref-verify SP <ref> [SP <old-ref>] LF
    ++	symref-delete SP <ref> [SP <old-target>] LF
    + 	symref-verify SP <ref> [SP <old-target>] LF
      	option SP <opt> LF
      	start LF
     @@ Documentation/git-update-ref.txt: quoting:
      	create SP <ref> NUL <new-oid> NUL
      	delete SP <ref> NUL [<old-oid>] NUL
      	verify SP <ref> NUL [<old-oid>] NUL
    -+	symref-delete SP <ref> [NUL <old-ref>] NUL
    - 	symref-verify SP <ref> [NUL <old-ref>] NUL
    ++	symref-delete SP <ref> [NUL <old-target>] NUL
    + 	symref-verify SP <ref> [NUL <old-target>] NUL
      	option SP <opt> NUL
      	start NUL
    -@@ Documentation/git-update-ref.txt: verify::
    +@@ Documentation/git-update-ref.txt: create::
    + 	exist.  The given <new-oid> may not be zero.
    + 
    + delete::
    +-	Delete <ref> after verifying it exists with <old-oid>, if
    +-	given.  If given, <old-oid> may not be zero.
    ++	Delete <ref> after verifying it exists with <old-oid>, if given.
    ++	If given, <old-oid> may not be zero.  If instead, ref:<old-target>
    ++	is provided, verify that the symbolic ref <ref> targets
    ++	<old-target> before deleting it.
    + 
    + verify::
      	Verify <ref> against <old-oid> but do not change it.  If
      	<old-oid> is zero or missing, the ref must not exist.
      
     +symref-delete::
    -+	Delete <ref> after verifying it exists with <old-ref>, if
    -+	given.
    ++	Delete <ref> after verifying it exists with <old-target>, if given.
     +
      symref-verify::
    - 	Verify symbolic <ref> against <old-ref> but do not change it.
    - 	If <old-ref> is missing, the ref must not exist.  Can only be
    + 	Verify symbolic <ref> against <old-target> but do not change it.
    + 	If <old-target> is missing, the ref must not exist.  Can only be
     
      ## builtin/fetch.c ##
     @@ builtin/fetch.c: static int prune_refs(struct display_state *display_state,
    @@ builtin/update-ref.c: static void parse_cmd_delete(struct ref_transaction *trans
      	strbuf_release(&err);
      }
      
    ++
     +static void parse_cmd_symref_delete(struct ref_transaction *transaction,
     +				    const char *next, const char *end)
     +{
     +	struct strbuf err = STRBUF_INIT;
    -+	char *refname, *old_ref;
    ++	char *refname, *old_target;
     +
     +	if (!(update_flags & REF_NO_DEREF))
    -+                die("symref-delete: cannot operate with deref mode");
    ++		die("symref-delete: cannot operate with deref mode");
     +
     +	refname = parse_refname(&next);
     +	if (!refname)
     +		die("symref-delete: missing <ref>");
     +
    -+        old_ref = parse_next_refname(&next);
    -+	if (old_ref && read_ref(old_ref, NULL))
    -+		die("symref-delete %s: invalid <old-ref>", refname);
    ++	old_target = parse_next_refname(&next);
     +
     +	if (*next != line_termination)
     +		die("symref-delete %s: extra input: %s", refname, next);
     +
     +	if (ref_transaction_delete(transaction, refname, NULL,
    -+				   update_flags | REF_SYMREF_UPDATE,
    -+				   old_ref, msg, &err))
    ++				   update_flags, old_target, msg, &err))
     +		die("%s", err.buf);
     +
     +	update_flags = default_flags;
     +	free(refname);
    -+	free(old_ref);
    ++	free(old_target);
     +	strbuf_release(&err);
     +}
    ++
     +
      static void parse_cmd_verify(struct ref_transaction *transaction,
      			     const char *next, const char *end)
    @@ refs.c: int refs_delete_ref(struct ref_store *refs, const char *msg,
      	    ref_transaction_commit(transaction, &err)) {
      		error("%s", err.buf);
      		ref_transaction_free(transaction);
    -@@ refs.c: void ref_transaction_free(struct ref_transaction *transaction)
    - 	for (i = 0; i < transaction->nr; i++) {
    - 		free(transaction->updates[i]->msg);
    - 		free((void *)transaction->updates[i]->old_ref);
    -+		free((void *)transaction->updates[i]->new_ref);
    - 		free(transaction->updates[i]);
    - 	}
    - 	free(transaction->updates);
    -@@ refs.c: struct ref_update *ref_transaction_add_update(
    - 	if (update->flags & REF_SYMREF_UPDATE) {
    - 		if (old_ref)
    - 			update->old_ref = xstrdup(old_ref);
    -+		if (new_ref)
    -+			update->new_ref = xstrdup(new_ref);
    - 	} else {
    - 		if (flags & REF_HAVE_NEW)
    - 			oidcpy(&update->new_oid, new_oid);
     @@ refs.c: int ref_transaction_create(struct ref_transaction *transaction,
      int ref_transaction_delete(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *old_oid,
     -			   unsigned int flags, const char *msg,
     +			   unsigned int flags,
    -+			   const char *old_ref,
    ++			   const char *old_target,
     +			   const char *msg,
      			   struct strbuf *err)
      {
    --	if (old_oid && is_null_oid(old_oid))
    -+	if (!(flags & REF_SYMREF_UPDATE) && old_oid &&
    -+	    is_null_oid(old_oid))
    + 	if (old_oid && is_null_oid(old_oid))
      		BUG("delete called with old_oid set to zeros");
    ++	if (old_target && !(flags & REF_NO_DEREF))
    ++		BUG("delete cannot operate on symrefs with deref mode");
      	return ref_transaction_update(transaction, refname,
      				      null_oid(), old_oid,
     -				      NULL, NULL, flags,
    -+				      NULL, old_ref, flags,
    ++				      NULL, old_target, flags,
      				      msg, err);
      }
      
    @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg,
      	for_each_string_list_item(item, refnames) {
      		ret = ref_transaction_delete(transaction, item->string,
     -					     NULL, flags, msg, &err);
    -+					     NULL, flags, 0, msg, &err);
    ++					     NULL, flags, NULL, msg, &err);
      		if (ret) {
      			warning(_("could not delete reference %s: %s"),
      				item->string, err.buf);
    @@ refs.h: int ref_transaction_create(struct ref_transaction *transaction,
      			   const struct object_id *old_oid,
     -			   unsigned int flags, const char *msg,
     +			   unsigned int flags,
    -+			   const char *old_ref,
    ++			   const char *old_target,
     +			   const char *msg,
      			   struct strbuf *err);
      
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      	files_assert_main_repository(refs, "lock_ref_for_update");
      
     -	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
    -+	if ((update->flags & REF_HAVE_NEW) && null_new_value(update))
    ++	if ((update->flags & REF_HAVE_NEW) && ref_update_is_null_new_value(update))
      		update->flags |= REF_DELETING;
      
      	if (head_ref) {
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      			continue;
      
     -		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
    -+		if (u->flags & REF_HAVE_NEW && null_new_value(u)) {
    ++		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
      			struct reftable_ref_record ref = {
      				.refname = (char *)u->refname,
      				.update_index = ts,
     
      ## t/t1400-update-ref.sh ##
    -@@ t/t1400-update-ref.sh: test_expect_success "stdin ${type} symref-verify fails for mistaken null value"
    - 	test_cmp expect actual
    - '
    +@@ t/t1400-update-ref.sh: do
    + 		test_cmp before after
    + 	'
      
    -+test_expect_success "stdin ${type} symref-delete fails without --no-deref" '
    -+	git symbolic-ref refs/heads/symref $a &&
    -+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
    -+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
    -+	grep "fatal: symref-delete: cannot operate with deref mode" err
    -+'
    +-	test_expect_success "stdin ${type} symref-verify no value is treated as zero value" '
    ++	test_expect_success "stdin ${type} symref-verify fails with no value" '
    + 		git symbolic-ref refs/heads/symref >expect &&
    + 		create_stdin_buf ${type} "symref-verify refs/heads/symref" "" &&
    + 		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
    +@@ t/t1400-update-ref.sh: do
    + 		test_cmp expect actual
    + 	'
    + 
    ++	test_expect_success "stdin ${type} symref-delete fails without --no-deref" '
    ++		git symbolic-ref refs/heads/symref $a &&
    ++		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
    ++		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
    ++		grep "fatal: symref-delete: cannot operate with deref mode" err
    ++	'
    ++
    ++	test_expect_success "stdin ${type} symref-delete fails with no ref" '
    ++		create_stdin_buf ${type} "symref-delete " &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    ++		grep "fatal: symref-delete: missing <ref>" err
    ++	'
    ++
    ++	test_expect_success "stdin ${type} symref-delete fails with too many arguments" '
    ++		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" "$a" &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    ++		if test "$type" = "-z"
    ++		then
    ++			grep "fatal: unknown command: $a" err
    ++		else
    ++			grep "fatal: symref-delete refs/heads/symref: extra input:  $a" err
    ++		fi
    ++	'
     +
    -+test_expect_success "stdin ${type} fails symref-delete with no ref" '
    -+	create_stdin_buf ${type} "symref-delete " &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    -+	grep "fatal: symref-delete: missing <ref>" err
    -+'
    ++	test_expect_success "stdin ${type} symref-delete fails with wrong old value" '
    ++		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$m" &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    ++		if test_have_prereq REFTABLE
    ++		then
    ++			grep "fatal: verifying symref target: ${SQ}refs/heads/symref${SQ}: is at $a but expected refs/heads/main" err
    ++		else
    ++			grep "fatal: cannot lock ref ${SQ}refs/heads/symref${SQ}" err
    ++		fi &&
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		echo $a >actual &&
    ++		test_cmp expect actual
    ++	'
     +
    -+test_expect_success "stdin ${type} fails symref-delete with too many arguments" '
    -+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" "$a" &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    -+	if test "$type" = "-z"
    -+	then
    -+		grep "fatal: unknown command: $a" err
    -+	else
    -+		grep "fatal: symref-delete refs/heads/symref: extra input:  $a" err
    -+	fi
    -+'
    ++	test_expect_success "stdin ${type} symref-delete works with right old value" '
    ++		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		test_must_fail git rev-parse --verify -q refs/heads/symref
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-delete ref fails with wrong old value" '
    -+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$m" &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    -+	grep "fatal: cannot lock ref '"'"'refs/heads/symref'"'"'" err &&
    -+	git symbolic-ref refs/heads/symref >expect &&
    -+	echo $a >actual &&
    -+	test_cmp expect actual
    -+'
    ++	test_expect_success "stdin ${type} symref-delete works with empty old value" '
    ++		git symbolic-ref refs/heads/symref $a &&
    ++		create_stdin_buf ${type} "symref-delete refs/heads/symref" "" &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		test_must_fail git rev-parse --verify -q $b
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-delete ref works with right old value" '
    -+	create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
    -+	git update-ref --stdin ${type} --no-deref <stdin &&
    -+	test_must_fail git rev-parse --verify -q $b
    -+'
    ++	test_expect_success "stdin ${type} symref-delete succeeds for dangling reference" '
    ++		test_must_fail git symbolic-ref refs/heads/nonexistent &&
    ++		git symbolic-ref refs/heads/symref2 refs/heads/nonexistent &&
    ++		create_stdin_buf ${type} "symref-delete refs/heads/symref2" "refs/heads/nonexistent" &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		test_must_fail git symbolic-ref -d refs/heads/symref2
    ++	'
     +
      done
      
5:  8fa0151f94 ! 5:  37f8be2f3f update-ref: add support for symref-create
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    update-ref: add support for symref-create
    +    update-ref: add support for 'symref-create' command
     
    -    Add 'symref-create' to allow creation of symbolic refs in a transaction
    -    via the 'git-update-ref' command. The 'symref-create' command takes in a
    -    <new-ref>, which the created <ref> will point to.
    +    Add 'symref-create' command to the '--stdin' mode 'git-update-ref' to
    +    allow creation of symbolic refs in a transaction. The 'symref-create'
    +    command takes in a <new-target>, which the created <ref> will point to.
     
    -    We also support the 'core.prefersymlinkrefs', wherein if the flag is set
    -    and the filesystem supports symlinks, we create the symbolic ref as a
    -    symlink.
    +    Also, support the 'core.prefersymlinkrefs' config, wherein if the config
    +    is set and the filesystem supports symlinks, we create the symbolic ref
    +    as a symlink. We fallback to creating a regular symref if creating the
    +    symlink is unsuccessful.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ Documentation/git-update-ref.txt: performs all modifications together.  Specify
      	create SP <ref> SP <new-oid> LF
      	delete SP <ref> [SP <old-oid>] LF
      	verify SP <ref> [SP <old-oid>] LF
    -+	symref-create SP <ref> SP <new-ref> LF
    - 	symref-delete SP <ref> [SP <old-ref>] LF
    - 	symref-verify SP <ref> [SP <old-ref>] LF
    ++	symref-create SP <ref> SP <new-target> LF
    + 	symref-delete SP <ref> [SP <old-target>] LF
    + 	symref-verify SP <ref> [SP <old-target>] LF
      	option SP <opt> LF
     @@ Documentation/git-update-ref.txt: quoting:
      	create SP <ref> NUL <new-oid> NUL
      	delete SP <ref> NUL [<old-oid>] NUL
      	verify SP <ref> NUL [<old-oid>] NUL
    -+	symref-create SP <ref> NUL <new-ref> NUL
    - 	symref-delete SP <ref> [NUL <old-ref>] NUL
    - 	symref-verify SP <ref> [NUL <old-ref>] NUL
    ++	symref-create SP <ref> NUL <new-target> NUL
    + 	symref-delete SP <ref> [NUL <old-target>] NUL
    + 	symref-verify SP <ref> [NUL <old-target>] NUL
      	option SP <opt> NUL
    +@@ Documentation/git-update-ref.txt: update::
    + 
    + create::
    + 	Create <ref> with <new-oid> after verifying it does not
    +-	exist.  The given <new-oid> may not be zero.
    ++	exist.  The given <new-oid> may not be zero.  If instead
    ++	ref:<new-target> is provided, a symbolic ref is created
    ++	which targets <new-target>.
    + 
    + delete::
    + 	Delete <ref> after verifying it exists with <old-oid>, if given.
     @@ Documentation/git-update-ref.txt: verify::
      	Verify <ref> against <old-oid> but do not change it.  If
      	<old-oid> is zero or missing, the ref must not exist.
      
     +symref-create::
    -+	Create symbolic ref <ref> with <new-ref> after verifying
    ++	Create symbolic ref <ref> with <new-target> after verifying
     +	it does not exist.  Can only be used in `no-deref` mode.
     +
      symref-delete::
    - 	Delete <ref> after verifying it exists with <old-ref>, if
    - 	given.
    + 	Delete <ref> after verifying it exists with <old-target>, if given.
    + 
     
      ## builtin/clone.c ##
     @@ builtin/clone.c: static void write_remote_refs(const struct ref *local_refs)
    @@ builtin/update-ref.c: static void parse_cmd_create(struct ref_transaction *trans
      	strbuf_release(&err);
      }
      
    ++
     +static void parse_cmd_symref_create(struct ref_transaction *transaction,
     +				    const char *next, const char *end)
     +{
     +	struct strbuf err = STRBUF_INIT;
    -+	char *refname, *new_ref;
    ++	char *refname, *new_target;
     +
     +	if (!(update_flags & REF_NO_DEREF))
    -+                die("symref-create: cannot operate with deref mode");
    ++		die("symref-create: cannot operate with deref mode");
     +
     +	refname = parse_refname(&next);
     +	if (!refname)
     +		die("symref-create: missing <ref>");
     +
    -+	new_ref = parse_next_refname(&next);
    -+	if (!new_ref)
    -+		die("symref-create %s: missing <new-ref>", refname);
    -+	if (read_ref(new_ref, NULL))
    -+		die("symref-create %s: invalid <new-ref>", refname);
    ++	new_target = parse_next_refname(&next);
    ++	if (!new_target)
    ++		die("symref-create %s: missing <new-target>", refname);
     +
     +	if (*next != line_termination)
     +		die("symref-create %s: extra input: %s", refname, next);
     +
    -+	if (ref_transaction_create(transaction, refname, NULL, new_ref,
    -+				   update_flags | create_reflog_flag |
    -+				   REF_SYMREF_UPDATE, msg, &err))
    ++	if (ref_transaction_create(transaction, refname, NULL, new_target,
    ++				   update_flags | create_reflog_flag,
    ++				   msg, &err))
     +		die("%s", err.buf);
     +
     +	update_flags = default_flags;
     +	free(refname);
    -+	free(new_ref);
    ++	free(new_target);
     +	strbuf_release(&err);
     +}
     +
    @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      int ref_transaction_create(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *new_oid,
    -+			   const char *new_ref,
    ++			   const char *new_target,
      			   unsigned int flags, const char *msg,
      			   struct strbuf *err)
      {
     -	if (!new_oid || is_null_oid(new_oid)) {
    -+	if ((flags & REF_SYMREF_UPDATE) && !new_ref) {
    -+		strbuf_addf(err, "'%s' has a no new ref", refname);
    -+		return 1;
    -+	}
    -+	if (!(flags & REF_SYMREF_UPDATE) && (!new_oid || is_null_oid(new_oid))) {
    - 		strbuf_addf(err, "'%s' has a null OID", refname);
    +-		strbuf_addf(err, "'%s' has a null OID", refname);
    ++	if ((!new_oid || is_null_oid(new_oid)) && !new_target) {
    ++		strbuf_addf(err, "'%s' has a null OID or no new target", refname);
      		return 1;
      	}
    ++	if (new_target && !(flags & REF_NO_DEREF))
    ++		BUG("create cannot operate on symrefs with deref mode");
      	return ref_transaction_update(transaction, refname, new_oid,
     -				      null_oid(), NULL, NULL, flags,
    -+				      null_oid(), new_ref, NULL, flags,
    ++				      null_oid(), new_target, NULL, flags,
      				      msg, err);
      }
      
    @@ refs.h: int ref_transaction_update(struct ref_transaction *transaction,
      int ref_transaction_create(struct ref_transaction *transaction,
      			   const char *refname,
      			   const struct object_id *new_oid,
    -+			   const char *new_ref,
    ++			   const char *new_target,
      			   unsigned int flags, const char *msg,
      			   struct strbuf *err);
      
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      		}
      	}
      
    -+	if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
    -+		if (create_symref_lock(refs, lock, update->refname, update->new_ref)) {
    ++	if (update->new_target) {
    ++		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
     +			ret = TRANSACTION_GENERIC_ERROR;
     +			goto out;
     +		}
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
      
      		if (update->flags & REF_NEEDS_COMMIT ||
      		    update->flags & REF_LOG_ONLY) {
    -+			if (update->flags & REF_SYMREF_UPDATE && update->new_ref) {
    -+				/* for dangling symrefs we gracefully set the oid to zero */
    -+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_ref,
    ++			if (update->new_target) {
    ++				/*
    ++				 * We want to get the resolved OID for the target, to ensure
    ++				 * that the correct value is added to the reflog.
    ++				 */
    ++				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
     +							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
    ++					/* for dangling symrefs we gracefully set the oid to zero */
     +					update->new_oid = *null_oid();
     +				}
     +			}
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
     +		 * We try creating a symlink, if that succeeds we continue to the
     +		 * next updated. If not, we try and create a regular symref.
     +		 */
    -+		if (update->flags & REF_SYMREF_UPDATE && prefer_symlink_refs)
    -+			if (!create_ref_symlink(lock, update->new_ref))
    ++		if (update->new_target && prefer_symlink_refs)
    ++			if (!create_ref_symlink(lock, update->new_target))
     +				continue;
     +
      		if (update->flags & REF_NEEDS_COMMIT) {
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      			 * when the reference in question doesn't exist.
      			 */
     -			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
    -+			 if (u->flags & REF_HAVE_NEW && !null_new_value(u)) {
    ++			 if (u->flags & REF_HAVE_NEW && !ref_update_is_null_new_value(u)) {
      				 ret = queue_transaction_update(refs, tx_data, u,
      								&current_oid, err);
      				 if (ret)
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      		 *   the given ref.
      		 */
     -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
    -+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && null_new_value(u)) {
    ++		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && ref_update_is_null_new_value(u)) {
      			struct reftable_log_record log = {0};
      			struct reftable_iterator it = {0};
      
    +@@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
    + 			    should_write_log(&arg->refs->base, u->refname))) {
    + 			struct reftable_log_record *log;
    + 
    ++			if (u->new_target)
    ++				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
    ++							     RESOLVE_REF_READING, &u->new_oid, NULL))
    ++					/* for dangling symrefs we gracefully set the oid to zero */
    ++					u->new_oid = *null_oid();
    ++
    + 			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
    + 			log = &logs[logs_nr++];
    + 			memset(log, 0, sizeof(*log));
     @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
      		if (u->flags & REF_LOG_ONLY)
      			continue;
      
    --		if (u->flags & REF_HAVE_NEW && null_new_value(u)) {
    -+		if (u->flags & REF_SYMREF_UPDATE &&
    -+		    u->flags & REF_HAVE_NEW &&
    -+		    !null_new_value(u)) {
    +-		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
    ++		if (u->flags & REF_HAVE_NEW && u->new_target) {
     +			struct reftable_ref_record ref = {
     +				.refname = (char *)u->refname,
     +				.value_type = REFTABLE_REF_SYMREF,
    -+				.value.symref = (char *)u->new_ref,
    ++				.value.symref = (char *)u->new_target,
     +				.update_index = ts,
     +			};
     +
     +			ret = reftable_writer_add_ref(writer, &ref);
     +			if (ret < 0)
     +				goto done;
    -+		} else if (u->flags & REF_HAVE_NEW && null_new_value(u)) {
    ++		} else if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
      			struct reftable_ref_record ref = {
      				.refname = (char *)u->refname,
      				.update_index = ts,
    @@ t/t0600-reffiles-backend.sh: test_expect_success POSIXPERM 'git reflog expire ho
      test_done
     
      ## t/t1400-update-ref.sh ##
    -@@ t/t1400-update-ref.sh: test_expect_success "stdin ${type} symref-delete ref works with right old value"
    - 	test_must_fail git rev-parse --verify -q $b
    - '
    +@@ t/t1400-update-ref.sh: do
    + 		test_must_fail git symbolic-ref -d refs/heads/symref2
    + 	'
      
    -+test_expect_success "stdin ${type} symref-create fails without --no-deref" '
    -+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" &&
    -+	test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
    -+	grep "fatal: symref-create: cannot operate with deref mode" err
    -+'
    ++	test_expect_success "stdin ${type} symref-create fails without --no-deref" '
    ++		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" &&
    ++		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
    ++		grep "fatal: symref-create: cannot operate with deref mode" err
    ++	'
     +
    -+test_expect_success "stdin ${type} fails symref-create with no ref" '
    -+	create_stdin_buf ${type} "symref-create " >stdin &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    -+	grep "fatal: symref-create: missing <ref>" err
    -+'
    ++	test_expect_success "stdin ${type} symref-create fails with too many arguments" '
    ++		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" "$a" >stdin &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    ++		if test "$type" = "-z"
    ++		then
    ++			grep "fatal: unknown command: $a" err
    ++		else
    ++			grep "fatal: symref-create refs/heads/symref: extra input:  $a" err
    ++		fi
    ++	'
     +
    -+test_expect_success "stdin ${type} fails symref-create with no new value" '
    -+	create_stdin_buf ${type} "symref-create refs/heads/symref" >stdin &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    -+	grep "fatal: symref-create refs/heads/symref: missing <new-ref>" err
    -+'
    ++	test_expect_success "stdin ${type} symref-create fails with no target" '
    ++		create_stdin_buf ${type} "symref-create refs/heads/symref" >stdin &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
    ++	'
     +
    -+test_expect_success "stdin ${type} fails symref-create with too many arguments" '
    -+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" "$a" >stdin &&
    -+	test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
    -+	if test "$type" = "-z"
    -+	then
    -+		grep "fatal: unknown command: $a" err
    -+	else
    -+		grep "fatal: symref-create refs/heads/symref: extra input:  $a" err
    -+	fi
    -+'
    ++	test_expect_success "stdin ${type} symref-create fails with empty target" '
    ++		create_stdin_buf ${type} "symref-create refs/heads/symref" "" >stdin &&
    ++		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-create ref works" '
    -+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
    -+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
    -+	git update-ref --stdin ${type} --no-deref <stdin &&
    -+	git symbolic-ref refs/heads/symref >expect &&
    -+	echo $a >actual &&
    -+	test_cmp expect actual
    -+'
    ++	test_expect_success "stdin ${type} symref-create works" '
    ++		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
    ++		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		echo $a >actual &&
    ++		test_cmp expect actual
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-create does not create reflogs by default" '
    -+	test_when_finished "git symbolic-ref -d refs/symref" &&
    -+	create_stdin_buf ${type} "symref-create refs/symref" "$a" >stdin &&
    -+	git update-ref --stdin ${type} --no-deref <stdin &&
    -+	git symbolic-ref refs/symref >expect &&
    -+	echo $a >actual &&
    -+	test_cmp expect actual &&
    -+	test_must_fail git reflog exists refs/symref
    -+'
    ++	test_expect_success "stdin ${type} create dangling symref ref works" '
    ++		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
    ++		create_stdin_buf ${type} "symref-create refs/heads/symref" "refs/heads/unkown" >stdin &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		echo refs/heads/unkown >actual &&
    ++		test_cmp expect actual
    ++	'
     +
    -+test_expect_success "stdin ${type} symref-create reflogs with --create-reflog" '
    -+	test_when_finished "git symbolic-ref -d refs/heads/symref" &&
    -+	create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
    -+	git update-ref --create-reflog --stdin ${type} --no-deref <stdin &&
    -+	git symbolic-ref refs/heads/symref >expect &&
    -+	echo $a >actual &&
    -+	test_cmp expect actual &&
    -+	git reflog exists refs/heads/symref
    -+'
    ++	test_expect_success "stdin ${type} symref-create does not create reflogs by default" '
    ++		test_when_finished "git symbolic-ref -d refs/symref" &&
    ++		create_stdin_buf ${type} "symref-create refs/symref" "$a" >stdin &&
    ++		git update-ref --stdin ${type} --no-deref <stdin &&
    ++		git symbolic-ref refs/symref >expect &&
    ++		echo $a >actual &&
    ++		test_cmp expect actual &&
    ++		test_must_fail git reflog exists refs/symref
    ++	'
    ++
    ++	test_expect_success "stdin ${type} symref-create reflogs with --create-reflog" '
    ++		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
    ++		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
    ++		git update-ref --create-reflog --stdin ${type} --no-deref <stdin &&
    ++		git symbolic-ref refs/heads/symref >expect &&
    ++		echo $a >actual &&
    ++		test_cmp expect actual &&
    ++		git reflog exists refs/heads/symref
    ++	'
     +
      done
      
6:  714492ede3 < -:  ---------- update-ref: add support for symref-update
-:  ---------- > 6:  b385f4d0d7 update-ref: add support for 'symref-update' command
7:  c483104562 ! 7:  ef335e47d1 refs: support symrefs in 'reference-transaction' hook
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    refs: support symrefs in 'reference-transaction' hook
    +    ref: support symrefs in 'reference-transaction' hook
     
         The 'reference-transaction' hook runs whenever a reference update is
    -    made to the system. In the previous commits, we added support for
    -    various symref commands in `git-update-ref`. While it allowed us to now
    +    made to the system. In the previous commits, we added symref support for
    +    various commands in `git-update-ref`. While it allowed us to now
         manipulate symbolic refs via `git-update-ref`, it didn't activate the
         'reference-transaction' hook.
     
    @@ Commit message
         new format described for this and we stick to the existing format of:
             <old-value> SP <new-value> SP <ref-name> LF
         but now, <old-value> and <new-value> could also denote references
    -    instead of objects.
    +    instead of objects, where the format is similar to that in
    +    'git-update-ref', i.e. 'ref:<ref-target>'.
     
         While this seems to be backward incompatible, it is okay, since the only
         way the `reference-transaction` hook has refs in its output is when
    -    `git-update-ref` is used with `update-symref` command. Also the
    -    documentation for reference-transaction hook always stated that support
    -    for symbolic references may be added in the future.
    +    `git-update-ref` is used to manipulate symrefs. Also the documentation
    +    for reference-transaction hook always stated that support for symbolic
    +    references may be added in the future.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ Documentation/githooks.txt: given reference transaction is in:
      `<ref-name>` via `git rev-parse`.
      
     +For symbolic reference updates the `<old_value>` and `<new-value>`
    -+fields could denote references instead of objects.
    ++fields could denote references instead of objects, denoted via the
    ++`ref:<ref-target>` format.
     +
      The exit status of the hook is ignored for any state except for the
      "prepared" state. In the "prepared" state, a non-zero exit status will
    @@ refs.c: static int run_transaction_hook(struct ref_transaction *transaction,
      
      	for (i = 0; i < transaction->nr; i++) {
      		struct ref_update *update = transaction->updates[i];
    -+		const char *new_value, *old_value;
    ++		strbuf_reset(&buf);
      
    --		if (update->flags & REF_SYMREF_UPDATE)
    +-		/*
    +-		 * Skip reference transaction for symbolic refs.
    +-		 */
    +-		if (update->new_target || update->old_target)
     -			continue;
    -+		new_value = oid_to_hex(&update->new_oid);
    -+		old_value = oid_to_hex(&update->old_oid);
    -+
    -+		if (update->flags & REF_SYMREF_UPDATE) {
    -+			if (update->flags & REF_HAVE_NEW && !null_new_value(update))
    -+				new_value = update->new_ref;
    -+			if (update->flags & REF_HAVE_OLD && update->old_ref)
    -+				old_value = update->old_ref;
    -+		}
    ++		if (update->flags & REF_HAVE_OLD && update->old_target)
    ++			strbuf_addf(&buf, "ref:%s ", update->old_target);
    ++		else
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
      
    - 		strbuf_reset(&buf);
    +-		strbuf_reset(&buf);
     -		strbuf_addf(&buf, "%s %s %s\n",
     -			    oid_to_hex(&update->old_oid),
     -			    oid_to_hex(&update->new_oid),
     -			    update->refname);
    -+		strbuf_addf(&buf, "%s %s %s\n", old_value, new_value, update->refname);
    ++		if (update->flags & REF_HAVE_NEW && update->new_target)
    ++			strbuf_addf(&buf, "ref:%s ", update->new_target);
    ++		else
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
    ++
    ++		strbuf_addf(&buf, "%s\n", update->refname);
      
      		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
      			if (errno != EPIPE) {
     
      ## t/t1416-ref-transaction-hooks.sh ##
    -@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'hook gets all queued updates in aborted state' '
    - 	test_cmp expect actual
    +@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
    + 	test_cmp expect target-repo.git/actual
      '
      
    -+# This test doesn't add a check for 'symref-delete' since there is a
    ++# This test doesn't add a check for symref 'delete' since there is a
     +# variation between the ref backends WRT 'delete'. In the files backend,
     +# 'delete' also triggers an additional transaction update on the
     +# packed-refs backend, which constitutes additional reflog entries.
    - test_expect_success 'interleaving hook calls succeed' '
    - 	test_when_finished "rm -r target-repo.git" &&
    - 
    -@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
    - 	test_cmp expect target-repo.git/actual
    - '
    - 
     +test_expect_success 'hook gets all queued symref updates' '
     +	test_when_finished "rm actual" &&
     +
    @@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls s
     +
     +	cat >expect <<-EOF &&
     +		prepared
    -+		refs/heads/main $ZERO_OID refs/heads/symref
    -+		$ZERO_OID refs/heads/main refs/heads/symrefc
    -+		refs/heads/main refs/heads/branch refs/heads/symrefu
    ++		ref:refs/heads/main $ZERO_OID refs/heads/symref
    ++		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
    ++		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
     +		committed
    -+		refs/heads/main $ZERO_OID refs/heads/symref
    -+		$ZERO_OID refs/heads/main refs/heads/symrefc
    -+		refs/heads/main refs/heads/branch refs/heads/symrefu
    ++		ref:refs/heads/main $ZERO_OID refs/heads/symref
    ++		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
    ++		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
     +	EOF
     +
     +	git update-ref --no-deref --stdin <<-EOF &&
     +		start
     +		symref-verify refs/heads/symref refs/heads/main
     +		symref-create refs/heads/symrefc refs/heads/main
    -+		symref-update refs/heads/symrefu refs/heads/branch refs/heads/main
    ++		symref-update refs/heads/symrefu refs/heads/branch ref refs/heads/main
     +		prepare
     +		commit
     +	EOF


Karthik Nayak (7):
  refs: accept symref values in `ref_transaction[_add]_update`
  files-backend: extract out `create_symref_lock`
  update-ref: add support for 'symref-verify' command
  update-ref: add support for 'symref-delete' command
  update-ref: add support for 'symref-create' command
  update-ref: add support for 'symref-update' command
  ref: support symrefs in 'reference-transaction' hook

 Documentation/git-update-ref.txt |  34 ++-
 Documentation/githooks.txt       |  14 +-
 branch.c                         |   2 +-
 builtin/clone.c                  |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   4 +-
 builtin/receive-pack.c           |   4 +-
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             | 240 ++++++++++++++++--
 refs.c                           |  78 ++++--
 refs.h                           |  23 +-
 refs/files-backend.c             | 141 +++++++++--
 refs/refs-internal.h             |  20 ++
 refs/reftable-backend.c          |  49 +++-
 sequencer.c                      |   9 +-
 t/t0600-reffiles-backend.sh      |  32 +++
 t/t1400-update-ref.sh            | 413 ++++++++++++++++++++++++++++++-
 t/t1416-ref-transaction-hooks.sh |  41 +++
 walker.c                         |   2 +-
 20 files changed, 1030 insertions(+), 86 deletions(-)

-- 
2.43.GIT

[PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `ref_transaction[_add]_update` functions obtain ref information and
flags to create a `ref_update` and add it to the transaction at hand.

To extend symref support in transactions, we need to also accept the
old and new ref targets and process it. In this commit, let's add the
required parameters to the function and modify all call sites.

The two parameters added are `new_target` and `old_target`. The
`new_target` is used to denote what the reference should point to when
the transaction is applied.

The `old_target` denotes the value the reference must have before the
update. Some functions allow this parameter to be NULL, meaning that the
old value of the reference is not checked.

The handling logic of these parameters will be added in consequent
commits as we add symref commands to the '--stdin' mode of
'git-update-ref'.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 17 ++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 13 +++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 69 insertions(+), 24 deletions(-)

diff --git a/branch.c b/branch.c
index e4a738fc7b..48af4c3ceb 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index dc5a9d32dd..297dfb91a1 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5857d860db..66840b7c5b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e8d7df14b6..b150ef39a8 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..7690687b0e 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 9a33cb50b4..40a65fdebc 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index e46afbc46d..21fdbf6ac8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
diff --git a/refs.c b/refs.c
index 55d2e0b2cb..060a31616d 100644
--- a/refs.c
+++ b/refs.c
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && !is_null_oid(old_oid) && old_target)
+		BUG("Only one of old_oid and old_target should be non NULL");
+	if (new_oid && !is_null_oid(new_oid) && new_target)
+		BUG("Only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index d278775e08..c792e13a64 100644
--- a/refs.h
+++ b/refs.h
@@ -648,6 +648,15 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         update to point to. This takes precedence over new_oid when
+ *         set. If the reference is a regular reference, it will be
+ *         converted to a symbolic reference.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Will only be taken into account when the reference is a symbolic
+ *         reference.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +722,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +735,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index a098d14ea0..e4d0aa3d41 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..3040d4797c 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -124,6 +124,18 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set and the reference is a symbolic ref, check that the
+	 * reference previously pointed to this value.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +185,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 1cda48c504..6104471199 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 2c19846385..af1b25692b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -616,7 +616,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1248,7 +1248,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3764,8 +3764,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index c0fd632d92..1b3df43906 100644
--- a/walker.c
+++ b/walker.c
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
-- 
2.43.GIT

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> From: Karthik Nayak <karthik.188@gmail.com>
> Subject: Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`
>
> The `ref_transaction[_add]_update` functions obtain ref information and
> flags to create a `ref_update` and add it to the transaction at hand.

Just a very minor irritation, but ref_transaction_add_update() is a
function used internally in the ref subsystem and is exported only
because its visibility needs to cross file boundaries between refs.c
and refs/*backend.c files.

It would be better to only mention ref_transaction_update() in the
title, and talk about the need to make matching adjustment to
ref_transaction_add_update(), which is an internal function, in the
body of the log message.

This is an unrelated #leftoverbits tangent, but while trying to find
out the reason why "[_add]" in the title looked irritating to me, I
noticed that builtin/show-ref.c includes <refs/refs-internal.h>.  I
do not know what it uses from the "internal" implementation detail,
but the API may have to be cleaned up so that a random "client"
caller do not have to do so.

The patch itself looked good.  Thanks.

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Jeff King]

On Fri, Apr 26, 2024 at 12:31:36PM -0700, Junio C Hamano wrote:

> This is an unrelated #leftoverbits tangent, but while trying to find
> out the reason why "[_add]" in the title looked irritating to me, I
> noticed that builtin/show-ref.c includes <refs/refs-internal.h>.  I
> do not know what it uses from the "internal" implementation detail,
> but the API may have to be cleaned up so that a random "client"
> caller do not have to do so.

There are two issues. One is the use of refs_read_raw_ref(), added by
Patrick's 9080a7f178 (builtin/show-ref: add new mode to check for
reference existence, 2023-10-31). And it argues there why the regular
API is unsufficient (mostly because it does not protect errno).

But the more interesting one is a call to refname_is_safe(), added
recently by Phillip's 1dbe401563 (show-ref --verify: accept pseudorefs,
2024-02-07). Looking at that commit, the intent was to allow pseudo-refs
by loosening the conditional that checked "HEAD" to allow "FOO_BAR" but
not "foobar" outside of "refs/". We enforce the all-caps pseudoref
syntax in is_refname_safe().

The proper API there is I think check_ref_format() with ALLOW_ONELEVEL.
But you shouldn't need to do that, because the refs code should be
checking the names itself (using check_ref_format() usually, but falling
back to refname_is_safe() if the ALLOW_BAD_NAME flag is passed).

And I think there _is_ a bug there. The idea of those two functions is
that check_ref_format() would allow a subset of what refname_is_safe()
does. We'd fall back to the latter when deleting, but not otherwise
allow creation or updates.

However, it looks like check_ref_format() doesn't enforce the pseudo-ref
syntax. It will happily resolve this:

  git rev-parse HEAD >.git/foo
  git rev-parse foo

and even update it:

  git update-ref foo HEAD

though curiously we will refuse to delete it:

  $ git update-ref -d foo
  error: refusing to update ref with bad name 'foo'

since that sets the ALLOW_BAD_NAME flag!

IMHO these should _all_ be forbidden, because we only want to allow the
more limited pseudoref names everywhere (and never mischievous ones like
"config" or whatever). And once we are doing that, then show-ref has no
need to check the format. It can just call read_ref() and it either gets
an answer or doesn't.

I don't know if that is a #leftoverbit though. It perhaps more
complicated than that.

-Peff

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3305 bytes --]

On Fri, Apr 26, 2024 at 05:15:29PM -0400, Jeff King wrote:
> On Fri, Apr 26, 2024 at 12:31:36PM -0700, Junio C Hamano wrote:
> 
> > This is an unrelated #leftoverbits tangent, but while trying to find
> > out the reason why "[_add]" in the title looked irritating to me, I
> > noticed that builtin/show-ref.c includes <refs/refs-internal.h>.  I
> > do not know what it uses from the "internal" implementation detail,
> > but the API may have to be cleaned up so that a random "client"
> > caller do not have to do so.
> 
> There are two issues. One is the use of refs_read_raw_ref(), added by
> Patrick's 9080a7f178 (builtin/show-ref: add new mode to check for
> reference existence, 2023-10-31). And it argues there why the regular
> API is unsufficient (mostly because it does not protect errno).
> 
> But the more interesting one is a call to refname_is_safe(), added
> recently by Phillip's 1dbe401563 (show-ref --verify: accept pseudorefs,
> 2024-02-07). Looking at that commit, the intent was to allow pseudo-refs
> by loosening the conditional that checked "HEAD" to allow "FOO_BAR" but
> not "foobar" outside of "refs/". We enforce the all-caps pseudoref
> syntax in is_refname_safe().
> 
> The proper API there is I think check_ref_format() with ALLOW_ONELEVEL.
> But you shouldn't need to do that, because the refs code should be
> checking the names itself (using check_ref_format() usually, but falling
> back to refname_is_safe() if the ALLOW_BAD_NAME flag is passed).
> 
> And I think there _is_ a bug there. The idea of those two functions is
> that check_ref_format() would allow a subset of what refname_is_safe()
> does. We'd fall back to the latter when deleting, but not otherwise
> allow creation or updates.
> 
> However, it looks like check_ref_format() doesn't enforce the pseudo-ref
> syntax. It will happily resolve this:
> 
>   git rev-parse HEAD >.git/foo
>   git rev-parse foo
> 
> and even update it:
> 
>   git update-ref foo HEAD
> 
> though curiously we will refuse to delete it:
> 
>   $ git update-ref -d foo
>   error: refusing to update ref with bad name 'foo'
> 
> since that sets the ALLOW_BAD_NAME flag!
> 
> IMHO these should _all_ be forbidden, because we only want to allow the
> more limited pseudoref names everywhere (and never mischievous ones like
> "config" or whatever). And once we are doing that, then show-ref has no
> need to check the format. It can just call read_ref() and it either gets
> an answer or doesn't.
> 
> I don't know if that is a #leftoverbit though. It perhaps more
> complicated than that.

Yeah, this is something that I've repeatedly stumbled over myself. If I
remember correctly, the plan was to clean up and consolidate all these
different functions we have for checking ref names such that they become
easier to use and hopefully lead to more consistent behaviour.

In any case, I very much agree that git-update-ref(1) should refuse to
write refs with names that are known-bad. There should probably be an
escape hatch though that at least allows you to _delete_ those, or
otherwise there is no way to remove such a ref in the reftable repo.
Well, except for meddling with the binary format, but I doubt that
anybody would really want to do that.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Jeff King]

On Mon, Apr 29, 2024 at 09:02:56AM +0200, Patrick Steinhardt wrote:

> > IMHO these should _all_ be forbidden, because we only want to allow the
> > more limited pseudoref names everywhere (and never mischievous ones like
> > "config" or whatever). And once we are doing that, then show-ref has no
> > need to check the format. It can just call read_ref() and it either gets
> > an answer or doesn't.
> > 
> > I don't know if that is a #leftoverbit though. It perhaps more
> > complicated than that.
> 
> Yeah, this is something that I've repeatedly stumbled over myself. If I
> remember correctly, the plan was to clean up and consolidate all these
> different functions we have for checking ref names such that they become
> easier to use and hopefully lead to more consistent behaviour.

I think the code changes here aren't too bad (modulo some
head-scratching I had to do around per-worktree refs). I'll post a
series in a moment (split off from here since we're far off topic from
the original thread).

> In any case, I very much agree that git-update-ref(1) should refuse to
> write refs with names that are known-bad. There should probably be an
> escape hatch though that at least allows you to _delete_ those, or
> otherwise there is no way to remove such a ref in the reftable repo.
> Well, except for meddling with the binary format, but I doubt that
> anybody would really want to do that.

Yeah, ironically deleting is the one thing you cannot do with them even
right now. ;) That is because the supposedly-looser refname_is_safe() is
actually tighter.

It would be tough to loosen it safely, since you don't want to allow
deleting arbitrary files in .git. Or worse, escaping .git via clever
paths, symlinks, etc. So I think at most you'd want some kind of "trust
me, for this command don't bother with ref format checks" flag.

Or alternatively, refname_is_safe() should become a per-backend thing.
And then reftables can say "everything is safe", because the ref names
are never used as paths (I think, anyway; I didn't follow all of the
discussion around per-worktree refs, pseudorefs, etc).

-Peff

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[phillip.wood123]

On 29/04/2024 08:55, Jeff King wrote:
> On Mon, Apr 29, 2024 at 09:02:56AM +0200, Patrick Steinhardt wrote:
> 
>> In any case, I very much agree that git-update-ref(1) should refuse to
>> write refs with names that are known-bad. There should probably be an
>> escape hatch though that at least allows you to _delete_ those, or
>> otherwise there is no way to remove such a ref in the reftable repo.
>> Well, except for meddling with the binary format, but I doubt that
>> anybody would really want to do that.
> 
> Yeah, ironically deleting is the one thing you cannot do with them even
> right now. ;) That is because the supposedly-looser refname_is_safe() is
> actually tighter.
> 
> It would be tough to loosen it safely, since you don't want to allow
> deleting arbitrary files in .git. Or worse, escaping .git via clever
> paths, symlinks, etc. So I think at most you'd want some kind of "trust
> me, for this command don't bother with ref format checks" flag.

Currently "git update-ref config HEAD" fails because the config file 
does not look like a ref - could we do similar check when deleting 
references at least for the files backend for arguments outside refs/?

Best Wishes

Phillip

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[phillip.wood123]

Hi Peff

On 26/04/2024 22:15, Jeff King wrote:
> On Fri, Apr 26, 2024 at 12:31:36PM -0700, Junio C Hamano wrote:
> 
> But the more interesting one is a call to refname_is_safe(), added
> recently by Phillip's 1dbe401563 (show-ref --verify: accept pseudorefs,
> 2024-02-07). Looking at that commit, the intent was to allow pseudo-refs
> by loosening the conditional that checked "HEAD" to allow "FOO_BAR" but
> not "foobar" outside of "refs/". We enforce the all-caps pseudoref
> syntax in is_refname_safe().
> 
> The proper API there is I think check_ref_format() with ALLOW_ONELEVEL.

ALLOW_ONELEVEL just disables the check that the refname contains a '/' 
and I think it is aimed at checking branch and tag names without a 
refs/{heads,tags} prefix. If we want to move away from using 
refname_is_safe() perhaps we could add an ALLOW_PSEUDOREF flag that only 
allows the name to contain '[A-Z_]' if there is no '/'.

Best Wishes

Phillip

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Junio C Hamano]

phillip.wood123@gmail.com writes:

> ALLOW_ONELEVEL just disables the check that the refname contains a '/'
> and I think it is aimed at checking branch and tag names without a
> refs/{heads,tags} prefix. If we want to move away from using
> refname_is_safe() perhaps we could add an ALLOW_PSEUDOREF flag that
> only allows the name to contain '[A-Z_]' if there is no '/'.

Makes sense.

I wonder if we eventually can get rid of ALLOW_ONELEVEL, though.  If
all callers that use ALLOW_ONELEVEL know under which prefix they
plan to hang the refname they are checking (if the refname passed
the check), we can force the check to be performed always on the
full refname, and it will become easier to make the check more
consistent---as the check will have full context information.

For example, with ALLOW_ONELEVEL the check may say "HEAD" is OK, but
if we get rid of ALLOW_ONELEVEL and force the callers to always test
the full refname, we may say "refs/heads/HEAD" is not acceptable,
neither is "refs/tags/HEAD", but "refs/remotes/origin/HEAD" is good.

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Jeff King]

On Mon, Apr 29, 2024 at 09:18:47AM -0700, Junio C Hamano wrote:

> phillip.wood123@gmail.com writes:
> 
> > ALLOW_ONELEVEL just disables the check that the refname contains a '/'
> > and I think it is aimed at checking branch and tag names without a
> > refs/{heads,tags} prefix. If we want to move away from using
> > refname_is_safe() perhaps we could add an ALLOW_PSEUDOREF flag that
> > only allows the name to contain '[A-Z_]' if there is no '/'.
> 
> Makes sense.
> 
> I wonder if we eventually can get rid of ALLOW_ONELEVEL, though.  If
> all callers that use ALLOW_ONELEVEL know under which prefix they
> plan to hang the refname they are checking (if the refname passed
> the check), we can force the check to be performed always on the
> full refname, and it will become easier to make the check more
> consistent---as the check will have full context information.
> 
> For example, with ALLOW_ONELEVEL the check may say "HEAD" is OK, but
> if we get rid of ALLOW_ONELEVEL and force the callers to always test
> the full refname, we may say "refs/heads/HEAD" is not acceptable,
> neither is "refs/tags/HEAD", but "refs/remotes/origin/HEAD" is good.

One case I ran into while working on my series is refspec parsing, where
we feed the left and right halves to check_refname_format(), albeit with
a special flag that allows the "*" wildcard character. And there we are
OK with one-level names because they will be passed through the
dwim_ref() lookup.

I don't know if there are other spots that do something similar. Most of
them would, I imagine, just take any input and leave it to the ref code
to enforce syntax after the dwim-ref has happened.

-Peff

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Jeff King]

On Mon, Apr 29, 2024 at 10:32:58AM +0100, phillip.wood123@gmail.com wrote:

> On 26/04/2024 22:15, Jeff King wrote:
> > On Fri, Apr 26, 2024 at 12:31:36PM -0700, Junio C Hamano wrote:
> > 
> > But the more interesting one is a call to refname_is_safe(), added
> > recently by Phillip's 1dbe401563 (show-ref --verify: accept pseudorefs,
> > 2024-02-07). Looking at that commit, the intent was to allow pseudo-refs
> > by loosening the conditional that checked "HEAD" to allow "FOO_BAR" but
> > not "foobar" outside of "refs/". We enforce the all-caps pseudoref
> > syntax in is_refname_safe().
> > 
> > The proper API there is I think check_ref_format() with ALLOW_ONELEVEL.
> 
> ALLOW_ONELEVEL just disables the check that the refname contains a '/' and I
> think it is aimed at checking branch and tag names without a
> refs/{heads,tags} prefix. If we want to move away from using
> refname_is_safe() perhaps we could add an ALLOW_PSEUDOREF flag that only
> allows the name to contain '[A-Z_]' if there is no '/'.

Right, I think that was the original reason for ALLOW_ONELEVEL. But then
lots of callers ended up using it because it was the only way to allow
"HEAD" or "FETCH_HEAD" to work alongside regular refs.

See the series I posted elsewhere which adds a new flag to cover that
case.

-Peff

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 689 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> From: Karthik Nayak <karthik.188@gmail.com>
>> Subject: Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`
>>
>> The `ref_transaction[_add]_update` functions obtain ref information and
>> flags to create a `ref_update` and add it to the transaction at hand.
>
> Just a very minor irritation, but ref_transaction_add_update() is a
> function used internally in the ref subsystem and is exported only
> because its visibility needs to cross file boundaries between refs.c
> and refs/*backend.c files.

Yes that is true. I'll amend this for the next version. Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Phillip Wood]

Hi Karthik

On 26/04/2024 16:24, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The `ref_transaction[_add]_update` functions obtain ref information and
> flags to create a `ref_update` and add it to the transaction at hand.
> 
> To extend symref support in transactions, we need to also accept the
> old and new ref targets and process it.

s/it/them/

> In this commit, let's add the

This commit adds?

> required parameters to the function and modify all call sites.
> 
> The two parameters added are `new_target` and `old_target`. The
> `new_target` is used to denote what the reference should point to when
> the transaction is applied.
> 
> The `old_target` denotes the value the reference must have before the
> update. Some functions allow this parameter to be NULL, meaning that the
> old value of the reference is not checked.
> 
> The handling logic of these parameters will be added in consequent
> commits as we add symref commands to the '--stdin' mode of
> 'git-update-ref'.
> 
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>

Thanks for updating the documentation, I've left a couple of comments below

> diff --git a/refs.h b/refs.h
> index d278775e08..c792e13a64 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -648,6 +648,15 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>    *         before the update. A copy of this value is made in the
>    *         transaction.
>    *
> + *     new_target -- the target reference that the reference will be
> + *         update to point to.

s/update/updated/

> This takes precedence over new_oid when set.

I thought it was a bug to set both new_oid and new_target.

> If the reference is a regular reference, it will be
> + *         converted to a symbolic reference.
 > + *
> + *     old_target -- the reference that the reference must be pointing to.
> + *         Will only be taken into account when the reference is a symbolic
> + *         reference.

Does this last sentence mean it is not possible to assert that it is 
currently a symbolic reference? I thought the point of being able to 
specify the old value of a ref when updating was to ensure it hadn't 
changed since it was read. This contradicts the documentation in the 
next hunk and the description in the commit message.

>    *     flags -- flags affecting the update, passed to
>    *         update_ref_lock(). Possible flags: REF_NO_DEREF,
>    *         REF_FORCE_CREATE_REFLOG. See those constants for more
> @@ -713,7 +722,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>    * beforehand. The old value is checked after the lock is taken to
>    * prevent races. If the old value doesn't agree with old_oid, the
>    * whole transaction fails. If old_oid is NULL, then the previous
> - * value is not checked.
> + * value is not checked. If `old_target` is not NULL, treat the reference
> + * as a symbolic ref and validate that its target before the update is
> + * `old_target`. If the `new_target` is not NULL, then the reference
> + * will be updated to a symbolic ref which targets `new_target`.

This looks good and describes the behavior I'd expected to see.

Best Wishes

Phillip

Re: [PATCH v4 1/7] refs: accept symref values in `ref_transaction[_add]_update`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2753 bytes --]

Phillip Wood <phillip.wood123@gmail.com> writes:

> Hi Karthik
>
> On 26/04/2024 16:24, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The `ref_transaction[_add]_update` functions obtain ref information and
>> flags to create a `ref_update` and add it to the transaction at hand.
>>
>> To extend symref support in transactions, we need to also accept the
>> old and new ref targets and process it.
>
> s/it/them/
>
>> In this commit, let's add the
>
> This commit adds?
>

Thanks, will add both the above.

>> required parameters to the function and modify all call sites.
>>
>> The two parameters added are `new_target` and `old_target`. The
>> `new_target` is used to denote what the reference should point to when
>> the transaction is applied.
>>
>> The `old_target` denotes the value the reference must have before the
>> update. Some functions allow this parameter to be NULL, meaning that the
>> old value of the reference is not checked.
>>
>> The handling logic of these parameters will be added in consequent
>> commits as we add symref commands to the '--stdin' mode of
>> 'git-update-ref'.
>>
>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>
> Thanks for updating the documentation, I've left a couple of comments below
>

Thanks for the review.

>> diff --git a/refs.h b/refs.h
>> index d278775e08..c792e13a64 100644
>> --- a/refs.h
>> +++ b/refs.h
>> @@ -648,6 +648,15 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>>    *         before the update. A copy of this value is made in the
>>    *         transaction.
>>    *
>> + *     new_target -- the target reference that the reference will be
>> + *         update to point to.
>
> s/update/updated/
>
>> This takes precedence over new_oid when set.
>
> I thought it was a bug to set both new_oid and new_target.
>

Yup. fixed both of these.

>
>> If the reference is a regular reference, it will be
>> + *         converted to a symbolic reference.
>  > + *
>> + *     old_target -- the reference that the reference must be pointing to.
>> + *         Will only be taken into account when the reference is a symbolic
>> + *         reference.
>
> Does this last sentence mean it is not possible to assert that it is
> currently a symbolic reference? I thought the point of being able to
> specify the old value of a ref when updating was to ensure it hadn't
> changed since it was read. This contradicts the documentation in the
> next hunk and the description in the commit message.
>

I see how this is vague, I was trying to imply that the old_target is
used to check a symref's old_value. But actually, if this is set and the
ref is a regular ref, we do fail the check. So this is wrong. Let me
just strip the last time.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v4 2/7] files-backend: extract out `create_symref_lock`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Split this into two individual functions `create_and_commit_symref` and
`create_symref_locked`. This way we can create the symref lock and
commit it at different times. This will be used to provide symref
support in `git-update-ref(1)`.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 40 +++++++++++++++++++++++++++-------------
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index e4d0aa3d41..2420dac2aa 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1920,26 +1920,39 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target)
 {
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	/* no error check; commit_ref will check ferror */
+	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
 	return 0;
 }
 
@@ -1960,7 +1973,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

Re: [PATCH v4 2/7] files-backend: extract out `create_symref_lock`

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> From: Karthik Nayak <karthik.188@gmail.com>
>
> The function `create_symref_locked` creates a symref by creating a
> '<symref>.lock' file and then committing the symref lock, which creates
> the final symref.
>
> Split this into two individual functions `create_and_commit_symref` and
> `create_symref_locked`. This way we can create the symref lock and
> commit it at different times. This will be used to provide symref
> support in `git-update-ref(1)`.

It is a confusing way to describe what this patch did, though.  If
you truly splitted create_symref_locked() into two, you would have
functions A and B, and existing callers of create_symref_locked()
would be changed to call A() and then B().  I do not think such a
split would make sense in this case, but the above description gives
an impression that it was what you did.

In reality, an early part of create_symref_locked() was made into a
separate helper function that can be called from callers other than
create_symref_locked(), and because the helper got a name too
similar to the original, you had to rename create_symref_locked() to
create_and_commit_symref().  The existing callers of it are not
affected, modulo the name change.

Perhaps

    Split the early half of create_symref_locked() into a new helper
    funciton create_symref_lock().  Because the name of the new
    function is too similar to the original, rename the original to
    create_and_commit_symref() to avoid confusion.

    The new helper will be used to ...

or something?

> -static int create_symref_locked(struct files_ref_store *refs,
> -				struct ref_lock *lock, const char *refname,
> -				const char *target, const char *logmsg)
> +static int create_symref_lock(struct files_ref_store *refs,
> +			      struct ref_lock *lock, const char *refname,
> +			      const char *target)
>  {
> +	if (!fdopen_lock_file(&lock->lk, "w"))
> +		return error("unable to fdopen %s: %s",
> +			     get_lock_file_path(&lock->lk), strerror(errno));
> +
> +	/* no error check; commit_ref will check ferror */
> +	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);

This was a bit puzzling (see below).

> +	return 0;
> +}
> +
> +static int create_and_commit_symref(struct files_ref_store *refs,
> +				    struct ref_lock *lock, const char *refname,
> +				    const char *target, const char *logmsg)
> +{
> +	int ret;
> +
>  	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
>  		update_symref_reflog(refs, lock, refname, target, logmsg);
>  		return 0;
>  	}

    Offtopic: we might want to start planning to deprecate creation
    of "symlink refs".  Linus originally used a symlink for
    .git/HEAD, but 9f0bb90d (core.prefersymlinkrefs: use symlinks
    for .git/HEAD, 2006-05-02) made it default not to use of
    symbolic links.  As long as we preserve the ability to work on a
    repository whose HEAD still uses a symbolic link, I'd hope
    nothing would break (#leftoverbits).

Let me rearrange this hunk to show the original first:

> -	if (!fdopen_lock_file(&lock->lk, "w"))
> -		return error("unable to fdopen %s: %s",
> -			     get_lock_file_path(&lock->lk), strerror(errno));
> -	update_symref_reflog(refs, lock, refname, target, logmsg);
> -	/* no error check; commit_ref will check ferror */
> -	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
> -	if (commit_ref(lock) < 0)
> -		return error("unable to write symref for %s: %s", refname,
> -			     strerror(errno));

The original in create_symref_locked() created a lockfile, called
update_symref_reflog(), and called commit_ref() to commit the thing.

The "no error check" comment is about detecting an error while
writing into the lock file.  It came from 370e5ad6 (create_symref:
use existing ref-lock code, 2015-12-29).  Because the fprintf() call
was immediately followed by commit_ref(), and the code assumed that
commit_ref() will check ferror(), we do not bother checking if the
fprintf() call fails to write the contents correctly.

> +	ret = create_symref_lock(refs, lock, refname, target);
> +	if (!ret) {
> +		update_symref_reflog(refs, lock, refname, target, logmsg);
>  
> +		if (commit_ref(lock) < 0)
> +			return error("unable to write symref for %s: %s", refname,
> +				     strerror(errno));
> +	}

The new code lets create_symref_lock() to create a lockfile, and
does the rest here.  commit_ref() does call commit_lock_file(),
which eventually passes the control to close_tempfile() and a
write error can be detected there.

But the point of this patch is that the creation of the locked
symref file PLUS writing its new contents (which is done by
create_symref_lock()) can be done way ahead of the remainder that
eventually does commit_ref().  So it smells a bit[*] dubious that we
still leave the error from fprintf() ignored in the "early half" in
the rearranged code.

	Side note: it is a "bit", as it is unlikely that we will do
	something to clear the ferror() from the (FILE *) in the
	meantime.

> @@ -1960,7 +1973,8 @@ static int files_create_symref(struct ref_store *ref_store,
>  		return -1;
>  	}
>  
> -	ret = create_symref_locked(refs, lock, refname, target, logmsg);
> +	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
> +
>  	unlock_ref(lock);
>  	return ret;
>  }

This hunk shows the "original function was renamed; there is no
other changes visible to the caller" nature of this rearrangement.

The extra blank line is probably a nice touch.

Re: [PATCH v4 2/7] files-backend: extract out `create_symref_lock`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 6127 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The function `create_symref_locked` creates a symref by creating a
>> '<symref>.lock' file and then committing the symref lock, which creates
>> the final symref.
>>
>> Split this into two individual functions `create_and_commit_symref` and
>> `create_symref_locked`. This way we can create the symref lock and
>> commit it at different times. This will be used to provide symref
>> support in `git-update-ref(1)`.
>
> It is a confusing way to describe what this patch did, though.  If
> you truly splitted create_symref_locked() into two, you would have
> functions A and B, and existing callers of create_symref_locked()
> would be changed to call A() and then B().  I do not think such a
> split would make sense in this case, but the above description gives
> an impression that it was what you did.
>
> In reality, an early part of create_symref_locked() was made into a
> separate helper function that can be called from callers other than
> create_symref_locked(), and because the helper got a name too
> similar to the original, you had to rename create_symref_locked() to
> create_and_commit_symref().  The existing callers of it are not
> affected, modulo the name change.
>
> Perhaps
>
>     Split the early half of create_symref_locked() into a new helper
>     funciton create_symref_lock().  Because the name of the new
>     function is too similar to the original, rename the original to
>     create_and_commit_symref() to avoid confusion.
>
>     The new helper will be used to ...
>
> or something?
>

Thanks. I agree with what you're saying. I would also s/Split/Extract
perhaps because it drives the point better.

>> -static int create_symref_locked(struct files_ref_store *refs,
>> -				struct ref_lock *lock, const char *refname,
>> -				const char *target, const char *logmsg)
>> +static int create_symref_lock(struct files_ref_store *refs,
>> +			      struct ref_lock *lock, const char *refname,
>> +			      const char *target)
>>  {
>> +	if (!fdopen_lock_file(&lock->lk, "w"))
>> +		return error("unable to fdopen %s: %s",
>> +			     get_lock_file_path(&lock->lk), strerror(errno));
>> +
>> +	/* no error check; commit_ref will check ferror */
>> +	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
>
> This was a bit puzzling (see below).
>
>> +	return 0;
>> +}
>> +
>> +static int create_and_commit_symref(struct files_ref_store *refs,
>> +				    struct ref_lock *lock, const char *refname,
>> +				    const char *target, const char *logmsg)
>> +{
>> +	int ret;
>> +
>>  	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
>>  		update_symref_reflog(refs, lock, refname, target, logmsg);
>>  		return 0;
>>  	}
>
>     Offtopic: we might want to start planning to deprecate creation
>     of "symlink refs".  Linus originally used a symlink for
>     .git/HEAD, but 9f0bb90d (core.prefersymlinkrefs: use symlinks
>     for .git/HEAD, 2006-05-02) made it default not to use of
>     symbolic links.  As long as we preserve the ability to work on a
>     repository whose HEAD still uses a symbolic link, I'd hope
>     nothing would break (#leftoverbits).
>
> Let me rearrange this hunk to show the original first:
>
>> -	if (!fdopen_lock_file(&lock->lk, "w"))
>> -		return error("unable to fdopen %s: %s",
>> -			     get_lock_file_path(&lock->lk), strerror(errno));
>> -	update_symref_reflog(refs, lock, refname, target, logmsg);
>> -	/* no error check; commit_ref will check ferror */
>> -	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
>> -	if (commit_ref(lock) < 0)
>> -		return error("unable to write symref for %s: %s", refname,
>> -			     strerror(errno));
>
> The original in create_symref_locked() created a lockfile, called
> update_symref_reflog(), and called commit_ref() to commit the thing.
>
> The "no error check" comment is about detecting an error while
> writing into the lock file.  It came from 370e5ad6 (create_symref:
> use existing ref-lock code, 2015-12-29).  Because the fprintf() call
> was immediately followed by commit_ref(), and the code assumed that
> commit_ref() will check ferror(), we do not bother checking if the
> fprintf() call fails to write the contents correctly.
>
>> +	ret = create_symref_lock(refs, lock, refname, target);
>> +	if (!ret) {
>> +		update_symref_reflog(refs, lock, refname, target, logmsg);
>>
>> +		if (commit_ref(lock) < 0)
>> +			return error("unable to write symref for %s: %s", refname,
>> +				     strerror(errno));
>> +	}
>
> The new code lets create_symref_lock() to create a lockfile, and
> does the rest here.  commit_ref() does call commit_lock_file(),
> which eventually passes the control to close_tempfile() and a
> write error can be detected there.
>
> But the point of this patch is that the creation of the locked
> symref file PLUS writing its new contents (which is done by
> create_symref_lock()) can be done way ahead of the remainder that
> eventually does commit_ref().  So it smells a bit[*] dubious that we
> still leave the error from fprintf() ignored in the "early half" in
> the rearranged code.
>
> 	Side note: it is a "bit", as it is unlikely that we will do
> 	something to clear the ferror() from the (FILE *) in the
> 	meantime.
>

You're right. I would say that perhaps it is a bit more than a 'bit
dubious' since `commit_ref()` being called after, is no longer a
guarantee. I'll go ahead and add error handling here.

>> @@ -1960,7 +1973,8 @@ static int files_create_symref(struct ref_store *ref_store,
>>  		return -1;
>>  	}
>>
>> -	ret = create_symref_locked(refs, lock, refname, target, logmsg);
>> +	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
>> +
>>  	unlock_ref(lock);
>>  	return ret;
>>  }
>
> This hunk shows the "original function was renamed; there is no
> other changes visible to the caller" nature of this rearrangement.
>
> The extra blank line is probably a nice touch.

Thanks. I'm sure it's not the best idea to introduce whitespace, but this
felt more readable here.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v4 3/7] update-ref: add support for 'symref-verify' command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

In the previous commits, we added the required base for adding symref
commands to the '--stdin' mode provided by 'git-update-ref(1)'. Using
them, add a new 'symref-verify' command to verify symrefs.

The 'symref-verify' command allows users to verify if a provided <ref>
contains the provided <old-target> without changing the <ref>. If
<old-target> is not provided, the command will verify that the <ref>
doesn't exist. Since we're checking for symbolic refs, this command will
only work with the 'no-deref' mode. This is because any dereferenced
symbolic ref will point to an object and not a ref and the regular
'verify' command can be used in such situations.

Add and use `ref_update_is_null_new_value`, a helper function which is
used to check if there is a new_value in a reference update. The new
value could either be a symref target `new_target` or a OID `new_oid`.
We also add tests to test the command in both the regular stdin mode and
also with the '-z' flag.

We also disable the reference-transaction hook for symref-updates which
will be tackled in its own commit.

Add required tests for symref support in 'verify' while also adding
reflog checks for the pre-existing 'verify' tests.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |  7 +++
 builtin/update-ref.c             | 80 +++++++++++++++++++++++----
 refs.c                           | 30 +++++++++--
 refs.h                           |  1 +
 refs/files-backend.c             | 43 +++++++++++++++
 refs/refs-internal.h             |  7 +++
 refs/reftable-backend.c          | 21 +++++++-
 t/t1400-update-ref.sh            | 93 +++++++++++++++++++++++++++++++-
 8 files changed, 264 insertions(+), 18 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 374a2ebd2b..9fe78b3501 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-verify SP <ref> [SP <old-target>] LF
 	option SP <opt> LF
 	start LF
 	prepare LF
@@ -86,6 +87,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-verify SP <ref> [NUL <old-target>] NUL
 	option SP <opt> NUL
 	start NUL
 	prepare NUL
@@ -117,6 +119,11 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-verify::
+	Verify symbolic <ref> against <old-target> but do not change it.
+	If <old-target> is missing, the ref must not exist.  Can only be
+	used in `no-deref` mode.
+
 option::
 	Modify the behavior of the next command naming a <ref>.
 	The only valid option is `no-deref` to avoid dereferencing
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 21fdbf6ac8..419b28169b 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -76,6 +76,29 @@ static char *parse_refname(const char **next)
 	return strbuf_detach(&ref, NULL);
 }
 
+/*
+ * Wrapper around parse_refname which skips the next delimiter.
+ */
+static char *parse_next_refname(const char **next)
+{
+	if (line_termination) {
+		/* Without -z, consume SP and use next argument */
+		if (!**next || **next == line_termination)
+			return NULL;
+		if (**next != ' ')
+			die("expected SP but got: %s", *next);
+	} else {
+		/* With -z, read the next NUL-terminated line */
+		if (**next)
+			return NULL;
+	}
+	/* Skip the delimiter */
+	(*next)++;
+
+	return parse_refname(next);
+}
+
+
 /*
  * The value being parsed is <old-oid> (as opposed to <new-oid>; the
  * difference affects which error messages are generated):
@@ -297,11 +320,47 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
 		die("verify %s: extra input: %s", refname, next);
 
 	if (ref_transaction_verify(transaction, refname, &old_oid,
-				   update_flags, &err))
+				   NULL, update_flags, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	strbuf_release(&err);
+}
+
+static void parse_cmd_symref_verify(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	struct object_id old_oid;
+	char *refname, *old_target;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-verify: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-verify: missing <ref>");
+
+	/*
+	 * old_ref is optional, but we want to differentiate between
+	 * a NULL and zero value.
+	 */
+	old_target = parse_next_refname(&next);
+	if (!old_target)
+		old_oid = *null_oid();
+
+	if (*next != line_termination)
+		die("symref-verify %s: extra input: %s", refname, next);
+
+	if (ref_transaction_verify(transaction, refname,
+				   old_target ? NULL : &old_oid,
+				   old_target, update_flags, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
 	free(refname);
+	free(old_target);
 	strbuf_release(&err);
 }
 
@@ -380,15 +439,16 @@ static const struct parse_cmd {
 	unsigned args;
 	enum update_refs_state state;
 } command[] = {
-	{ "update",  parse_cmd_update,  3, UPDATE_REFS_OPEN },
-	{ "create",  parse_cmd_create,  2, UPDATE_REFS_OPEN },
-	{ "delete",  parse_cmd_delete,  2, UPDATE_REFS_OPEN },
-	{ "verify",  parse_cmd_verify,  2, UPDATE_REFS_OPEN },
-	{ "option",  parse_cmd_option,  1, UPDATE_REFS_OPEN },
-	{ "start",   parse_cmd_start,   0, UPDATE_REFS_STARTED },
-	{ "prepare", parse_cmd_prepare, 0, UPDATE_REFS_PREPARED },
-	{ "abort",   parse_cmd_abort,   0, UPDATE_REFS_CLOSED },
-	{ "commit",  parse_cmd_commit,  0, UPDATE_REFS_CLOSED },
+	{ "update",        parse_cmd_update,        3, UPDATE_REFS_OPEN },
+	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
+	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
+	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
+	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
+	{ "start",         parse_cmd_start,         0, UPDATE_REFS_STARTED },
+	{ "prepare",       parse_cmd_prepare,       0, UPDATE_REFS_PREPARED },
+	{ "abort",         parse_cmd_abort,         0, UPDATE_REFS_CLOSED },
+	{ "commit",        parse_cmd_commit,        0, UPDATE_REFS_CLOSED },
 };
 
 static void update_refs_stdin(void)
diff --git a/refs.c b/refs.c
index 060a31616d..0e1013b5ab 100644
--- a/refs.c
+++ b/refs.c
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((void *)transaction->updates[i]->old_target);
+		free((void *)transaction->updates[i]->new_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,9 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);
+	if (new_oid && flags & REF_HAVE_NEW)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if (old_oid && flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);
 	update->msg = normalize_reflog_message(msg);
 	return update;
@@ -1286,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -1325,14 +1332,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err)
 {
-	if (!old_oid)
-		BUG("verify called with old_oid set to NULL");
+	if (!old_target && !old_oid)
+		BUG("verify called with old_oid and old_target set to NULL");
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("verify cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
-				      NULL, NULL,
+				      NULL, old_target,
 				      flags, NULL, err);
 }
 
@@ -2349,6 +2359,12 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
 
+		/*
+		 * Skip reference transaction for symbolic refs.
+		 */
+		if (update->new_target || update->old_target)
+			continue;
+
 		strbuf_reset(&buf);
 		strbuf_addf(&buf, "%s %s %s\n",
 			    oid_to_hex(&update->old_oid),
@@ -2802,3 +2818,7 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int ref_update_is_null_new_value(struct ref_update *update) {
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
diff --git a/refs.h b/refs.h
index c792e13a64..27b9aeaf54 100644
--- a/refs.h
+++ b/refs.h
@@ -780,6 +780,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 2420dac2aa..53197fa3af 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2425,6 +2425,37 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+/*
+ * Check whether the REF_HAVE_OLD and old_target values stored in
+ * update are consistent with ref, which is the symbolic reference's
+ * current value. If everything is OK, return 0; otherwise, write an
+ * error message to err and return -1.
+ */
+static int check_old_target(struct ref_update *update, char *ref,
+			    struct strbuf *err)
+{
+	if (!(update->flags & REF_HAVE_OLD) ||
+	    !strcmp(update->old_target, ref))
+		return 0;
+
+	if (!strcmp(update->old_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference already exists",
+			    original_update_refname(update));
+	else if (!strcmp(ref, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    ref, update->old_target);
+
+	return -1;
+}
+
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2528,6 +2559,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
+			}
+
+			/*
+			 * For symref verification, we need to check the reference value
+			 * rather than the oid. If we're dealing with regular refs or we're
+			 * verifying a dereferenced symref, we then check the oid.
+			 */
+			if (update->old_target) {
+				if (check_old_target(update, referent.buf, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}
 			} else if (check_old_oid(update, &lock->old_oid, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto out;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3040d4797c..23e65f65e8 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -748,4 +748,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_is_null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 6104471199..a2474245aa 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -938,7 +938,26 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {
+				if (!strcmp(u->old_target, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "provided target is empty",
+						    original_update_refname(u));
+				else if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
+				ret = -1;
+				goto done;
+			}
+		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 					    "reference already exists"),
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index ec3443cc87..34b29eeac8 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -890,17 +890,23 @@ test_expect_success 'stdin update/create/verify combination works' '
 '
 
 test_expect_success 'stdin verify succeeds for correct value' '
+	test-tool ref-store main for-each-reflog-ent $m >before &&
 	git rev-parse $m >expect &&
 	echo "verify $m $m" >stdin &&
 	git update-ref --stdin <stdin &&
 	git rev-parse $m >actual &&
-	test_cmp expect actual
+	test_cmp expect actual &&
+	test-tool ref-store main for-each-reflog-ent $m >after &&
+	test_cmp before after
 '
 
 test_expect_success 'stdin verify succeeds for missing reference' '
+	test-tool ref-store main for-each-reflog-ent $m >before &&
 	echo "verify refs/heads/missing $Z" >stdin &&
 	git update-ref --stdin <stdin &&
-	test_must_fail git rev-parse --verify -q refs/heads/missing
+	test_must_fail git rev-parse --verify -q refs/heads/missing &&
+	test-tool ref-store main for-each-reflog-ent $m >after &&
+	test_cmp before after
 '
 
 test_expect_success 'stdin verify treats no value as missing' '
@@ -1641,4 +1647,87 @@ test_expect_success PIPE 'transaction flushes status updates' '
 	test_cmp expected actual
 '
 
+create_stdin_buf () {
+	if test "$1" = "-z"
+	then
+		shift
+		printf "$F" "$@" >stdin
+	else
+		echo "$@" >stdin
+	fi
+}
+
+for type in "" "-z"
+do
+
+	test_expect_success "stdin ${type} symref-verify fails without --no-deref" '
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: symref-verify: cannot operate with deref mode" err
+	'
+
+	test_expect_success "stdin ${type} symref-verify fails with too many arguments" '
+		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" "$a" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err  &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: $a" err
+		else
+			grep "fatal: symref-verify refs/heads/symref: extra input:  $a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} symref-verify succeeds for correct value" '
+		git symbolic-ref refs/heads/symref >expect &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
+		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$a" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
+		test_cmp before after
+	'
+
+	test_expect_success "stdin ${type} symref-verify no value is treated as zero value" '
+		git symbolic-ref refs/heads/symref >expect &&
+		create_stdin_buf ${type} "symref-verify refs/heads/symref" "" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
+	'
+
+	test_expect_success "stdin ${type} symref-verify succeeds for dangling reference" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref2" &&
+		test_must_fail git symbolic-ref refs/heads/nonexistent &&
+		git symbolic-ref refs/heads/symref2 refs/heads/nonexistent &&
+		create_stdin_buf ${type} "symref-verify refs/heads/symref2" "refs/heads/nonexistent" &&
+		git update-ref --stdin ${type} --no-deref <stdin
+	'
+
+	test_expect_success "stdin ${type} symref-verify succeeds for missing reference" '
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >before &&
+		create_stdin_buf ${type} "symref-verify refs/heads/missing" "$Z" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		test_must_fail git rev-parse --verify -q refs/heads/missing &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >after &&
+		test_cmp before after
+	'
+
+	test_expect_success "stdin ${type} symref-verify fails for wrong value" '
+		git symbolic-ref refs/heads/symref >expect &&
+		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$b" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-verify fails for mistaken null value" '
+		git symbolic-ref refs/heads/symref >expect &&
+		create_stdin_buf ${type} "symref-verify refs/heads/symref" "$Z" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+done
+
 test_done
-- 
2.43.GIT

Re: [PATCH v4 3/7] update-ref: add support for 'symref-verify' command

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> From: Karthik Nayak <karthik.188@gmail.com>
>
> In the previous commits, we added the required base for adding symref
> commands to the '--stdin' mode provided by 'git-update-ref(1)'. Using
> them, add a new 'symref-verify' command to verify symrefs.
>
> The 'symref-verify' command allows users to verify if a provided <ref>
> contains the provided <old-target> without changing the <ref>. If
> <old-target> is not provided, the command will verify that the <ref>
> doesn't exist. Since we're checking for symbolic refs, this command will
> only work with the 'no-deref' mode. This is because any dereferenced
> symbolic ref will point to an object and not a ref and the regular
> 'verify' command can be used in such situations.

All makes sense, but a naïve reader may find it helpful if you
explained why having "verify" command is a good idea in the first
place ("I can just do 'git symoblic-ref' to read the current value,
and see if it is what I expect").  Presumably the value of "verify"
is that you can have it in a transaction and fail other operations
in the same transaction if the symref moved from what you expected
it to point at?

> Add and use `ref_update_is_null_new_value`, a helper function which is
> used to check if there is a new_value in a reference update. The new
> value could either be a symref target `new_target` or a OID `new_oid`.
> We also add tests to test the command in both the regular stdin mode and
> also with the '-z' flag.

This looks out of place, primarily because the helper function is
*NOT* used in this step.  Without any actual user, and with the name
that says only what it checks without hinting why a caller may want
to check the condition it checks, it is hard to guess if it is a
good idea to have such a helper.

"If a ref_update object specifies no new-oid and no new-target, it
is not about updating but just validating" is how the callers are
expected to use it, then instead of is_null_new_value that says
what it checks, something like is_verify_only that says what the
caller may want to use it for would be a more friendly name for
readers and future developers.

> @@ -297,11 +320,47 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
>  		die("verify %s: extra input: %s", refname, next);
>  
>  	if (ref_transaction_verify(transaction, refname, &old_oid,
> -				   update_flags, &err))
> +				   NULL, update_flags, &err))
>  
>  	update_flags = default_flags;
>  	free(refname);
>  	strbuf_release(&err);
>  }

The only damage by this patch to parse_cmd_verify() is that
ref_transaction_verify() gained another parameter NULL, but with the
default "--diff-algorithm=myers" algorithm, it is very hard to see.

The "--patience" algorithm does a much beter job on this hunk.

And the following function is entirely new.

> +static void parse_cmd_symref_verify(struct ref_transaction *transaction,
> +				    const char *next, const char *end)
> +{
> +	struct strbuf err = STRBUF_INIT;
> +	struct object_id old_oid;
> +	char *refname, *old_target;
> +
> +	if (!(update_flags & REF_NO_DEREF))
> +		die("symref-verify: cannot operate with deref mode");
> +
> +	refname = parse_refname(&next);
> +	if (!refname)
> +		die("symref-verify: missing <ref>");
> +
> +	/*
> +	 * old_ref is optional, but we want to differentiate between
> +	 * a NULL and zero value.
> +	 */
> +	old_target = parse_next_refname(&next);
> +	if (!old_target)
> +		old_oid = *null_oid();

In many existing code paths, we do not do structure assignment like
this. Instead we do

		oidcpy(&old_oid, null_oid());

We can see an existing example in a common context in a hunk for
refs.c in this patch.

> +	if (*next != line_termination)
> +		die("symref-verify %s: extra input: %s", refname, next);
> +
> +	if (ref_transaction_verify(transaction, refname,
> +				   old_target ? NULL : &old_oid,
> +				   old_target, update_flags, &err))
> +		die("%s", err.buf);

Are static analyzers smart enough to notice that we will not be
using old_oid uninitialized here?  Just wondering.

Anyway.  This ensures ref_transaction_verify() gets either
old_target or old_oid, but never both at the same time.  The caller
to ref_transaction_verify() in the previous function passed NULL for
old_target but it always had a non-NULL old_oid so that is perfectly
fine.

> +	update_flags = default_flags;
> +	free(refname);
> +	free(old_target);
> +	strbuf_release(&err);
> +}

> diff --git a/refs.c b/refs.c
> index 060a31616d..0e1013b5ab 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
>  
>  	for (i = 0; i < transaction->nr; i++) {
>  		free(transaction->updates[i]->msg);
> +		free((void *)transaction->updates[i]->old_target);
> +		free((void *)transaction->updates[i]->new_target);
>  		free(transaction->updates[i]);
>  	}
>  	free(transaction->updates);
> @@ -1247,9 +1249,13 @@ struct ref_update *ref_transaction_add_update(
>  
>  	update->flags = flags;
>  
> -	if (flags & REF_HAVE_NEW)
> +	if (new_target)
> +		update->new_target = xstrdup(new_target);
> +	if (old_target)
> +		update->old_target = xstrdup(old_target);

Presumably "update" structure, when freshly initialized, has NULL in
both of these _target members?  Otherwise ref_transaction_free()
would get in trouble, so double checking.

> +	if (new_oid && flags & REF_HAVE_NEW)
>  		oidcpy(&update->new_oid, new_oid);
> -	if (flags & REF_HAVE_OLD)
> +	if (old_oid && flags & REF_HAVE_OLD)
>  		oidcpy(&update->old_oid, old_oid);

Since we can ask to work on a symbolic ref, new_oid / old_oid can be
NULL when REF_HAVE_NEW / REF_HAVE_OLD bit is on for _target members.

Makes me wonder if the code becomes easier to follow if the flag
bits are split into four (_NEW -> _NEW_OID + _NEW_TARGET), but let's
not worry about that for now.

> @@ -1286,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>  	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
>  
>  	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
> +	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);

> @@ -1325,14 +1332,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
>  int ref_transaction_verify(struct ref_transaction *transaction,
>  			   const char *refname,
>  			   const struct object_id *old_oid,
> +			   const char *old_target,
>  			   unsigned int flags,
>  			   struct strbuf *err)
>  {
> -	if (!old_oid)
> -		BUG("verify called with old_oid set to NULL");
> +	if (!old_target && !old_oid)
> +		BUG("verify called with old_oid and old_target set to NULL");

Is it normal if you get _both_ set, or is it equally a BUG()?
The parse_*_verify() codepaths we saw earlier both made sure
only one of the two is non-NULL, and it is unclear what should
happen if both are non-NULL.

> +	if (old_target && !(flags & REF_NO_DEREF))
> +		BUG("verify cannot operate on symrefs with deref mode");
>  	return ref_transaction_update(transaction, refname,
>  				      NULL, old_oid,
> -				      NULL, NULL,
> +				      NULL, old_target,
>  				      flags, NULL, err);
>  }

So this queues an ref_update object whose .new_oid and .new_target
are NULL, and .old_oid and .old_target are what the caller gave us
to check.  The NULLs in .new* members hopefully do not mean "delete
this thing" ;-)

> @@ -2349,6 +2359,12 @@ static int run_transaction_hook(struct ref_transaction *transaction,
>  	for (i = 0; i < transaction->nr; i++) {
>  		struct ref_update *update = transaction->updates[i];
>  
> +		/*
> +		 * Skip reference transaction for symbolic refs.
> +		 */
> +		if (update->new_target || update->old_target)
> +			continue;

Is that a final design, or will the hooks have a chance to interfere?

> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 2420dac2aa..53197fa3af 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2425,6 +2425,37 @@ static const char *original_update_refname(struct ref_update *update)
>  	return update->refname;
>  }
>  
> +/*
> + * Check whether the REF_HAVE_OLD and old_target values stored in
> + * update are consistent with ref, which is the symbolic reference's
> + * current value. If everything is OK, return 0; otherwise, write an
> + * error message to err and return -1.
> + */
> +static int check_old_target(struct ref_update *update, char *ref,
> +			    struct strbuf *err)
> +{
> +	if (!(update->flags & REF_HAVE_OLD) ||
> +	    !strcmp(update->old_target, ref))
> +		return 0;

Earlier on the assignment side for "update" structure we saw above,
the guard was (old_target && flags & REF_HAVE_OLD), but here we
assume old_target is valid, which feels a bit asymmetric.

Yes, I can see that the caller does not call us when !old_target,
but still...  Perhaps

	if ((update->flags & REF_HAVE_OLD) && !update->old_target)
		BUG(...);

or something?  Or alternatively, perhaps !!update->old_target should
be the only thing we should check and ignore REF_HAVE_OLD bit?  I am
not sure, but it smells like that the non-NULL-ness of old_target is
the only thing that matters (if it is not NULL, very early in the
control flow somebody would have set REF_HAVE_OLD bit to flags, no?).

It brings me back to my earlier question.  Does REF_HAVE_OLD bit
serve a useful purpose in this code?

> +	if (!strcmp(update->old_target, ""))
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "reference already exists",
> +			    original_update_refname(update));
> +	else if (!strcmp(ref, ""))
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "reference is missing but expected %s",
> +			    original_update_refname(update),
> +			    update->old_target);

So... for old_target and ref, an empty string is a special value?
How?  Shouldn't that be documented in the comment before the
function?

> +	else
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "is at %s but expected %s",
> +			    original_update_refname(update),
> +			    ref, update->old_target);
> +
> +	return -1;
> +}
> +
>  /*
>   * Check whether the REF_HAVE_OLD and old_oid values stored in update
>   * are consistent with oid, which is the reference's current value. If
> @@ -2528,6 +2559,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>  					ret = TRANSACTION_GENERIC_ERROR;
>  					goto out;
>  				}
> +			}
> +
> +			/*
> +			 * For symref verification, we need to check the reference value
> +			 * rather than the oid. If we're dealing with regular refs or we're
> +			 * verifying a dereferenced symref, we then check the oid.
> +			 */
> +			if (update->old_target) {
> +				if (check_old_target(update, referent.buf, err)) {
> +					ret = TRANSACTION_GENERIC_ERROR;
> +					goto out;
> +				}

We come here only when update->type has REF_ISSYMREF bit on (we
learned that value by calling lock_raw_ref()), and know referent.buf
has the current "target" value.  That is consumed as "ref" parameter
to check_old_target() we just saw.  OK.

> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index 6104471199..a2474245aa 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -938,7 +938,26 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>  		 * individual refs. But the error messages match what the files
>  		 * backend returns, which keeps our tests happy.
>  		 */
> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
> +		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
> +			if (strcmp(referent.buf, u->old_target)) {
> +				if (!strcmp(u->old_target, ""))
> +					strbuf_addf(err, "verifying symref target: '%s': "
> +						    "provided target is empty",
> +						    original_update_refname(u));
> +				else if (!strcmp(referent.buf, ""))
> +					strbuf_addf(err, "verifying symref target: '%s': "
> +						    "reference is missing but expected %s",
> +						    original_update_refname(u),
> +						    u->old_target);
> +				else
> +					strbuf_addf(err, "verifying symref target: '%s': "
> +						    "is at %s but expected %s",
> +						    original_update_refname(u),
> +						    referent.buf, u->old_target);
> +				ret = -1;
> +				goto done;
> +			}

Again, the puzzling "empty string"s are handled here.

Re: [PATCH v4 3/7] update-ref: add support for 'symref-verify' command

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 14893 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> In the previous commits, we added the required base for adding symref
>> commands to the '--stdin' mode provided by 'git-update-ref(1)'. Using
>> them, add a new 'symref-verify' command to verify symrefs.
>>
>> The 'symref-verify' command allows users to verify if a provided <ref>
>> contains the provided <old-target> without changing the <ref>. If
>> <old-target> is not provided, the command will verify that the <ref>
>> doesn't exist. Since we're checking for symbolic refs, this command will
>> only work with the 'no-deref' mode. This is because any dereferenced
>> symbolic ref will point to an object and not a ref and the regular
>> 'verify' command can be used in such situations.
>
> All makes sense, but a naïve reader may find it helpful if you
> explained why having "verify" command is a good idea in the first
> place ("I can just do 'git symoblic-ref' to read the current value,
> and see if it is what I expect").  Presumably the value of "verify"
> is that you can have it in a transaction and fail other operations
> in the same transaction if the symref moved from what you expected
> it to point at?
>

I would say none of the commits drive this point, and I would go ahead
and add something on these lines to each of them. I think it would add
good value to readers.

>> Add and use `ref_update_is_null_new_value`, a helper function which is
>> used to check if there is a new_value in a reference update. The new
>> value could either be a symref target `new_target` or a OID `new_oid`.
>> We also add tests to test the command in both the regular stdin mode and
>> also with the '-z' flag.
>
> This looks out of place, primarily because the helper function is
> *NOT* used in this step.  Without any actual user, and with the name
> that says only what it checks without hinting why a caller may want
> to check the condition it checks, it is hard to guess if it is a
> good idea to have such a helper.
>

I think over the revision, its usage from this commit was removed. It
makes sense to move it to a commit where its used, I'll do that.

> "If a ref_update object specifies no new-oid and no new-target, it
> is not about updating but just validating" is how the callers are
> expected to use it, then instead of is_null_new_value that says
> what it checks, something like is_verify_only that says what the
> caller may want to use it for would be a more friendly name for
> readers and future developers.

This is true for the old-oid and old-target. That is, when they are set
to null, we're validating.

With the new-oid and new-target, if they're null, it usually signifies
deletion. We could rename it to 'is_delete_only', but that would also
need checking the 'REF_HAVE_NEW' flag. So we could ideally change it to

```
int ref_update_is_delete_only(struct ref_update *update) {
	return (update->flags & REF_HAVE_NEW) && !update->new_target &&
is_null_oid(&update->new_oid);
}
```

I'm okay with making this change.

>> @@ -297,11 +320,47 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
>>  		die("verify %s: extra input: %s", refname, next);
>>
>>  	if (ref_transaction_verify(transaction, refname, &old_oid,
>> -				   update_flags, &err))
>> +				   NULL, update_flags, &err))
>>
>>  	update_flags = default_flags;
>>  	free(refname);
>>  	strbuf_release(&err);
>>  }
>
> The only damage by this patch to parse_cmd_verify() is that
> ref_transaction_verify() gained another parameter NULL, but with the
> default "--diff-algorithm=myers" algorithm, it is very hard to see.
>
> The "--patience" algorithm does a much beter job on this hunk.
>
> And the following function is entirely new.
>
>> +static void parse_cmd_symref_verify(struct ref_transaction *transaction,
>> +				    const char *next, const char *end)
>> +{
>> +	struct strbuf err = STRBUF_INIT;
>> +	struct object_id old_oid;
>> +	char *refname, *old_target;
>> +
>> +	if (!(update_flags & REF_NO_DEREF))
>> +		die("symref-verify: cannot operate with deref mode");
>> +
>> +	refname = parse_refname(&next);
>> +	if (!refname)
>> +		die("symref-verify: missing <ref>");
>> +
>> +	/*
>> +	 * old_ref is optional, but we want to differentiate between
>> +	 * a NULL and zero value.
>> +	 */
>> +	old_target = parse_next_refname(&next);
>> +	if (!old_target)
>> +		old_oid = *null_oid();
>
> In many existing code paths, we do not do structure assignment like
> this. Instead we do
>
> 		oidcpy(&old_oid, null_oid());
>
> We can see an existing example in a common context in a hunk for
> refs.c in this patch.
>

Yeah, makes sense to switch this. Will do.

>> +	if (*next != line_termination)
>> +		die("symref-verify %s: extra input: %s", refname, next);
>> +
>> +	if (ref_transaction_verify(transaction, refname,
>> +				   old_target ? NULL : &old_oid,
>> +				   old_target, update_flags, &err))
>> +		die("%s", err.buf);
>
> Are static analyzers smart enough to notice that we will not be
> using old_oid uninitialized here?  Just wondering.

Yup, at least the clang LSP server seems to detect and not bug me about
it.

> Anyway.  This ensures ref_transaction_verify() gets either
> old_target or old_oid, but never both at the same time.  The caller
> to ref_transaction_verify() in the previous function passed NULL for
> old_target but it always had a non-NULL old_oid so that is perfectly
> fine.
>
>> +	update_flags = default_flags;
>> +	free(refname);
>> +	free(old_target);
>> +	strbuf_release(&err);
>> +}
>
>> diff --git a/refs.c b/refs.c
>> index 060a31616d..0e1013b5ab 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
>>
>>  	for (i = 0; i < transaction->nr; i++) {
>>  		free(transaction->updates[i]->msg);
>> +		free((void *)transaction->updates[i]->old_target);
>> +		free((void *)transaction->updates[i]->new_target);
>>  		free(transaction->updates[i]);
>>  	}
>>  	free(transaction->updates);
>> @@ -1247,9 +1249,13 @@ struct ref_update *ref_transaction_add_update(
>>
>>  	update->flags = flags;
>>
>> -	if (flags & REF_HAVE_NEW)
>> +	if (new_target)
>> +		update->new_target = xstrdup(new_target);
>> +	if (old_target)
>> +		update->old_target = xstrdup(old_target);
>
> Presumably "update" structure, when freshly initialized, has NULL in
> both of these _target members?  Otherwise ref_transaction_free()
> would get in trouble, so double checking.
>

This is a good point. My understanding was that FLEX_ALLOC_MEM should
set everything to 0.

>> +	if (new_oid && flags & REF_HAVE_NEW)
>>  		oidcpy(&update->new_oid, new_oid);
>> -	if (flags & REF_HAVE_OLD)
>> +	if (old_oid && flags & REF_HAVE_OLD)
>>  		oidcpy(&update->old_oid, old_oid);
>
> Since we can ask to work on a symbolic ref, new_oid / old_oid can be
> NULL when REF_HAVE_NEW / REF_HAVE_OLD bit is on for _target members.
>
> Makes me wonder if the code becomes easier to follow if the flag
> bits are split into four (_NEW -> _NEW_OID + _NEW_TARGET), but let's
> not worry about that for now.
>

The intersection of this is quite low currently, so I'm not really sure
if there's added benefit. I did start that way before, but perhaps with
the iterations in the last few version, maybe it makes the code simpler.

>> @@ -1286,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
>>  	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
>>
>>  	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
>> +	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
>
>> @@ -1325,14 +1332,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
>>  int ref_transaction_verify(struct ref_transaction *transaction,
>>  			   const char *refname,
>>  			   const struct object_id *old_oid,
>> +			   const char *old_target,
>>  			   unsigned int flags,
>>  			   struct strbuf *err)
>>  {
>> -	if (!old_oid)
>> -		BUG("verify called with old_oid set to NULL");
>> +	if (!old_target && !old_oid)
>> +		BUG("verify called with old_oid and old_target set to NULL");
>
> Is it normal if you get _both_ set, or is it equally a BUG()?
> The parse_*_verify() codepaths we saw earlier both made sure
> only one of the two is non-NULL, and it is unclear what should
> happen if both are non-NULL.
>

It is a bug and this is caught in `ref_transaction_add_update`.
Introduced in the first patch of the series.

>> +	if (old_target && !(flags & REF_NO_DEREF))
>> +		BUG("verify cannot operate on symrefs with deref mode");
>>  	return ref_transaction_update(transaction, refname,
>>  				      NULL, old_oid,
>> -				      NULL, NULL,
>> +				      NULL, old_target,
>>  				      flags, NULL, err);
>>  }
>
> So this queues an ref_update object whose .new_oid and .new_target
> are NULL, and .old_oid and .old_target are what the caller gave us
> to check.  The NULLs in .new* members hopefully do not mean "delete
> this thing" ;-)
>

So the 'new_oid' being set to zero should be the delete this thing
queue.

>> @@ -2349,6 +2359,12 @@ static int run_transaction_hook(struct ref_transaction *transaction,
>>  	for (i = 0; i < transaction->nr; i++) {
>>  		struct ref_update *update = transaction->updates[i];
>>
>> +		/*
>> +		 * Skip reference transaction for symbolic refs.
>> +		 */
>> +		if (update->new_target || update->old_target)
>> +			continue;
>
> Is that a final design, or will the hooks have a chance to interfere?
>

The last patch adds hook support.

>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index 2420dac2aa..53197fa3af 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -2425,6 +2425,37 @@ static const char *original_update_refname(struct ref_update *update)
>>  	return update->refname;
>>  }
>>
>> +/*
>> + * Check whether the REF_HAVE_OLD and old_target values stored in
>> + * update are consistent with ref, which is the symbolic reference's
>> + * current value. If everything is OK, return 0; otherwise, write an
>> + * error message to err and return -1.
>> + */
>> +static int check_old_target(struct ref_update *update, char *ref,
>> +			    struct strbuf *err)
>> +{
>> +	if (!(update->flags & REF_HAVE_OLD) ||
>> +	    !strcmp(update->old_target, ref))
>> +		return 0;
>
> Earlier on the assignment side for "update" structure we saw above,
> the guard was (old_target && flags & REF_HAVE_OLD), but here we
> assume old_target is valid, which feels a bit asymmetric.
>
> Yes, I can see that the caller does not call us when !old_target,
> but still...  Perhaps
>
> 	if ((update->flags & REF_HAVE_OLD) && !update->old_target)
> 		BUG(...);
>

I will add something like this.

> or something?  Or alternatively, perhaps !!update->old_target should
> be the only thing we should check and ignore REF_HAVE_OLD bit?  I am
> not sure, but it smells like that the non-NULL-ness of old_target is
> the only thing that matters (if it is not NULL, very early in the
> control flow somebody would have set REF_HAVE_OLD bit to flags, no?).
>
> It brings me back to my earlier question.  Does REF_HAVE_OLD bit
> serve a useful purpose in this code?
>

I checked and it doesn't, it can be removed from usage in this code.
Will cleanup this part.

>> +	if (!strcmp(update->old_target, ""))
>> +		strbuf_addf(err, "cannot lock ref '%s': "
>> +			    "reference already exists",
>> +			    original_update_refname(update));
>> +	else if (!strcmp(ref, ""))
>> +		strbuf_addf(err, "cannot lock ref '%s': "
>> +			    "reference is missing but expected %s",
>> +			    original_update_refname(update),
>> +			    update->old_target);
>
> So... for old_target and ref, an empty string is a special value?
> How?  Shouldn't that be documented in the comment before the
> function?
>
>> +	else
>> +		strbuf_addf(err, "cannot lock ref '%s': "
>> +			    "is at %s but expected %s",
>> +			    original_update_refname(update),
>> +			    ref, update->old_target);
>> +
>> +	return -1;
>> +}
>> +
>>  /*
>>   * Check whether the REF_HAVE_OLD and old_oid values stored in update
>>   * are consistent with oid, which is the reference's current value. If
>> @@ -2528,6 +2559,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>>  					ret = TRANSACTION_GENERIC_ERROR;
>>  					goto out;
>>  				}
>> +			}
>> +
>> +			/*
>> +			 * For symref verification, we need to check the reference value
>> +			 * rather than the oid. If we're dealing with regular refs or we're
>> +			 * verifying a dereferenced symref, we then check the oid.
>> +			 */
>> +			if (update->old_target) {
>> +				if (check_old_target(update, referent.buf, err)) {
>> +					ret = TRANSACTION_GENERIC_ERROR;
>> +					goto out;
>> +				}
>
> We come here only when update->type has REF_ISSYMREF bit on (we
> learned that value by calling lock_raw_ref()), and know referent.buf
> has the current "target" value.  That is consumed as "ref" parameter
> to check_old_target() we just saw.  OK.
>
>> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
>> index 6104471199..a2474245aa 100644
>> --- a/refs/reftable-backend.c
>> +++ b/refs/reftable-backend.c
>> @@ -938,7 +938,26 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>>  		 * individual refs. But the error messages match what the files
>>  		 * backend returns, which keeps our tests happy.
>>  		 */
>> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>> +		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
>> +			if (strcmp(referent.buf, u->old_target)) {
>> +				if (!strcmp(u->old_target, ""))
>> +					strbuf_addf(err, "verifying symref target: '%s': "
>> +						    "provided target is empty",
>> +						    original_update_refname(u));
>> +				else if (!strcmp(referent.buf, ""))
>> +					strbuf_addf(err, "verifying symref target: '%s': "
>> +						    "reference is missing but expected %s",
>> +						    original_update_refname(u),
>> +						    u->old_target);
>> +				else
>> +					strbuf_addf(err, "verifying symref target: '%s': "
>> +						    "is at %s but expected %s",
>> +						    original_update_refname(u),
>> +						    referent.buf, u->old_target);
>> +				ret = -1;
>> +				goto done;
>> +			}
>
> Again, the puzzling "empty string"s are handled here.

For here and above, this too is dead code and no longer needed,
old_target being empty string is left over code from before we decided
to use zero_oid for deleting. I'll remove it. Thanks.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v4 4/7] update-ref: add support for 'symref-delete' command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Add a new command 'symref-delete' to allow deletions of symbolic refs in
a transaction via the '--stdin' mode of the 'git-update-ref' command.
The 'symref-delete' command can, when given an <old-target>, delete the
provided <ref> only when it points to <old-target>. This will only work
when used with the 'no-deref' mode as it doesn't make sense to deref a
symref during deletion.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt | 11 ++++--
 builtin/fetch.c                  |  2 +-
 builtin/receive-pack.c           |  3 +-
 builtin/update-ref.c             | 33 ++++++++++++++++-
 refs.c                           | 12 ++++---
 refs.h                           |  4 ++-
 refs/files-backend.c             |  2 +-
 refs/reftable-backend.c          |  2 +-
 t/t1400-update-ref.sh            | 61 +++++++++++++++++++++++++++++++-
 9 files changed, 117 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 9fe78b3501..2924b9437e 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-delete SP <ref> [SP <old-target>] LF
 	symref-verify SP <ref> [SP <old-target>] LF
 	option SP <opt> LF
 	start LF
@@ -87,6 +88,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-delete SP <ref> [NUL <old-target>] NUL
 	symref-verify SP <ref> [NUL <old-target>] NUL
 	option SP <opt> NUL
 	start NUL
@@ -112,13 +114,18 @@ create::
 	exist.  The given <new-oid> may not be zero.
 
 delete::
-	Delete <ref> after verifying it exists with <old-oid>, if
-	given.  If given, <old-oid> may not be zero.
+	Delete <ref> after verifying it exists with <old-oid>, if given.
+	If given, <old-oid> may not be zero.  If instead, ref:<old-target>
+	is provided, verify that the symbolic ref <ref> targets
+	<old-target> before deleting it.
 
 verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-delete::
+	Delete <ref> after verifying it exists with <old-target>, if given.
+
 symref-verify::
 	Verify symbolic <ref> against <old-target> but do not change it.
 	If <old-target> is missing, the ref must not exist.  Can only be
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 66840b7c5b..d02592efca 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1383,7 +1383,7 @@ static int prune_refs(struct display_state *display_state,
 		if (transaction) {
 			for (ref = stale_refs; ref; ref = ref->next) {
 				result = ref_transaction_delete(transaction, ref->name, NULL, 0,
-								"fetch: prune", &err);
+								NULL, "fetch: prune", &err);
 				if (result)
 					goto cleanup;
 			}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b150ef39a8..9a4667d57d 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1576,7 +1576,8 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_delete(transaction,
 					   namespaced_name,
 					   old_oid,
-					   0, "push", &err)) {
+					   0, NULL,
+					   "push", &err)) {
 			rp_error("%s", err.buf);
 			ret = "failed to delete";
 		} else {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 419b28169b..8fef3aed0a 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -293,7 +293,7 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 
 	if (ref_transaction_delete(transaction, refname,
 				   have_old ? &old_oid : NULL,
-				   update_flags, msg, &err))
+				   update_flags, NULL, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
@@ -301,6 +301,36 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+
+static void parse_cmd_symref_delete(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *old_target;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-delete: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-delete: missing <ref>");
+
+	old_target = parse_next_refname(&next);
+
+	if (*next != line_termination)
+		die("symref-delete %s: extra input: %s", refname, next);
+
+	if (ref_transaction_delete(transaction, refname, NULL,
+				   update_flags, old_target, msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(old_target);
+	strbuf_release(&err);
+}
+
+
 static void parse_cmd_verify(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -443,6 +473,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
 	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
 	{ "start",         parse_cmd_start,         0, UPDATE_REFS_STARTED },
diff --git a/refs.c b/refs.c
index 0e1013b5ab..6b7c46bfd8 100644
--- a/refs.c
+++ b/refs.c
@@ -979,7 +979,7 @@ int refs_delete_ref(struct ref_store *refs, const char *msg,
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, refname, old_oid,
-				   flags, msg, &err) ||
+				   flags, NULL, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ref_transaction_free(transaction);
@@ -1318,14 +1318,18 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_target,
+			   const char *msg,
 			   struct strbuf *err)
 {
 	if (old_oid && is_null_oid(old_oid))
 		BUG("delete called with old_oid set to zeros");
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("delete cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      NULL, NULL, flags,
+				      NULL, old_target, flags,
 				      msg, err);
 }
 
@@ -2752,7 +2756,7 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg,
 
 	for_each_string_list_item(item, refnames) {
 		ret = ref_transaction_delete(transaction, item->string,
-					     NULL, flags, msg, &err);
+					     NULL, flags, NULL, msg, &err);
 		if (ret) {
 			warning(_("could not delete reference %s: %s"),
 				item->string, err.buf);
diff --git a/refs.h b/refs.h
index 27b9aeaf54..4be4930f04 100644
--- a/refs.h
+++ b/refs.h
@@ -766,7 +766,9 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_target,
+			   const char *msg,
 			   struct strbuf *err);
 
 /*
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 53197fa3af..fc5037fe5a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2516,7 +2516,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && ref_update_is_null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index a2474245aa..2b2cbca8c0 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1120,7 +1120,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 34b29eeac8..8efddac013 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1689,7 +1689,7 @@ do
 		test_cmp before after
 	'
 
-	test_expect_success "stdin ${type} symref-verify no value is treated as zero value" '
+	test_expect_success "stdin ${type} symref-verify fails with no value" '
 		git symbolic-ref refs/heads/symref >expect &&
 		create_stdin_buf ${type} "symref-verify refs/heads/symref" "" &&
 		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
@@ -1728,6 +1728,65 @@ do
 		test_cmp expect actual
 	'
 
+	test_expect_success "stdin ${type} symref-delete fails without --no-deref" '
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: symref-delete: cannot operate with deref mode" err
+	'
+
+	test_expect_success "stdin ${type} symref-delete fails with no ref" '
+		create_stdin_buf ${type} "symref-delete " &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		grep "fatal: symref-delete: missing <ref>" err
+	'
+
+	test_expect_success "stdin ${type} symref-delete fails with too many arguments" '
+		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" "$a" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: $a" err
+		else
+			grep "fatal: symref-delete refs/heads/symref: extra input:  $a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} symref-delete fails with wrong old value" '
+		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$m" &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test_have_prereq REFTABLE
+		then
+			grep "fatal: verifying symref target: ${SQ}refs/heads/symref${SQ}: is at $a but expected refs/heads/main" err
+		else
+			grep "fatal: cannot lock ref ${SQ}refs/heads/symref${SQ}" err
+		fi &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-delete works with right old value" '
+		create_stdin_buf ${type} "symref-delete refs/heads/symref" "$a" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		test_must_fail git rev-parse --verify -q refs/heads/symref
+	'
+
+	test_expect_success "stdin ${type} symref-delete works with empty old value" '
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "symref-delete refs/heads/symref" "" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		test_must_fail git rev-parse --verify -q $b
+	'
+
+	test_expect_success "stdin ${type} symref-delete succeeds for dangling reference" '
+		test_must_fail git symbolic-ref refs/heads/nonexistent &&
+		git symbolic-ref refs/heads/symref2 refs/heads/nonexistent &&
+		create_stdin_buf ${type} "symref-delete refs/heads/symref2" "refs/heads/nonexistent" &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		test_must_fail git symbolic-ref -d refs/heads/symref2
+	'
+
 done
 
 test_done
-- 
2.43.GIT

[PATCH v4 5/7] update-ref: add support for 'symref-create' command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Add 'symref-create' command to the '--stdin' mode 'git-update-ref' to
allow creation of symbolic refs in a transaction. The 'symref-create'
command takes in a <new-target>, which the created <ref> will point to.

Also, support the 'core.prefersymlinkrefs' config, wherein if the config
is set and the filesystem supports symlinks, we create the symbolic ref
as a symlink. We fallback to creating a regular symref if creating the
symlink is unsuccessful.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt | 10 ++++-
 builtin/clone.c                  |  2 +-
 builtin/update-ref.c             | 35 ++++++++++++++++-
 refs.c                           |  9 +++--
 refs.h                           |  1 +
 refs/files-backend.c             | 42 +++++++++++++++++++++
 refs/reftable-backend.c          | 23 +++++++++--
 t/t0600-reffiles-backend.sh      | 32 ++++++++++++++++
 t/t1400-update-ref.sh            | 65 ++++++++++++++++++++++++++++++++
 9 files changed, 210 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 2924b9437e..7a33f70767 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-create SP <ref> SP <new-target> LF
 	symref-delete SP <ref> [SP <old-target>] LF
 	symref-verify SP <ref> [SP <old-target>] LF
 	option SP <opt> LF
@@ -88,6 +89,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-create SP <ref> NUL <new-target> NUL
 	symref-delete SP <ref> [NUL <old-target>] NUL
 	symref-verify SP <ref> [NUL <old-target>] NUL
 	option SP <opt> NUL
@@ -111,7 +113,9 @@ update::
 
 create::
 	Create <ref> with <new-oid> after verifying it does not
-	exist.  The given <new-oid> may not be zero.
+	exist.  The given <new-oid> may not be zero.  If instead
+	ref:<new-target> is provided, a symbolic ref is created
+	which targets <new-target>.
 
 delete::
 	Delete <ref> after verifying it exists with <old-oid>, if given.
@@ -123,6 +127,10 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-create::
+	Create symbolic ref <ref> with <new-target> after verifying
+	it does not exist.  Can only be used in `no-deref` mode.
+
 symref-delete::
 	Delete <ref> after verifying it exists with <old-target>, if given.
 
diff --git a/builtin/clone.c b/builtin/clone.c
index 74ec14542e..c0eed8e795 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -547,7 +547,7 @@ static void write_remote_refs(const struct ref *local_refs)
 		if (!r->peer_ref)
 			continue;
 		if (ref_transaction_create(t, r->peer_ref->name, &r->old_oid,
-					   0, NULL, &err))
+					   NULL, 0, NULL, &err))
 			die("%s", err.buf);
 	}
 
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 8fef3aed0a..ae68ffde5e 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -257,7 +257,7 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	if (*next != line_termination)
 		die("create %s: extra input: %s", refname, next);
 
-	if (ref_transaction_create(transaction, refname, &new_oid,
+	if (ref_transaction_create(transaction, refname, &new_oid, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
@@ -267,6 +267,38 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+
+static void parse_cmd_symref_create(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *new_target;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-create: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-create: missing <ref>");
+
+	new_target = parse_next_refname(&next);
+	if (!new_target)
+		die("symref-create %s: missing <new-target>", refname);
+
+	if (*next != line_termination)
+		die("symref-create %s: extra input: %s", refname, next);
+
+	if (ref_transaction_create(transaction, refname, NULL, new_target,
+				   update_flags | create_reflog_flag,
+				   msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(new_target);
+	strbuf_release(&err);
+}
+
 static void parse_cmd_delete(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -473,6 +505,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
 	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
 	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
diff --git a/refs.c b/refs.c
index 6b7c46bfd8..42cb4126a7 100644
--- a/refs.c
+++ b/refs.c
@@ -1303,15 +1303,18 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
-	if (!new_oid || is_null_oid(new_oid)) {
-		strbuf_addf(err, "'%s' has a null OID", refname);
+	if ((!new_oid || is_null_oid(new_oid)) && !new_target) {
+		strbuf_addf(err, "'%s' has a null OID or no new target", refname);
 		return 1;
 	}
+	if (new_target && !(flags & REF_NO_DEREF))
+		BUG("create cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), NULL, NULL, flags,
+				      null_oid(), new_target, NULL, flags,
 				      msg, err);
 }
 
diff --git a/refs.h b/refs.h
index 4be4930f04..bde8606213 100644
--- a/refs.h
+++ b/refs.h
@@ -752,6 +752,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index fc5037fe5a..f5e271a442 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2610,6 +2610,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
+	if (update->new_target) {
+		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		update->flags |= REF_NEEDS_COMMIT;
+	}
+
+
 	if ((update->flags & REF_HAVE_NEW) &&
 	    !(update->flags & REF_DELETING) &&
 	    !(update->flags & REF_LOG_ONLY)) {
@@ -2905,6 +2926,18 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
+			if (update->new_target) {
+				/*
+				 * We want to get the resolved OID for the target, to ensure
+				 * that the correct value is added to the reflog.
+				 */
+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
+					/* for dangling symrefs we gracefully set the oid to zero */
+					update->new_oid = *null_oid();
+				}
+			}
+
 			if (files_log_ref_write(refs,
 						lock->ref_name,
 						&lock->old_oid,
@@ -2922,6 +2955,15 @@ static int files_transaction_finish(struct ref_store *ref_store,
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next updated. If not, we try and create a regular symref.
+		 */
+		if (update->new_target && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->new_target))
+				continue;
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 2b2cbca8c0..e203c697f2 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -856,7 +856,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if (u->flags & REF_HAVE_NEW && !ref_update_is_null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -1062,7 +1062,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && ref_update_is_null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1104,6 +1104,12 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
 
+			if (u->new_target)
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
+							     RESOLVE_REF_READING, &u->new_oid, NULL))
+					/* for dangling symrefs we gracefully set the oid to zero */
+					u->new_oid = *null_oid();
+
 			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
 			log = &logs[logs_nr++];
 			memset(log, 0, sizeof(*log));
@@ -1120,7 +1126,18 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
+		if (u->flags & REF_HAVE_NEW && u->new_target) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_target,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh
index 64214340e7..c5061c26cf 100755
--- a/t/t0600-reffiles-backend.sh
+++ b/t/t0600-reffiles-backend.sh
@@ -472,4 +472,36 @@ test_expect_success POSIXPERM 'git reflog expire honors core.sharedRepository' '
 	esac
 '
 
+test_expect_success SYMLINKS 'symref transaction supports symlinks' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs true &&
+	cat >stdin <<-EOF &&
+	start
+	symref-create TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_symlink .git/TESTSYMREFONE &&
+	test "$(test_readlink .git/TESTSYMREFONE)" = refs/heads/new
+'
+
+test_expect_success 'symref transaction supports false symlink config' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs false &&
+	cat >stdin <<-EOF &&
+	start
+	symref-create TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_file .git/TESTSYMREFONE &&
+	git symbolic-ref TESTSYMREFONE >actual &&
+	echo refs/heads/new >expect &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 8efddac013..452fc1da50 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1787,6 +1787,71 @@ do
 		test_must_fail git symbolic-ref -d refs/heads/symref2
 	'
 
+	test_expect_success "stdin ${type} symref-create fails without --no-deref" '
+		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: symref-create: cannot operate with deref mode" err
+	'
+
+	test_expect_success "stdin ${type} symref-create fails with too many arguments" '
+		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" "$a" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: $a" err
+		else
+			grep "fatal: symref-create refs/heads/symref: extra input:  $a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} symref-create fails with no target" '
+		create_stdin_buf ${type} "symref-create refs/heads/symref" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
+	'
+
+	test_expect_success "stdin ${type} symref-create fails with empty target" '
+		create_stdin_buf ${type} "symref-create refs/heads/symref" "" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin
+	'
+
+	test_expect_success "stdin ${type} symref-create works" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} create dangling symref ref works" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "symref-create refs/heads/symref" "refs/heads/unkown" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo refs/heads/unkown >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-create does not create reflogs by default" '
+		test_when_finished "git symbolic-ref -d refs/symref" &&
+		create_stdin_buf ${type} "symref-create refs/symref" "$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual &&
+		test_must_fail git reflog exists refs/symref
+	'
+
+	test_expect_success "stdin ${type} symref-create reflogs with --create-reflog" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "symref-create refs/heads/symref" "$a" >stdin &&
+		git update-ref --create-reflog --stdin ${type} --no-deref <stdin &&
+		git symbolic-ref refs/heads/symref >expect &&
+		echo $a >actual &&
+		test_cmp expect actual &&
+		git reflog exists refs/heads/symref
+	'
+
 done
 
 test_done
-- 
2.43.GIT

[PATCH v4 6/7] update-ref: add support for 'symref-update' command

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

Add 'symref-update' command to the '--stdin' mode of 'git-update-ref' to
allow updates of symbolic refs. The 'symref-update' command takes in a
<new-target>, which the <ref> will be updated to. If the <ref> doesn't
exist it will be created.

It also optionally takes either an `ref <old-target>` or `oid
<old-oid>`. If the <old-target> is provided, it checks to see if the
<ref> targets the <old-target> before the update. If <old-oid> is provided
it checks <ref> to ensure that it is a regular ref and <old-oid> is the
OID before the update. This by extension also means that this when a
zero <old-oid> is provided, it ensures that the ref didn't exist before.

This command will also support deref mode, to ensure that we can update
dereferenced regular refs to symrefs.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/git-update-ref.txt |   6 +
 builtin/update-ref.c             |  91 +++++++++++++-
 refs/files-backend.c             |  14 +--
 refs/reftable-backend.c          |   3 +-
 t/t1400-update-ref.sh            | 196 +++++++++++++++++++++++++++++++
 5 files changed, 299 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 7a33f70767..0cd4c37820 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-update SP <ref> SP <new-target> [SP (ref SP <old-target> | oid SP <old-oid>)] LF
 	symref-create SP <ref> SP <new-target> LF
 	symref-delete SP <ref> [SP <old-target>] LF
 	symref-verify SP <ref> [SP <old-target>] LF
@@ -89,6 +90,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-update SP <ref> NUL <new-target> [NUL (ref NUL <old-target> | oid NUL <old-oid>)] NUL
 	symref-create SP <ref> NUL <new-target> NUL
 	symref-delete SP <ref> [NUL <old-target>] NUL
 	symref-verify SP <ref> [NUL <old-target>] NUL
@@ -123,6 +125,10 @@ delete::
 	is provided, verify that the symbolic ref <ref> targets
 	<old-target> before deleting it.
 
+symref-update::
+	Set <ref> to <new-target> after verifying <old-target> or <old-oid>,
+	if given.  Can be used to delete or create symrefs too.
+
 verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index ae68ffde5e..5001d80ae4 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -98,6 +98,41 @@ static char *parse_next_refname(const char **next)
 	return parse_refname(next);
 }
 
+/*
+ * Wrapper around parse_arg which skips the next delimiter.
+ */
+static char *parse_next_arg(const char **next)
+{
+	struct strbuf arg = STRBUF_INIT;
+
+	if (line_termination) {
+		/* Without -z, consume SP and use next argument */
+		if (!**next || **next == line_termination)
+			return NULL;
+		if (**next != ' ')
+			die("expected SP but got: %s", *next);
+	} else {
+		/* With -z, read the next NUL-terminated line */
+		if (**next)
+			return NULL;
+	}
+	/* Skip the delimiter */
+	(*next)++;
+
+	if (line_termination) {
+		/* Without -z, use the next argument */
+		*next = parse_arg(*next, &arg);
+	} else {
+		/* With -z, use everything up to the next NUL */
+		strbuf_addstr(&arg, *next);
+		*next += arg.len;
+	}
+
+	if (arg.len)
+		return strbuf_detach(&arg, NULL);
+	return NULL;
+}
+
 
 /*
  * The value being parsed is <old-oid> (as opposed to <new-oid>; the
@@ -225,8 +260,8 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	if (*next != line_termination)
 		die("update %s: extra input: %s", refname, next);
 
-	if (ref_transaction_update(transaction, refname,
-				   &new_oid, have_old ? &old_oid : NULL,
+	if (ref_transaction_update(transaction, refname, &new_oid,
+				   have_old ? &old_oid : NULL,
 				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
@@ -237,6 +272,57 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+static void parse_cmd_symref_update(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	char *refname, *new_target, *old_arg;
+	char *old_target = NULL;
+	struct strbuf err = STRBUF_INIT;
+	struct object_id old_oid;
+	int have_old = 0;
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-update: missing <ref>");
+
+	new_target = parse_next_refname(&next);
+	if (!new_target)
+		die("symref-update %s: missing <new-target>", refname);
+
+	old_arg = parse_next_arg(&next);
+	if (old_arg) {
+		old_target = parse_next_refname(&next);
+		if (!old_target)
+			die("symref-update %s: expected old value", refname);
+
+		if (!strcmp(old_arg, "oid") &&
+		    !repo_get_oid(the_repository, old_target, &old_oid)) {
+			old_target = NULL;
+			have_old = 1;
+		} else if (strcmp(old_arg, "ref"))
+			die("symref-update %s: invalid arg '%s' for old value", refname, old_arg);
+	}
+
+
+	if (*next != line_termination)
+		die("symref-update %s: extra input: %s", refname, next);
+
+
+	if (ref_transaction_update(transaction, refname, NULL,
+				   have_old ? &old_oid : NULL,
+				   new_target, old_target,
+				   update_flags |= create_reflog_flag,
+				   msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(old_arg);
+	free(old_target);
+	free(new_target);
+	strbuf_release(&err);
+}
+
 static void parse_cmd_create(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -505,6 +591,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-update", parse_cmd_symref_update, 4, UPDATE_REFS_OPEN },
 	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
 	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f5e271a442..59d1ab3eeb 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2386,7 +2386,8 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target, update->old_target,
+			update->msg);
 
 	new_update->parent_update = update;
 
@@ -2610,7 +2611,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if (update->new_target) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
 		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
@@ -2628,12 +2629,9 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		 * phase of the transaction only needs to commit the lock.
 		 */
 		update->flags |= REF_NEEDS_COMMIT;
-	}
-
-
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index e203c697f2..a00f55802a 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -908,7 +908,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+						&u->new_oid, &u->old_oid, u->new_target,
+						u->old_target, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 452fc1da50..3cfef9fbe3 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1360,6 +1360,7 @@ test_expect_success 'fails with duplicate HEAD update' '
 '
 
 test_expect_success 'fails with duplicate ref update via symref' '
+	test_when_finished "git symbolic-ref -d refs/heads/symref2" &&
 	git branch target2 $A &&
 	git symbolic-ref refs/heads/symref2 refs/heads/target2 &&
 	cat >stdin <<-EOF &&
@@ -1852,6 +1853,201 @@ do
 		git reflog exists refs/heads/symref
 	'
 
+	test_expect_success "stdin ${type} symref-update fails with too many arguments" '
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "ref" "$a" "$a" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test "$type" = "-z"
+		then
+			grep "fatal: unknown command: $a" err
+		else
+			grep "fatal: symref-update refs/heads/symref: extra input:  $a" err
+		fi
+	'
+
+	test_expect_success "stdin ${type} symref-update fails with wrong old value argument" '
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "foo" "$a" "$a" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		grep "fatal: symref-update refs/heads/symref: invalid arg ${SQ}foo${SQ} for old value" err
+	'
+
+	test_expect_success "stdin ${type} symref-update creates with zero old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "oid" "$Z" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $a >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update creates with no old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $a >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update creates dangling" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		test_must_fail git rev-parse refs/heads/nonexistent &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "refs/heads/nonexistent" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo refs/heads/nonexistent >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update fails with wrong old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$m" "ref" "$b" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin 2>err &&
+		if test_have_prereq REFTABLE
+		then
+			grep "fatal: verifying symref target: ${SQ}refs/heads/symref${SQ}: is at $a but expected $b" err
+		else
+			grep "fatal: cannot lock ref ${SQ}refs/heads/symref${SQ}: is at $a but expected $b" err
+		fi &&
+		test_must_fail git rev-parse --verify -q $c
+	'
+
+	test_expect_success "stdin ${type} symref-update updates dangling ref" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		test_must_fail git rev-parse refs/heads/nonexistent &&
+		git symbolic-ref refs/heads/symref refs/heads/nonexistent &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $a >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update updates dangling ref with old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		test_must_fail git rev-parse refs/heads/nonexistent &&
+		git symbolic-ref refs/heads/symref refs/heads/nonexistent &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "ref" "refs/heads/nonexistent" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $a >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update fails update dangling ref with wrong old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		test_must_fail git rev-parse refs/heads/nonexistent &&
+		git symbolic-ref refs/heads/symref refs/heads/nonexistent &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "ref" "refs/heads/wrongref" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+		echo refs/heads/nonexistent >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update works with right old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$m" "ref" "$a" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $m >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update works with no old value" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$m" >stdin &&
+		git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $m >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update fails with empty old ref-target" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref $a &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$m" "ref" "" >stdin &&
+		test_must_fail git update-ref --stdin ${type} --no-deref <stdin &&
+		echo $a >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update creates (with deref)" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" >stdin &&
+		git update-ref --stdin ${type} <stdin &&
+		echo $a >expect &&
+		git symbolic-ref --no-recurse refs/heads/symref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+		grep "$Z $(git rev-parse $a)" actual
+	'
+
+	test_expect_success "stdin ${type} symref-update regular ref to symref with correct old-oid" '
+		test_when_finished "git symbolic-ref -d --no-recurse refs/heads/regularref" &&
+		git update-ref --no-deref refs/heads/regularref $a &&
+		create_stdin_buf ${type} "symref-update refs/heads/regularref" "$a" "oid" "$(git rev-parse $a)" >stdin &&
+		git update-ref --stdin ${type} <stdin &&
+		echo $a >expect &&
+		git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+		grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+	'
+
+	test_expect_success "stdin ${type} symref-update regular ref to symref fails with wrong old-oid" '
+		test_when_finished "git update-ref -d refs/heads/regularref" &&
+		git update-ref --no-deref refs/heads/regularref $a &&
+		create_stdin_buf ${type} "symref-update refs/heads/regularref" "$a" "oid" "$(git rev-parse refs/heads/target2)" >stdin &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		echo $(git rev-parse $a) >expect &&
+		git rev-parse refs/heads/regularref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update existing symref with zero old-oid" '
+		test_when_finished "git symbolic-ref -d --no-recurse refs/heads/symref" &&
+		git symbolic-ref refs/heads/symref refs/heads/target2 &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" "oid" "$Z" >stdin &&
+		test_must_fail git update-ref --stdin ${type} <stdin 2>err &&
+		grep "fatal: cannot lock ref ${SQ}refs/heads/symref${SQ}: reference already exists" err &&
+		echo refs/heads/target2 >expect &&
+		git symbolic-ref refs/heads/symref >actual &&
+		test_cmp expect actual
+	'
+
+	test_expect_success "stdin ${type} symref-update regular ref to symref (with deref)" '
+		test_when_finished "git symbolic-ref -d refs/heads/symref" &&
+		test_when_finished "git update-ref -d --no-deref refs/heads/symref2" &&
+		git update-ref refs/heads/symref2 $a &&
+		git symbolic-ref --no-recurse refs/heads/symref refs/heads/symref2 &&
+		create_stdin_buf ${type} "symref-update refs/heads/symref" "$a" >stdin &&
+		git update-ref ${type} --stdin <stdin &&
+		echo $a >expect &&
+		git symbolic-ref --no-recurse refs/heads/symref2 >actual &&
+		test_cmp expect actual &&
+		echo refs/heads/symref2 >expect &&
+		git symbolic-ref --no-recurse refs/heads/symref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/symref >actual &&
+		grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+	'
+
+	test_expect_success "stdin ${type} symref-update regular ref to symref" '
+		test_when_finished "git symbolic-ref -d --no-recurse refs/heads/regularref" &&
+		git update-ref --no-deref refs/heads/regularref $a &&
+		create_stdin_buf ${type} "symref-update refs/heads/regularref" "$a" >stdin &&
+		git update-ref ${type} --stdin <stdin &&
+		echo $a >expect &&
+		git symbolic-ref --no-recurse refs/heads/regularref >actual &&
+		test_cmp expect actual &&
+		test-tool ref-store main for-each-reflog-ent refs/heads/regularref >actual &&
+		grep "$(git rev-parse $a) $(git rev-parse $a)" actual
+	'
+
 done
 
 test_done
-- 
2.43.GIT

[PATCH v4 7/7] ref: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In the previous commits, we added symref support for
various commands in `git-update-ref`. While it allowed us to now
manipulate symbolic refs via `git-update-ref`, it didn't activate the
'reference-transaction' hook.

Let's activate the hook for symbolic reference updates too. There is no
new format described for this and we stick to the existing format of:
    <old-value> SP <new-value> SP <ref-name> LF
but now, <old-value> and <new-value> could also denote references
instead of objects, where the format is similar to that in
'git-update-ref', i.e. 'ref:<ref-target>'.

While this seems to be backward incompatible, it is okay, since the only
way the `reference-transaction` hook has refs in its output is when
`git-update-ref` is used to manipulate symrefs. Also the documentation
for reference-transaction hook always stated that support for symbolic
references may be added in the future.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/githooks.txt       | 14 +++++++----
 refs.c                           | 21 ++++++++--------
 t/t1416-ref-transaction-hooks.sh | 41 ++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 15 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ee9b92c90d..0bf8ca87a6 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also cover symbolic references.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects, denoted via the
+`ref:<ref-target>` format.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index 42cb4126a7..9a510744a7 100644
--- a/refs.c
+++ b/refs.c
@@ -2365,18 +2365,19 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		strbuf_reset(&buf);
 
-		/*
-		 * Skip reference transaction for symbolic refs.
-		 */
-		if (update->new_target || update->old_target)
-			continue;
+		if (update->flags & REF_HAVE_OLD && update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
 
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+		if (update->flags & REF_HAVE_NEW && update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..0a7e86062e 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -134,4 +134,45 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+# This test doesn't add a check for symref 'delete' since there is a
+# variation between the ref backends WRT 'delete'. In the files backend,
+# 'delete' also triggers an additional transaction update on the
+# packed-refs backend, which constitutes additional reflog entries.
+test_expect_success 'hook gets all queued symref updates' '
+	test_when_finished "rm actual" &&
+
+	git update-ref refs/heads/branch $POST_OID &&
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+	git symbolic-ref refs/heads/symrefu refs/heads/main &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	cat >expect <<-EOF &&
+		prepared
+		ref:refs/heads/main $ZERO_OID refs/heads/symref
+		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
+		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
+		committed
+		ref:refs/heads/main $ZERO_OID refs/heads/symref
+		$ZERO_OID ref:refs/heads/main refs/heads/symrefc
+		ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
+	EOF
+
+	git update-ref --no-deref --stdin <<-EOF &&
+		start
+		symref-verify refs/heads/symref refs/heads/main
+		symref-create refs/heads/symrefc refs/heads/main
+		symref-update refs/heads/symrefu refs/heads/branch ref refs/heads/main
+		prepare
+		commit
+	EOF
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

Re: [PATCH v4 0/7] add symref-* commands to 'git-update-ref --stdin'

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 2148 bytes --]

Karthik Nayak <karthik.188@gmail.com> writes:

> From: Karthik Nayak <karthik.188@gmail.com>
>
> The 'git-update-ref(1)' command allows transactional reference updates.
> But currently only supports regular reference updates. Meaning, if one
> wants to update HEAD (symbolic ref) in a transaction, there is no tool
> to do so.
>
> One option to obtain transactional updates for the HEAD ref is to
> manually create the HEAD.lock file and commit. This is intrusive, where
> the user needs to mimic internal git behavior. Also, this only works
> when using the files backend.
>
> At GitLab, we've been using the manual process till date, to allow users
> to set and change their default branch. But with the introduction of
> reftables as a reference backend, this becomes a necessity to be solved
> within git.
>
> This patch series goes about introducing a set of commands
> symref-{create,verify,delete,update} to work with symrefs complimenting
> the existing commands for the regular refs in the '--stdin' mode of
> 'git-update-ref'.
>
> The 'symref-verify' command can be used to verify if a symref exists and
> its existing value.
>
> The 'symref-create' command can be used to create a new symref.
>
> The 'symref-delete' command can be used to delete an existing symref while
> optionally checking its existing value.
>
> The 'symref-update' command can be used to update a symref, create a symref,
> delete a symref or even convert an existing regular ref to a symref. Wherein
> like the regular 'update' command, the zero OID can be used to create/delete
> a symref.
>
> While this series adds the commands and the required ground work, it only
> is accessile within the '--stdin' mode of 'git-update-ref'. However, it makes
> it easy to extend it further to the command line too, which will be present
> in a follow up series.
>

Also, I'll be pushing a new version with reduced scope soon. The idea
being that we can review/merge the part which adds the symref support to
the refs library first and then we can have a followup for exposing the
same to the users via git-update-ref. This should make it also easier
for reviewers.

[snip]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v5 0/7] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The patch series takes over from the existing patch series, wherein we
introduced symref-* commands to git-update-ref. Since there was a ton of
discussions on the UX of the patch series and its application, I thought it
would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then
we switch refs_create_symref() to start using transactions for symref updates.
This allows us to deprecate the create_symref code in the ref_storage_be
interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the
previous series. While pertaining the user facing components into another set
of patches wherein deeper discussion on the UX can be held without worrying
about the internal implementation. Also by using this new functionality in a
pre-existing command, we can leverage the existing tests to catch any
inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for
dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref
update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com/
V2: https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com/
V3: https://lore.kernel.org/git/20240423212818.574123-1-knayak@gitlab.com/
V4: https://lore.kernel.org/r/20240426152449.228860-1-knayak@gitlab.com

Changes over v4 are:
- Dropped the patches for adding support in git-update-ref.
- Added changes to use transaction in 'refs_create_symref()' and
  deprecate 'create_symref'.
- Cleaned up the commit messages, documentation and comments.
- Added better handling, like if 'old_target' is set, the value of the
  reference is checked, irrelevant of its type.

Range diff:

1:  4a56e3ede4 ! 1:  a354190905 refs: accept symref values in `ref_transaction[_add]_update`
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    refs: accept symref values in `ref_transaction[_add]_update`
    +    refs: accept symref values in `ref_transaction_update()`
     
    -    The `ref_transaction[_add]_update` functions obtain ref information and
    -    flags to create a `ref_update` and add it to the transaction at hand.
    +    The function `ref_transaction_update()` obtains ref information and
    +    flags to create a `ref_update` and add them to the transaction at hand.
     
         To extend symref support in transactions, we need to also accept the
    -    old and new ref targets and process it. In this commit, let's add the
    -    required parameters to the function and modify all call sites.
    +    old and new ref targets and process it. This commit adds the required
    +    parameters to the function and modifies all call sites.
     
         The two parameters added are `new_target` and `old_target`. The
         `new_target` is used to denote what the reference should point to when
    -    the transaction is applied.
    +    the transaction is applied. Some functions allow this parameter to be
    +    NULL, meaning that the reference is not changed.
     
         The `old_target` denotes the value the reference must have before the
         update. Some functions allow this parameter to be NULL, meaning that the
         old value of the reference is not checked.
     
    -    The handling logic of these parameters will be added in consequent
    -    commits as we add symref commands to the '--stdin' mode of
    -    'git-update-ref'.
    +    We also update the internal function `ref_transaction_add_update()`
    +    similarly to take the two new parameters.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      		BUG("update called for transaction that is not open");
      
     +	if (old_oid && !is_null_oid(old_oid) && old_target)
    -+		BUG("Only one of old_oid and old_target should be non NULL");
    ++		BUG("only one of old_oid and old_target should be non NULL");
     +	if (new_oid && !is_null_oid(new_oid) && new_target)
    -+		BUG("Only one of new_oid and new_target should be non NULL");
    ++		BUG("only one of new_oid and new_target should be non NULL");
     +
      	FLEX_ALLOC_STR(update, refname, refname);
      	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
    @@ refs.h: struct ref_transaction *ref_transaction_begin(struct strbuf *err);
       *         transaction.
       *
     + *     new_target -- the target reference that the reference will be
    -+ *         update to point to. This takes precedence over new_oid when
    -+ *         set. If the reference is a regular reference, it will be
    -+ *         converted to a symbolic reference.
    ++ *         updated to point to. If the reference is a regular reference,
    ++ *         it will be converted to a symbolic reference. Cannot be set
    ++ *         together with `new_oid`. A copy of this value is made in the
    ++ *         transaction.
     + *
     + *     old_target -- the reference that the reference must be pointing to.
    -+ *         Will only be taken into account when the reference is a symbolic
    -+ *         reference.
    ++ *         Canont be set together with `old_oid`. A copy of this value is
    ++ *         made in the transaction.
     + *
       *     flags -- flags affecting the update, passed to
       *         update_ref_lock(). Possible flags: REF_NO_DEREF,
    @@ refs/refs-internal.h: struct ref_update {
     +	/*
     +	 * If set, point the reference to this value. This can also be
     +	 * used to convert regular references to become symbolic refs.
    ++	 * Cannot be set together with `new_oid`.
     +	 */
     +	const char *new_target;
     +
     +	/*
    -+	 * If set and the reference is a symbolic ref, check that the
    -+	 * reference previously pointed to this value.
    ++	 * If set, check that the reference previously pointed to this
    ++	 * value. Cannot be set together with `old_oid`.
     +	 */
     +	const char *old_target;
     +
2:  496bf14f28 ! 2:  7dff21dbef files-backend: extract out `create_symref_lock`
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    files-backend: extract out `create_symref_lock`
    +    files-backend: extract out `create_symref_lock()`
     
    -    The function `create_symref_locked` creates a symref by creating a
    +    The function `create_symref_locked()` creates a symref by creating a
         '<symref>.lock' file and then committing the symref lock, which creates
         the final symref.
     
    -    Split this into two individual functions `create_and_commit_symref` and
    -    `create_symref_locked`. This way we can create the symref lock and
    -    commit it at different times. This will be used to provide symref
    -    support in `git-update-ref(1)`.
    +    Extract the early half of `create_symref_locked()` into a new helper
    +    function `create_symref_lock()`. Because the name of the new function is
    +    too similar to the original, rename the original to
    +    `create_and_commit_symref()` to avoid confusion.
    +
    +    The new function `create_symref_locked()` can be used to create the
    +    symref lock in a separate step from that of committing it. This allows
    +    to add transactional support for symrefs, where the lock would be
    +    created in the preparation step and the lock would be committed in the
    +    finish step.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     +		return error("unable to fdopen %s: %s",
     +			     get_lock_file_path(&lock->lk), strerror(errno));
     +
    -+	/* no error check; commit_ref will check ferror */
    -+	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
    ++	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
    ++		return error("unable to fprintf %s: %s",
    ++			     get_lock_file_path(&lock->lk), strerror(errno));
     +	return 0;
     +}
     +
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     -	if (commit_ref(lock) < 0)
     -		return error("unable to write symref for %s: %s", refname,
     -			     strerror(errno));
    - 	return 0;
    +-	return 0;
    ++	return ret;
      }
      
    + static int files_create_symref(struct ref_store *ref_store,
     @@ refs/files-backend.c: static int files_create_symref(struct ref_store *ref_store,
      		return -1;
      	}
3:  6337859cbb < -:  ---------- update-ref: add support for 'symref-verify' command
4:  e611cb5a8c < -:  ---------- update-ref: add support for 'symref-delete' command
5:  37f8be2f3f < -:  ---------- update-ref: add support for 'symref-create' command
6:  b385f4d0d7 < -:  ---------- update-ref: add support for 'symref-update' command
7:  ef335e47d1 < -:  ---------- ref: support symrefs in 'reference-transaction' hook
-:  ---------- > 3:  901a586683 refs: support symrefs in 'reference-transaction' hook
-:  ---------- > 4:  6c97f6a660 refs: add support for transactional symref updates
-:  ---------- > 5:  5b55406430 refs: use transaction in `refs_create_symref()`
-:  ---------- > 6:  9e25816e68 refs: rename `refs_create_symref()` to `refs_update_symref()`
-:  ---------- > 7:  3836e25932 refs: remove `create_symref` and associated dead code


Karthik Nayak (7):
  refs: accept symref values in `ref_transaction_update()`
  files-backend: extract out `create_symref_lock()`
  refs: support symrefs in 'reference-transaction' hook
  refs: add support for transactional symref updates
  refs: use transaction in `refs_create_symref()`
  refs: rename `refs_create_symref()` to `refs_update_symref()`
  refs: remove `create_symref` and associated dead code

 Documentation/githooks.txt       |  14 ++-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           |  87 +++++++++++----
 refs.h                           |  20 +++-
 refs/debug.c                     |  13 ---
 refs/files-backend.c             | 185 +++++++++++++++++++------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  26 ++++-
 refs/reftable-backend.c          | 159 +++++++++-----------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t0610-reftable-basics.sh       |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 22 files changed, 321 insertions(+), 240 deletions(-)

-- 
2.43.GIT

[PATCH v5 1/7] refs: accept symref values in `ref_transaction_update()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `ref_transaction_update()` obtains ref information and
flags to create a `ref_update` and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the
old and new ref targets and process it. This commit adds the required
parameters to the function and modifies all call sites.

The two parameters added are `new_target` and `old_target`. The
`new_target` is used to denote what the reference should point to when
the transaction is applied. Some functions allow this parameter to be
NULL, meaning that the reference is not changed.

The `old_target` denotes the value the reference must have before the
update. Some functions allow this parameter to be NULL, meaning that the
old value of the reference is not checked.

We also update the internal function `ref_transaction_add_update()`
similarly to take the two new parameters.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)

diff --git a/branch.c b/branch.c
index e4a738fc7b..48af4c3ceb 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index dc5a9d32dd..297dfb91a1 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5857d860db..66840b7c5b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e8d7df14b6..b150ef39a8 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..7690687b0e 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 9a33cb50b4..40a65fdebc 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index e46afbc46d..21fdbf6ac8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
diff --git a/refs.c b/refs.c
index 55d2e0b2cb..47bc9dd103 100644
--- a/refs.c
+++ b/refs.c
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && !is_null_oid(old_oid) && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && !is_null_oid(new_oid) && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index d278775e08..c7851bf587 100644
--- a/refs.h
+++ b/refs.h
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index a098d14ea0..e4d0aa3d41 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..108f4ec419 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 1cda48c504..6104471199 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 88de4dc20f..61e007d85f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index c0fd632d92..1b3df43906 100644
--- a/walker.c
+++ b/walker.c
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
-- 
2.43.GIT

[PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked()` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Extract the early half of `create_symref_locked()` into a new helper
function `create_symref_lock()`. Because the name of the new function is
too similar to the original, rename the original to
`create_and_commit_symref()` to avoid confusion.

The new function `create_symref_locked()` can be used to create the
symref lock in a separate step from that of committing it. This allows
to add transactional support for symrefs, where the lock would be
created in the preparation step and the lock would be committed in the
finish step.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 43 +++++++++++++++++++++++++++++--------------
 1 file changed, 29 insertions(+), 14 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index e4d0aa3d41..878601ced0 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1920,27 +1920,41 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target)
 {
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
+		return error("unable to fprintf %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1960,7 +1974,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> +	if (!fdopen_lock_file(&lock->lk, "w"))
> +		return error("unable to fdopen %s: %s",
> +			     get_lock_file_path(&lock->lk), strerror(errno));
> +
> +	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
> +		return error("unable to fprintf %s: %s",
> +			     get_lock_file_path(&lock->lk), strerror(errno));

error() is end-user facing, so "fprintf" is probably a bit too
precise?  "fprintf" -> "write to"

Also we may want to make them (not just this new message but other
error() messages in related code paths) localizable but that is
probably beyond the scope of this topic.

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 980 bytes --]

On Wed, May 01, 2024 at 03:06:19PM -0700, Junio C Hamano wrote:
> Karthik Nayak <karthik.188@gmail.com> writes:
> 
> > +	if (!fdopen_lock_file(&lock->lk, "w"))
> > +		return error("unable to fdopen %s: %s",
> > +			     get_lock_file_path(&lock->lk), strerror(errno));
> > +
> > +	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
> > +		return error("unable to fprintf %s: %s",
> > +			     get_lock_file_path(&lock->lk), strerror(errno));
> 
> error() is end-user facing, so "fprintf" is probably a bit too
> precise?  "fprintf" -> "write to"
> 
> Also we may want to make them (not just this new message but other
> error() messages in related code paths) localizable but that is
> probably beyond the scope of this topic.

It only occurred to me now, but shouldn't we also support passing in a
`struct strbuf *err` here? The transactional code doesn't want us to
print error messages to `stderr`, but always supplies a buffer.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1093 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Wed, May 01, 2024 at 03:06:19PM -0700, Junio C Hamano wrote:
>> Karthik Nayak <karthik.188@gmail.com> writes:
>>
>> > +	if (!fdopen_lock_file(&lock->lk, "w"))
>> > +		return error("unable to fdopen %s: %s",
>> > +			     get_lock_file_path(&lock->lk), strerror(errno));
>> > +
>> > +	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
>> > +		return error("unable to fprintf %s: %s",
>> > +			     get_lock_file_path(&lock->lk), strerror(errno));
>>
>> error() is end-user facing, so "fprintf" is probably a bit too
>> precise?  "fprintf" -> "write to"
>>
>> Also we may want to make them (not just this new message but other
>> error() messages in related code paths) localizable but that is
>> probably beyond the scope of this topic.
>
> It only occurred to me now, but shouldn't we also support passing in a
> `struct strbuf *err` here? The transactional code doesn't want us to
> print error messages to `stderr`, but always supplies a buffer.
>
> Patrick

Yes I think that would fit better with the existing transaction code.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> On Wed, May 01, 2024 at 03:06:19PM -0700, Junio C Hamano wrote:
>> Karthik Nayak <karthik.188@gmail.com> writes:
>> 
>> > +	if (!fdopen_lock_file(&lock->lk, "w"))
>> > +		return error("unable to fdopen %s: %s",
>> > +			     get_lock_file_path(&lock->lk), strerror(errno));
>> > +
>> > +	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
>> > +		return error("unable to fprintf %s: %s",
>> > +			     get_lock_file_path(&lock->lk), strerror(errno));
>> 
>> error() is end-user facing, so "fprintf" is probably a bit too
>> precise?  "fprintf" -> "write to"
>> 
>> Also we may want to make them (not just this new message but other
>> error() messages in related code paths) localizable but that is
>> probably beyond the scope of this topic.
>
> It only occurred to me now, but shouldn't we also support passing in a
> `struct strbuf *err` here? The transactional code doesn't want us to
> print error messages to `stderr`, but always supplies a buffer.

Sounds sensible.  Thanks.

[PATCH v5 3/7] refs: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In a previous commit, we added the `old_target` and
`new_target` fields to the `reference_transaction_update()`. In
following commits we'll also add the code to handle symref's in the
reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying
the current format:
    <old-oid> SP <new-oid> SP <ref-name> LF
to be be:
    <old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same.
But when either 'old-value' or 'new-value' is a symref, we print the ref
as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction'
hook's documentation always stated that support for symbolic references
may be added in the future.

We do not add any tests in this commit since there is no git command
which activates this flow, in an upcoming commit, we'll start using
transaction based symref updates as the default, we'll add tests there
for the hook too.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 16 ++++++++++++----
 2 files changed, 21 insertions(+), 9 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ee9b92c90d..06e997131b 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index 47bc9dd103..5dfe93060a 100644
--- a/refs.c
+++ b/refs.c
@@ -2350,10 +2350,18 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (update->flags & REF_HAVE_OLD && update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (update->flags & REF_HAVE_NEW && update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
-- 
2.43.GIT

Re: [PATCH v5 3/7] refs: support symrefs in 'reference-transaction' hook

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> +		if (update->flags & REF_HAVE_OLD && update->old_target)

Although the precedence rule does not require it,

		if ((update->flags & REF_HAVE_OLD) && update_old_target)

is probably easier to read.

> +			strbuf_addf(&buf, "ref:%s ", update->old_target);
> +		else
> +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));

So the promise this code assumes is that .old_target member is
non-NULL if and only if the ref originally is a symbolic ref?

And if the "we do not care what the original value is, whether it is
a normal ref or a symbolic one" case, .old_oid would be all '\0' and
REF_HAVE_OLD bit is not set?

If we can write it like so:

	if (!(update->flags & REF_HAVE_OLD))
		strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
	else if (update->old_target)
		strbuf_addf(&buf, "ref:%s ", update->old_target);
	else
		strbuf_addf(&buf, "ref:%s ", oid_to_hex(update->old_oid));

it may make the intent of the code a lot more clear.  If we are
operating in "!HAVE_OLD" mode, we show 0{40}.  Otherwise, old_target
is non-NULL when the thing is symbolic, and if old_target is NULL,
it is not symbolic and has its own value.

The same comment applies to the other side.

> +		if (update->flags & REF_HAVE_NEW && update->new_target)
> +			strbuf_addf(&buf, "ref:%s ", update->new_target);
> +		else
> +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));


> +		strbuf_addf(&buf, "%s\n", update->refname);
>  
>  		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
>  			if (errno != EPIPE) {

Re: [PATCH v5 3/7] refs: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1958 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> +		if (update->flags & REF_HAVE_OLD && update->old_target)
>
> Although the precedence rule does not require it,
>
> 		if ((update->flags & REF_HAVE_OLD) && update_old_target)
>
> is probably easier to read.
>

Will add.

>> +			strbuf_addf(&buf, "ref:%s ", update->old_target);
>> +		else
>> +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
>
> So the promise this code assumes is that .old_target member is
> non-NULL if and only if the ref originally is a symbolic ref?
>

Yes, for old_target this is correct. new_target could be set for a ref
to convert it to a symbolic ref.

> And if the "we do not care what the original value is, whether it is
> a normal ref or a symbolic one" case, .old_oid would be all '\0' and
> REF_HAVE_OLD bit is not set?
>

Yup that's accurate.

> If we can write it like so:
>
> 	if (!(update->flags & REF_HAVE_OLD))
> 		strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
> 	else if (update->old_target)
> 		strbuf_addf(&buf, "ref:%s ", update->old_target);
> 	else
> 		strbuf_addf(&buf, "ref:%s ", oid_to_hex(update->old_oid));
>
> it may make the intent of the code a lot more clear.  If we are
> operating in "!HAVE_OLD" mode, we show 0{40}.  Otherwise, old_target
> is non-NULL when the thing is symbolic, and if old_target is NULL,
> it is not symbolic and has its own value.
>
> The same comment applies to the other side.
>

I see how it makes it clearer, but I think the intent with the existing
code was clear too. I'll add this change to my local for the next
version.

>> +		if (update->flags & REF_HAVE_NEW && update->new_target)
>> +			strbuf_addf(&buf, "ref:%s ", update->new_target);
>> +		else
>> +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
>
>
>> +		strbuf_addf(&buf, "%s\n", update->refname);
>>
>>  		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
>>  			if (errno != EPIPE) {

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v5 4/7] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The reference backends currently support transactional reference
updates. While this is exposed to users via 'git-update-ref' and its
'--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's add
support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use `ref_update_is_null_new_value()`, a helper function
which is used to check if there is a new_value in a reference update.
The new value could either be a symref target `new_target` or a OID
`new_oid`.

With this, now transactional updates (verify, create, delete, update)
can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in
'git-update-ref' in the future.

We do not add reflog for dangling symref updates, because currently
'git-symbolic-ref' doesn't add reflog for dangling symref updates and it
would be best to keep this behavior consistent as we would move it to
start using transaction based updates in the following commit.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                  |  17 +++++-
 refs/files-backend.c    | 112 +++++++++++++++++++++++++++++++++++-----
 refs/refs-internal.h    |   7 +++
 refs/reftable-backend.c |  71 +++++++++++++++++++------
 4 files changed, 176 insertions(+), 31 deletions(-)

diff --git a/refs.c b/refs.c
index 5dfe93060a..a4dca08244 100644
--- a/refs.c
+++ b/refs.c
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,10 +1249,15 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);
+	if (new_oid && flags & REF_HAVE_NEW)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if (old_oid && flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1286,6 +1293,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2810,3 +2818,8 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int ref_update_is_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 878601ced0..85c4af7e89 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2426,6 +2426,36 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with current_target, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+static int check_old_target(struct ref_update *update,
+			    const char *current_target,
+			    struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(update->old_target, current_target))
+		return 0;
+
+	if (!strcmp(current_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    current_target, update->old_target);
+
+	return -1;
+}
+
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2486,7 +2516,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && ref_update_is_null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
@@ -2529,7 +2559,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
-			} else if (check_old_oid(update, &lock->old_oid, err)) {
+			}
+
+			if (update->old_target) {
+				if (check_old_target(update, referent.buf, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}
+			} else if  (check_old_oid(update, &lock->old_oid, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto out;
 			}
@@ -2550,7 +2587,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 	} else {
 		struct ref_update *parent_update;
 
-		if (check_old_oid(update, &lock->old_oid, err)) {
+		/*
+		 * Even if the ref is a regular ref, if `old_target` is set, we
+		 * check the referent value. Ideally `old_target` should only
+		 * be set for symrefs, but we're strict about its usage.
+		 */
+		if (update->old_target) {
+			if (check_old_target(update, referent.buf, err)) {
+				ret = TRANSACTION_GENERIC_ERROR;
+				goto out;
+			}
+		} else if  (check_old_oid(update, &lock->old_oid, err)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
 		}
@@ -2568,9 +2615,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
+		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		update->flags |= REF_NEEDS_COMMIT;
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
@@ -2863,12 +2928,26 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
-			if (files_log_ref_write(refs,
-						lock->ref_name,
-						&lock->old_oid,
-						&update->new_oid,
-						update->msg, update->flags,
-						err)) {
+			int create_reflog = 1;
+
+			if (update->new_target) {
+				/*
+				 * We want to get the resolved OID for the target, to ensure
+				 * that the correct value is added to the reflog.
+				 */
+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
+					/* for dangling symrefs we skip creating a reflog entry. */
+					create_reflog = 0;
+				}
+			}
+
+			if (create_reflog && files_log_ref_write(refs,
+								 lock->ref_name,
+								 &lock->old_oid,
+								 &update->new_oid,
+								 update->msg, update->flags,
+								 err)) {
 				char *old_msg = strbuf_detach(err, NULL);
 
 				strbuf_addf(err, "cannot update the ref '%s': %s",
@@ -2880,6 +2959,15 @@ static int files_transaction_finish(struct ref_store *ref_store,
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next update. If not, we try and create a regular symref.
+		 */
+		if (update->new_target && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->new_target))
+				continue;
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 108f4ec419..9578665243 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -749,4 +749,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_is_null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 6104471199..5e8a696d40 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -856,7 +856,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if (u->flags & REF_HAVE_NEW && !ref_update_is_null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -907,8 +907,10 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 * intertwined with the locking in files-backend.c.
 				 */
 				new_update = ref_transaction_add_update(
-						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+					transaction, referent.buf, new_flags,
+					&u->new_oid, &u->old_oid, u->new_target,
+					u->old_target, u->msg);
+
 				new_update->parent_update = u;
 
 				/*
@@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {
+				if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
+				ret = -1;
+				goto done;
+			}
+		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 					    "reference already exists"),
@@ -1043,7 +1060,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && ref_update_is_null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1084,24 +1101,44 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			   (u->flags & REF_FORCE_CREATE_REFLOG ||
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
+			int create_reflog = 1;
 
-			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
-			log = &logs[logs_nr++];
-			memset(log, 0, sizeof(*log));
-
-			fill_reftable_log_record(log);
-			log->update_index = ts;
-			log->refname = xstrdup(u->refname);
-			memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
-			memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
-			log->value.update.message =
-				xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			if (u->new_target)
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
+							     RESOLVE_REF_READING, &u->new_oid, NULL))
+					/* for dangling symrefs we skip creating reflog */
+					create_reflog = 0;
+
+			if (create_reflog) {
+				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
+				log = &logs[logs_nr++];
+				memset(log, 0, sizeof(*log));
+
+				fill_reftable_log_record(log);
+				log->update_index = ts;
+				log->refname = xstrdup(u->refname);
+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
+				log->value.update.message =
+					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			}
 		}
 
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && u->new_target) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_target,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
-- 
2.43.GIT

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> From: Karthik Nayak <karthik.188@gmail.com>
>
> The reference backends currently support transactional reference
> updates. While this is exposed to users via 'git-update-ref' and its
> '--stdin' mode, it is also used internally within various commands.
>
> However, we never supported transactional updates of symrefs. Let's add
> support for symrefs in both the 'files' and the 'reftable' backend.
>
> Here, we add and use `ref_update_is_null_new_value()`, a helper function
> which is used to check if there is a new_value in a reference update.

I know you want to express a condition where you answer yes to "Is
the new value specified in this ref update NULL?", but "is" at that
position in the name somehow sounds awkward.  Any of

	ref_update_has_null_new_value
        ref_update_with_no_new_value
        ref_update_without_new_value

might be nicer to ears.

> We do not add reflog for dangling symref updates, because currently
> 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it
> would be best to keep this behavior consistent as we would move it to
> start using transaction based updates in the following commit.

If we are not changing the behaviour, does it deserve a four-line
paragraph?  It is not like we describe every no changes (i.e. "we
could break the behaviour by introducing this and that bugs, but we
did not" is not something we usually say in proposed log messages).

At most, if you want to highlight that behaviour, I would expect a
brief mention like:

    Note that a dangling symref update does not record a new reflog
    entry, which is unchanged before and after this commit.

As a reflog entry records name of the object that is pointed by the
ref (either directly or indirectly) before and after an operation,
an operation that involve a dangling reflog that does not point at
any object cannot be expressed in a reflog, no?  It is way too late
to change this, but it would have been interesting if the design of
reflog had a room to log the change of symbolic ref target as well
as object names.  It would have allowed us to say "HEAD at time T
pointed at refs/heads/main (which did not exist)", "HEAD at time T+1
directly pointed at commit X (detached)", "HEAD at time T+2 pointed
at refs/heads/next", etc. and allowed us to much more cleanly
support "previous branch".

> @@ -1247,10 +1249,15 @@ struct ref_update *ref_transaction_add_update(
>  
>  	update->flags = flags;
>  
> -	if (flags & REF_HAVE_NEW)
> +	if (new_target)
> +		update->new_target = xstrdup(new_target);
> +	if (old_target)
> +		update->old_target = xstrdup(old_target);

"Is the assumption that *update is 0-initialized?" was the first
question that came to my mind.

Doing an unconditional

	update->new_target = xstrdup_or_null(new_target);
	update->old_target = xstrdup_or_null(old_target);

would convey the intention much more clearly without having readers
guess the answer to the above question.

> +	if (new_oid && flags & REF_HAVE_NEW)

Even though syntactically not required, 

	if (new_oid && (flags & REF_HAVE_NEW))

or better yet

	if ((flags & REF_HAVE_NEW) && new_oid)

would be easier to see.

>  		oidcpy(&update->new_oid, new_oid);

Again is the expectation that update->new_oid is initialized to
all-0?  I am wondering if we want an else clause, i.e.

	if (!(flags & REF_HAVE_NEW))
		oidcpy(&update->new_oid, null_oid());
	else
		oidcpy(&update->new_oid, new_oid ? new_oid : null_oid());

to clarify the intention of the code, since the way you wrote the
consumer of thes two members and REF_HAVE_NEW bit in the previous
step implied that the new_oid member gets used even when REF_HAVE_*
bit is off, only for its null_oid() value.

I'll stop here for now.

Thanks.

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 5013 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The reference backends currently support transactional reference
>> updates. While this is exposed to users via 'git-update-ref' and its
>> '--stdin' mode, it is also used internally within various commands.
>>
>> However, we never supported transactional updates of symrefs. Let's add
>> support for symrefs in both the 'files' and the 'reftable' backend.
>>
>> Here, we add and use `ref_update_is_null_new_value()`, a helper function
>> which is used to check if there is a new_value in a reference update.
>
> I know you want to express a condition where you answer yes to "Is
> the new value specified in this ref update NULL?", but "is" at that
> position in the name somehow sounds awkward.  Any of
>
> 	ref_update_has_null_new_value
>         ref_update_with_no_new_value
>         ref_update_without_new_value
>
> might be nicer to ears.
>

Yes, thanks with this, I do agree that `ref_update_has_null_new_value`
sounds better.

>> We do not add reflog for dangling symref updates, because currently
>> 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it
>> would be best to keep this behavior consistent as we would move it to
>> start using transaction based updates in the following commit.
>
> If we are not changing the behaviour, does it deserve a four-line
> paragraph?  It is not like we describe every no changes (i.e. "we
> could break the behaviour by introducing this and that bugs, but we
> did not" is not something we usually say in proposed log messages).
>
> At most, if you want to highlight that behaviour, I would expect a
> brief mention like:
>
>     Note that a dangling symref update does not record a new reflog
>     entry, which is unchanged before and after this commit.
>
> As a reflog entry records name of the object that is pointed by the
> ref (either directly or indirectly) before and after an operation,
> an operation that involve a dangling reflog that does not point at
> any object cannot be expressed in a reflog, no?  It is way too late
> to change this, but it would have been interesting if the design of
> reflog had a room to log the change of symbolic ref target as well
> as object names.  It would have allowed us to say "HEAD at time T
> pointed at refs/heads/main (which did not exist)", "HEAD at time T+1
> directly pointed at commit X (detached)", "HEAD at time T+2 pointed
> at refs/heads/next", etc. and allowed us to much more cleanly
> support "previous branch".
>

While I agree that four lines may seem excessive, I think it is indeed
an important point to note. Mostly because this shows that when doing
dangling symref updates, there is no record of this update. The best
situation would be like you mentioned, to record the symref target
changes. But even with the current design, it would have been nice to at
least acknowledge that there was some update done to the symref. By
having zero-oid for the new and old value in the reflog. But seems like
we can't do that either.

>> @@ -1247,10 +1249,15 @@ struct ref_update *ref_transaction_add_update(
>>
>>  	update->flags = flags;
>>
>> -	if (flags & REF_HAVE_NEW)
>> +	if (new_target)
>> +		update->new_target = xstrdup(new_target);
>> +	if (old_target)
>> +		update->old_target = xstrdup(old_target);
>
> "Is the assumption that *update is 0-initialized?" was the first
> question that came to my mind.
>
> Doing an unconditional
>
> 	update->new_target = xstrdup_or_null(new_target);
> 	update->old_target = xstrdup_or_null(old_target);
>
> would convey the intention much more clearly without having readers
> guess the answer to the above question.
>

Right, I didn't catch the nuance last time, thanks for the explanation.

>> +	if (new_oid && flags & REF_HAVE_NEW)
>
> Even though syntactically not required,
>
> 	if (new_oid && (flags & REF_HAVE_NEW))
>
> or better yet
>
> 	if ((flags & REF_HAVE_NEW) && new_oid)
>
> would be easier to see.
>
>>  		oidcpy(&update->new_oid, new_oid);
>
> Again is the expectation that update->new_oid is initialized to
> all-0?  I am wondering if we want an else clause, i.e.
>
> 	if (!(flags & REF_HAVE_NEW))
> 		oidcpy(&update->new_oid, null_oid());
> 	else
> 		oidcpy(&update->new_oid, new_oid ? new_oid : null_oid());
>
> to clarify the intention of the code, since the way you wrote the
> consumer of thes two members and REF_HAVE_NEW bit in the previous
> step implied that the new_oid member gets used even when REF_HAVE_*
> bit is off, only for its null_oid() value.
>

Yes I understand what you're saying, but since we're doing a
`FLEX_ALLOC_MEM` right above this code, I thought the fact that the
'update' struct is 0 initialized is known. With this, and the fact that
here `update->new_oid` is a static variable, while `new_oid` is a
pointer. I think being too verbose is not required.

> I'll stop here for now.
>
> Thanks.
>

Thanks Junio for taking the time to review.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2947 bytes --]

On Thu, May 02, 2024 at 05:50:47AM +0000, Karthik Nayak wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> > Karthik Nayak <karthik.188@gmail.com> writes:
> >> From: Karthik Nayak <karthik.188@gmail.com>
> >> We do not add reflog for dangling symref updates, because currently
> >> 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it
> >> would be best to keep this behavior consistent as we would move it to
> >> start using transaction based updates in the following commit.
> >
> > If we are not changing the behaviour, does it deserve a four-line
> > paragraph?  It is not like we describe every no changes (i.e. "we
> > could break the behaviour by introducing this and that bugs, but we
> > did not" is not something we usually say in proposed log messages).
> >
> > At most, if you want to highlight that behaviour, I would expect a
> > brief mention like:
> >
> >     Note that a dangling symref update does not record a new reflog
> >     entry, which is unchanged before and after this commit.
> >
> > As a reflog entry records name of the object that is pointed by the
> > ref (either directly or indirectly) before and after an operation,
> > an operation that involve a dangling reflog that does not point at
> > any object cannot be expressed in a reflog, no?  It is way too late
> > to change this, but it would have been interesting if the design of
> > reflog had a room to log the change of symbolic ref target as well
> > as object names.  It would have allowed us to say "HEAD at time T
> > pointed at refs/heads/main (which did not exist)", "HEAD at time T+1
> > directly pointed at commit X (detached)", "HEAD at time T+2 pointed
> > at refs/heads/next", etc. and allowed us to much more cleanly
> > support "previous branch".
> >
> 
> While I agree that four lines may seem excessive, I think it is indeed
> an important point to note. Mostly because this shows that when doing
> dangling symref updates, there is no record of this update. The best
> situation would be like you mentioned, to record the symref target
> changes. But even with the current design, it would have been nice to at
> least acknowledge that there was some update done to the symref. By
> having zero-oid for the new and old value in the reflog. But seems like
> we can't do that either.

I wouldn't say we can't do that. We already do log when symrefs become
dangling when updating references via HEAD by logging a zero OID as new
OID. That is, if we have "HEAD -> refs/heads/foo" and you delete the
latter, then we create a new reflog message for "HEAD" with zero OID as
new OID.

I would claim that the current behaviour where we don't create a reflog
entry when updating a ref to become dangling is a mere bug. I think it's
fair to declare this a #leftoverbit and handle it in a follow-up patch
series. But it would be nice to say so in an in-code comment.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 3298 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Thu, May 02, 2024 at 05:50:47AM +0000, Karthik Nayak wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>> > Karthik Nayak <karthik.188@gmail.com> writes:
>> >> From: Karthik Nayak <karthik.188@gmail.com>
>> >> We do not add reflog for dangling symref updates, because currently
>> >> 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it
>> >> would be best to keep this behavior consistent as we would move it to
>> >> start using transaction based updates in the following commit.
>> >
>> > If we are not changing the behaviour, does it deserve a four-line
>> > paragraph?  It is not like we describe every no changes (i.e. "we
>> > could break the behaviour by introducing this and that bugs, but we
>> > did not" is not something we usually say in proposed log messages).
>> >
>> > At most, if you want to highlight that behaviour, I would expect a
>> > brief mention like:
>> >
>> >     Note that a dangling symref update does not record a new reflog
>> >     entry, which is unchanged before and after this commit.
>> >
>> > As a reflog entry records name of the object that is pointed by the
>> > ref (either directly or indirectly) before and after an operation,
>> > an operation that involve a dangling reflog that does not point at
>> > any object cannot be expressed in a reflog, no?  It is way too late
>> > to change this, but it would have been interesting if the design of
>> > reflog had a room to log the change of symbolic ref target as well
>> > as object names.  It would have allowed us to say "HEAD at time T
>> > pointed at refs/heads/main (which did not exist)", "HEAD at time T+1
>> > directly pointed at commit X (detached)", "HEAD at time T+2 pointed
>> > at refs/heads/next", etc. and allowed us to much more cleanly
>> > support "previous branch".
>> >
>>
>> While I agree that four lines may seem excessive, I think it is indeed
>> an important point to note. Mostly because this shows that when doing
>> dangling symref updates, there is no record of this update. The best
>> situation would be like you mentioned, to record the symref target
>> changes. But even with the current design, it would have been nice to at
>> least acknowledge that there was some update done to the symref. By
>> having zero-oid for the new and old value in the reflog. But seems like
>> we can't do that either.
>
> I wouldn't say we can't do that. We already do log when symrefs become
> dangling when updating references via HEAD by logging a zero OID as new
> OID. That is, if we have "HEAD -> refs/heads/foo" and you delete the
> latter, then we create a new reflog message for "HEAD" with zero OID as
> new OID.
>
> I would claim that the current behaviour where we don't create a reflog
> entry when updating a ref to become dangling is a mere bug. I think it's
> fair to declare this a #leftoverbit and handle it in a follow-up patch
> series. But it would be nice to say so in an in-code comment.
>

I think _can't_ wasn't the best terminology. My previous series actually
added a reflog, but I noticed a bunch of tests were failing and I think
it made sense to keep the existing behaviour.

But addressing it as a bug would definitely be a good way to go and fix
this, I'll add a comment in the code for now.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> I wouldn't say we can't do that. We already do log when symrefs become
> dangling when updating references via HEAD by logging a zero OID as new
> OID. That is, if we have "HEAD -> refs/heads/foo" and you delete the
> latter, then we create a new reflog message for "HEAD" with zero OID as
> new OID.
>
> I would claim that the current behaviour where we don't create a reflog
> entry when updating a ref to become dangling is a mere bug. I think it's
> fair to declare this a #leftoverbit and handle it in a follow-up patch
> series. But it would be nice to say so in an in-code comment.

I like that.  Thanks.

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> ... By
> having zero-oid for the new and old value in the reflog. But seems like
> we can't do that either.

Even if you could, the transition from refs/heads/main to refs/heads/next
would not be captured.

In any case, that is not something this step changes.  My assumption
is that for most of developers it would be unexpected for the theme
to "allow symbolic updates to be transactional" to change it, hence
"we do not explain what we did not change".

Thanks.

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> @@ -2863,12 +2928,26 @@ static int files_transaction_finish(struct ref_store *ref_store,
>  
>  		if (update->flags & REF_NEEDS_COMMIT ||
>  		    update->flags & REF_LOG_ONLY) {
> -			if (files_log_ref_write(refs,
> -						lock->ref_name,
> -						&lock->old_oid,
> -						&update->new_oid,
> -						update->msg, update->flags,
> -						err)) {
> +			int create_reflog = 1;
> +
> +			if (update->new_target) {
> +				/*
> +				 * We want to get the resolved OID for the target, to ensure
> +				 * that the correct value is added to the reflog.
> +				 */
> +				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
> +							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
> +					/* for dangling symrefs we skip creating a reflog entry. */
> +					create_reflog = 0;
> +				}
> +			}
> +
> +			if (create_reflog && files_log_ref_write(refs,
> +								 lock->ref_name,
> +								 &lock->old_oid,
> +								 &update->new_oid,
> +								 update->msg, update->flags,
> +								 err)) {
>  				char *old_msg = strbuf_detach(err, NULL);
>  
>  				strbuf_addf(err, "cannot update the ref '%s': %s",

This hunk is overly wide.

You could of course fix it mechanically (e.g., by rewrapping overly
wide comment block, wrapping an expression after &&-), but a code
path that is too deeply indented may be a sign that the function may
want to be split into calls to a smaller helper function for
readability.

[PATCH v5 5/7] refs: use transaction in `refs_create_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function updates a symref to a given new
target. To do this, it uses a ref-backend specific function
`create_symref()`.

In this previous commit, we introduce symref support in transactions.
This means we can now use transactions to perform symref updates and not
have to resort to `create_symref()`. Doing this allows us to remove and
cleanup `create_symref()`, which we will do in the following commit.

Modify the expected error message for a test in
't/t0610-reftable-basics.sh', since the error is now thrown from the
'refs.c'. This is because in transactional updates, F/D conflicts are
caught before we're in the reference backend.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                           | 24 +++++++++++++++++-------
 t/t0610-reftable-basics.sh       |  2 +-
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 3 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/refs.c b/refs.c
index a4dca08244..1b67c87f47 100644
--- a/refs.c
+++ b/refs.c
@@ -2291,14 +2291,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index 178791e086..9e8d22bcbd 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
 	git init repo &&
 	test_commit -C repo A &&
 	cat >expect <<-EOF &&
-	error: unable to write symref for refs/heads: file/directory conflict
+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
 	EOF
 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
 	test_cmp expect err
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..4433ac2177 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+		prepared
+		$ZERO_OID ref:refs/heads/main refs/heads/symref
+		committed
+		$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

Re: [PATCH v5 5/7] refs: use transaction in `refs_create_symref()`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2226 bytes --]

On Wed, May 01, 2024 at 10:22:27PM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The `refs_create_symref()` function updates a symref to a given new
> target. To do this, it uses a ref-backend specific function
> `create_symref()`.
> 
> In this previous commit, we introduce symref support in transactions.
> This means we can now use transactions to perform symref updates and not
> have to resort to `create_symref()`. Doing this allows us to remove and

Nit: "not have to" -> "don't have to"

[snip]
> diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
> index 178791e086..9e8d22bcbd 100755
> --- a/t/t0610-reftable-basics.sh
> +++ b/t/t0610-reftable-basics.sh
> @@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
>  	git init repo &&
>  	test_commit -C repo A &&
>  	cat >expect <<-EOF &&
> -	error: unable to write symref for refs/heads: file/directory conflict
> +	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
>  	EOF
>  	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
>  	test_cmp expect err

Nice. Not only do we have less code to worry about, but the error
message is better, too.

> diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
> index 2092488090..4433ac2177 100755
> --- a/t/t1416-ref-transaction-hooks.sh
> +++ b/t/t1416-ref-transaction-hooks.sh
> @@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
>  	test_cmp expect target-repo.git/actual
>  '
>  
> +test_expect_success 'hook captures git-symbolic-ref updates' '
> +	test_when_finished "rm actual" &&
> +
> +	test_hook reference-transaction <<-\EOF &&
> +		echo "$*" >>actual
> +		while read -r line
> +		do
> +			printf "%s\n" "$line"
> +		done >>actual
> +	EOF
> +
> +	git symbolic-ref refs/heads/symref refs/heads/main &&
> +
> +	cat >expect <<-EOF &&
> +		prepared
> +		$ZERO_OID ref:refs/heads/main refs/heads/symref
> +		committed
> +		$ZERO_OID ref:refs/heads/main refs/heads/symref
> +	EOF

Nit: the contents of the heredoc should be indented one level less.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v5 6/7] refs: rename `refs_create_symref()` to `refs_update_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function is used to update/create a symref.
But it doesn't check the old target of the symref, if existing. It force
updates the symref. In this regard, the name `refs_create_symref()` is a
bit misleading. So let's rename it to `refs_update_symref()`. This is
akin to how 'git-update-ref(1)' also allows us to create apart from
update.

While we're here, rename the arguments in the function to clarify what
they actually signify and reduce confusion.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index dd3e3a7dc0..4491f7a20c 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 7c6c72536b..480202c517 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
diff --git a/refs.c b/refs.c
index 1b67c87f47..1215b0fae2 100644
--- a/refs.c
+++ b/refs.c
@@ -2286,10 +2286,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2297,8 +2295,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2314,7 +2312,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
diff --git a/refs.h b/refs.h
index c7851bf587..71cc1c58e0 100644
--- a/refs.h
+++ b/refs.h
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 82bbf6e2e6..4651e4ced7 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
-- 
2.43.GIT

Re: [PATCH v5 6/7] refs: rename `refs_create_symref()` to `refs_update_symref()`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 998 bytes --]

On Wed, May 01, 2024 at 10:22:28PM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The `refs_create_symref()` function is used to update/create a symref.
> But it doesn't check the old target of the symref, if existing. It force
> updates the symref. In this regard, the name `refs_create_symref()` is a
> bit misleading. So let's rename it to `refs_update_symref()`. This is
> akin to how 'git-update-ref(1)' also allows us to create apart from
> update.

Arguably, as we are already updating all callsites anyway, I don't see a
reason why we shouldn't also update the function to accept the old OID
or old target so that callers can make raceless updates.

The only problem is that we don't have any users yet, and consequently
we have no way to verify that it works as intended. So maybe this is
better left for a future patch series, unless we have places where we
can reasonably update the callers to pass in the old value, as well.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v5 6/7] refs: rename `refs_create_symref()` to `refs_update_symref()`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1193 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> On Wed, May 01, 2024 at 10:22:28PM +0200, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The `refs_create_symref()` function is used to update/create a symref.
>> But it doesn't check the old target of the symref, if existing. It force
>> updates the symref. In this regard, the name `refs_create_symref()` is a
>> bit misleading. So let's rename it to `refs_update_symref()`. This is
>> akin to how 'git-update-ref(1)' also allows us to create apart from
>> update.
>
> Arguably, as we are already updating all callsites anyway, I don't see a
> reason why we shouldn't also update the function to accept the old OID
> or old target so that callers can make raceless updates.
>
> The only problem is that we don't have any users yet, and consequently
> we have no way to verify that it works as intended. So maybe this is
> better left for a future patch series, unless we have places where we
> can reasonably update the callers to pass in the old value, as well.
>
> Patrick

Yeah that did run through my mind too, but without a usecase, its hard
to justify a change and also like you mentioned harder to write tests
for.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v5 7/7] refs: remove `create_symref` and associated dead code

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

In the previous commits, we converted `refs_create_symref()` to utilize
transactions to perform symref updates. Earlier `refs_create_symref()`
used `create_symref()` to do the same.

This means, we can now remove `create_symref()` and any code associated
with it which is no longer used. We remove `create_symref()` code from
all the reference backends and also remove it entirely from the
`ref_storage_be` struct.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/debug.c            | 13 -------
 refs/files-backend.c    | 64 ------------------------------
 refs/packed-backend.c   |  1 -
 refs/refs-internal.h    |  5 ---
 refs/reftable-backend.c | 86 -----------------------------------------
 5 files changed, 169 deletions(-)

diff --git a/refs/debug.c b/refs/debug.c
index c7531b17f0..8be316bb67 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 85c4af7e89..7597760f63 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1903,23 +1903,6 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)
 	return ret;
 }
 
-static void update_symref_reflog(struct files_ref_store *refs,
-				 struct ref_lock *lock, const char *refname,
-				 const char *target, const char *logmsg)
-{
-	struct strbuf err = STRBUF_INIT;
-	struct object_id new_oid;
-
-	if (logmsg &&
-	    refs_resolve_ref_unsafe(&refs->base, target,
-				    RESOLVE_REF_READING, &new_oid, NULL) &&
-	    files_log_ref_write(refs, refname, &lock->old_oid,
-				&new_oid, logmsg, 0, &err)) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-	}
-}
-
 static int create_symref_lock(struct files_ref_store *refs,
 			      struct ref_lock *lock, const char *refname,
 			      const char *target)
@@ -1934,52 +1917,6 @@ static int create_symref_lock(struct files_ref_store *refs,
 	return 0;
 }
 
-static int create_and_commit_symref(struct files_ref_store *refs,
-				    struct ref_lock *lock, const char *refname,
-				    const char *target, const char *logmsg)
-{
-	int ret;
-
-	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-		return 0;
-	}
-
-	ret = create_symref_lock(refs, lock, refname, target);
-	if (!ret) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-
-		if (commit_ref(lock) < 0)
-			return error("unable to write symref for %s: %s", refname,
-				     strerror(errno));
-	}
-
-	return ret;
-}
-
-static int files_create_symref(struct ref_store *ref_store,
-			       const char *refname, const char *target,
-			       const char *logmsg)
-{
-	struct files_ref_store *refs =
-		files_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct strbuf err = STRBUF_INIT;
-	struct ref_lock *lock;
-	int ret;
-
-	lock = lock_ref_oid_basic(refs, refname, &err);
-	if (!lock) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-		return -1;
-	}
-
-	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
-
-	unlock_ref(lock);
-	return ret;
-}
-
 static int files_reflog_exists(struct ref_store *ref_store,
 			       const char *refname)
 {
@@ -3394,7 +3331,6 @@ struct ref_storage_be refs_be_files = {
 	.initial_transaction_commit = files_initial_transaction_commit,
 
 	.pack_refs = files_pack_refs,
-	.create_symref = files_create_symref,
 	.rename_ref = files_rename_ref,
 	.copy_ref = files_copy_ref,
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 4e826c05ff..a937e7dbfc 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 9578665243..fa46548a2f 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 5e8a696d40..d4221898e4 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1269,91 +1269,6 @@ struct write_create_symref_arg {
 	const char *logmsg;
 };
 
-static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
-{
-	struct write_create_symref_arg *create = cb_data;
-	uint64_t ts = reftable_stack_next_update_index(create->stack);
-	struct reftable_ref_record ref = {
-		.refname = (char *)create->refname,
-		.value_type = REFTABLE_REF_SYMREF,
-		.value.symref = (char *)create->target,
-		.update_index = ts,
-	};
-	struct reftable_log_record log = {0};
-	struct object_id new_oid;
-	struct object_id old_oid;
-	int ret;
-
-	reftable_writer_set_limits(writer, ts, ts);
-
-	ret = reftable_writer_add_ref(writer, &ref);
-	if (ret)
-		return ret;
-
-	/*
-	 * Note that it is important to try and resolve the reference before we
-	 * write the log entry. This is because `should_write_log()` will munge
-	 * `core.logAllRefUpdates`, which is undesirable when we create a new
-	 * repository because it would be written into the config. As HEAD will
-	 * not resolve for new repositories this ordering will ensure that this
-	 * never happens.
-	 */
-	if (!create->logmsg ||
-	    !refs_resolve_ref_unsafe(&create->refs->base, create->target,
-				     RESOLVE_REF_READING, &new_oid, NULL) ||
-	    !should_write_log(&create->refs->base, create->refname))
-		return 0;
-
-	fill_reftable_log_record(&log);
-	log.refname = xstrdup(create->refname);
-	log.update_index = ts;
-	log.value.update.message = xstrndup(create->logmsg,
-					    create->refs->write_options.block_size / 2);
-	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
-	if (refs_resolve_ref_unsafe(&create->refs->base, create->refname,
-				    RESOLVE_REF_READING, &old_oid, NULL))
-		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
-
-	ret = reftable_writer_add_log(writer, &log);
-	reftable_log_record_release(&log);
-	return ret;
-}
-
-static int reftable_be_create_symref(struct ref_store *ref_store,
-				     const char *refname,
-				     const char *target,
-				     const char *logmsg)
-{
-	struct reftable_ref_store *refs =
-		reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct reftable_stack *stack = stack_for(refs, refname, &refname);
-	struct write_create_symref_arg arg = {
-		.refs = refs,
-		.stack = stack,
-		.refname = refname,
-		.target = target,
-		.logmsg = logmsg,
-	};
-	int ret;
-
-	ret = refs->err;
-	if (ret < 0)
-		goto done;
-
-	ret = reftable_stack_reload(stack);
-	if (ret)
-		goto done;
-
-	ret = reftable_stack_add(stack, &write_create_symref_table, &arg);
-
-done:
-	assert(ret != REFTABLE_API_ERROR);
-	if (ret)
-		error("unable to write symref for %s: %s", refname,
-		      reftable_error_str(ret));
-	return ret;
-}
-
 struct write_copy_arg {
 	struct reftable_ref_store *refs;
 	struct reftable_stack *stack;
@@ -2261,7 +2176,6 @@ struct ref_storage_be refs_be_reftable = {
 	.initial_transaction_commit = reftable_be_initial_transaction_commit,
 
 	.pack_refs = reftable_be_pack_refs,
-	.create_symref = reftable_be_create_symref,
 	.rename_ref = reftable_be_rename_ref,
 	.copy_ref = reftable_be_copy_ref,
 
-- 
2.43.GIT

Re: [PATCH v5 7/7] refs: remove `create_symref` and associated dead code

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 579 bytes --]

On Wed, May 01, 2024 at 10:22:29PM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> In the previous commits, we converted `refs_create_symref()` to utilize
> transactions to perform symref updates. Earlier `refs_create_symref()`
> used `create_symref()` to do the same.
> 
> This means, we can now remove `create_symref()` and any code associated
> with it which is no longer used. We remove `create_symref()` code from
> all the reference backends and also remove it entirely from the
> `ref_storage_be` struct.

Very nice.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v5 7/7] refs: remove `create_symref` and associated dead code

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> On Wed, May 01, 2024 at 10:22:29PM +0200, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
>> 
>> In the previous commits, we converted `refs_create_symref()` to utilize
>> transactions to perform symref updates. Earlier `refs_create_symref()`
>> used `create_symref()` to do the same.
>> 
>> This means, we can now remove `create_symref()` and any code associated
>> with it which is no longer used. We remove `create_symref()` code from
>> all the reference backends and also remove it entirely from the
>> `ref_storage_be` struct.
>
> Very nice.

Indeed.

The last paragraph reads well without "This means, ", by the way.

Re: [PATCH v5 0/7] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> From: Karthik Nayak <karthik.188@gmail.com>
>
> The patch series takes over from the existing patch series, wherein we
> introduced symref-* commands to git-update-ref. Since there was a ton of
> discussions on the UX of the patch series and its application, I thought it
> would be best to shorten the series and split it into multiple smaller series.
>
> This series adds transactional support for symrefs in the reference db. Then
> we switch refs_create_symref() to start using transactions for symref updates.
> This allows us to deprecate the create_symref code in the ref_storage_be
> interface and remove all associated code which is no longer used.
>
> The split was primarily done so we can merge the non-user facing parts of the
> previous series. While pertaining the user facing components into another set
> of patches wherein deeper discussion on the UX can be held without worrying
> about the internal implementation.

This split probably makes sense in the context of the evolution of
this series.  If this were without any prior discussion, a change to
the internal mechanism, without showing how the end-user facing half
would use it fully, would have been hard to evaluate, but now that
we know where the new mechanism wants to take us, we can fairly
evaluate it alone without the end-user facing part.

I've read only the earlier half of the series but so far the pieces
make sense to me.

Re: [PATCH v5 0/7] refs: add support for transactional symref updates

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1723 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The patch series takes over from the existing patch series, wherein we
>> introduced symref-* commands to git-update-ref. Since there was a ton of
>> discussions on the UX of the patch series and its application, I thought it
>> would be best to shorten the series and split it into multiple smaller series.
>>
>> This series adds transactional support for symrefs in the reference db. Then
>> we switch refs_create_symref() to start using transactions for symref updates.
>> This allows us to deprecate the create_symref code in the ref_storage_be
>> interface and remove all associated code which is no longer used.
>>
>> The split was primarily done so we can merge the non-user facing parts of the
>> previous series. While pertaining the user facing components into another set
>> of patches wherein deeper discussion on the UX can be held without worrying
>> about the internal implementation.
>
> This split probably makes sense in the context of the evolution of
> this series.  If this were without any prior discussion, a change to
> the internal mechanism, without showing how the end-user facing half
> would use it fully, would have been hard to evaluate, but now that
> we know where the new mechanism wants to take us, we can fairly
> evaluate it alone without the end-user facing part.
>
> I've read only the earlier half of the series but so far the pieces
> make sense to me.

Yes I agree, I think it's also nice to see how a bunch of code can be
removed to use this generic functionality.

I'll wait for a day/two for other reviews. Thanks for your review.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

[PATCH v6 0/7] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The patch series takes over from the existing patch series, wherein we
introduced symref-* commands to git-update-ref. Since there was a ton of
discussions on the UX of the patch series and its application, I thought it
would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then
we switch refs_create_symref() to start using transactions for symref updates.
This allows us to deprecate the create_symref code in the ref_storage_be
interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the
previous series. While pertaining the user facing components into another set
of patches wherein deeper discussion on the UX can be held without worrying
about the internal implementation. Also by using this new functionality in a
pre-existing command, we can leverage the existing tests to catch any
inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for
dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref
update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com/
V2: https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com/
V3: https://lore.kernel.org/git/20240423212818.574123-1-knayak@gitlab.com/
V4: https://lore.kernel.org/r/20240426152449.228860-1-knayak@gitlab.com
V5: https://lore.kernel.org/r/20240501202229.2695774-1-knayak@gitlab.com

Changes over v5 are:
- More user friendly error messages.
- `create_symref_lock` now writes to an err buf, instead of directly to stderr.
- Refactor code to make it easier to read around logical operations.
- Cleanup commit message and fix typos.

Thanks to all reviewers!

Range diff:

1:  a354190905 = 1:  a354190905 refs: accept symref values in `ref_transaction_update()`
2:  7dff21dbef ! 2:  0d9c5b9804 files-backend: extract out `create_symref_lock()`
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     -				const char *target, const char *logmsg)
     +static int create_symref_lock(struct files_ref_store *refs,
     +			      struct ref_lock *lock, const char *refname,
    -+			      const char *target)
    ++			      const char *target, struct strbuf *err)
      {
    -+	if (!fdopen_lock_file(&lock->lk, "w"))
    -+		return error("unable to fdopen %s: %s",
    ++	if (!fdopen_lock_file(&lock->lk, "w")) {
    ++		strbuf_addf(err, "unable to fdopen %s: %s",
     +			     get_lock_file_path(&lock->lk), strerror(errno));
    ++		return -1;
    ++	}
     +
    -+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
    -+		return error("unable to fprintf %s: %s",
    ++	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
    ++		strbuf_addf(err, "unable to write to %s: %s",
     +			     get_lock_file_path(&lock->lk), strerror(errno));
    ++		return -1;
    ++	}
    ++
     +	return 0;
     +}
     +
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     +				    struct ref_lock *lock, const char *refname,
     +				    const char *target, const char *logmsg)
     +{
    ++	struct strbuf err = STRBUF_INIT;
     +	int ret;
     +
      	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     -	if (!fdopen_lock_file(&lock->lk, "w"))
     -		return error("unable to fdopen %s: %s",
     -			     get_lock_file_path(&lock->lk), strerror(errno));
    -+	ret = create_symref_lock(refs, lock, refname, target);
    ++	ret = create_symref_lock(refs, lock, refname, target, &err);
     +	if (!ret) {
     +		update_symref_reflog(refs, lock, refname, target, logmsg);
      
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     +		if (commit_ref(lock) < 0)
     +			return error("unable to write symref for %s: %s", refname,
     +				     strerror(errno));
    ++	} else {
    ++		return error("%s", err.buf);
     +	}
      
     -	/* no error check; commit_ref will check ferror */
3:  901a586683 ! 3:  e0219ffd31 refs: support symrefs in 'reference-transaction' hook
    @@ refs.c: static int run_transaction_hook(struct ref_transaction *transaction,
     -			    oid_to_hex(&update->new_oid),
     -			    update->refname);
     +
    -+		if (update->flags & REF_HAVE_OLD && update->old_target)
    ++		if (!(update->flags & REF_HAVE_OLD))
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
    ++		else if (update->old_target)
     +			strbuf_addf(&buf, "ref:%s ", update->old_target);
     +		else
     +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
     +
    -+		if (update->flags & REF_HAVE_NEW && update->new_target)
    ++		if (!(update->flags & REF_HAVE_NEW))
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
    ++		else if (update->new_target)
     +			strbuf_addf(&buf, "ref:%s ", update->new_target);
     +		else
     +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
4:  6c97f6a660 ! 4:  b22c59c722 refs: add support for transactional symref updates
    @@ Commit message
         However, we never supported transactional updates of symrefs. Let's add
         support for symrefs in both the 'files' and the 'reftable' backend.
     
    -    Here, we add and use `ref_update_is_null_new_value()`, a helper function
    -    which is used to check if there is a new_value in a reference update.
    -    The new value could either be a symref target `new_target` or a OID
    -    `new_oid`.
    +    Here, we add and use `ref_update_has_null_new_value()`, a helper
    +    function which is used to check if there is a new_value in a reference
    +    update. The new value could either be a symref target `new_target` or a
    +    OID `new_oid`.
     
         With this, now transactional updates (verify, create, delete, update)
         can be used for:
    @@ Commit message
         This also allows us to expose this to users via new commands in
         'git-update-ref' in the future.
     
    -    We do not add reflog for dangling symref updates, because currently
    -    'git-symbolic-ref' doesn't add reflog for dangling symref updates and it
    -    would be best to keep this behavior consistent as we would move it to
    -    start using transaction based updates in the following commit.
    +    Note that a dangling symref update does not record a new reflog entry,
    +    which is unchanged before and after this commit.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      	update->flags = flags;
      
     -	if (flags & REF_HAVE_NEW)
    -+	if (new_target)
    -+		update->new_target = xstrdup(new_target);
    -+	if (old_target)
    -+		update->old_target = xstrdup(old_target);
    -+	if (new_oid && flags & REF_HAVE_NEW)
    ++	update->new_target = xstrdup_or_null(new_target);
    ++	update->old_target = xstrdup_or_null(old_target);
    ++	if ((flags & REF_HAVE_NEW) && new_oid)
      		oidcpy(&update->new_oid, new_oid);
     -	if (flags & REF_HAVE_OLD)
    -+	if (old_oid && flags & REF_HAVE_OLD)
    ++	if ((flags & REF_HAVE_OLD) && old_oid)
      		oidcpy(&update->old_oid, old_oid);
     +
      	update->msg = normalize_reflog_message(msg);
    @@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char
      	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
      }
     +
    -+int ref_update_is_null_new_value(struct ref_update *update)
    ++int ref_update_has_null_new_value(struct ref_update *update)
     +{
     +	return !update->new_target && is_null_oid(&update->new_oid);
     +}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      	files_assert_main_repository(refs, "lock_ref_for_update");
      
     -	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
    -+	if ((update->flags & REF_HAVE_NEW) && ref_update_is_null_new_value(update))
    ++	if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update))
      		update->flags |= REF_DELETING;
      
      	if (head_ref) {
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     -	    !(update->flags & REF_DELETING) &&
     -	    !(update->flags & REF_LOG_ONLY)) {
     +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
    -+		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
    ++		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
     +			ret = TRANSACTION_GENERIC_ERROR;
     +			goto out;
     +		}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      		if (!(update->type & REF_ISSYMREF) &&
      		    oideq(&lock->old_oid, &update->new_oid)) {
      			/*
    +@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
    + 	return ret;
    + }
    + 
    ++static int parse_and_write_reflog(struct files_ref_store *refs,
    ++				  struct ref_update *update,
    ++				  struct ref_lock *lock,
    ++				  struct strbuf *err)
    ++{
    ++	if (update->new_target) {
    ++		/*
    ++		 * We want to get the resolved OID for the target, to ensure
    ++		 * that the correct value is added to the reflog.
    ++		 */
    ++		if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
    ++					     RESOLVE_REF_READING,
    ++					     &update->new_oid, NULL)) {
    ++			/*
    ++			 * TODO: currently we skip creating reflogs for dangling
    ++			 * symref updates. It would be nice to capture this as
    ++			 * zero oid updates however.
    ++			 */
    ++			return 0;
    ++		}
    ++	}
    ++
    ++	if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid,
    ++				&update->new_oid, update->msg, update->flags, err)) {
    ++		char *old_msg = strbuf_detach(err, NULL);
    ++
    ++		strbuf_addf(err, "cannot update the ref '%s': %s",
    ++			    lock->ref_name, old_msg);
    ++		free(old_msg);
    ++		unlock_ref(lock);
    ++		update->backend_data = NULL;
    ++		return -1;
    ++	}
    ++
    ++	return 0;
    ++}
    ++
    + static int files_transaction_finish(struct ref_store *ref_store,
    + 				    struct ref_transaction *transaction,
    + 				    struct strbuf *err)
     @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
      
      		if (update->flags & REF_NEEDS_COMMIT ||
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
     -						&update->new_oid,
     -						update->msg, update->flags,
     -						err)) {
    -+			int create_reflog = 1;
    -+
    -+			if (update->new_target) {
    -+				/*
    -+				 * We want to get the resolved OID for the target, to ensure
    -+				 * that the correct value is added to the reflog.
    -+				 */
    -+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
    -+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
    -+					/* for dangling symrefs we skip creating a reflog entry. */
    -+					create_reflog = 0;
    -+				}
    -+			}
    -+
    -+			if (create_reflog && files_log_ref_write(refs,
    -+								 lock->ref_name,
    -+								 &lock->old_oid,
    -+								 &update->new_oid,
    -+								 update->msg, update->flags,
    -+								 err)) {
    - 				char *old_msg = strbuf_detach(err, NULL);
    - 
    - 				strbuf_addf(err, "cannot update the ref '%s': %s",
    -@@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
    +-				char *old_msg = strbuf_detach(err, NULL);
    +-
    +-				strbuf_addf(err, "cannot update the ref '%s': %s",
    +-					    lock->ref_name, old_msg);
    +-				free(old_msg);
    +-				unlock_ref(lock);
    +-				update->backend_data = NULL;
    ++			if (parse_and_write_reflog(refs, update, lock, err)) {
    + 				ret = TRANSACTION_GENERIC_ERROR;
      				goto cleanup;
      			}
      		}
    @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
     + * takes into consideration that the update could be a regular
     + * ref or a symbolic ref.
     + */
    -+int ref_update_is_null_new_value(struct ref_update *update);
    ++int ref_update_has_null_new_value(struct ref_update *update);
     +
      #endif /* REFS_REFS_INTERNAL_H */
     
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      			 * when the reference in question doesn't exist.
      			 */
     -			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
    -+			 if (u->flags & REF_HAVE_NEW && !ref_update_is_null_new_value(u)) {
    ++			 if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
      				 ret = queue_transaction_update(refs, tx_data, u,
      								&current_oid, err);
      				 if (ret)
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
     +				ret = -1;
     +				goto done;
     +			}
    -+		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
    ++		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
      			if (is_null_oid(&u->old_oid))
      				strbuf_addf(err, _("cannot lock ref '%s': "
      					    "reference already exists"),
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      		 *   the given ref.
      		 */
     -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
    -+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && ref_update_is_null_new_value(u)) {
    ++		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
      			struct reftable_log_record log = {0};
      			struct reftable_iterator it = {0};
      
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      			    should_write_log(&arg->refs->base, u->refname))) {
      			struct reftable_log_record *log;
     +			int create_reflog = 1;
    ++
    ++			if (u->new_target) {
    ++				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
    ++							     RESOLVE_REF_READING, &u->new_oid, NULL)) {
    ++					/*
    ++					 * TODO: currently we skip creating reflogs for dangling
    ++					 * symref updates. It would be nice to capture this as
    ++					 * zero oid updates however.
    ++					 */
    ++					create_reflog = 0;
    ++				}
    ++			}
      
     -			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
     -			log = &logs[logs_nr++];
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     -			memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
     -			log->value.update.message =
     -				xstrndup(u->msg, arg->refs->write_options.block_size / 2);
    -+			if (u->new_target)
    -+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
    -+							     RESOLVE_REF_READING, &u->new_oid, NULL))
    -+					/* for dangling symrefs we skip creating reflog */
    -+					create_reflog = 0;
    -+
     +			if (create_reflog) {
     +				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
     +				log = &logs[logs_nr++];
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      			continue;
      
     -		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
    -+		if (u->flags & REF_HAVE_NEW && u->new_target) {
    ++		if (u->new_target) {
     +			struct reftable_ref_record ref = {
     +				.refname = (char *)u->refname,
     +				.value_type = REFTABLE_REF_SYMREF,
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     +			ret = reftable_writer_add_ref(writer, &ref);
     +			if (ret < 0)
     +				goto done;
    -+		} else if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
    ++		} else if ((u->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(u)) {
      			struct reftable_ref_record ref = {
      				.refname = (char *)u->refname,
      				.update_index = ts,
5:  5b55406430 ! 5:  636bf5ce98 refs: use transaction in `refs_create_symref()`
    @@ Commit message
         target. To do this, it uses a ref-backend specific function
         `create_symref()`.
     
    -    In this previous commit, we introduce symref support in transactions.
    -    This means we can now use transactions to perform symref updates and not
    -    have to resort to `create_symref()`. Doing this allows us to remove and
    -    cleanup `create_symref()`, which we will do in the following commit.
    +    In the previous commits, we introduced symref support in transactions.
    +    This means we can now use transactions to perform symref updates and
    +    don't have to resort to `create_symref()`. Doing this allows us to
    +    remove and cleanup `create_symref()`, which we will do in the following
    +    commit.
     
         Modify the expected error message for a test in
    -    't/t0610-reftable-basics.sh', since the error is now thrown from the
    +    't/t0610-reftable-basics.sh', since the error is now thrown from
         'refs.c'. This is because in transactional updates, F/D conflicts are
         caught before we're in the reference backend.
     
    @@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls s
     +	git symbolic-ref refs/heads/symref refs/heads/main &&
     +
     +	cat >expect <<-EOF &&
    -+		prepared
    -+		$ZERO_OID ref:refs/heads/main refs/heads/symref
    -+		committed
    -+		$ZERO_OID ref:refs/heads/main refs/heads/symref
    ++	prepared
    ++	$ZERO_OID ref:refs/heads/main refs/heads/symref
    ++	committed
    ++	$ZERO_OID ref:refs/heads/main refs/heads/symref
     +	EOF
     +
     +	test_cmp expect actual
6:  9e25816e68 = 6:  07fb23374f refs: rename `refs_create_symref()` to `refs_update_symref()`
7:  3836e25932 ! 7:  5c05813bcc refs: remove `create_symref` and associated dead code
    @@ Commit message
         transactions to perform symref updates. Earlier `refs_create_symref()`
         used `create_symref()` to do the same.
     
    -    This means, we can now remove `create_symref()` and any code associated
    -    with it which is no longer used. We remove `create_symref()` code from
    -    all the reference backends and also remove it entirely from the
    -    `ref_storage_be` struct.
    +    We can now remove `create_symref()` and any code associated with it
    +    which is no longer used. We remove `create_symref()` code from all the
    +    reference backends and also remove it entirely from the `ref_storage_be`
    +    struct.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
    @@ refs/files-backend.c: static int create_ref_symlink(struct ref_lock *lock, const
     -
      static int create_symref_lock(struct files_ref_store *refs,
      			      struct ref_lock *lock, const char *refname,
    - 			      const char *target)
    + 			      const char *target, struct strbuf *err)
     @@ refs/files-backend.c: static int create_symref_lock(struct files_ref_store *refs,
      	return 0;
      }
    @@ refs/files-backend.c: static int create_symref_lock(struct files_ref_store *refs
     -				    struct ref_lock *lock, const char *refname,
     -				    const char *target, const char *logmsg)
     -{
    +-	struct strbuf err = STRBUF_INIT;
     -	int ret;
     -
     -	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
    @@ refs/files-backend.c: static int create_symref_lock(struct files_ref_store *refs
     -		return 0;
     -	}
     -
    --	ret = create_symref_lock(refs, lock, refname, target);
    +-	ret = create_symref_lock(refs, lock, refname, target, &err);
     -	if (!ret) {
     -		update_symref_reflog(refs, lock, refname, target, logmsg);
     -
     -		if (commit_ref(lock) < 0)
     -			return error("unable to write symref for %s: %s", refname,
     -				     strerror(errno));
    +-	} else {
    +-		return error("%s", err.buf);
     -	}
     -
     -	return ret;


Karthik Nayak (7):
  refs: accept symref values in `ref_transaction_update()`
  files-backend: extract out `create_symref_lock()`
  refs: support symrefs in 'reference-transaction' hook
  refs: add support for transactional symref updates
  refs: use transaction in `refs_create_symref()`
  refs: rename `refs_create_symref()` to `refs_update_symref()`
  refs: remove `create_symref` and associated dead code

 Documentation/githooks.txt       |  14 +-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           |  89 +++++++++----
 refs.h                           |  20 ++-
 refs/debug.c                     |  13 --
 refs/files-backend.c             | 213 +++++++++++++++++++------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  26 +++-
 refs/reftable-backend.c          | 165 +++++++++---------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t0610-reftable-basics.sh       |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 22 files changed, 351 insertions(+), 246 deletions(-)

-- 
2.43.GIT

[PATCH v6 1/7] refs: accept symref values in `ref_transaction_update()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `ref_transaction_update()` obtains ref information and
flags to create a `ref_update` and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the
old and new ref targets and process it. This commit adds the required
parameters to the function and modifies all call sites.

The two parameters added are `new_target` and `old_target`. The
`new_target` is used to denote what the reference should point to when
the transaction is applied. Some functions allow this parameter to be
NULL, meaning that the reference is not changed.

The `old_target` denotes the value the reference must have before the
update. Some functions allow this parameter to be NULL, meaning that the
old value of the reference is not checked.

We also update the internal function `ref_transaction_add_update()`
similarly to take the two new parameters.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)

diff --git a/branch.c b/branch.c
index e4a738fc7b..48af4c3ceb 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index dc5a9d32dd..297dfb91a1 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5857d860db..66840b7c5b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e8d7df14b6..b150ef39a8 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..7690687b0e 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 9a33cb50b4..40a65fdebc 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index e46afbc46d..21fdbf6ac8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
diff --git a/refs.c b/refs.c
index 55d2e0b2cb..47bc9dd103 100644
--- a/refs.c
+++ b/refs.c
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && !is_null_oid(old_oid) && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && !is_null_oid(new_oid) && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index d278775e08..c7851bf587 100644
--- a/refs.h
+++ b/refs.h
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index a098d14ea0..e4d0aa3d41 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..108f4ec419 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 1cda48c504..6104471199 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 88de4dc20f..61e007d85f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index c0fd632d92..1b3df43906 100644
--- a/walker.c
+++ b/walker.c
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
-- 
2.43.GIT

Re: [PATCH v6 1/7] refs: accept symref values in `ref_transaction_update()`

[Phillip Wood]

Hi Karthik

On 03/05/2024 13:41, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The function `ref_transaction_update()` obtains ref information and
> flags to create a `ref_update` and add them to the transaction at hand.
> 
> To extend symref support in transactions, we need to also accept the
> old and new ref targets and process it. This commit adds the required
> parameters to the function and modifies all call sites.
> 
> The two parameters added are `new_target` and `old_target`. The
> `new_target` is used to denote what the reference should point to when
> the transaction is applied. Some functions allow this parameter to be
> NULL, meaning that the reference is not changed.
> 
> The `old_target` denotes the value the reference must have before the
> update. Some functions allow this parameter to be NULL, meaning that the
> old value of the reference is not checked.
> 
> We also update the internal function `ref_transaction_add_update()`
> similarly to take the two new parameters.

The documentation for the new parameters looks good to me now - thanks 
for updating it. I'm confused about the assertions though as I mentioned 
in my other message [1].

Best Wishes

Phillip

[1] 
https://www.lore.kernel.org/git/7ca8c2c4-a9cc-4bec-b13c-95d7854b664b@gmail.com

> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
>   branch.c                |  2 +-
>   builtin/fast-import.c   |  5 +++--
>   builtin/fetch.c         |  2 +-
>   builtin/receive-pack.c  |  1 +
>   builtin/replace.c       |  2 +-
>   builtin/tag.c           |  1 +
>   builtin/update-ref.c    |  1 +
>   refs.c                  | 22 +++++++++++++++++-----
>   refs.h                  | 18 +++++++++++++++++-
>   refs/files-backend.c    | 12 ++++++------
>   refs/refs-internal.h    | 14 ++++++++++++++
>   refs/reftable-backend.c |  4 ++--
>   sequencer.c             |  9 +++++----
>   walker.c                |  2 +-
>   14 files changed, 71 insertions(+), 24 deletions(-)
> 
> diff --git a/branch.c b/branch.c
> index e4a738fc7b..48af4c3ceb 100644
> --- a/branch.c
> +++ b/branch.c
> @@ -627,7 +627,7 @@ void create_branch(struct repository *r,
>   	if (!transaction ||
>   		ref_transaction_update(transaction, ref.buf,
>   					&oid, forcing ? NULL : null_oid(),
> -					0, msg, &err) ||
> +					NULL, NULL, 0, msg, &err) ||
>   		ref_transaction_commit(transaction, &err))
>   		die("%s", err.buf);
>   	ref_transaction_free(transaction);
> diff --git a/builtin/fast-import.c b/builtin/fast-import.c
> index dc5a9d32dd..297dfb91a1 100644
> --- a/builtin/fast-import.c
> +++ b/builtin/fast-import.c
> @@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
>   	transaction = ref_transaction_begin(&err);
>   	if (!transaction ||
>   	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
> -				   0, msg, &err) ||
> +				   NULL, NULL, 0, msg, &err) ||
>   	    ref_transaction_commit(transaction, &err)) {
>   		ref_transaction_free(transaction);
>   		error("%s", err.buf);
> @@ -1675,7 +1675,8 @@ static void dump_tags(void)
>   		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
>   
>   		if (ref_transaction_update(transaction, ref_name.buf,
> -					   &t->oid, NULL, 0, msg, &err)) {
> +					   &t->oid, NULL, NULL, NULL,
> +					   0, msg, &err)) {
>   			failure |= error("%s", err.buf);
>   			goto cleanup;
>   		}
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index 5857d860db..66840b7c5b 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
>   
>   	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
>   				     check_old ? &ref->old_oid : NULL,
> -				     0, msg, &err);
> +				     NULL, NULL, 0, msg, &err);
>   	if (ret) {
>   		ret = STORE_REF_ERROR_OTHER;
>   		goto out;
> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
> index e8d7df14b6..b150ef39a8 100644
> --- a/builtin/receive-pack.c
> +++ b/builtin/receive-pack.c
> @@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
>   		if (ref_transaction_update(transaction,
>   					   namespaced_name,
>   					   new_oid, old_oid,
> +					   NULL, NULL,
>   					   0, "push",
>   					   &err)) {
>   			rp_error("%s", err.buf);
> diff --git a/builtin/replace.c b/builtin/replace.c
> index da59600ad2..7690687b0e 100644
> --- a/builtin/replace.c
> +++ b/builtin/replace.c
> @@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
>   	transaction = ref_transaction_begin(&err);
>   	if (!transaction ||
>   	    ref_transaction_update(transaction, ref.buf, repl, &prev,
> -				   0, NULL, &err) ||
> +				   NULL, NULL, 0, NULL, &err) ||
>   	    ref_transaction_commit(transaction, &err))
>   		res = error("%s", err.buf);
>   
> diff --git a/builtin/tag.c b/builtin/tag.c
> index 9a33cb50b4..40a65fdebc 100644
> --- a/builtin/tag.c
> +++ b/builtin/tag.c
> @@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
>   	transaction = ref_transaction_begin(&err);
>   	if (!transaction ||
>   	    ref_transaction_update(transaction, ref.buf, &object, &prev,
> +				   NULL, NULL,
>   				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
>   				   reflog_msg.buf, &err) ||
>   	    ref_transaction_commit(transaction, &err)) {
> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
> index e46afbc46d..21fdbf6ac8 100644
> --- a/builtin/update-ref.c
> +++ b/builtin/update-ref.c
> @@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
>   
>   	if (ref_transaction_update(transaction, refname,
>   				   &new_oid, have_old ? &old_oid : NULL,
> +				   NULL, NULL,
>   				   update_flags | create_reflog_flag,
>   				   msg, &err))
>   		die("%s", err.buf);
> diff --git a/refs.c b/refs.c
> index 55d2e0b2cb..47bc9dd103 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
>   		const char *refname, unsigned int flags,
>   		const struct object_id *new_oid,
>   		const struct object_id *old_oid,
> +		const char *new_target, const char *old_target,
>   		const char *msg)
>   {
>   	struct ref_update *update;
> @@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
>   	if (transaction->state != REF_TRANSACTION_OPEN)
>   		BUG("update called for transaction that is not open");
>   
> +	if (old_oid && !is_null_oid(old_oid) && old_target)
> +		BUG("only one of old_oid and old_target should be non NULL");
> +	if (new_oid && !is_null_oid(new_oid) && new_target)
> +		BUG("only one of new_oid and new_target should be non NULL");
> +
>   	FLEX_ALLOC_STR(update, refname, refname);
>   	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
>   	transaction->updates[transaction->nr++] = update;
> @@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
>   			   const char *refname,
>   			   const struct object_id *new_oid,
>   			   const struct object_id *old_oid,
> +			   const char *new_target,
> +			   const char *old_target,
>   			   unsigned int flags, const char *msg,
>   			   struct strbuf *err)
>   {
> @@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
>   	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
>   
>   	ref_transaction_add_update(transaction, refname, flags,
> -				   new_oid, old_oid, msg);
> +				   new_oid, old_oid, new_target,
> +				   old_target, msg);
>   	return 0;
>   }
>   
> @@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
>   		return 1;
>   	}
>   	return ref_transaction_update(transaction, refname, new_oid,
> -				      null_oid(), flags, msg, err);
> +				      null_oid(), NULL, NULL, flags,
> +				      msg, err);
>   }
>   
>   int ref_transaction_delete(struct ref_transaction *transaction,
> @@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
>   		BUG("delete called with old_oid set to zeros");
>   	return ref_transaction_update(transaction, refname,
>   				      null_oid(), old_oid,
> -				      flags, msg, err);
> +				      NULL, NULL, flags,
> +				      msg, err);
>   }
>   
>   int ref_transaction_verify(struct ref_transaction *transaction,
> @@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
>   		BUG("verify called with old_oid set to NULL");
>   	return ref_transaction_update(transaction, refname,
>   				      NULL, old_oid,
> +				      NULL, NULL,
>   				      flags, NULL, err);
>   }
>   
> @@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
>   
>   	t = ref_store_transaction_begin(refs, &err);
>   	if (!t ||
> -	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
> -				   &err) ||
> +	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
> +				   flags, msg, &err) ||
>   	    ref_transaction_commit(t, &err)) {
>   		ret = 1;
>   		ref_transaction_free(t);
> diff --git a/refs.h b/refs.h
> index d278775e08..c7851bf587 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>    *         before the update. A copy of this value is made in the
>    *         transaction.
>    *
> + *     new_target -- the target reference that the reference will be
> + *         updated to point to. If the reference is a regular reference,
> + *         it will be converted to a symbolic reference. Cannot be set
> + *         together with `new_oid`. A copy of this value is made in the
> + *         transaction.
> + *
> + *     old_target -- the reference that the reference must be pointing to.
> + *         Canont be set together with `old_oid`. A copy of this value is
> + *         made in the transaction.
> + *
>    *     flags -- flags affecting the update, passed to
>    *         update_ref_lock(). Possible flags: REF_NO_DEREF,
>    *         REF_FORCE_CREATE_REFLOG. See those constants for more
> @@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
>    * beforehand. The old value is checked after the lock is taken to
>    * prevent races. If the old value doesn't agree with old_oid, the
>    * whole transaction fails. If old_oid is NULL, then the previous
> - * value is not checked.
> + * value is not checked. If `old_target` is not NULL, treat the reference
> + * as a symbolic ref and validate that its target before the update is
> + * `old_target`. If the `new_target` is not NULL, then the reference
> + * will be updated to a symbolic ref which targets `new_target`.
> + * Together, these allow us to update between regular refs and symrefs.
>    *
>    * See the above comment "Reference transaction updates" for more
>    * information.
> @@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
>   			   const char *refname,
>   			   const struct object_id *new_oid,
>   			   const struct object_id *old_oid,
> +			   const char *new_target,
> +			   const char *old_target,
>   			   unsigned int flags, const char *msg,
>   			   struct strbuf *err);
>   
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index a098d14ea0..e4d0aa3d41 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
>   	ref_transaction_add_update(
>   			transaction, r->name,
>   			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
> -			null_oid(), &r->oid, NULL);
> +			null_oid(), &r->oid, NULL, NULL, NULL);
>   	if (ref_transaction_commit(transaction, &err))
>   		goto cleanup;
>   
> @@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
>   		 * packed-refs transaction:
>   		 */
>   		if (ref_transaction_update(transaction, iter->refname,
> -					   iter->oid, NULL,
> +					   iter->oid, NULL, NULL, NULL,
>   					   REF_NO_DEREF, NULL, &err))
>   			die("failure preparing to create packed reference %s: %s",
>   			    iter->refname, err.buf);
> @@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
>   			transaction, "HEAD",
>   			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
>   			&update->new_oid, &update->old_oid,
> -			update->msg);
> +			NULL, NULL, update->msg);
>   
>   	/*
>   	 * Add "HEAD". This insertion is O(N) in the transaction
> @@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
>   	new_update = ref_transaction_add_update(
>   			transaction, referent, new_flags,
>   			&update->new_oid, &update->old_oid,
> -			update->msg);
> +			NULL, NULL, update->msg);
>   
>   	new_update->parent_update = update;
>   
> @@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
>   					packed_transaction, update->refname,
>   					REF_HAVE_NEW | REF_NO_DEREF,
>   					&update->new_oid, NULL,
> -					NULL);
> +					NULL, NULL, NULL);
>   		}
>   	}
>   
> @@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
>   		ref_transaction_add_update(packed_transaction, update->refname,
>   					   update->flags & ~REF_HAVE_OLD,
>   					   &update->new_oid, &update->old_oid,
> -					   NULL);
> +					   NULL, NULL, NULL);
>   	}
>   
>   	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 56641aa57a..108f4ec419 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -124,6 +124,19 @@ struct ref_update {
>   	 */
>   	struct object_id old_oid;
>   
> +	/*
> +	 * If set, point the reference to this value. This can also be
> +	 * used to convert regular references to become symbolic refs.
> +	 * Cannot be set together with `new_oid`.
> +	 */
> +	const char *new_target;
> +
> +	/*
> +	 * If set, check that the reference previously pointed to this
> +	 * value. Cannot be set together with `old_oid`.
> +	 */
> +	const char *old_target;
> +
>   	/*
>   	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
>   	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
> @@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
>   		const char *refname, unsigned int flags,
>   		const struct object_id *new_oid,
>   		const struct object_id *old_oid,
> +		const char *new_target, const char *old_target,
>   		const char *msg);
>   
>   /*
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index 1cda48c504..6104471199 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>   			new_update = ref_transaction_add_update(
>   					transaction, "HEAD",
>   					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
> -					&u->new_oid, &u->old_oid, u->msg);
> +					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
>   			string_list_insert(&affected_refnames, new_update->refname);
>   		}
>   
> @@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>   				 */
>   				new_update = ref_transaction_add_update(
>   						transaction, referent.buf, new_flags,
> -						&u->new_oid, &u->old_oid, u->msg);
> +						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
>   				new_update->parent_update = u;
>   
>   				/*
> diff --git a/sequencer.c b/sequencer.c
> index 88de4dc20f..61e007d85f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
>   	if (!transaction ||
>   	    ref_transaction_update(transaction, "HEAD",
>   				   to, unborn && !is_rebase_i(opts) ?
> -				   null_oid() : from,
> +				   null_oid() : from, NULL, NULL,
>   				   0, sb.buf, &err) ||
>   	    ref_transaction_commit(transaction, &err)) {
>   		ref_transaction_free(transaction);
> @@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
>   	if (!transaction ||
>   	    ref_transaction_update(transaction, "HEAD", new_head,
>   				   old_head ? &old_head->object.oid : null_oid(),
> -				   0, sb.buf, err) ||
> +				   NULL, NULL, 0, sb.buf, err) ||
>   	    ref_transaction_commit(transaction, err)) {
>   		ret = -1;
>   	}
> @@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
>   	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
>   		error(_("could not read HEAD"));
>   		ret = -1;
> -	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
> -					  NULL, 0, msg.buf, &err) < 0 ||
> +	} else if (ref_transaction_update(transaction, ref_name.buf,
> +					  &head_oid, NULL, NULL, NULL,
> +					  0, msg.buf, &err) < 0 ||
>   		   ref_transaction_commit(transaction, &err)) {
>   		error("%s", err.buf);
>   		ret = -1;
> diff --git a/walker.c b/walker.c
> index c0fd632d92..1b3df43906 100644
> --- a/walker.c
> +++ b/walker.c
> @@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
>   		strbuf_reset(&refname);
>   		strbuf_addf(&refname, "refs/%s", write_ref[i]);
>   		if (ref_transaction_update(transaction, refname.buf,
> -					   oids + i, NULL, 0,
> +					   oids + i, NULL, NULL, NULL, 0,
>   					   msg ? msg : "fetch (unknown)",
>   					   &err)) {
>   			error("%s", err.buf);

Re: [PATCH v6 1/7] refs: accept symref values in `ref_transaction_update()`

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 3324 bytes --]

Hey Phillip,

Phillip Wood <phillip.wood123@gmail.com> writes:

> Hi Karthik
>
> On 03/05/2024 13:41, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The function `ref_transaction_update()` obtains ref information and
>> flags to create a `ref_update` and add them to the transaction at hand.
>>
>> To extend symref support in transactions, we need to also accept the
>> old and new ref targets and process it. This commit adds the required
>> parameters to the function and modifies all call sites.
>>
>> The two parameters added are `new_target` and `old_target`. The
>> `new_target` is used to denote what the reference should point to when
>> the transaction is applied. Some functions allow this parameter to be
>> NULL, meaning that the reference is not changed.
>>
>> The `old_target` denotes the value the reference must have before the
>> update. Some functions allow this parameter to be NULL, meaning that the
>> old value of the reference is not checked.
>>
>> We also update the internal function `ref_transaction_add_update()`
>> similarly to take the two new parameters.
>
> The documentation for the new parameters looks good to me now - thanks
> for updating it. I'm confused about the assertions though as I mentioned
> in my other message [1].
>
> Best Wishes
>
> Phillip
>
> [1]
> https://www.lore.kernel.org/git/7ca8c2c4-a9cc-4bec-b13c-95d7854b664b@gmail.com
>

Responding here since this is a newer thread.

This is done because in files-backend we split symref updates (see
`split_symref_update`) and add a new one for the target reference. Here
we pass along the update struct. This update struct is memset to 0 and
this is after the checks we do. So the 'new_oid' here would be set to 0
(null oid) even if the 'new_target' value is set. This made more sense
in the earlier set of patches, but probably a diff like this should work
for this series and can be amended later as needed (in the series which
adds the symref-* commands).

diff --git a/refs.c b/refs.c
index 3645b805c1..20d26da372 100644
--- a/refs.c
+++ b/refs.c
@@ -1238,9 +1238,9 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");

-	if (old_oid && !is_null_oid(old_oid) && old_target)
+	if (old_oid && old_target)
 		BUG("only one of old_oid and old_target should be non NULL");
-	if (new_oid && !is_null_oid(new_oid) && new_target)
+	if (new_oid && new_target)
 		BUG("only one of new_oid and new_target should be non NULL");

 	FLEX_ALLOC_STR(update, refname, refname);
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3ce260d07d..a718164798 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2328,8 +2328,9 @@ static int split_symref_update(struct ref_update *update,

 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
-			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target ? NULL : &update->new_oid,
+			update->old_target ? NULL : &update->old_oid,
+			update->new_target, update->old_target, update->msg);

 	new_update->parent_update = update;

I think it makes sense to make it fool proof and add this, I'll wait for
more reviews and re-roll in a day or so.

Thanks for following through.

[snip]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v6 1/7] refs: accept symref values in `ref_transaction_update()`

[phillip.wood123]

Hi Karthik

On 05/05/2024 16:10, Karthik Nayak wrote:
> Hey Phillip,
> 
> Phillip Wood <phillip.wood123@gmail.com> writes:
> 
>> Hi Karthik
>>
>> On 03/05/2024 13:41, Karthik Nayak wrote:
>>> From: Karthik Nayak <karthik.188@gmail.com>
>>>
>>> The function `ref_transaction_update()` obtains ref information and
>>> flags to create a `ref_update` and add them to the transaction at hand.
>>>
>>> To extend symref support in transactions, we need to also accept the
>>> old and new ref targets and process it. This commit adds the required
>>> parameters to the function and modifies all call sites.
>>>
>>> The two parameters added are `new_target` and `old_target`. The
>>> `new_target` is used to denote what the reference should point to when
>>> the transaction is applied. Some functions allow this parameter to be
>>> NULL, meaning that the reference is not changed.
>>>
>>> The `old_target` denotes the value the reference must have before the
>>> update. Some functions allow this parameter to be NULL, meaning that the
>>> old value of the reference is not checked.
>>>
>>> We also update the internal function `ref_transaction_add_update()`
>>> similarly to take the two new parameters.
>>
>> The documentation for the new parameters looks good to me now - thanks
>> for updating it. I'm confused about the assertions though as I mentioned
>> in my other message [1].
>>
>> Best Wishes
>>
>> Phillip
>>
>> [1]
>> https://www.lore.kernel.org/git/7ca8c2c4-a9cc-4bec-b13c-95d7854b664b@gmail.com
>>
> 
> Responding here since this is a newer thread.
> 
> This is done because in files-backend we split symref updates (see
> `split_symref_update`) and add a new one for the target reference. Here
> we pass along the update struct. This update struct is memset to 0 and
> this is after the checks we do. So the 'new_oid' here would be set to 0
> (null oid) even if the 'new_target' value is set. This made more sense
> in the earlier set of patches, but probably a diff like this should work
> for this series and can be amended later as needed (in the series which
> adds the symref-* commands).

Thanks for the explanation - it would be good to fix this because the 
assertions don't catch misuses of ref_transaction_update() at the moment 
when old_oid points to the null oid and old_target is also set.

Best Wishes

Phillip

> diff --git a/refs.c b/refs.c
> index 3645b805c1..20d26da372 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1238,9 +1238,9 @@ struct ref_update *ref_transaction_add_update(
>   	if (transaction->state != REF_TRANSACTION_OPEN)
>   		BUG("update called for transaction that is not open");
> 
> -	if (old_oid && !is_null_oid(old_oid) && old_target)
> +	if (old_oid && old_target)
>   		BUG("only one of old_oid and old_target should be non NULL");
> -	if (new_oid && !is_null_oid(new_oid) && new_target)
> +	if (new_oid && new_target)
>   		BUG("only one of new_oid and new_target should be non NULL");
> 
>   	FLEX_ALLOC_STR(update, refname, refname);
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 3ce260d07d..a718164798 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2328,8 +2328,9 @@ static int split_symref_update(struct ref_update *update,
> 
>   	new_update = ref_transaction_add_update(
>   			transaction, referent, new_flags,
> -			&update->new_oid, &update->old_oid,
> -			NULL, NULL, update->msg);
> +			update->new_target ? NULL : &update->new_oid,
> +			update->old_target ? NULL : &update->old_oid,
> +			update->new_target, update->old_target, update->msg);
> 
>   	new_update->parent_update = update;
> 
> I think it makes sense to make it fool proof and add this, I'll wait for
> more reviews and re-roll in a day or so.
> 
> Thanks for following through.
> 
> [snip]

[PATCH v6 2/7] files-backend: extract out `create_symref_lock()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked()` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Extract the early half of `create_symref_locked()` into a new helper
function `create_symref_lock()`. Because the name of the new function is
too similar to the original, rename the original to
`create_and_commit_symref()` to avoid confusion.

The new function `create_symref_locked()` can be used to create the
symref lock in a separate step from that of committing it. This allows
to add transactional support for symrefs, where the lock would be
created in the preparation step and the lock would be committed in the
finish step.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 51 ++++++++++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 14 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index e4d0aa3d41..74a713090c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1920,27 +1920,49 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target, struct strbuf *err)
 {
+	if (!fdopen_lock_file(&lock->lk, "w")) {
+		strbuf_addf(err, "unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
+		strbuf_addf(err, "unable to write to %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	struct strbuf err = STRBUF_INIT;
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target, &err);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	} else {
+		return error("%s", err.buf);
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1960,7 +1982,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

[PATCH v6 3/7] refs: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In a previous commit, we added the `old_target` and
`new_target` fields to the `reference_transaction_update()`. In
following commits we'll also add the code to handle symref's in the
reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying
the current format:
    <old-oid> SP <new-oid> SP <ref-name> LF
to be be:
    <old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same.
But when either 'old-value' or 'new-value' is a symref, we print the ref
as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction'
hook's documentation always stated that support for symbolic references
may be added in the future.

We do not add any tests in this commit since there is no git command
which activates this flow, in an upcoming commit, we'll start using
transaction based symref updates as the default, we'll add tests there
for the hook too.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 20 ++++++++++++++++----
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ee9b92c90d..06e997131b 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index 47bc9dd103..203a101988 100644
--- a/refs.c
+++ b/refs.c
@@ -2350,10 +2350,22 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (!(update->flags & REF_HAVE_OLD))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (!(update->flags & REF_HAVE_NEW))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
-- 
2.43.GIT

[PATCH v6 4/7] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The reference backends currently support transactional reference
updates. While this is exposed to users via 'git-update-ref' and its
'--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's add
support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use `ref_update_has_null_new_value()`, a helper
function which is used to check if there is a new_value in a reference
update. The new value could either be a symref target `new_target` or a
OID `new_oid`.

With this, now transactional updates (verify, create, delete, update)
can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in
'git-update-ref' in the future.

Note that a dangling symref update does not record a new reflog entry,
which is unchanged before and after this commit.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                  |  15 ++++-
 refs/files-backend.c    | 137 ++++++++++++++++++++++++++++++++++------
 refs/refs-internal.h    |   7 ++
 refs/reftable-backend.c |  77 +++++++++++++++++-----
 4 files changed, 198 insertions(+), 38 deletions(-)

diff --git a/refs.c b/refs.c
index 203a101988..acb153b6f8 100644
--- a/refs.c
+++ b/refs.c
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,10 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	update->new_target = xstrdup_or_null(new_target);
+	update->old_target = xstrdup_or_null(old_target);
+	if ((flags & REF_HAVE_NEW) && new_oid)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if ((flags & REF_HAVE_OLD) && old_oid)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1286,6 +1291,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2814,3 +2820,8 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int ref_update_has_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 74a713090c..e1f0ca74c0 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2434,6 +2434,36 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with current_target, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+static int check_old_target(struct ref_update *update,
+			    const char *current_target,
+			    struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(update->old_target, current_target))
+		return 0;
+
+	if (!strcmp(current_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    current_target, update->old_target);
+
+	return -1;
+}
+
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2494,7 +2524,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
@@ -2537,7 +2567,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
-			} else if (check_old_oid(update, &lock->old_oid, err)) {
+			}
+
+			if (update->old_target) {
+				if (check_old_target(update, referent.buf, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}
+			} else if  (check_old_oid(update, &lock->old_oid, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto out;
 			}
@@ -2558,7 +2595,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 	} else {
 		struct ref_update *parent_update;
 
-		if (check_old_oid(update, &lock->old_oid, err)) {
+		/*
+		 * Even if the ref is a regular ref, if `old_target` is set, we
+		 * check the referent value. Ideally `old_target` should only
+		 * be set for symrefs, but we're strict about its usage.
+		 */
+		if (update->old_target) {
+			if (check_old_target(update, referent.buf, err)) {
+				ret = TRANSACTION_GENERIC_ERROR;
+				goto out;
+			}
+		} else if  (check_old_oid(update, &lock->old_oid, err)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
 		}
@@ -2576,9 +2623,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		update->flags |= REF_NEEDS_COMMIT;
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
@@ -2841,6 +2906,43 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 	return ret;
 }
 
+static int parse_and_write_reflog(struct files_ref_store *refs,
+				  struct ref_update *update,
+				  struct ref_lock *lock,
+				  struct strbuf *err)
+{
+	if (update->new_target) {
+		/*
+		 * We want to get the resolved OID for the target, to ensure
+		 * that the correct value is added to the reflog.
+		 */
+		if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
+					     RESOLVE_REF_READING,
+					     &update->new_oid, NULL)) {
+			/*
+			 * TODO: currently we skip creating reflogs for dangling
+			 * symref updates. It would be nice to capture this as
+			 * zero oid updates however.
+			 */
+			return 0;
+		}
+	}
+
+	if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid,
+				&update->new_oid, update->msg, update->flags, err)) {
+		char *old_msg = strbuf_detach(err, NULL);
+
+		strbuf_addf(err, "cannot update the ref '%s': %s",
+			    lock->ref_name, old_msg);
+		free(old_msg);
+		unlock_ref(lock);
+		update->backend_data = NULL;
+		return -1;
+	}
+
+	return 0;
+}
+
 static int files_transaction_finish(struct ref_store *ref_store,
 				    struct ref_transaction *transaction,
 				    struct strbuf *err)
@@ -2871,23 +2973,20 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
-			if (files_log_ref_write(refs,
-						lock->ref_name,
-						&lock->old_oid,
-						&update->new_oid,
-						update->msg, update->flags,
-						err)) {
-				char *old_msg = strbuf_detach(err, NULL);
-
-				strbuf_addf(err, "cannot update the ref '%s': %s",
-					    lock->ref_name, old_msg);
-				free(old_msg);
-				unlock_ref(lock);
-				update->backend_data = NULL;
+			if (parse_and_write_reflog(refs, update, lock, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next update. If not, we try and create a regular symref.
+		 */
+		if (update->new_target && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->new_target))
+				continue;
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 108f4ec419..03eda70ea4 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -749,4 +749,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_has_null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 6104471199..54c4d8b771 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -856,7 +856,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -907,8 +907,10 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 * intertwined with the locking in files-backend.c.
 				 */
 				new_update = ref_transaction_add_update(
-						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+					transaction, referent.buf, new_flags,
+					&u->new_oid, &u->old_oid, u->new_target,
+					u->old_target, u->msg);
+
 				new_update->parent_update = u;
 
 				/*
@@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {
+				if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
+				ret = -1;
+				goto done;
+			}
+		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 					    "reference already exists"),
@@ -1043,7 +1060,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1084,24 +1101,50 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			   (u->flags & REF_FORCE_CREATE_REFLOG ||
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
+			int create_reflog = 1;
+
+			if (u->new_target) {
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
+							     RESOLVE_REF_READING, &u->new_oid, NULL)) {
+					/*
+					 * TODO: currently we skip creating reflogs for dangling
+					 * symref updates. It would be nice to capture this as
+					 * zero oid updates however.
+					 */
+					create_reflog = 0;
+				}
+			}
 
-			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
-			log = &logs[logs_nr++];
-			memset(log, 0, sizeof(*log));
-
-			fill_reftable_log_record(log);
-			log->update_index = ts;
-			log->refname = xstrdup(u->refname);
-			memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
-			memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
-			log->value.update.message =
-				xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			if (create_reflog) {
+				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
+				log = &logs[logs_nr++];
+				memset(log, 0, sizeof(*log));
+
+				fill_reftable_log_record(log);
+				log->update_index = ts;
+				log->refname = xstrdup(u->refname);
+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
+				log->value.update.message =
+					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			}
 		}
 
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->new_target) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_target,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if ((u->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
-- 
2.43.GIT

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Phillip Wood]

Hi Karthik

I've left a few comments below - the most important one is about the 
error messages in the reftable backend, non of the others are worth 
re-rolling for on their own.

On 03/05/2024 13:41, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The reference backends currently support transactional reference
> updates. While this is exposed to users via 'git-update-ref' and its
> '--stdin' mode, it is also used internally within various commands.
> 
> However, we never supported transactional updates of symrefs. Let's adds

s/we never supported/we do not support/

s/Let's/This commit/

> support for symrefs in both the 'files' and the 'reftable' backend.

s/backend/backends/

> Here, we add and use `ref_update_has_null_new_value()`, a helper
> function which is used to check if there is a new_value in a reference
> update. The new value could either be a symref target `new_target` or a
> OID `new_oid`.
> 
> With this, now transactional updates (verify, create, delete, update)

s/With this, //

> can be used for:
> - regular refs
> - symbolic refs
> - conversion of regular to symbolic refs and vice versa

Excellent

> This also allows us to expose this to users via new commands in
> 'git-update-ref' in the future.

I'm slightly concerned that splitting out the update-ref changes means 
we don't have any test coverage of the new code beyond the part that is 
used by refs_create_symref()

> Note that a dangling symref update does not record a new reflog entry,
> which is unchanged before and after this commit.
> 
> +/*
> + * Check whether the old_target values stored in update are consistent
> + * with current_target, which is the symbolic reference's current value.
> + * If everything is OK, return 0; otherwise, write an error message to
> + * err and return -1.
> + */
> +static int check_old_target(struct ref_update *update,
> +			    const char *current_target,
> +			    struct strbuf *err)
> +{
> +	if (!update->old_target)
> +		BUG("called without old_target set");
> +
> +	if (!strcmp(update->old_target, current_target))
> +		return 0;
> +
> +	if (!strcmp(current_target, ""))
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "reference is missing but expected %s",
> +			    original_update_refname(update),
> +			    update->old_target);
> +	else
> +		strbuf_addf(err, "cannot lock ref '%s': "
> +			    "is at %s but expected %s",
> +			    original_update_refname(update),
> +			    current_target, update->old_target);
> +
> +	return -1;
> +}
> @@ -2576,9 +2623,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>   		}
>   	}
>   
> -	if ((update->flags & REF_HAVE_NEW) &&
> -	    !(update->flags & REF_DELETING) &&
> -	    !(update->flags & REF_LOG_ONLY)) {
> +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
> +		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {

This line looks quite long

> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>   		 * individual refs. But the error messages match what the files
>   		 * backend returns, which keeps our tests happy.
>   		 */
> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
> +		if (u->old_target) {
> +			if (strcmp(referent.buf, u->old_target)) {
> +				if (!strcmp(referent.buf, ""))
> +					strbuf_addf(err, "verifying symref target: '%s': "
> +						    "reference is missing but expected %s",
> +						    original_update_refname(u),
> +						    u->old_target);
> +				else
> +					strbuf_addf(err, "verifying symref target: '%s': "
> +						    "is at %s but expected %s",
> +						    original_update_refname(u),
> +						    referent.buf, u->old_target);

The messages in this function differ from the equivalent messages in 
check_old_target() from the files backend above. This is potentially 
confusing to users, creates more work for translators and makes it hard 
to write tests that are independent of the backend. Can we export 
check_old_target() so it can be reused here. If not we should reword 
these messages to match the other messages all of which talk about not 
being able to lock the ref.

> +				ret = -1;
> +				goto done;
> +			}
> +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
>   			if (is_null_oid(&u->old_oid))
>   				strbuf_addf(err, _("cannot lock ref '%s': "
>   					    "reference already exists"),
> @@ -1043,7 +1060,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
>   		 * - `core.logAllRefUpdates` tells us to create the reflog for
>   		 *   the given ref.
>   		 */
> -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
> +		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {

The old line was already quite long and the new one is even longer - 
perhaps we could break it after the second "&&"

> +			if (create_reflog) {
> +				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
> +				log = &logs[logs_nr++];
> +				memset(log, 0, sizeof(*log));
> +
> +				fill_reftable_log_record(log);
> +				log->update_index = ts;
> +				log->refname = xstrdup(u->refname);
> +				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
> +				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);

Both these lines would benefit from being folded

Best Wishes

Phillip

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 6563 bytes --]

Hello Phillip,


Phillip Wood <phillip.wood123@gmail.com> writes:

> Hi Karthik
>
> I've left a few comments below - the most important one is about the
> error messages in the reftable backend, non of the others are worth
> re-rolling for on their own.
>
> On 03/05/2024 13:41, Karthik Nayak wrote:
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> The reference backends currently support transactional reference
>> updates. While this is exposed to users via 'git-update-ref' and its
>> '--stdin' mode, it is also used internally within various commands.
>>
>> However, we never supported transactional updates of symrefs. Let's adds
>
> s/we never supported/we do not support/
>
> s/Let's/This commit/
>
>> support for symrefs in both the 'files' and the 'reftable' backend.
>
> s/backend/backends/
>
>> Here, we add and use `ref_update_has_null_new_value()`, a helper
>> function which is used to check if there is a new_value in a reference
>> update. The new value could either be a symref target `new_target` or a
>> OID `new_oid`.
>>
>> With this, now transactional updates (verify, create, delete, update)
>
> s/With this, //
>

Thanks, will make the above changes.

>> can be used for:
>> - regular refs
>> - symbolic refs
>> - conversion of regular to symbolic refs and vice versa
>
> Excellent
>
>> This also allows us to expose this to users via new commands in
>> 'git-update-ref' in the future.
>
> I'm slightly concerned that splitting out the update-ref changes means
> we don't have any test coverage of the new code beyond the part that is
> used by refs_create_symref()
>

This is definitely true. But I also caught a bunch of edge cases this
way because the tests which indirectly use 'refs_create_symref()' are
quite intensive.

>> Note that a dangling symref update does not record a new reflog entry,
>> which is unchanged before and after this commit.
>>
>> +/*
>> + * Check whether the old_target values stored in update are consistent
>> + * with current_target, which is the symbolic reference's current value.
>> + * If everything is OK, return 0; otherwise, write an error message to
>> + * err and return -1.
>> + */
>> +static int check_old_target(struct ref_update *update,
>> +			    const char *current_target,
>> +			    struct strbuf *err)
>> +{
>> +	if (!update->old_target)
>> +		BUG("called without old_target set");
>> +
>> +	if (!strcmp(update->old_target, current_target))
>> +		return 0;
>> +
>> +	if (!strcmp(current_target, ""))
>> +		strbuf_addf(err, "cannot lock ref '%s': "
>> +			    "reference is missing but expected %s",
>> +			    original_update_refname(update),
>> +			    update->old_target);
>> +	else
>> +		strbuf_addf(err, "cannot lock ref '%s': "
>> +			    "is at %s but expected %s",
>> +			    original_update_refname(update),
>> +			    current_target, update->old_target);
>> +
>> +	return -1;
>> +}
>> @@ -2576,9 +2623,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
>>   		}
>>   	}
>>
>> -	if ((update->flags & REF_HAVE_NEW) &&
>> -	    !(update->flags & REF_DELETING) &&
>> -	    !(update->flags & REF_LOG_ONLY)) {
>> +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
>> +		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
>
> This line looks quite long
>

Somehow I find it much easier to read these longer lines, unless there
is a logical operator, but I'll split the line mid way with the
arguments.

>> --- a/refs/reftable-backend.c
>> +++ b/refs/reftable-backend.c
>> @@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>>   		 * individual refs. But the error messages match what the files
>>   		 * backend returns, which keeps our tests happy.
>>   		 */
>> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>> +		if (u->old_target) {
>> +			if (strcmp(referent.buf, u->old_target)) {
>> +				if (!strcmp(referent.buf, ""))
>> +					strbuf_addf(err, "verifying symref target: '%s': "
>> +						    "reference is missing but expected %s",
>> +						    original_update_refname(u),
>> +						    u->old_target);
>> +				else
>> +					strbuf_addf(err, "verifying symref target: '%s': "
>> +						    "is at %s but expected %s",
>> +						    original_update_refname(u),
>> +						    referent.buf, u->old_target);
>
> The messages in this function differ from the equivalent messages in
> check_old_target() from the files backend above. This is potentially
> confusing to users, creates more work for translators and makes it hard
> to write tests that are independent of the backend. Can we export
> check_old_target() so it can be reused here. If not we should reword
> these messages to match the other messages all of which talk about not
> being able to lock the ref.
>

This is very intentional, the way the backends work at this point are
quite different and while in the files backend, we talk about locking a
particular ref. In the reftable backend we do not lock single refs. We
lock tables. So keeping it consistent doesn't make sense here.

However, we could make the files backend similar to this one, I would be
okay doing that.

>> +				ret = -1;
>> +				goto done;
>> +			}
>> +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
>>   			if (is_null_oid(&u->old_oid))
>>   				strbuf_addf(err, _("cannot lock ref '%s': "
>>   					    "reference already exists"),
>> @@ -1043,7 +1060,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
>>   		 * - `core.logAllRefUpdates` tells us to create the reflog for
>>   		 *   the given ref.
>>   		 */
>> -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
>> +		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
>
> The old line was already quite long and the new one is even longer -
> perhaps we could break it after the second "&&"
>

I will break it both the && I think that is easier on the eyes.

>> +			if (create_reflog) {
>> +				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
>> +				log = &logs[logs_nr++];
>> +				memset(log, 0, sizeof(*log));
>> +
>> +				fill_reftable_log_record(log);
>> +				log->update_index = ts;
>> +				log->refname = xstrdup(u->refname);
>> +				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
>> +				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
>
> Both these lines would benefit from being folded
>
> Best Wishes
>
> Phillip


Thanks for the review.

Karthik

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Phillip Wood]

Hi Karthik

On 05/05/2024 17:09, Karthik Nayak wrote:
> Phillip Wood <phillip.wood123@gmail.com> writes:
>> On 03/05/2024 13:41, Karthik Nayak wrote:
>>> --- a/refs/reftable-backend.c
>>> +++ b/refs/reftable-backend.c
>>> @@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>>>    		 * individual refs. But the error messages match what the files
>>>    		 * backend returns, which keeps our tests happy.
>>>    		 */
>>> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>>> +		if (u->old_target) {
>>> +			if (strcmp(referent.buf, u->old_target)) {
>>> +				if (!strcmp(referent.buf, ""))
>>> +					strbuf_addf(err, "verifying symref target: '%s': "
>>> +						    "reference is missing but expected %s",
>>> +						    original_update_refname(u),
>>> +						    u->old_target);
>>> +				else
>>> +					strbuf_addf(err, "verifying symref target: '%s': "
>>> +						    "is at %s but expected %s",
>>> +						    original_update_refname(u),
>>> +						    referent.buf, u->old_target);
>>
>> The messages in this function differ from the equivalent messages in
>> check_old_target() from the files backend above. This is potentially
>> confusing to users, creates more work for translators and makes it hard
>> to write tests that are independent of the backend. Can we export
>> check_old_target() so it can be reused here. If not we should reword
>> these messages to match the other messages all of which talk about not
>> being able to lock the ref.
>>
> 
> This is very intentional, the way the backends work at this point are
> quite different and while in the files backend, we talk about locking a
> particular ref.

I agree that the existing messages could be improved - these messages 
are returned when checking the old value of the ref so talking about 
being unable to lock the ref is not helpful as the important information 
is that the old value does not match the expected value. However that is 
not dependent on the backend or on whether the expected value is a 
symref or an oid so it feels a bit random to make these two messages 
different.

> In the reftable backend we do not lock single refs. We
> lock tables. So keeping it consistent doesn't make sense here.

Where an update is prevented by another process holding a lock I think 
that the granularity of the lock that prevents the ref from being 
updated is not particularly relevant as far as the user is concerned. As 
far as I can see the existing error messages in the reftable backend try 
to be consistent with the messages in the files backend.

> However, we could make the files backend similar to this one, I would be
> okay doing that.

I would be very happy to see the messages improved for both backends 
when the old value does not match the expected (oid or symref) value. I 
do think we should have consistent error messages in this case that are 
essentially independent of the backend and type of the expected value.

Best Wishes

Phillip

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 3771 bytes --]

Phillip Wood <phillip.wood123@gmail.com> writes:

> Hi Karthik
>
> On 05/05/2024 17:09, Karthik Nayak wrote:
>> Phillip Wood <phillip.wood123@gmail.com> writes:
>>> On 03/05/2024 13:41, Karthik Nayak wrote:
>>>> --- a/refs/reftable-backend.c
>>>> +++ b/refs/reftable-backend.c
>>>> @@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>>>>    		 * individual refs. But the error messages match what the files
>>>>    		 * backend returns, which keeps our tests happy.
>>>>    		 */
>>>> -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>>>> +		if (u->old_target) {
>>>> +			if (strcmp(referent.buf, u->old_target)) {
>>>> +				if (!strcmp(referent.buf, ""))
>>>> +					strbuf_addf(err, "verifying symref target: '%s': "
>>>> +						    "reference is missing but expected %s",
>>>> +						    original_update_refname(u),
>>>> +						    u->old_target);
>>>> +				else
>>>> +					strbuf_addf(err, "verifying symref target: '%s': "
>>>> +						    "is at %s but expected %s",
>>>> +						    original_update_refname(u),
>>>> +						    referent.buf, u->old_target);
>>>
>>> The messages in this function differ from the equivalent messages in
>>> check_old_target() from the files backend above. This is potentially
>>> confusing to users, creates more work for translators and makes it hard
>>> to write tests that are independent of the backend. Can we export
>>> check_old_target() so it can be reused here. If not we should reword
>>> these messages to match the other messages all of which talk about not
>>> being able to lock the ref.
>>>
>>
>> This is very intentional, the way the backends work at this point are
>> quite different and while in the files backend, we talk about locking a
>> particular ref.
>
> I agree that the existing messages could be improved - these messages
> are returned when checking the old value of the ref so talking about
> being unable to lock the ref is not helpful as the important information
> is that the old value does not match the expected value. However that is
> not dependent on the backend or on whether the expected value is a
> symref or an oid so it feels a bit random to make these two messages
> different.
>
>> In the reftable backend we do not lock single refs. We
>> lock tables. So keeping it consistent doesn't make sense here.
>
> Where an update is prevented by another process holding a lock I think
> that the granularity of the lock that prevents the ref from being
> updated is not particularly relevant as far as the user is concerned. As
> far as I can see the existing error messages in the reftable backend try
> to be consistent with the messages in the files backend.
>

Well, I agree. But we do have to note, that the files backend always had
`check_old_oid` which has messages along the lines of 'cannot lock ref
...' and now the new `check_old_target` will not be consistent with
those messages. Which is OK, since, like you mentioned, these messages
could be improved.

>> However, we could make the files backend similar to this one, I would be
>> okay doing that.
>
> I would be very happy to see the messages improved for both backends
> when the old value does not match the expected (oid or symref) value. I
> do think we should have consistent error messages in this case that are
> essentially independent of the backend and type of the expected value.
>
> Best Wishes
>
> Phillip

Overall, I think we're reaching the same consensus, i.e. to export this
functionality to a generic function and shared between the both
backends. I will make this change. I think the message used in the
reftable backend along the lines of 'verifying symref target ...' is
verbose yet generic enough, so I'll keep those messages.

Thanks
Karthik

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Phillip Wood]

Hi Karthik

On 06/05/2024 12:19, Karthik Nayak wrote:
>
> Overall, I think we're reaching the same consensus, i.e. to export this
> functionality to a generic function and shared between the both
> backends. I will make this change. I think the message used in the
> reftable backend along the lines of 'verifying symref target ...' is
> verbose yet generic enough, so I'll keep those messages.

That sounds good to me

Thanks

Phillip

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Phillip Wood]

Hi Karthik

On 05/05/2024 17:09, Karthik Nayak wrote:
> Phillip Wood <phillip.wood123@gmail.com> writes:
>> I'm slightly concerned that splitting out the update-ref changes means
>> we don't have any test coverage of the new code beyond the part that is
>> used by refs_create_symref()
>>
> 
> This is definitely true. But I also caught a bunch of edge cases this
> way because the tests which indirectly use 'refs_create_symref()' are
> quite intensive.

I forgot to say in my last mail that this is good to know. So it sounds 
like the only new code that isn't being exercised by the tests is the 
check for the old value?

Best Wishes

Phillip

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

Hello,


Phillip Wood <phillip.wood123@gmail.com> writes:

> Hi Karthik
>
> On 05/05/2024 17:09, Karthik Nayak wrote:
>> Phillip Wood <phillip.wood123@gmail.com> writes:
>>> I'm slightly concerned that splitting out the update-ref changes means
>>> we don't have any test coverage of the new code beyond the part that is
>>> used by refs_create_symref()
>>>
>>
>> This is definitely true. But I also caught a bunch of edge cases this
>> way because the tests which indirectly use 'refs_create_symref()' are
>> quite intensive.
>
> I forgot to say in my last mail that this is good to know. So it sounds
> like the only new code that isn't being exercised by the tests is the
> check for the old value?
>
> Best Wishes
>
> Phillip

That's correct. I think testing that _currently_ would require us to
probably expose and test via the unit testing library. I plan to follow
this patch series soon with the symref-* ones. While that's not the best
argument for not having full test coverage, I hope it is an OK state to
be since that path has no users as of this point.

Karthik

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

[Phillip Wood]

Hi Karthik
On 06/05/2024 12:22, Karthik Nayak wrote:
> 
> That's correct. I think testing that _currently_ would require us to
> probably expose and test via the unit testing library. I plan to follow
> this patch series soon with the symref-* ones. While that's not the best
> argument for not having full test coverage, I hope it is an OK state to
> be since that path has no users as of this point.

That seems reasonable - we have good test coverage for the code we're 
currently using and you're clearly committed to adding tests for the 
rest when you extend "git update-ref" to accommodate symrefs.

Thanks

Phillip

[PATCH v6 5/7] refs: use transaction in `refs_create_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function updates a symref to a given new
target. To do this, it uses a ref-backend specific function
`create_symref()`.

In the previous commits, we introduced symref support in transactions.
This means we can now use transactions to perform symref updates and
don't have to resort to `create_symref()`. Doing this allows us to
remove and cleanup `create_symref()`, which we will do in the following
commit.

Modify the expected error message for a test in
't/t0610-reftable-basics.sh', since the error is now thrown from
'refs.c'. This is because in transactional updates, F/D conflicts are
caught before we're in the reference backend.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                           | 24 +++++++++++++++++-------
 t/t0610-reftable-basics.sh       |  2 +-
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 3 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/refs.c b/refs.c
index acb153b6f8..195861b09b 100644
--- a/refs.c
+++ b/refs.c
@@ -2289,14 +2289,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index 178791e086..9e8d22bcbd 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
 	git init repo &&
 	test_commit -C repo A &&
 	cat >expect <<-EOF &&
-	error: unable to write symref for refs/heads: file/directory conflict
+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
 	EOF
 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
 	test_cmp expect err
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..067fd57290 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+	prepared
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	committed
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

[PATCH v6 6/7] refs: rename `refs_create_symref()` to `refs_update_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function is used to update/create a symref.
But it doesn't check the old target of the symref, if existing. It force
updates the symref. In this regard, the name `refs_create_symref()` is a
bit misleading. So let's rename it to `refs_update_symref()`. This is
akin to how 'git-update-ref(1)' also allows us to create apart from
update.

While we're here, rename the arguments in the function to clarify what
they actually signify and reduce confusion.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index dd3e3a7dc0..4491f7a20c 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 7c6c72536b..480202c517 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
diff --git a/refs.c b/refs.c
index 195861b09b..3645b805c1 100644
--- a/refs.c
+++ b/refs.c
@@ -2284,10 +2284,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2295,8 +2293,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2312,7 +2310,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
diff --git a/refs.h b/refs.h
index c7851bf587..71cc1c58e0 100644
--- a/refs.h
+++ b/refs.h
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 82bbf6e2e6..4651e4ced7 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
-- 
2.43.GIT

[PATCH v6 7/7] refs: remove `create_symref` and associated dead code

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

In the previous commits, we converted `refs_create_symref()` to utilize
transactions to perform symref updates. Earlier `refs_create_symref()`
used `create_symref()` to do the same.

We can now remove `create_symref()` and any code associated with it
which is no longer used. We remove `create_symref()` code from all the
reference backends and also remove it entirely from the `ref_storage_be`
struct.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/debug.c            | 13 -------
 refs/files-backend.c    | 67 --------------------------------
 refs/packed-backend.c   |  1 -
 refs/refs-internal.h    |  5 ---
 refs/reftable-backend.c | 86 -----------------------------------------
 5 files changed, 172 deletions(-)

diff --git a/refs/debug.c b/refs/debug.c
index c7531b17f0..8be316bb67 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index e1f0ca74c0..3ce260d07d 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1903,23 +1903,6 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)
 	return ret;
 }
 
-static void update_symref_reflog(struct files_ref_store *refs,
-				 struct ref_lock *lock, const char *refname,
-				 const char *target, const char *logmsg)
-{
-	struct strbuf err = STRBUF_INIT;
-	struct object_id new_oid;
-
-	if (logmsg &&
-	    refs_resolve_ref_unsafe(&refs->base, target,
-				    RESOLVE_REF_READING, &new_oid, NULL) &&
-	    files_log_ref_write(refs, refname, &lock->old_oid,
-				&new_oid, logmsg, 0, &err)) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-	}
-}
-
 static int create_symref_lock(struct files_ref_store *refs,
 			      struct ref_lock *lock, const char *refname,
 			      const char *target, struct strbuf *err)
@@ -1939,55 +1922,6 @@ static int create_symref_lock(struct files_ref_store *refs,
 	return 0;
 }
 
-static int create_and_commit_symref(struct files_ref_store *refs,
-				    struct ref_lock *lock, const char *refname,
-				    const char *target, const char *logmsg)
-{
-	struct strbuf err = STRBUF_INIT;
-	int ret;
-
-	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-		return 0;
-	}
-
-	ret = create_symref_lock(refs, lock, refname, target, &err);
-	if (!ret) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-
-		if (commit_ref(lock) < 0)
-			return error("unable to write symref for %s: %s", refname,
-				     strerror(errno));
-	} else {
-		return error("%s", err.buf);
-	}
-
-	return ret;
-}
-
-static int files_create_symref(struct ref_store *ref_store,
-			       const char *refname, const char *target,
-			       const char *logmsg)
-{
-	struct files_ref_store *refs =
-		files_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct strbuf err = STRBUF_INIT;
-	struct ref_lock *lock;
-	int ret;
-
-	lock = lock_ref_oid_basic(refs, refname, &err);
-	if (!lock) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-		return -1;
-	}
-
-	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
-
-	unlock_ref(lock);
-	return ret;
-}
-
 static int files_reflog_exists(struct ref_store *ref_store,
 			       const char *refname)
 {
@@ -3413,7 +3347,6 @@ struct ref_storage_be refs_be_files = {
 	.initial_transaction_commit = files_initial_transaction_commit,
 
 	.pack_refs = files_pack_refs,
-	.create_symref = files_create_symref,
 	.rename_ref = files_rename_ref,
 	.copy_ref = files_copy_ref,
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 4e826c05ff..a937e7dbfc 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 03eda70ea4..0c620d4bce 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 54c4d8b771..999b5bca21 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1275,91 +1275,6 @@ struct write_create_symref_arg {
 	const char *logmsg;
 };
 
-static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
-{
-	struct write_create_symref_arg *create = cb_data;
-	uint64_t ts = reftable_stack_next_update_index(create->stack);
-	struct reftable_ref_record ref = {
-		.refname = (char *)create->refname,
-		.value_type = REFTABLE_REF_SYMREF,
-		.value.symref = (char *)create->target,
-		.update_index = ts,
-	};
-	struct reftable_log_record log = {0};
-	struct object_id new_oid;
-	struct object_id old_oid;
-	int ret;
-
-	reftable_writer_set_limits(writer, ts, ts);
-
-	ret = reftable_writer_add_ref(writer, &ref);
-	if (ret)
-		return ret;
-
-	/*
-	 * Note that it is important to try and resolve the reference before we
-	 * write the log entry. This is because `should_write_log()` will munge
-	 * `core.logAllRefUpdates`, which is undesirable when we create a new
-	 * repository because it would be written into the config. As HEAD will
-	 * not resolve for new repositories this ordering will ensure that this
-	 * never happens.
-	 */
-	if (!create->logmsg ||
-	    !refs_resolve_ref_unsafe(&create->refs->base, create->target,
-				     RESOLVE_REF_READING, &new_oid, NULL) ||
-	    !should_write_log(&create->refs->base, create->refname))
-		return 0;
-
-	fill_reftable_log_record(&log);
-	log.refname = xstrdup(create->refname);
-	log.update_index = ts;
-	log.value.update.message = xstrndup(create->logmsg,
-					    create->refs->write_options.block_size / 2);
-	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
-	if (refs_resolve_ref_unsafe(&create->refs->base, create->refname,
-				    RESOLVE_REF_READING, &old_oid, NULL))
-		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
-
-	ret = reftable_writer_add_log(writer, &log);
-	reftable_log_record_release(&log);
-	return ret;
-}
-
-static int reftable_be_create_symref(struct ref_store *ref_store,
-				     const char *refname,
-				     const char *target,
-				     const char *logmsg)
-{
-	struct reftable_ref_store *refs =
-		reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct reftable_stack *stack = stack_for(refs, refname, &refname);
-	struct write_create_symref_arg arg = {
-		.refs = refs,
-		.stack = stack,
-		.refname = refname,
-		.target = target,
-		.logmsg = logmsg,
-	};
-	int ret;
-
-	ret = refs->err;
-	if (ret < 0)
-		goto done;
-
-	ret = reftable_stack_reload(stack);
-	if (ret)
-		goto done;
-
-	ret = reftable_stack_add(stack, &write_create_symref_table, &arg);
-
-done:
-	assert(ret != REFTABLE_API_ERROR);
-	if (ret)
-		error("unable to write symref for %s: %s", refname,
-		      reftable_error_str(ret));
-	return ret;
-}
-
 struct write_copy_arg {
 	struct reftable_ref_store *refs;
 	struct reftable_stack *stack;
@@ -2267,7 +2182,6 @@ struct ref_storage_be refs_be_reftable = {
 	.initial_transaction_commit = reftable_be_initial_transaction_commit,
 
 	.pack_refs = reftable_be_pack_refs,
-	.create_symref = reftable_be_create_symref,
 	.rename_ref = reftable_be_rename_ref,
 	.copy_ref = reftable_be_copy_ref,
 
-- 
2.43.GIT

Re: [PATCH v6 7/7] refs: remove `create_symref` and associated dead code

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> From: Karthik Nayak <karthik.188@gmail.com>
>
> In the previous commits, we converted `refs_create_symref()` to utilize
> transactions to perform symref updates. Earlier `refs_create_symref()`
> used `create_symref()` to do the same.
>
> We can now remove `create_symref()` and any code associated with it
> which is no longer used. We remove `create_symref()` code from all the
> reference backends and also remove it entirely from the `ref_storage_be`
> struct.
>
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---

This has serious semantic conflicts with in-flight topics.  I think
I resolved them all correctly while merging it in 'seen', but please
double check the result after I push it out.

This comment equally applies to the "force all callers to use
get_main_ref_store() on the_repository and remove functions that
implicitly used the main ref store of the_repository" topic by
Patrick, but we really should devise a bit more smoother way to cope
with out of tree and in-flight topics.  For example, as the new
refs_update_symref() function works exactly like the existing
refs_create_symref() function, after renaming all the in-base (i.e.,
in-tree at the point this topic forks from) users to call the new
function, instead of just removing the original one, it would have
been nice to guide authors of other in-flight topics by (1) causing
build failure and at the same time (2) telling them what they need
to do to adjust to the new world order.  This patch does only (1)
but does a poor job for (2).  We may want to establish a better
convention than just outright removing and breaking others' topics.

Thanks.

Re: [PATCH v6 7/7] refs: remove `create_symref` and associated dead code

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1944 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> From: Karthik Nayak <karthik.188@gmail.com>
>>
>> In the previous commits, we converted `refs_create_symref()` to utilize
>> transactions to perform symref updates. Earlier `refs_create_symref()`
>> used `create_symref()` to do the same.
>>
>> We can now remove `create_symref()` and any code associated with it
>> which is no longer used. We remove `create_symref()` code from all the
>> reference backends and also remove it entirely from the `ref_storage_be`
>> struct.
>>
>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>> ---
>
> This has serious semantic conflicts with in-flight topics.  I think
> I resolved them all correctly while merging it in 'seen', but please
> double check the result after I push it out.
>

I had a look at the commit pushed to 'seen', looked good to me. Thanks.

> This comment equally applies to the "force all callers to use
> get_main_ref_store() on the_repository and remove functions that
> implicitly used the main ref store of the_repository" topic by
> Patrick, but we really should devise a bit more smoother way to cope
> with out of tree and in-flight topics.  For example, as the new
> refs_update_symref() function works exactly like the existing
> refs_create_symref() function, after renaming all the in-base (i.e.,
> in-tree at the point this topic forks from) users to call the new
> function, instead of just removing the original one, it would have
> been nice to guide authors of other in-flight topics by (1) causing
> build failure and at the same time (2) telling them what they need
> to do to adjust to the new world order.  This patch does only (1)
> but does a poor job for (2).  We may want to establish a better
> convention than just outright removing and breaking others' topics.
>
> Thanks.

Just so I can do better next time, what is your suggestion for doing (2)
better?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v6 0/7] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> Previous versions:
> V1: https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com/
> V2: https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com/
> V3: https://lore.kernel.org/git/20240423212818.574123-1-knayak@gitlab.com/
> V4: https://lore.kernel.org/r/20240426152449.228860-1-knayak@gitlab.com
> V5: https://lore.kernel.org/r/20240501202229.2695774-1-knayak@gitlab.com
>
> Changes over v5 are:
> - More user friendly error messages.
> - `create_symref_lock` now writes to an err buf, instead of directly to stderr.
> - Refactor code to make it easier to read around logical operations.

All changes I saw in the range-diff and diff output with v5 looked
sensible improvements.

Will wait for comments and then hopefully we can mark it for 'next'
sometime next week, if nothing happens.

Thanks.

Re: [PATCH v6 0/7] refs: add support for transactional symref updates

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2265 bytes --]

On Fri, May 03, 2024 at 02:41:08PM +0200, Karthik Nayak wrote:
> From: Karthik Nayak <karthik.188@gmail.com>
> 
> The patch series takes over from the existing patch series, wherein we
> introduced symref-* commands to git-update-ref. Since there was a ton of
> discussions on the UX of the patch series and its application, I thought it
> would be best to shorten the series and split it into multiple smaller series.
> 
> This series adds transactional support for symrefs in the reference db. Then
> we switch refs_create_symref() to start using transactions for symref updates.
> This allows us to deprecate the create_symref code in the ref_storage_be
> interface and remove all associated code which is no longer used.
> 
> The split was primarily done so we can merge the non-user facing parts of the
> previous series. While pertaining the user facing components into another set
> of patches wherein deeper discussion on the UX can be held without worrying
> about the internal implementation. Also by using this new functionality in a
> pre-existing command, we can leverage the existing tests to catch any
> inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for
> dangling symrefs, which I've modified my patch to do the same.
> 
> We also modify the reference transaction hook to support symrefs. For any symref
> update the reference transaction hook will output the value with a 'ref:' prefix.
> 
> Previous versions:
> V1: https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com/
> V2: https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com/
> V3: https://lore.kernel.org/git/20240423212818.574123-1-knayak@gitlab.com/
> V4: https://lore.kernel.org/r/20240426152449.228860-1-knayak@gitlab.com
> V5: https://lore.kernel.org/r/20240501202229.2695774-1-knayak@gitlab.com
> 
> Changes over v5 are:
> - More user friendly error messages.
> - `create_symref_lock` now writes to an err buf, instead of directly to stderr.
> - Refactor code to make it easier to read around logical operations.
> - Cleanup commit message and fix typos.

Just for the record, I don't really have much to add to this series
besides what has already been said.

Thanks!

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v7 0/8] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The patch series takes over from the existing patch series, wherein we
introduced symref-* commands to git-update-ref. Since there was a ton of
discussions on the UX of the patch series and its application, I thought it
would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then
we switch refs_create_symref() to start using transactions for symref updates.
This allows us to deprecate the create_symref code in the ref_storage_be
interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the
previous series. While pertaining the user facing components into another set
of patches wherein deeper discussion on the UX can be held without worrying
about the internal implementation. Also by using this new functionality in a
pre-existing command, we can leverage the existing tests to catch any
inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for
dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref
update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com
V2: https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com
V3: https://lore.kernel.org/git/20240423212818.574123-1-knayak@gitlab.com
V4: https://lore.kernel.org/r/20240426152449.228860-1-knayak@gitlab.com
V5: https://lore.kernel.org/r/20240501202229.2695774-1-knayak@gitlab.com
V6: https://lore.kernel.org/r/20240503124115.252413-1-knayak@gitlab.com

Thanks for all the reviews!

Changes since v6:
* Made the check for old/new oid & target more stricter by removing
the exception for null oid's.
* Extracted `ref_update_check_old_target` and `original_update_refname`
to refs.c
* ^This allowed us to generalize the error messages for non-matching
old_target values between files and reftable backend.
* Better line wrapping in code.
* Fixed some grammar in commit messages.

Junio, I've rebased my patches on top of, not sure if there is something else I
missed here:
- ps/ci-python-2-deprecation (2a686e00d2)
- ps/reftable-write-options (f5545b173b)
- ps/pseudo-ref-terminology (ad302a68fd)

Range diff against v6:

1:  a354190905 ! 1:  2a09350661 refs: accept symref values in `ref_transaction_update()`
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      	if (transaction->state != REF_TRANSACTION_OPEN)
      		BUG("update called for transaction that is not open");
      
    -+	if (old_oid && !is_null_oid(old_oid) && old_target)
    ++	if (old_oid && old_target)
     +		BUG("only one of old_oid and old_target should be non NULL");
    -+	if (new_oid && !is_null_oid(new_oid) && new_target)
    ++	if (new_oid && new_target)
     +		BUG("only one of new_oid and new_target should be non NULL");
     +
      	FLEX_ALLOC_STR(update, refname, refname);
2:  0d9c5b9804 = 2:  9e76b768b9 files-backend: extract out `create_symref_lock()`
3:  e0219ffd31 = 3:  c1d1c5a651 refs: support symrefs in 'reference-transaction' hook
-:  ---------- > 4:  d7d455c9b6 refs: move `original_update_refname` to 'refs.c'
4:  b22c59c722 ! 5:  1c00cd69c6 refs: add support for transactional symref updates
    @@ Commit message
         updates. While this is exposed to users via 'git-update-ref' and its
         '--stdin' mode, it is also used internally within various commands.
     
    -    However, we never supported transactional updates of symrefs. Let's add
    -    support for symrefs in both the 'files' and the 'reftable' backend.
    +    However, we do not support transactional updates of symrefs. This commit
    +    adds support for symrefs in both the 'files' and the 'reftable' backend.
     
         Here, we add and use `ref_update_has_null_new_value()`, a helper
         function which is used to check if there is a new_value in a reference
         update. The new value could either be a symref target `new_target` or a
         OID `new_oid`.
     
    -    With this, now transactional updates (verify, create, delete, update)
    -    can be used for:
    +    We also add another common function `ref_update_check_old_target` which
    +    will be used to check if the update's old_target corresponds to a
    +    reference's current target.
    +
    +    Now transactional updates (verify, create, delete, update) can be used
    +    for:
         - regular refs
         - symbolic refs
         - conversion of regular to symbolic refs and vice versa
    @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      
      	ref_transaction_add_update(transaction, refname, flags,
      				   new_oid, old_oid, new_target,
    -@@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
    - {
    - 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
    +@@ refs.c: const char *ref_update_original_update_refname(struct ref_update *update)
    + 	return update->refname;
      }
    -+
    + 
     +int ref_update_has_null_new_value(struct ref_update *update)
     +{
     +	return !update->new_target && is_null_oid(&update->new_oid);
     +}
    -
    - ## refs/files-backend.c ##
    -@@ refs/files-backend.c: static const char *original_update_refname(struct ref_update *update)
    - 	return update->refname;
    - }
    - 
    -+/*
    -+ * Check whether the old_target values stored in update are consistent
    -+ * with current_target, which is the symbolic reference's current value.
    -+ * If everything is OK, return 0; otherwise, write an error message to
    -+ * err and return -1.
    -+ */
    -+static int check_old_target(struct ref_update *update,
    -+			    const char *current_target,
    -+			    struct strbuf *err)
    ++
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err)
     +{
     +	if (!update->old_target)
     +		BUG("called without old_target set");
     +
    -+	if (!strcmp(update->old_target, current_target))
    ++	if (!strcmp(referent, update->old_target))
     +		return 0;
     +
    -+	if (!strcmp(current_target, ""))
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++	if (!strcmp(referent, ""))
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "reference is missing but expected %s",
    -+			    original_update_refname(update),
    ++			    ref_update_original_update_refname(update),
     +			    update->old_target);
     +	else
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "is at %s but expected %s",
    -+			    original_update_refname(update),
    -+			    current_target, update->old_target);
    -+
    ++			    ref_update_original_update_refname(update),
    ++			    referent, update->old_target);
     +	return -1;
     +}
    -+
    - /*
    -  * Check whether the REF_HAVE_OLD and old_oid values stored in update
    -  * are consistent with oid, which is the reference's current value. If
    +
    + ## refs/files-backend.c ##
    +@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
    + 
    + 	new_update = ref_transaction_add_update(
    + 			transaction, referent, new_flags,
    +-			&update->new_oid, &update->old_oid,
    +-			NULL, NULL, update->msg);
    ++			update->new_target ? NULL : &update->new_oid,
    ++			update->old_target ? NULL : &update->old_oid,
    ++			update->new_target, update->old_target, update->msg);
    + 
    + 	new_update->parent_update = update;
    + 
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
      
      	files_assert_main_repository(refs, "lock_ref_for_update");
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +			}
     +
     +			if (update->old_target) {
    -+				if (check_old_target(update, referent.buf, err)) {
    ++				if (ref_update_check_old_target(referent.buf, update, err)) {
     +					ret = TRANSACTION_GENERIC_ERROR;
     +					goto out;
     +				}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +		 * be set for symrefs, but we're strict about its usage.
     +		 */
     +		if (update->old_target) {
    -+			if (check_old_target(update, referent.buf, err)) {
    ++			if (ref_update_check_old_target(referent.buf, update, err)) {
     +				ret = TRANSACTION_GENERIC_ERROR;
     +				goto out;
     +			}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     -	    !(update->flags & REF_DELETING) &&
     -	    !(update->flags & REF_LOG_ONLY)) {
     +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
    -+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
    ++		if (create_symref_lock(refs, lock, update->refname,
    ++				       update->new_target, err)) {
     +			ret = TRANSACTION_GENERIC_ERROR;
     +			goto out;
     +		}
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
      			if (commit_ref(lock)) {
     
      ## refs/refs-internal.h ##
    -@@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct repository *repo,
    +@@ refs/refs-internal.h: struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
       */
    - struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
    + const char *ref_update_original_update_refname(struct ref_update *update);
      
     +/*
     + * Helper function to check if the new value is null, this
    @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
     + * ref or a symbolic ref.
     + */
     +int ref_update_has_null_new_value(struct ref_update *update);
    ++
    ++/*
    ++ * Check whether the old_target values stored in update are consistent
    ++ * with the referent, which is the symbolic reference's current value.
    ++ * If everything is OK, return 0; otherwise, write an error message to
    ++ * err and return -1.
    ++ */
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err);
     +
      #endif /* REFS_REFS_INTERNAL_H */
     
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      		 */
     -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
     +		if (u->old_target) {
    -+			if (strcmp(referent.buf, u->old_target)) {
    -+				if (!strcmp(referent.buf, ""))
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "reference is missing but expected %s",
    -+						    original_update_refname(u),
    -+						    u->old_target);
    -+				else
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "is at %s but expected %s",
    -+						    original_update_refname(u),
    -+						    referent.buf, u->old_target);
    ++			if (ref_update_check_old_target(referent.buf, u, err)) {
     +				ret = -1;
     +				goto done;
     +			}
     +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
      			if (is_null_oid(&u->old_oid))
      				strbuf_addf(err, _("cannot lock ref '%s': "
    - 					    "reference already exists"),
    + 						   "reference already exists"),
     @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
      		 * - `core.logAllRefUpdates` tells us to create the reflog for
      		 *   the given ref.
      		 */
     -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
    -+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
    ++		if ((u->flags & REF_HAVE_NEW) &&
    ++		    !(u->type & REF_ISSYMREF) &&
    ++		    ref_update_has_null_new_value(u)) {
      			struct reftable_log_record log = {0};
      			struct reftable_iterator it = {0};
      
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     -			log = &logs[logs_nr++];
     -			memset(log, 0, sizeof(*log));
     -
    --			fill_reftable_log_record(log);
    +-			fill_reftable_log_record(log, &committer_ident);
     -			log->update_index = ts;
     -			log->refname = xstrdup(u->refname);
     -			memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     +				log = &logs[logs_nr++];
     +				memset(log, 0, sizeof(*log));
     +
    -+				fill_reftable_log_record(log);
    ++				fill_reftable_log_record(log, &committer_ident);
     +				log->update_index = ts;
     +				log->refname = xstrdup(u->refname);
    -+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
    -+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.new_hash,
    ++				       u->new_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.old_hash,
    ++				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
     +				log->value.update.message =
     +					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
     +			}
5:  636bf5ce98 ! 6:  027106cfe6 refs: use transaction in `refs_create_symref()`
    @@ refs.c: int refs_create_symref(struct ref_store *refs,
      
      int create_symref(const char *ref_target, const char *refs_heads_master,
     
    - ## t/t0610-reftable-basics.sh ##
    -@@ t/t0610-reftable-basics.sh: test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
    - 	git init repo &&
    - 	test_commit -C repo A &&
    - 	cat >expect <<-EOF &&
    --	error: unable to write symref for refs/heads: file/directory conflict
    -+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
    - 	EOF
    - 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
    - 	test_cmp expect err
    -
      ## t/t1416-ref-transaction-hooks.sh ##
     @@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
      	test_cmp expect target-repo.git/actual
6:  07fb23374f = 7:  6939406143 refs: rename `refs_create_symref()` to `refs_update_symref()`
7:  5c05813bcc ! 8:  bc787b903b refs: remove `create_symref` and associated dead code
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -		.value.symref = (char *)create->target,
     -		.update_index = ts,
     -	};
    +-	struct ident_split committer_ident = {0};
     -	struct reftable_log_record log = {0};
     -	struct object_id new_oid;
     -	struct object_id old_oid;
    +-	const char *committer_info;
     -	int ret;
     -
     -	reftable_writer_set_limits(writer, ts, ts);
     -
    +-	ret = refs_verify_refname_available(&create->refs->base, create->refname,
    +-					    NULL, NULL, create->err);
    +-	if (ret < 0)
    +-		return ret;
    +-
     -	ret = reftable_writer_add_ref(writer, &ref);
     -	if (ret)
     -		return ret;
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -	    !should_write_log(&create->refs->base, create->refname))
     -		return 0;
     -
    --	fill_reftable_log_record(&log);
    +-	committer_info = git_committer_info(0);
    +-	if (split_ident_line(&committer_ident, committer_info, strlen(committer_info)))
    +-		BUG("failed splitting committer info");
    +-
    +-	fill_reftable_log_record(&log, &committer_ident);
     -	log.refname = xstrdup(create->refname);
     -	log.update_index = ts;
     -	log.value.update.message = xstrndup(create->logmsg,
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -	struct reftable_ref_store *refs =
     -		reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_symref");
     -	struct reftable_stack *stack = stack_for(refs, refname, &refname);
    +-	struct strbuf err = STRBUF_INIT;
     -	struct write_create_symref_arg arg = {
     -		.refs = refs,
     -		.stack = stack,
     -		.refname = refname,
     -		.target = target,
     -		.logmsg = logmsg,
    +-		.err = &err,
     -	};
     -	int ret;
     -
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -
     -done:
     -	assert(ret != REFTABLE_API_ERROR);
    --	if (ret)
    --		error("unable to write symref for %s: %s", refname,
    --		      reftable_error_str(ret));
    +-	if (ret) {
    +-		if (err.len)
    +-			error("%s", err.buf);
    +-		else
    +-			error("unable to write symref for %s: %s", refname,
    +-			      reftable_error_str(ret));
    +-	}
    +-
    +-	strbuf_release(&err);
     -	return ret;
     -}
     -

Karthik Nayak (8):
  refs: accept symref values in `ref_transaction_update()`
  files-backend: extract out `create_symref_lock()`
  refs: support symrefs in 'reference-transaction' hook
  refs: move `original_update_refname` to 'refs.c'
  refs: add support for transactional symref updates
  refs: use transaction in `refs_create_symref()`
  refs: rename `refs_create_symref()` to `refs_update_symref()`
  refs: remove `create_symref` and associated dead code

 Documentation/githooks.txt       |  14 ++-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           | 119 ++++++++++++++----
 refs.h                           |  20 ++-
 refs/debug.c                     |  13 --
 refs/files-backend.c             | 208 +++++++++++++++++--------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  40 +++++-
 refs/reftable-backend.c          | 202 +++++++++---------------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 21 files changed, 373 insertions(+), 298 deletions(-)

-- 
2.43.GIT

[PATCH v7 1/8] refs: accept symref values in `ref_transaction_update()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `ref_transaction_update()` obtains ref information and
flags to create a `ref_update` and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the
old and new ref targets and process it. This commit adds the required
parameters to the function and modifies all call sites.

The two parameters added are `new_target` and `old_target`. The
`new_target` is used to denote what the reference should point to when
the transaction is applied. Some functions allow this parameter to be
NULL, meaning that the reference is not changed.

The `old_target` denotes the value the reference must have before the
update. Some functions allow this parameter to be NULL, meaning that the
old value of the reference is not checked.

We also update the internal function `ref_transaction_add_update()`
similarly to take the two new parameters.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)

diff --git a/branch.c b/branch.c
index e4a738fc7b..48af4c3ceb 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index dc5a9d32dd..297dfb91a1 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5857d860db..66840b7c5b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e8d7df14b6..b150ef39a8 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..7690687b0e 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 9a33cb50b4..40a65fdebc 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index e46afbc46d..21fdbf6ac8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
diff --git a/refs.c b/refs.c
index 7c3c7465a4..d7df137904 100644
--- a/refs.c
+++ b/refs.c
@@ -1268,6 +1268,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1275,6 +1276,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1293,6 +1299,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1327,7 +1335,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1342,7 +1351,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1355,7 +1365,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1368,6 +1379,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1382,8 +1394,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index 8255989e7e..9abbfd35a2 100644
--- a/refs.h
+++ b/refs.h
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ea927c516d..7e432cf026 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1197,7 +1197,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1291,7 +1291,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2308,7 +2308,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2371,7 +2371,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2762,7 +2762,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3047,7 +3047,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..108f4ec419 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 10ee92e1f7..f48ab5ad22 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -870,7 +870,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -949,7 +949,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 88de4dc20f..61e007d85f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index c0fd632d92..1b3df43906 100644
--- a/walker.c
+++ b/walker.c
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
-- 
2.43.GIT

[PATCH v7 2/8] files-backend: extract out `create_symref_lock()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked()` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Extract the early half of `create_symref_locked()` into a new helper
function `create_symref_lock()`. Because the name of the new function is
too similar to the original, rename the original to
`create_and_commit_symref()` to avoid confusion.

The new function `create_symref_locked()` can be used to create the
symref lock in a separate step from that of committing it. This allows
to add transactional support for symrefs, where the lock would be
created in the preparation step and the lock would be committed in the
finish step.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 51 ++++++++++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 14 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7e432cf026..40cc715ea7 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1919,27 +1919,49 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target, struct strbuf *err)
 {
+	if (!fdopen_lock_file(&lock->lk, "w")) {
+		strbuf_addf(err, "unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
+		strbuf_addf(err, "unable to write to %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	struct strbuf err = STRBUF_INIT;
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target, &err);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	} else {
+		return error("%s", err.buf);
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1959,7 +1981,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

[PATCH v7 3/8] refs: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In a previous commit, we added the `old_target` and
`new_target` fields to the `reference_transaction_update()`. In
following commits we'll also add the code to handle symref's in the
reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying
the current format:
    <old-oid> SP <new-oid> SP <ref-name> LF
to be be:
    <old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same.
But when either 'old-value' or 'new-value' is a symref, we print the ref
as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction'
hook's documentation always stated that support for symbolic references
may be added in the future.

We do not add any tests in this commit since there is no git command
which activates this flow, in an upcoming commit, we'll start using
transaction based symref updates as the default, we'll add tests there
for the hook too.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 20 ++++++++++++++++----
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ee9b92c90d..06e997131b 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index d7df137904..2bb3e09425 100644
--- a/refs.c
+++ b/refs.c
@@ -2366,10 +2366,22 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (!(update->flags & REF_HAVE_OLD))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (!(update->flags & REF_HAVE_NEW))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
-- 
2.43.GIT

[PATCH v7 4/8] refs: move `original_update_refname` to 'refs.c'

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The files backend and the reftable backend implement
`original_update_refname` to obtain the original refname of the update.
Move it out to 'refs.c' and only expose it internally to the refs
library. This will be used in an upcoming commit to also introduce
another common functionality for the two backends.

We also rename the function to `ref_update_original_update_refname` to
keep it consistent with the upcoming other 'ref_update_*' functions
that'll be introduced.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                  |  9 +++++++++
 refs/files-backend.c    | 21 +++++----------------
 refs/refs-internal.h    |  5 +++++
 refs/reftable-backend.c | 24 +++++++-----------------
 4 files changed, 26 insertions(+), 33 deletions(-)

diff --git a/refs.c b/refs.c
index 2bb3e09425..d65e4cc41e 100644
--- a/refs.c
+++ b/refs.c
@@ -2830,3 +2830,12 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+const char *ref_update_original_update_refname(struct ref_update *update)
+{
+	while (update->parent_update)
+		update = update->parent_update;
+
+	return update->refname;
+}
+
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 40cc715ea7..64d2a50e97 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2422,17 +2422,6 @@ static int split_symref_update(struct ref_update *update,
 	return 0;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-
-	return update->refname;
-}
-
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2449,16 +2438,16 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
 	if (is_null_oid(&update->old_oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference already exists",
-			    original_update_refname(update));
+			    ref_update_original_update_refname(update));
 	else if (is_null_oid(oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference is missing but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(&update->old_oid));
 	else
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "is at %s but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(oid),
 			    oid_to_hex(&update->old_oid));
 
@@ -2512,7 +2501,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 		reason = strbuf_detach(err, NULL);
 		strbuf_addf(err, "cannot lock ref '%s': %s",
-			    original_update_refname(update), reason);
+			    ref_update_original_update_refname(update), reason);
 		free(reason);
 		goto out;
 	}
@@ -2532,7 +2521,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 				if (update->flags & REF_HAVE_OLD) {
 					strbuf_addf(err, "cannot lock ref '%s': "
 						    "error reading reference",
-						    original_update_refname(update));
+						    ref_update_original_update_refname(update));
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 108f4ec419..617b93a6c8 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -749,4 +749,9 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Return the refname under which update was originally requested.
+ */
+const char *ref_update_original_update_refname(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index f48ab5ad22..0988f59ed4 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -622,16 +622,6 @@ static int reftable_be_read_symbolic_ref(struct ref_store *ref_store,
 	return ret;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-	return update->refname;
-}
-
 struct reftable_transaction_update {
 	struct ref_update *update;
 	struct object_id current_oid;
@@ -910,7 +900,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			/* The reference does not exist, but we expected it to. */
 			strbuf_addf(err, _("cannot lock ref '%s': "
 				    "unable to resolve reference '%s'"),
-				    original_update_refname(u), u->refname);
+				    ref_update_original_update_refname(u), u->refname);
 			ret = -1;
 			goto done;
 		}
@@ -982,17 +972,17 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference already exists"),
-					    original_update_refname(u));
+						   "reference already exists"),
+					    ref_update_original_update_refname(u));
 			else if (is_null_oid(&current_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference is missing but expected %s"),
-					    original_update_refname(u),
+						   "reference is missing but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&u->old_oid));
 			else
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "is at %s but expected %s"),
-					    original_update_refname(u),
+						   "is at %s but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&current_oid),
 					    oid_to_hex(&u->old_oid));
 			ret = -1;
-- 
2.43.GIT

[PATCH v7 5/8] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The reference backends currently support transactional reference
updates. While this is exposed to users via 'git-update-ref' and its
'--stdin' mode, it is also used internally within various commands.

However, we do not support transactional updates of symrefs. This commit
adds support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use `ref_update_has_null_new_value()`, a helper
function which is used to check if there is a new_value in a reference
update. The new value could either be a symref target `new_target` or a
OID `new_oid`.

We also add another common function `ref_update_check_old_target` which
will be used to check if the update's old_target corresponds to a
reference's current target.

Now transactional updates (verify, create, delete, update) can be used
for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in
'git-update-ref' in the future.

Note that a dangling symref update does not record a new reflog entry,
which is unchanged before and after this commit.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                  |  36 ++++++++++++-
 refs/files-backend.c    | 113 ++++++++++++++++++++++++++++++++--------
 refs/refs-internal.h    |  16 ++++++
 refs/reftable-backend.c |  71 +++++++++++++++++++------
 4 files changed, 196 insertions(+), 40 deletions(-)

diff --git a/refs.c b/refs.c
index d65e4cc41e..f3d6b3b8e3 100644
--- a/refs.c
+++ b/refs.c
@@ -1257,6 +1257,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1287,10 +1289,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	update->new_target = xstrdup_or_null(new_target);
+	update->old_target = xstrdup_or_null(old_target);
+	if ((flags & REF_HAVE_NEW) && new_oid)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if ((flags & REF_HAVE_OLD) && old_oid)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1333,6 +1338,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2839,3 +2845,29 @@ const char *ref_update_original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+int ref_update_has_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
+
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(referent, update->old_target))
+		return 0;
+
+	if (!strcmp(referent, ""))
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "reference is missing but expected %s",
+			    ref_update_original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "is at %s but expected %s",
+			    ref_update_original_update_refname(update),
+			    referent, update->old_target);
+	return -1;
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 64d2a50e97..1d4650b7cb 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2393,8 +2393,9 @@ static int split_symref_update(struct ref_update *update,
 
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
-			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target ? NULL : &update->new_oid,
+			update->old_target ? NULL : &update->old_oid,
+			update->new_target, update->old_target, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2482,7 +2483,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
@@ -2525,7 +2526,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
-			} else if (check_old_oid(update, &lock->old_oid, err)) {
+			}
+
+			if (update->old_target) {
+				if (ref_update_check_old_target(referent.buf, update, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}
+			} else if  (check_old_oid(update, &lock->old_oid, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto out;
 			}
@@ -2546,7 +2554,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 	} else {
 		struct ref_update *parent_update;
 
-		if (check_old_oid(update, &lock->old_oid, err)) {
+		/*
+		 * Even if the ref is a regular ref, if `old_target` is set, we
+		 * check the referent value. Ideally `old_target` should only
+		 * be set for symrefs, but we're strict about its usage.
+		 */
+		if (update->old_target) {
+			if (ref_update_check_old_target(referent.buf, update, err)) {
+				ret = TRANSACTION_GENERIC_ERROR;
+				goto out;
+			}
+		} else if  (check_old_oid(update, &lock->old_oid, err)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
 		}
@@ -2564,9 +2582,28 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
+		if (create_symref_lock(refs, lock, update->refname,
+				       update->new_target, err)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		update->flags |= REF_NEEDS_COMMIT;
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
@@ -2829,6 +2866,43 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 	return ret;
 }
 
+static int parse_and_write_reflog(struct files_ref_store *refs,
+				  struct ref_update *update,
+				  struct ref_lock *lock,
+				  struct strbuf *err)
+{
+	if (update->new_target) {
+		/*
+		 * We want to get the resolved OID for the target, to ensure
+		 * that the correct value is added to the reflog.
+		 */
+		if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
+					     RESOLVE_REF_READING,
+					     &update->new_oid, NULL)) {
+			/*
+			 * TODO: currently we skip creating reflogs for dangling
+			 * symref updates. It would be nice to capture this as
+			 * zero oid updates however.
+			 */
+			return 0;
+		}
+	}
+
+	if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid,
+				&update->new_oid, update->msg, update->flags, err)) {
+		char *old_msg = strbuf_detach(err, NULL);
+
+		strbuf_addf(err, "cannot update the ref '%s': %s",
+			    lock->ref_name, old_msg);
+		free(old_msg);
+		unlock_ref(lock);
+		update->backend_data = NULL;
+		return -1;
+	}
+
+	return 0;
+}
+
 static int files_transaction_finish(struct ref_store *ref_store,
 				    struct ref_transaction *transaction,
 				    struct strbuf *err)
@@ -2859,23 +2933,20 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
-			if (files_log_ref_write(refs,
-						lock->ref_name,
-						&lock->old_oid,
-						&update->new_oid,
-						update->msg, update->flags,
-						err)) {
-				char *old_msg = strbuf_detach(err, NULL);
-
-				strbuf_addf(err, "cannot update the ref '%s': %s",
-					    lock->ref_name, old_msg);
-				free(old_msg);
-				unlock_ref(lock);
-				update->backend_data = NULL;
+			if (parse_and_write_reflog(refs, update, lock, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next update. If not, we try and create a regular symref.
+		 */
+		if (update->new_target && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->new_target))
+				continue;
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 617b93a6c8..819157256e 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -754,4 +754,20 @@ struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
  */
 const char *ref_update_original_update_refname(struct ref_update *update);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_has_null_new_value(struct ref_update *update);
+
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with the referent, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 0988f59ed4..4817dc2f0b 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -887,7 +887,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -938,8 +938,10 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 * intertwined with the locking in files-backend.c.
 				 */
 				new_update = ref_transaction_add_update(
-						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+					transaction, referent.buf, new_flags,
+					&u->new_oid, &u->old_oid, u->new_target,
+					u->old_target, u->msg);
+
 				new_update->parent_update = u;
 
 				/*
@@ -969,7 +971,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (ref_update_check_old_target(referent.buf, u, err)) {
+				ret = -1;
+				goto done;
+			}
+		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 						   "reference already exists"),
@@ -1080,7 +1087,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if ((u->flags & REF_HAVE_NEW) &&
+		    !(u->type & REF_ISSYMREF) &&
+		    ref_update_has_null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1121,24 +1130,52 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			   (u->flags & REF_FORCE_CREATE_REFLOG ||
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
+			int create_reflog = 1;
+
+			if (u->new_target) {
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
+							     RESOLVE_REF_READING, &u->new_oid, NULL)) {
+					/*
+					 * TODO: currently we skip creating reflogs for dangling
+					 * symref updates. It would be nice to capture this as
+					 * zero oid updates however.
+					 */
+					create_reflog = 0;
+				}
+			}
 
-			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
-			log = &logs[logs_nr++];
-			memset(log, 0, sizeof(*log));
-
-			fill_reftable_log_record(log, &committer_ident);
-			log->update_index = ts;
-			log->refname = xstrdup(u->refname);
-			memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
-			memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
-			log->value.update.message =
-				xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			if (create_reflog) {
+				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
+				log = &logs[logs_nr++];
+				memset(log, 0, sizeof(*log));
+
+				fill_reftable_log_record(log, &committer_ident);
+				log->update_index = ts;
+				log->refname = xstrdup(u->refname);
+				memcpy(log->value.update.new_hash,
+				       u->new_oid.hash, GIT_MAX_RAWSZ);
+				memcpy(log->value.update.old_hash,
+				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
+				log->value.update.message =
+					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			}
 		}
 
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->new_target) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_target,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if ((u->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
-- 
2.43.GIT

[PATCH v7 6/8] refs: use transaction in `refs_create_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function updates a symref to a given new
target. To do this, it uses a ref-backend specific function
`create_symref()`.

In the previous commits, we introduced symref support in transactions.
This means we can now use transactions to perform symref updates and
don't have to resort to `create_symref()`. Doing this allows us to
remove and cleanup `create_symref()`, which we will do in the following
commit.

Modify the expected error message for a test in
't/t0610-reftable-basics.sh', since the error is now thrown from
'refs.c'. This is because in transactional updates, F/D conflicts are
caught before we're in the reference backend.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                           | 24 +++++++++++++++++-------
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 2 files changed, 40 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index f3d6b3b8e3..7a693dbcfb 100644
--- a/refs.c
+++ b/refs.c
@@ -2305,14 +2305,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..067fd57290 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+	prepared
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	committed
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

[PATCH v7 7/8] refs: rename `refs_create_symref()` to `refs_update_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function is used to update/create a symref.
But it doesn't check the old target of the symref, if existing. It force
updates the symref. In this regard, the name `refs_create_symref()` is a
bit misleading. So let's rename it to `refs_update_symref()`. This is
akin to how 'git-update-ref(1)' also allows us to create apart from
update.

While we're here, rename the arguments in the function to clarify what
they actually signify and reduce confusion.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index dd3e3a7dc0..4491f7a20c 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 7c6c72536b..480202c517 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
diff --git a/refs.c b/refs.c
index 7a693dbcfb..85f53a2cbe 100644
--- a/refs.c
+++ b/refs.c
@@ -2300,10 +2300,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2311,8 +2309,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2328,7 +2326,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
diff --git a/refs.h b/refs.h
index 9abbfd35a2..7c69e8cc0d 100644
--- a/refs.h
+++ b/refs.h
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 82bbf6e2e6..4651e4ced7 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
-- 
2.43.GIT

[PATCH v7 8/8] refs: remove `create_symref` and associated dead code

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

In the previous commits, we converted `refs_create_symref()` to utilize
transactions to perform symref updates. Earlier `refs_create_symref()`
used `create_symref()` to do the same.

We can now remove `create_symref()` and any code associated with it
which is no longer used. We remove `create_symref()` code from all the
reference backends and also remove it entirely from the `ref_storage_be`
struct.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/debug.c            |  13 -----
 refs/files-backend.c    |  67 -------------------------
 refs/packed-backend.c   |   1 -
 refs/refs-internal.h    |   5 --
 refs/reftable-backend.c | 105 ----------------------------------------
 5 files changed, 191 deletions(-)

diff --git a/refs/debug.c b/refs/debug.c
index c7531b17f0..8be316bb67 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 1d4650b7cb..dc0490f0db 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1902,23 +1902,6 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)
 	return ret;
 }
 
-static void update_symref_reflog(struct files_ref_store *refs,
-				 struct ref_lock *lock, const char *refname,
-				 const char *target, const char *logmsg)
-{
-	struct strbuf err = STRBUF_INIT;
-	struct object_id new_oid;
-
-	if (logmsg &&
-	    refs_resolve_ref_unsafe(&refs->base, target,
-				    RESOLVE_REF_READING, &new_oid, NULL) &&
-	    files_log_ref_write(refs, refname, &lock->old_oid,
-				&new_oid, logmsg, 0, &err)) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-	}
-}
-
 static int create_symref_lock(struct files_ref_store *refs,
 			      struct ref_lock *lock, const char *refname,
 			      const char *target, struct strbuf *err)
@@ -1938,55 +1921,6 @@ static int create_symref_lock(struct files_ref_store *refs,
 	return 0;
 }
 
-static int create_and_commit_symref(struct files_ref_store *refs,
-				    struct ref_lock *lock, const char *refname,
-				    const char *target, const char *logmsg)
-{
-	struct strbuf err = STRBUF_INIT;
-	int ret;
-
-	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-		return 0;
-	}
-
-	ret = create_symref_lock(refs, lock, refname, target, &err);
-	if (!ret) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-
-		if (commit_ref(lock) < 0)
-			return error("unable to write symref for %s: %s", refname,
-				     strerror(errno));
-	} else {
-		return error("%s", err.buf);
-	}
-
-	return ret;
-}
-
-static int files_create_symref(struct ref_store *ref_store,
-			       const char *refname, const char *target,
-			       const char *logmsg)
-{
-	struct files_ref_store *refs =
-		files_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct strbuf err = STRBUF_INIT;
-	struct ref_lock *lock;
-	int ret;
-
-	lock = lock_ref_oid_basic(refs, refname, &err);
-	if (!lock) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-		return -1;
-	}
-
-	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
-
-	unlock_ref(lock);
-	return ret;
-}
-
 static int files_reflog_exists(struct ref_store *ref_store,
 			       const char *refname)
 {
@@ -3373,7 +3307,6 @@ struct ref_storage_be refs_be_files = {
 	.initial_transaction_commit = files_initial_transaction_commit,
 
 	.pack_refs = files_pack_refs,
-	.create_symref = files_create_symref,
 	.rename_ref = files_rename_ref,
 	.copy_ref = files_copy_ref,
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 4e826c05ff..a937e7dbfc 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 819157256e..53a6c5d842 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 4817dc2f0b..d8e06767a4 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1307,110 +1307,6 @@ struct write_create_symref_arg {
 	const char *logmsg;
 };
 
-static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
-{
-	struct write_create_symref_arg *create = cb_data;
-	uint64_t ts = reftable_stack_next_update_index(create->stack);
-	struct reftable_ref_record ref = {
-		.refname = (char *)create->refname,
-		.value_type = REFTABLE_REF_SYMREF,
-		.value.symref = (char *)create->target,
-		.update_index = ts,
-	};
-	struct ident_split committer_ident = {0};
-	struct reftable_log_record log = {0};
-	struct object_id new_oid;
-	struct object_id old_oid;
-	const char *committer_info;
-	int ret;
-
-	reftable_writer_set_limits(writer, ts, ts);
-
-	ret = refs_verify_refname_available(&create->refs->base, create->refname,
-					    NULL, NULL, create->err);
-	if (ret < 0)
-		return ret;
-
-	ret = reftable_writer_add_ref(writer, &ref);
-	if (ret)
-		return ret;
-
-	/*
-	 * Note that it is important to try and resolve the reference before we
-	 * write the log entry. This is because `should_write_log()` will munge
-	 * `core.logAllRefUpdates`, which is undesirable when we create a new
-	 * repository because it would be written into the config. As HEAD will
-	 * not resolve for new repositories this ordering will ensure that this
-	 * never happens.
-	 */
-	if (!create->logmsg ||
-	    !refs_resolve_ref_unsafe(&create->refs->base, create->target,
-				     RESOLVE_REF_READING, &new_oid, NULL) ||
-	    !should_write_log(&create->refs->base, create->refname))
-		return 0;
-
-	committer_info = git_committer_info(0);
-	if (split_ident_line(&committer_ident, committer_info, strlen(committer_info)))
-		BUG("failed splitting committer info");
-
-	fill_reftable_log_record(&log, &committer_ident);
-	log.refname = xstrdup(create->refname);
-	log.update_index = ts;
-	log.value.update.message = xstrndup(create->logmsg,
-					    create->refs->write_options.block_size / 2);
-	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
-	if (refs_resolve_ref_unsafe(&create->refs->base, create->refname,
-				    RESOLVE_REF_READING, &old_oid, NULL))
-		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
-
-	ret = reftable_writer_add_log(writer, &log);
-	reftable_log_record_release(&log);
-	return ret;
-}
-
-static int reftable_be_create_symref(struct ref_store *ref_store,
-				     const char *refname,
-				     const char *target,
-				     const char *logmsg)
-{
-	struct reftable_ref_store *refs =
-		reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct reftable_stack *stack = stack_for(refs, refname, &refname);
-	struct strbuf err = STRBUF_INIT;
-	struct write_create_symref_arg arg = {
-		.refs = refs,
-		.stack = stack,
-		.refname = refname,
-		.target = target,
-		.logmsg = logmsg,
-		.err = &err,
-	};
-	int ret;
-
-	ret = refs->err;
-	if (ret < 0)
-		goto done;
-
-	ret = reftable_stack_reload(stack);
-	if (ret)
-		goto done;
-
-	ret = reftable_stack_add(stack, &write_create_symref_table, &arg);
-
-done:
-	assert(ret != REFTABLE_API_ERROR);
-	if (ret) {
-		if (err.len)
-			error("%s", err.buf);
-		else
-			error("unable to write symref for %s: %s", refname,
-			      reftable_error_str(ret));
-	}
-
-	strbuf_release(&err);
-	return ret;
-}
-
 struct write_copy_arg {
 	struct reftable_ref_store *refs;
 	struct reftable_stack *stack;
@@ -2325,7 +2221,6 @@ struct ref_storage_be refs_be_reftable = {
 	.initial_transaction_commit = reftable_be_initial_transaction_commit,
 
 	.pack_refs = reftable_be_pack_refs,
-	.create_symref = reftable_be_create_symref,
 	.rename_ref = reftable_be_rename_ref,
 	.copy_ref = reftable_be_copy_ref,
 
-- 
2.43.GIT

Re: [PATCH v7 0/8] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> Junio, I've rebased my patches on top of, not sure if there is something else I
> missed here:
> - ps/ci-python-2-deprecation (2a686e00d2)
> - ps/reftable-write-options (f5545b173b)
> - ps/pseudo-ref-terminology (ad302a68fd)

This is a somewhat puzzling set of topics.  You are not touching
ci/lib.sh at all so I do not see the point of including the
ci-python one, for example.

Re: [PATCH v7 0/8] refs: add support for transactional symref updates

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> Junio, I've rebased my patches on top of, not sure if there is something else I
>> missed here:
>> - ps/ci-python-2-deprecation (2a686e00d2)
>> - ps/reftable-write-options (f5545b173b)
>> - ps/pseudo-ref-terminology (ad302a68fd)
>
> This is a somewhat puzzling set of topics.  You are not touching
> ci/lib.sh at all so I do not see the point of including the
> ci-python one, for example.

Oh, it is even worse.  2a686e00d2 is not the ci-python thing at all.
It is a random point between 'master' and 'seen' that happens to be
a merge of that topic, so you basically based your series on top of
random collection of many topics that are not even in 'next'.  More
than 30 of them.

2a686e00d2 Merge branch 'ps/ci-python-2-deprecation' into seen
cfae85760b Merge branch 'ow/refspec-glossary-update' into seen
0ac6476973 Merge branch 'ps/ci-enable-minimal-fuzzers-at-gitlab' into seen
2e6fe8d39b Merge branch 'jp/tag-trailer' into seen
2954339080 Merge branch 'rs/external-diff-with-exit-code' into seen
b16fe0b6f5 Merge branch 'it/refs-name-conflict' into seen
bdec3a9a17 Merge branch 'ps/refs-without-the-repository' into seen
b30d6b985f Merge branch 'kn/ref-transaction-symref' into seen
33c1562652 Merge branch 'ds/doc-config-reflow' into seen
053736edd8 Merge branch 'ie/config-includeif-hostname' into seen
db453e610c Merge branch 'cw/git-std-lib' into seen
6858402556 Merge branch 'bk/complete-dirname-for-am-and-format-patch' into seen
2ec214595a Merge branch 'bk/complete-send-email' into seen
0810be3417 Merge branch 'jc/rerere-cleanup' into seen
a3d2847041 Merge branch 'ds/send-email-per-message-block' into seen
919b280123 Merge branch 'tb/attr-limits' into jch
27f675fb6d Merge branch 'ps/config-subcommands' into jch
f5545b173b Merge branch 'ps/reftable-write-options' into jch
c86de5db47 Merge branch 'ew/khash-to-khashl' into jch
ef2bdd9348 Merge branch 'jc/test-workaround-broken-mv' into jch
fb794ba19c Merge branch 'jc/rev-parse-fatal-doc' into jch
96aa517d85 Merge branch 'jt/port-ci-whitespace-check-to-gitlab' into jch
941a03e7c9 Merge branch 'jl/git-no-advice' into jch
56c4ee21a1 Merge branch 'ma/win32-unix-domain-socket' into jch
9d2ea80011 Merge branch 'tb/pseudo-merge-reachability-bitmap' into jch
ad302a68fd Merge branch 'ps/pseudo-ref-terminology' into jch
2754234e89 Merge branch 'ds/scalar-reconfigure-all-fix' into jch
c0340717db Merge branch 'pw/rebase-i-error-message' into jch
89b67e5799 Merge branch 'tb/path-filter-fix' into jch
f40942e3c0 Merge branch 'la/hide-trailer-info' into jch
7b725fb75c Merge branch 'js/cmake-with-test-tool' into jch
1ddb41fd10 Merge branch 'js/unit-test-suite-runner' into jch
e95b4f5420 Merge branch 'jc/no-default-attr-tree-in-bare' into jch
ed26589653 ### match next

I'll not look at this round, as I know you know better than building
a series deliberately on top of these random topics and this was an
accidental mistake you'd rather want to correct before asking folks
to take a look at the resulting series.

Thanks.

[PATCH v8 0/8] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The patch series takes over from the existing patch series, wherein we
introduced symref-* commands to git-update-ref. Since there was a ton of
discussions on the UX of the patch series and its application, I thought it
would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then
we switch refs_create_symref() to start using transactions for symref updates.
This allows us to deprecate the create_symref code in the ref_storage_be
interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the
previous series. While pertaining the user facing components into another set
of patches wherein deeper discussion on the UX can be held without worrying
about the internal implementation. Also by using this new functionality in a
pre-existing command, we can leverage the existing tests to catch any
inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for
dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref
update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/20240330224623.579457-1-knayak@gitlab.com
V2: https://lore.kernel.org/git/20240412095908.1134387-1-knayak@gitlab.com
V3: https://lore.kernel.org/git/20240423212818.574123-1-knayak@gitlab.com
V4: https://lore.kernel.org/r/20240426152449.228860-1-knayak@gitlab.com
V5: https://lore.kernel.org/r/20240501202229.2695774-1-knayak@gitlab.com
V6: https://lore.kernel.org/r/20240503124115.252413-1-knayak@gitlab.com
V7: https://lore.kernel.org/r/20240507060035.28602-1-knayak@gitlab.com

Thanks for all the reviews!

Changes since v7:
* I had rebased v7 on next. I've rebased v8 on master. That's the only difference
between the two versions.

Junio, this might cause conflicts when merging, I think you resolved them for
v6 and hope its the same now. Let me know if I can help otherwise somehow. 

Changes since v6:
* Made the check for old/new oid & target more stricter by removing
the exception for null oid's.
* Extracted `ref_update_check_old_target` and `original_update_refname`
to refs.c
* ^This allowed us to generalize the error messages for non-matching
old_target values between files and reftable backend.
* Better line wrapping in code.
* Fixed some grammar in commit messages.

Range diff against v6:

1:  a354190905 ! 1:  defc1b3521 refs: accept symref values in `ref_transaction_update()`
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      	if (transaction->state != REF_TRANSACTION_OPEN)
      		BUG("update called for transaction that is not open");
      
    -+	if (old_oid && !is_null_oid(old_oid) && old_target)
    ++	if (old_oid && old_target)
     +		BUG("only one of old_oid and old_target should be non NULL");
    -+	if (new_oid && !is_null_oid(new_oid) && new_target)
    ++	if (new_oid && new_target)
     +		BUG("only one of new_oid and new_target should be non NULL");
     +
      	FLEX_ALLOC_STR(update, refname, refname);
2:  0d9c5b9804 = 2:  54bb78a27c files-backend: extract out `create_symref_lock()`
3:  e0219ffd31 = 3:  c16b7c5da0 refs: support symrefs in 'reference-transaction' hook
-:  ---------- > 4:  2d268f12cc refs: move `original_update_refname` to 'refs.c'
4:  b22c59c722 ! 5:  7db3a2245f refs: add support for transactional symref updates
    @@ Commit message
         updates. While this is exposed to users via 'git-update-ref' and its
         '--stdin' mode, it is also used internally within various commands.
     
    -    However, we never supported transactional updates of symrefs. Let's add
    -    support for symrefs in both the 'files' and the 'reftable' backend.
    +    However, we do not support transactional updates of symrefs. This commit
    +    adds support for symrefs in both the 'files' and the 'reftable' backend.
     
         Here, we add and use `ref_update_has_null_new_value()`, a helper
         function which is used to check if there is a new_value in a reference
         update. The new value could either be a symref target `new_target` or a
         OID `new_oid`.
     
    -    With this, now transactional updates (verify, create, delete, update)
    -    can be used for:
    +    We also add another common function `ref_update_check_old_target` which
    +    will be used to check if the update's old_target corresponds to a
    +    reference's current target.
    +
    +    Now transactional updates (verify, create, delete, update) can be used
    +    for:
         - regular refs
         - symbolic refs
         - conversion of regular to symbolic refs and vice versa
    @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      
      	ref_transaction_add_update(transaction, refname, flags,
      				   new_oid, old_oid, new_target,
    -@@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
    - {
    - 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
    +@@ refs.c: const char *ref_update_original_update_refname(struct ref_update *update)
    + 	return update->refname;
      }
    -+
    + 
     +int ref_update_has_null_new_value(struct ref_update *update)
     +{
     +	return !update->new_target && is_null_oid(&update->new_oid);
     +}
    -
    - ## refs/files-backend.c ##
    -@@ refs/files-backend.c: static const char *original_update_refname(struct ref_update *update)
    - 	return update->refname;
    - }
    - 
    -+/*
    -+ * Check whether the old_target values stored in update are consistent
    -+ * with current_target, which is the symbolic reference's current value.
    -+ * If everything is OK, return 0; otherwise, write an error message to
    -+ * err and return -1.
    -+ */
    -+static int check_old_target(struct ref_update *update,
    -+			    const char *current_target,
    -+			    struct strbuf *err)
    ++
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err)
     +{
     +	if (!update->old_target)
     +		BUG("called without old_target set");
     +
    -+	if (!strcmp(update->old_target, current_target))
    ++	if (!strcmp(referent, update->old_target))
     +		return 0;
     +
    -+	if (!strcmp(current_target, ""))
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++	if (!strcmp(referent, ""))
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "reference is missing but expected %s",
    -+			    original_update_refname(update),
    ++			    ref_update_original_update_refname(update),
     +			    update->old_target);
     +	else
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "is at %s but expected %s",
    -+			    original_update_refname(update),
    -+			    current_target, update->old_target);
    -+
    ++			    ref_update_original_update_refname(update),
    ++			    referent, update->old_target);
     +	return -1;
     +}
    -+
    - /*
    -  * Check whether the REF_HAVE_OLD and old_oid values stored in update
    -  * are consistent with oid, which is the reference's current value. If
    +
    + ## refs/files-backend.c ##
    +@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
    + 
    + 	new_update = ref_transaction_add_update(
    + 			transaction, referent, new_flags,
    +-			&update->new_oid, &update->old_oid,
    +-			NULL, NULL, update->msg);
    ++			update->new_target ? NULL : &update->new_oid,
    ++			update->old_target ? NULL : &update->old_oid,
    ++			update->new_target, update->old_target, update->msg);
    + 
    + 	new_update->parent_update = update;
    + 
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
      
      	files_assert_main_repository(refs, "lock_ref_for_update");
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +			}
     +
     +			if (update->old_target) {
    -+				if (check_old_target(update, referent.buf, err)) {
    ++				if (ref_update_check_old_target(referent.buf, update, err)) {
     +					ret = TRANSACTION_GENERIC_ERROR;
     +					goto out;
     +				}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +		 * be set for symrefs, but we're strict about its usage.
     +		 */
     +		if (update->old_target) {
    -+			if (check_old_target(update, referent.buf, err)) {
    ++			if (ref_update_check_old_target(referent.buf, update, err)) {
     +				ret = TRANSACTION_GENERIC_ERROR;
     +				goto out;
     +			}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     -	    !(update->flags & REF_DELETING) &&
     -	    !(update->flags & REF_LOG_ONLY)) {
     +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
    -+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
    ++		if (create_symref_lock(refs, lock, update->refname,
    ++				       update->new_target, err)) {
     +			ret = TRANSACTION_GENERIC_ERROR;
     +			goto out;
     +		}
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
      			if (commit_ref(lock)) {
     
      ## refs/refs-internal.h ##
    -@@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct repository *repo,
    +@@ refs/refs-internal.h: struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
       */
    - struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
    + const char *ref_update_original_update_refname(struct ref_update *update);
      
     +/*
     + * Helper function to check if the new value is null, this
    @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
     + * ref or a symbolic ref.
     + */
     +int ref_update_has_null_new_value(struct ref_update *update);
    ++
    ++/*
    ++ * Check whether the old_target values stored in update are consistent
    ++ * with the referent, which is the symbolic reference's current value.
    ++ * If everything is OK, return 0; otherwise, write an error message to
    ++ * err and return -1.
    ++ */
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err);
     +
      #endif /* REFS_REFS_INTERNAL_H */
     
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      		 */
     -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
     +		if (u->old_target) {
    -+			if (strcmp(referent.buf, u->old_target)) {
    -+				if (!strcmp(referent.buf, ""))
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "reference is missing but expected %s",
    -+						    original_update_refname(u),
    -+						    u->old_target);
    -+				else
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "is at %s but expected %s",
    -+						    original_update_refname(u),
    -+						    referent.buf, u->old_target);
    ++			if (ref_update_check_old_target(referent.buf, u, err)) {
     +				ret = -1;
     +				goto done;
     +			}
     +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
      			if (is_null_oid(&u->old_oid))
      				strbuf_addf(err, _("cannot lock ref '%s': "
    - 					    "reference already exists"),
    + 						   "reference already exists"),
     @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
      		 * - `core.logAllRefUpdates` tells us to create the reflog for
      		 *   the given ref.
      		 */
     -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
    -+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
    ++		if ((u->flags & REF_HAVE_NEW) &&
    ++		    !(u->type & REF_ISSYMREF) &&
    ++		    ref_update_has_null_new_value(u)) {
      			struct reftable_log_record log = {0};
      			struct reftable_iterator it = {0};
      
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     +				fill_reftable_log_record(log);
     +				log->update_index = ts;
     +				log->refname = xstrdup(u->refname);
    -+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
    -+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.new_hash,
    ++				       u->new_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.old_hash,
    ++				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
     +				log->value.update.message =
     +					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
     +			}
5:  636bf5ce98 = 6:  354ebbe17f refs: use transaction in `refs_create_symref()`
6:  07fb23374f = 7:  c8a23b3454 refs: rename `refs_create_symref()` to `refs_update_symref()`
7:  5c05813bcc = 8:  fa1b8f445b refs: remove `create_symref` and associated dead code


Karthik Nayak (8):
  refs: accept symref values in `ref_transaction_update()`
  files-backend: extract out `create_symref_lock()`
  refs: support symrefs in 'reference-transaction' hook
  refs: move `original_update_refname` to 'refs.c'
  refs: add support for transactional symref updates
  refs: use transaction in `refs_create_symref()`
  refs: rename `refs_create_symref()` to `refs_update_symref()`
  refs: remove `create_symref` and associated dead code

 Documentation/githooks.txt       |  14 ++-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           | 119 ++++++++++++++----
 refs.h                           |  20 ++-
 refs/debug.c                     |  13 --
 refs/files-backend.c             | 208 +++++++++++++++++--------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  40 +++++-
 refs/reftable-backend.c          | 183 +++++++++------------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t0610-reftable-basics.sh       |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 22 files changed, 374 insertions(+), 280 deletions(-)

-- 
2.43.GIT

[PATCH v8 1/8] refs: accept symref values in `ref_transaction_update()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `ref_transaction_update()` obtains ref information and
flags to create a `ref_update` and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the
old and new ref targets and process it. This commit adds the required
parameters to the function and modifies all call sites.

The two parameters added are `new_target` and `old_target`. The
`new_target` is used to denote what the reference should point to when
the transaction is applied. Some functions allow this parameter to be
NULL, meaning that the reference is not changed.

The `old_target` denotes the value the reference must have before the
update. Some functions allow this parameter to be NULL, meaning that the
old value of the reference is not checked.

We also update the internal function `ref_transaction_add_update()`
similarly to take the two new parameters.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)

diff --git a/branch.c b/branch.c
index e4a738fc7b..48af4c3ceb 100644
--- a/branch.c
+++ b/branch.c
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index dc5a9d32dd..297dfb91a1 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 5857d860db..66840b7c5b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e8d7df14b6..b150ef39a8 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index da59600ad2..7690687b0e 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 9a33cb50b4..40a65fdebc 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index e46afbc46d..21fdbf6ac8 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
diff --git a/refs.c b/refs.c
index 55d2e0b2cb..e7b7c48d92 100644
--- a/refs.c
+++ b/refs.c
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
diff --git a/refs.h b/refs.h
index d278775e08..c7851bf587 100644
--- a/refs.h
+++ b/refs.h
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index a098d14ea0..e4d0aa3d41 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 56641aa57a..108f4ec419 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 1cda48c504..6104471199 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
diff --git a/sequencer.c b/sequencer.c
index 88de4dc20f..61e007d85f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
diff --git a/walker.c b/walker.c
index c0fd632d92..1b3df43906 100644
--- a/walker.c
+++ b/walker.c
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
-- 
2.43.GIT

[PATCH v8 2/8] files-backend: extract out `create_symref_lock()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The function `create_symref_locked()` creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates
the final symref.

Extract the early half of `create_symref_locked()` into a new helper
function `create_symref_lock()`. Because the name of the new function is
too similar to the original, rename the original to
`create_and_commit_symref()` to avoid confusion.

The new function `create_symref_locked()` can be used to create the
symref lock in a separate step from that of committing it. This allows
to add transactional support for symrefs, where the lock would be
created in the preparation step and the lock would be committed in the
finish step.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/files-backend.c | 51 ++++++++++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 14 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index e4d0aa3d41..74a713090c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1920,27 +1920,49 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target, struct strbuf *err)
 {
+	if (!fdopen_lock_file(&lock->lk, "w")) {
+		strbuf_addf(err, "unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
+		strbuf_addf(err, "unable to write to %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	struct strbuf err = STRBUF_INIT;
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target, &err);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	} else {
+		return error("%s", err.buf);
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1960,7 +1982,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
-- 
2.43.GIT

[PATCH v8 3/8] refs: support symrefs in 'reference-transaction' hook

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The 'reference-transaction' hook runs whenever a reference update is
made to the system. In a previous commit, we added the `old_target` and
`new_target` fields to the `reference_transaction_update()`. In
following commits we'll also add the code to handle symref's in the
reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying
the current format:
    <old-oid> SP <new-oid> SP <ref-name> LF
to be be:
    <old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same.
But when either 'old-value' or 'new-value' is a symref, we print the ref
as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction'
hook's documentation always stated that support for symbolic references
may be added in the future.

We do not add any tests in this commit since there is no git command
which activates this flow, in an upcoming commit, we'll start using
transaction based symref updates as the default, we'll add tests there
for the hook too.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 20 ++++++++++++++++----
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ee9b92c90d..06e997131b 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
diff --git a/refs.c b/refs.c
index e7b7c48d92..9d722d798a 100644
--- a/refs.c
+++ b/refs.c
@@ -2350,10 +2350,22 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (!(update->flags & REF_HAVE_OLD))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (!(update->flags & REF_HAVE_NEW))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
-- 
2.43.GIT

[PATCH v8 4/8] refs: move `original_update_refname` to 'refs.c'

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The files backend and the reftable backend implement
`original_update_refname` to obtain the original refname of the update.
Move it out to 'refs.c' and only expose it internally to the refs
library. This will be used in an upcoming commit to also introduce
another common functionality for the two backends.

We also rename the function to `ref_update_original_update_refname` to
keep it consistent with the upcoming other 'ref_update_*' functions
that'll be introduced.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                  |  9 +++++++++
 refs/files-backend.c    | 21 +++++----------------
 refs/refs-internal.h    |  5 +++++
 refs/reftable-backend.c | 24 +++++++-----------------
 4 files changed, 26 insertions(+), 33 deletions(-)

diff --git a/refs.c b/refs.c
index 9d722d798a..8de43dc0fc 100644
--- a/refs.c
+++ b/refs.c
@@ -2814,3 +2814,12 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+const char *ref_update_original_update_refname(struct ref_update *update)
+{
+	while (update->parent_update)
+		update = update->parent_update;
+
+	return update->refname;
+}
+
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 74a713090c..25e5d03496 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2423,17 +2423,6 @@ static int split_symref_update(struct ref_update *update,
 	return 0;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-
-	return update->refname;
-}
-
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2450,16 +2439,16 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
 	if (is_null_oid(&update->old_oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference already exists",
-			    original_update_refname(update));
+			    ref_update_original_update_refname(update));
 	else if (is_null_oid(oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference is missing but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(&update->old_oid));
 	else
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "is at %s but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(oid),
 			    oid_to_hex(&update->old_oid));
 
@@ -2513,7 +2502,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 		reason = strbuf_detach(err, NULL);
 		strbuf_addf(err, "cannot lock ref '%s': %s",
-			    original_update_refname(update), reason);
+			    ref_update_original_update_refname(update), reason);
 		free(reason);
 		goto out;
 	}
@@ -2533,7 +2522,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 				if (update->flags & REF_HAVE_OLD) {
 					strbuf_addf(err, "cannot lock ref '%s': "
 						    "error reading reference",
-						    original_update_refname(update));
+						    ref_update_original_update_refname(update));
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 108f4ec419..617b93a6c8 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -749,4 +749,9 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Return the refname under which update was originally requested.
+ */
+const char *ref_update_original_update_refname(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 6104471199..8bba5d5096 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -581,16 +581,6 @@ static int reftable_be_read_symbolic_ref(struct ref_store *ref_store,
 	return ret;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-	return update->refname;
-}
-
 struct reftable_transaction_update {
 	struct ref_update *update;
 	struct object_id current_oid;
@@ -869,7 +859,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			/* The reference does not exist, but we expected it to. */
 			strbuf_addf(err, _("cannot lock ref '%s': "
 				    "unable to resolve reference '%s'"),
-				    original_update_refname(u), u->refname);
+				    ref_update_original_update_refname(u), u->refname);
 			ret = -1;
 			goto done;
 		}
@@ -941,17 +931,17 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference already exists"),
-					    original_update_refname(u));
+						   "reference already exists"),
+					    ref_update_original_update_refname(u));
 			else if (is_null_oid(&current_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference is missing but expected %s"),
-					    original_update_refname(u),
+						   "reference is missing but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&u->old_oid));
 			else
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "is at %s but expected %s"),
-					    original_update_refname(u),
+						   "is at %s but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&current_oid),
 					    oid_to_hex(&u->old_oid));
 			ret = -1;
-- 
2.43.GIT

[PATCH v8 5/8] refs: add support for transactional symref updates

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The reference backends currently support transactional reference
updates. While this is exposed to users via 'git-update-ref' and its
'--stdin' mode, it is also used internally within various commands.

However, we do not support transactional updates of symrefs. This commit
adds support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use `ref_update_has_null_new_value()`, a helper
function which is used to check if there is a new_value in a reference
update. The new value could either be a symref target `new_target` or a
OID `new_oid`.

We also add another common function `ref_update_check_old_target` which
will be used to check if the update's old_target corresponds to a
reference's current target.

Now transactional updates (verify, create, delete, update) can be used
for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in
'git-update-ref' in the future.

Note that a dangling symref update does not record a new reflog entry,
which is unchanged before and after this commit.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                  |  36 ++++++++++++-
 refs/files-backend.c    | 113 ++++++++++++++++++++++++++++++++--------
 refs/refs-internal.h    |  16 ++++++
 refs/reftable-backend.c |  71 +++++++++++++++++++------
 4 files changed, 196 insertions(+), 40 deletions(-)

diff --git a/refs.c b/refs.c
index 8de43dc0fc..d0ea7573d8 100644
--- a/refs.c
+++ b/refs.c
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,10 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	update->new_target = xstrdup_or_null(new_target);
+	update->old_target = xstrdup_or_null(old_target);
+	if ((flags & REF_HAVE_NEW) && new_oid)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if ((flags & REF_HAVE_OLD) && old_oid)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1286,6 +1291,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2823,3 +2829,29 @@ const char *ref_update_original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+int ref_update_has_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
+
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(referent, update->old_target))
+		return 0;
+
+	if (!strcmp(referent, ""))
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "reference is missing but expected %s",
+			    ref_update_original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "is at %s but expected %s",
+			    ref_update_original_update_refname(update),
+			    referent, update->old_target);
+	return -1;
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 25e5d03496..2d1525b240 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2394,8 +2394,9 @@ static int split_symref_update(struct ref_update *update,
 
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
-			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target ? NULL : &update->new_oid,
+			update->old_target ? NULL : &update->old_oid,
+			update->new_target, update->old_target, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2483,7 +2484,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
@@ -2526,7 +2527,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
-			} else if (check_old_oid(update, &lock->old_oid, err)) {
+			}
+
+			if (update->old_target) {
+				if (ref_update_check_old_target(referent.buf, update, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}
+			} else if  (check_old_oid(update, &lock->old_oid, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto out;
 			}
@@ -2547,7 +2555,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 	} else {
 		struct ref_update *parent_update;
 
-		if (check_old_oid(update, &lock->old_oid, err)) {
+		/*
+		 * Even if the ref is a regular ref, if `old_target` is set, we
+		 * check the referent value. Ideally `old_target` should only
+		 * be set for symrefs, but we're strict about its usage.
+		 */
+		if (update->old_target) {
+			if (ref_update_check_old_target(referent.buf, update, err)) {
+				ret = TRANSACTION_GENERIC_ERROR;
+				goto out;
+			}
+		} else if  (check_old_oid(update, &lock->old_oid, err)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
 		}
@@ -2565,9 +2583,28 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
+		if (create_symref_lock(refs, lock, update->refname,
+				       update->new_target, err)) {
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		if (close_ref_gently(lock)) {
+			strbuf_addf(err, "couldn't close '%s.lock'",
+				    update->refname);
+			ret = TRANSACTION_GENERIC_ERROR;
+			goto out;
+		}
+
+		/*
+		 * Once we have created the symref lock, the commit
+		 * phase of the transaction only needs to commit the lock.
+		 */
+		update->flags |= REF_NEEDS_COMMIT;
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
@@ -2830,6 +2867,43 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 	return ret;
 }
 
+static int parse_and_write_reflog(struct files_ref_store *refs,
+				  struct ref_update *update,
+				  struct ref_lock *lock,
+				  struct strbuf *err)
+{
+	if (update->new_target) {
+		/*
+		 * We want to get the resolved OID for the target, to ensure
+		 * that the correct value is added to the reflog.
+		 */
+		if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
+					     RESOLVE_REF_READING,
+					     &update->new_oid, NULL)) {
+			/*
+			 * TODO: currently we skip creating reflogs for dangling
+			 * symref updates. It would be nice to capture this as
+			 * zero oid updates however.
+			 */
+			return 0;
+		}
+	}
+
+	if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid,
+				&update->new_oid, update->msg, update->flags, err)) {
+		char *old_msg = strbuf_detach(err, NULL);
+
+		strbuf_addf(err, "cannot update the ref '%s': %s",
+			    lock->ref_name, old_msg);
+		free(old_msg);
+		unlock_ref(lock);
+		update->backend_data = NULL;
+		return -1;
+	}
+
+	return 0;
+}
+
 static int files_transaction_finish(struct ref_store *ref_store,
 				    struct ref_transaction *transaction,
 				    struct strbuf *err)
@@ -2860,23 +2934,20 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
-			if (files_log_ref_write(refs,
-						lock->ref_name,
-						&lock->old_oid,
-						&update->new_oid,
-						update->msg, update->flags,
-						err)) {
-				char *old_msg = strbuf_detach(err, NULL);
-
-				strbuf_addf(err, "cannot update the ref '%s': %s",
-					    lock->ref_name, old_msg);
-				free(old_msg);
-				unlock_ref(lock);
-				update->backend_data = NULL;
+			if (parse_and_write_reflog(refs, update, lock, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto cleanup;
 			}
 		}
+
+		/*
+		 * We try creating a symlink, if that succeeds we continue to the
+		 * next update. If not, we try and create a regular symref.
+		 */
+		if (update->new_target && prefer_symlink_refs)
+			if (!create_ref_symlink(lock, update->new_target))
+				continue;
+
 		if (update->flags & REF_NEEDS_COMMIT) {
 			clear_loose_ref_cache(refs);
 			if (commit_ref(lock)) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 617b93a6c8..819157256e 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -754,4 +754,20 @@ struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
  */
 const char *ref_update_original_update_refname(struct ref_update *update);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_has_null_new_value(struct ref_update *update);
+
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with the referent, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 8bba5d5096..3eab57cf95 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -846,7 +846,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -897,8 +897,10 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 * intertwined with the locking in files-backend.c.
 				 */
 				new_update = ref_transaction_add_update(
-						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+					transaction, referent.buf, new_flags,
+					&u->new_oid, &u->old_oid, u->new_target,
+					u->old_target, u->msg);
+
 				new_update->parent_update = u;
 
 				/*
@@ -928,7 +930,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (ref_update_check_old_target(referent.buf, u, err)) {
+				ret = -1;
+				goto done;
+			}
+		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 						   "reference already exists"),
@@ -1033,7 +1040,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if ((u->flags & REF_HAVE_NEW) &&
+		    !(u->type & REF_ISSYMREF) &&
+		    ref_update_has_null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1074,24 +1083,52 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			   (u->flags & REF_FORCE_CREATE_REFLOG ||
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
+			int create_reflog = 1;
+
+			if (u->new_target) {
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
+							     RESOLVE_REF_READING, &u->new_oid, NULL)) {
+					/*
+					 * TODO: currently we skip creating reflogs for dangling
+					 * symref updates. It would be nice to capture this as
+					 * zero oid updates however.
+					 */
+					create_reflog = 0;
+				}
+			}
 
-			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
-			log = &logs[logs_nr++];
-			memset(log, 0, sizeof(*log));
-
-			fill_reftable_log_record(log);
-			log->update_index = ts;
-			log->refname = xstrdup(u->refname);
-			memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
-			memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
-			log->value.update.message =
-				xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			if (create_reflog) {
+				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
+				log = &logs[logs_nr++];
+				memset(log, 0, sizeof(*log));
+
+				fill_reftable_log_record(log);
+				log->update_index = ts;
+				log->refname = xstrdup(u->refname);
+				memcpy(log->value.update.new_hash,
+				       u->new_oid.hash, GIT_MAX_RAWSZ);
+				memcpy(log->value.update.old_hash,
+				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
+				log->value.update.message =
+					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
+			}
 		}
 
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->new_target) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_target,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if ((u->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
-- 
2.43.GIT

[PATCH v8 6/8] refs: use transaction in `refs_create_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function updates a symref to a given new
target. To do this, it uses a ref-backend specific function
`create_symref()`.

In the previous commits, we introduced symref support in transactions.
This means we can now use transactions to perform symref updates and
don't have to resort to `create_symref()`. Doing this allows us to
remove and cleanup `create_symref()`, which we will do in the following
commit.

Modify the expected error message for a test in
't/t0610-reftable-basics.sh', since the error is now thrown from
'refs.c'. This is because in transactional updates, F/D conflicts are
caught before we're in the reference backend.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs.c                           | 24 +++++++++++++++++-------
 t/t0610-reftable-basics.sh       |  2 +-
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 3 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/refs.c b/refs.c
index d0ea7573d8..507f5f0525 100644
--- a/refs.c
+++ b/refs.c
@@ -2289,14 +2289,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index 178791e086..9e8d22bcbd 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
 	git init repo &&
 	test_commit -C repo A &&
 	cat >expect <<-EOF &&
-	error: unable to write symref for refs/heads: file/directory conflict
+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
 	EOF
 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
 	test_cmp expect err
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 2092488090..067fd57290 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+	prepared
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	committed
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
2.43.GIT

[PATCH v8 7/8] refs: rename `refs_create_symref()` to `refs_update_symref()`

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

The `refs_create_symref()` function is used to update/create a symref.
But it doesn't check the old target of the symref, if existing. It force
updates the symref. In this regard, the name `refs_create_symref()` is a
bit misleading. So let's rename it to `refs_update_symref()`. This is
akin to how 'git-update-ref(1)' also allows us to create apart from
update.

While we're here, rename the arguments in the function to clarify what
they actually signify and reduce confusion.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index dd3e3a7dc0..4491f7a20c 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 7c6c72536b..480202c517 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
diff --git a/refs.c b/refs.c
index 507f5f0525..fa5471d219 100644
--- a/refs.c
+++ b/refs.c
@@ -2284,10 +2284,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2295,8 +2293,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2312,7 +2310,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
diff --git a/refs.h b/refs.h
index c7851bf587..71cc1c58e0 100644
--- a/refs.h
+++ b/refs.h
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 82bbf6e2e6..4651e4ced7 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
-- 
2.43.GIT

[PATCH v8 8/8] refs: remove `create_symref` and associated dead code

[Karthik Nayak]

From: Karthik Nayak <karthik.188@gmail.com>

In the previous commits, we converted `refs_create_symref()` to utilize
transactions to perform symref updates. Earlier `refs_create_symref()`
used `create_symref()` to do the same.

We can now remove `create_symref()` and any code associated with it
which is no longer used. We remove `create_symref()` code from all the
reference backends and also remove it entirely from the `ref_storage_be`
struct.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 refs/debug.c            | 13 -------
 refs/files-backend.c    | 67 --------------------------------
 refs/packed-backend.c   |  1 -
 refs/refs-internal.h    |  5 ---
 refs/reftable-backend.c | 86 -----------------------------------------
 5 files changed, 172 deletions(-)

diff --git a/refs/debug.c b/refs/debug.c
index c7531b17f0..8be316bb67 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 2d1525b240..3957bfa579 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1903,23 +1903,6 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)
 	return ret;
 }
 
-static void update_symref_reflog(struct files_ref_store *refs,
-				 struct ref_lock *lock, const char *refname,
-				 const char *target, const char *logmsg)
-{
-	struct strbuf err = STRBUF_INIT;
-	struct object_id new_oid;
-
-	if (logmsg &&
-	    refs_resolve_ref_unsafe(&refs->base, target,
-				    RESOLVE_REF_READING, &new_oid, NULL) &&
-	    files_log_ref_write(refs, refname, &lock->old_oid,
-				&new_oid, logmsg, 0, &err)) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-	}
-}
-
 static int create_symref_lock(struct files_ref_store *refs,
 			      struct ref_lock *lock, const char *refname,
 			      const char *target, struct strbuf *err)
@@ -1939,55 +1922,6 @@ static int create_symref_lock(struct files_ref_store *refs,
 	return 0;
 }
 
-static int create_and_commit_symref(struct files_ref_store *refs,
-				    struct ref_lock *lock, const char *refname,
-				    const char *target, const char *logmsg)
-{
-	struct strbuf err = STRBUF_INIT;
-	int ret;
-
-	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-		return 0;
-	}
-
-	ret = create_symref_lock(refs, lock, refname, target, &err);
-	if (!ret) {
-		update_symref_reflog(refs, lock, refname, target, logmsg);
-
-		if (commit_ref(lock) < 0)
-			return error("unable to write symref for %s: %s", refname,
-				     strerror(errno));
-	} else {
-		return error("%s", err.buf);
-	}
-
-	return ret;
-}
-
-static int files_create_symref(struct ref_store *ref_store,
-			       const char *refname, const char *target,
-			       const char *logmsg)
-{
-	struct files_ref_store *refs =
-		files_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct strbuf err = STRBUF_INIT;
-	struct ref_lock *lock;
-	int ret;
-
-	lock = lock_ref_oid_basic(refs, refname, &err);
-	if (!lock) {
-		error("%s", err.buf);
-		strbuf_release(&err);
-		return -1;
-	}
-
-	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
-
-	unlock_ref(lock);
-	return ret;
-}
-
 static int files_reflog_exists(struct ref_store *ref_store,
 			       const char *refname)
 {
@@ -3374,7 +3308,6 @@ struct ref_storage_be refs_be_files = {
 	.initial_transaction_commit = files_initial_transaction_commit,
 
 	.pack_refs = files_pack_refs,
-	.create_symref = files_create_symref,
 	.rename_ref = files_rename_ref,
 	.copy_ref = files_copy_ref,
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 4e826c05ff..a937e7dbfc 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 819157256e..53a6c5d842 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 3eab57cf95..1b92c396b6 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1259,91 +1259,6 @@ struct write_create_symref_arg {
 	const char *logmsg;
 };
 
-static int write_create_symref_table(struct reftable_writer *writer, void *cb_data)
-{
-	struct write_create_symref_arg *create = cb_data;
-	uint64_t ts = reftable_stack_next_update_index(create->stack);
-	struct reftable_ref_record ref = {
-		.refname = (char *)create->refname,
-		.value_type = REFTABLE_REF_SYMREF,
-		.value.symref = (char *)create->target,
-		.update_index = ts,
-	};
-	struct reftable_log_record log = {0};
-	struct object_id new_oid;
-	struct object_id old_oid;
-	int ret;
-
-	reftable_writer_set_limits(writer, ts, ts);
-
-	ret = reftable_writer_add_ref(writer, &ref);
-	if (ret)
-		return ret;
-
-	/*
-	 * Note that it is important to try and resolve the reference before we
-	 * write the log entry. This is because `should_write_log()` will munge
-	 * `core.logAllRefUpdates`, which is undesirable when we create a new
-	 * repository because it would be written into the config. As HEAD will
-	 * not resolve for new repositories this ordering will ensure that this
-	 * never happens.
-	 */
-	if (!create->logmsg ||
-	    !refs_resolve_ref_unsafe(&create->refs->base, create->target,
-				     RESOLVE_REF_READING, &new_oid, NULL) ||
-	    !should_write_log(&create->refs->base, create->refname))
-		return 0;
-
-	fill_reftable_log_record(&log);
-	log.refname = xstrdup(create->refname);
-	log.update_index = ts;
-	log.value.update.message = xstrndup(create->logmsg,
-					    create->refs->write_options.block_size / 2);
-	memcpy(log.value.update.new_hash, new_oid.hash, GIT_MAX_RAWSZ);
-	if (refs_resolve_ref_unsafe(&create->refs->base, create->refname,
-				    RESOLVE_REF_READING, &old_oid, NULL))
-		memcpy(log.value.update.old_hash, old_oid.hash, GIT_MAX_RAWSZ);
-
-	ret = reftable_writer_add_log(writer, &log);
-	reftable_log_record_release(&log);
-	return ret;
-}
-
-static int reftable_be_create_symref(struct ref_store *ref_store,
-				     const char *refname,
-				     const char *target,
-				     const char *logmsg)
-{
-	struct reftable_ref_store *refs =
-		reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_symref");
-	struct reftable_stack *stack = stack_for(refs, refname, &refname);
-	struct write_create_symref_arg arg = {
-		.refs = refs,
-		.stack = stack,
-		.refname = refname,
-		.target = target,
-		.logmsg = logmsg,
-	};
-	int ret;
-
-	ret = refs->err;
-	if (ret < 0)
-		goto done;
-
-	ret = reftable_stack_reload(stack);
-	if (ret)
-		goto done;
-
-	ret = reftable_stack_add(stack, &write_create_symref_table, &arg);
-
-done:
-	assert(ret != REFTABLE_API_ERROR);
-	if (ret)
-		error("unable to write symref for %s: %s", refname,
-		      reftable_error_str(ret));
-	return ret;
-}
-
 struct write_copy_arg {
 	struct reftable_ref_store *refs;
 	struct reftable_stack *stack;
@@ -2251,7 +2166,6 @@ struct ref_storage_be refs_be_reftable = {
 	.initial_transaction_commit = reftable_be_initial_transaction_commit,
 
 	.pack_refs = reftable_be_pack_refs,
-	.create_symref = reftable_be_create_symref,
 	.rename_ref = reftable_be_rename_ref,
 	.copy_ref = reftable_be_copy_ref,
 
-- 
2.43.GIT

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

[phillip.wood123]

Hi Karthik

On 07/05/2024 13:58, Karthik Nayak wrote:
> Changes since v6:
> * Made the check for old/new oid & target more stricter by removing
> the exception for null oid's.
> * Extracted `ref_update_check_old_target` and `original_update_refname`
> to refs.c
> * ^This allowed us to generalize the error messages for non-matching
> old_target values between files and reftable backend.
> * Better line wrapping in code.
> * Fixed some grammar in commit messages.

Thanks for re-rolling - these changes address all of my comments on v6 
and the range-diff below looks good.

Best Wishes

Phillip

> Range diff against v6:
> 
> 1:  a354190905 ! 1:  defc1b3521 refs: accept symref values in `ref_transaction_update()`
>      @@ refs.c: struct ref_update *ref_transaction_add_update(
>        	if (transaction->state != REF_TRANSACTION_OPEN)
>        		BUG("update called for transaction that is not open");
>        
>      -+	if (old_oid && !is_null_oid(old_oid) && old_target)
>      ++	if (old_oid && old_target)
>       +		BUG("only one of old_oid and old_target should be non NULL");
>      -+	if (new_oid && !is_null_oid(new_oid) && new_target)
>      ++	if (new_oid && new_target)
>       +		BUG("only one of new_oid and new_target should be non NULL");
>       +
>        	FLEX_ALLOC_STR(update, refname, refname);
> 2:  0d9c5b9804 = 2:  54bb78a27c files-backend: extract out `create_symref_lock()`
> 3:  e0219ffd31 = 3:  c16b7c5da0 refs: support symrefs in 'reference-transaction' hook
> -:  ---------- > 4:  2d268f12cc refs: move `original_update_refname` to 'refs.c'
> 4:  b22c59c722 ! 5:  7db3a2245f refs: add support for transactional symref updates
>      @@ Commit message
>           updates. While this is exposed to users via 'git-update-ref' and its
>           '--stdin' mode, it is also used internally within various commands.
>       
>      -    However, we never supported transactional updates of symrefs. Let's add
>      -    support for symrefs in both the 'files' and the 'reftable' backend.
>      +    However, we do not support transactional updates of symrefs. This commit
>      +    adds support for symrefs in both the 'files' and the 'reftable' backend.
>       
>           Here, we add and use `ref_update_has_null_new_value()`, a helper
>           function which is used to check if there is a new_value in a reference
>           update. The new value could either be a symref target `new_target` or a
>           OID `new_oid`.
>       
>      -    With this, now transactional updates (verify, create, delete, update)
>      -    can be used for:
>      +    We also add another common function `ref_update_check_old_target` which
>      +    will be used to check if the update's old_target corresponds to a
>      +    reference's current target.
>      +
>      +    Now transactional updates (verify, create, delete, update) can be used
>      +    for:
>           - regular refs
>           - symbolic refs
>           - conversion of regular to symbolic refs and vice versa
>      @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
>        
>        	ref_transaction_add_update(transaction, refname, flags,
>        				   new_oid, old_oid, new_target,
>      -@@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
>      - {
>      - 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
>      +@@ refs.c: const char *ref_update_original_update_refname(struct ref_update *update)
>      + 	return update->refname;
>        }
>      -+
>      +
>       +int ref_update_has_null_new_value(struct ref_update *update)
>       +{
>       +	return !update->new_target && is_null_oid(&update->new_oid);
>       +}
>      -
>      - ## refs/files-backend.c ##
>      -@@ refs/files-backend.c: static const char *original_update_refname(struct ref_update *update)
>      - 	return update->refname;
>      - }
>      -
>      -+/*
>      -+ * Check whether the old_target values stored in update are consistent
>      -+ * with current_target, which is the symbolic reference's current value.
>      -+ * If everything is OK, return 0; otherwise, write an error message to
>      -+ * err and return -1.
>      -+ */
>      -+static int check_old_target(struct ref_update *update,
>      -+			    const char *current_target,
>      -+			    struct strbuf *err)
>      ++
>      ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
>      ++				struct strbuf *err)
>       +{
>       +	if (!update->old_target)
>       +		BUG("called without old_target set");
>       +
>      -+	if (!strcmp(update->old_target, current_target))
>      ++	if (!strcmp(referent, update->old_target))
>       +		return 0;
>       +
>      -+	if (!strcmp(current_target, ""))
>      -+		strbuf_addf(err, "cannot lock ref '%s': "
>      ++	if (!strcmp(referent, ""))
>      ++		strbuf_addf(err, "verifying symref target: '%s': "
>       +			    "reference is missing but expected %s",
>      -+			    original_update_refname(update),
>      ++			    ref_update_original_update_refname(update),
>       +			    update->old_target);
>       +	else
>      -+		strbuf_addf(err, "cannot lock ref '%s': "
>      ++		strbuf_addf(err, "verifying symref target: '%s': "
>       +			    "is at %s but expected %s",
>      -+			    original_update_refname(update),
>      -+			    current_target, update->old_target);
>      -+
>      ++			    ref_update_original_update_refname(update),
>      ++			    referent, update->old_target);
>       +	return -1;
>       +}
>      -+
>      - /*
>      -  * Check whether the REF_HAVE_OLD and old_oid values stored in update
>      -  * are consistent with oid, which is the reference's current value. If
>      +
>      + ## refs/files-backend.c ##
>      +@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
>      +
>      + 	new_update = ref_transaction_add_update(
>      + 			transaction, referent, new_flags,
>      +-			&update->new_oid, &update->old_oid,
>      +-			NULL, NULL, update->msg);
>      ++			update->new_target ? NULL : &update->new_oid,
>      ++			update->old_target ? NULL : &update->old_oid,
>      ++			update->new_target, update->old_target, update->msg);
>      +
>      + 	new_update->parent_update = update;
>      +
>       @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
>        
>        	files_assert_main_repository(refs, "lock_ref_for_update");
>      @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
>       +			}
>       +
>       +			if (update->old_target) {
>      -+				if (check_old_target(update, referent.buf, err)) {
>      ++				if (ref_update_check_old_target(referent.buf, update, err)) {
>       +					ret = TRANSACTION_GENERIC_ERROR;
>       +					goto out;
>       +				}
>      @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
>       +		 * be set for symrefs, but we're strict about its usage.
>       +		 */
>       +		if (update->old_target) {
>      -+			if (check_old_target(update, referent.buf, err)) {
>      ++			if (ref_update_check_old_target(referent.buf, update, err)) {
>       +				ret = TRANSACTION_GENERIC_ERROR;
>       +				goto out;
>       +			}
>      @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
>       -	    !(update->flags & REF_DELETING) &&
>       -	    !(update->flags & REF_LOG_ONLY)) {
>       +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
>      -+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
>      ++		if (create_symref_lock(refs, lock, update->refname,
>      ++				       update->new_target, err)) {
>       +			ret = TRANSACTION_GENERIC_ERROR;
>       +			goto out;
>       +		}
>      @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
>        			if (commit_ref(lock)) {
>       
>        ## refs/refs-internal.h ##
>      -@@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct repository *repo,
>      +@@ refs/refs-internal.h: struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
>         */
>      - struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
>      + const char *ref_update_original_update_refname(struct ref_update *update);
>        
>       +/*
>       + * Helper function to check if the new value is null, this
>      @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
>       + * ref or a symbolic ref.
>       + */
>       +int ref_update_has_null_new_value(struct ref_update *update);
>      ++
>      ++/*
>      ++ * Check whether the old_target values stored in update are consistent
>      ++ * with the referent, which is the symbolic reference's current value.
>      ++ * If everything is OK, return 0; otherwise, write an error message to
>      ++ * err and return -1.
>      ++ */
>      ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
>      ++				struct strbuf *err);
>       +
>        #endif /* REFS_REFS_INTERNAL_H */
>       
>      @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
>        		 */
>       -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
>       +		if (u->old_target) {
>      -+			if (strcmp(referent.buf, u->old_target)) {
>      -+				if (!strcmp(referent.buf, ""))
>      -+					strbuf_addf(err, "verifying symref target: '%s': "
>      -+						    "reference is missing but expected %s",
>      -+						    original_update_refname(u),
>      -+						    u->old_target);
>      -+				else
>      -+					strbuf_addf(err, "verifying symref target: '%s': "
>      -+						    "is at %s but expected %s",
>      -+						    original_update_refname(u),
>      -+						    referent.buf, u->old_target);
>      ++			if (ref_update_check_old_target(referent.buf, u, err)) {
>       +				ret = -1;
>       +				goto done;
>       +			}
>       +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
>        			if (is_null_oid(&u->old_oid))
>        				strbuf_addf(err, _("cannot lock ref '%s': "
>      - 					    "reference already exists"),
>      + 						   "reference already exists"),
>       @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
>        		 * - `core.logAllRefUpdates` tells us to create the reflog for
>        		 *   the given ref.
>        		 */
>       -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
>      -+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
>      ++		if ((u->flags & REF_HAVE_NEW) &&
>      ++		    !(u->type & REF_ISSYMREF) &&
>      ++		    ref_update_has_null_new_value(u)) {
>        			struct reftable_log_record log = {0};
>        			struct reftable_iterator it = {0};
>        
>      @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
>       +				fill_reftable_log_record(log);
>       +				log->update_index = ts;
>       +				log->refname = xstrdup(u->refname);
>      -+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
>      -+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
>      ++				memcpy(log->value.update.new_hash,
>      ++				       u->new_oid.hash, GIT_MAX_RAWSZ);
>      ++				memcpy(log->value.update.old_hash,
>      ++				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
>       +				log->value.update.message =
>       +					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
>       +			}
> 5:  636bf5ce98 = 6:  354ebbe17f refs: use transaction in `refs_create_symref()`
> 6:  07fb23374f = 7:  c8a23b3454 refs: rename `refs_create_symref()` to `refs_update_symref()`
> 7:  5c05813bcc = 8:  fa1b8f445b refs: remove `create_symref` and associated dead code
> 
> 
> Karthik Nayak (8):
>    refs: accept symref values in `ref_transaction_update()`
>    files-backend: extract out `create_symref_lock()`
>    refs: support symrefs in 'reference-transaction' hook
>    refs: move `original_update_refname` to 'refs.c'
>    refs: add support for transactional symref updates
>    refs: use transaction in `refs_create_symref()`
>    refs: rename `refs_create_symref()` to `refs_update_symref()`
>    refs: remove `create_symref` and associated dead code
> 
>   Documentation/githooks.txt       |  14 ++-
>   branch.c                         |   2 +-
>   builtin/branch.c                 |   2 +-
>   builtin/fast-import.c            |   5 +-
>   builtin/fetch.c                  |   2 +-
>   builtin/receive-pack.c           |   1 +
>   builtin/replace.c                |   2 +-
>   builtin/tag.c                    |   1 +
>   builtin/update-ref.c             |   1 +
>   builtin/worktree.c               |   2 +-
>   refs.c                           | 119 ++++++++++++++----
>   refs.h                           |  20 ++-
>   refs/debug.c                     |  13 --
>   refs/files-backend.c             | 208 +++++++++++++++++--------------
>   refs/packed-backend.c            |   1 -
>   refs/refs-internal.h             |  40 +++++-
>   refs/reftable-backend.c          | 183 +++++++++------------------
>   sequencer.c                      |   9 +-
>   t/helper/test-ref-store.c        |   2 +-
>   t/t0610-reftable-basics.sh       |   2 +-
>   t/t1416-ref-transaction-hooks.sh |  23 ++++
>   walker.c                         |   2 +-
>   22 files changed, 374 insertions(+), 280 deletions(-)
> 

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> Changes since v7:
> * I had rebased v7 on next. I've rebased v8 on master. That's the only difference
> between the two versions.

I've applied them to the same base as used to queue the previous
round, which I think is "436d4e5b14 The seventeenth batch".  It went
without conflicts, and tests fine in isolation.  I'll see if it plays
well with other topics in 'seen' later in the day but not now.

Thanks.

> Junio, this might cause conflicts when merging, I think you resolved them for
> v6 and hope its the same now. Let me know if I can help otherwise somehow. 

The easiest for both of us would be to do this:

 (1) Build on whatever base you want, and format-patch the series.
     If you are doing "rebase -i" in-place to update from the
     previous round, this will reuse the previous base so (2) and
     (3) may become trivial.

 (2) Find the base of where the last round was queued, something like

     $ mine='kn/ref-transaction-symref'
     $ git checkout "origin/seen^{/^Merge branch '$mine'}...master"

 (3) Apply your format-patch result.  There are three cases

     (3)-1. Things apply cleanly and tests fine.  Go to (4).

     (3)-2. Things apply cleanly but does not build or test fails,
	    or things do not apply cleanly.

     In the latter case, you have textual or semantic conflicts
     coming from the difference between the old base and the base
     you used to build in (1).  Identify what caused the breakages
     (e.g., a topic or two may have merged since the base used by
     (2) until the base used by (1)).

     Check out the latest 'origin/master' (which may be newer than
     the base used by (2)), "merge --no-ff" the topics you newly
     depend on in there, and use the result of the merge(s) as the
     base, rebuild the series and test again.  Run format-patch from
     the last such merges to the tip of your topic.  If you did

     $ git checkout origin/master
     $ git merge --no-ff --into kn/ref-transaction-symref fo/obar
     $ git merge --no-ff --into kn/ref-transaction-symref ba/zqux
     ... rebuild the topic ...

     Then you'd just format your topic above these "preparing the
     ground" merges, e.g.

     $ git format-patch "HEAD^{/^Merge branch 'ba/zqux'}"..HEAD

     Do not forget to write in the cover letter you did this,
     including the topics you have in your base on top of 'master'.
     Then go to (4).

 (4) Make a trial merge of your topic into 'next' and 'seen', e.g.

     $ git checkout --detach 'origin/seen' &&
       git revert -m 1 <the merge of the previous iteration into seen> &&
       git merge kn/ref-transaction-symref

     The "revert" is needed if the previous iteration of your topic
     is already in 'seen' (like in this case).  You could choose to
     rebuild master..origin/seen from scratch while excluding your
     previous iteration, which may emulate what happens on my end
     more closely.

     This trial merge may conflict.  It is primarily to see what
     conflicts _other_ topics may have with your topic.  In other
     words, you do not have to depend on to make your topic work on
     'master'.  It may become the job of the other topic owners to
     resolve conflicts if your topic goes to 'next' before theirs.

     Make a note on what conflict you saw in the cover letter.  You
     do not necessarily have to resolve them, but it would be a good
     opportunity to learn what others are doing in an related area.

     $ git checkout --detach 'origin/next' &&
       git merge kn/ref-transaction-symref

     This is to see what conflicts your topic has with other topics
     that are already cooking.  This should not conflict if (3)-2
     prepared a base on top of updated master plus dependent topics
     taken from 'next'.  Unless the context is severe (one way to
     tell is try the same trial merge with your old iteration, which
     may conflict in a similar way), expect that it will be handled
     on my end (if it gets unmanageable, I'll ask to rebase when I
     receive your patches).

Something like the above should be added to the SubmittingPatches
document (or its successor to cover more advanced topics, perhaps).

Thanks.

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 5083 bytes --]

Junio C Hamano <gitster@pobox.com> writes:

> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> Changes since v7:
>> * I had rebased v7 on next. I've rebased v8 on master. That's the only difference
>> between the two versions.
>
> I've applied them to the same base as used to queue the previous
> round, which I think is "436d4e5b14 The seventeenth batch".  It went
> without conflicts, and tests fine in isolation.  I'll see if it plays
> well with other topics in 'seen' later in the day but not now.
>
> Thanks.
>
>> Junio, this might cause conflicts when merging, I think you resolved them for
>> v6 and hope its the same now. Let me know if I can help otherwise somehow.
>
> The easiest for both of us would be to do this:
>
>  (1) Build on whatever base you want, and format-patch the series.
>      If you are doing "rebase -i" in-place to update from the
>      previous round, this will reuse the previous base so (2) and
>      (3) may become trivial.
>
>  (2) Find the base of where the last round was queued, something like
>
>      $ mine='kn/ref-transaction-symref'
>      $ git checkout "origin/seen^{/^Merge branch '$mine'}...master"
>

I find the '...' always so confusing, I would say suggesting to use
'git-merge-base' would be much nicer here.

>  (3) Apply your format-patch result.  There are three cases
>
>      (3)-1. Things apply cleanly and tests fine.  Go to (4).
>
>      (3)-2. Things apply cleanly but does not build or test fails,
> 	    or things do not apply cleanly.
>
>      In the latter case, you have textual or semantic conflicts
>      coming from the difference between the old base and the base
>      you used to build in (1).  Identify what caused the breakages
>      (e.g., a topic or two may have merged since the base used by
>      (2) until the base used by (1)).
>
>      Check out the latest 'origin/master' (which may be newer than
>      the base used by (2)), "merge --no-ff" the topics you newly

For my own understanding, even if we use '--ff' the end result should be
the same, but using '--no-ff' would ensure that the changes and
conflicts are isolated to the merge commit, right?

>      depend on in there, and use the result of the merge(s) as the
>      base, rebuild the series and test again.  Run format-patch from
>      the last such merges to the tip of your topic.  If you did
>
>      $ git checkout origin/master
>      $ git merge --no-ff --into kn/ref-transaction-symref fo/obar
>      $ git merge --no-ff --into kn/ref-transaction-symref ba/zqux
>      ... rebuild the topic ...
>

I guess you mean '--into-name' here? I would skip mentioning this since
it doesn't have any real effect and is perhaps confusing.

>      Then you'd just format your topic above these "preparing the
>      ground" merges, e.g.
>
>      $ git format-patch "HEAD^{/^Merge branch 'ba/zqux'}"..HEAD
>
>      Do not forget to write in the cover letter you did this,
>      including the topics you have in your base on top of 'master'.
>      Then go to (4).
>
>  (4) Make a trial merge of your topic into 'next' and 'seen', e.g.
>
>      $ git checkout --detach 'origin/seen' &&
>        git revert -m 1 <the merge of the previous iteration into seen> &&
>        git merge kn/ref-transaction-symref
>
>      The "revert" is needed if the previous iteration of your topic
>      is already in 'seen' (like in this case).  You could choose to
>      rebuild master..origin/seen from scratch while excluding your
>      previous iteration, which may emulate what happens on my end
>      more closely.
>
>      This trial merge may conflict.  It is primarily to see what
>      conflicts _other_ topics may have with your topic.  In other
>      words, you do not have to depend on to make your topic work on
>      'master'.  It may become the job of the other topic owners to
>      resolve conflicts if your topic goes to 'next' before theirs.
>
>      Make a note on what conflict you saw in the cover letter.  You
>      do not necessarily have to resolve them, but it would be a good
>      opportunity to learn what others are doing in an related area.
>
>      $ git checkout --detach 'origin/next' &&
>        git merge kn/ref-transaction-symref
>
>      This is to see what conflicts your topic has with other topics
>      that are already cooking.  This should not conflict if (3)-2
>      prepared a base on top of updated master plus dependent topics
>      taken from 'next'.  Unless the context is severe (one way to
>      tell is try the same trial merge with your old iteration, which
>      may conflict in a similar way), expect that it will be handled
>      on my end (if it gets unmanageable, I'll ask to rebase when I
>      receive your patches).
>
> Something like the above should be added to the SubmittingPatches
> document (or its successor to cover more advanced topics, perhaps).
>
> Thanks.

The rest of this looks good, I'll cleanup add the appropriate syntax,
merge in your patches [1] and send something soon!

[1]: https://lore.kernel.org/all/20240510165526.1412338-1-gitster@pobox.com/#t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

[Junio C Hamano]

Karthik Nayak <karthik.188@gmail.com> writes:

> I find the '...' always so confusing, I would say suggesting to use
> 'git-merge-base' would be much nicer here.

They are equivalent, except that "..." in

	git checkout --detach A...B


is internal as opposed to

	git checkout --detach $(merge-base A B)

that uses one extra process.  I thought people loath spawning
processes?

>>      Check out the latest 'origin/master' (which may be newer than
>>      the base used by (2)), "merge --no-ff" the topics you newly
>
> For my own understanding, even if we use '--ff' the end result should be
> the same, but using '--no-ff' would ensure that the changes and
> conflicts are isolated to the merge commit, right?

It will make it easy for you to look at 

	git log --first-parent --oneline master..

by hiding the base commits you depend on behind a single merge
commit.

>>      $ git merge --no-ff --into kn/ref-transaction-symref ba/zqux
>>      ... rebuild the topic ...
>
> I guess you mean '--into-name' here? I would skip mentioning this since
> it doesn't have any real effect and is perhaps confusing.

Again this is to help that "one liner first-parent chain" output.