Transaction patch series overview

Transaction patch series overview

[Ronnie Sahlberg]

List, please see here an overview and ordering of the ref transaction
patch series.
These series build on each other and needs to be applied in the order
listed below.


rs/ref-transaction-0
---------------------------
    Early part of the "ref transaction" topic.

    * rs/ref-transaction-0:
      refs.c: change ref_transaction_update() to do error checking and
return status
      refs.c: remove the onerr argument to ref_transaction_commit
      update-ref: use err argument to get error from ref_transaction_commit
      refs.c: make update_ref_write update a strbuf on failure
      refs.c: make ref_update_reject_duplicates take a strbuf argument
for errors
      refs.c: log_ref_write should try to return meaningful errno
      refs.c: make resolve_ref_unsafe set errno to something meaningful on error
      refs.c: commit_packed_refs to return a meaningful errno on failure
      refs.c: make remove_empty_directories always set errno to something sane
      refs.c: verify_lock should set errno to something meaningful
      refs.c: make sure log_ref_setup returns a meaningful errno
      refs.c: add an err argument to repack_without_refs
      lockfile.c: make lock_file return a meaningful errno on failurei
      lockfile.c: add a new public function unable_to_lock_message
      refs.c: add a strbuf argument to ref_transaction_commit for error logging
      refs.c: allow passing NULL to ref_transaction_free
      refs.c: constify the sha arguments for
ref_transaction_create|delete|update
      refs.c: ref_transaction_commit should not free the transaction
      refs.c: remove ref_transaction_rollback

Has been merged into next.



ref-transaction-1 (2014-07-16) 20 commits
-------------------------------------------------------------
Second batch of ref transactions

 - refs.c: make delete_ref use a transaction
 - refs.c: make prune_ref use a transaction to delete the ref
 - refs.c: remove lock_ref_sha1
 - refs.c: remove the update_ref_write function
 - refs.c: remove the update_ref_lock function
 - refs.c: make lock_ref_sha1 static
 - walker.c: use ref transaction for ref updates
 - fast-import.c: use a ref transaction when dumping tags
 - receive-pack.c: use a reference transaction for updating the refs
 - refs.c: change update_ref to use a transaction
 - branch.c: use ref transaction for all ref updates
 - fast-import.c: change update_branch to use ref transactions
 - sequencer.c: use ref transactions for all ref updates
 - commit.c: use ref transactions for updates
 - replace.c: use the ref transaction functions for updates
 - tag.c: use ref transactions when doing updates
 - refs.c: add transaction.status and track OPEN/CLOSED/ERROR
 - refs.c: make ref_transaction_begin take an err argument
 - refs.c: update ref_transaction_delete to check for error and return status
 - refs.c: change ref_transaction_create to do error checking and return status
 (this branch is used by rs/ref-transaction, rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename.)

 The second batch of the transactional ref update series.

Has been merged into pu



rs/ref-transaction (2014-07-17) 12 commits
-----------------------------------------------------------------
 - refs.c: fix handling of badly named refs
 - refs.c: make write_ref_sha1 static
 - fetch.c: change s_update_ref to use a ref transaction
 - refs.c: propagate any errno==ENOTDIR from _commit back to the callers
 - refs.c: pass a skip list to name_conflict_fn
 - refs.c: call lock_ref_sha1_basic directly from commit
 - refs.c: move the check for valid refname to lock_ref_sha1_basic
 - refs.c: pass NULL as *flags to read_ref_full
 - refs.c: pass the ref log message to _create/delete/update instead of _commit
 - refs.c: add an err argument to delete_ref_loose
 - wrapper.c: add a new function unlink_or_msg
 - wrapper.c: simplify warn_if_unremovable
 (this branch is used by rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename; uses
rs/ref-transaction-1.)

The third and final part of the basic ref-transaction work.

Has been merged into pu.




rs/ref-transaction-reflog (2014-07-23) 15 commits
-----------------------------------------------------------------------
 - refs.c: allow deleting refs with a broken sha1
 - refs.c: remove lock_any_ref_for_update
 - refs.c: make unlock_ref/close_ref/commit_ref static
 - refs.c: rename log_ref_setup to create_reflog
 - reflog.c: use a reflog transaction when writing during expire
 - refs.c: allow multiple reflog updates during a single transaction
 - refs.c: only write reflog update if msg is non-NULL
 - refs.c: add a flag to allow reflog updates to truncate the log
 - refs.c: add a transaction function to append a reflog entry
 - lockfile.c: make hold_lock_file_for_append preserve meaningful errno
 - refs.c: add a function to append a reflog entry to a fd
 - refs.c: add a new update_type field to ref_update
 - refs.c: rename the transaction functions
 - refs.c: make ref_transaction_delete a wrapper for ref_transaction_update
 - refs.c: make ref_transaction_create a wrapper to ref_transaction_update
 (this branch is used by rs/ref-transaction-multi and
rs/ref-transaction-rename; uses rs/ref-transaction and
rs/ref-transaction-1.)

This patch series adds support for reflog updates to the transaction subsystem.
Once it has refactored the builtin/reflog.c to use a transaction
instead of accessing the refs and
reflogs directly it allows us to remove
unlock_ref/close_ref/commit_ref/lock_any_ref_for_update
from the public API.

As part of the reflog work I also refactor how reflog creation works
renaming log_ref_setup to create_reflog. These changes allow us then
to reduce the number of places where
we expose the global log_all_ref_updates to the callers.
This leads to a much cleaner api for reflog creation and avoids completely the
"manipulate the global log_all_ref_updates before calling
log_ref_setup" thing that was done
in checkout.c. Now checkout.c can just call create_reflog() and the
right thing will happen.

At the end of the patch series is an unrelated patch for repairing the
"allow deletion of a broken ref" functionality that was lost a while
back. This patch is independent of the reflog work but is placed here
just to avoid creating additional conflicts with this series and
series that follow.

Has been merged to pu.





Following are the patch series that has not yet made its way into pu.




ref-transactions-rename
-----------------------------------
refs.c: allow passing raw git_committer_info as email to _update_reflog
refs.c: return error instead of dying when locking fails during transaction
refs.c: use packed refs when deleting refs during a transaction
refs.c: update rename_ref to use a transaction
refs.c: rollback the lockfile before we die() in repack_without_refs

This series focuses on reworking how rename_ref works and to make it
use an atomic transaction for the rename.
The first three patches lead up to a point where we can now perform
the "delete the old ref and add the new ref" as a single atomic
operation to the packed refs file followed by the fourth patch where
we rework rename_ref() completely to perform the rename and the reflog
updates as a single atomic transaction.

As part of this the rename_ref logic is greatly simplified and we
reach several nice goals:
* no need to disallow rename_ref() if the reflog is a symlink
* no need to rename the reflog via a third filename
* make the rename become completely atomic both for the caller but
also for any external observers.

The fifth patch is independent to the rename work itself but was
discovered during the work for this patch. It could be applied
separately but I did not see any urgency to push it separetely and
left it as a patch in this series.





ref-transactions-req-packed-refs
----------------------------------------------
refs.c: move reflog updates into its own function
refs.c: write updates to packed refs when a transaction has more than one ref
remote.c: use a transaction for deleting refs
refs.c: make repack_without_refs static
refs.c: make the *_packed_refs functions static

These patches expands on the "perform ref operations as a packed refs
file commit" and changes the transaction subsystem to always perform
multi-ref updates as a packed ref commit instead of discreete
operations on loose refs.

The new logic is basically:
IF the transaction only touches a single ref, then just do the normal
loose ref operation as before.
   This is to make sure that the very common "git commit" command will
still always be fast.

IF the transaction touches multiple refs, then
 * first copy/move all affected refs to the packed refs file and
commit, then re-lock the packed
   refs file until the transaction has completed.
 * then delete any affected loose refs files (they are already in the
packed refs file)
 * perform the operation by adding/removing/updating packed refs entries
 * finally, commit the packed refs file

This leads to several new nice properties.
First, the transaction subsystem will now do the "right thing" and
automatically convert any
multi-ref updates to use the packed refs file. This then means that we
no longer need to have
special casing for clone.c to use a special packed_refs api. Clone.c
can now just use a transaction for its "add all refs" and the right
thing will happen.
Similar optimizations can be done for remote.c which now also can just
use a transaction
and know that the right thing will happen.
As we now no longer need to expose the special purpose packed-refs api
to external callers we can now make all the packed refs functions
private to refs.c and shrink the size of the public api.

Note: we still do have packed refs and they still work the same way as
before. It is just that the packed refs are now an internal
optimization for the refs code and no longer part of the public api.

Second, all transaction that affects multiple refs will now also
become atomic for any external observer regardless of whether they
used the packed refs api or not.


The following functions are now static to refs.c and no longer part of
the public api:
 add_packed_ref/lock_packed_refs/commit_packed_refs/rollback_packed_refs/repack_without_refs




ref-transactions-req-strbuf-err
-------------------------------------------
This series just contains various changes internally to refs.c to
expand on the use of a strbuf *err as argument for passing error
messages back to the caller.
Only thing by note here is tgat by doing this to update_ref() we can
finally get rid of the action_on_err enum:

-enum action_on_err {
-       UPDATE_REFS_MSG_ON_ERR,
-       UPDATE_REFS_DIE_ON_ERR,
-       UPDATE_REFS_QUIET_ON_ERR
-};

wooohooo

As part of this patch series I noticed that there were two places in
transport*.c where we were calling update_ref() and passing '0' as
argument instead of UPDATE_REFS_MSG_ON_ERR.
Not a bug and not a change in functionality but it would be nice if
the compiler could warn for this.



ref-transactions-send-pack
---------------------------------------
Various changes to allow a new argument  --atomic-push, and protocol
changes, to allow a client to request/negotiate that a multi-ref push
should be atomic. I.e. apply all changes or none.

Not sent to the list yet.


ref-transactions-send-pack-post
----------------------------------------------
Various other changes to add a strbuf *err argument to error handling.

Not sent to the list yet.



And that concludes the basic ref-transaction stuff.


Once this is in I have a very large patch series that is mainly
rearranging code and to add support for pluggable ref backends.
But that is for when the ref transaction patches are in.



regards
ronnie sahlberg

Re: Transaction patch series overview

[Ronnie Sahlberg]

List, please see here an overview and ordering of the ref transaction
patch series.
These series build on each other and needs to be applied in the order
listed below.

This is an update.



rs/ref-transaction-0
---------------------------
    Early part of the "ref transaction" topic.

    * rs/ref-transaction-0:
      refs.c: change ref_transaction_update() to do error checking and
return status
      refs.c: remove the onerr argument to ref_transaction_commit
      update-ref: use err argument to get error from ref_transaction_commit
      refs.c: make update_ref_write update a strbuf on failure
      refs.c: make ref_update_reject_duplicates take a strbuf argument
for errors
      refs.c: log_ref_write should try to return meaningful errno
      refs.c: make resolve_ref_unsafe set errno to something meaningful on error
      refs.c: commit_packed_refs to return a meaningful errno on failure
      refs.c: make remove_empty_directories always set errno to something sane
      refs.c: verify_lock should set errno to something meaningful
      refs.c: make sure log_ref_setup returns a meaningful errno
      refs.c: add an err argument to repack_without_refs
      lockfile.c: make lock_file return a meaningful errno on failurei
      lockfile.c: add a new public function unable_to_lock_message
      refs.c: add a strbuf argument to ref_transaction_commit for error logging
      refs.c: allow passing NULL to ref_transaction_free
      refs.c: constify the sha arguments for
ref_transaction_create|delete|update
      refs.c: ref_transaction_commit should not free the transaction
      refs.c: remove ref_transaction_rollback

Has been merged into next.



ref-transaction-1 (2014-07-16) 20 commits
-------------------------------------------------------------
Second batch of ref transactions

 - refs.c: make delete_ref use a transaction
 - refs.c: make prune_ref use a transaction to delete the ref
 - refs.c: remove lock_ref_sha1
 - refs.c: remove the update_ref_write function
 - refs.c: remove the update_ref_lock function
 - refs.c: make lock_ref_sha1 static
 - walker.c: use ref transaction for ref updates
 - fast-import.c: use a ref transaction when dumping tags
 - receive-pack.c: use a reference transaction for updating the refs
 - refs.c: change update_ref to use a transaction
 - branch.c: use ref transaction for all ref updates
 - fast-import.c: change update_branch to use ref transactions
 - sequencer.c: use ref transactions for all ref updates
 - commit.c: use ref transactions for updates
 - replace.c: use the ref transaction functions for updates
 - tag.c: use ref transactions when doing updates
 - refs.c: add transaction.status and track OPEN/CLOSED/ERROR
 - refs.c: make ref_transaction_begin take an err argument
 - refs.c: update ref_transaction_delete to check for error and return status
 - refs.c: change ref_transaction_create to do error checking and return status
 (this branch is used by rs/ref-transaction, rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename.)

 The second batch of the transactional ref update series.

Has been merged into pu



rs/ref-transaction (2014-07-17) 12 commits
-----------------------------------------------------------------
 - refs.c: fix handling of badly named refs
 - refs.c: make write_ref_sha1 static
 - fetch.c: change s_update_ref to use a ref transaction
 - refs.c: propagate any errno==ENOTDIR from _commit back to the callers
 - refs.c: pass a skip list to name_conflict_fn
 - refs.c: call lock_ref_sha1_basic directly from commit
 - refs.c: move the check for valid refname to lock_ref_sha1_basic
 - refs.c: pass NULL as *flags to read_ref_full
 - refs.c: pass the ref log message to _create/delete/update instead of _commit
 - refs.c: add an err argument to delete_ref_loose
 - wrapper.c: add a new function unlink_or_msg
 - wrapper.c: simplify warn_if_unremovable
 (this branch is used by rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename; uses
rs/ref-transaction-1.)

The third and final part of the basic ref-transaction work.

Has been merged into pu.




rs/ref-transaction-reflog (2014-07-23) 15 commits
-----------------------------------------------------------------------
 - refs.c: allow deleting refs with a broken sha1
 - refs.c: remove lock_any_ref_for_update
 - refs.c: make unlock_ref/close_ref/commit_ref static
 - refs.c: rename log_ref_setup to create_reflog
 - reflog.c: use a reflog transaction when writing during expire
 - refs.c: allow multiple reflog updates during a single transaction
 - refs.c: only write reflog update if msg is non-NULL
 - refs.c: add a flag to allow reflog updates to truncate the log
 - refs.c: add a transaction function to append a reflog entry
 - lockfile.c: make hold_lock_file_for_append preserve meaningful errno
 - refs.c: add a function to append a reflog entry to a fd
 - refs.c: add a new update_type field to ref_update
 - refs.c: rename the transaction functions
 - refs.c: make ref_transaction_delete a wrapper for ref_transaction_update
 - refs.c: make ref_transaction_create a wrapper to ref_transaction_update
 (this branch is used by rs/ref-transaction-multi and
rs/ref-transaction-rename; uses rs/ref-transaction and
rs/ref-transaction-1.)

This patch series adds support for reflog updates to the transaction subsystem.
Once it has refactored the builtin/reflog.c to use a transaction
instead of accessing the refs and
reflogs directly it allows us to remove
unlock_ref/close_ref/commit_ref/lock_any_ref_for_update
from the public API.

As part of the reflog work I also refactor how reflog creation works
renaming log_ref_setup to create_reflog. These changes allow us then
to reduce the number of places where
we expose the global log_all_ref_updates to the callers.
This leads to a much cleaner api for reflog creation and avoids completely the
"manipulate the global log_all_ref_updates before calling
log_ref_setup" thing that was done
in checkout.c. Now checkout.c can just call create_reflog() and the
right thing will happen.

At the end of the patch series is an unrelated patch for repairing the
"allow deletion of a broken ref" functionality that was lost a while
back. This patch is independent of the reflog work but is placed here
just to avoid creating additional conflicts with this series and
series that follow.

Has been merged to pu.


ref-transactions-rename
-----------------------------------
refs.c: allow passing raw git_committer_info as email to _update_reflog
refs.c: return error instead of dying when locking fails during transaction
refs.c: use packed refs when deleting refs during a transaction
refs.c: update rename_ref to use a transaction
refs.c: rollback the lockfile before we die() in repack_without_refs

This series focuses on reworking how rename_ref works and to make it
use an atomic transaction for the rename.
The first three patches lead up to a point where we can now perform
the "delete the old ref and add the new ref" as a single atomic
operation to the packed refs file followed by the fourth patch where
we rework rename_ref() completely to perform the rename and the reflog
updates as a single atomic transaction.

As part of this the rename_ref logic is greatly simplified and we
reach several nice goals:
* no need to disallow rename_ref() if the reflog is a symlink
* no need to rename the reflog via a third filename
* make the rename become completely atomic both for the caller but
also for any external observers.

The fifth patch is independent to the rename work itself but was
discovered during the work for this patch. It could be applied
separately but I did not see any urgency to push it separetely and
left it as a patch in this series.

Has been merged into pu.



ref-transactions-req-packed-refs
----------------------------------------------
refs.c: move reflog updates into its own function
refs.c: write updates to packed refs when a transaction has more than one ref
remote.c: use a transaction for deleting refs
refs.c: make repack_without_refs static
refs.c: make the *_packed_refs functions static

These patches expands on the "perform ref operations as a packed refs
file commit" and changes the transaction subsystem to always perform
multi-ref updates as a packed ref commit instead of discreete
operations on loose refs.

The new logic is basically:
IF the transaction only touches a single ref, then just do the normal
loose ref operation as before.
   This is to make sure that the very common "git commit" command will
still always be fast.

IF the transaction touches multiple refs, then
 * first copy/move all affected refs to the packed refs file and
commit, then re-lock the packed
   refs file until the transaction has completed.
 * then delete any affected loose refs files (they are already in the
packed refs file)
 * perform the operation by adding/removing/updating packed refs entries
 * finally, commit the packed refs file

This leads to several new nice properties.
First, the transaction subsystem will now do the "right thing" and
automatically convert any
multi-ref updates to use the packed refs file. This then means that we
no longer need to have
special casing for clone.c to use a special packed_refs api. Clone.c
can now just use a transaction for its "add all refs" and the right
thing will happen.
Similar optimizations can be done for remote.c which now also can just
use a transaction
and know that the right thing will happen.
As we now no longer need to expose the special purpose packed-refs api
to external callers we can now make all the packed refs functions
private to refs.c and shrink the size of the public api.

Note: we still do have packed refs and they still work the same way as
before. It is just that the packed refs are now an internal
optimization for the refs code and no longer part of the public api.

Second, all transaction that affects multiple refs will now also
become atomic for any external observer regardless of whether they
used the packed refs api or not.


The following functions are now static to refs.c and no longer part of
the public api:
 add_packed_ref/lock_packed_refs/commit_packed_refs/rollback_packed_refs/repack_without_refs

Has been merged into pu.



Following are the patch series that has not yet made its way into pu.


ref-transactions-req-strbuf-err
-------------------------------------------
This series just contains various changes internally to refs.c to
expand on the use of a strbuf *err as argument for passing error
messages back to the caller.
Only thing by note here is tgat by doing this to update_ref() we can
finally get rid of the action_on_err enum:

-enum action_on_err {
-       UPDATE_REFS_MSG_ON_ERR,
-       UPDATE_REFS_DIE_ON_ERR,
-       UPDATE_REFS_QUIET_ON_ERR
-};

wooohooo

As part of this patch series I noticed that there were two places in
transport*.c where we were calling update_ref() and passing '0' as
argument instead of UPDATE_REFS_MSG_ON_ERR.
Not a bug and not a change in functionality but it would be nice if
the compiler could warn for this.



ref-transactions-send-pack
---------------------------------------
Various changes to allow a new argument  --atomic-push, and protocol
changes, to allow a client to request/negotiate that a multi-ref push
should be atomic. I.e. apply all changes or none.



ref-transactions-send-pack-post
----------------------------------------------
Various other changes to add a strbuf *err argument to error handling.

Not sent to the list yet.



And that concludes the basic ref-transaction stuff.


Once this is in I have a very large patch series that is mainly
rearranging code and to add support for pluggable ref backends.
But that is for when the ref transaction patches are in.



regards

Re: Transaction patch series overview

[Ronnie Sahlberg]

List, please see here an overview and ordering of the ref transaction
patch series.
These series build on each other and needs to be applied in the order
listed below.

This is an update.




rs/ref-transaction-0
---------------------------
    Early part of the "ref transaction" topic.

    * rs/ref-transaction-0:
      refs.c: change ref_transaction_update() to do error checking and
return status
      refs.c: remove the onerr argument to ref_transaction_commit
      update-ref: use err argument to get error from ref_transaction_commit
      refs.c: make update_ref_write update a strbuf on failure
      refs.c: make ref_update_reject_duplicates take a strbuf argument
for errors
      refs.c: log_ref_write should try to return meaningful errno
      refs.c: make resolve_ref_unsafe set errno to something meaningful on error
      refs.c: commit_packed_refs to return a meaningful errno on failure
      refs.c: make remove_empty_directories always set errno to something sane
      refs.c: verify_lock should set errno to something meaningful
      refs.c: make sure log_ref_setup returns a meaningful errno
      refs.c: add an err argument to repack_without_refs
      lockfile.c: make lock_file return a meaningful errno on failurei
      lockfile.c: add a new public function unable_to_lock_message
      refs.c: add a strbuf argument to ref_transaction_commit for error logging
      refs.c: allow passing NULL to ref_transaction_free
      refs.c: constify the sha arguments for
ref_transaction_create|delete|update
      refs.c: ref_transaction_commit should not free the transaction
      refs.c: remove ref_transaction_rollback

Has been merged into next.



ref-transaction-1 (2014-07-16) 20 commits
-------------------------------------------------------------
Second batch of ref transactions

 - refs.c: make delete_ref use a transaction
 - refs.c: make prune_ref use a transaction to delete the ref
 - refs.c: remove lock_ref_sha1
 - refs.c: remove the update_ref_write function
 - refs.c: remove the update_ref_lock function
 - refs.c: make lock_ref_sha1 static
 - walker.c: use ref transaction for ref updates
 - fast-import.c: use a ref transaction when dumping tags
 - receive-pack.c: use a reference transaction for updating the refs
 - refs.c: change update_ref to use a transaction
 - branch.c: use ref transaction for all ref updates
 - fast-import.c: change update_branch to use ref transactions
 - sequencer.c: use ref transactions for all ref updates
 - commit.c: use ref transactions for updates
 - replace.c: use the ref transaction functions for updates
 - tag.c: use ref transactions when doing updates
 - refs.c: add transaction.status and track OPEN/CLOSED/ERROR
 - refs.c: make ref_transaction_begin take an err argument
 - refs.c: update ref_transaction_delete to check for error and return status
 - refs.c: change ref_transaction_create to do error checking and return status
 (this branch is used by rs/ref-transaction, rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename.)

 The second batch of the transactional ref update series.

Has been merged into pu



rs/ref-transaction (2014-07-17) 12 commits
-----------------------------------------------------------------
 - refs.c: fix handling of badly named refs
 - refs.c: make write_ref_sha1 static
 - fetch.c: change s_update_ref to use a ref transaction
 - refs.c: propagate any errno==ENOTDIR from _commit back to the callers
 - refs.c: pass a skip list to name_conflict_fn
 - refs.c: call lock_ref_sha1_basic directly from commit
 - refs.c: move the check for valid refname to lock_ref_sha1_basic
 - refs.c: pass NULL as *flags to read_ref_full
 - refs.c: pass the ref log message to _create/delete/update instead of _commit
 - refs.c: add an err argument to delete_ref_loose
 - wrapper.c: add a new function unlink_or_msg
 - wrapper.c: simplify warn_if_unremovable
 (this branch is used by rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename; uses
rs/ref-transaction-1.)

The third and final part of the basic ref-transaction work.

Has been merged into pu.




rs/ref-transaction-reflog (2014-07-23) 15 commits
-----------------------------------------------------------------------
 - refs.c: allow deleting refs with a broken sha1
 - refs.c: remove lock_any_ref_for_update
 - refs.c: make unlock_ref/close_ref/commit_ref static
 - refs.c: rename log_ref_setup to create_reflog
 - reflog.c: use a reflog transaction when writing during expire
 - refs.c: allow multiple reflog updates during a single transaction
 - refs.c: only write reflog update if msg is non-NULL
 - refs.c: add a flag to allow reflog updates to truncate the log
 - refs.c: add a transaction function to append a reflog entry
 - lockfile.c: make hold_lock_file_for_append preserve meaningful errno
 - refs.c: add a function to append a reflog entry to a fd
 - refs.c: add a new update_type field to ref_update
 - refs.c: rename the transaction functions
 - refs.c: make ref_transaction_delete a wrapper for ref_transaction_update
 - refs.c: make ref_transaction_create a wrapper to ref_transaction_update
 (this branch is used by rs/ref-transaction-multi and
rs/ref-transaction-rename; uses rs/ref-transaction and
rs/ref-transaction-1.)

This patch series adds support for reflog updates to the transaction subsystem.
Once it has refactored the builtin/reflog.c to use a transaction
instead of accessing the refs and
reflogs directly it allows us to remove
unlock_ref/close_ref/commit_ref/lock_any_ref_for_update
from the public API.

As part of the reflog work I also refactor how reflog creation works
renaming log_ref_setup to create_reflog. These changes allow us then
to reduce the number of places where
we expose the global log_all_ref_updates to the callers.
This leads to a much cleaner api for reflog creation and avoids completely the
"manipulate the global log_all_ref_updates before calling
log_ref_setup" thing that was done
in checkout.c. Now checkout.c can just call create_reflog() and the
right thing will happen.

At the end of the patch series is an unrelated patch for repairing the
"allow deletion of a broken ref" functionality that was lost a while
back. This patch is independent of the reflog work but is placed here
just to avoid creating additional conflicts with this series and
series that follow.

Has been merged to pu.


ref-transactions-rename
-----------------------------------
refs.c: allow passing raw git_committer_info as email to _update_reflog
refs.c: return error instead of dying when locking fails during transaction
refs.c: use packed refs when deleting refs during a transaction
refs.c: update rename_ref to use a transaction
refs.c: rollback the lockfile before we die() in repack_without_refs

This series focuses on reworking how rename_ref works and to make it
use an atomic transaction for the rename.
The first three patches lead up to a point where we can now perform
the "delete the old ref and add the new ref" as a single atomic
operation to the packed refs file followed by the fourth patch where
we rework rename_ref() completely to perform the rename and the reflog
updates as a single atomic transaction.

As part of this the rename_ref logic is greatly simplified and we
reach several nice goals:
* no need to disallow rename_ref() if the reflog is a symlink
* no need to rename the reflog via a third filename
* make the rename become completely atomic both for the caller but
also for any external observers.

The fifth patch is independent to the rename work itself but was
discovered during the work for this patch. It could be applied
separately but I did not see any urgency to push it separetely and
left it as a patch in this series.

Has been merged into pu.



ref-transactions-req-packed-refs
----------------------------------------------
refs.c: move reflog updates into its own function
refs.c: write updates to packed refs when a transaction has more than one ref
remote.c: use a transaction for deleting refs
refs.c: make repack_without_refs static
refs.c: make the *_packed_refs functions static

These patches expands on the "perform ref operations as a packed refs
file commit" and changes the transaction subsystem to always perform
multi-ref updates as a packed ref commit instead of discreete
operations on loose refs.

The new logic is basically:
IF the transaction only touches a single ref, then just do the normal
loose ref operation as before.
   This is to make sure that the very common "git commit" command will
still always be fast.

IF the transaction touches multiple refs, then
 * first copy/move all affected refs to the packed refs file and
commit, then re-lock the packed
   refs file until the transaction has completed.
 * then delete any affected loose refs files (they are already in the
packed refs file)
 * perform the operation by adding/removing/updating packed refs entries
 * finally, commit the packed refs file

This leads to several new nice properties.
First, the transaction subsystem will now do the "right thing" and
automatically convert any
multi-ref updates to use the packed refs file. This then means that we
no longer need to have
special casing for clone.c to use a special packed_refs api. Clone.c
can now just use a transaction for its "add all refs" and the right
thing will happen.
Similar optimizations can be done for remote.c which now also can just
use a transaction
and know that the right thing will happen.
As we now no longer need to expose the special purpose packed-refs api
to external callers we can now make all the packed refs functions
private to refs.c and shrink the size of the public api.

Note: we still do have packed refs and they still work the same way as
before. It is just that the packed refs are now an internal
optimization for the refs code and no longer part of the public api.

Second, all transaction that affects multiple refs will now also
become atomic for any external observer regardless of whether they
used the packed refs api or not.


The following functions are now static to refs.c and no longer part of
the public api:
 add_packed_ref/lock_packed_refs/commit_packed_refs/rollback_packed_refs/repack_without_refs

Has been merged into pu.



Following are the patch series that has not yet made its way into pu.


ref-transactions-req-strbuf-err
-------------------------------------------
This series just contains various changes internally to refs.c to
expand on the use of a strbuf *err as argument for passing error
messages back to the caller.
Only thing by note here is tgat by doing this to update_ref() we can
finally get rid of the action_on_err enum:

-enum action_on_err {
-       UPDATE_REFS_MSG_ON_ERR,
-       UPDATE_REFS_DIE_ON_ERR,
-       UPDATE_REFS_QUIET_ON_ERR
-};

wooohooo

As part of this patch series I noticed that there were two places in
transport*.c where we were calling update_ref() and passing '0' as
argument instead of UPDATE_REFS_MSG_ON_ERR.
Not a bug and not a change in functionality but it would be nice if
the compiler could warn for this.

Sent to the list.


ref-transactions-send-pack
---------------------------------------
Various changes to allow a new argument  --atomic-push, and protocol
changes, to allow a client to request/negotiate that a multi-ref push
should be atomic. I.e. apply all changes or none.

Sent to the list.


backend-struct-db
--------------------------
Various changes that mainly moves functions between files to split the
current refs.c file into functions that are backend agnostic and the
functions that implement the files based ref backend.
It finally adds a backend structure and the methods that define a
backend datastore for refs.

This series concludes the work to define pluggable backends for refs.
At this stage it will now be possible to start building and adding new
types of ref backends to git.

Sent to the list




regards
ronnie sahlberg

Re: Transaction patch series overview

[Ronnie Sahlberg]

List, please see here an overview and ordering of the ref transaction
patch series.
These series build on each other and needs to be applied in the order
listed below.

This is an update.




rs/ref-transaction-0
---------------------------
    Early part of the "ref transaction" topic.

    * rs/ref-transaction-0:
      refs.c: change ref_transaction_update() to do error checking and
return status
      refs.c: remove the onerr argument to ref_transaction_commit
      update-ref: use err argument to get error from ref_transaction_commit
      refs.c: make update_ref_write update a strbuf on failure
      refs.c: make ref_update_reject_duplicates take a strbuf argument
for errors
      refs.c: log_ref_write should try to return meaningful errno
      refs.c: make resolve_ref_unsafe set errno to something meaningful on error
      refs.c: commit_packed_refs to return a meaningful errno on failure
      refs.c: make remove_empty_directories always set errno to something sane
      refs.c: verify_lock should set errno to something meaningful
      refs.c: make sure log_ref_setup returns a meaningful errno
      refs.c: add an err argument to repack_without_refs
      lockfile.c: make lock_file return a meaningful errno on failurei
      lockfile.c: add a new public function unable_to_lock_message
      refs.c: add a strbuf argument to ref_transaction_commit for error logging
      refs.c: allow passing NULL to ref_transaction_free
      refs.c: constify the sha arguments for
ref_transaction_create|delete|update
      refs.c: ref_transaction_commit should not free the transaction
      refs.c: remove ref_transaction_rollback

Has been merged into next.



ref-transaction-1 (2014-07-16) 20 commits
-------------------------------------------------------------
Second batch of ref transactions

 - refs.c: make delete_ref use a transaction
 - refs.c: make prune_ref use a transaction to delete the ref
 - refs.c: remove lock_ref_sha1
 - refs.c: remove the update_ref_write function
 - refs.c: remove the update_ref_lock function
 - refs.c: make lock_ref_sha1 static
 - walker.c: use ref transaction for ref updates
 - fast-import.c: use a ref transaction when dumping tags
 - receive-pack.c: use a reference transaction for updating the refs
 - refs.c: change update_ref to use a transaction
 - branch.c: use ref transaction for all ref updates
 - fast-import.c: change update_branch to use ref transactions
 - sequencer.c: use ref transactions for all ref updates
 - commit.c: use ref transactions for updates
 - replace.c: use the ref transaction functions for updates
 - tag.c: use ref transactions when doing updates
 - refs.c: add transaction.status and track OPEN/CLOSED/ERROR
 - refs.c: make ref_transaction_begin take an err argument
 - refs.c: update ref_transaction_delete to check for error and return status
 - refs.c: change ref_transaction_create to do error checking and return status
 (this branch is used by rs/ref-transaction, rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename.)

 The second batch of the transactional ref update series.

Has been merged into pu



rs/ref-transaction (2014-07-17) 12 commits
-----------------------------------------------------------------
 - refs.c: fix handling of badly named refs
 - refs.c: make write_ref_sha1 static
 - fetch.c: change s_update_ref to use a ref transaction
 - refs.c: propagate any errno==ENOTDIR from _commit back to the callers
 - refs.c: pass a skip list to name_conflict_fn
 - refs.c: call lock_ref_sha1_basic directly from commit
 - refs.c: move the check for valid refname to lock_ref_sha1_basic
 - refs.c: pass NULL as *flags to read_ref_full
 - refs.c: pass the ref log message to _create/delete/update instead of _commit
 - refs.c: add an err argument to delete_ref_loose
 - wrapper.c: add a new function unlink_or_msg
 - wrapper.c: simplify warn_if_unremovable
 (this branch is used by rs/ref-transaction-multi,
rs/ref-transaction-reflog and rs/ref-transaction-rename; uses
rs/ref-transaction-1.)

The third and final part of the basic ref-transaction work.

Has been merged into pu.




rs/ref-transaction-reflog (2014-07-23) 15 commits
-----------------------------------------------------------------------
 - refs.c: allow deleting refs with a broken sha1
 - refs.c: remove lock_any_ref_for_update
 - refs.c: make unlock_ref/close_ref/commit_ref static
 - refs.c: rename log_ref_setup to create_reflog
 - reflog.c: use a reflog transaction when writing during expire
 - refs.c: allow multiple reflog updates during a single transaction
 - refs.c: only write reflog update if msg is non-NULL
 - refs.c: add a flag to allow reflog updates to truncate the log
 - refs.c: add a transaction function to append a reflog entry
 - lockfile.c: make hold_lock_file_for_append preserve meaningful errno
 - refs.c: add a function to append a reflog entry to a fd
 - refs.c: add a new update_type field to ref_update
 - refs.c: rename the transaction functions
 - refs.c: make ref_transaction_delete a wrapper for ref_transaction_update
 - refs.c: make ref_transaction_create a wrapper to ref_transaction_update
 (this branch is used by rs/ref-transaction-multi and
rs/ref-transaction-rename; uses rs/ref-transaction and
rs/ref-transaction-1.)

This patch series adds support for reflog updates to the transaction subsystem.
Once it has refactored the builtin/reflog.c to use a transaction
instead of accessing the refs and
reflogs directly it allows us to remove
unlock_ref/close_ref/commit_ref/lock_any_ref_for_update
from the public API.

As part of the reflog work I also refactor how reflog creation works
renaming log_ref_setup to create_reflog. These changes allow us then
to reduce the number of places where
we expose the global log_all_ref_updates to the callers.
This leads to a much cleaner api for reflog creation and avoids completely the
"manipulate the global log_all_ref_updates before calling
log_ref_setup" thing that was done
in checkout.c. Now checkout.c can just call create_reflog() and the
right thing will happen.

At the end of the patch series is an unrelated patch for repairing the
"allow deletion of a broken ref" functionality that was lost a while
back. This patch is independent of the reflog work but is placed here
just to avoid creating additional conflicts with this series and
series that follow.

Has been merged to pu.


ref-transactions-rename
-----------------------------------
refs.c: allow passing raw git_committer_info as email to _update_reflog
refs.c: return error instead of dying when locking fails during transaction
refs.c: use packed refs when deleting refs during a transaction
refs.c: update rename_ref to use a transaction
refs.c: rollback the lockfile before we die() in repack_without_refs

This series focuses on reworking how rename_ref works and to make it
use an atomic transaction for the rename.
The first three patches lead up to a point where we can now perform
the "delete the old ref and add the new ref" as a single atomic
operation to the packed refs file followed by the fourth patch where
we rework rename_ref() completely to perform the rename and the reflog
updates as a single atomic transaction.

As part of this the rename_ref logic is greatly simplified and we
reach several nice goals:
* no need to disallow rename_ref() if the reflog is a symlink
* no need to rename the reflog via a third filename
* make the rename become completely atomic both for the caller but
also for any external observers.

The fifth patch is independent to the rename work itself but was
discovered during the work for this patch. It could be applied
separately but I did not see any urgency to push it separetely and
left it as a patch in this series.

Has been merged into pu.



ref-transactions-req-packed-refs
----------------------------------------------
refs.c: move reflog updates into its own function
refs.c: write updates to packed refs when a transaction has more than one ref
remote.c: use a transaction for deleting refs
refs.c: make repack_without_refs static
refs.c: make the *_packed_refs functions static

These patches expands on the "perform ref operations as a packed refs
file commit" and changes the transaction subsystem to always perform
multi-ref updates as a packed ref commit instead of discreete
operations on loose refs.

The new logic is basically:
IF the transaction only touches a single ref, then just do the normal
loose ref operation as before.
   This is to make sure that the very common "git commit" command will
still always be fast.

IF the transaction touches multiple refs, then
 * first copy/move all affected refs to the packed refs file and
commit, then re-lock the packed
   refs file until the transaction has completed.
 * then delete any affected loose refs files (they are already in the
packed refs file)
 * perform the operation by adding/removing/updating packed refs entries
 * finally, commit the packed refs file

This leads to several new nice properties.
First, the transaction subsystem will now do the "right thing" and
automatically convert any
multi-ref updates to use the packed refs file. This then means that we
no longer need to have
special casing for clone.c to use a special packed_refs api. Clone.c
can now just use a transaction for its "add all refs" and the right
thing will happen.
Similar optimizations can be done for remote.c which now also can just
use a transaction
and know that the right thing will happen.
As we now no longer need to expose the special purpose packed-refs api
to external callers we can now make all the packed refs functions
private to refs.c and shrink the size of the public api.

Note: we still do have packed refs and they still work the same way as
before. It is just that the packed refs are now an internal
optimization for the refs code and no longer part of the public api.

Second, all transaction that affects multiple refs will now also
become atomic for any external observer regardless of whether they
used the packed refs api or not.


The following functions are now static to refs.c and no longer part of
the public api:
 add_packed_ref/lock_packed_refs/commit_packed_refs/rollback_packed_refs/repack_without_refs

Has been merged into pu.



Following are the patch series that has not yet made its way into pu.


ref-transactions-req-strbuf-err
-------------------------------------------
This series just contains various changes internally to refs.c to
expand on the use of a strbuf *err as argument for passing error
messages back to the caller.
Only thing by note here is tgat by doing this to update_ref() we can
finally get rid of the action_on_err enum:

-enum action_on_err {
-       UPDATE_REFS_MSG_ON_ERR,
-       UPDATE_REFS_DIE_ON_ERR,
-       UPDATE_REFS_QUIET_ON_ERR
-};

wooohooo

As part of this patch series I noticed that there were two places in
transport*.c where we were calling update_ref() and passing '0' as
argument instead of UPDATE_REFS_MSG_ON_ERR.
Not a bug and not a change in functionality but it would be nice if
the compiler could warn for this.

Sent to the list.


ref-transactions-send-pack
---------------------------------------
Various changes to allow a new argument  --atomic-push, and protocol
changes, to allow a client to request/negotiate that a multi-ref push
should be atomic. I.e. apply all changes or none.

Sent to the list.


backend-struct-db
--------------------------
Various changes that mainly moves functions between files to split the
current refs.c file into functions that are backend agnostic and the
functions that implement the files based ref backend.
It finally adds a backend structure and the methods that define a
backend datastore for refs.

This series concludes the work to define pluggable backends for refs.
At this stage it will now be possible to start building and adding new
types of ref backends to git.

Sent to the list





In addition to these patches, there is one more patchseries with 4 patches at :
https://github.com/rsahlberg/git/tree/backend-struct-db-2

This series is for once the previous transaction changes are in and
this series will add
a new backend for refs: refs-be-db.c which offloads all refs access to
an external database daemon.
An example reference implementation for an external daemon is also
provided and can be used as basis for
creating a daemon to plug into, say, a SQL database or similar.


regards
ronnie sahlberg

Re: Transaction patch series overview

[Junio C Hamano]

Ronnie Sahlberg <sahlberg@google.com> writes:

> List, please see here an overview and ordering of the ref transaction
> patch series.
> These series build on each other and needs to be applied in the order
> listed below.
>
> This is an update.
>
> rs/ref-transaction-0
> ---------------------------
>     Early part of the "ref transaction" topic.
>
>     * rs/ref-transaction-0:
>       refs.c: change ref_transaction_update() to do error checking and
> return status
>       refs.c: remove the onerr argument to ref_transaction_commit
>       update-ref: use err argument to get error from ref_transaction_commit
>       refs.c: make update_ref_write update a strbuf on failure
>       refs.c: make ref_update_reject_duplicates take a strbuf argument
> for errors
>       refs.c: log_ref_write should try to return meaningful errno
>       refs.c: make resolve_ref_unsafe set errno to something meaningful on error
>       refs.c: commit_packed_refs to return a meaningful errno on failure
>       refs.c: make remove_empty_directories always set errno to something sane
>       refs.c: verify_lock should set errno to something meaningful
>       refs.c: make sure log_ref_setup returns a meaningful errno
>       refs.c: add an err argument to repack_without_refs
>       lockfile.c: make lock_file return a meaningful errno on failurei
>       lockfile.c: add a new public function unable_to_lock_message
>       refs.c: add a strbuf argument to ref_transaction_commit for error logging
>       refs.c: allow passing NULL to ref_transaction_free
>       refs.c: constify the sha arguments for
> ref_transaction_create|delete|update
>       refs.c: ref_transaction_commit should not free the transaction
>       refs.c: remove ref_transaction_rollback
>
> Has been merged into next.

I gave a quick re-read-thru on this and did not find anything
questionable, nor recall any issue pending.  Unless somebody
objects in the coming few days, let's move it to 'master' as part of
the first batch for 2.2.

> ref-transaction-1 (2014-07-16) 20 commits
> ...
>  The second batch of the transactional ref update series.
>
> Has been merged into pu
>
> rs/ref-transaction (2014-07-17) 12 commits
> ...
> The third and final part of the basic ref-transaction work.
> 
> Has been merged into pu.

I'll re-read these two before deciding to merge them to 'next'.  Any
pending issues from the previous discussions?

> This series is for once the previous transaction changes are in and
> this series will add
> a new backend for refs: refs-be-db.c which offloads all refs access to
> an external database daemon.
> An example reference implementation for an external daemon is also
> provided and can be used as basis for
> creating a daemon to plug into, say, a SQL database or similar.

A filesystem based backend/external daemon that is compatible with
the traditional on-disk format might be another fun demonstration.

Looking forward to shepherding the whole thing ahead ;-)  The most
fun part awaits us but there is a long way.

Thanks.

Re: Transaction patch series overview

[Jonathan Nieder]

Hi,

Ronnie Sahlberg wrote:

> List, please see here an overview and ordering of the ref transaction
> patch series.

Thanks much for this.

[...]
> rs/ref-transaction-0
[...]
> Has been merged into next.

This is even part of "master" now, so if people have review comments
then they can make them most easily by submitting new patches on top.

> ref-transaction-1 (2014-07-16) 20 commits
> -------------------------------------------------------------
> Second batch of ref transactions
>
>  - refs.c: make delete_ref use a transaction
>  - refs.c: make prune_ref use a transaction to delete the ref
>  - refs.c: remove lock_ref_sha1
>  - refs.c: remove the update_ref_write function
>  - refs.c: remove the update_ref_lock function
>  - refs.c: make lock_ref_sha1 static
>  - walker.c: use ref transaction for ref updates
>  - fast-import.c: use a ref transaction when dumping tags
>  - receive-pack.c: use a reference transaction for updating the refs
>  - refs.c: change update_ref to use a transaction
>  - branch.c: use ref transaction for all ref updates
>  - fast-import.c: change update_branch to use ref transactions
>  - sequencer.c: use ref transactions for all ref updates
>  - commit.c: use ref transactions for updates
>  - replace.c: use the ref transaction functions for updates
>  - tag.c: use ref transactions when doing updates
>  - refs.c: add transaction.status and track OPEN/CLOSED/ERROR
>  - refs.c: make ref_transaction_begin take an err argument
>  - refs.c: update ref_transaction_delete to check for error and return status
>  - refs.c: change ref_transaction_create to do error checking and return status
>  (this branch is used by rs/ref-transaction, rs/ref-transaction-multi,
> rs/ref-transaction-reflog and rs/ref-transaction-rename.)
>
>  The second batch of the transactional ref update series.
>
> Has been merged into pu

Last reviewed at
http://thread.gmane.org/gmane.comp.version-control.git/252226, if
I am using gmane search correctly.

Are there any pending questions for this part?

I've having trouble keeping track of how patches change, which patches
have been reviewed and which haven't, unaddressed comments, and so on,
so as an experiment I've pushed this part of the series to the Gerrit
server at

 https://code-review.googlesource.com/#/q/topic:ref-transaction-1

It's running a copy of gerrit code review
(https://code.google.com/p/gerrit).

Maybe this can be useful as a way of keeping track of the state of
long and long-lived series like this one.  It works something like
this:

Preparation
-----------
Get a password from https://code-review.googlesource.com/new-password
and put it in netrc.

Install the hook from
https://code-review.googlesource.com/tools/hooks/commit-msg to
.git/hooks/commit-msg.  This automatically populates a Change-Id
line in the commit message if none is present so gerrit can
tell when you are uploading a new version of an existing patch.

(The commit-msg hook can be annoying when not working with gerrit
code review.  To turn it off, use 'git config gerrit.createChangeId false'.)

Uploading patches
-----------------
Write a patch series against 'maint' or 'master' as usual, then push:

	git push https://code.googlesource.com/git \
		HEAD:refs/for/master; # or refs/for/maint

There can be a parameter '%topic=<name>' after the refs/for/<branch>
(e.g., refs/for/master%topic=ref-transaction-1) if the series should
be labelled with a topic name, or that can be set in the web UI or
using the HTTP API after the fact.

Commenting on patches
---------------------
Visit https://code-review.googlesource.com.  Patches can be found
under "All" -> "Open".  Patches you're involved with somehow are
at "My" -> "Changes".

To prepare inline comments, choose a file, highlight the text to comment
on or click a line number, and comment.

To publish comments, go back up to the change overview screen (using
the up arrow button or by pressing "u"), click "Reply", and say
something.

'?' brings up keyboard shortcuts.

Downloading patches
-------------------
In the change overview screen, there's a 'Download' menu in the
top-right corner with commands to download the patch.

Revising patches
----------------
After modifying a patch locally using the usual tools such as rebase
--interactive, push again:

	git push https://code.googlesource.com/git \
		HEAD:refs/for/master; # or refs/for/maint

When a patch seems polished
---------------------------
Get rid of the Change-Id lines --- they shouldn't be needed any
more.  Send the patches or a request to pull from a public git
repository, as usual.  A link (in the top-left corner where it says
'Change 1000', the "1000" is a link to the change) can be helpful
for letting people on-list see how the patch got into the form it's
in today.

Maybe it can be useful.

Thanks,
Jonathan

Re: Transaction patch series overview

[Jonathan Nieder]

Jonathan Nieder wrote:
> Ronnie Sahlberg wrote:

>> ref-transaction-1 (2014-07-16) 20 commits
>> -------------------------------------------------------------
>> Second batch of ref transactions
>>
>>  - refs.c: make delete_ref use a transaction
>>  - refs.c: make prune_ref use a transaction to delete the ref
>>  - refs.c: remove lock_ref_sha1
>>  - refs.c: remove the update_ref_write function
>>  - refs.c: remove the update_ref_lock function
>>  - refs.c: make lock_ref_sha1 static
>>  - walker.c: use ref transaction for ref updates
>>  - fast-import.c: use a ref transaction when dumping tags
>>  - receive-pack.c: use a reference transaction for updating the refs
>>  - refs.c: change update_ref to use a transaction
>>  - branch.c: use ref transaction for all ref updates
>>  - fast-import.c: change update_branch to use ref transactions
>>  - sequencer.c: use ref transactions for all ref updates
>>  - commit.c: use ref transactions for updates
>>  - replace.c: use the ref transaction functions for updates
>>  - tag.c: use ref transactions when doing updates
>>  - refs.c: add transaction.status and track OPEN/CLOSED/ERROR
>>  - refs.c: make ref_transaction_begin take an err argument
>>  - refs.c: update ref_transaction_delete to check for error and return status
>>  - refs.c: change ref_transaction_create to do error checking and return status
>>  (this branch is used by rs/ref-transaction, rs/ref-transaction-multi,
>> rs/ref-transaction-reflog and rs/ref-transaction-rename.)
[...]
> I've having trouble keeping track of how patches change, which patches
> have been reviewed and which haven't, unaddressed comments, and so on,
> so as an experiment I've pushed this part of the series to the Gerrit
> server at
>
>  https://code-review.googlesource.com/#/q/topic:ref-transaction-1

Outcome of the experiment: patches gained some minor changes and then

 1-12
	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

 13
	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

 14
	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

 15-16
	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

 17
	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>

 18
	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

 19
	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

 20
	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

I found the web UI helpful in seeing how each patch evolved since I
last looked at it.  Interdiff relative to the version in pu is below.
I'm still hoping for a tweak in response to a minor comment and then I
can put up a copy of the updated series to pull.

The next series from Ronnie's collection is available at
https://code-review.googlesource.com/#/q/topic:ref-transaction in case
someone wants a fresh series to look at.

diff --git a/branch.c b/branch.c
index c1eae00..37ac555 100644
--- a/branch.c
+++ b/branch.c
@@ -305,6 +305,7 @@ void create_branch(const char *head,
 		    ref_transaction_commit(transaction, msg, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
+		strbuf_release(&err);
 	}
 
 	if (real_ref && track)
diff --git a/builtin/commit.c b/builtin/commit.c
index 668fa6a..9bf1003 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1765,8 +1765,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", sha1,
-				   current_head ?
-				   current_head->object.sha1 : NULL,
+				   current_head
+				   ? current_head->object.sha1 : NULL,
 				   0, !!current_head, &err) ||
 	    ref_transaction_commit(transaction, sb.buf, &err)) {
 		rollback_index_files();
@@ -1801,5 +1801,6 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	if (!quiet)
 		print_summary(prefix, sha1, !current_head);
 
+	strbuf_release(&err);
 	return 0;
 }
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 91099ad..224fadc 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -194,7 +194,7 @@ static void write_head_info(void)
 
 struct command {
 	struct command *next;
-	char *error_string;
+	const char *error_string;
 	unsigned int skip_update:1,
 		     did_not_exist:1;
 	int index;
@@ -468,7 +468,7 @@ static int update_shallow_ref(struct command *cmd, struct shallow_info *si)
 	return 0;
 }
 
-static char *update(struct command *cmd, struct shallow_info *si)
+static const char *update(struct command *cmd, struct shallow_info *si)
 {
 	const char *name = cmd->ref_name;
 	struct strbuf namespaced_name_buf = STRBUF_INIT;
@@ -479,7 +479,7 @@ static char *update(struct command *cmd, struct shallow_info *si)
 	/* only refs/... are allowed */
 	if (!starts_with(name, "refs/") || check_refname_format(name + 5, 0)) {
 		rp_error("refusing to create funny ref '%s' remotely", name);
-		return xstrdup("funny refname");
+		return "funny refname";
 	}
 
 	strbuf_addf(&namespaced_name_buf, "%s%s", get_git_namespace(), name);
@@ -497,20 +497,20 @@ static char *update(struct command *cmd, struct shallow_info *si)
 			rp_error("refusing to update checked out branch: %s", name);
 			if (deny_current_branch == DENY_UNCONFIGURED)
 				refuse_unconfigured_deny();
-			return xstrdup("branch is currently checked out");
+			return "branch is currently checked out";
 		}
 	}
 
 	if (!is_null_sha1(new_sha1) && !has_sha1_file(new_sha1)) {
 		error("unpack should have generated %s, "
 		      "but I can't find it!", sha1_to_hex(new_sha1));
-		return xstrdup("bad pack");
+		return "bad pack";
 	}
 
 	if (!is_null_sha1(old_sha1) && is_null_sha1(new_sha1)) {
 		if (deny_deletes && starts_with(name, "refs/heads/")) {
 			rp_error("denying ref deletion for %s", name);
-			return xstrdup("deletion prohibited");
+			return "deletion prohibited";
 		}
 
 		if (!strcmp(namespaced_name, head_name)) {
@@ -525,7 +525,7 @@ static char *update(struct command *cmd, struct shallow_info *si)
 				if (deny_delete_current == DENY_UNCONFIGURED)
 					refuse_unconfigured_deny_delete_current();
 				rp_error("refusing to delete the current branch: %s", name);
-				return xstrdup("deletion of the current branch prohibited");
+				return "deletion of the current branch prohibited";
 			}
 		}
 	}
@@ -543,19 +543,19 @@ static char *update(struct command *cmd, struct shallow_info *si)
 		    old_object->type != OBJ_COMMIT ||
 		    new_object->type != OBJ_COMMIT) {
 			error("bad sha1 objects for %s", name);
-			return xstrdup("bad ref");
+			return "bad ref";
 		}
 		old_commit = (struct commit *)old_object;
 		new_commit = (struct commit *)new_object;
 		if (!in_merge_bases(old_commit, new_commit)) {
 			rp_error("denying non-fast-forward %s"
 				 " (you should pull first)", name);
-			return xstrdup("non-fast-forward");
+			return "non-fast-forward";
 		}
 	}
 	if (run_update_hook(cmd)) {
 		rp_error("hook declined to update %s", name);
-		return xstrdup("hook declined");
+		return "hook declined";
 	}
 
 	if (is_null_sha1(new_sha1)) {
@@ -570,7 +570,7 @@ static char *update(struct command *cmd, struct shallow_info *si)
 		}
 		if (delete_ref(namespaced_name, old_sha1, 0)) {
 			rp_error("failed to delete %s", name);
-			return xstrdup("failed to delete");
+			return "failed to delete";
 		}
 		return NULL; /* good */
 	}
@@ -580,18 +580,18 @@ static char *update(struct command *cmd, struct shallow_info *si)
 
 		if (shallow_update && si->shallow_ref[cmd->index] &&
 		    update_shallow_ref(cmd, si))
-			return xstrdup("shallow error");
+			return "shallow error";
 
 		transaction = ref_transaction_begin(&err);
 		if (!transaction ||
 		    ref_transaction_update(transaction, namespaced_name,
 					   new_sha1, old_sha1, 0, 1, &err) ||
 		    ref_transaction_commit(transaction, "push", &err)) {
-			char *str = strbuf_detach(&err, NULL);
 			ref_transaction_free(transaction);
 
-			rp_error("%s", str);
-			return str;
+			rp_error("%s", err.buf);
+			strbuf_release(&err);
+			return "failed to update ref";
 		}
 
 		ref_transaction_free(transaction);
@@ -654,9 +654,6 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)
 	char cmd_oldh[41], cmd_newh[41], dst_oldh[41], dst_newh[41];
 	int flag;
 
-	if (cmd->error_string)
-		die("BUG: check_aliased_update called with failed cmd");
-
 	strbuf_addf(&buf, "%s%s", get_git_namespace(), cmd->ref_name);
 	dst_name = resolve_ref_unsafe(buf.buf, sha1, 0, &flag);
 	strbuf_release(&buf);
@@ -668,7 +665,7 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)
 	if (!dst_name) {
 		rp_error("refusing update to broken symref '%s'", cmd->ref_name);
 		cmd->skip_update = 1;
-		cmd->error_string = xstrdup("broken symref");
+		cmd->error_string = "broken symref";
 		return;
 	}
 
@@ -694,9 +691,8 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)
 		 cmd->ref_name, cmd_oldh, cmd_newh,
 		 dst_cmd->ref_name, dst_oldh, dst_newh);
 
-	cmd->error_string = xstrdup("inconsistent aliased update");
-	free(dst_cmd->error_string);
-	dst_cmd->error_string = xstrdup("inconsistent aliased update");
+	cmd->error_string = dst_cmd->error_string =
+		"inconsistent aliased update";
 }
 
 static void check_aliased_updates(struct command *commands)
@@ -744,9 +740,7 @@ static void set_connectivity_errors(struct command *commands,
 		if (!check_everything_connected(command_singleton_iterator,
 						0, &singleton))
 			continue;
-		if (cmd->error_string)  /* can't happen */
-			continue;
-		cmd->error_string = xstrdup("missing necessary objects");
+		cmd->error_string = "missing necessary objects";
 	}
 }
 
@@ -783,9 +777,9 @@ static void reject_updates_to_hidden(struct command *commands)
 		if (cmd->error_string || !ref_is_hidden(cmd->ref_name))
 			continue;
 		if (is_null_sha1(cmd->new_sha1))
-			cmd->error_string = xstrdup("deny deleting a hidden ref");
+			cmd->error_string = "deny deleting a hidden ref";
 		else
-			cmd->error_string = xstrdup("deny updating a hidden ref");
+			cmd->error_string = "deny updating a hidden ref";
 	}
 }
 
@@ -799,11 +793,8 @@ static void execute_commands(struct command *commands,
 	struct iterate_data data;
 
 	if (unpacker_error) {
-		for (cmd = commands; cmd; cmd = cmd->next) {
-			if (cmd->error_string)  /* can't happen */
-				continue;
-			cmd->error_string = xstrdup("unpacker error");
-		}
+		for (cmd = commands; cmd; cmd = cmd->next)
+			cmd->error_string = "unpacker error";
 		return;
 	}
 
@@ -816,9 +807,8 @@ static void execute_commands(struct command *commands,
 
 	if (run_receive_hook(commands, "pre-receive", 0)) {
 		for (cmd = commands; cmd; cmd = cmd->next) {
-			if (cmd->error_string)
-				continue;
-			cmd->error_string = xstrdup("pre-receive hook declined");
+			if (!cmd->error_string)
+				cmd->error_string = "pre-receive hook declined";
 		}
 		return;
 	}
@@ -1096,8 +1086,7 @@ static void update_shallow_info(struct command *commands,
 		if (is_null_sha1(cmd->new_sha1))
 			continue;
 		if (ref_status[cmd->index]) {
-			free(cmd->error_string);
-			cmd->error_string = xstrdup("shallow update not allowed");
+			cmd->error_string = "shallow update not allowed";
 			cmd->skip_update = 1;
 		}
 	}
@@ -1138,17 +1127,6 @@ static int delete_only(struct command *commands)
 	return 1;
 }
 
-static void free_commands(struct command *commands)
-{
-	while (commands) {
-		struct command *next = commands->next;
-
-		free(commands->error_string);
-		free(commands);
-		commands = next;
-	}
-}
-
 int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 {
 	int advertise_refs = 0;
@@ -1244,6 +1222,5 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 		packet_flush(1);
 	sha1_array_clear(&shallow);
 	sha1_array_clear(&ref);
-	free_commands(commands);
 	return 0;
 }
diff --git a/builtin/replace.c b/builtin/replace.c
index 7528f3d..1fcd06d 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -169,8 +169,7 @@ static int replace_object_sha1(const char *object_ref,
 
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref, repl, prev,
-				   0, !is_null_sha1(prev), &err) ||
+	    ref_transaction_update(transaction, ref, repl, prev, 0, 1, &err) ||
 	    ref_transaction_commit(transaction, NULL, &err))
 		die("%s", err.buf);
 
diff --git a/builtin/tag.c b/builtin/tag.c
index 1aa88a2..f3f172f 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -712,6 +712,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (force && !is_null_sha1(prev) && hashcmp(prev, object))
 		printf(_("Updated tag '%s' (was %s)\n"), tag, find_unique_abbrev(prev, DEFAULT_ABBREV));
 
+	strbuf_release(&err);
 	strbuf_release(&buf);
 	strbuf_release(&ref);
 	return 0;
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index c6ad0be..96a53b9 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -366,6 +366,8 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 
 	if (read_stdin) {
 		transaction = ref_transaction_begin(&err);
+		if (!transaction)
+			die("%s", err.buf);
 		if (delete || no_deref || argc > 0)
 			usage_with_options(git_update_ref_usage, options);
 		if (end_null)
@@ -374,6 +376,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		if (ref_transaction_commit(transaction, msg, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
+		strbuf_release(&err);
 		return 0;
 	}
 
diff --git a/fast-import.c b/fast-import.c
index a95e1be..e7f6e37 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1716,6 +1716,7 @@ static int update_branch(struct branch *b)
 		return -1;
 	}
 	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	return 0;
 }
 
diff --git a/refs.c b/refs.c
index 0017d9c..819e25f 100644
--- a/refs.c
+++ b/refs.c
@@ -2074,7 +2074,10 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 	return logs_found;
 }
 
-/* This function should make sure errno is meaningful on error */
+/*
+ * Locks a "refs/" ref returning the lock on success and NULL on failure.
+ * On failure errno is set to something meaningful.
+ */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
 					    int flags, int *type_p)
@@ -2401,6 +2404,7 @@ static void prune_ref(struct ref_to_prune *r)
 		return;
 	}
 	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	try_remove_empty_parents(r->name);
 }
 
@@ -2575,6 +2579,7 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 		return 1;
 	}
 	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	return 0;
 }
 
@@ -3352,19 +3357,15 @@ struct ref_update {
  * Transaction states.
  * OPEN:   The transaction is in a valid state and can accept new updates.
  *         An OPEN transaction can be committed.
- * CLOSED: If an open transaction is successfully committed the state will
- *         change to CLOSED. No further changes can be made to a CLOSED
- *         transaction.
- *         CLOSED means that all updates have been successfully committed and
- *         the only thing that remains is to free the completed transaction.
- * ERROR:  The transaction has failed and is no longer committable.
- *         No further changes can be made to a CLOSED transaction and it must
- *         be rolled back using transaction_free.
+ * CLOSED: A closed transaction is no longer active and no other operations
+ *         than free can be used on it in this state.
+ *         A transaction can either become closed by successfully committing
+ *         an active transaction or if there is a failure while building
+ *         the transaction thus rendering it failed/inactive.
  */
 enum ref_transaction_state {
 	REF_TRANSACTION_OPEN   = 0,
-	REF_TRANSACTION_CLOSED = 1,
-	REF_TRANSACTION_ERROR  = 2,
+	REF_TRANSACTION_CLOSED = 1
 };
 
 /*
@@ -3509,6 +3510,7 @@ int update_ref(const char *action, const char *refname,
 		strbuf_release(&err);
 		return 1;
 	}
+	strbuf_release(&err);
 	return 0;
 }
 
@@ -3588,10 +3590,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 					     msg);
 			update->lock = NULL; /* freed by write_ref_sha1 */
 			if (ret) {
-				const char *str = "Cannot update the ref '%s'.";
-
 				if (err)
-					strbuf_addf(err, str, update->refname);
+					strbuf_addf(err, "Cannot update the "
+						    "ref '%s'.",
+						    update->refname);
 				goto cleanup;
 			}
 		}
@@ -3614,8 +3616,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	clear_loose_ref_cache(&ref_cache);
 
 cleanup:
-	transaction->state = ret ? REF_TRANSACTION_ERROR
-		: REF_TRANSACTION_CLOSED;
+	transaction->state = REF_TRANSACTION_CLOSED;
 
 	for (i = 0; i < n; i++)
 		if (updates[i]->lock)
diff --git a/refs.h b/refs.h
index aad846c..69ef28c 100644
--- a/refs.h
+++ b/refs.h
@@ -180,8 +180,7 @@ extern int peel_ref(const char *refname, unsigned char *sha1);
  */
 #define REF_NODEREF	0x01
 /*
- * Locks any ref (for 'HEAD' type refs) and sets errno to something
- * meaningful on failure.
+ * This function sets errno to something meaningful on failure.
  */
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
diff --git a/sequencer.c b/sequencer.c
index cf17c69..5e93b6a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -296,6 +296,7 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	}
 
 	strbuf_release(&sb);
+	strbuf_release(&err);
 	ref_transaction_free(transaction);
 	return 0;
 }
diff --git a/walker.c b/walker.c
index 60d9f9e..b8a5441 100644
--- a/walker.c
+++ b/walker.c
@@ -251,12 +251,12 @@ void walker_targets_free(int targets, char **target, const char **write_ref)
 int walker_fetch(struct walker *walker, int targets, char **target,
 		 const char **write_ref, const char *write_ref_log_details)
 {
-	struct strbuf ref_name = STRBUF_INIT;
+	struct strbuf refname = STRBUF_INIT;
 	struct strbuf err = STRBUF_INIT;
 	struct ref_transaction *transaction = NULL;
 	unsigned char *sha1 = xmalloc(targets * 20);
 	char *msg = NULL;
-	int i;
+	int i, ret = -1;
 
 	save_commit_buffer = 0;
 
@@ -264,7 +264,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		transaction = ref_transaction_begin(&err);
 		if (!transaction) {
 			error("%s", err.buf);
-			goto rollback_and_fail;
+			goto done;
 		}
 	}
 	if (!walker->get_recover)
@@ -273,15 +273,18 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 	for (i = 0; i < targets; i++) {
 		if (interpret_target(walker, target[i], &sha1[20 * i])) {
 			error("Could not interpret response from server '%s' as something to pull", target[i]);
-			goto rollback_and_fail;
+			goto done;
 		}
 		if (process(walker, lookup_unknown_object(&sha1[20 * i])))
-			goto rollback_and_fail;
+			goto done;
 	}
 
 	if (loop(walker))
-		goto rollback_and_fail;
-
+		goto done;
+	if (!write_ref) {
+		ret = 0;
+		goto done;
+	}
 	if (write_ref_log_details) {
 		msg = xmalloc(strlen(write_ref_log_details) + 12);
 		sprintf(msg, "fetch from %s", write_ref_log_details);
@@ -289,37 +292,33 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		msg = NULL;
 	}
 	for (i = 0; i < targets; i++) {
-		if (!write_ref || !write_ref[i])
+		if (!write_ref[i])
 			continue;
-		strbuf_reset(&ref_name);
-		strbuf_addf(&ref_name, "refs/%s", write_ref[i]);
-		if (ref_transaction_update(transaction, ref_name.buf,
+		strbuf_reset(&refname);
+		strbuf_addf(&refname, "refs/%s", write_ref[i]);
+		if (ref_transaction_update(transaction, refname.buf,
 					   &sha1[20 * i], NULL, 0, 0,
 					   &err)) {
 			error("%s", err.buf);
-			goto rollback_and_fail;
+			goto done;
 		}
 	}
-	if (write_ref) {
-		if (ref_transaction_commit(transaction,
-					   msg ? msg : "fetch (unknown)",
-					   &err)) {
-			error("%s", err.buf);
-			goto rollback_and_fail;
-		}
-		ref_transaction_free(transaction);
+	if (ref_transaction_commit(transaction,
+				   msg ? msg : "fetch (unknown)",
+				   &err)) {
+		error("%s", err.buf);
+		goto done;
 	}
 
-	free(msg);
-	return 0;
+	ret = 0;
 
-rollback_and_fail:
+done:
 	ref_transaction_free(transaction);
 	free(msg);
+	free(sha1);
 	strbuf_release(&err);
-	strbuf_release(&ref_name);
-
-	return -1;
+	strbuf_release(&refname);
+	return ret;
 }
 
 void walker_free(struct walker *walker)

Re: Transaction patch series overview

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

>>  https://code-review.googlesource.com/#/q/topic:ref-transaction-1
>
> Outcome of the experiment: patches gained some minor changes and then
>
>  1-12
> 	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
>  13
> 	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
> 	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
>  14
> 	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
>  15-16
> 	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
> 	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
>  17
> 	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
>
>  18
> 	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
> 	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
>  19
> 	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
>  20
> 	Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
> 	Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
> I found the web UI helpful in seeing how each patch evolved since I
> last looked at it.  Interdiff relative to the version in pu is below.

Thanks for the interdiff, it really helps to be able to take a
glance without having to click around.

It seems that I can hold the topic in 'pu' a bit longer and expect a
reroll from this effort before merging it to 'next'?

> The next series from Ronnie's collection is available at
> https://code-review.googlesource.com/#/q/topic:ref-transaction in case
> someone wants a fresh series to look at.

Thanks.

> diff --git a/builtin/commit.c b/builtin/commit.c
> index 668fa6a..9bf1003 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1765,8 +1765,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
>  	transaction = ref_transaction_begin(&err);
>  	if (!transaction ||
>  	    ref_transaction_update(transaction, "HEAD", sha1,
> -				   current_head ?
> -				   current_head->object.sha1 : NULL,
> +				   current_head
> +				   ? current_head->object.sha1 : NULL,
>  				   0, !!current_head, &err) ||
>  	    ref_transaction_commit(transaction, sb.buf, &err)) {
>  		rollback_index_files();

Perhaps this is nicer, but probably most people would not care.

> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
> index 91099ad..224fadc 100644
> --- a/builtin/receive-pack.c
> +++ b/builtin/receive-pack.c
> @@ -580,18 +580,18 @@ static char *update(struct command *cmd, struct shallow_info *si)
>  
>  		if (shallow_update && si->shallow_ref[cmd->index] &&
>  		    update_shallow_ref(cmd, si))
> -			return xstrdup("shallow error");
> +			return "shallow error";
>  
>  		transaction = ref_transaction_begin(&err);
>  		if (!transaction ||
>  		    ref_transaction_update(transaction, namespaced_name,
>  					   new_sha1, old_sha1, 0, 1, &err) ||
>  		    ref_transaction_commit(transaction, "push", &err)) {
> -			char *str = strbuf_detach(&err, NULL);
>  			ref_transaction_free(transaction);
>  
> -			rp_error("%s", str);
> -			return str;
> +			rp_error("%s", err.buf);
> +			strbuf_release(&err);
> +			return "failed to update ref";
>  		}

I am guessing that the purpose of this change is to make sure that
cmd->error_string is taken from a fixed set of strings, not an
arbitrary collection of errors from the transaction subsystem, but
what is the significance of doing so?  Do the other side i18n while
running print_ref_status() or something?

> diff --git a/refs.h b/refs.h
> index aad846c..69ef28c 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -180,8 +180,7 @@ extern int peel_ref(const char *refname, unsigned char *sha1);
>   */
>  #define REF_NODEREF	0x01
>  /*
> - * Locks any ref (for 'HEAD' type refs) and sets errno to something
> - * meaningful on failure.
> + * This function sets errno to something meaningful on failure.
>   */
>  extern struct ref_lock *lock_any_ref_for_update(const char *refname,
>  						const unsigned char *old_sha1,

To contrast this function with lock_ref_sha1() that only allowed
refs under refs/ hierarchy, the comment may have made sense, but now
that other function is gone, losing the comment makes sense.

I removed from the above the interdiff hunks I did not have any
comments or questions on.

Thanks again.

Re: Transaction patch series overview

[Jonathan Nieder]

Hi again,

Junio C Hamano wrote:

> It seems that I can hold the topic in 'pu' a bit longer and expect a
> reroll from this effort before merging it to 'next'?

Sorry for the delay.  The following changes on top of "master"
(refs.c: change ref_transaction_update() to do error checking and
return status, 2014-07-14) are available in the git repository at

  git://repo.or.cz/git/jrn.git tags/rs/ref-transaction-1

for you to fetch changes up to 432ad1f39fd773b37757dd8f10b3075dc3d0d0a2:

  refs.c: make delete_ref use a transaction (2014-08-26 14:22:53 -0700)

----------------------------------------------------------------
"Use ref transactions", early part

Ronnie explains:

	This patch series expands on the transaction API. It converts
	ref updates, inside refs.c as well as external, to use the
	transaction API for updates. This makes most ref updates
	atomic when there are failures locking or writing to a ref.

This series teaches the tag, replace, commit, cherry-pick,
fast-import, checkout -b, branch, receive-pack, and http-fetch
commands and all update_ref and delete_ref callers to use the ref
transaction API instead of lock_ref_sha1.

The main user-visible change should be some cosmetic changes to error
messages.  The series also combines multiple ref updates into a single
transaction in 'git http-fetch -w' and when writing tags in
fast-import but otherwise doesn't change the granularity of
transactions.

Reviewed at https://code-review.googlesource.com/#/q/topic:ref-transaction-1

----------------------------------------------------------------
Ronnie Sahlberg (20):
      refs.c: change ref_transaction_create to do error checking and return status
      refs.c: update ref_transaction_delete to check for error and return status
      refs.c: make ref_transaction_begin take an err argument
      refs.c: add transaction.status and track OPEN/CLOSED
      tag.c: use ref transactions when doing updates
      replace.c: use the ref transaction functions for updates
      commit.c: use ref transactions for updates
      sequencer.c: use ref transactions for all ref updates
      fast-import.c: change update_branch to use ref transactions
      branch.c: use ref transaction for all ref updates
      refs.c: change update_ref to use a transaction
      receive-pack.c: use a reference transaction for updating the refs
      fast-import.c: use a ref transaction when dumping tags
      walker.c: use ref transaction for ref updates
      refs.c: make lock_ref_sha1 static
      refs.c: remove the update_ref_lock function
      refs.c: remove the update_ref_write function
      refs.c: remove lock_ref_sha1
      refs.c: make prune_ref use a transaction to delete the ref
      refs.c: make delete_ref use a transaction

 branch.c               |  31 ++++---
 builtin/commit.c       |  25 +++--
 builtin/receive-pack.c |  25 +++--
 builtin/replace.c      |  14 +--
 builtin/tag.c          |  16 ++--
 builtin/update-ref.c   |  14 ++-
 fast-import.c          |  54 +++++++----
 refs.c                 | 244 ++++++++++++++++++++++++++++---------------------
 refs.h                 |  77 ++++++++++++----
 sequencer.c            |  26 ++++--
 walker.c               |  70 ++++++++------
 11 files changed, 367 insertions(+), 229 deletions(-)

[...]
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> --- a/builtin/receive-pack.c
>> +++ b/builtin/receive-pack.c
>> @@ -580,18 +580,18 @@ static char *update(struct command *cmd, struct shallow_info *si)
[...]
>>  		if (!transaction ||
>>  		    ref_transaction_update(transaction, namespaced_name,
>>  					   new_sha1, old_sha1, 0, 1, &err) ||
>>  		    ref_transaction_commit(transaction, "push", &err)) {
>> -			char *str = strbuf_detach(&err, NULL);
>>  			ref_transaction_free(transaction);
>>  
>> -			rp_error("%s", str);
>> -			return str;
>> +			rp_error("%s", err.buf);
>> +			strbuf_release(&err);
>> +			return "failed to update ref";
>>  		}
>
> I am guessing that the purpose of this change is to make sure that
> cmd->error_string is taken from a fixed set of strings, not an
> arbitrary collection of errors from the transaction subsystem, but
> what is the significance of doing so?  Do the other side i18n while
> running print_ref_status() or something?

It's about keeping the message in parentheses on the "! rejected
master -> master" line short and simple, since the more detailed
diagnosis is redundant next to the full message that is sent over
sideband earlier and gets a line to itself.

Side effects:

 * a less invasive patch --- no more need to change the calling
   convention to return a dynamically allocated string, with
   assertions throughout the file that check for leaks

 * using the same "all lowercase and brief" convention makes the
   message less jarring next to other statuses for "! rejected" lines,
   like "non-fast-forward"

 * no more O(n) stall from traversing the linked list to free messages
   when receive-pack is about to exit

[...]
>> diff --git a/refs.h b/refs.h
>> index aad846c..69ef28c 100644
>> --- a/refs.h
>> +++ b/refs.h
>> @@ -180,8 +180,7 @@ extern int peel_ref(const char *refname, unsigned char *sha1);
>>   */
>>  #define REF_NODEREF	0x01
>>  /*
>> - * Locks any ref (for 'HEAD' type refs) and sets errno to something
>> - * meaningful on failure.
>> + * This function sets errno to something meaningful on failure.
>>   */
>>  extern struct ref_lock *lock_any_ref_for_update(const char *refname,
>>  						const unsigned char *old_sha1,
>
> To contrast this function with lock_ref_sha1() that only allowed
> refs under refs/ hierarchy, the comment may have made sense, but now
> that other function is gone, losing the comment makes sense.

Ahhh.  Thanks for explaining.

Jonathan

[PATCH 0/20] rs/ref-transaction-1 (Re: Transaction patch series overview)

[Jonathan Nieder]

Jonathan Nieder wrote:

> This series teaches the tag, replace, commit, cherry-pick,
> fast-import, checkout -b, branch, receive-pack, and http-fetch
> commands and all update_ref and delete_ref callers to use the ref
> transaction API instead of lock_ref_sha1.
>
> The main user-visible change should be some cosmetic changes to error
> messages.  The series also combines multiple ref updates into a single
> transaction in 'git http-fetch -w' and when writing tags in
> fast-import but otherwise doesn't change the granularity of
> transactions.
>
> Reviewed at https://code-review.googlesource.com/#/q/topic:ref-transaction-1
>
> ----------------------------------------------------------------
> Ronnie Sahlberg (20):
>       refs.c: change ref_transaction_create to do error checking and return status
>       refs.c: update ref_transaction_delete to check for error and return status
>       refs.c: make ref_transaction_begin take an err argument
>       refs.c: add transaction.status and track OPEN/CLOSED
>       tag.c: use ref transactions when doing updates
>       replace.c: use the ref transaction functions for updates
>       commit.c: use ref transactions for updates
>       sequencer.c: use ref transactions for all ref updates
>       fast-import.c: change update_branch to use ref transactions
>       branch.c: use ref transaction for all ref updates
>       refs.c: change update_ref to use a transaction
>       receive-pack.c: use a reference transaction for updating the refs
>       fast-import.c: use a ref transaction when dumping tags
>       walker.c: use ref transaction for ref updates
>       refs.c: make lock_ref_sha1 static
>       refs.c: remove the update_ref_lock function
>       refs.c: remove the update_ref_write function
>       refs.c: remove lock_ref_sha1
>       refs.c: make prune_ref use a transaction to delete the ref
>       refs.c: make delete_ref use a transaction

And here are the patches.

[PATCH 01/20] refs.c: change ref_transaction_create to do error checking and return status

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:26:44 -0700

Do basic error checking in ref_transaction_create() and make it return
non-zero on error. Update all callers to check the result of
ref_transaction_create(). There are currently no conditions in _create that
will return error but there will be in the future. Add an err argument that
will be updated on failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/update-ref.c |  4 +++-
 refs.c               | 18 ++++++++++++------
 refs.h               | 48 +++++++++++++++++++++++++++++++++++++++++-------
 3 files changed, 56 insertions(+), 14 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 3067b11..41121fa 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -226,7 +226,9 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)
 	if (*next != line_termination)
 		die("create %s: extra input: %s", refname, next);
 
-	ref_transaction_create(transaction, refname, new_sha1, update_flags);
+	if (ref_transaction_create(transaction, refname, new_sha1,
+				   update_flags, &err))
+		die("%s", err.buf);
 
 	update_flags = 0;
 	free(refname);
diff --git a/refs.c b/refs.c
index 3f05e88..c49f1c6 100644
--- a/refs.c
+++ b/refs.c
@@ -3449,18 +3449,24 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	return 0;
 }
 
-void ref_transaction_create(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *new_sha1,
-			    int flags)
+int ref_transaction_create(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *new_sha1,
+			   int flags,
+			   struct strbuf *err)
 {
-	struct ref_update *update = add_update(transaction, refname);
+	struct ref_update *update;
+
+	if (!new_sha1 || is_null_sha1(new_sha1))
+		die("BUG: create ref with null new_sha1");
+
+	update = add_update(transaction, refname);
 
-	assert(!is_null_sha1(new_sha1));
 	hashcpy(update->new_sha1, new_sha1);
 	hashclr(update->old_sha1);
 	update->flags = flags;
 	update->have_old = 1;
+	return 0;
 }
 
 void ref_transaction_delete(struct ref_transaction *transaction,
diff --git a/refs.h b/refs.h
index c5376ce..b648819 100644
--- a/refs.h
+++ b/refs.h
@@ -10,6 +10,38 @@ struct ref_lock {
 	int force_write;
 };
 
+/*
+ * A ref_transaction represents a collection of ref updates
+ * that should succeed or fail together.
+ *
+ * Calling sequence
+ * ----------------
+ * - Allocate and initialize a `struct ref_transaction` by calling
+ *   `ref_transaction_begin()`.
+ *
+ * - List intended ref updates by calling functions like
+ *   `ref_transaction_update()` and `ref_transaction_create()`.
+ *
+ * - Call `ref_transaction_commit()` to execute the transaction.
+ *   If this succeeds, the ref updates will have taken place and
+ *   the transaction cannot be rolled back.
+ *
+ * - At any time call `ref_transaction_free()` to discard the
+ *   transaction and free associated resources.  In particular,
+ *   this rolls back the transaction if it has not been
+ *   successfully committed.
+ *
+ * Error handling
+ * --------------
+ *
+ * On error, transaction functions append a message about what
+ * went wrong to the 'err' argument.  The message mentions what
+ * ref was being updated (if any) when the error occurred so it
+ * can be passed to 'die' or 'error' as-is.
+ *
+ * The message is appended to err without first clearing err.
+ * err will not be '\n' terminated.
+ */
 struct ref_transaction;
 
 /*
@@ -248,7 +280,7 @@ struct ref_transaction *ref_transaction_begin(void);
  * it must not have existed beforehand.
  * Function returns 0 on success and non-zero on failure. A failure to update
  * means that the transaction as a whole has failed and will need to be
- * rolled back. On failure the err buffer will be updated.
+ * rolled back.
  */
 int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
@@ -262,11 +294,15 @@ int ref_transaction_update(struct ref_transaction *transaction,
  * that the reference should have after the update; it must not be the
  * null SHA-1.  It is verified that the reference does not exist
  * already.
+ * Function returns 0 on success and non-zero on failure. A failure to create
+ * means that the transaction as a whole has failed and will need to be
+ * rolled back.
  */
-void ref_transaction_create(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *new_sha1,
-			    int flags);
+int ref_transaction_create(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *new_sha1,
+			   int flags,
+			   struct strbuf *err);
 
 /*
  * Add a reference deletion to transaction.  If have_old is true, then
@@ -282,8 +318,6 @@ void ref_transaction_delete(struct ref_transaction *transaction,
  * Commit all of the changes that have been queued in transaction, as
  * atomically as possible.  Return a nonzero value if there is a
  * problem.
- * If err is non-NULL we will add an error string to it to explain why
- * the transaction failed. The string does not end in newline.
  */
 int ref_transaction_commit(struct ref_transaction *transaction,
 			   const char *msg, struct strbuf *err);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 02/20] refs.c: update ref_transaction_delete to check for error and return status

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:27:45 -0700

Change ref_transaction_delete() to do basic error checking and return
non-zero of error. Update all callers to check the return for
ref_transaction_delete(). There are currently no conditions in _delete that
will return error but there will be in the future. Add an err argument that
will be updated on failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/update-ref.c |  5 +++--
 refs.c               | 16 +++++++++++-----
 refs.h               | 12 ++++++++----
 3 files changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 41121fa..7c9c248 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -258,8 +258,9 @@ static const char *parse_cmd_delete(struct strbuf *input, const char *next)
 	if (*next != line_termination)
 		die("delete %s: extra input: %s", refname, next);
 
-	ref_transaction_delete(transaction, refname, old_sha1,
-			       update_flags, have_old);
+	if (ref_transaction_delete(transaction, refname, old_sha1,
+				   update_flags, have_old, &err))
+		die("%s", err.buf);
 
 	update_flags = 0;
 	free(refname);
diff --git a/refs.c b/refs.c
index c49f1c6..40f04f4 100644
--- a/refs.c
+++ b/refs.c
@@ -3469,19 +3469,25 @@ int ref_transaction_create(struct ref_transaction *transaction,
 	return 0;
 }
 
-void ref_transaction_delete(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *old_sha1,
-			    int flags, int have_old)
+int ref_transaction_delete(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *old_sha1,
+			   int flags, int have_old,
+			   struct strbuf *err)
 {
-	struct ref_update *update = add_update(transaction, refname);
+	struct ref_update *update;
 
+	if (have_old && !old_sha1)
+		die("BUG: have_old is true but old_sha1 is NULL");
+
+	update = add_update(transaction, refname);
 	update->flags = flags;
 	update->have_old = have_old;
 	if (have_old) {
 		assert(!is_null_sha1(old_sha1));
 		hashcpy(update->old_sha1, old_sha1);
 	}
+	return 0;
 }
 
 int update_ref(const char *action, const char *refname,
diff --git a/refs.h b/refs.h
index b648819..71389a1 100644
--- a/refs.h
+++ b/refs.h
@@ -308,11 +308,15 @@ int ref_transaction_create(struct ref_transaction *transaction,
  * Add a reference deletion to transaction.  If have_old is true, then
  * old_sha1 holds the value that the reference should have had before
  * the update (which must not be the null SHA-1).
+ * Function returns 0 on success and non-zero on failure. A failure to delete
+ * means that the transaction as a whole has failed and will need to be
+ * rolled back.
  */
-void ref_transaction_delete(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *old_sha1,
-			    int flags, int have_old);
+int ref_transaction_delete(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *old_sha1,
+			   int flags, int have_old,
+			   struct strbuf *err);
 
 /*
  * Commit all of the changes that have been queued in transaction, as
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 03/20] refs.c: make ref_transaction_begin take an err argument

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 19 May 2014 10:42:34 -0700

Add an err argument to _begin so that on non-fatal failures in future ref
backends we can report a nice error back to the caller.
While _begin can currently never fail for other reasons than OOM, in which
case we die() anyway, we may add other types of backends in the future.
For example, a hypothetical MySQL backend could fail in _begin with
"Can not connect to MySQL server. No route to host".

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/update-ref.c | 5 ++++-
 refs.c               | 2 +-
 refs.h               | 2 +-
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 7c9c248..96a53b9 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -365,7 +365,9 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		die("Refusing to perform update with empty message.");
 
 	if (read_stdin) {
-		transaction = ref_transaction_begin();
+		transaction = ref_transaction_begin(&err);
+		if (!transaction)
+			die("%s", err.buf);
 		if (delete || no_deref || argc > 0)
 			usage_with_options(git_update_ref_usage, options);
 		if (end_null)
@@ -374,6 +376,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		if (ref_transaction_commit(transaction, msg, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
+		strbuf_release(&err);
 		return 0;
 	}
 
diff --git a/refs.c b/refs.c
index 40f04f4..9cb7908 100644
--- a/refs.c
+++ b/refs.c
@@ -3397,7 +3397,7 @@ struct ref_transaction {
 	size_t nr;
 };
 
-struct ref_transaction *ref_transaction_begin(void)
+struct ref_transaction *ref_transaction_begin(struct strbuf *err)
 {
 	return xcalloc(1, sizeof(struct ref_transaction));
 }
diff --git a/refs.h b/refs.h
index 71389a1..3f37c65 100644
--- a/refs.h
+++ b/refs.h
@@ -262,7 +262,7 @@ enum action_on_err {
  * Begin a reference transaction.  The reference transaction must
  * be freed by calling ref_transaction_free().
  */
-struct ref_transaction *ref_transaction_begin(void);
+struct ref_transaction *ref_transaction_begin(struct strbuf *err);
 
 /*
  * The following functions add a reference check or update to a
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 04/20] refs.c: add transaction.status and track OPEN/CLOSED

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 12:06:19 -0700

Track the state of a transaction in a new state field. Check the field for
sanity, i.e. that state must be OPEN when _commit/_create/_delete or
_update is called or else die(BUG:...)

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 9cb7908..cc63056 100644
--- a/refs.c
+++ b/refs.c
@@ -3387,6 +3387,21 @@ struct ref_update {
 };
 
 /*
+ * Transaction states.
+ * OPEN:   The transaction is in a valid state and can accept new updates.
+ *         An OPEN transaction can be committed.
+ * CLOSED: A closed transaction is no longer active and no other operations
+ *         than free can be used on it in this state.
+ *         A transaction can either become closed by successfully committing
+ *         an active transaction or if there is a failure while building
+ *         the transaction thus rendering it failed/inactive.
+ */
+enum ref_transaction_state {
+	REF_TRANSACTION_OPEN   = 0,
+	REF_TRANSACTION_CLOSED = 1
+};
+
+/*
  * Data structure for holding a reference transaction, which can
  * consist of checks and updates to multiple references, carried out
  * as atomically as possible.  This structure is opaque to callers.
@@ -3395,6 +3410,7 @@ struct ref_transaction {
 	struct ref_update **updates;
 	size_t alloc;
 	size_t nr;
+	enum ref_transaction_state state;
 };
 
 struct ref_transaction *ref_transaction_begin(struct strbuf *err)
@@ -3437,6 +3453,9 @@ int ref_transaction_update(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: update called for transaction that is not open");
+
 	if (have_old && !old_sha1)
 		die("BUG: have_old is true but old_sha1 is NULL");
 
@@ -3457,6 +3476,9 @@ int ref_transaction_create(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: create called for transaction that is not open");
+
 	if (!new_sha1 || is_null_sha1(new_sha1))
 		die("BUG: create ref with null new_sha1");
 
@@ -3477,6 +3499,9 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: delete called for transaction that is not open");
+
 	if (have_old && !old_sha1)
 		die("BUG: have_old is true but old_sha1 is NULL");
 
@@ -3532,8 +3557,13 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	int n = transaction->nr;
 	struct ref_update **updates = transaction->updates;
 
-	if (!n)
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: commit called for transaction that is not open");
+
+	if (!n) {
+		transaction->state = REF_TRANSACTION_CLOSED;
 		return 0;
+	}
 
 	/* Allocate work space */
 	delnames = xmalloc(sizeof(*delnames) * n);
@@ -3595,6 +3625,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	clear_loose_ref_cache(&ref_cache);
 
 cleanup:
+	transaction->state = REF_TRANSACTION_CLOSED;
+
 	for (i = 0; i < n; i++)
 		if (updates[i]->lock)
 			unlock_ref(updates[i]->lock);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 05/20] tag.c: use ref transactions when doing updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:30:41 -0700

Change tag.c to use ref transactions for all ref updates.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/tag.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/builtin/tag.c b/builtin/tag.c
index c6e8a71..f3f172f 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -548,7 +548,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	struct strbuf ref = STRBUF_INIT;
 	unsigned char object[20], prev[20];
 	const char *object_ref, *tag;
-	struct ref_lock *lock;
 	struct create_tag_options opt;
 	char *cleanup_arg = NULL;
 	int annotate = 0, force = 0, lines = -1;
@@ -556,6 +555,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	const char *msgfile = NULL, *keyid = NULL;
 	struct msg_arg msg = { 0, STRBUF_INIT };
 	struct commit_list *with_commit = NULL;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 	struct option options[] = {
 		OPT_CMDMODE('l', "list", &cmdmode, N_("list tag names"), 'l'),
 		{ OPTION_INTEGER, 'n', NULL, &lines, N_("n"),
@@ -701,14 +702,17 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (annotate)
 		create_tag(object, tag, &buf, &opt, prev, object);
 
-	lock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);
-	if (!lock)
-		die(_("%s: cannot lock the ref"), ref.buf);
-	if (write_ref_sha1(lock, object, NULL) < 0)
-		die(_("%s: cannot update the ref"), ref.buf);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref.buf, object, prev,
+				   0, 1, &err) ||
+	    ref_transaction_commit(transaction, NULL, &err))
+		die("%s", err.buf);
+	ref_transaction_free(transaction);
 	if (force && !is_null_sha1(prev) && hashcmp(prev, object))
 		printf(_("Updated tag '%s' (was %s)\n"), tag, find_unique_abbrev(prev, DEFAULT_ABBREV));
 
+	strbuf_release(&err);
 	strbuf_release(&buf);
 	strbuf_release(&ref);
 	return 0;
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 06/20] replace.c: use the ref transaction functions for updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:32:29 -0700

Update replace.c to use ref transactions for updates.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/replace.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/builtin/replace.c b/builtin/replace.c
index 1bb491d..1fcd06d 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -153,7 +153,8 @@ static int replace_object_sha1(const char *object_ref,
 	unsigned char prev[20];
 	enum object_type obj_type, repl_type;
 	char ref[PATH_MAX];
-	struct ref_lock *lock;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
 	obj_type = sha1_object_info(object, NULL);
 	repl_type = sha1_object_info(repl, NULL);
@@ -166,12 +167,13 @@ static int replace_object_sha1(const char *object_ref,
 
 	check_ref_valid(object, prev, ref, sizeof(ref), force);
 
-	lock = lock_any_ref_for_update(ref, prev, 0, NULL);
-	if (!lock)
-		die("%s: cannot lock the ref", ref);
-	if (write_ref_sha1(lock, repl, NULL) < 0)
-		die("%s: cannot update the ref", ref);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref, repl, prev, 0, 1, &err) ||
+	    ref_transaction_commit(transaction, NULL, &err))
+		die("%s", err.buf);
 
+	ref_transaction_free(transaction);
 	return 0;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 07/20] commit.c: use ref transactions for updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:34:19 -0700

Change commit.c to use ref transactions for all ref updates.
Make sure we pass a NULL pointer to ref_transaction_update if have_old
is false.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/commit.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 5e2221c..9bf1003 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1627,11 +1627,12 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	const char *index_file, *reflog_msg;
 	char *nl;
 	unsigned char sha1[20];
-	struct ref_lock *ref_lock;
 	struct commit_list *parents = NULL, **pptr = &parents;
 	struct stat statbuf;
 	struct commit *current_head = NULL;
 	struct commit_extra_header *extra = NULL;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
 	if (argc == 2 && !strcmp(argv[1], "-h"))
 		usage_with_options(builtin_commit_usage, builtin_commit_options);
@@ -1753,16 +1754,6 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	strbuf_release(&author_ident);
 	free_commit_extra_headers(extra);
 
-	ref_lock = lock_any_ref_for_update("HEAD",
-					   !current_head
-					   ? NULL
-					   : current_head->object.sha1,
-					   0, NULL);
-	if (!ref_lock) {
-		rollback_index_files();
-		die(_("cannot lock HEAD ref"));
-	}
-
 	nl = strchr(sb.buf, '\n');
 	if (nl)
 		strbuf_setlen(&sb, nl + 1 - sb.buf);
@@ -1771,10 +1762,17 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	strbuf_insert(&sb, 0, reflog_msg, strlen(reflog_msg));
 	strbuf_insert(&sb, strlen(reflog_msg), ": ", 2);
 
-	if (write_ref_sha1(ref_lock, sha1, sb.buf) < 0) {
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, "HEAD", sha1,
+				   current_head
+				   ? current_head->object.sha1 : NULL,
+				   0, !!current_head, &err) ||
+	    ref_transaction_commit(transaction, sb.buf, &err)) {
 		rollback_index_files();
-		die(_("cannot update HEAD ref"));
+		die("%s", err.buf);
 	}
+	ref_transaction_free(transaction);
 
 	unlink(git_path("CHERRY_PICK_HEAD"));
 	unlink(git_path("REVERT_HEAD"));
@@ -1803,5 +1801,6 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	if (!quiet)
 		print_summary(prefix, sha1, !current_head);
 
+	strbuf_release(&err);
 	return 0;
 }
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 08/20] sequencer.c: use ref transactions for all ref updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:37:45 -0700

Change to use ref transactions for all updates to refs.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sequencer.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 0a80c58..5e93b6a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -272,23 +272,33 @@ static int error_dirty_index(struct replay_opts *opts)
 static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 			int unborn, struct replay_opts *opts)
 {
-	struct ref_lock *ref_lock;
+	struct ref_transaction *transaction;
 	struct strbuf sb = STRBUF_INIT;
-	int ret;
+	struct strbuf err = STRBUF_INIT;
 
 	read_cache();
 	if (checkout_fast_forward(from, to, 1))
 		exit(1); /* the callee should have complained already */
-	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from,
-					   0, NULL);
-	if (!ref_lock)
-		return error(_("Failed to lock HEAD during fast_forward_to"));
 
 	strbuf_addf(&sb, "%s: fast-forward", action_name(opts));
-	ret = write_ref_sha1(ref_lock, to, sb.buf);
+
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, "HEAD",
+				   to, unborn ? null_sha1 : from,
+				   0, 1, &err) ||
+	    ref_transaction_commit(transaction, sb.buf, &err)) {
+		ref_transaction_free(transaction);
+		error("%s", err.buf);
+		strbuf_release(&sb);
+		strbuf_release(&err);
+		return -1;
+	}
 
 	strbuf_release(&sb);
-	return ret;
+	strbuf_release(&err);
+	ref_transaction_free(transaction);
+	return 0;
 }
 
 static int do_recursive_merge(struct commit *base, struct commit *next,
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 09/20] fast-import.c: change update_branch to use ref transactions

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 16:21:13 -0700

Change update_branch() to use ref transactions for updates.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 fast-import.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index 6707a66..79160d5 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1679,8 +1679,9 @@ found_entry:
 static int update_branch(struct branch *b)
 {
 	static const char *msg = "fast-import";
-	struct ref_lock *lock;
+	struct ref_transaction *transaction;
 	unsigned char old_sha1[20];
+	struct strbuf err = STRBUF_INIT;
 
 	if (read_ref(b->name, old_sha1))
 		hashclr(old_sha1);
@@ -1689,29 +1690,33 @@ static int update_branch(struct branch *b)
 			delete_ref(b->name, old_sha1, 0);
 		return 0;
 	}
-	lock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL);
-	if (!lock)
-		return error("Unable to lock %s", b->name);
 	if (!force_update && !is_null_sha1(old_sha1)) {
 		struct commit *old_cmit, *new_cmit;
 
 		old_cmit = lookup_commit_reference_gently(old_sha1, 0);
 		new_cmit = lookup_commit_reference_gently(b->sha1, 0);
-		if (!old_cmit || !new_cmit) {
-			unlock_ref(lock);
+		if (!old_cmit || !new_cmit)
 			return error("Branch %s is missing commits.", b->name);
-		}
 
 		if (!in_merge_bases(old_cmit, new_cmit)) {
-			unlock_ref(lock);
 			warning("Not updating %s"
 				" (new tip %s does not contain %s)",
 				b->name, sha1_to_hex(b->sha1), sha1_to_hex(old_sha1));
 			return -1;
 		}
 	}
-	if (write_ref_sha1(lock, b->sha1, msg) < 0)
-		return error("Unable to update %s", b->name);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, b->name, b->sha1, old_sha1,
+				   0, 1, &err) ||
+	    ref_transaction_commit(transaction, msg, &err)) {
+		ref_transaction_free(transaction);
+		error("%s", err.buf);
+		strbuf_release(&err);
+		return -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	return 0;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 10/20] branch.c: use ref transaction for all ref updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 16:21:53 -0700

Change create_branch to use a ref transaction when creating the new branch.

This also fixes a race condition in the old code where two concurrent
create_branch could race since the lock_any_ref_for_update/write_ref_sha1
did not protect against the ref already existing. I.e. one thread could end up
overwriting a branch even if the forcing flag is false.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 branch.c | 31 +++++++++++++++++--------------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/branch.c b/branch.c
index 660097b..37ac555 100644
--- a/branch.c
+++ b/branch.c
@@ -226,7 +226,6 @@ void create_branch(const char *head,
 		   int force, int reflog, int clobber_head,
 		   int quiet, enum branch_track track)
 {
-	struct ref_lock *lock = NULL;
 	struct commit *commit;
 	unsigned char sha1[20];
 	char *real_ref, msg[PATH_MAX + 20];
@@ -285,15 +284,6 @@ void create_branch(const char *head,
 		die(_("Not a valid branch point: '%s'."), start_name);
 	hashcpy(sha1, commit->object.sha1);
 
-	if (!dont_change_ref) {
-		lock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);
-		if (!lock)
-			die_errno(_("Failed to lock ref for update"));
-	}
-
-	if (reflog)
-		log_all_ref_updates = 1;
-
 	if (forcing)
 		snprintf(msg, sizeof msg, "branch: Reset to %s",
 			 start_name);
@@ -301,13 +291,26 @@ void create_branch(const char *head,
 		snprintf(msg, sizeof msg, "branch: Created from %s",
 			 start_name);
 
+	if (reflog)
+		log_all_ref_updates = 1;
+
+	if (!dont_change_ref) {
+		struct ref_transaction *transaction;
+		struct strbuf err = STRBUF_INIT;
+
+		transaction = ref_transaction_begin(&err);
+		if (!transaction ||
+		    ref_transaction_update(transaction, ref.buf, sha1,
+					   null_sha1, 0, !forcing, &err) ||
+		    ref_transaction_commit(transaction, msg, &err))
+			die("%s", err.buf);
+		ref_transaction_free(transaction);
+		strbuf_release(&err);
+	}
+
 	if (real_ref && track)
 		setup_tracking(ref.buf + 11, real_ref, track, quiet);
 
-	if (!dont_change_ref)
-		if (write_ref_sha1(lock, sha1, msg) < 0)
-			die_errno(_("Failed to write ref"));
-
 	strbuf_release(&ref);
 	free(real_ref);
 }
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 11/20] refs.c: change update_ref to use a transaction

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 24 Apr 2014 16:36:55 -0700

Change the update_ref helper function to use a ref transaction internally.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)

diff --git a/refs.c b/refs.c
index cc63056..dcc877b 100644
--- a/refs.c
+++ b/refs.c
@@ -3519,11 +3519,32 @@ int update_ref(const char *action, const char *refname,
 	       const unsigned char *sha1, const unsigned char *oldval,
 	       int flags, enum action_on_err onerr)
 {
-	struct ref_lock *lock;
-	lock = update_ref_lock(refname, oldval, flags, NULL, onerr);
-	if (!lock)
+	struct ref_transaction *t;
+	struct strbuf err = STRBUF_INIT;
+
+	t = ref_transaction_begin(&err);
+	if (!t ||
+	    ref_transaction_update(t, refname, sha1, oldval, flags,
+				   !!oldval, &err) ||
+	    ref_transaction_commit(t, action, &err)) {
+		const char *str = "update_ref failed for ref '%s': %s";
+
+		ref_transaction_free(t);
+		switch (onerr) {
+		case UPDATE_REFS_MSG_ON_ERR:
+			error(str, refname, err.buf);
+			break;
+		case UPDATE_REFS_DIE_ON_ERR:
+			die(str, refname, err.buf);
+			break;
+		case UPDATE_REFS_QUIET_ON_ERR:
+			break;
+		}
+		strbuf_release(&err);
 		return 1;
-	return update_ref_write(action, refname, sha1, lock, NULL, onerr);
+	}
+	strbuf_release(&err);
+	return 0;
 }
 
 static int ref_update_compare(const void *r1, const void *r2)
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 12/20] receive-pack.c: use a reference transaction for updating the refs

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 14:36:15 -0700

Wrap all the ref updates inside a transaction.

In the new API there is no distinction between failure to lock and
failure to write a ref.  Both can be permanent (e.g., a ref
"refs/heads/topic" is blocking creation of the lock file
"refs/heads/topic/1.lock") or transient (e.g., file system full) and
there's no clear difference in how the client should respond, so
replace the two statuses "failed to lock" and "failed to write" with
a single status "failed to update ref".  In both cases a more
detailed message is sent by sideband to diagnose the problem.

Example, before:

 error: there are still refs under 'refs/heads/topic'
 remote: error: failed to lock refs/heads/topic
 To foo
  ! [remote rejected] HEAD -> topic (failed to lock)

After:

 error: there are still refs under 'refs/heads/topic'
 remote: error: Cannot lock the ref 'refs/heads/topic'.
 To foo
  ! [remote rejected] HEAD -> topic (failed to update ref)

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/receive-pack.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index c323081..224fadc 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -475,7 +475,6 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 	const char *namespaced_name;
 	unsigned char *old_sha1 = cmd->old_sha1;
 	unsigned char *new_sha1 = cmd->new_sha1;
-	struct ref_lock *lock;
 
 	/* only refs/... are allowed */
 	if (!starts_with(name, "refs/") || check_refname_format(name + 5, 0)) {
@@ -576,19 +575,27 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		return NULL; /* good */
 	}
 	else {
+		struct strbuf err = STRBUF_INIT;
+		struct ref_transaction *transaction;
+
 		if (shallow_update && si->shallow_ref[cmd->index] &&
 		    update_shallow_ref(cmd, si))
 			return "shallow error";
 
-		lock = lock_any_ref_for_update(namespaced_name, old_sha1,
-					       0, NULL);
-		if (!lock) {
-			rp_error("failed to lock %s", name);
-			return "failed to lock";
-		}
-		if (write_ref_sha1(lock, new_sha1, "push")) {
-			return "failed to write"; /* error() already called */
+		transaction = ref_transaction_begin(&err);
+		if (!transaction ||
+		    ref_transaction_update(transaction, namespaced_name,
+					   new_sha1, old_sha1, 0, 1, &err) ||
+		    ref_transaction_commit(transaction, "push", &err)) {
+			ref_transaction_free(transaction);
+
+			rp_error("%s", err.buf);
+			strbuf_release(&err);
+			return "failed to update ref";
 		}
+
+		ref_transaction_free(transaction);
+		strbuf_release(&err);
 		return NULL; /* good */
 	}
 }
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 13/20] fast-import.c: use a ref transaction when dumping tags

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 15:23:58 -0700

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 fast-import.c | 29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index 79160d5..e7f6e37 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1735,15 +1735,32 @@ static void dump_tags(void)
 {
 	static const char *msg = "fast-import";
 	struct tag *t;
-	struct ref_lock *lock;
-	char ref_name[PATH_MAX];
+	struct strbuf ref_name = STRBUF_INIT;
+	struct strbuf err = STRBUF_INIT;
+	struct ref_transaction *transaction;
 
+	transaction = ref_transaction_begin(&err);
+	if (!transaction) {
+		failure |= error("%s", err.buf);
+		goto cleanup;
+	}
 	for (t = first_tag; t; t = t->next_tag) {
-		sprintf(ref_name, "tags/%s", t->name);
-		lock = lock_ref_sha1(ref_name, NULL);
-		if (!lock || write_ref_sha1(lock, t->sha1, msg) < 0)
-			failure |= error("Unable to update %s", ref_name);
+		strbuf_reset(&ref_name);
+		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
+
+		if (ref_transaction_update(transaction, ref_name.buf, t->sha1,
+					   NULL, 0, 0, &err)) {
+			failure |= error("%s", err.buf);
+			goto cleanup;
+		}
 	}
+	if (ref_transaction_commit(transaction, msg, &err))
+		failure |= error("%s", err.buf);
+
+ cleanup:
+	ref_transaction_free(transaction);
+	strbuf_release(&ref_name);
+	strbuf_release(&err);
 }
 
 static void dump_marks_helper(FILE *f,
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 14/20] walker.c: use ref transaction for ref updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 17 Apr 2014 11:31:06 -0700

Switch to using ref transactions in walker_fetch(). As part of the refactoring
to use ref transactions we also fix a potential memory leak where in the
original code if write_ref_sha1() would fail we would end up returning from
the function without free()ing the msg string.

Note that this function is only called when fetching from a remote HTTP
repository onto the local (most of the time single-user) repository which
likely means that the type of collisions that the previous locking would
protect against and cause the fetch to fail for are even more rare.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 walker.c | 72 ++++++++++++++++++++++++++++++++++++----------------------------
 1 file changed, 41 insertions(+), 31 deletions(-)

diff --git a/walker.c b/walker.c
index 1dd86b8..b8a5441 100644
--- a/walker.c
+++ b/walker.c
@@ -251,40 +251,40 @@ void walker_targets_free(int targets, char **target, const char **write_ref)
 int walker_fetch(struct walker *walker, int targets, char **target,
 		 const char **write_ref, const char *write_ref_log_details)
 {
-	struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));
+	struct strbuf refname = STRBUF_INIT;
+	struct strbuf err = STRBUF_INIT;
+	struct ref_transaction *transaction = NULL;
 	unsigned char *sha1 = xmalloc(targets * 20);
-	char *msg;
-	int ret;
-	int i;
+	char *msg = NULL;
+	int i, ret = -1;
 
 	save_commit_buffer = 0;
 
-	for (i = 0; i < targets; i++) {
-		if (!write_ref || !write_ref[i])
-			continue;
-
-		lock[i] = lock_ref_sha1(write_ref[i], NULL);
-		if (!lock[i]) {
-			error("Can't lock ref %s", write_ref[i]);
-			goto unlock_and_fail;
+	if (write_ref) {
+		transaction = ref_transaction_begin(&err);
+		if (!transaction) {
+			error("%s", err.buf);
+			goto done;
 		}
 	}
-
 	if (!walker->get_recover)
 		for_each_ref(mark_complete, NULL);
 
 	for (i = 0; i < targets; i++) {
 		if (interpret_target(walker, target[i], &sha1[20 * i])) {
 			error("Could not interpret response from server '%s' as something to pull", target[i]);
-			goto unlock_and_fail;
+			goto done;
 		}
 		if (process(walker, lookup_unknown_object(&sha1[20 * i])))
-			goto unlock_and_fail;
+			goto done;
 	}
 
 	if (loop(walker))
-		goto unlock_and_fail;
-
+		goto done;
+	if (!write_ref) {
+		ret = 0;
+		goto done;
+	}
 	if (write_ref_log_details) {
 		msg = xmalloc(strlen(write_ref_log_details) + 12);
 		sprintf(msg, "fetch from %s", write_ref_log_details);
@@ -292,23 +292,33 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		msg = NULL;
 	}
 	for (i = 0; i < targets; i++) {
-		if (!write_ref || !write_ref[i])
+		if (!write_ref[i])
 			continue;
-		ret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : "fetch (unknown)");
-		lock[i] = NULL;
-		if (ret)
-			goto unlock_and_fail;
+		strbuf_reset(&refname);
+		strbuf_addf(&refname, "refs/%s", write_ref[i]);
+		if (ref_transaction_update(transaction, refname.buf,
+					   &sha1[20 * i], NULL, 0, 0,
+					   &err)) {
+			error("%s", err.buf);
+			goto done;
+		}
 	}
+	if (ref_transaction_commit(transaction,
+				   msg ? msg : "fetch (unknown)",
+				   &err)) {
+		error("%s", err.buf);
+		goto done;
+	}
+
+	ret = 0;
+
+done:
+	ref_transaction_free(transaction);
 	free(msg);
-
-	return 0;
-
-unlock_and_fail:
-	for (i = 0; i < targets; i++)
-		if (lock[i])
-			unlock_ref(lock[i]);
-
-	return -1;
+	free(sha1);
+	strbuf_release(&err);
+	strbuf_release(&refname);
+	return ret;
 }
 
 void walker_free(struct walker *walker)
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 15/20] refs.c: make lock_ref_sha1 static

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 15:38:47 -0700

No external callers reference lock_ref_sha1 any more so let's declare it
static.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 7 +++++--
 refs.h | 6 ------
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/refs.c b/refs.c
index dcc877b..0dc093c 100644
--- a/refs.c
+++ b/refs.c
@@ -2069,7 +2069,10 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 	return logs_found;
 }
 
-/* This function should make sure errno is meaningful on error */
+/*
+ * Locks a "refs/" ref returning the lock on success and NULL on failure.
+ * On failure errno is set to something meaningful.
+ */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
 					    int flags, int *type_p)
@@ -2170,7 +2173,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	return NULL;
 }
 
-struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)
+static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)
 {
 	char refpath[PATH_MAX];
 	if (check_refname_format(refname, 0))
diff --git a/refs.h b/refs.h
index 3f37c65..65dd593 100644
--- a/refs.h
+++ b/refs.h
@@ -170,12 +170,6 @@ extern int ref_exists(const char *);
  */
 extern int peel_ref(const char *refname, unsigned char *sha1);
 
-/*
- * Locks a "refs/" ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
- */
-extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1);
-
 /** Locks any ref (for 'HEAD' type refs). */
 #define REF_NODEREF	0x01
 /* errno is set to something meaningful on failure */
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 16/20] refs.c: remove the update_ref_lock function

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 12:14:47 -0700

Since we now only call update_ref_lock with onerr==QUIET_ON_ERR we no longer
need this function and can replace it with just calling lock_any_ref_for_update
directly.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/refs.c b/refs.c
index 0dc093c..de07791 100644
--- a/refs.c
+++ b/refs.c
@@ -3336,24 +3336,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-static struct ref_lock *update_ref_lock(const char *refname,
-					const unsigned char *oldval,
-					int flags, int *type_p,
-					enum action_on_err onerr)
-{
-	struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
-	if (!lock) {
-		const char *str = "Cannot lock the ref '%s'.";
-		switch (onerr) {
-		case UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;
-		case UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;
-		case UPDATE_REFS_QUIET_ON_ERR: break;
-		}
-	}
-	return lock;
-}
-
 static int update_ref_write(const char *action, const char *refname,
 			    const unsigned char *sha1, struct ref_lock *lock,
 			    struct strbuf *err, enum action_on_err onerr)
@@ -3602,12 +3584,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	for (i = 0; i < n; i++) {
 		struct ref_update *update = updates[i];
 
-		update->lock = update_ref_lock(update->refname,
-					       (update->have_old ?
-						update->old_sha1 : NULL),
-					       update->flags,
-					       &update->type,
-					       UPDATE_REFS_QUIET_ON_ERR);
+		update->lock = lock_any_ref_for_update(update->refname,
+						       (update->have_old ?
+							update->old_sha1 :
+							NULL),
+						       update->flags,
+						       &update->type);
 		if (!update->lock) {
 			if (err)
 				strbuf_addf(err, "Cannot lock the ref '%s'.",
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 17/20] refs.c: remove the update_ref_write function

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 13:42:07 -0700

Since we only call update_ref_write from a single place and we only call it
with onerr==QUIET_ON_ERR we can just as well get rid of it and just call
write_ref_sha1 directly. This changes the return status for _commit from
1 to -1 on failures when writing to the ref. Eventually we will want
_commit to start returning more detailed error conditions than the current
simple success/failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 34 ++++++++--------------------------
 1 file changed, 8 insertions(+), 26 deletions(-)

diff --git a/refs.c b/refs.c
index de07791..ef7660a 100644
--- a/refs.c
+++ b/refs.c
@@ -3336,25 +3336,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-static int update_ref_write(const char *action, const char *refname,
-			    const unsigned char *sha1, struct ref_lock *lock,
-			    struct strbuf *err, enum action_on_err onerr)
-{
-	if (write_ref_sha1(lock, sha1, action) < 0) {
-		const char *str = "Cannot update the ref '%s'.";
-		if (err)
-			strbuf_addf(err, str, refname);
-
-		switch (onerr) {
-		case UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;
-		case UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;
-		case UPDATE_REFS_QUIET_ON_ERR: break;
-		}
-		return 1;
-	}
-	return 0;
-}
-
 /**
  * Information needed for a single ref update.  Set new_sha1 to the
  * new value or to zero to delete the ref.  To check the old value
@@ -3604,14 +3585,15 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (!is_null_sha1(update->new_sha1)) {
-			ret = update_ref_write(msg,
-					       update->refname,
-					       update->new_sha1,
-					       update->lock, err,
-					       UPDATE_REFS_QUIET_ON_ERR);
-			update->lock = NULL; /* freed by update_ref_write */
-			if (ret)
+			ret = write_ref_sha1(update->lock, update->new_sha1,
+					     msg);
+			update->lock = NULL; /* freed by write_ref_sha1 */
+			if (ret) {
+				if (err)
+					strbuf_addf(err, "Cannot update the ref '%s'.",
+						    update->refname);
 				goto cleanup;
+			}
 		}
 	}
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 18/20] refs.c: remove lock_ref_sha1

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 15:45:52 -0700

lock_ref_sha1 was only called from one place in refs.c and only provided
a check that the refname was sane before adding back the initial "refs/"
part of the ref path name, the initial "refs/" that this caller had already
stripped off before calling lock_ref_sha1.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index ef7660a..f0883d0 100644
--- a/refs.c
+++ b/refs.c
@@ -2173,15 +2173,6 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	return NULL;
 }
 
-static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)
-{
-	char refpath[PATH_MAX];
-	if (check_refname_format(refname, 0))
-		return NULL;
-	strcpy(refpath, mkpath("refs/%s", refname));
-	return lock_ref_sha1_basic(refpath, old_sha1, 0, NULL);
-}
-
 struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
@@ -2391,8 +2382,12 @@ static void try_remove_empty_parents(char *name)
 /* make sure nobody touched the ref, and unlink */
 static void prune_ref(struct ref_to_prune *r)
 {
-	struct ref_lock *lock = lock_ref_sha1(r->name + 5, r->sha1);
+	struct ref_lock *lock;
 
+	if (check_refname_format(r->name + 5, 0))
+		return;
+
+	lock = lock_ref_sha1_basic(r->name, r->sha1, 0, NULL);
 	if (lock) {
 		unlink_or_warn(git_path("%s", r->name));
 		unlock_ref(lock);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 19/20] refs.c: make prune_ref use a transaction to delete the ref

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 09:03:36 -0700

Change prune_ref to delete the ref using a ref transaction. To do this we also
need to add a new flag REF_ISPRUNING that will tell the transaction that we
do not want to delete this ref from the packed refs. This flag is private to
refs.c and not exposed to external callers.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 28 +++++++++++++++++++++-------
 refs.h | 13 +++++++++++--
 2 files changed, 32 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index f0883d0..5b2d335 100644
--- a/refs.c
+++ b/refs.c
@@ -25,6 +25,11 @@ static unsigned char refname_disposition[256] = {
 };
 
 /*
+ * Used as a flag to ref_transaction_delete when a loose ref is being
+ * pruned.
+ */
+#define REF_ISPRUNING	0x0100
+/*
  * Try to read one refname component from the front of refname.
  * Return the length of the component found, or -1 if the component is
  * not legal.  It is legal if it is something reasonable to have under
@@ -2382,17 +2387,25 @@ static void try_remove_empty_parents(char *name)
 /* make sure nobody touched the ref, and unlink */
 static void prune_ref(struct ref_to_prune *r)
 {
-	struct ref_lock *lock;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
 	if (check_refname_format(r->name + 5, 0))
 		return;
 
-	lock = lock_ref_sha1_basic(r->name, r->sha1, 0, NULL);
-	if (lock) {
-		unlink_or_warn(git_path("%s", r->name));
-		unlock_ref(lock);
-		try_remove_empty_parents(r->name);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_delete(transaction, r->name, r->sha1,
+				   REF_ISPRUNING, 1, &err) ||
+	    ref_transaction_commit(transaction, NULL, &err)) {
+		ref_transaction_free(transaction);
+		error("%s", err.buf);
+		strbuf_release(&err);
+		return;
 	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	try_remove_empty_parents(r->name);
 }
 
 static void prune_refs(struct ref_to_prune *r)
@@ -3597,8 +3610,9 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			delnames[delnum++] = update->lock->ref_name;
 			ret |= delete_ref_loose(update->lock, update->type);
+			if (!(update->flags & REF_ISPRUNING))
+				delnames[delnum++] = update->lock->ref_name;
 		}
 	}
 
diff --git a/refs.h b/refs.h
index 65dd593..69ef28c 100644
--- a/refs.h
+++ b/refs.h
@@ -170,9 +170,18 @@ extern int ref_exists(const char *);
  */
 extern int peel_ref(const char *refname, unsigned char *sha1);
 
-/** Locks any ref (for 'HEAD' type refs). */
+/*
+ * Flags controlling lock_any_ref_for_update(), ref_transaction_update(),
+ * ref_transaction_create(), etc.
+ * REF_NODEREF: act on the ref directly, instead of dereferencing
+ *              symbolic references.
+ *
+ * Flags >= 0x100 are reserved for internal use.
+ */
 #define REF_NODEREF	0x01
-/* errno is set to something meaningful on failure */
+/*
+ * This function sets errno to something meaningful on failure.
+ */
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
 						int flags, int *type_p);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 20/20] refs.c: make delete_ref use a transaction

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 09:22:45 -0700

Change delete_ref to use a ref transaction for the deletion. At the same time
since we no longer have any callers of repack_without_ref we can now delete
this function.

Change delete_ref to return 0 on success and 1 on failure instead of the
previous 0 on success either 1 or -1 on failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Thanks for reading.

 refs.c | 35 ++++++++++++++---------------------
 1 file changed, 14 insertions(+), 21 deletions(-)

diff --git a/refs.c b/refs.c
index 5b2d335..7996be9 100644
--- a/refs.c
+++ b/refs.c
@@ -2548,11 +2548,6 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 	return ret;
 }
 
-static int repack_without_ref(const char *refname)
-{
-	return repack_without_refs(&refname, 1, NULL);
-}
-
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
@@ -2570,24 +2565,22 @@ static int delete_ref_loose(struct ref_lock *lock, int flag)
 
 int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 {
-	struct ref_lock *lock;
-	int ret = 0, flag = 0;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
-	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
-	if (!lock)
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_delete(transaction, refname, sha1, delopt,
+				   sha1 && !is_null_sha1(sha1), &err) ||
+	    ref_transaction_commit(transaction, NULL, &err)) {
+		error("%s", err.buf);
+		ref_transaction_free(transaction);
+		strbuf_release(&err);
 		return 1;
-	ret |= delete_ref_loose(lock, flag);
-
-	/* removing the loose one could have resurrected an earlier
-	 * packed one.  Also, if it was not loose we need to repack
-	 * without it.
-	 */
-	ret |= repack_without_ref(lock->ref_name);
-
-	unlink_or_warn(git_path("logs/%s", lock->ref_name));
-	clear_loose_ref_cache(&ref_cache);
-	unlock_ref(lock);
-	return ret;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	return 0;
 }
 
 /*
-- 
2.1.0.rc2.206.gedb03e5

Re: [PATCH 0/20] rs/ref-transaction-1 (Re: Transaction patch series overview)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Jonathan Nieder wrote:
>
>> This series teaches the tag, replace, commit, cherry-pick,
>> fast-import, checkout -b, branch, receive-pack, and http-fetch
>> commands and all update_ref and delete_ref callers to use the ref
>> transaction API instead of lock_ref_sha1.
>>
>> The main user-visible change should be some cosmetic changes to error
>> messages.  The series also combines multiple ref updates into a single
>> transaction in 'git http-fetch -w' and when writing tags in
>> fast-import but otherwise doesn't change the granularity of
>> transactions.
>>
>> Reviewed at https://code-review.googlesource.com/#/q/topic:ref-transaction-1

Thanks.

Will queue, but the some other topics may have to disappear from my
tree while I try to rebase them on top (or while I wait for you guys
to send a reroll, making it unnecesary to do the rebase myself).

Using Gerrit to review Git patches (was: Re: Transaction patch series overview)

[Michael Haggerty]

On 08/26/2014 02:03 AM, Jonathan Nieder wrote:
> Jonathan Nieder wrote:
> [...]
>> I've having trouble keeping track of how patches change, which patches
>> have been reviewed and which haven't, unaddressed comments, and so on,
>> so as an experiment I've pushed this part of the series to the Gerrit
>> server at
>>
>>  https://code-review.googlesource.com/#/q/topic:ref-transaction-1
> 
> Outcome of the experiment: patches gained some minor changes and then
> [...]
> 
> I found the web UI helpful in seeing how each patch evolved since I
> last looked at it.  Interdiff relative to the version in pu is below.
> I'm still hoping for a tweak in response to a minor comment and then I
> can put up a copy of the updated series to pull.

Thanks for organizing this "experiment". I was one of the guinea pigs
:-) I have wanted to review more of Ronnie's patches (actually, all of
them!) but have been overwhelmed by the number of iterations and the
number of patch series flying around in parallel. I was also interested
to try out Gerrit, which I haven't used before. So I took up Jonathan's
invitation and reviewed the first patch series in Gerrit. Here are some
of my first impressions.

* Overall, I found it easier to review commits in Gerrit than on the
mailing list, especially a long patch series like this one that has seen
so much flux. It was easier to see the comments from all reviewers that
apply to a patch, which is difficult on the mailing list when comments
are scattered over the many iterations of the patch series. It was
easier to incrementally increase the context around a patch. It was easy
to use the copy-paste commands provided in the "download" menu to fetch
the commit that I was reviewing into my local Git repository, and from
there to build it or investigate it using other tools.

* The Gerrit interface is very busy. It was somewhat overwhelming to me
as a beginner. On the other hand, the help menus ("?" key) are good and
the keyboard shortcuts are convenient. I didn't have to read much
documentation to get started doing review in Gerrit, at least at a basic
level.

* During two of my big Gerrit sessions the website was very responsive
and pleasant to use. During the third, it was terribly slow, like 5 - 15
seconds per page update. If I had only experienced the slow behavior, I
would have rejected Gerrit immediately. I hope that the slow behavior
was a rare anomaly.

* Gerrit sends out an endless flood of emails that mostly seem pretty
useless to me. I wish it weren't so chatty and that its emails were
better organized.

* At one point a back-and-forth in a line comment grew into a more
general issue that was more appropriate for the mailing list. The
transition from Gerrit to mailing list was a bit awkward.

So, overall I found it pleasant and efficient to review patches in
Gerrit. I would welcome more such "experiments". It would have been even
better if Gerrit would generate more useful notification emails.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu

Re: [PATCH 0/20] rs/ref-transaction-1 (Re: Transaction patch series overview)

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Jonathan Nieder wrote:
>>
>>> This series teaches the tag, replace, commit, cherry-pick,
>>> fast-import, checkout -b, branch, receive-pack, and http-fetch
>>> commands and all update_ref and delete_ref callers to use the ref
>>> transaction API instead of lock_ref_sha1.
>>>
>>> The main user-visible change should be some cosmetic changes to error
>>> messages.  The series also combines multiple ref updates into a single
>>> transaction in 'git http-fetch -w' and when writing tags in
>>> fast-import but otherwise doesn't change the granularity of
>>> transactions.
>>>
>>> Reviewed at https://code-review.googlesource.com/#/q/topic:ref-transaction-1
>
> Thanks.
>
> Will queue, but the some other topics may have to disappear from my
> tree while I try to rebase them on top (or while I wait for you guys
> to send a reroll, making it unnecesary to do the rebase myself).

I managed to rebase all the rs/ref-transaction* topics and pushed
out the result in 'pu'.  Please eyeball the result.

I had to bump nd/multiple-work-trees, which Duy asked to kick out of
'next' earlier, out of 'pu' because I did not want to deal with
conflict resolution that will go to waste when the topic is rerolled
which would happen soon.

Re: Using Gerrit to review Git patches (was: Re: Transaction patch series overview)

[Shawn Pearce]

On Wed, Aug 27, 2014 at 2:53 PM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> On 08/26/2014 02:03 AM, Jonathan Nieder wrote:
>> Jonathan Nieder wrote:
>> [...]
>>> I've having trouble keeping track of how patches change, which patches
>>> have been reviewed and which haven't, unaddressed comments, and so on,
>>> so as an experiment I've pushed this part of the series to the Gerrit
>>> server at
>>>
>>>  https://code-review.googlesource.com/#/q/topic:ref-transaction-1
>>
>> Outcome of the experiment: patches gained some minor changes and then
>> [...]
>>
>> I found the web UI helpful in seeing how each patch evolved since I
>> last looked at it.  Interdiff relative to the version in pu is below.
>> I'm still hoping for a tweak in response to a minor comment and then I
>> can put up a copy of the updated series to pull.
>
> Thanks for organizing this "experiment". I was one of the guinea pigs
> :-) I have wanted to review more of Ronnie's patches (actually, all of
> them!) but have been overwhelmed by the number of iterations and the
> number of patch series flying around in parallel. I was also interested
> to try out Gerrit, which I haven't used before. So I took up Jonathan's
> invitation and reviewed the first patch series in Gerrit. Here are some
> of my first impressions.

Thank you for making this writeup. It contains many valuable nuggets
that we should follow up on.

> * Overall, I found it easier to review commits in Gerrit than on the
> mailing list, especially a long patch series like this one that has seen
> so much flux. It was easier to see the comments from all reviewers that
> apply to a patch, which is difficult on the mailing list when comments
> are scattered over the many iterations of the patch series. It was
> easier to incrementally increase the context around a patch. It was easy
> to use the copy-paste commands provided in the "download" menu to fetch
> the commit that I was reviewing into my local Git repository, and from
> there to build it or investigate it using other tools.
>
> * The Gerrit interface is very busy. It was somewhat overwhelming to me
> as a beginner.

Very astute observation. I have been using it for 5+ years so its hard
for me to see things like this.

Unfortunately many parts of the UI have grown over the years.
Contributors come along and want to add X, months later another adds
field Y. Suddenly we have the entire alphabet and we don't know how it
got that way. Sounds like git reset --{hard|soft|mixed|wtf}. :)

Its not an excuse. The UI should be less overwhelming. Its going to be
a challenge.

> On the other hand, the help menus ("?" key) are good and
> the keyboard shortcuts are convenient. I didn't have to read much
> documentation to get started doing review in Gerrit, at least at a basic
> level.

Yay!

> * During two of my big Gerrit sessions the website was very responsive
> and pleasant to use. During the third, it was terribly slow, like 5 - 15
> seconds per page update. If I had only experienced the slow behavior, I
> would have rejected Gerrit immediately. I hope that the slow behavior
> was a rare anomaly.

This third session was a rare anomaly unique to
code-review.googlesource.com. It isn't typical for Gerrit Code Review.

We have a server set in Europe that was mostly answering you in the
first two sessions. During the third session this server set was
temporarily offline. Requests degraded to a server set in the US,
which pushed us into a worst-case scenario due to a misconfiguration
of our load balancers. The lag you experienced was mostly due to this
misconfiguration, not EU-US networking.

Earlier today I brought back up the European server set, so latency
should be reduced again.

Unfortunately I haven't figured out how to fix the load balancer
misconfiguration. :(

> * Gerrit sends out an endless flood of emails that mostly seem pretty
> useless to me. I wish it weren't so chatty and that its emails were
> better organized.

Absolutely agree.

> * At one point a back-and-forth in a line comment grew into a more
> general issue that was more appropriate for the mailing list. The
> transition from Gerrit to mailing list was a bit awkward.

We find this a bit awkward in JGit too. Our solution has been to
(mostly) keep discussion within the code review context. This has to
some extent forced more discussion to be around specific code, and
avoid wandering off into the philosophical weeds. Without concrete
code to talk about, there is nothing to say. :)

A nice feature of Gerrit is another contributor can add their own
iterations to the same commit in the series, offering up alternative
proposals. Like on the mailing list, this allows someone to just say
"what if you did it this way...", and give everyone a concrete chunk
of code to consider and discuss. Again this avoids wandering off into
the weeds.

> So, overall I found it pleasant and efficient to review patches in
> Gerrit. I would welcome more such "experiments". It would have been even
> better if Gerrit would generate more useful notification emails.

One idea has been to just stop sending automatic emails, and instead
require the author to send a message saying "please take another look
at my series". Just disable the #@!(@(! automatic email delivery that
stems out of `git push ...` creating or updating pending reviews.

Unfortunately there are factions within the Gerrit contributor base
that find value in those automated emails. :(

[PATCH v22 0/22] rs/ref-transaction-1 (Re: Transaction patch series overview)

[Jonathan Nieder]

Hi Junio,

Michael reviewed the rest of rs/ref-transaction-1 (thanks!) and some
more small tweaks resulted --- in particular, there was a missing call
to ref_transaction_free.  Interdiff below.  This is meant to replace
rs/ref-transaction-1 in 'pu' and should be ready for 'next'.

These patches are also available from the git repository at

  git://repo.or.cz/git/jrn.git tags/rs/ref-transaction-1

"Use ref transactions", early part

Ronnie explains:

	This patch series expands on the transaction API. It converts
	ref updates, inside refs.c as well as external, to use the
	transaction API for updates. This makes most ref updates
	atomic when there are failures locking or writing to a ref.

This series teaches the tag, replace, commit, cherry-pick, fast-import,
checkout -b, branch, receive-pack, and http-fetch commands and all
update_ref and delete_ref callers to use the ref transaction API instead
of lock_ref_sha1.

The main user-visible change should be some cosmetic changes to error
messages.  The series also combines multiple ref updates into a single
transaction in 'git http-fetch -w' and when writing tags in fast-import
but otherwise doesn't change the granularity of transactions.

Jonathan Nieder (2):
      update-ref --stdin: narrow scope of err strbuf
      update-ref --stdin: pass transaction around explicitly

Ronnie Sahlberg (20):
      refs.c: change ref_transaction_create to do error checking and return status
      refs.c: update ref_transaction_delete to check for error and return status
      refs.c: make ref_transaction_begin take an err argument
      refs.c: add transaction.status and track OPEN/CLOSED
      tag.c: use ref transactions when doing updates
      replace.c: use the ref transaction functions for updates
      commit.c: use ref transactions for updates
      sequencer.c: use ref transactions for all ref updates
      fast-import.c: change update_branch to use ref transactions
      branch.c: use ref transaction for all ref updates
      refs.c: change update_ref to use a transaction
      receive-pack.c: use a reference transaction for updating the refs
      fast-import.c: use a ref transaction when dumping tags
      walker.c: use ref transaction for ref updates
      refs.c: make lock_ref_sha1 static
      refs.c: remove the update_ref_lock function
      refs.c: remove the update_ref_write function
      refs.c: remove lock_ref_sha1
      refs.c: make prune_ref use a transaction to delete the ref
      refs.c: make delete_ref use a transaction

 branch.c               |  31 ++++---
 builtin/commit.c       |  25 +++--
 builtin/receive-pack.c |  25 +++--
 builtin/replace.c      |  14 +--
 builtin/tag.c          |  16 ++--
 builtin/update-ref.c   |  52 +++++++----
 fast-import.c          |  54 +++++++----
 refs.c                 | 245 ++++++++++++++++++++++++++++---------------------
 refs.h                 |  77 ++++++++++++----
 sequencer.c            |  26 ++++--
 walker.c               |  70 ++++++++------
 11 files changed, 393 insertions(+), 242 deletions(-)
---
Changes since v21:

 builtin/update-ref.c | 38 +++++++++++++++++++++++++-------------
 refs.c               |  1 +
 2 files changed, 26 insertions(+), 13 deletions(-)

diff --git c/builtin/update-ref.c w/builtin/update-ref.c
index 96a53b9..54a48c0 100644
--- c/builtin/update-ref.c
+++ w/builtin/update-ref.c
@@ -12,11 +12,8 @@ static const char * const git_update_ref_usage[] = {
 	NULL
 };
 
-static struct ref_transaction *transaction;
-
 static char line_termination = '\n';
 static int update_flags;
-static struct strbuf err = STRBUF_INIT;
 
 /*
  * Parse one whitespace- or NUL-terminated, possibly C-quoted argument
@@ -177,8 +174,10 @@ static int parse_next_sha1(struct strbuf *input, const char **next,
  * depending on how line_termination is set.
  */
 
-static const char *parse_cmd_update(struct strbuf *input, const char *next)
+static const char *parse_cmd_update(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char new_sha1[20];
 	unsigned char old_sha1[20];
@@ -204,12 +203,15 @@ static const char *parse_cmd_update(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
 
-static const char *parse_cmd_create(struct strbuf *input, const char *next)
+static const char *parse_cmd_create(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char new_sha1[20];
 
@@ -232,12 +234,15 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
 
-static const char *parse_cmd_delete(struct strbuf *input, const char *next)
+static const char *parse_cmd_delete(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char old_sha1[20];
 	int have_old;
@@ -264,12 +269,15 @@ static const char *parse_cmd_delete(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
 
-static const char *parse_cmd_verify(struct strbuf *input, const char *next)
+static const char *parse_cmd_verify(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char new_sha1[20];
 	unsigned char old_sha1[20];
@@ -297,6 +305,7 @@ static const char *parse_cmd_verify(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
@@ -310,7 +319,7 @@ static const char *parse_cmd_option(struct strbuf *input, const char *next)
 	return next + 8;
 }
 
-static void update_refs_stdin(void)
+static void update_refs_stdin(struct ref_transaction *transaction)
 {
 	struct strbuf input = STRBUF_INIT;
 	const char *next;
@@ -325,13 +334,13 @@ static void update_refs_stdin(void)
 		else if (isspace(*next))
 			die("whitespace before command: %s", next);
 		else if (starts_with(next, "update "))
-			next = parse_cmd_update(&input, next + 7);
+			next = parse_cmd_update(transaction, &input, next + 7);
 		else if (starts_with(next, "create "))
-			next = parse_cmd_create(&input, next + 7);
+			next = parse_cmd_create(transaction, &input, next + 7);
 		else if (starts_with(next, "delete "))
-			next = parse_cmd_delete(&input, next + 7);
+			next = parse_cmd_delete(transaction, &input, next + 7);
 		else if (starts_with(next, "verify "))
-			next = parse_cmd_verify(&input, next + 7);
+			next = parse_cmd_verify(transaction, &input, next + 7);
 		else if (starts_with(next, "option "))
 			next = parse_cmd_option(&input, next + 7);
 		else
@@ -365,6 +374,9 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		die("Refusing to perform update with empty message.");
 
 	if (read_stdin) {
+		struct strbuf err = STRBUF_INIT;
+		struct ref_transaction *transaction;
+
 		transaction = ref_transaction_begin(&err);
 		if (!transaction)
 			die("%s", err.buf);
@@ -372,7 +384,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 			usage_with_options(git_update_ref_usage, options);
 		if (end_null)
 			line_termination = '\0';
-		update_refs_stdin();
+		update_refs_stdin(transaction);
 		if (ref_transaction_commit(transaction, msg, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
diff --git c/refs.c w/refs.c
index 7996be9..7235574 100644
--- c/refs.c
+++ w/refs.c
@@ -3511,6 +3511,7 @@ int update_ref(const char *action, const char *refname,
 		return 1;
 	}
 	strbuf_release(&err);
+	ref_transaction_free(t);
 	return 0;
 }
 

[PATCH 01/22] refs.c: change ref_transaction_create to do error checking and return status

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:26:44 -0700

Do basic error checking in ref_transaction_create() and make it return
non-zero on error. Update all callers to check the result of
ref_transaction_create(). There are currently no conditions in _create that
will return error but there will be in the future. Add an err argument that
will be updated on failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/update-ref.c |  4 +++-
 refs.c               | 18 ++++++++++++------
 refs.h               | 48 +++++++++++++++++++++++++++++++++++++++++-------
 3 files changed, 56 insertions(+), 14 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 3067b11..41121fa 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -226,7 +226,9 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)
 	if (*next != line_termination)
 		die("create %s: extra input: %s", refname, next);
 
-	ref_transaction_create(transaction, refname, new_sha1, update_flags);
+	if (ref_transaction_create(transaction, refname, new_sha1,
+				   update_flags, &err))
+		die("%s", err.buf);
 
 	update_flags = 0;
 	free(refname);
diff --git a/refs.c b/refs.c
index 3f05e88..c49f1c6 100644
--- a/refs.c
+++ b/refs.c
@@ -3449,18 +3449,24 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	return 0;
 }
 
-void ref_transaction_create(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *new_sha1,
-			    int flags)
+int ref_transaction_create(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *new_sha1,
+			   int flags,
+			   struct strbuf *err)
 {
-	struct ref_update *update = add_update(transaction, refname);
+	struct ref_update *update;
+
+	if (!new_sha1 || is_null_sha1(new_sha1))
+		die("BUG: create ref with null new_sha1");
+
+	update = add_update(transaction, refname);
 
-	assert(!is_null_sha1(new_sha1));
 	hashcpy(update->new_sha1, new_sha1);
 	hashclr(update->old_sha1);
 	update->flags = flags;
 	update->have_old = 1;
+	return 0;
 }
 
 void ref_transaction_delete(struct ref_transaction *transaction,
diff --git a/refs.h b/refs.h
index c5376ce..b648819 100644
--- a/refs.h
+++ b/refs.h
@@ -10,6 +10,38 @@ struct ref_lock {
 	int force_write;
 };
 
+/*
+ * A ref_transaction represents a collection of ref updates
+ * that should succeed or fail together.
+ *
+ * Calling sequence
+ * ----------------
+ * - Allocate and initialize a `struct ref_transaction` by calling
+ *   `ref_transaction_begin()`.
+ *
+ * - List intended ref updates by calling functions like
+ *   `ref_transaction_update()` and `ref_transaction_create()`.
+ *
+ * - Call `ref_transaction_commit()` to execute the transaction.
+ *   If this succeeds, the ref updates will have taken place and
+ *   the transaction cannot be rolled back.
+ *
+ * - At any time call `ref_transaction_free()` to discard the
+ *   transaction and free associated resources.  In particular,
+ *   this rolls back the transaction if it has not been
+ *   successfully committed.
+ *
+ * Error handling
+ * --------------
+ *
+ * On error, transaction functions append a message about what
+ * went wrong to the 'err' argument.  The message mentions what
+ * ref was being updated (if any) when the error occurred so it
+ * can be passed to 'die' or 'error' as-is.
+ *
+ * The message is appended to err without first clearing err.
+ * err will not be '\n' terminated.
+ */
 struct ref_transaction;
 
 /*
@@ -248,7 +280,7 @@ struct ref_transaction *ref_transaction_begin(void);
  * it must not have existed beforehand.
  * Function returns 0 on success and non-zero on failure. A failure to update
  * means that the transaction as a whole has failed and will need to be
- * rolled back. On failure the err buffer will be updated.
+ * rolled back.
  */
 int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
@@ -262,11 +294,15 @@ int ref_transaction_update(struct ref_transaction *transaction,
  * that the reference should have after the update; it must not be the
  * null SHA-1.  It is verified that the reference does not exist
  * already.
+ * Function returns 0 on success and non-zero on failure. A failure to create
+ * means that the transaction as a whole has failed and will need to be
+ * rolled back.
  */
-void ref_transaction_create(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *new_sha1,
-			    int flags);
+int ref_transaction_create(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *new_sha1,
+			   int flags,
+			   struct strbuf *err);
 
 /*
  * Add a reference deletion to transaction.  If have_old is true, then
@@ -282,8 +318,6 @@ void ref_transaction_delete(struct ref_transaction *transaction,
  * Commit all of the changes that have been queued in transaction, as
  * atomically as possible.  Return a nonzero value if there is a
  * problem.
- * If err is non-NULL we will add an error string to it to explain why
- * the transaction failed. The string does not end in newline.
  */
 int ref_transaction_commit(struct ref_transaction *transaction,
 			   const char *msg, struct strbuf *err);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 02/22] refs.c: update ref_transaction_delete to check for error and return status

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:27:45 -0700

Change ref_transaction_delete() to do basic error checking and return
non-zero on error. Update all callers to check the return for
ref_transaction_delete(). There are currently no conditions in _delete that
will return error but there will be in the future. Add an err argument that
will be updated on failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/update-ref.c |  5 +++--
 refs.c               | 16 +++++++++++-----
 refs.h               | 12 ++++++++----
 3 files changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 41121fa..7c9c248 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -258,8 +258,9 @@ static const char *parse_cmd_delete(struct strbuf *input, const char *next)
 	if (*next != line_termination)
 		die("delete %s: extra input: %s", refname, next);
 
-	ref_transaction_delete(transaction, refname, old_sha1,
-			       update_flags, have_old);
+	if (ref_transaction_delete(transaction, refname, old_sha1,
+				   update_flags, have_old, &err))
+		die("%s", err.buf);
 
 	update_flags = 0;
 	free(refname);
diff --git a/refs.c b/refs.c
index c49f1c6..40f04f4 100644
--- a/refs.c
+++ b/refs.c
@@ -3469,19 +3469,25 @@ int ref_transaction_create(struct ref_transaction *transaction,
 	return 0;
 }
 
-void ref_transaction_delete(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *old_sha1,
-			    int flags, int have_old)
+int ref_transaction_delete(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *old_sha1,
+			   int flags, int have_old,
+			   struct strbuf *err)
 {
-	struct ref_update *update = add_update(transaction, refname);
+	struct ref_update *update;
 
+	if (have_old && !old_sha1)
+		die("BUG: have_old is true but old_sha1 is NULL");
+
+	update = add_update(transaction, refname);
 	update->flags = flags;
 	update->have_old = have_old;
 	if (have_old) {
 		assert(!is_null_sha1(old_sha1));
 		hashcpy(update->old_sha1, old_sha1);
 	}
+	return 0;
 }
 
 int update_ref(const char *action, const char *refname,
diff --git a/refs.h b/refs.h
index b648819..71389a1 100644
--- a/refs.h
+++ b/refs.h
@@ -308,11 +308,15 @@ int ref_transaction_create(struct ref_transaction *transaction,
  * Add a reference deletion to transaction.  If have_old is true, then
  * old_sha1 holds the value that the reference should have had before
  * the update (which must not be the null SHA-1).
+ * Function returns 0 on success and non-zero on failure. A failure to delete
+ * means that the transaction as a whole has failed and will need to be
+ * rolled back.
  */
-void ref_transaction_delete(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *old_sha1,
-			    int flags, int have_old);
+int ref_transaction_delete(struct ref_transaction *transaction,
+			   const char *refname,
+			   const unsigned char *old_sha1,
+			   int flags, int have_old,
+			   struct strbuf *err);
 
 /*
  * Commit all of the changes that have been queued in transaction, as
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 03/22] refs.c: make ref_transaction_begin take an err argument

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 19 May 2014 10:42:34 -0700

Add an err argument to _begin so that on non-fatal failures in future ref
backends we can report a nice error back to the caller.
While _begin can currently never fail for other reasons than OOM, in which
case we die() anyway, we may add other types of backends in the future.
For example, a hypothetical MySQL backend could fail in _begin with
"Can not connect to MySQL server. No route to host".

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/update-ref.c | 5 ++++-
 refs.c               | 2 +-
 refs.h               | 2 +-
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 7c9c248..96a53b9 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -365,7 +365,9 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		die("Refusing to perform update with empty message.");
 
 	if (read_stdin) {
-		transaction = ref_transaction_begin();
+		transaction = ref_transaction_begin(&err);
+		if (!transaction)
+			die("%s", err.buf);
 		if (delete || no_deref || argc > 0)
 			usage_with_options(git_update_ref_usage, options);
 		if (end_null)
@@ -374,6 +376,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		if (ref_transaction_commit(transaction, msg, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
+		strbuf_release(&err);
 		return 0;
 	}
 
diff --git a/refs.c b/refs.c
index 40f04f4..9cb7908 100644
--- a/refs.c
+++ b/refs.c
@@ -3397,7 +3397,7 @@ struct ref_transaction {
 	size_t nr;
 };
 
-struct ref_transaction *ref_transaction_begin(void)
+struct ref_transaction *ref_transaction_begin(struct strbuf *err)
 {
 	return xcalloc(1, sizeof(struct ref_transaction));
 }
diff --git a/refs.h b/refs.h
index 71389a1..3f37c65 100644
--- a/refs.h
+++ b/refs.h
@@ -262,7 +262,7 @@ enum action_on_err {
  * Begin a reference transaction.  The reference transaction must
  * be freed by calling ref_transaction_free().
  */
-struct ref_transaction *ref_transaction_begin(void);
+struct ref_transaction *ref_transaction_begin(struct strbuf *err);
 
 /*
  * The following functions add a reference check or update to a
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 04/22] refs.c: add transaction.status and track OPEN/CLOSED

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 12:06:19 -0700

Track the state of a transaction in a new state field. Check the field for
sanity, i.e. that state must be OPEN when _commit/_create/_delete or
_update is called or else die(BUG:...)

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 9cb7908..cc63056 100644
--- a/refs.c
+++ b/refs.c
@@ -3387,6 +3387,21 @@ struct ref_update {
 };
 
 /*
+ * Transaction states.
+ * OPEN:   The transaction is in a valid state and can accept new updates.
+ *         An OPEN transaction can be committed.
+ * CLOSED: A closed transaction is no longer active and no other operations
+ *         than free can be used on it in this state.
+ *         A transaction can either become closed by successfully committing
+ *         an active transaction or if there is a failure while building
+ *         the transaction thus rendering it failed/inactive.
+ */
+enum ref_transaction_state {
+	REF_TRANSACTION_OPEN   = 0,
+	REF_TRANSACTION_CLOSED = 1
+};
+
+/*
  * Data structure for holding a reference transaction, which can
  * consist of checks and updates to multiple references, carried out
  * as atomically as possible.  This structure is opaque to callers.
@@ -3395,6 +3410,7 @@ struct ref_transaction {
 	struct ref_update **updates;
 	size_t alloc;
 	size_t nr;
+	enum ref_transaction_state state;
 };
 
 struct ref_transaction *ref_transaction_begin(struct strbuf *err)
@@ -3437,6 +3453,9 @@ int ref_transaction_update(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: update called for transaction that is not open");
+
 	if (have_old && !old_sha1)
 		die("BUG: have_old is true but old_sha1 is NULL");
 
@@ -3457,6 +3476,9 @@ int ref_transaction_create(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: create called for transaction that is not open");
+
 	if (!new_sha1 || is_null_sha1(new_sha1))
 		die("BUG: create ref with null new_sha1");
 
@@ -3477,6 +3499,9 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: delete called for transaction that is not open");
+
 	if (have_old && !old_sha1)
 		die("BUG: have_old is true but old_sha1 is NULL");
 
@@ -3532,8 +3557,13 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	int n = transaction->nr;
 	struct ref_update **updates = transaction->updates;
 
-	if (!n)
+	if (transaction->state != REF_TRANSACTION_OPEN)
+		die("BUG: commit called for transaction that is not open");
+
+	if (!n) {
+		transaction->state = REF_TRANSACTION_CLOSED;
 		return 0;
+	}
 
 	/* Allocate work space */
 	delnames = xmalloc(sizeof(*delnames) * n);
@@ -3595,6 +3625,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	clear_loose_ref_cache(&ref_cache);
 
 cleanup:
+	transaction->state = REF_TRANSACTION_CLOSED;
+
 	for (i = 0; i < n; i++)
 		if (updates[i]->lock)
 			unlock_ref(updates[i]->lock);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 05/22] tag.c: use ref transactions when doing updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:30:41 -0700

Change tag.c to use ref transactions for all ref updates.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/tag.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/builtin/tag.c b/builtin/tag.c
index c6e8a71..f3f172f 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -548,7 +548,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	struct strbuf ref = STRBUF_INIT;
 	unsigned char object[20], prev[20];
 	const char *object_ref, *tag;
-	struct ref_lock *lock;
 	struct create_tag_options opt;
 	char *cleanup_arg = NULL;
 	int annotate = 0, force = 0, lines = -1;
@@ -556,6 +555,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	const char *msgfile = NULL, *keyid = NULL;
 	struct msg_arg msg = { 0, STRBUF_INIT };
 	struct commit_list *with_commit = NULL;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 	struct option options[] = {
 		OPT_CMDMODE('l', "list", &cmdmode, N_("list tag names"), 'l'),
 		{ OPTION_INTEGER, 'n', NULL, &lines, N_("n"),
@@ -701,14 +702,17 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (annotate)
 		create_tag(object, tag, &buf, &opt, prev, object);
 
-	lock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);
-	if (!lock)
-		die(_("%s: cannot lock the ref"), ref.buf);
-	if (write_ref_sha1(lock, object, NULL) < 0)
-		die(_("%s: cannot update the ref"), ref.buf);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref.buf, object, prev,
+				   0, 1, &err) ||
+	    ref_transaction_commit(transaction, NULL, &err))
+		die("%s", err.buf);
+	ref_transaction_free(transaction);
 	if (force && !is_null_sha1(prev) && hashcmp(prev, object))
 		printf(_("Updated tag '%s' (was %s)\n"), tag, find_unique_abbrev(prev, DEFAULT_ABBREV));
 
+	strbuf_release(&err);
 	strbuf_release(&buf);
 	strbuf_release(&ref);
 	return 0;
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 06/22] replace.c: use the ref transaction functions for updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:32:29 -0700

Update replace.c to use ref transactions for updates.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/replace.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/builtin/replace.c b/builtin/replace.c
index 1bb491d..1fcd06d 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -153,7 +153,8 @@ static int replace_object_sha1(const char *object_ref,
 	unsigned char prev[20];
 	enum object_type obj_type, repl_type;
 	char ref[PATH_MAX];
-	struct ref_lock *lock;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
 	obj_type = sha1_object_info(object, NULL);
 	repl_type = sha1_object_info(repl, NULL);
@@ -166,12 +167,13 @@ static int replace_object_sha1(const char *object_ref,
 
 	check_ref_valid(object, prev, ref, sizeof(ref), force);
 
-	lock = lock_any_ref_for_update(ref, prev, 0, NULL);
-	if (!lock)
-		die("%s: cannot lock the ref", ref);
-	if (write_ref_sha1(lock, repl, NULL) < 0)
-		die("%s: cannot update the ref", ref);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref, repl, prev, 0, 1, &err) ||
+	    ref_transaction_commit(transaction, NULL, &err))
+		die("%s", err.buf);
 
+	ref_transaction_free(transaction);
 	return 0;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 07/22] commit.c: use ref transactions for updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:34:19 -0700

Change commit.c to use ref transactions for all ref updates.
Make sure we pass a NULL pointer to ref_transaction_update if have_old
is false.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/commit.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 5e2221c..9bf1003 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1627,11 +1627,12 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	const char *index_file, *reflog_msg;
 	char *nl;
 	unsigned char sha1[20];
-	struct ref_lock *ref_lock;
 	struct commit_list *parents = NULL, **pptr = &parents;
 	struct stat statbuf;
 	struct commit *current_head = NULL;
 	struct commit_extra_header *extra = NULL;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
 	if (argc == 2 && !strcmp(argv[1], "-h"))
 		usage_with_options(builtin_commit_usage, builtin_commit_options);
@@ -1753,16 +1754,6 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	strbuf_release(&author_ident);
 	free_commit_extra_headers(extra);
 
-	ref_lock = lock_any_ref_for_update("HEAD",
-					   !current_head
-					   ? NULL
-					   : current_head->object.sha1,
-					   0, NULL);
-	if (!ref_lock) {
-		rollback_index_files();
-		die(_("cannot lock HEAD ref"));
-	}
-
 	nl = strchr(sb.buf, '\n');
 	if (nl)
 		strbuf_setlen(&sb, nl + 1 - sb.buf);
@@ -1771,10 +1762,17 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	strbuf_insert(&sb, 0, reflog_msg, strlen(reflog_msg));
 	strbuf_insert(&sb, strlen(reflog_msg), ": ", 2);
 
-	if (write_ref_sha1(ref_lock, sha1, sb.buf) < 0) {
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, "HEAD", sha1,
+				   current_head
+				   ? current_head->object.sha1 : NULL,
+				   0, !!current_head, &err) ||
+	    ref_transaction_commit(transaction, sb.buf, &err)) {
 		rollback_index_files();
-		die(_("cannot update HEAD ref"));
+		die("%s", err.buf);
 	}
+	ref_transaction_free(transaction);
 
 	unlink(git_path("CHERRY_PICK_HEAD"));
 	unlink(git_path("REVERT_HEAD"));
@@ -1803,5 +1801,6 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	if (!quiet)
 		print_summary(prefix, sha1, !current_head);
 
+	strbuf_release(&err);
 	return 0;
 }
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 08/22] sequencer.c: use ref transactions for all ref updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 15:37:45 -0700

Change to use ref transactions for all updates to refs.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sequencer.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 0a80c58..5e93b6a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -272,23 +272,33 @@ static int error_dirty_index(struct replay_opts *opts)
 static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 			int unborn, struct replay_opts *opts)
 {
-	struct ref_lock *ref_lock;
+	struct ref_transaction *transaction;
 	struct strbuf sb = STRBUF_INIT;
-	int ret;
+	struct strbuf err = STRBUF_INIT;
 
 	read_cache();
 	if (checkout_fast_forward(from, to, 1))
 		exit(1); /* the callee should have complained already */
-	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from,
-					   0, NULL);
-	if (!ref_lock)
-		return error(_("Failed to lock HEAD during fast_forward_to"));
 
 	strbuf_addf(&sb, "%s: fast-forward", action_name(opts));
-	ret = write_ref_sha1(ref_lock, to, sb.buf);
+
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, "HEAD",
+				   to, unborn ? null_sha1 : from,
+				   0, 1, &err) ||
+	    ref_transaction_commit(transaction, sb.buf, &err)) {
+		ref_transaction_free(transaction);
+		error("%s", err.buf);
+		strbuf_release(&sb);
+		strbuf_release(&err);
+		return -1;
+	}
 
 	strbuf_release(&sb);
-	return ret;
+	strbuf_release(&err);
+	ref_transaction_free(transaction);
+	return 0;
 }
 
 static int do_recursive_merge(struct commit *base, struct commit *next,
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 09/22] fast-import.c: change update_branch to use ref transactions

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 16:21:13 -0700

Change update_branch() to use ref transactions for updates.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 fast-import.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index 6707a66..79160d5 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1679,8 +1679,9 @@ found_entry:
 static int update_branch(struct branch *b)
 {
 	static const char *msg = "fast-import";
-	struct ref_lock *lock;
+	struct ref_transaction *transaction;
 	unsigned char old_sha1[20];
+	struct strbuf err = STRBUF_INIT;
 
 	if (read_ref(b->name, old_sha1))
 		hashclr(old_sha1);
@@ -1689,29 +1690,33 @@ static int update_branch(struct branch *b)
 			delete_ref(b->name, old_sha1, 0);
 		return 0;
 	}
-	lock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL);
-	if (!lock)
-		return error("Unable to lock %s", b->name);
 	if (!force_update && !is_null_sha1(old_sha1)) {
 		struct commit *old_cmit, *new_cmit;
 
 		old_cmit = lookup_commit_reference_gently(old_sha1, 0);
 		new_cmit = lookup_commit_reference_gently(b->sha1, 0);
-		if (!old_cmit || !new_cmit) {
-			unlock_ref(lock);
+		if (!old_cmit || !new_cmit)
 			return error("Branch %s is missing commits.", b->name);
-		}
 
 		if (!in_merge_bases(old_cmit, new_cmit)) {
-			unlock_ref(lock);
 			warning("Not updating %s"
 				" (new tip %s does not contain %s)",
 				b->name, sha1_to_hex(b->sha1), sha1_to_hex(old_sha1));
 			return -1;
 		}
 	}
-	if (write_ref_sha1(lock, b->sha1, msg) < 0)
-		return error("Unable to update %s", b->name);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, b->name, b->sha1, old_sha1,
+				   0, 1, &err) ||
+	    ref_transaction_commit(transaction, msg, &err)) {
+		ref_transaction_free(transaction);
+		error("%s", err.buf);
+		strbuf_release(&err);
+		return -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	return 0;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 10/22] branch.c: use ref transaction for all ref updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Apr 2014 16:21:53 -0700

Change create_branch to use a ref transaction when creating the new branch.

This also fixes a race condition in the old code where two concurrent
create_branch could race since the lock_any_ref_for_update/write_ref_sha1
did not protect against the ref already existing. I.e. one thread could end up
overwriting a branch even if the forcing flag is false.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 branch.c | 31 +++++++++++++++++--------------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/branch.c b/branch.c
index 660097b..37ac555 100644
--- a/branch.c
+++ b/branch.c
@@ -226,7 +226,6 @@ void create_branch(const char *head,
 		   int force, int reflog, int clobber_head,
 		   int quiet, enum branch_track track)
 {
-	struct ref_lock *lock = NULL;
 	struct commit *commit;
 	unsigned char sha1[20];
 	char *real_ref, msg[PATH_MAX + 20];
@@ -285,15 +284,6 @@ void create_branch(const char *head,
 		die(_("Not a valid branch point: '%s'."), start_name);
 	hashcpy(sha1, commit->object.sha1);
 
-	if (!dont_change_ref) {
-		lock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);
-		if (!lock)
-			die_errno(_("Failed to lock ref for update"));
-	}
-
-	if (reflog)
-		log_all_ref_updates = 1;
-
 	if (forcing)
 		snprintf(msg, sizeof msg, "branch: Reset to %s",
 			 start_name);
@@ -301,13 +291,26 @@ void create_branch(const char *head,
 		snprintf(msg, sizeof msg, "branch: Created from %s",
 			 start_name);
 
+	if (reflog)
+		log_all_ref_updates = 1;
+
+	if (!dont_change_ref) {
+		struct ref_transaction *transaction;
+		struct strbuf err = STRBUF_INIT;
+
+		transaction = ref_transaction_begin(&err);
+		if (!transaction ||
+		    ref_transaction_update(transaction, ref.buf, sha1,
+					   null_sha1, 0, !forcing, &err) ||
+		    ref_transaction_commit(transaction, msg, &err))
+			die("%s", err.buf);
+		ref_transaction_free(transaction);
+		strbuf_release(&err);
+	}
+
 	if (real_ref && track)
 		setup_tracking(ref.buf + 11, real_ref, track, quiet);
 
-	if (!dont_change_ref)
-		if (write_ref_sha1(lock, sha1, msg) < 0)
-			die_errno(_("Failed to write ref"));
-
 	strbuf_release(&ref);
 	free(real_ref);
 }
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 11/22] refs.c: change update_ref to use a transaction

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 24 Apr 2014 16:36:55 -0700

Change the update_ref helper function to use a ref transaction internally.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/refs.c b/refs.c
index cc63056..a3532ab 100644
--- a/refs.c
+++ b/refs.c
@@ -3519,11 +3519,33 @@ int update_ref(const char *action, const char *refname,
 	       const unsigned char *sha1, const unsigned char *oldval,
 	       int flags, enum action_on_err onerr)
 {
-	struct ref_lock *lock;
-	lock = update_ref_lock(refname, oldval, flags, NULL, onerr);
-	if (!lock)
+	struct ref_transaction *t;
+	struct strbuf err = STRBUF_INIT;
+
+	t = ref_transaction_begin(&err);
+	if (!t ||
+	    ref_transaction_update(t, refname, sha1, oldval, flags,
+				   !!oldval, &err) ||
+	    ref_transaction_commit(t, action, &err)) {
+		const char *str = "update_ref failed for ref '%s': %s";
+
+		ref_transaction_free(t);
+		switch (onerr) {
+		case UPDATE_REFS_MSG_ON_ERR:
+			error(str, refname, err.buf);
+			break;
+		case UPDATE_REFS_DIE_ON_ERR:
+			die(str, refname, err.buf);
+			break;
+		case UPDATE_REFS_QUIET_ON_ERR:
+			break;
+		}
+		strbuf_release(&err);
 		return 1;
-	return update_ref_write(action, refname, sha1, lock, NULL, onerr);
+	}
+	strbuf_release(&err);
+	ref_transaction_free(t);
+	return 0;
 }
 
 static int ref_update_compare(const void *r1, const void *r2)
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 12/22] receive-pack.c: use a reference transaction for updating the refs

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 14:36:15 -0700

Wrap all the ref updates inside a transaction.

In the new API there is no distinction between failure to lock and
failure to write a ref.  Both can be permanent (e.g., a ref
"refs/heads/topic" is blocking creation of the lock file
"refs/heads/topic/1.lock") or transient (e.g., file system full) and
there's no clear difference in how the client should respond, so
replace the two statuses "failed to lock" and "failed to write" with
a single status "failed to update ref".  In both cases a more
detailed message is sent by sideband to diagnose the problem.

Example, before:

 error: there are still refs under 'refs/heads/topic'
 remote: error: failed to lock refs/heads/topic
 To foo
  ! [remote rejected] HEAD -> topic (failed to lock)

After:

 error: there are still refs under 'refs/heads/topic'
 remote: error: Cannot lock the ref 'refs/heads/topic'.
 To foo
  ! [remote rejected] HEAD -> topic (failed to update ref)

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/receive-pack.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index c323081..224fadc 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -475,7 +475,6 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 	const char *namespaced_name;
 	unsigned char *old_sha1 = cmd->old_sha1;
 	unsigned char *new_sha1 = cmd->new_sha1;
-	struct ref_lock *lock;
 
 	/* only refs/... are allowed */
 	if (!starts_with(name, "refs/") || check_refname_format(name + 5, 0)) {
@@ -576,19 +575,27 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		return NULL; /* good */
 	}
 	else {
+		struct strbuf err = STRBUF_INIT;
+		struct ref_transaction *transaction;
+
 		if (shallow_update && si->shallow_ref[cmd->index] &&
 		    update_shallow_ref(cmd, si))
 			return "shallow error";
 
-		lock = lock_any_ref_for_update(namespaced_name, old_sha1,
-					       0, NULL);
-		if (!lock) {
-			rp_error("failed to lock %s", name);
-			return "failed to lock";
-		}
-		if (write_ref_sha1(lock, new_sha1, "push")) {
-			return "failed to write"; /* error() already called */
+		transaction = ref_transaction_begin(&err);
+		if (!transaction ||
+		    ref_transaction_update(transaction, namespaced_name,
+					   new_sha1, old_sha1, 0, 1, &err) ||
+		    ref_transaction_commit(transaction, "push", &err)) {
+			ref_transaction_free(transaction);
+
+			rp_error("%s", err.buf);
+			strbuf_release(&err);
+			return "failed to update ref";
 		}
+
+		ref_transaction_free(transaction);
+		strbuf_release(&err);
 		return NULL; /* good */
 	}
 }
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 13/22] fast-import.c: use a ref transaction when dumping tags

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 15:23:58 -0700

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 fast-import.c | 29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index 79160d5..e7f6e37 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1735,15 +1735,32 @@ static void dump_tags(void)
 {
 	static const char *msg = "fast-import";
 	struct tag *t;
-	struct ref_lock *lock;
-	char ref_name[PATH_MAX];
+	struct strbuf ref_name = STRBUF_INIT;
+	struct strbuf err = STRBUF_INIT;
+	struct ref_transaction *transaction;
 
+	transaction = ref_transaction_begin(&err);
+	if (!transaction) {
+		failure |= error("%s", err.buf);
+		goto cleanup;
+	}
 	for (t = first_tag; t; t = t->next_tag) {
-		sprintf(ref_name, "tags/%s", t->name);
-		lock = lock_ref_sha1(ref_name, NULL);
-		if (!lock || write_ref_sha1(lock, t->sha1, msg) < 0)
-			failure |= error("Unable to update %s", ref_name);
+		strbuf_reset(&ref_name);
+		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
+
+		if (ref_transaction_update(transaction, ref_name.buf, t->sha1,
+					   NULL, 0, 0, &err)) {
+			failure |= error("%s", err.buf);
+			goto cleanup;
+		}
 	}
+	if (ref_transaction_commit(transaction, msg, &err))
+		failure |= error("%s", err.buf);
+
+ cleanup:
+	ref_transaction_free(transaction);
+	strbuf_release(&ref_name);
+	strbuf_release(&err);
 }
 
 static void dump_marks_helper(FILE *f,
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 14/22] walker.c: use ref transaction for ref updates

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 17 Apr 2014 11:31:06 -0700

Switch to using ref transactions in walker_fetch(). As part of the refactoring
to use ref transactions we also fix a potential memory leak where in the
original code if write_ref_sha1() would fail we would end up returning from
the function without free()ing the msg string.

Note that this function is only called when fetching from a remote HTTP
repository onto the local (most of the time single-user) repository which
likely means that the type of collisions that the previous locking would
protect against and cause the fetch to fail for are even more rare.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 walker.c | 72 ++++++++++++++++++++++++++++++++++++----------------------------
 1 file changed, 41 insertions(+), 31 deletions(-)

diff --git a/walker.c b/walker.c
index 1dd86b8..b8a5441 100644
--- a/walker.c
+++ b/walker.c
@@ -251,40 +251,40 @@ void walker_targets_free(int targets, char **target, const char **write_ref)
 int walker_fetch(struct walker *walker, int targets, char **target,
 		 const char **write_ref, const char *write_ref_log_details)
 {
-	struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));
+	struct strbuf refname = STRBUF_INIT;
+	struct strbuf err = STRBUF_INIT;
+	struct ref_transaction *transaction = NULL;
 	unsigned char *sha1 = xmalloc(targets * 20);
-	char *msg;
-	int ret;
-	int i;
+	char *msg = NULL;
+	int i, ret = -1;
 
 	save_commit_buffer = 0;
 
-	for (i = 0; i < targets; i++) {
-		if (!write_ref || !write_ref[i])
-			continue;
-
-		lock[i] = lock_ref_sha1(write_ref[i], NULL);
-		if (!lock[i]) {
-			error("Can't lock ref %s", write_ref[i]);
-			goto unlock_and_fail;
+	if (write_ref) {
+		transaction = ref_transaction_begin(&err);
+		if (!transaction) {
+			error("%s", err.buf);
+			goto done;
 		}
 	}
-
 	if (!walker->get_recover)
 		for_each_ref(mark_complete, NULL);
 
 	for (i = 0; i < targets; i++) {
 		if (interpret_target(walker, target[i], &sha1[20 * i])) {
 			error("Could not interpret response from server '%s' as something to pull", target[i]);
-			goto unlock_and_fail;
+			goto done;
 		}
 		if (process(walker, lookup_unknown_object(&sha1[20 * i])))
-			goto unlock_and_fail;
+			goto done;
 	}
 
 	if (loop(walker))
-		goto unlock_and_fail;
-
+		goto done;
+	if (!write_ref) {
+		ret = 0;
+		goto done;
+	}
 	if (write_ref_log_details) {
 		msg = xmalloc(strlen(write_ref_log_details) + 12);
 		sprintf(msg, "fetch from %s", write_ref_log_details);
@@ -292,23 +292,33 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		msg = NULL;
 	}
 	for (i = 0; i < targets; i++) {
-		if (!write_ref || !write_ref[i])
+		if (!write_ref[i])
 			continue;
-		ret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : "fetch (unknown)");
-		lock[i] = NULL;
-		if (ret)
-			goto unlock_and_fail;
+		strbuf_reset(&refname);
+		strbuf_addf(&refname, "refs/%s", write_ref[i]);
+		if (ref_transaction_update(transaction, refname.buf,
+					   &sha1[20 * i], NULL, 0, 0,
+					   &err)) {
+			error("%s", err.buf);
+			goto done;
+		}
 	}
+	if (ref_transaction_commit(transaction,
+				   msg ? msg : "fetch (unknown)",
+				   &err)) {
+		error("%s", err.buf);
+		goto done;
+	}
+
+	ret = 0;
+
+done:
+	ref_transaction_free(transaction);
 	free(msg);
-
-	return 0;
-
-unlock_and_fail:
-	for (i = 0; i < targets; i++)
-		if (lock[i])
-			unlock_ref(lock[i]);
-
-	return -1;
+	free(sha1);
+	strbuf_release(&err);
+	strbuf_release(&refname);
+	return ret;
 }
 
 void walker_free(struct walker *walker)
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 15/22] refs.c: make lock_ref_sha1 static

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 15:38:47 -0700

No external callers reference lock_ref_sha1 any more so let's declare it
static.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 7 +++++--
 refs.h | 6 ------
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/refs.c b/refs.c
index a3532ab..d4cd44b 100644
--- a/refs.c
+++ b/refs.c
@@ -2069,7 +2069,10 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 	return logs_found;
 }
 
-/* This function should make sure errno is meaningful on error */
+/*
+ * Locks a "refs/" ref returning the lock on success and NULL on failure.
+ * On failure errno is set to something meaningful.
+ */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
 					    int flags, int *type_p)
@@ -2170,7 +2173,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	return NULL;
 }
 
-struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)
+static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)
 {
 	char refpath[PATH_MAX];
 	if (check_refname_format(refname, 0))
diff --git a/refs.h b/refs.h
index 3f37c65..65dd593 100644
--- a/refs.h
+++ b/refs.h
@@ -170,12 +170,6 @@ extern int ref_exists(const char *);
  */
 extern int peel_ref(const char *refname, unsigned char *sha1);
 
-/*
- * Locks a "refs/" ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
- */
-extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1);
-
 /** Locks any ref (for 'HEAD' type refs). */
 #define REF_NODEREF	0x01
 /* errno is set to something meaningful on failure */
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 16/22] refs.c: remove the update_ref_lock function

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 12:14:47 -0700

Since we now only call update_ref_lock with onerr==QUIET_ON_ERR we no longer
need this function and can replace it with just calling lock_any_ref_for_update
directly.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/refs.c b/refs.c
index d4cd44b..a4445a1 100644
--- a/refs.c
+++ b/refs.c
@@ -3336,24 +3336,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-static struct ref_lock *update_ref_lock(const char *refname,
-					const unsigned char *oldval,
-					int flags, int *type_p,
-					enum action_on_err onerr)
-{
-	struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
-	if (!lock) {
-		const char *str = "Cannot lock the ref '%s'.";
-		switch (onerr) {
-		case UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;
-		case UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;
-		case UPDATE_REFS_QUIET_ON_ERR: break;
-		}
-	}
-	return lock;
-}
-
 static int update_ref_write(const char *action, const char *refname,
 			    const unsigned char *sha1, struct ref_lock *lock,
 			    struct strbuf *err, enum action_on_err onerr)
@@ -3603,12 +3585,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	for (i = 0; i < n; i++) {
 		struct ref_update *update = updates[i];
 
-		update->lock = update_ref_lock(update->refname,
-					       (update->have_old ?
-						update->old_sha1 : NULL),
-					       update->flags,
-					       &update->type,
-					       UPDATE_REFS_QUIET_ON_ERR);
+		update->lock = lock_any_ref_for_update(update->refname,
+						       (update->have_old ?
+							update->old_sha1 :
+							NULL),
+						       update->flags,
+						       &update->type);
 		if (!update->lock) {
 			if (err)
 				strbuf_addf(err, "Cannot lock the ref '%s'.",
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 17/22] refs.c: remove the update_ref_write function

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 13:42:07 -0700

Since we only call update_ref_write from a single place and we only call it
with onerr==QUIET_ON_ERR we can just as well get rid of it and just call
write_ref_sha1 directly. This changes the return status for _commit from
1 to -1 on failures when writing to the ref. Eventually we will want
_commit to start returning more detailed error conditions than the current
simple success/failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 34 ++++++++--------------------------
 1 file changed, 8 insertions(+), 26 deletions(-)

diff --git a/refs.c b/refs.c
index a4445a1..a6b39ec 100644
--- a/refs.c
+++ b/refs.c
@@ -3336,25 +3336,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-static int update_ref_write(const char *action, const char *refname,
-			    const unsigned char *sha1, struct ref_lock *lock,
-			    struct strbuf *err, enum action_on_err onerr)
-{
-	if (write_ref_sha1(lock, sha1, action) < 0) {
-		const char *str = "Cannot update the ref '%s'.";
-		if (err)
-			strbuf_addf(err, str, refname);
-
-		switch (onerr) {
-		case UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;
-		case UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;
-		case UPDATE_REFS_QUIET_ON_ERR: break;
-		}
-		return 1;
-	}
-	return 0;
-}
-
 /**
  * Information needed for a single ref update.  Set new_sha1 to the
  * new value or to zero to delete the ref.  To check the old value
@@ -3605,14 +3586,15 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (!is_null_sha1(update->new_sha1)) {
-			ret = update_ref_write(msg,
-					       update->refname,
-					       update->new_sha1,
-					       update->lock, err,
-					       UPDATE_REFS_QUIET_ON_ERR);
-			update->lock = NULL; /* freed by update_ref_write */
-			if (ret)
+			ret = write_ref_sha1(update->lock, update->new_sha1,
+					     msg);
+			update->lock = NULL; /* freed by write_ref_sha1 */
+			if (ret) {
+				if (err)
+					strbuf_addf(err, "Cannot update the ref '%s'.",
+						    update->refname);
 				goto cleanup;
+			}
 		}
 	}
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 18/22] refs.c: remove lock_ref_sha1

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 29 Apr 2014 15:45:52 -0700

lock_ref_sha1 was only called from one place in refs.c and only provided
a check that the refname was sane before adding back the initial "refs/"
part of the ref path name, the initial "refs/" that this caller had already
stripped off before calling lock_ref_sha1.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index a6b39ec..fd67684 100644
--- a/refs.c
+++ b/refs.c
@@ -2173,15 +2173,6 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	return NULL;
 }
 
-static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha1)
-{
-	char refpath[PATH_MAX];
-	if (check_refname_format(refname, 0))
-		return NULL;
-	strcpy(refpath, mkpath("refs/%s", refname));
-	return lock_ref_sha1_basic(refpath, old_sha1, 0, NULL);
-}
-
 struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
@@ -2391,8 +2382,12 @@ static void try_remove_empty_parents(char *name)
 /* make sure nobody touched the ref, and unlink */
 static void prune_ref(struct ref_to_prune *r)
 {
-	struct ref_lock *lock = lock_ref_sha1(r->name + 5, r->sha1);
+	struct ref_lock *lock;
 
+	if (check_refname_format(r->name + 5, 0))
+		return;
+
+	lock = lock_ref_sha1_basic(r->name, r->sha1, 0, NULL);
 	if (lock) {
 		unlink_or_warn(git_path("%s", r->name));
 		unlock_ref(lock);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 19/22] refs.c: make prune_ref use a transaction to delete the ref

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 09:03:36 -0700

Change prune_ref to delete the ref using a ref transaction. To do this we also
need to add a new flag REF_ISPRUNING that will tell the transaction that we
do not want to delete this ref from the packed refs. This flag is private to
refs.c and not exposed to external callers.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 28 +++++++++++++++++++++-------
 refs.h | 13 +++++++++++--
 2 files changed, 32 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index fd67684..22eb3dd 100644
--- a/refs.c
+++ b/refs.c
@@ -25,6 +25,11 @@ static unsigned char refname_disposition[256] = {
 };
 
 /*
+ * Used as a flag to ref_transaction_delete when a loose ref is being
+ * pruned.
+ */
+#define REF_ISPRUNING	0x0100
+/*
  * Try to read one refname component from the front of refname.
  * Return the length of the component found, or -1 if the component is
  * not legal.  It is legal if it is something reasonable to have under
@@ -2382,17 +2387,25 @@ static void try_remove_empty_parents(char *name)
 /* make sure nobody touched the ref, and unlink */
 static void prune_ref(struct ref_to_prune *r)
 {
-	struct ref_lock *lock;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
 	if (check_refname_format(r->name + 5, 0))
 		return;
 
-	lock = lock_ref_sha1_basic(r->name, r->sha1, 0, NULL);
-	if (lock) {
-		unlink_or_warn(git_path("%s", r->name));
-		unlock_ref(lock);
-		try_remove_empty_parents(r->name);
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_delete(transaction, r->name, r->sha1,
+				   REF_ISPRUNING, 1, &err) ||
+	    ref_transaction_commit(transaction, NULL, &err)) {
+		ref_transaction_free(transaction);
+		error("%s", err.buf);
+		strbuf_release(&err);
+		return;
 	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	try_remove_empty_parents(r->name);
 }
 
 static void prune_refs(struct ref_to_prune *r)
@@ -3598,8 +3611,9 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			delnames[delnum++] = update->lock->ref_name;
 			ret |= delete_ref_loose(update->lock, update->type);
+			if (!(update->flags & REF_ISPRUNING))
+				delnames[delnum++] = update->lock->ref_name;
 		}
 	}
 
diff --git a/refs.h b/refs.h
index 65dd593..69ef28c 100644
--- a/refs.h
+++ b/refs.h
@@ -170,9 +170,18 @@ extern int ref_exists(const char *);
  */
 extern int peel_ref(const char *refname, unsigned char *sha1);
 
-/** Locks any ref (for 'HEAD' type refs). */
+/*
+ * Flags controlling lock_any_ref_for_update(), ref_transaction_update(),
+ * ref_transaction_create(), etc.
+ * REF_NODEREF: act on the ref directly, instead of dereferencing
+ *              symbolic references.
+ *
+ * Flags >= 0x100 are reserved for internal use.
+ */
 #define REF_NODEREF	0x01
-/* errno is set to something meaningful on failure */
+/*
+ * This function sets errno to something meaningful on failure.
+ */
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
 						int flags, int *type_p);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 20/22] refs.c: make delete_ref use a transaction

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 09:22:45 -0700

Change delete_ref to use a ref transaction for the deletion. At the same time
since we no longer have any callers of repack_without_ref we can now delete
this function.

Change delete_ref to return 0 on success and 1 on failure instead of the
previous 0 on success either 1 or -1 on failure.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 35 ++++++++++++++---------------------
 1 file changed, 14 insertions(+), 21 deletions(-)

diff --git a/refs.c b/refs.c
index 22eb3dd..7235574 100644
--- a/refs.c
+++ b/refs.c
@@ -2548,11 +2548,6 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 	return ret;
 }
 
-static int repack_without_ref(const char *refname)
-{
-	return repack_without_refs(&refname, 1, NULL);
-}
-
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
@@ -2570,24 +2565,22 @@ static int delete_ref_loose(struct ref_lock *lock, int flag)
 
 int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 {
-	struct ref_lock *lock;
-	int ret = 0, flag = 0;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
 
-	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
-	if (!lock)
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_delete(transaction, refname, sha1, delopt,
+				   sha1 && !is_null_sha1(sha1), &err) ||
+	    ref_transaction_commit(transaction, NULL, &err)) {
+		error("%s", err.buf);
+		ref_transaction_free(transaction);
+		strbuf_release(&err);
 		return 1;
-	ret |= delete_ref_loose(lock, flag);
-
-	/* removing the loose one could have resurrected an earlier
-	 * packed one.  Also, if it was not loose we need to repack
-	 * without it.
-	 */
-	ret |= repack_without_ref(lock->ref_name);
-
-	unlink_or_warn(git_path("logs/%s", lock->ref_name));
-	clear_loose_ref_cache(&ref_cache);
-	unlock_ref(lock);
-	return ret;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	return 0;
 }
 
 /*
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 21/22] update-ref --stdin: narrow scope of err strbuf

[Jonathan Nieder]

Making the strbuf local in each function that needs to print errors
saves the reader from having to think about action at a distance,
such as

 * errors piling up and being concatenated with no newline between
   them
 * errors unhandled in one function, to be later handled in another
 * concurrency issues, if this code starts using threads some day

No functional change intended.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/update-ref.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 96a53b9..866bbee 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -16,7 +16,6 @@ static struct ref_transaction *transaction;
 
 static char line_termination = '\n';
 static int update_flags;
-static struct strbuf err = STRBUF_INIT;
 
 /*
  * Parse one whitespace- or NUL-terminated, possibly C-quoted argument
@@ -179,6 +178,7 @@ static int parse_next_sha1(struct strbuf *input, const char **next,
 
 static const char *parse_cmd_update(struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char new_sha1[20];
 	unsigned char old_sha1[20];
@@ -204,12 +204,14 @@ static const char *parse_cmd_update(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
 
 static const char *parse_cmd_create(struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char new_sha1[20];
 
@@ -232,12 +234,14 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
 
 static const char *parse_cmd_delete(struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char old_sha1[20];
 	int have_old;
@@ -264,12 +268,14 @@ static const char *parse_cmd_delete(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
 
 static const char *parse_cmd_verify(struct strbuf *input, const char *next)
 {
+	struct strbuf err = STRBUF_INIT;
 	char *refname;
 	unsigned char new_sha1[20];
 	unsigned char old_sha1[20];
@@ -297,6 +303,7 @@ static const char *parse_cmd_verify(struct strbuf *input, const char *next)
 
 	update_flags = 0;
 	free(refname);
+	strbuf_release(&err);
 
 	return next;
 }
@@ -365,6 +372,8 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		die("Refusing to perform update with empty message.");
 
 	if (read_stdin) {
+		struct strbuf err = STRBUF_INIT;
+
 		transaction = ref_transaction_begin(&err);
 		if (!transaction)
 			die("%s", err.buf);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 22/22] update-ref --stdin: pass transaction around explicitly

[Jonathan Nieder]

This makes it more obvious at a glance where the output of functions
parsing the --stdin stream goes.

No functional change intended.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
---
Thanks for reading.

 builtin/update-ref.c | 27 +++++++++++++++------------
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 866bbee..54a48c0 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -12,8 +12,6 @@ static const char * const git_update_ref_usage[] = {
 	NULL
 };
 
-static struct ref_transaction *transaction;
-
 static char line_termination = '\n';
 static int update_flags;
 
@@ -176,7 +174,8 @@ static int parse_next_sha1(struct strbuf *input, const char **next,
  * depending on how line_termination is set.
  */
 
-static const char *parse_cmd_update(struct strbuf *input, const char *next)
+static const char *parse_cmd_update(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
 	struct strbuf err = STRBUF_INIT;
 	char *refname;
@@ -209,7 +208,8 @@ static const char *parse_cmd_update(struct strbuf *input, const char *next)
 	return next;
 }
 
-static const char *parse_cmd_create(struct strbuf *input, const char *next)
+static const char *parse_cmd_create(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
 	struct strbuf err = STRBUF_INIT;
 	char *refname;
@@ -239,7 +239,8 @@ static const char *parse_cmd_create(struct strbuf *input, const char *next)
 	return next;
 }
 
-static const char *parse_cmd_delete(struct strbuf *input, const char *next)
+static const char *parse_cmd_delete(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
 	struct strbuf err = STRBUF_INIT;
 	char *refname;
@@ -273,7 +274,8 @@ static const char *parse_cmd_delete(struct strbuf *input, const char *next)
 	return next;
 }
 
-static const char *parse_cmd_verify(struct strbuf *input, const char *next)
+static const char *parse_cmd_verify(struct ref_transaction *transaction,
+				    struct strbuf *input, const char *next)
 {
 	struct strbuf err = STRBUF_INIT;
 	char *refname;
@@ -317,7 +319,7 @@ static const char *parse_cmd_option(struct strbuf *input, const char *next)
 	return next + 8;
 }
 
-static void update_refs_stdin(void)
+static void update_refs_stdin(struct ref_transaction *transaction)
 {
 	struct strbuf input = STRBUF_INIT;
 	const char *next;
@@ -332,13 +334,13 @@ static void update_refs_stdin(void)
 		else if (isspace(*next))
 			die("whitespace before command: %s", next);
 		else if (starts_with(next, "update "))
-			next = parse_cmd_update(&input, next + 7);
+			next = parse_cmd_update(transaction, &input, next + 7);
 		else if (starts_with(next, "create "))
-			next = parse_cmd_create(&input, next + 7);
+			next = parse_cmd_create(transaction, &input, next + 7);
 		else if (starts_with(next, "delete "))
-			next = parse_cmd_delete(&input, next + 7);
+			next = parse_cmd_delete(transaction, &input, next + 7);
 		else if (starts_with(next, "verify "))
-			next = parse_cmd_verify(&input, next + 7);
+			next = parse_cmd_verify(transaction, &input, next + 7);
 		else if (starts_with(next, "option "))
 			next = parse_cmd_option(&input, next + 7);
 		else
@@ -373,6 +375,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 
 	if (read_stdin) {
 		struct strbuf err = STRBUF_INIT;
+		struct ref_transaction *transaction;
 
 		transaction = ref_transaction_begin(&err);
 		if (!transaction)
@@ -381,7 +384,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 			usage_with_options(git_update_ref_usage, options);
 		if (end_null)
 			line_termination = '\0';
-		update_refs_stdin();
+		update_refs_stdin(transaction);
 		if (ref_transaction_commit(transaction, msg, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Jonathan Nieder]

Jonathan Nieder wrote:

> The next series from Ronnie's collection is available at
> https://code-review.googlesource.com/#/q/topic:ref-transaction in case
> someone wants a fresh series to look at.

Here is the outcome of that review.  It could use another set of eyes
(hint, hint) but should be mostly ready.  Interdiff below.  This is meant
to replace rs/ref-transaction in 'pu' and I'd prefer to wait a little
for more comments before merging to 'next'.

These patches are also available from the git repository at

  git://repo.or.cz/git/jrn.git tags/rs/ref-transaction

"Use ref transactions", part 3

Ronnie explains:

	This is the third and final part of the original 48 patch
	series for basic transaction support.  This version implements
	some changes suggested by mhagger for the warn_if_unremovable
	changes.  It also adds a new patch "fix handling of badly
	named refs" that repairs the handling of badly named refs.

This includes some improvements to the transaction API (in
particular allowing different reflog messages per ref update in
a transaction), some cleanups and consistency improvements, and
preparation for an implementation of ref renaming in terms of
the transaction API.

It also improves handling of refs with invalid names.  Today "git
branch --list" and "git for-each-ref" do not provide a way to discover
refs with invalid names and "git branch -d" and "git update-ref -d"
cannot delete them.  After this series, such bad refs will be
discoverable and deletable again as in olden times.

Jonathan Nieder (5):
  mv test: recreate mod/ directory instead of relying on stale copy
  branch -d: avoid repeated symref resolution
  refs.c: do not permit err == NULL
  lockfile: remove unable_to_lock_error
  ref_transaction_commit: bail out on failure to remove a ref

Ronnie Sahlberg (14):
  wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as success
  wrapper.c: add a new function unlink_or_msg
  refs.c: add an err argument to delete_ref_loose
  refs.c: pass the ref log message to _create/delete/update instead of
    _commit
  rename_ref: don't ask read_ref_full where the ref came from
  refs.c: move the check for valid refname to lock_ref_sha1_basic
  refs.c: call lock_ref_sha1_basic directly from commit
  refs.c: pass a skip list to name_conflict_fn
  refs.c: ref_transaction_commit: distinguish name conflicts from other
    errors
  fetch.c: change s_update_ref to use a ref transaction
  refs.c: make write_ref_sha1 static
  refs.c: change resolve_ref_unsafe reading argument to be a flags field
  refs.c: fix handling of badly named refs
  for-each-ref.c: improve message before aborting on broken ref

 branch.c                    |   6 +-
 builtin/blame.c             |   2 +-
 builtin/branch.c            |  19 ++-
 builtin/checkout.c          |   6 +-
 builtin/clone.c             |   2 +-
 builtin/commit.c            |   6 +-
 builtin/fetch.c             |  34 +++--
 builtin/fmt-merge-msg.c     |   2 +-
 builtin/for-each-ref.c      |  12 +-
 builtin/fsck.c              |   2 +-
 builtin/log.c               |   3 +-
 builtin/merge.c             |   2 +-
 builtin/notes.c             |   2 +-
 builtin/receive-pack.c      |   9 +-
 builtin/remote.c            |   5 +-
 builtin/replace.c           |   5 +-
 builtin/show-branch.c       |   6 +-
 builtin/symbolic-ref.c      |   2 +-
 builtin/tag.c               |   4 +-
 builtin/update-ref.c        |  16 ++-
 bundle.c                    |   2 +-
 cache.h                     |  31 +++--
 fast-import.c               |   8 +-
 git-compat-util.h           |  16 ++-
 http-backend.c              |   3 +-
 lockfile.c                  |  10 --
 notes-merge.c               |   2 +-
 reflog-walk.c               |   5 +-
 refs.c                      | 321 ++++++++++++++++++++++++++++++--------------
 refs.h                      |  18 ++-
 remote.c                    |  10 +-
 sequencer.c                 |   8 +-
 t/t1400-update-ref.sh       |  16 +++
 t/t1402-check-ref-format.sh |  48 +++++++
 t/t3200-branch.sh           |   9 ++
 t/t7001-mv.sh               |  15 ++-
 transport-helper.c          |   4 +-
 transport.c                 |   5 +-
 upload-pack.c               |   2 +-
 walker.c                    |   5 +-
 wrapper.c                   |  28 ++--
 wt-status.c                 |   2 +-
 42 files changed, 487 insertions(+), 226 deletions(-)
---
Changes since last appearance:

diff --git c/branch.c w/branch.c
index 76a8ec9..ba3e1c1 100644
--- c/branch.c
+++ w/branch.c
@@ -186,7 +186,7 @@ int validate_new_branchname(const char *name, struct strbuf *ref,
 		const char *head;
 		unsigned char sha1[20];
 
-		head = resolve_ref_unsafe("HEAD", sha1, 0, NULL);
+		head = resolve_ref_unsafe("HEAD", sha1, NULL, 0);
 		if (!is_bare_repository() && head && !strcmp(head, ref->buf))
 			die(_("Cannot force update the current branch."));
 	}
diff --git c/builtin/blame.c w/builtin/blame.c
index 1568bd8..b8bec0a 100644
--- c/builtin/blame.c
+++ w/builtin/blame.c
@@ -2292,7 +2292,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 	commit->object.type = OBJ_COMMIT;
 	parent_tail = &commit->parents;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, RESOLVE_REF_READING, NULL))
+	if (!resolve_ref_unsafe("HEAD", head_sha1, NULL, RESOLVE_REF_READING))
 		die("no such ref: HEAD");
 
 	parent_tail = append_parent(parent_tail, head_sha1);
diff --git c/builtin/branch.c w/builtin/branch.c
index 5c95656..5d5bc56 100644
--- c/builtin/branch.c
+++ w/builtin/branch.c
@@ -130,7 +130,7 @@ static int branch_merged(int kind, const char *name,
 		    branch->merge[0]->dst &&
 		    (reference_name = reference_name_to_free =
 		     resolve_refdup(branch->merge[0]->dst, sha1,
-				    RESOLVE_REF_READING, NULL)) != NULL)
+				    NULL, RESOLVE_REF_READING)) != NULL)
 			reference_rev = lookup_commit_reference(sha1);
 	}
 	if (!reference_rev)
@@ -234,10 +234,13 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, sha1,
-					    RESOLVE_REF_ALLOW_BAD_NAME, &flags);
+		target = resolve_ref_unsafe(name, sha1, &flags,
+					    RESOLVE_REF_READING
+					    | RESOLVE_REF_NODEREF
+					    | RESOLVE_REF_ALLOW_BAD_NAME);
 		if (!target ||
-		    (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {
+		    (!(flags & (REF_ISSYMREF|REF_ISBROKEN)) &&
+		     is_null_sha1(sha1))) {
 			error(remote_branch
 			      ? _("remote branch '%s' not found.")
 			      : _("branch '%s' not found."), bname.buf);
@@ -245,14 +248,14 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 			continue;
 		}
 
-		if (!(flags & REF_ISSYMREF) &&
+		if (!(flags & (REF_ISSYMREF|REF_ISBROKEN)) &&
 		    check_branch_commit(bname.buf, name, sha1, head_rev, kinds,
 					force)) {
 			ret = 1;
 			continue;
 		}
 
-		if (delete_ref(name, sha1, REF_NODEREF)) {
+		if (delete_ref(name, sha1, REF_NODEREF|REF_BADNAMEOK)) {
 			error(remote_branch
 			      ? _("Error deleting remote branch '%s'")
 			      : _("Error deleting branch '%s'"),
@@ -298,7 +301,7 @@ static char *resolve_symref(const char *src, const char *prefix)
 	int flag;
 	const char *dst, *cp;
 
-	dst = resolve_ref_unsafe(src, sha1, 0, &flag);
+	dst = resolve_ref_unsafe(src, sha1, &flag, 0);
 	if (!(dst && (flag & REF_ISSYMREF)))
 		return NULL;
 	if (prefix && (cp = skip_prefix(dst, prefix)))
@@ -864,7 +867,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	track = git_branch_track;
 
-	head = resolve_refdup("HEAD", head_sha1, 0, NULL);
+	head = resolve_refdup("HEAD", head_sha1, NULL, 0);
 	if (!head)
 		die(_("Failed to resolve HEAD as a valid ref."));
 	if (!strcmp(head, "HEAD")) {
diff --git c/builtin/checkout.c w/builtin/checkout.c
index f1dc56e..64af1f7 100644
--- c/builtin/checkout.c
+++ w/builtin/checkout.c
@@ -356,7 +356,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 	    commit_locked_index(lock_file))
 		die(_("unable to write new index file"));
 
-	read_ref_full("HEAD", rev, 0, &flag);
+	read_ref_full("HEAD", rev, &flag, 0);
 	head = lookup_commit_reference_gently(rev, 1);
 
 	errs |= post_checkout_hook(head, head, 0);
@@ -771,7 +771,7 @@ static int switch_branches(const struct checkout_opts *opts,
 	unsigned char rev[20];
 	int flag, writeout_error = 0;
 	memset(&old, 0, sizeof(old));
-	old.path = path_to_free = resolve_refdup("HEAD", rev, 0, &flag);
+	old.path = path_to_free = resolve_refdup("HEAD", rev, &flag, 0);
 	old.commit = lookup_commit_reference_gently(rev, 1);
 	if (!(flag & REF_ISSYMREF))
 		old.path = NULL;
@@ -1068,7 +1068,7 @@ static int checkout_branch(struct checkout_opts *opts,
 		unsigned char rev[20];
 		int flag;
 
-		if (!read_ref_full("HEAD", rev, 0, &flag) &&
+		if (!read_ref_full("HEAD", rev, &flag, 0) &&
 		    (flag & REF_ISSYMREF) && is_null_sha1(rev))
 			return switch_unborn_to_new_branch(opts);
 	}
diff --git c/builtin/clone.c w/builtin/clone.c
index f7307e6..12a78e1 100644
--- c/builtin/clone.c
+++ w/builtin/clone.c
@@ -622,7 +622,7 @@ static int checkout(void)
 	if (option_no_checkout)
 		return 0;
 
-	head = resolve_refdup("HEAD", sha1, RESOLVE_REF_READING, NULL);
+	head = resolve_refdup("HEAD", sha1, NULL, RESOLVE_REF_READING);
 	if (!head) {
 		warning(_("remote HEAD refers to nonexistent ref, "
 			  "unable to checkout.\n"));
diff --git c/builtin/commit.c w/builtin/commit.c
index d23e876..3536330 100644
--- c/builtin/commit.c
+++ w/builtin/commit.c
@@ -1468,7 +1468,7 @@ static void print_summary(const char *prefix, const unsigned char *sha1,
 	rev.diffopt.break_opt = 0;
 	diff_setup_done(&rev.diffopt);
 
-	head = resolve_ref_unsafe("HEAD", junk_sha1, 0, NULL);
+	head = resolve_ref_unsafe("HEAD", junk_sha1, NULL, 0);
 	printf("[%s%s ",
 		starts_with(head, "refs/heads/") ?
 			head + 11 :
diff --git c/builtin/fetch.c w/builtin/fetch.c
index 52f1ebc..2e3bc73 100644
--- c/builtin/fetch.c
+++ w/builtin/fetch.c
@@ -398,6 +398,7 @@ static int s_update_ref(const char *action,
 		goto fail;
 
 	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	return 0;
 fail:
 	ref_transaction_free(transaction);
@@ -686,9 +687,10 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 			}
 		}
 	}
+
 	if (rc & STORE_REF_ERROR_DF_CONFLICT)
 		error(_("some local refs could not be updated; try running\n"
-		      "'git remote prune %s' to remove any old, conflicting "
+		      " 'git remote prune %s' to remove any old, conflicting "
 		      "branches"), remote_name);
 
  abort:
diff --git c/builtin/fmt-merge-msg.c w/builtin/fmt-merge-msg.c
index d8ab177..b2355ad 100644
--- c/builtin/fmt-merge-msg.c
+++ w/builtin/fmt-merge-msg.c
@@ -602,7 +602,7 @@ int fmt_merge_msg(struct strbuf *in, struct strbuf *out,
 
 	/* get current branch */
 	current_branch = current_branch_to_free =
-		resolve_refdup("HEAD", head_sha1, RESOLVE_REF_READING, NULL);
+		resolve_refdup("HEAD", head_sha1, NULL, RESOLVE_REF_READING);
 	if (!current_branch)
 		die("No current branch");
 	if (starts_with(current_branch, "refs/heads/"))
diff --git c/builtin/for-each-ref.c w/builtin/for-each-ref.c
index 7c5b479..090390c 100644
--- c/builtin/for-each-ref.c
+++ w/builtin/for-each-ref.c
@@ -650,7 +650,7 @@ static void populate_value(struct refinfo *ref)
 	if (need_symref && (ref->flag & REF_ISSYMREF) && !ref->symref) {
 		unsigned char unused1[20];
 		ref->symref = resolve_refdup(ref->refname, unused1,
-					     RESOLVE_REF_READING, NULL);
+					     NULL, RESOLVE_REF_READING);
 		if (!ref->symref)
 			ref->symref = "";
 	}
@@ -709,7 +709,7 @@ static void populate_value(struct refinfo *ref)
 			unsigned char sha1[20];
 
 			head = resolve_ref_unsafe("HEAD", sha1,
-						  RESOLVE_REF_READING, NULL);
+						  NULL, RESOLVE_REF_READING);
 			if (!strcmp(ref->refname, head))
 				v->s = "*";
 			else
@@ -853,6 +853,12 @@ static int grab_single_ref(const char *refname, const unsigned char *sha1, int f
 	struct refinfo *ref;
 	int cnt;
 
+	if ((flag & REF_ISBROKEN) &&
+	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		  warning("ignoring ref with broken name %s", refname);
+		  return 0;
+	}
+
 	if (*cb->grab_pattern) {
 		const char **pattern;
 		int namelen = strlen(refname);
diff --git c/builtin/fsck.c w/builtin/fsck.c
index fc150c8..506e32b 100644
--- c/builtin/fsck.c
+++ w/builtin/fsck.c
@@ -560,7 +560,7 @@ static int fsck_head_link(void)
 	if (verbose)
 		fprintf(stderr, "Checking HEAD link\n");
 
-	head_points_at = resolve_ref_unsafe("HEAD", head_sha1, 0, &flag);
+	head_points_at = resolve_ref_unsafe("HEAD", head_sha1, &flag, 0);
 	if (!head_points_at)
 		return error("Invalid HEAD");
 	if (!strcmp(head_points_at, "HEAD"))
diff --git c/builtin/log.c w/builtin/log.c
index 92db809..230a9ef 100644
--- c/builtin/log.c
+++ w/builtin/log.c
@@ -1395,8 +1395,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (check_head) {
 			unsigned char sha1[20];
 			const char *ref;
-			ref = resolve_ref_unsafe("HEAD", sha1,
-						 RESOLVE_REF_READING, NULL);
+			ref = resolve_ref_unsafe("HEAD", sha1, NULL,
+						 RESOLVE_REF_READING);
 			if (ref && starts_with(ref, "refs/heads/"))
 				branch_name = xstrdup(ref + strlen("refs/heads/"));
 			else
diff --git c/builtin/merge.c w/builtin/merge.c
index 428ca24..d262279 100644
--- c/builtin/merge.c
+++ w/builtin/merge.c
@@ -1108,7 +1108,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
 	 * Check if we are _not_ on a detached HEAD, i.e. if there is a
 	 * current branch.
 	 */
-	branch = branch_to_free = resolve_refdup("HEAD", head_sha1, 0, &flag);
+	branch = branch_to_free = resolve_refdup("HEAD", head_sha1, &flag, 0);
 	if (branch && starts_with(branch, "refs/heads/"))
 		branch += 11;
 	if (!branch || is_null_sha1(head_sha1))
diff --git c/builtin/notes.c w/builtin/notes.c
index 820c341..16df78c 100644
--- c/builtin/notes.c
+++ w/builtin/notes.c
@@ -703,7 +703,7 @@ static int merge_commit(struct notes_merge_options *o)
 	init_notes(t, "NOTES_MERGE_PARTIAL", combine_notes_overwrite, 0);
 
 	o->local_ref = local_ref_to_free =
-		resolve_refdup("NOTES_MERGE_REF", sha1, 0, NULL);
+		resolve_refdup("NOTES_MERGE_REF", sha1, NULL, 0);
 	if (!o->local_ref)
 		die("Failed to resolve NOTES_MERGE_REF");
 
diff --git c/builtin/receive-pack.c w/builtin/receive-pack.c
index d1f4cf7..555a4a6 100644
--- c/builtin/receive-pack.c
+++ w/builtin/receive-pack.c
@@ -656,7 +656,7 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)
 	int flag;
 
 	strbuf_addf(&buf, "%s%s", get_git_namespace(), cmd->ref_name);
-	dst_name = resolve_ref_unsafe(buf.buf, sha1, 0, &flag);
+	dst_name = resolve_ref_unsafe(buf.buf, sha1, &flag, 0);
 	strbuf_release(&buf);
 
 	if (!(flag & REF_ISSYMREF))
@@ -817,7 +817,7 @@ static void execute_commands(struct command *commands,
 	check_aliased_updates(commands);
 
 	free(head_name_to_free);
-	head_name = head_name_to_free = resolve_refdup("HEAD", sha1, 0, NULL);
+	head_name = head_name_to_free = resolve_refdup("HEAD", sha1, NULL, 0);
 
 	checked_connectivity = 1;
 	for (cmd = commands; cmd; cmd = cmd->next) {
diff --git c/builtin/remote.c w/builtin/remote.c
index be8ebac..6eaeee7 100644
--- c/builtin/remote.c
+++ w/builtin/remote.c
@@ -568,8 +568,8 @@ static int read_remote_branches(const char *refname,
 	strbuf_addf(&buf, "refs/remotes/%s/", rename->old);
 	if (starts_with(refname, buf.buf)) {
 		item = string_list_append(rename->remote_branches, xstrdup(refname));
-		symref = resolve_ref_unsafe(refname, orig_sha1,
-					    RESOLVE_REF_READING, &flag);
+		symref = resolve_ref_unsafe(refname, orig_sha1, &flag,
+					    RESOLVE_REF_READING);
 		if (flag & REF_ISSYMREF)
 			item->util = xstrdup(symref);
 		else
@@ -705,7 +705,7 @@ static int mv(int argc, const char **argv)
 		int flag = 0;
 		unsigned char sha1[20];
 
-		read_ref_full(item->string, sha1, RESOLVE_REF_READING, &flag);
+		read_ref_full(item->string, sha1, &flag, RESOLVE_REF_READING);
 		if (!(flag & REF_ISSYMREF))
 			continue;
 		if (delete_ref(item->string, NULL, REF_NODEREF))
diff --git c/builtin/replace.c w/builtin/replace.c
index df060f8..9d03b84 100644
--- c/builtin/replace.c
+++ w/builtin/replace.c
@@ -170,7 +170,7 @@ static int replace_object_sha1(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref, repl, prev,
-				   0, !is_null_sha1(prev), NULL, &err) ||
+				   0, 1, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 
diff --git c/builtin/show-branch.c w/builtin/show-branch.c
index a9a5eb3..ef6ea52 100644
--- c/builtin/show-branch.c
+++ w/builtin/show-branch.c
@@ -727,8 +727,8 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		if (ac == 0) {
 			static const char *fake_av[2];
 
-			fake_av[0] = resolve_refdup("HEAD", sha1,
-						    RESOLVE_REF_READING, NULL);
+			fake_av[0] = resolve_refdup("HEAD", sha1, NULL,
+						    RESOLVE_REF_READING);
 			fake_av[1] = NULL;
 			av = fake_av;
 			ac = 1;
@@ -790,8 +790,8 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		}
 	}
 
-	head_p = resolve_ref_unsafe("HEAD", head_sha1,
-				    RESOLVE_REF_READING, NULL);
+	head_p = resolve_ref_unsafe("HEAD", head_sha1, NULL,
+				    RESOLVE_REF_READING);
 	if (head_p) {
 		head_len = strlen(head_p);
 		memcpy(head, head_p, head_len + 1);
diff --git c/builtin/symbolic-ref.c w/builtin/symbolic-ref.c
index b6a711d..1dd04af 100644
--- c/builtin/symbolic-ref.c
+++ w/builtin/symbolic-ref.c
@@ -13,7 +13,7 @@ static int check_symref(const char *HEAD, int quiet, int shorten, int print)
 {
 	unsigned char sha1[20];
 	int flag;
-	const char *refname = resolve_ref_unsafe(HEAD, sha1, 0, &flag);
+	const char *refname = resolve_ref_unsafe(HEAD, sha1, &flag, 0);
 
 	if (!refname)
 		die("No such ref: %s", HEAD);
diff --git c/builtin/update-ref.c w/builtin/update-ref.c
index 6c9be05..e379fdd 100644
--- c/builtin/update-ref.c
+++ w/builtin/update-ref.c
@@ -419,7 +419,8 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (no_deref)
 		flags = REF_NODEREF;
 	if (delete)
-		return delete_ref(refname, oldval ? oldsha1 : NULL, flags);
+		return delete_ref(refname, oldval ? oldsha1 : NULL,
+				  flags|REF_BADNAMEOK);
 	else
 		return update_ref(msg, refname, sha1, oldval ? oldsha1 : NULL,
 				  flags, UPDATE_REFS_DIE_ON_ERR);
diff --git c/bundle.c w/bundle.c
index 8aaf5f8..32dd2f7 100644
--- c/bundle.c
+++ w/bundle.c
@@ -311,7 +311,7 @@ int create_bundle(struct bundle_header *header, const char *path,
 			continue;
 		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
 			continue;
-		if (read_ref_full(e->name, sha1, RESOLVE_REF_READING, &flag))
+		if (read_ref_full(e->name, sha1, &flag, RESOLVE_REF_READING))
 			flag = 0;
 		display_ref = (flag & REF_ISSYMREF) ? e->name : ref;
 
diff --git c/cache.h w/cache.h
index 3a4b087..995729f 100644
--- c/cache.h
+++ w/cache.h
@@ -558,7 +558,6 @@ struct lock_file {
 };
 #define LOCK_DIE_ON_ERROR 1
 #define LOCK_NODEREF 2
-extern int unable_to_lock_error(const char *path, int err);
 extern void unable_to_lock_message(const char *path, int err,
 				   struct strbuf *buf);
 extern NORETURN void unable_to_lock_index_die(const char *path, int err);
@@ -948,7 +947,7 @@ extern int get_sha1_hex(const char *hex, unsigned char *sha1);
 
 extern char *sha1_to_hex(const unsigned char *sha1);	/* static buffer result! */
 extern int read_ref_full(const char *refname, unsigned char *sha1,
-			 int flags, int *ref_flag);
+			 int *flags, int resolve_flags);
 extern int read_ref(const char *refname, unsigned char *sha1);
 
 /*
@@ -970,21 +969,27 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  *   "writing" to the ref, the return value is the name of the ref
  *   that will actually be created or changed.
  *
- * If ref_flag is non-NULL, set the value that it points to the
+ * If flags is non-NULL, set the value that it points to the
  * combination of REF_ISPACKED (if the reference was found among the
- * packed references) and REF_ISSYMREF (if the initial reference was a
- * symbolic reference).
+ * packed references), REF_ISSYMREF (if the initial reference was a
+ * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
  *
  * If ref is not a properly-formatted, normalized reference, return
  * NULL.  If more than MAXDEPTH recursive symbolic lookups are needed,
  * give up and return NULL.
  *
- * errno is set to something meaningful on error.
+ * RESOLVE_REF_ALLOW_BAD_NAME disables most of the ref name checking except
+ * for names that are absolute paths or contain ".." components. For both
+ * these cases the function will return NULL and set errno to EINVAL.
+ * If the name is bad then the function will set the REF_ISBROKEN flag and
+ * return the name, if the ref exists, or NULL, if it does not.
+ * When this flag is set, any badly named refs will resolve to nullsha1.
  */
-#define RESOLVE_REF_READING        0x01
-#define RESOLVE_REF_ALLOW_BAD_NAME 0x02
-extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int flags, int *ref_flag);
-extern char *resolve_refdup(const char *ref, unsigned char *sha1, int flags, int *ref_flag);
+#define RESOLVE_REF_READING 0x01
+#define RESOLVE_REF_NODEREF 0x02
+#define RESOLVE_REF_ALLOW_BAD_NAME 0x04
+extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
+extern char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
 
 extern int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref);
 extern int dwim_log(const char *str, int len, unsigned char *sha1, char **ref);
diff --git c/git-compat-util.h w/git-compat-util.h
index 426bc98..5ee140c 100644
--- c/git-compat-util.h
+++ w/git-compat-util.h
@@ -307,6 +307,8 @@ extern char *gitbasename(char *);
 
 #include "wildmatch.h"
 
+struct strbuf;
+
 /* General helper functions */
 extern void vreportf(const char *prefix, const char *err, va_list params);
 extern void vwritef(int fd, const char *prefix, const char *err, va_list params);
@@ -704,19 +706,23 @@ void git_qsort(void *base, size_t nmemb, size_t size,
 #endif
 #endif
 
-#include "strbuf.h"
-
 /*
  * Preserves errno, prints a message, but gives no warning for ENOENT.
- * Always returns the return value of unlink(2).
+ * Returns 0 on success which includes trying to unlink an object that does
+ * not exist.
  */
 int unlink_or_warn(const char *path);
-/*
- * Like unlink_or_warn but populates a strbuf
- */
+ /*
+  * Tries to unlink file.  Returns 0 if unlink succeeded
+  * or the file already didn't exist.  Returns -1 and
+  * appends a message to err suitable for
+  * 'error("%s", err->buf)' on error.
+  */
 int unlink_or_msg(const char *file, struct strbuf *err);
 /*
- * Likewise for rmdir(2).
+ * Preserves errno, prints a message, but gives no warning for ENOENT.
+ * Returns 0 on success which includes trying to remove a directory that does
+ * not exist.
  */
 int rmdir_or_warn(const char *path);
 /*
diff --git c/http-backend.c w/http-backend.c
index 059f790..8f94f9b 100644
--- c/http-backend.c
+++ w/http-backend.c
@@ -418,7 +418,7 @@ static int show_head_ref(const char *refname, const unsigned char *sha1,
 	if (flag & REF_ISSYMREF) {
 		unsigned char unused[20];
 		const char *target = resolve_ref_unsafe(refname, unused,
-						RESOLVE_REF_READING, NULL);
+						NULL, RESOLVE_REF_READING);
 		const char *target_nons = strip_namespace(target);
 
 		strbuf_addf(buf, "ref: %s\n", target_nons);
diff --git c/lockfile.c w/lockfile.c
index a921d77..dbd4101 100644
--- c/lockfile.c
+++ w/lockfile.c
@@ -176,16 +176,6 @@ void unable_to_lock_message(const char *path, int err, struct strbuf *buf)
 			    absolute_path(path), strerror(err));
 }
 
-int unable_to_lock_error(const char *path, int err)
-{
-	struct strbuf buf = STRBUF_INIT;
-
-	unable_to_lock_message(path, err, &buf);
-	error("%s", buf.buf);
-	strbuf_release(&buf);
-	return -1;
-}
-
 NORETURN void unable_to_lock_index_die(const char *path, int err)
 {
 	struct strbuf buf = STRBUF_INIT;
diff --git c/notes-merge.c w/notes-merge.c
index 94a1a8a..ffca602 100644
--- c/notes-merge.c
+++ w/notes-merge.c
@@ -549,7 +549,7 @@ int notes_merge(struct notes_merge_options *o,
 	       o->local_ref, o->remote_ref);
 
 	/* Dereference o->local_ref into local_sha1 */
-	if (read_ref_full(o->local_ref, local_sha1, 0, NULL))
+	if (read_ref_full(o->local_ref, local_sha1, NULL, 0))
 		die("Failed to resolve local notes ref '%s'", o->local_ref);
 	else if (!check_refname_format(o->local_ref, 0) &&
 		is_null_sha1(local_sha1))
diff --git c/reflog-walk.c w/reflog-walk.c
index d80a42a..feeaf0a 100644
--- c/reflog-walk.c
+++ w/reflog-walk.c
@@ -48,8 +48,8 @@ static struct complete_reflogs *read_complete_reflog(const char *ref)
 		unsigned char sha1[20];
 		const char *name;
 		void *name_to_free;
-		name = name_to_free = resolve_refdup(ref, sha1,
-						     RESOLVE_REF_READING, NULL);
+		name = name_to_free = resolve_refdup(ref, sha1, NULL,
+						     RESOLVE_REF_READING);
 		if (name) {
 			for_each_reflog_ent(name, read_one_reflog, reflogs);
 			free(name_to_free);
@@ -175,7 +175,7 @@ int add_reflog_for_walk(struct reflog_walk_info *info,
 		if (*branch == '\0') {
 			unsigned char sha1[20];
 			free(branch);
-			branch = resolve_refdup("HEAD", sha1, 0, NULL);
+			branch = resolve_refdup("HEAD", sha1, NULL, 0);
 			if (!branch)
 				die ("No current branch");
 
diff --git c/refs.c w/refs.c
index b14bbf0..3b27758 100644
--- c/refs.c
+++ w/refs.c
@@ -281,11 +281,15 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
 }
 
 static struct ref_entry *create_ref_entry(const char *refname,
-					  const unsigned char *sha1, int flag)
+					  const unsigned char *sha1, int flag,
+					  int check_name)
 {
 	int len;
 	struct ref_entry *ref;
 
+	if (check_name &&
+	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT))
+		die("Reference has invalid format: '%s'", refname);
 	len = strlen(refname) + 1;
 	ref = xmalloc(sizeof(struct ref_entry) + len);
 	hashcpy(ref->u.value.sha1, sha1);
@@ -798,17 +802,16 @@ static int names_conflict(const char *refname1, const char *refname2)
 struct name_conflict_cb {
 	const char *refname;
 	const char *conflicting_refname;
-	const char **skip;
-	int skipnum;
+	struct string_list *skiplist;
 };
 
 static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
 {
 	struct name_conflict_cb *data = (struct name_conflict_cb *)cb_data;
-	int i;
-	for (i = 0; i < data->skipnum; i++)
-		if (!strcmp(entry->name, data->skip[i]))
-			return 0;
+
+	if (data->skiplist &&
+	    string_list_has_string(data->skiplist, entry->name))
+		return 0;
 	if (names_conflict(data->refname, entry->name)) {
 		data->conflicting_refname = entry->name;
 		return 1;
@@ -821,18 +824,17 @@ static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
  * conflicting with the name of an existing reference in dir.  If
  * oldrefname is non-NULL, ignore potential conflicts with oldrefname
  * (e.g., because oldrefname is scheduled for deletion in the same
- * operation). skip contains a list of refs we want to skip checking for
- * conflicts with.
+ * operation). skiplist contains a list of refs we want to skip checking for
+ * conflicts with. skiplist must be sorted.
  */
 static int is_refname_available(const char *refname,
 				struct ref_dir *dir,
-				const char **skip, int skipnum)
+				struct string_list *skiplist)
 {
 	struct name_conflict_cb data;
 	data.refname = refname;
 	data.conflicting_refname = NULL;
-	data.skip = skip;
-	data.skipnum = skipnum;
+	data.skiplist = skiplist;
 
 	sort_ref_dir(dir);
 	if (do_for_each_entry_in_dir(dir, 0, name_conflict_fn, &data)) {
@@ -1062,8 +1064,9 @@ static void read_packed_refs(FILE *f, struct ref_dir *dir)
 
 			if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT)) {
 				flag |= REF_ISBROKEN;
+				hashclr(sha1);
 			}
-			last = create_ref_entry(refname, sha1, flag);
+			last = create_ref_entry(refname, sha1, flag, 0);
 			if (peeled == PEELED_FULLY ||
 			    (peeled == PEELED_TAGS && starts_with(refname, "refs/tags/")))
 				last->flag |= REF_KNOWS_PEELED;
@@ -1136,10 +1139,8 @@ void add_packed_ref(const char *refname, const unsigned char *sha1)
 
 	if (!packed_ref_cache->lock)
 		die("internal error: packed refs not locked");
-	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT))
-		die("Reference has invalid format: '%s'", refname);
 	add_ref(get_packed_ref_dir(packed_ref_cache),
-		create_ref_entry(refname, sha1, REF_ISPACKED));
+		create_ref_entry(refname, sha1, REF_ISPACKED, 1));
 }
 
 /*
@@ -1197,17 +1198,19 @@ static void read_loose_refs(const char *dirname, struct ref_dir *dir)
 					hashclr(sha1);
 					flag |= REF_ISBROKEN;
 				}
-			} else if (read_ref_full(refname.buf, sha1,
-						 RESOLVE_REF_READING, &flag)) {
+			} else if (read_ref_full(refname.buf, sha1, &flag,
+						 RESOLVE_REF_READING)) {
 				hashclr(sha1);
 				flag |= REF_ISBROKEN;
 			}
-			if (check_refname_format(refname.buf, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT)) {
+			if (check_refname_format(refname.buf,
+						 REFNAME_ALLOW_ONELEVEL|
+						 REFNAME_DOT_COMPONENT)) {
 				hashclr(sha1);
 				flag |= REF_ISBROKEN;
 			}
 			add_entry_to_dir(dir,
-					 create_ref_entry(refname.buf, sha1, flag));
+					 create_ref_entry(refname.buf, sha1, flag, 0));
 		}
 		strbuf_setlen(&refname, dirnamelen);
 	}
@@ -1353,21 +1356,40 @@ static const char *handle_missing_loose_ref(const char *refname,
 	}
 }
 
+static int escapes_cwd(const char *path) {
+	char *buf;
+	int result;
+
+	if (is_absolute_path(path))
+		return 1;
+	buf = xmalloc(strlen(path) + 1);
+	result = normalize_path_copy(buf, path);
+	free(buf);
+	return result;
+}
+
 /* This function needs to return a meaningful errno on failure */
-const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int flags, int *ref_flag)
+const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *flags, int resolve_flags)
 {
 	int depth = MAXDEPTH;
 	ssize_t len;
 	char buffer[256];
 	static char refname_buffer[256];
+	int bad_name = 0;
 
-	if (ref_flag)
-		*ref_flag = 0;
+	if (flags)
+		*flags = 0;
 
-	if (!(flags & RESOLVE_REF_ALLOW_BAD_NAME) &&
-	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
-		errno = EINVAL;
-		return NULL;
+	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+		    escapes_cwd(refname)) {
+			errno = EINVAL;
+			return NULL;
+		}
+		hashclr(sha1);
+		if (flags)
+			*flags |= REF_ISBROKEN;
+		bad_name = 1;
 	}
 	for (;;) {
 		char path[PATH_MAX];
@@ -1395,8 +1417,8 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int fla
 		if (lstat(path, &st) < 0) {
 			if (errno == ENOENT)
 				return handle_missing_loose_ref(refname, sha1,
-						flags & RESOLVE_REF_READING,
-						ref_flag);
+					resolve_flags & RESOLVE_REF_READING,
+					flags);
 			else
 				return NULL;
 		}
@@ -1416,8 +1438,12 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int fla
 					!check_refname_format(buffer, 0)) {
 				strcpy(refname_buffer, buffer);
 				refname = refname_buffer;
-				if (ref_flag)
-					*ref_flag |= REF_ISSYMREF;
+				if (flags)
+					*flags |= REF_ISSYMREF;
+				if (resolve_flags & RESOLVE_REF_NODEREF) {
+					hashclr(sha1);
+					return refname;
+				}
 				continue;
 			}
 		}
@@ -1462,31 +1488,37 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int fla
 			 */
 			if (get_sha1_hex(buffer, sha1) ||
 			    (buffer[40] != '\0' && !isspace(buffer[40]))) {
-				if (ref_flag)
-					*ref_flag |= REF_ISBROKEN;
+				if (flags)
+					*flags |= REF_ISBROKEN;
 				errno = EINVAL;
 				return NULL;
 			}
+			if (bad_name)
+				hashclr(sha1);
 			return refname;
 		}
-		if (ref_flag)
-			*ref_flag |= REF_ISSYMREF;
+		if (flags)
+			*flags |= REF_ISSYMREF;
 		buf = buffer + 4;
 		while (isspace(*buf))
 			buf++;
 		if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
-			if (ref_flag)
-				*ref_flag |= REF_ISBROKEN;
+			if (flags)
+				*flags |= REF_ISBROKEN;
 			errno = EINVAL;
 			return NULL;
 		}
 		refname = strcpy(refname_buffer, buf);
+		if (resolve_flags & RESOLVE_REF_NODEREF) {
+			hashclr(sha1);
+			return refname;
+		}
 	}
 }
 
-char *resolve_refdup(const char *ref, unsigned char *sha1, int flags, int *ref_flag)
+char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags)
 {
-	const char *ret = resolve_ref_unsafe(ref, sha1, flags, ref_flag);
+	const char *ret = resolve_ref_unsafe(ref, sha1, flags, resolve_flags);
 	return ret ? xstrdup(ret) : NULL;
 }
 
@@ -1497,22 +1529,22 @@ struct ref_filter {
 	void *cb_data;
 };
 
-int read_ref_full(const char *refname, unsigned char *sha1, int flags, int *ref_flag)
+int read_ref_full(const char *refname, unsigned char *sha1, int *flags, int resolve_flags)
 {
-	if (resolve_ref_unsafe(refname, sha1, flags, ref_flag))
+	if (resolve_ref_unsafe(refname, sha1, flags, resolve_flags))
 		return 0;
 	return -1;
 }
 
 int read_ref(const char *refname, unsigned char *sha1)
 {
-	return read_ref_full(refname, sha1, RESOLVE_REF_READING, NULL);
+	return read_ref_full(refname, sha1, NULL, RESOLVE_REF_READING);
 }
 
 int ref_exists(const char *refname)
 {
 	unsigned char sha1[20];
-	return !!resolve_ref_unsafe(refname, sha1, RESOLVE_REF_READING, NULL);
+	return !!resolve_ref_unsafe(refname, sha1, NULL, RESOLVE_REF_READING);
 }
 
 static int filter_refs(const char *refname, const unsigned char *sha1, int flags,
@@ -1626,7 +1658,7 @@ int peel_ref(const char *refname, unsigned char *sha1)
 		return 0;
 	}
 
-	if (read_ref_full(refname, base, RESOLVE_REF_READING, &flag))
+	if (read_ref_full(refname, base, &flag, RESOLVE_REF_READING))
 		return -1;
 
 	/*
@@ -1667,7 +1699,7 @@ static int warn_if_dangling_symref(const char *refname, const unsigned char *sha
 	if (!(flags & REF_ISSYMREF))
 		return 0;
 
-	resolves_to = resolve_ref_unsafe(refname, junk, 0, NULL);
+	resolves_to = resolve_ref_unsafe(refname, junk, NULL, 0);
 	if (!resolves_to
 	    || (d->refname
 		? strcmp(resolves_to, d->refname)
@@ -1792,7 +1824,7 @@ static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
 		return 0;
 	}
 
-	if (!read_ref_full("HEAD", sha1, RESOLVE_REF_READING, &flag))
+	if (!read_ref_full("HEAD", sha1, &flag, RESOLVE_REF_READING))
 		return fn("HEAD", sha1, flag, cb_data);
 
 	return 0;
@@ -1872,7 +1904,7 @@ int head_ref_namespaced(each_ref_fn fn, void *cb_data)
 	int flag;
 
 	strbuf_addf(&buf, "%sHEAD", get_git_namespace());
-	if (!read_ref_full(buf.buf, sha1, RESOLVE_REF_READING, &flag))
+	if (!read_ref_full(buf.buf, sha1, &flag, RESOLVE_REF_READING))
 		ret = fn(buf.buf, sha1, flag, cb_data);
 	strbuf_release(&buf);
 
@@ -1967,8 +1999,8 @@ int refname_match(const char *abbrev_name, const char *full_name)
 static struct ref_lock *verify_lock(struct ref_lock *lock,
 	const unsigned char *old_sha1, int mustexist)
 {
-	if (read_ref_full(lock->ref_name, lock->old_sha1,
-			  mustexist ? RESOLVE_REF_READING : 0, NULL)) {
+	if (read_ref_full(lock->ref_name, lock->old_sha1, NULL,
+			  mustexist ? RESOLVE_REF_READING : 0)) {
 		int save_errno = errno;
 		error("Can't verify ref %s", lock->ref_name);
 		unlock_ref(lock);
@@ -2041,8 +2073,8 @@ int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
 
 		this_result = refs_found ? sha1_from_ref : sha1;
 		mksnpath(fullref, sizeof(fullref), *p, len, str);
-		r = resolve_ref_unsafe(fullref, this_result,
-				       RESOLVE_REF_READING, &flag);
+		r = resolve_ref_unsafe(fullref, this_result, &flag,
+				       RESOLVE_REF_READING);
 		if (r) {
 			if (!refs_found++)
 				*ref = xstrdup(r);
@@ -2071,7 +2103,7 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 		const char *ref, *it;
 
 		mksnpath(path, sizeof(path), *p, len, str);
-		ref = resolve_ref_unsafe(path, hash, RESOLVE_REF_READING, NULL);
+		ref = resolve_ref_unsafe(path, hash, NULL, RESOLVE_REF_READING);
 		if (!ref)
 			continue;
 		if (reflog_exists(path))
@@ -2097,8 +2129,8 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
  */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
-					    int flags, int *type_p,
-					    const char **skip, int skipnum)
+					    struct string_list *skiplist,
+					    int flags, int *type_p)
 {
 	char *ref_file;
 	const char *orig_refname = refname;
@@ -2106,22 +2138,23 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	int last_errno = 0;
 	int type, lflags;
 	int mustexist = (old_sha1 && !is_null_sha1(old_sha1));
-	int resolve_flags;
+	int resolve_flags = 0;
 	int missing = 0;
 	int attempts_remaining = 3;
-	int bad_refname;
 
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
-	bad_refname = check_refname_format(refname, REFNAME_ALLOW_ONELEVEL);
+	if (flags & REF_BADNAMEOK)
+		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
-	resolve_flags = RESOLVE_REF_ALLOW_BAD_NAME;
 	if (mustexist)
 		resolve_flags |= RESOLVE_REF_READING;
+	if (flags & REF_NODEREF)
+		resolve_flags |= RESOLVE_REF_NODEREF;
 
-	refname = resolve_ref_unsafe(refname, lock->old_sha1, resolve_flags,
-				     &type);
+	refname = resolve_ref_unsafe(refname, lock->old_sha1, &type,
+				     resolve_flags);
 	if (!refname && errno == EISDIR) {
 		/* we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -2135,7 +2168,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 			goto error_return;
 		}
 		refname = resolve_ref_unsafe(orig_refname, lock->old_sha1,
-					     resolve_flags, &type);
+					     &type, resolve_flags);
 	}
 	if (type_p)
 	    *type_p = type;
@@ -2153,7 +2186,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	 */
 	if (missing &&
 	     !is_refname_available(refname, get_packed_refs(&ref_cache),
-				   skip, skipnum)) {
+				   skiplist)) {
 		last_errno = ENOTDIR;
 		goto error_return;
 	}
@@ -2199,8 +2232,6 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 		else
 			unable_to_lock_index_die(ref_file, errno);
 	}
-	if (bad_refname)
-		return lock;
 	return old_sha1 ? verify_lock(lock, old_sha1, mustexist) : lock;
 
  error_return:
@@ -2213,7 +2244,7 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
 {
-	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p, NULL, 0);
+	return lock_ref_sha1_basic(refname, old_sha1, NULL, flags, type_p);
 }
 
 /*
@@ -2364,9 +2395,8 @@ static int pack_if_possible_fn(struct ref_entry *entry, void *cb_data)
 		packed_entry->flag = REF_ISPACKED | REF_KNOWS_PEELED;
 		hashcpy(packed_entry->u.value.sha1, entry->u.value.sha1);
 	} else {
-		packed_entry = create_ref_entry(entry->name,
-					entry->u.value.sha1,
-					REF_ISPACKED | REF_KNOWS_PEELED);
+		packed_entry = create_ref_entry(entry->name, entry->u.value.sha1,
+						REF_ISPACKED | REF_KNOWS_PEELED, 0);
 		add_ref(cb->packed_refs, packed_entry);
 	}
 	hashcpy(packed_entry->u.value.peeled, entry->u.value.peeled);
@@ -2491,7 +2521,7 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 		unsigned char sha1[20];
 		int flags;
 
-		if (read_ref_full(entry->name, sha1, 0, &flags))
+		if (read_ref_full(entry->name, sha1, &flags, 0))
 			/* We should at least have found the packed ref. */
 			die("Internal error");
 		if ((flags & REF_ISSYMREF) || !(flags & REF_ISPACKED)) {
@@ -2530,6 +2560,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 	struct string_list_item *ref_to_delete;
 	int i, ret, removed = 0;
 
+	assert(err);
+
 	/* Look for a packed ref */
 	for (i = 0; i < n; i++)
 		if (get_packed_ref(refnames[i]))
@@ -2540,13 +2572,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
-		if (err) {
-			unable_to_lock_message(git_path("packed-refs"), errno,
-					       err);
-			return -1;
-		}
-		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refnames[i]);
+		unable_to_lock_message(git_path("packed-refs"), errno, err);
+		return -1;
 	}
 	packed = get_packed_refs(&ref_cache);
 
@@ -2572,7 +2599,7 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 
 	/* Write what remains */
 	ret = commit_packed_refs();
-	if (ret && err)
+	if (ret)
 		strbuf_addf(err, "unable to overwrite old ref-pack file: %s",
 			    strerror(errno));
 	return ret;
@@ -2580,6 +2607,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 
 static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
 {
+	assert(err);
+
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
 		/* loose */
 		int res, i = strlen(lock->lk->filename) - 5; /* .lock */
@@ -2678,25 +2707,31 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	struct stat loginfo;
 	int log = !lstat(git_path("logs/%s", oldrefname), &loginfo);
 	const char *symref = NULL;
+	struct string_list skiplist = STRING_LIST_INIT_NODUP;
 
 	if (log && S_ISLNK(loginfo.st_mode))
 		return error("reflog for %s is a symlink", oldrefname);
 
-	symref = resolve_ref_unsafe(oldrefname, orig_sha1,
-				    RESOLVE_REF_READING, &flag);
+	symref = resolve_ref_unsafe(oldrefname, orig_sha1, &flag,
+				    RESOLVE_REF_READING);
 	if (flag & REF_ISSYMREF)
 		return error("refname %s is a symbolic ref, renaming it is not supported",
 			oldrefname);
 	if (!symref)
 		return error("refname %s not found", oldrefname);
 
+	string_list_insert(&skiplist, oldrefname);
 	if (!is_refname_available(newrefname, get_packed_refs(&ref_cache),
-				  &oldrefname, 1))
+				  &skiplist)) {
+		string_list_clear(&skiplist, 0);
 		return 1;
-
+	}
 	if (!is_refname_available(newrefname, get_loose_refs(&ref_cache),
-				  &oldrefname, 1))
+				  &skiplist)) {
+		string_list_clear(&skiplist, 0);
 		return 1;
+	}
+	string_list_clear(&skiplist, 0);
 
 	if (log && rename(git_path("logs/%s", oldrefname), git_path(TMP_RENAMED_LOG)))
 		return error("unable to move logfile logs/%s to "TMP_RENAMED_LOG": %s",
@@ -2707,7 +2742,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 		goto rollback;
 	}
 
-	if (!read_ref_full(newrefname, sha1, RESOLVE_REF_READING, NULL) &&
+	if (!read_ref_full(newrefname, sha1, NULL, RESOLVE_REF_READING) &&
 	    delete_ref(newrefname, sha1, REF_NODEREF)) {
 		if (errno==EISDIR) {
 			if (remove_empty_directories(git_path("%s", newrefname))) {
@@ -2725,7 +2760,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 
 	logmoved = log;
 
-	lock = lock_ref_sha1_basic(newrefname, NULL, 0, NULL, NULL, 0);
+	lock = lock_ref_sha1_basic(newrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for update", newrefname);
 		goto rollback;
@@ -2740,7 +2775,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	return 0;
 
  rollback:
-	lock = lock_ref_sha1_basic(oldrefname, NULL, 0, NULL, NULL, 0);
+	lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for rollback", oldrefname);
 		goto rollbacklog;
@@ -2985,8 +3020,8 @@ static int write_ref_sha1(struct ref_lock *lock,
 		unsigned char head_sha1[20];
 		int head_flag;
 		const char *head_ref;
-		head_ref = resolve_ref_unsafe("HEAD", head_sha1,
-					      RESOLVE_REF_READING, &head_flag);
+		head_ref = resolve_ref_unsafe("HEAD", head_sha1, &head_flag,
+					      RESOLVE_REF_READING);
 		if (head_ref && (head_flag & REF_ISSYMREF) &&
 		    !strcmp(head_ref, lock->ref_name))
 			log_ref_write("HEAD", lock->old_sha1, sha1, logmsg);
@@ -3353,7 +3388,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data
 				retval = do_for_each_reflog(name, fn, cb_data);
 			} else {
 				unsigned char sha1[20];
-				if (read_ref_full(name->buf, sha1, 0, NULL))
+				if (read_ref_full(name->buf, sha1, NULL, 0))
 					retval = error("bad ref for %s", name->buf);
 				else
 					retval = fn(name->buf, sha1, 0, cb_data);
@@ -3423,6 +3458,8 @@ struct ref_transaction {
 
 struct ref_transaction *ref_transaction_begin(struct strbuf *err)
 {
+	assert(err);
+
 	return xcalloc(1, sizeof(struct ref_transaction));
 }
 
@@ -3462,6 +3499,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: update called for transaction that is not open");
 
@@ -3470,7 +3509,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 
 	if (!is_null_sha1(new_sha1) &&
 	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
-		strbuf_addf(err, "Bad refname: %s", refname);
+		strbuf_addf(err, "refusing to update ref with bad name %s",
+			    refname);
 		return -1;
 	}
 
@@ -3493,6 +3533,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: create called for transaction that is not open");
 
@@ -3500,7 +3542,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		die("BUG: create ref with null new_sha1");
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
-		strbuf_addf(err, "Bad refname: %s", refname);
+		strbuf_addf(err, "refusing to create ref with bad name %s",
+			    refname);
 		return -1;
 	}
 
@@ -3523,6 +3566,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: delete called for transaction that is not open");
 
@@ -3585,13 +3630,14 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,
 					struct strbuf *err)
 {
 	int i;
+
+	assert(err);
+
 	for (i = 1; i < n; i++)
 		if (!strcmp(updates[i - 1]->refname, updates[i]->refname)) {
-			const char *str =
-				"Multiple updates for ref '%s' not allowed.";
-			if (err)
-				strbuf_addf(err, str, updates[i]->refname);
-
+			strbuf_addf(err,
+				    "Multiple updates for ref '%s' not allowed.",
+				    updates[i]->refname);
 			return 1;
 		}
 	return 0;
@@ -3600,11 +3646,13 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,
 int ref_transaction_commit(struct ref_transaction *transaction,
 			   struct strbuf *err)
 {
-	int ret = 0, delnum = 0, i, df_conflict = 0;
+	int ret = 0, delnum = 0, i;
 	const char **delnames;
 	int n = transaction->nr;
 	struct ref_update **updates = transaction->updates;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: commit called for transaction that is not open");
 
@@ -3631,16 +3679,15 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 						   (update->have_old ?
 						    update->old_sha1 :
 						    NULL),
+						   NULL,
 						   update->flags,
-						   &update->type,
-						   delnames, delnum);
+						   &update->type);
 		if (!update->lock) {
-			if (errno == ENOTDIR)
-				df_conflict = 1;
-			if (err)
-				strbuf_addf(err, "Cannot lock the ref '%s'.",
-					    update->refname);
-			ret = -1;
+			int df_conflict = (errno == ENOTDIR);
+
+			strbuf_addf(err, "Cannot lock the ref '%s'.",
+				    update->refname);
+			ret = df_conflict ? UPDATE_REFS_NAME_CONFLICT : -1;
 			goto cleanup;
 		}
 	}
@@ -3654,9 +3701,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 					     update->msg);
 			update->lock = NULL; /* freed by write_ref_sha1 */
 			if (ret) {
-				if (err)
-					strbuf_addf(err, "Cannot update the ref '%s'.",
-						    update->refname);
+				strbuf_addf(err, "Cannot update the ref '%s'.",
+					    update->refname);
 				ret = -1;
 				goto cleanup;
 			}
@@ -3668,16 +3714,20 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			if (delete_ref_loose(update->lock, update->type, err))
+			if (delete_ref_loose(update->lock, update->type, err)) {
 				ret = -1;
+				goto cleanup;
+			}
 
 			if (!(update->flags & REF_ISPRUNING))
 				delnames[delnum++] = update->lock->ref_name;
 		}
 	}
 
-	if (repack_without_refs(delnames, delnum, err))
+	if (repack_without_refs(delnames, delnum, err)) {
 		ret = -1;
+		goto cleanup;
+	}
 	for (i = 0; i < delnum; i++)
 		unlink_or_warn(git_path("logs/%s", delnames[i]));
 	clear_loose_ref_cache(&ref_cache);
@@ -3689,8 +3739,6 @@ cleanup:
 		if (updates[i]->lock)
 			unlock_ref(updates[i]->lock);
 	free(delnames);
-	if (df_conflict)
-		ret = -2;
 	return ret;
 }
 
diff --git c/refs.h w/refs.h
index a7b8009..a96e174 100644
--- c/refs.h
+++ w/refs.h
@@ -175,10 +175,12 @@ extern int peel_ref(const char *refname, unsigned char *sha1);
  * ref_transaction_create(), etc.
  * REF_NODEREF: act on the ref directly, instead of dereferencing
  *              symbolic references.
+ * REF_BADNAMEOK: allow locking a ref that has a bad name.
  *
  * Flags >= 0x100 are reserved for internal use.
  */
 #define REF_NODEREF	0x01
+#define REF_BADNAMEOK	0x02
 /*
  * This function sets errno to something meaningful on failure.
  */
@@ -322,11 +324,10 @@ int ref_transaction_delete(struct ref_transaction *transaction,
  * Commit all of the changes that have been queued in transaction, as
  * atomically as possible.  Return a nonzero value if there is a
  * problem.
- * If the transaction is already in failed state this function will return
- * an error.
+ *
  * Function returns 0 on success, -1 for generic failures and
- * UPDATE_REFS_NAME_CONFLICT (-2) if the failure was due to a name
- * collision (ENOTDIR).
+ * UPDATE_REFS_NAME_CONFLICT (-2) if the failure was due to a naming conflict.
+ * For example, the ref names A and A/B conflict.
  */
 #define UPDATE_REFS_NAME_CONFLICT -2
 int ref_transaction_commit(struct ref_transaction *transaction,
diff --git c/remote.c w/remote.c
index 2adb31c..67c375d 100644
--- c/remote.c
+++ w/remote.c
@@ -486,7 +486,7 @@ static void read_config(void)
 		return;
 	default_remote_name = "origin";
 	current_branch = NULL;
-	head_ref = resolve_ref_unsafe("HEAD", sha1, 0, &flag);
+	head_ref = resolve_ref_unsafe("HEAD", sha1, &flag, 0);
 	if (head_ref && (flag & REF_ISSYMREF) &&
 	    starts_with(head_ref, "refs/heads/")) {
 		current_branch =
@@ -1121,8 +1121,8 @@ static char *guess_ref(const char *name, struct ref *peer)
 	struct strbuf buf = STRBUF_INIT;
 	unsigned char sha1[20];
 
-	const char *r = resolve_ref_unsafe(peer->name, sha1,
-					   RESOLVE_REF_READING, NULL);
+	const char *r = resolve_ref_unsafe(peer->name, sha1, NULL,
+					   RESOLVE_REF_READING);
 	if (!r)
 		return NULL;
 
@@ -1183,8 +1183,8 @@ static int match_explicit(struct ref *src, struct ref *dst,
 		unsigned char sha1[20];
 		int flag;
 
-		dst_value = resolve_ref_unsafe(matched_src->name, sha1,
-					       RESOLVE_REF_READING, &flag);
+		dst_value = resolve_ref_unsafe(matched_src->name, sha1, &flag,
+					       RESOLVE_REF_READING);
 		if (!dst_value ||
 		    ((flag & REF_ISSYMREF) &&
 		     !starts_with(dst_value, "refs/heads/")))
@@ -1658,7 +1658,7 @@ static int ignore_symref_update(const char *refname)
 	unsigned char sha1[20];
 	int flag;
 
-	if (!resolve_ref_unsafe(refname, sha1, 0, &flag))
+	if (!resolve_ref_unsafe(refname, sha1, &flag, 0))
 		return 0; /* non-existing refs are OK */
 	return (flag & REF_ISSYMREF);
 }
diff --git c/sequencer.c w/sequencer.c
index ebe8713..6a05ad4 100644
--- c/sequencer.c
+++ w/sequencer.c
@@ -366,7 +366,7 @@ static int is_index_unchanged(void)
 	unsigned char head_sha1[20];
 	struct commit *head_commit;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, RESOLVE_REF_READING, NULL))
+	if (!resolve_ref_unsafe("HEAD", head_sha1, NULL, RESOLVE_REF_READING))
 		return error(_("Could not resolve HEAD commit\n"));
 
 	head_commit = lookup_commit(head_sha1);
@@ -912,7 +912,7 @@ static int rollback_single_pick(void)
 	if (!file_exists(git_path("CHERRY_PICK_HEAD")) &&
 	    !file_exists(git_path("REVERT_HEAD")))
 		return error(_("no cherry-pick or revert in progress"));
-	if (read_ref_full("HEAD", head_sha1, 0, NULL))
+	if (read_ref_full("HEAD", head_sha1, NULL, 0))
 		return error(_("cannot resolve HEAD"));
 	if (is_null_sha1(head_sha1))
 		return error(_("cannot abort from a branch yet to be born"));
diff --git c/t/t1400-update-ref.sh w/t/t1400-update-ref.sh
index 0218e96..ff4607b 100755
--- c/t/t1400-update-ref.sh
+++ w/t/t1400-update-ref.sh
@@ -110,6 +110,22 @@ test_expect_success "delete symref without dereference when the referred ref is
 cp -f .git/HEAD.orig .git/HEAD
 git update-ref -d $m
 
+test_expect_success 'update-ref -d is not confused by self-reference' '
+	git symbolic-ref refs/heads/self refs/heads/self &&
+	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_path_is_file .git/refs/heads/self &&
+	test_must_fail git update-ref -d refs/heads/self &&
+	test_path_is_file .git/refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete self-reference' '
+	git symbolic-ref refs/heads/self refs/heads/self &&
+	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_path_is_file .git/refs/heads/self &&
+	git update-ref --no-deref -d refs/heads/self &&
+	test_path_is_missing .git/refs/heads/self
+'
+
 test_expect_success '(not) create HEAD with old sha1' "
 	test_must_fail git update-ref HEAD $A $B
 "
diff --git c/t/t1402-check-ref-format.sh w/t/t1402-check-ref-format.sh
index 1a5a5f3..058fa37 100755
--- c/t/t1402-check-ref-format.sh
+++ w/t/t1402-check-ref-format.sh
@@ -196,4 +196,52 @@ invalid_ref_normalized 'heads///foo.lock'
 invalid_ref_normalized 'foo.lock/bar'
 invalid_ref_normalized 'foo.lock///bar'
 
+test_expect_success 'git branch shows badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch >output &&
+	grep -e "broken...ref" output
+'
+
+test_expect_success 'git branch -d can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -d broken...ref &&
+	git branch >output &&
+	! grep -e "broken...ref" output
+'
+
+test_expect_success 'git branch -D can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -D broken...ref &&
+	git branch >output &&
+	! grep -e "broken...ref" output
+'
+
+test_expect_success 'git update-ref -d can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git update-ref -d refs/heads/broken...ref &&
+	git branch >output &&
+	! grep -e "broken...ref" output
+'
+
+test_expect_success 'git branch can not create a badly named ref' '
+	test_must_fail git branch broken...ref
+'
+
+test_expect_success 'git branch can not rename to a bad ref name' '
+	git branch goodref &&
+	test_must_fail git branch -m goodref broken...ref
+'
+
+test_expect_failure 'git branch can rename from a bad ref name' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -m broken...ref renamed &&
+	test_must_fail git rev-parse broken...ref &&
+	test_cmp_rev master renamed
+'
+
 test_done
diff --git c/t/t3200-branch.sh w/t/t3200-branch.sh
index ac31b71..432921b 100755
--- c/t/t3200-branch.sh
+++ w/t/t3200-branch.sh
@@ -285,6 +285,15 @@ test_expect_success 'deleting a dangling symref' '
 	test_i18ncmp expect actual
 '
 
+test_expect_success 'deleting a self-referential symref' '
+	git symbolic-ref refs/heads/self-reference refs/heads/self-reference &&
+	test_path_is_file .git/refs/heads/self-reference &&
+	echo "Deleted branch self-reference (was refs/heads/self-reference)." >expect &&
+	git branch -d self-reference >actual &&
+	test_path_is_missing .git/refs/heads/self-reference &&
+	test_i18ncmp expect actual
+'
+
 test_expect_success 'renaming a symref is not allowed' '
 	git symbolic-ref refs/heads/master2 refs/heads/master &&
 	test_must_fail git branch -m master2 master3 &&
diff --git c/t/t7001-mv.sh w/t/t7001-mv.sh
index 54d7807..69f11bd 100755
--- c/t/t7001-mv.sh
+++ w/t/t7001-mv.sh
@@ -350,10 +350,11 @@ test_expect_success 'git mv moves a submodule with a .git directory and .gitmodu
 '
 
 test_expect_success 'git mv moves a submodule with gitfile' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	(
 		cd mod &&
 		git mv ../sub/ .
@@ -372,11 +373,12 @@ test_expect_success 'git mv moves a submodule with gitfile' '
 '
 
 test_expect_success 'mv does not complain when no .gitmodules file is found' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git rm .gitmodules &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	git mv sub mod/sub 2>actual.err &&
 	! test -s actual.err &&
 	! test -e sub &&
@@ -390,11 +392,12 @@ test_expect_success 'mv does not complain when no .gitmodules file is found' '
 '
 
 test_expect_success 'mv will error out on a modified .gitmodules file unless staged' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git config -f .gitmodules foo.bar true &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	test_must_fail git mv sub mod/sub 2>actual.err &&
 	test -s actual.err &&
 	test -e sub &&
@@ -413,13 +416,14 @@ test_expect_success 'mv will error out on a modified .gitmodules file unless sta
 '
 
 test_expect_success 'mv issues a warning when section is not found in .gitmodules' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git config -f .gitmodules --remove-section submodule.sub &&
 	git add .gitmodules &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
 	echo "warning: Could not find section in .gitmodules where path=sub" >expect.err &&
+	mkdir mod &&
 	git mv sub mod/sub 2>actual.err &&
 	test_i18ncmp expect.err actual.err &&
 	! test -e sub &&
@@ -433,9 +437,10 @@ test_expect_success 'mv issues a warning when section is not found in .gitmodule
 '
 
 test_expect_success 'mv --dry-run does not touch the submodule or .gitmodules' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
+	mkdir mod &&
 	git mv -n sub mod/sub 2>actual.err &&
 	test -f sub/.git &&
 	git diff-index --exit-code HEAD &&
diff --git c/transport-helper.c w/transport-helper.c
index 270ae28..8365441 100644
--- c/transport-helper.c
+++ w/transport-helper.c
@@ -889,7 +889,9 @@ static int push_refs_with_export(struct transport *transport,
 					int flag;
 
 					/* Follow symbolic refs (mainly for HEAD). */
-					name = resolve_ref_unsafe(ref->peer_ref->name, sha1, RESOLVE_REF_READING, &flag);
+					name = resolve_ref_unsafe(
+						 ref->peer_ref->name, sha1,
+						 &flag, RESOLVE_REF_READING);
 					if (!name || !(flag & REF_ISSYMREF))
 						name = ref->peer_ref->name;
 
diff --git c/transport.c w/transport.c
index f40e950..3ba7bbf 100644
--- c/transport.c
+++ w/transport.c
@@ -168,8 +168,8 @@ static void set_upstreams(struct transport *transport, struct ref *refs,
 		/* Follow symbolic refs (mainly for HEAD). */
 		localname = ref->peer_ref->name;
 		remotename = ref->name;
-		tmp = resolve_ref_unsafe(localname, sha,
-					 RESOLVE_REF_READING, &flag);
+		tmp = resolve_ref_unsafe(localname, sha, &flag,
+					 RESOLVE_REF_READING);
 		if (tmp && flag & REF_ISSYMREF &&
 			starts_with(tmp, "refs/heads/"))
 			localname = tmp;
@@ -754,7 +754,7 @@ void transport_print_push_status(const char *dest, struct ref *refs,
 	unsigned char head_sha1[20];
 	char *head;
 
-	head = resolve_refdup("HEAD", head_sha1, RESOLVE_REF_READING, NULL);
+	head = resolve_refdup("HEAD", head_sha1, NULL, RESOLVE_REF_READING);
 
 	if (verbose) {
 		for (ref = refs; ref; ref = ref->next)
diff --git c/upload-pack.c w/upload-pack.c
index 01de944..3b51ccb 100644
--- c/upload-pack.c
+++ w/upload-pack.c
@@ -743,7 +743,7 @@ static int find_symref(const char *refname, const unsigned char *sha1, int flag,
 
 	if ((flag & REF_ISSYMREF) == 0)
 		return 0;
-	symref_target = resolve_ref_unsafe(refname, unused, 0, &flag);
+	symref_target = resolve_ref_unsafe(refname, unused, &flag, 0);
 	if (!symref_target || (flag & REF_ISSYMREF) == 0)
 		die("'%s' is a symref but it is not?", refname);
 	item = string_list_append(cb_data, refname);
diff --git c/wrapper.c w/wrapper.c
index 74a0cc0..ec7a08b 100644
--- c/wrapper.c
+++ w/wrapper.c
@@ -430,8 +430,8 @@ int xmkstemp_mode(char *template, int mode)
 static int warn_if_unremovable(const char *op, const char *file, int rc)
 {
 	int err;
-	if (rc >= 0 || errno == ENOENT)
-		return rc;
+	if (!rc || errno == ENOENT)
+		return 0;
 	err = errno;
 	warning("unable to %s %s: %s", op, file, strerror(errno));
 	errno = err;
@@ -440,20 +440,16 @@ static int warn_if_unremovable(const char *op, const char *file, int rc)
 
 int unlink_or_msg(const char *file, struct strbuf *err)
 {
-	if (err) {
-		int rc = unlink(file);
-		int save_errno = errno;
+	int rc = unlink(file);
 
-		if (rc < 0 && errno != ENOENT) {
-			strbuf_addf(err, "unable to unlink %s: %s",
-				    file, strerror(errno));
-			errno = save_errno;
-			return -1;
-		}
+	assert(err);
+
+	if (!rc || errno == ENOENT)
 		return 0;
-	}
 
-	return unlink_or_warn(file);
+	strbuf_addf(err, "unable to unlink %s: %s",
+		    file, strerror(errno));
+	return -1;
 }
 
 int unlink_or_warn(const char *file)
diff --git c/wt-status.c w/wt-status.c
index 318a191..6819e2b 100644
--- c/wt-status.c
+++ w/wt-status.c
@@ -128,7 +128,7 @@ void wt_status_prepare(struct wt_status *s)
 	s->show_untracked_files = SHOW_NORMAL_UNTRACKED_FILES;
 	s->use_color = -1;
 	s->relative_paths = 1;
-	s->branch = resolve_refdup("HEAD", sha1, 0, NULL);
+	s->branch = resolve_refdup("HEAD", sha1, NULL, 0);
 	s->reference = "HEAD";
 	s->fp = stdout;
 	s->index_file = get_index_file();

[PATCH 01/19] mv test: recreate mod/ directory instead of relying on stale copy

[Jonathan Nieder]

The tests for 'git mv moves a submodule' functionality often run
commands like

	git mv sub mod/sub

to move a submodule into a subdirectory.  Just like plain /bin/mv,
this is supposed to succeed if the mod/ parent directory exists
and fail if it doesn't exist.

Usually these tests mkdir the parent directory beforehand, but some
instead rely on it being left behind by previous tests.

More precisely, when 'git reset --hard' tries to move to a state where
mod/sub is not present any more, it would perform the following
operations:

	rmdir("mod/sub")
	rmdir("mod")

The first fails with ENOENT because the test script removed mod/sub
with "rm -rf" already, so 'reset --hard' doesn't bother to move on to
the second, and the mod/ directory is kept around.

Better to explicitly remove and re-create the mod/ directory so later
tests don't have to depend on the directory left behind by the earlier
ones at all (making it easier to rearrange or skip some tests in the
file or to tweak 'reset --hard' behavior without breaking unrelated
tests).

Noticed while testing a patch that fixes the reset --hard behavior
described above.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
 t/t7001-mv.sh | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/t/t7001-mv.sh b/t/t7001-mv.sh
index 54d7807..69f11bd 100755
--- a/t/t7001-mv.sh
+++ b/t/t7001-mv.sh
@@ -350,10 +350,11 @@ test_expect_success 'git mv moves a submodule with a .git directory and .gitmodu
 '
 
 test_expect_success 'git mv moves a submodule with gitfile' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	(
 		cd mod &&
 		git mv ../sub/ .
@@ -372,11 +373,12 @@ test_expect_success 'git mv moves a submodule with gitfile' '
 '
 
 test_expect_success 'mv does not complain when no .gitmodules file is found' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git rm .gitmodules &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	git mv sub mod/sub 2>actual.err &&
 	! test -s actual.err &&
 	! test -e sub &&
@@ -390,11 +392,12 @@ test_expect_success 'mv does not complain when no .gitmodules file is found' '
 '
 
 test_expect_success 'mv will error out on a modified .gitmodules file unless staged' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git config -f .gitmodules foo.bar true &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	test_must_fail git mv sub mod/sub 2>actual.err &&
 	test -s actual.err &&
 	test -e sub &&
@@ -413,13 +416,14 @@ test_expect_success 'mv will error out on a modified .gitmodules file unless sta
 '
 
 test_expect_success 'mv issues a warning when section is not found in .gitmodules' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git config -f .gitmodules --remove-section submodule.sub &&
 	git add .gitmodules &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
 	echo "warning: Could not find section in .gitmodules where path=sub" >expect.err &&
+	mkdir mod &&
 	git mv sub mod/sub 2>actual.err &&
 	test_i18ncmp expect.err actual.err &&
 	! test -e sub &&
@@ -433,9 +437,10 @@ test_expect_success 'mv issues a warning when section is not found in .gitmodule
 '
 
 test_expect_success 'mv --dry-run does not touch the submodule or .gitmodules' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
+	mkdir mod &&
 	git mv -n sub mod/sub 2>actual.err &&
 	test -f sub/.git &&
 	git diff-index --exit-code HEAD &&
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 02/19] wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as success

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>

Simplify the function warn_if_unremovable slightly. Additionally, change
behaviour slightly. If we failed to remove the object because the object
does not exist, we can still return success back to the caller since none of
the callers depend on "fail if the file did not exist".

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 git-compat-util.h |  7 +++++--
 wrapper.c         | 14 ++++++--------
 2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/git-compat-util.h b/git-compat-util.h
index b6f03b3..611e77b 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -706,11 +706,14 @@ void git_qsort(void *base, size_t nmemb, size_t size,
 
 /*
  * Preserves errno, prints a message, but gives no warning for ENOENT.
- * Always returns the return value of unlink(2).
+ * Returns 0 on success which includes trying to unlink an object that does
+ * not exist.
  */
 int unlink_or_warn(const char *path);
 /*
- * Likewise for rmdir(2).
+ * Preserves errno, prints a message, but gives no warning for ENOENT.
+ * Returns 0 on success which includes trying to remove a directory that does
+ * not exist.
  */
 int rmdir_or_warn(const char *path);
 /*
diff --git a/wrapper.c b/wrapper.c
index bc1bfb8..c9605cd 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -429,14 +429,12 @@ int xmkstemp_mode(char *template, int mode)
 
 static int warn_if_unremovable(const char *op, const char *file, int rc)
 {
-	if (rc < 0) {
-		int err = errno;
-		if (ENOENT != err) {
-			warning("unable to %s %s: %s",
-				op, file, strerror(errno));
-			errno = err;
-		}
-	}
+	int err;
+	if (!rc || errno == ENOENT)
+		return 0;
+	err = errno;
+	warning("unable to %s %s: %s", op, file, strerror(errno));
+	errno = err;
 	return rc;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 03/19] wrapper.c: add a new function unlink_or_msg

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Jul 2014 11:20:36 -0700

This behaves like unlink_or_warn except that on failure it writes the message
to its 'err' argument, which the caller can display in an appropriate way or
ignore.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 git-compat-util.h |  9 +++++++++
 wrapper.c         | 14 ++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index 611e77b..5ee140c 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -307,6 +307,8 @@ extern char *gitbasename(char *);
 
 #include "wildmatch.h"
 
+struct strbuf;
+
 /* General helper functions */
 extern void vreportf(const char *prefix, const char *err, va_list params);
 extern void vwritef(int fd, const char *prefix, const char *err, va_list params);
@@ -710,6 +712,13 @@ void git_qsort(void *base, size_t nmemb, size_t size,
  * not exist.
  */
 int unlink_or_warn(const char *path);
+ /*
+  * Tries to unlink file.  Returns 0 if unlink succeeded
+  * or the file already didn't exist.  Returns -1 and
+  * appends a message to err suitable for
+  * 'error("%s", err->buf)' on error.
+  */
+int unlink_or_msg(const char *file, struct strbuf *err);
 /*
  * Preserves errno, prints a message, but gives no warning for ENOENT.
  * Returns 0 on success which includes trying to remove a directory that does
diff --git a/wrapper.c b/wrapper.c
index c9605cd..ec7a08b 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -438,6 +438,20 @@ static int warn_if_unremovable(const char *op, const char *file, int rc)
 	return rc;
 }
 
+int unlink_or_msg(const char *file, struct strbuf *err)
+{
+	int rc = unlink(file);
+
+	assert(err);
+
+	if (!rc || errno == ENOENT)
+		return 0;
+
+	strbuf_addf(err, "unable to unlink %s: %s",
+		    file, strerror(errno));
+	return -1;
+}
+
 int unlink_or_warn(const char *file)
 {
 	return warn_if_unremovable("unlink", file, unlink(file));
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 04/19] refs.c: add an err argument to delete_ref_loose

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 15 May 2014 08:25:23 -0700

Add an err argument to delete_loose_ref so that we can pass a descriptive
error string back to the caller. Pass the err argument from transaction
commit to this function so that transaction users will have a nice error
string if the transaction failed due to delete_loose_ref.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index 7235574..5609622 100644
--- a/refs.c
+++ b/refs.c
@@ -2548,16 +2548,16 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 	return ret;
 }
 
-static int delete_ref_loose(struct ref_lock *lock, int flag)
+static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
 		/* loose */
-		int err, i = strlen(lock->lk->filename) - 5; /* .lock */
+		int res, i = strlen(lock->lk->filename) - 5; /* .lock */
 
 		lock->lk->filename[i] = 0;
-		err = unlink_or_warn(lock->lk->filename);
+		res = unlink_or_msg(lock->lk->filename, err);
 		lock->lk->filename[i] = '.';
-		if (err && errno != ENOENT)
+		if (res)
 			return 1;
 	}
 	return 0;
@@ -3604,7 +3604,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			ret |= delete_ref_loose(update->lock, update->type);
+			ret |= delete_ref_loose(update->lock, update->type,
+						err);
 			if (!(update->flags & REF_ISPRUNING))
 				delnames[delnum++] = update->lock->ref_name;
 		}
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 05/19] refs.c: pass the ref log message to _create/delete/update instead of _commit

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 12:22:42 -0700

Change the reference transactions so that we pass the reflog message
through to the create/delete/update function instead of the commit message.
This allows for individual messages for each change in a multi ref
transaction.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 branch.c               |  4 ++--
 builtin/commit.c       |  4 ++--
 builtin/receive-pack.c |  5 +++--
 builtin/replace.c      |  5 +++--
 builtin/tag.c          |  4 ++--
 builtin/update-ref.c   | 13 +++++++------
 fast-import.c          |  8 ++++----
 refs.c                 | 34 +++++++++++++++++++++-------------
 refs.h                 |  8 ++++----
 sequencer.c            |  4 ++--
 walker.c               |  5 ++---
 11 files changed, 52 insertions(+), 42 deletions(-)

diff --git a/branch.c b/branch.c
index 37ac555..76a8ec9 100644
--- a/branch.c
+++ b/branch.c
@@ -301,8 +301,8 @@ void create_branch(const char *head,
 		transaction = ref_transaction_begin(&err);
 		if (!transaction ||
 		    ref_transaction_update(transaction, ref.buf, sha1,
-					   null_sha1, 0, !forcing, &err) ||
-		    ref_transaction_commit(transaction, msg, &err))
+					   null_sha1, 0, !forcing, msg, &err) ||
+		    ref_transaction_commit(transaction, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
 		strbuf_release(&err);
diff --git a/builtin/commit.c b/builtin/commit.c
index 9bf1003..d23e876 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1767,8 +1767,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	    ref_transaction_update(transaction, "HEAD", sha1,
 				   current_head
 				   ? current_head->object.sha1 : NULL,
-				   0, !!current_head, &err) ||
-	    ref_transaction_commit(transaction, sb.buf, &err)) {
+				   0, !!current_head, sb.buf, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		rollback_index_files();
 		die("%s", err.buf);
 	}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 224fadc..d1f4cf7 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -585,8 +585,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		transaction = ref_transaction_begin(&err);
 		if (!transaction ||
 		    ref_transaction_update(transaction, namespaced_name,
-					   new_sha1, old_sha1, 0, 1, &err) ||
-		    ref_transaction_commit(transaction, "push", &err)) {
+					   new_sha1, old_sha1, 0, 1, "push",
+					   &err) ||
+		    ref_transaction_commit(transaction, &err)) {
 			ref_transaction_free(transaction);
 
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index 1fcd06d..9d03b84 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -169,8 +169,9 @@ static int replace_object_sha1(const char *object_ref,
 
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref, repl, prev, 0, 1, &err) ||
-	    ref_transaction_commit(transaction, NULL, &err))
+	    ref_transaction_update(transaction, ref, repl, prev,
+				   0, 1, NULL, &err) ||
+	    ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 
 	ref_transaction_free(transaction);
diff --git a/builtin/tag.c b/builtin/tag.c
index f3f172f..70d28c5 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -705,8 +705,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, object, prev,
-				   0, 1, &err) ||
-	    ref_transaction_commit(transaction, NULL, &err))
+				   0, 1, NULL, &err) ||
+	    ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
 	if (force && !is_null_sha1(prev) && hashcmp(prev, object))
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 54a48c0..6c9be05 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -14,6 +14,7 @@ static const char * const git_update_ref_usage[] = {
 
 static char line_termination = '\n';
 static int update_flags;
+static const char *msg;
 
 /*
  * Parse one whitespace- or NUL-terminated, possibly C-quoted argument
@@ -198,7 +199,7 @@ static const char *parse_cmd_update(struct ref_transaction *transaction,
 		die("update %s: extra input: %s", refname, next);
 
 	if (ref_transaction_update(transaction, refname, new_sha1, old_sha1,
-				   update_flags, have_old, &err))
+				   update_flags, have_old, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -229,7 +230,7 @@ static const char *parse_cmd_create(struct ref_transaction *transaction,
 		die("create %s: extra input: %s", refname, next);
 
 	if (ref_transaction_create(transaction, refname, new_sha1,
-				   update_flags, &err))
+				   update_flags, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -264,7 +265,7 @@ static const char *parse_cmd_delete(struct ref_transaction *transaction,
 		die("delete %s: extra input: %s", refname, next);
 
 	if (ref_transaction_delete(transaction, refname, old_sha1,
-				   update_flags, have_old, &err))
+				   update_flags, have_old, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -300,7 +301,7 @@ static const char *parse_cmd_verify(struct ref_transaction *transaction,
 		die("verify %s: extra input: %s", refname, next);
 
 	if (ref_transaction_update(transaction, refname, new_sha1, old_sha1,
-				   update_flags, have_old, &err))
+				   update_flags, have_old, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -354,7 +355,7 @@ static void update_refs_stdin(struct ref_transaction *transaction)
 
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
-	const char *refname, *oldval, *msg = NULL;
+	const char *refname, *oldval;
 	unsigned char sha1[20], oldsha1[20];
 	int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;
 	struct option options[] = {
@@ -385,7 +386,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		if (end_null)
 			line_termination = '\0';
 		update_refs_stdin(transaction);
-		if (ref_transaction_commit(transaction, msg, &err))
+		if (ref_transaction_commit(transaction, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
 		strbuf_release(&err);
diff --git a/fast-import.c b/fast-import.c
index e7f6e37..51dfaad 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1708,8 +1708,8 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, b->sha1, old_sha1,
-				   0, 1, &err) ||
-	    ref_transaction_commit(transaction, msg, &err)) {
+				   0, 1, msg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
 		strbuf_release(&err);
@@ -1749,12 +1749,12 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf, t->sha1,
-					   NULL, 0, 0, &err)) {
+					   NULL, 0, 0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
 	}
-	if (ref_transaction_commit(transaction, msg, &err))
+	if (ref_transaction_commit(transaction, &err))
 		failure |= error("%s", err.buf);
 
  cleanup:
diff --git a/refs.c b/refs.c
index 5609622..99a9b86 100644
--- a/refs.c
+++ b/refs.c
@@ -2396,8 +2396,8 @@ static void prune_ref(struct ref_to_prune *r)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, r->name, r->sha1,
-				   REF_ISPRUNING, 1, &err) ||
-	    ref_transaction_commit(transaction, NULL, &err)) {
+				   REF_ISPRUNING, 1, NULL, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
 		strbuf_release(&err);
@@ -2571,8 +2571,8 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, refname, sha1, delopt,
-				   sha1 && !is_null_sha1(sha1), &err) ||
-	    ref_transaction_commit(transaction, NULL, &err)) {
+				   sha1 && !is_null_sha1(sha1), NULL, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ref_transaction_free(transaction);
 		strbuf_release(&err);
@@ -3350,6 +3350,7 @@ struct ref_update {
 	int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
 	struct ref_lock *lock;
 	int type;
+	char *msg;
 	const char refname[FLEX_ARRAY];
 };
 
@@ -3392,9 +3393,10 @@ void ref_transaction_free(struct ref_transaction *transaction)
 	if (!transaction)
 		return;
 
-	for (i = 0; i < transaction->nr; i++)
+	for (i = 0; i < transaction->nr; i++) {
+		free(transaction->updates[i]->msg);
 		free(transaction->updates[i]);
-
+	}
 	free(transaction->updates);
 	free(transaction);
 }
@@ -3415,7 +3417,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err)
 {
 	struct ref_update *update;
@@ -3432,13 +3434,15 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	update->have_old = have_old;
 	if (have_old)
 		hashcpy(update->old_sha1, old_sha1);
+	if (msg)
+		update->msg = xstrdup(msg);
 	return 0;
 }
 
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
-			   int flags,
+			   int flags, const char *msg,
 			   struct strbuf *err)
 {
 	struct ref_update *update;
@@ -3455,13 +3459,15 @@ int ref_transaction_create(struct ref_transaction *transaction,
 	hashclr(update->old_sha1);
 	update->flags = flags;
 	update->have_old = 1;
+	if (msg)
+		update->msg = xstrdup(msg);
 	return 0;
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err)
 {
 	struct ref_update *update;
@@ -3479,6 +3485,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		assert(!is_null_sha1(old_sha1));
 		hashcpy(update->old_sha1, old_sha1);
 	}
+	if (msg)
+		update->msg = xstrdup(msg);
 	return 0;
 }
 
@@ -3492,8 +3500,8 @@ int update_ref(const char *action, const char *refname,
 	t = ref_transaction_begin(&err);
 	if (!t ||
 	    ref_transaction_update(t, refname, sha1, oldval, flags,
-				   !!oldval, &err) ||
-	    ref_transaction_commit(t, action, &err)) {
+				   !!oldval, action, &err) ||
+	    ref_transaction_commit(t, &err)) {
 		const char *str = "update_ref failed for ref '%s': %s";
 
 		ref_transaction_free(t);
@@ -3539,7 +3547,7 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,
 }
 
 int ref_transaction_commit(struct ref_transaction *transaction,
-			   const char *msg, struct strbuf *err)
+			   struct strbuf *err)
 {
 	int ret = 0, delnum = 0, i;
 	const char **delnames;
@@ -3588,7 +3596,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 
 		if (!is_null_sha1(update->new_sha1)) {
 			ret = write_ref_sha1(update->lock, update->new_sha1,
-					     msg);
+					     update->msg);
 			update->lock = NULL; /* freed by write_ref_sha1 */
 			if (ret) {
 				if (err)
diff --git a/refs.h b/refs.h
index 69ef28c..7c1bf95 100644
--- a/refs.h
+++ b/refs.h
@@ -289,7 +289,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err);
 
 /*
@@ -304,7 +304,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
-			   int flags,
+			   int flags, const char *msg,
 			   struct strbuf *err);
 
 /*
@@ -318,7 +318,7 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err);
 
 /*
@@ -327,7 +327,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
  * problem.
  */
 int ref_transaction_commit(struct ref_transaction *transaction,
-			   const char *msg, struct strbuf *err);
+			   struct strbuf *err);
 
 /*
  * Free an existing transaction and all associated data.
diff --git a/sequencer.c b/sequencer.c
index 5e93b6a..c5b7b8a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -286,8 +286,8 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn ? null_sha1 : from,
-				   0, 1, &err) ||
-	    ref_transaction_commit(transaction, sb.buf, &err)) {
+				   0, 1, sb.buf, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
 		strbuf_release(&sb);
diff --git a/walker.c b/walker.c
index b8a5441..3c6fb8a 100644
--- a/walker.c
+++ b/walker.c
@@ -298,14 +298,13 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
 					   &sha1[20 * i], NULL, 0, 0,
+					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
 			goto done;
 		}
 	}
-	if (ref_transaction_commit(transaction,
-				   msg ? msg : "fetch (unknown)",
-				   &err)) {
+	if (ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		goto done;
 	}
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 06/19] rename_ref: don't ask read_ref_full where the ref came from

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 12:41:04 -0700

We call read_ref_full with a pointer to flags from rename_ref but since
we never actually use the returned flags we can just pass NULL here instead.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 99a9b86..39571f5 100644
--- a/refs.c
+++ b/refs.c
@@ -2671,7 +2671,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 		goto rollback;
 	}
 
-	if (!read_ref_full(newrefname, sha1, 1, &flag) &&
+	if (!read_ref_full(newrefname, sha1, 1, NULL) &&
 	    delete_ref(newrefname, sha1, REF_NODEREF)) {
 		if (errno==EISDIR) {
 			if (remove_empty_directories(git_path("%s", newrefname))) {
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 07/19] refs.c: move the check for valid refname to lock_ref_sha1_basic

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 1 May 2014 10:40:10 -0700

Move the check for check_refname_format from lock_any_ref_for_update
to lock_ref_sha1_basic. At some later stage we will get rid of
lock_any_ref_for_update completely. This has no visible impact to callers,
except for inability to lock badly named refs which is not possible today
anyway but for other reasons.

Keep lock_any_ref_for_update as a no-op wrapper --- it is the public facing
version of this interface and keeping it as a separate function will make
it easier to experiment with the internal lock_ref_sha1_basic signature.

Note that if lock_ref_sha1_basic now checks the refname format and fails to
lock the ref it will not be possible to delete such a ref since deletion
implies we first lock the ref. In fact, we currently fail even earlier than
that since these refs are not even recognized to exist.

Example:
$ cp .git/refs/heads/master .git/refs/heads/echo...\*\*
$ ./git branch -D .git/refs/heads/echo...\*\*
error: branch '.git/refs/heads/echo...**' not found.

This is not a new regression and this has been broken for a while.
Later patches in the series will start repairing the handling of badly
named refs, at which time we will need to modify lock_ref_sha1_basic once more
in order to allow locking these refs for certain use cases such as rename
and delete.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index 39571f5..3c2ce57 100644
--- a/refs.c
+++ b/refs.c
@@ -2091,6 +2091,11 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	int missing = 0;
 	int attempts_remaining = 3;
 
+	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		errno = EINVAL;
+		return NULL;
+	}
+
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
@@ -2182,8 +2187,6 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
 {
-	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
-		return NULL;
 	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 08/19] refs.c: call lock_ref_sha1_basic directly from commit

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 1 May 2014 10:43:39 -0700

Skip using the lock_any_ref_for_update wrapper and call lock_ref_sha1_basic
directly from the commit function.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index 3c2ce57..f124c2b 100644
--- a/refs.c
+++ b/refs.c
@@ -3578,12 +3578,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	for (i = 0; i < n; i++) {
 		struct ref_update *update = updates[i];
 
-		update->lock = lock_any_ref_for_update(update->refname,
-						       (update->have_old ?
-							update->old_sha1 :
-							NULL),
-						       update->flags,
-						       &update->type);
+		update->lock = lock_ref_sha1_basic(update->refname,
+						   (update->have_old ?
+						    update->old_sha1 :
+						    NULL),
+						   update->flags,
+						   &update->type);
 		if (!update->lock) {
 			if (err)
 				strbuf_addf(err, "Cannot lock the ref '%s'.",
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 09/19] refs.c: pass a skip list to name_conflict_fn

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 1 May 2014 11:16:07 -0700

Allow passing a list of refs to skip checking to name_conflict_fn.
There are some conditions where we want to allow a temporary conflict and skip
checking those refs. For example if we have a transaction that
1, guarantees that m is a packed refs and there is no loose ref for m
2, the transaction will delete m from the packed ref
3, the transaction will create conflicting m/m

For this case we want to be able to lock and create m/m since we know that the
conflict is only transient. I.e. the conflict will be automatically resolved
by the transaction when it deletes m.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 41 ++++++++++++++++++++++++++++-------------
 1 file changed, 28 insertions(+), 13 deletions(-)

diff --git a/refs.c b/refs.c
index f124c2b..b63ab2f 100644
--- a/refs.c
+++ b/refs.c
@@ -801,14 +801,16 @@ static int names_conflict(const char *refname1, const char *refname2)
 
 struct name_conflict_cb {
 	const char *refname;
-	const char *oldrefname;
 	const char *conflicting_refname;
+	struct string_list *skiplist;
 };
 
 static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
 {
 	struct name_conflict_cb *data = (struct name_conflict_cb *)cb_data;
-	if (data->oldrefname && !strcmp(data->oldrefname, entry->name))
+
+	if (data->skiplist &&
+	    string_list_has_string(data->skiplist, entry->name))
 		return 0;
 	if (names_conflict(data->refname, entry->name)) {
 		data->conflicting_refname = entry->name;
@@ -822,15 +824,17 @@ static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
  * conflicting with the name of an existing reference in dir.  If
  * oldrefname is non-NULL, ignore potential conflicts with oldrefname
  * (e.g., because oldrefname is scheduled for deletion in the same
- * operation).
+ * operation). skiplist contains a list of refs we want to skip checking for
+ * conflicts with. skiplist must be sorted.
  */
-static int is_refname_available(const char *refname, const char *oldrefname,
-				struct ref_dir *dir)
+static int is_refname_available(const char *refname,
+				struct ref_dir *dir,
+				struct string_list *skiplist)
 {
 	struct name_conflict_cb data;
 	data.refname = refname;
-	data.oldrefname = oldrefname;
 	data.conflicting_refname = NULL;
+	data.skiplist = skiplist;
 
 	sort_ref_dir(dir);
 	if (do_for_each_entry_in_dir(dir, 0, name_conflict_fn, &data)) {
@@ -2080,6 +2084,7 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
  */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
+					    struct string_list *skiplist,
 					    int flags, int *type_p)
 {
 	char *ref_file;
@@ -2129,7 +2134,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	 * name is a proper prefix of our refname.
 	 */
 	if (missing &&
-	     !is_refname_available(refname, NULL, get_packed_refs(&ref_cache))) {
+	     !is_refname_available(refname, get_packed_refs(&ref_cache),
+				   skiplist)) {
 		last_errno = ENOTDIR;
 		goto error_return;
 	}
@@ -2187,7 +2193,7 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
 {
-	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
+	return lock_ref_sha1_basic(refname, old_sha1, NULL, flags, type_p);
 }
 
 /*
@@ -2648,6 +2654,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	struct stat loginfo;
 	int log = !lstat(git_path("logs/%s", oldrefname), &loginfo);
 	const char *symref = NULL;
+	struct string_list skiplist = STRING_LIST_INIT_NODUP;
 
 	if (log && S_ISLNK(loginfo.st_mode))
 		return error("reflog for %s is a symlink", oldrefname);
@@ -2659,11 +2666,18 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	if (!symref)
 		return error("refname %s not found", oldrefname);
 
-	if (!is_refname_available(newrefname, oldrefname, get_packed_refs(&ref_cache)))
+	string_list_insert(&skiplist, oldrefname);
+	if (!is_refname_available(newrefname, get_packed_refs(&ref_cache),
+				  &skiplist)) {
+		string_list_clear(&skiplist, 0);
 		return 1;
-
-	if (!is_refname_available(newrefname, oldrefname, get_loose_refs(&ref_cache)))
+	}
+	if (!is_refname_available(newrefname, get_loose_refs(&ref_cache),
+				  &skiplist)) {
+		string_list_clear(&skiplist, 0);
 		return 1;
+	}
+	string_list_clear(&skiplist, 0);
 
 	if (log && rename(git_path("logs/%s", oldrefname), git_path(TMP_RENAMED_LOG)))
 		return error("unable to move logfile logs/%s to "TMP_RENAMED_LOG": %s",
@@ -2692,7 +2706,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 
 	logmoved = log;
 
-	lock = lock_ref_sha1_basic(newrefname, NULL, 0, NULL);
+	lock = lock_ref_sha1_basic(newrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for update", newrefname);
 		goto rollback;
@@ -2707,7 +2721,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	return 0;
 
  rollback:
-	lock = lock_ref_sha1_basic(oldrefname, NULL, 0, NULL);
+	lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for rollback", oldrefname);
 		goto rollbacklog;
@@ -3582,6 +3596,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 						   (update->have_old ?
 						    update->old_sha1 :
 						    NULL),
+						   NULL,
 						   update->flags,
 						   &update->type);
 		if (!update->lock) {
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 10/19] refs.c: ref_transaction_commit: distinguish name conflicts from other errors

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Fri, 16 May 2014 14:14:38 -0700

In _commit, ENOTDIR can happen in the call to lock_ref_sha1_basic, either when
we lstat the new refname and it returns ENOTDIR or if the name checking
function reports that the same type of conflict happened. In both cases it
means that we can not create the new ref due to a name conflict.

Start defining specific return codes for _commit: assign -1 as a generic
error and UPDATE_REFS_NAME_CONFLICT (-2) as the error that refers to a name
conflict.

When "git fetch" is creating refs, name conflicts differ from other errors in
that they are likely to be resolved by running "git remote prune <remote>".
"git fetch" currently inspects errno to decide whether to give that advice.
Once it switches to the transaction API, it can check for
UPDATE_REFS_NAME_CONFLICT instead.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 18 ++++++++++++------
 refs.h |  5 +++++
 2 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index b63ab2f..86c708a 100644
--- a/refs.c
+++ b/refs.c
@@ -3584,9 +3584,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 
 	/* Copy, sort, and reject duplicate refs */
 	qsort(updates, n, sizeof(*updates), ref_update_compare);
-	ret = ref_update_reject_duplicates(updates, n, err);
-	if (ret)
+	if (ref_update_reject_duplicates(updates, n, err)) {
+		ret = -1;
 		goto cleanup;
+	}
 
 	/* Acquire all locks while verifying old values */
 	for (i = 0; i < n; i++) {
@@ -3600,10 +3601,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 						   update->flags,
 						   &update->type);
 		if (!update->lock) {
+			int df_conflict = (errno == ENOTDIR);
+
 			if (err)
 				strbuf_addf(err, "Cannot lock the ref '%s'.",
 					    update->refname);
-			ret = 1;
+			ret = df_conflict ? UPDATE_REFS_NAME_CONFLICT : -1;
 			goto cleanup;
 		}
 	}
@@ -3620,6 +3623,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 				if (err)
 					strbuf_addf(err, "Cannot update the ref '%s'.",
 						    update->refname);
+				ret = -1;
 				goto cleanup;
 			}
 		}
@@ -3630,14 +3634,16 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			ret |= delete_ref_loose(update->lock, update->type,
-						err);
+			if (delete_ref_loose(update->lock, update->type, err))
+				ret = -1;
+
 			if (!(update->flags & REF_ISPRUNING))
 				delnames[delnum++] = update->lock->ref_name;
 		}
 	}
 
-	ret |= repack_without_refs(delnames, delnum, err);
+	if (repack_without_refs(delnames, delnum, err))
+		ret = -1;
 	for (i = 0; i < delnum; i++)
 		unlink_or_warn(git_path("logs/%s", delnames[i]));
 	clear_loose_ref_cache(&ref_cache);
diff --git a/refs.h b/refs.h
index 7c1bf95..e14aa31 100644
--- a/refs.h
+++ b/refs.h
@@ -325,7 +325,12 @@ int ref_transaction_delete(struct ref_transaction *transaction,
  * Commit all of the changes that have been queued in transaction, as
  * atomically as possible.  Return a nonzero value if there is a
  * problem.
+ *
+ * Function returns 0 on success, -1 for generic failures and
+ * UPDATE_REFS_NAME_CONFLICT (-2) if the failure was due to a naming conflict.
+ * For example, the ref names A and A/B conflict.
  */
+#define UPDATE_REFS_NAME_CONFLICT -2
 int ref_transaction_commit(struct ref_transaction *transaction,
 			   struct strbuf *err);
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 11/19] fetch.c: change s_update_ref to use a ref transaction

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 13:49:07 -0700

Change s_update_ref to use a ref transaction for the ref update.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/fetch.c | 34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 55f457c..2e3bc73 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -375,23 +375,37 @@ static int s_update_ref(const char *action,
 {
 	char msg[1024];
 	char *rla = getenv("GIT_REFLOG_ACTION");
-	static struct ref_lock *lock;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret, df_conflict = 0;
 
 	if (dry_run)
 		return 0;
 	if (!rla)
 		rla = default_rla.buf;
 	snprintf(msg, sizeof(msg), "%s: %s", rla, action);
-	lock = lock_any_ref_for_update(ref->name,
-				       check_old ? ref->old_sha1 : NULL,
-				       0, NULL);
-	if (!lock)
-		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
-					  STORE_REF_ERROR_OTHER;
-	if (write_ref_sha1(lock, ref->new_sha1, msg) < 0)
-		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
-					  STORE_REF_ERROR_OTHER;
+
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref->name, ref->new_sha1,
+				   ref->old_sha1, 0, check_old, msg, &err))
+		goto fail;
+
+	ret = ref_transaction_commit(transaction, &err);
+	if (ret == UPDATE_REFS_NAME_CONFLICT)
+		df_conflict = 1;
+	if (ret)
+		goto fail;
+
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	return 0;
+fail:
+	ref_transaction_free(transaction);
+	error("%s", err.buf);
+	strbuf_release(&err);
+	return df_conflict ? STORE_REF_ERROR_DF_CONFLICT
+			   : STORE_REF_ERROR_OTHER;
 }
 
 #define REFCOL_WIDTH  10
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 12/19] refs.c: make write_ref_sha1 static

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 15:36:58 -0700

No external users call write_ref_sha1 any more so lets declare it static.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 10 ++++++++--
 refs.h |  3 ---
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index 86c708a..c2dab4a 100644
--- a/refs.c
+++ b/refs.c
@@ -2646,6 +2646,9 @@ static int rename_tmp_log(const char *newrefname)
 	return 0;
 }
 
+static int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1,
+			  const char *logmsg);
+
 int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)
 {
 	unsigned char sha1[20], orig_sha1[20];
@@ -2901,8 +2904,11 @@ static int is_branch(const char *refname)
 	return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
 }
 
-/* This function must return a meaningful errno */
-int write_ref_sha1(struct ref_lock *lock,
+/*
+ * Writes sha1 into the ref specified by the lock. Makes sure that errno
+ * is sane on error.
+ */
+static int write_ref_sha1(struct ref_lock *lock,
 	const unsigned char *sha1, const char *logmsg)
 {
 	static char term = '\n';
diff --git a/refs.h b/refs.h
index e14aa31..fafa493 100644
--- a/refs.h
+++ b/refs.h
@@ -195,9 +195,6 @@ extern int commit_ref(struct ref_lock *lock);
 /** Release any lock taken but not written. **/
 extern void unlock_ref(struct ref_lock *lock);
 
-/** Writes sha1 into the ref specified by the lock. **/
-extern int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1, const char *msg);
-
 /*
  * Setup reflog before using. Set errno to something meaningful on failure.
  */
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 13/19] refs.c: change resolve_ref_unsafe reading argument to be a flags field

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 15 Jul 2014 12:59:36 -0700

resolve_ref_unsafe takes a boolean argument for reading.
Change this to be a flags field instead and pass the new constant
RESOLVE_REF_READING when we want this behaviour.

Swap two of the arguments in the function to make sure that we capture any
instances where callers have not been updated/aware of the new api so that
they can be audited.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 branch.c                |  2 +-
 builtin/blame.c         |  2 +-
 builtin/branch.c        |  9 +++---
 builtin/checkout.c      |  6 ++--
 builtin/clone.c         |  2 +-
 builtin/commit.c        |  2 +-
 builtin/fmt-merge-msg.c |  2 +-
 builtin/for-each-ref.c  |  6 ++--
 builtin/fsck.c          |  2 +-
 builtin/log.c           |  3 +-
 builtin/merge.c         |  2 +-
 builtin/notes.c         |  2 +-
 builtin/receive-pack.c  |  4 +--
 builtin/remote.c        |  5 ++--
 builtin/show-branch.c   |  6 ++--
 builtin/symbolic-ref.c  |  2 +-
 bundle.c                |  2 +-
 cache.h                 | 21 ++++++-------
 http-backend.c          |  3 +-
 notes-merge.c           |  2 +-
 reflog-walk.c           |  5 ++--
 refs.c                  | 79 ++++++++++++++++++++++++++++---------------------
 remote.c                | 10 ++++---
 sequencer.c             |  4 +--
 transport-helper.c      |  4 ++-
 transport.c             |  5 ++--
 upload-pack.c           |  2 +-
 wt-status.c             |  2 +-
 28 files changed, 111 insertions(+), 85 deletions(-)

diff --git a/branch.c b/branch.c
index 76a8ec9..ba3e1c1 100644
--- a/branch.c
+++ b/branch.c
@@ -186,7 +186,7 @@ int validate_new_branchname(const char *name, struct strbuf *ref,
 		const char *head;
 		unsigned char sha1[20];
 
-		head = resolve_ref_unsafe("HEAD", sha1, 0, NULL);
+		head = resolve_ref_unsafe("HEAD", sha1, NULL, 0);
 		if (!is_bare_repository() && head && !strcmp(head, ref->buf))
 			die(_("Cannot force update the current branch."));
 	}
diff --git a/builtin/blame.c b/builtin/blame.c
index a52a279..b8bec0a 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -2292,7 +2292,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 	commit->object.type = OBJ_COMMIT;
 	parent_tail = &commit->parents;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, 1, NULL))
+	if (!resolve_ref_unsafe("HEAD", head_sha1, NULL, RESOLVE_REF_READING))
 		die("no such ref: HEAD");
 
 	parent_tail = append_parent(parent_tail, head_sha1);
diff --git a/builtin/branch.c b/builtin/branch.c
index 652b1d2..f144808 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -129,7 +129,8 @@ static int branch_merged(int kind, const char *name,
 		    branch->merge[0] &&
 		    branch->merge[0]->dst &&
 		    (reference_name = reference_name_to_free =
-		     resolve_refdup(branch->merge[0]->dst, sha1, 1, NULL)) != NULL)
+		     resolve_refdup(branch->merge[0]->dst, sha1,
+				    NULL, RESOLVE_REF_READING)) != NULL)
 			reference_rev = lookup_commit_reference(sha1);
 	}
 	if (!reference_rev)
@@ -233,7 +234,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, sha1, 0, &flags);
+		target = resolve_ref_unsafe(name, sha1, &flags, 0);
 		if (!target ||
 		    (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {
 			error(remote_branch
@@ -296,7 +297,7 @@ static char *resolve_symref(const char *src, const char *prefix)
 	int flag;
 	const char *dst, *cp;
 
-	dst = resolve_ref_unsafe(src, sha1, 0, &flag);
+	dst = resolve_ref_unsafe(src, sha1, &flag, 0);
 	if (!(dst && (flag & REF_ISSYMREF)))
 		return NULL;
 	if (prefix && (cp = skip_prefix(dst, prefix)))
@@ -862,7 +863,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	track = git_branch_track;
 
-	head = resolve_refdup("HEAD", head_sha1, 0, NULL);
+	head = resolve_refdup("HEAD", head_sha1, NULL, 0);
 	if (!head)
 		die(_("Failed to resolve HEAD as a valid ref."));
 	if (!strcmp(head, "HEAD")) {
diff --git a/builtin/checkout.c b/builtin/checkout.c
index f1dc56e..64af1f7 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -356,7 +356,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 	    commit_locked_index(lock_file))
 		die(_("unable to write new index file"));
 
-	read_ref_full("HEAD", rev, 0, &flag);
+	read_ref_full("HEAD", rev, &flag, 0);
 	head = lookup_commit_reference_gently(rev, 1);
 
 	errs |= post_checkout_hook(head, head, 0);
@@ -771,7 +771,7 @@ static int switch_branches(const struct checkout_opts *opts,
 	unsigned char rev[20];
 	int flag, writeout_error = 0;
 	memset(&old, 0, sizeof(old));
-	old.path = path_to_free = resolve_refdup("HEAD", rev, 0, &flag);
+	old.path = path_to_free = resolve_refdup("HEAD", rev, &flag, 0);
 	old.commit = lookup_commit_reference_gently(rev, 1);
 	if (!(flag & REF_ISSYMREF))
 		old.path = NULL;
@@ -1068,7 +1068,7 @@ static int checkout_branch(struct checkout_opts *opts,
 		unsigned char rev[20];
 		int flag;
 
-		if (!read_ref_full("HEAD", rev, 0, &flag) &&
+		if (!read_ref_full("HEAD", rev, &flag, 0) &&
 		    (flag & REF_ISSYMREF) && is_null_sha1(rev))
 			return switch_unborn_to_new_branch(opts);
 	}
diff --git a/builtin/clone.c b/builtin/clone.c
index b12989d..12a78e1 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -622,7 +622,7 @@ static int checkout(void)
 	if (option_no_checkout)
 		return 0;
 
-	head = resolve_refdup("HEAD", sha1, 1, NULL);
+	head = resolve_refdup("HEAD", sha1, NULL, RESOLVE_REF_READING);
 	if (!head) {
 		warning(_("remote HEAD refers to nonexistent ref, "
 			  "unable to checkout.\n"));
diff --git a/builtin/commit.c b/builtin/commit.c
index d23e876..3536330 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1468,7 +1468,7 @@ static void print_summary(const char *prefix, const unsigned char *sha1,
 	rev.diffopt.break_opt = 0;
 	diff_setup_done(&rev.diffopt);
 
-	head = resolve_ref_unsafe("HEAD", junk_sha1, 0, NULL);
+	head = resolve_ref_unsafe("HEAD", junk_sha1, NULL, 0);
 	printf("[%s%s ",
 		starts_with(head, "refs/heads/") ?
 			head + 11 :
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index 3906eda..b2355ad 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -602,7 +602,7 @@ int fmt_merge_msg(struct strbuf *in, struct strbuf *out,
 
 	/* get current branch */
 	current_branch = current_branch_to_free =
-		resolve_refdup("HEAD", head_sha1, 1, NULL);
+		resolve_refdup("HEAD", head_sha1, NULL, RESOLVE_REF_READING);
 	if (!current_branch)
 		die("No current branch");
 	if (starts_with(current_branch, "refs/heads/"))
diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c
index 3e1d5c3..e193073 100644
--- a/builtin/for-each-ref.c
+++ b/builtin/for-each-ref.c
@@ -649,7 +649,8 @@ static void populate_value(struct refinfo *ref)
 
 	if (need_symref && (ref->flag & REF_ISSYMREF) && !ref->symref) {
 		unsigned char unused1[20];
-		ref->symref = resolve_refdup(ref->refname, unused1, 1, NULL);
+		ref->symref = resolve_refdup(ref->refname, unused1,
+					     NULL, RESOLVE_REF_READING);
 		if (!ref->symref)
 			ref->symref = "";
 	}
@@ -707,7 +708,8 @@ static void populate_value(struct refinfo *ref)
 			const char *head;
 			unsigned char sha1[20];
 
-			head = resolve_ref_unsafe("HEAD", sha1, 1, NULL);
+			head = resolve_ref_unsafe("HEAD", sha1,
+						  NULL, RESOLVE_REF_READING);
 			if (!strcmp(ref->refname, head))
 				v->s = "*";
 			else
diff --git a/builtin/fsck.c b/builtin/fsck.c
index fc150c8..506e32b 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -560,7 +560,7 @@ static int fsck_head_link(void)
 	if (verbose)
 		fprintf(stderr, "Checking HEAD link\n");
 
-	head_points_at = resolve_ref_unsafe("HEAD", head_sha1, 0, &flag);
+	head_points_at = resolve_ref_unsafe("HEAD", head_sha1, &flag, 0);
 	if (!head_points_at)
 		return error("Invalid HEAD");
 	if (!strcmp(head_points_at, "HEAD"))
diff --git a/builtin/log.c b/builtin/log.c
index a7ba211..230a9ef 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1395,7 +1395,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (check_head) {
 			unsigned char sha1[20];
 			const char *ref;
-			ref = resolve_ref_unsafe("HEAD", sha1, 1, NULL);
+			ref = resolve_ref_unsafe("HEAD", sha1, NULL,
+						 RESOLVE_REF_READING);
 			if (ref && starts_with(ref, "refs/heads/"))
 				branch_name = xstrdup(ref + strlen("refs/heads/"));
 			else
diff --git a/builtin/merge.c b/builtin/merge.c
index 428ca24..d262279 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -1108,7 +1108,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
 	 * Check if we are _not_ on a detached HEAD, i.e. if there is a
 	 * current branch.
 	 */
-	branch = branch_to_free = resolve_refdup("HEAD", head_sha1, 0, &flag);
+	branch = branch_to_free = resolve_refdup("HEAD", head_sha1, &flag, 0);
 	if (branch && starts_with(branch, "refs/heads/"))
 		branch += 11;
 	if (!branch || is_null_sha1(head_sha1))
diff --git a/builtin/notes.c b/builtin/notes.c
index 820c341..16df78c 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -703,7 +703,7 @@ static int merge_commit(struct notes_merge_options *o)
 	init_notes(t, "NOTES_MERGE_PARTIAL", combine_notes_overwrite, 0);
 
 	o->local_ref = local_ref_to_free =
-		resolve_refdup("NOTES_MERGE_REF", sha1, 0, NULL);
+		resolve_refdup("NOTES_MERGE_REF", sha1, NULL, 0);
 	if (!o->local_ref)
 		die("Failed to resolve NOTES_MERGE_REF");
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index d1f4cf7..555a4a6 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -656,7 +656,7 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)
 	int flag;
 
 	strbuf_addf(&buf, "%s%s", get_git_namespace(), cmd->ref_name);
-	dst_name = resolve_ref_unsafe(buf.buf, sha1, 0, &flag);
+	dst_name = resolve_ref_unsafe(buf.buf, sha1, &flag, 0);
 	strbuf_release(&buf);
 
 	if (!(flag & REF_ISSYMREF))
@@ -817,7 +817,7 @@ static void execute_commands(struct command *commands,
 	check_aliased_updates(commands);
 
 	free(head_name_to_free);
-	head_name = head_name_to_free = resolve_refdup("HEAD", sha1, 0, NULL);
+	head_name = head_name_to_free = resolve_refdup("HEAD", sha1, NULL, 0);
 
 	checked_connectivity = 1;
 	for (cmd = commands; cmd; cmd = cmd->next) {
diff --git a/builtin/remote.c b/builtin/remote.c
index 401feb3..6eaeee7 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -568,7 +568,8 @@ static int read_remote_branches(const char *refname,
 	strbuf_addf(&buf, "refs/remotes/%s/", rename->old);
 	if (starts_with(refname, buf.buf)) {
 		item = string_list_append(rename->remote_branches, xstrdup(refname));
-		symref = resolve_ref_unsafe(refname, orig_sha1, 1, &flag);
+		symref = resolve_ref_unsafe(refname, orig_sha1, &flag,
+					    RESOLVE_REF_READING);
 		if (flag & REF_ISSYMREF)
 			item->util = xstrdup(symref);
 		else
@@ -704,7 +705,7 @@ static int mv(int argc, const char **argv)
 		int flag = 0;
 		unsigned char sha1[20];
 
-		read_ref_full(item->string, sha1, 1, &flag);
+		read_ref_full(item->string, sha1, &flag, RESOLVE_REF_READING);
 		if (!(flag & REF_ISSYMREF))
 			continue;
 		if (delete_ref(item->string, NULL, REF_NODEREF))
diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index d873172..ef6ea52 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -727,7 +727,8 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		if (ac == 0) {
 			static const char *fake_av[2];
 
-			fake_av[0] = resolve_refdup("HEAD", sha1, 1, NULL);
+			fake_av[0] = resolve_refdup("HEAD", sha1, NULL,
+						    RESOLVE_REF_READING);
 			fake_av[1] = NULL;
 			av = fake_av;
 			ac = 1;
@@ -789,7 +790,8 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		}
 	}
 
-	head_p = resolve_ref_unsafe("HEAD", head_sha1, 1, NULL);
+	head_p = resolve_ref_unsafe("HEAD", head_sha1, NULL,
+				    RESOLVE_REF_READING);
 	if (head_p) {
 		head_len = strlen(head_p);
 		memcpy(head, head_p, head_len + 1);
diff --git a/builtin/symbolic-ref.c b/builtin/symbolic-ref.c
index b6a711d..1dd04af 100644
--- a/builtin/symbolic-ref.c
+++ b/builtin/symbolic-ref.c
@@ -13,7 +13,7 @@ static int check_symref(const char *HEAD, int quiet, int shorten, int print)
 {
 	unsigned char sha1[20];
 	int flag;
-	const char *refname = resolve_ref_unsafe(HEAD, sha1, 0, &flag);
+	const char *refname = resolve_ref_unsafe(HEAD, sha1, &flag, 0);
 
 	if (!refname)
 		die("No such ref: %s", HEAD);
diff --git a/bundle.c b/bundle.c
index 1222952..32dd2f7 100644
--- a/bundle.c
+++ b/bundle.c
@@ -311,7 +311,7 @@ int create_bundle(struct bundle_header *header, const char *path,
 			continue;
 		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
 			continue;
-		if (read_ref_full(e->name, sha1, 1, &flag))
+		if (read_ref_full(e->name, sha1, &flag, RESOLVE_REF_READING))
 			flag = 0;
 		display_ref = (flag & REF_ISSYMREF) ? e->name : ref;
 
diff --git a/cache.h b/cache.h
index e7ec626..2d3c5ed 100644
--- a/cache.h
+++ b/cache.h
@@ -948,7 +948,7 @@ extern int get_sha1_hex(const char *hex, unsigned char *sha1);
 
 extern char *sha1_to_hex(const unsigned char *sha1);	/* static buffer result! */
 extern int read_ref_full(const char *refname, unsigned char *sha1,
-			 int reading, int *flags);
+			 int *flags, int resolve_flags);
 extern int read_ref(const char *refname, unsigned char *sha1);
 
 /*
@@ -960,20 +960,20 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  * or the input ref.
  *
  * If the reference cannot be resolved to an object, the behavior
- * depends on the "reading" argument:
+ * depends on the RESOLVE_REF_READING flag:
  *
- * - If reading is set, return NULL.
+ * - If RESOLVE_REF_READING is set, return NULL.
  *
- * - If reading is not set, clear sha1 and return the name of the last
- *   reference name in the chain, which will either be a non-symbolic
+ * - If RESOLVE_REF_READING is not set, clear sha1 and return the name of
+ *   the last reference name in the chain, which will either be a non-symbolic
  *   reference or an undefined reference.  If this is a prelude to
  *   "writing" to the ref, the return value is the name of the ref
  *   that will actually be created or changed.
  *
- * If flag is non-NULL, set the value that it points to the
+ * If flags is non-NULL, set the value that it points to the
  * combination of REF_ISPACKED (if the reference was found among the
- * packed references) and REF_ISSYMREF (if the initial reference was a
- * symbolic reference).
+ * packed references), REF_ISSYMREF (if the initial reference was a
+ * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
  *
  * If ref is not a properly-formatted, normalized reference, return
  * NULL.  If more than MAXDEPTH recursive symbolic lookups are needed,
@@ -981,8 +981,9 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  *
  * errno is set to something meaningful on error.
  */
-extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int reading, int *flag);
-extern char *resolve_refdup(const char *ref, unsigned char *sha1, int reading, int *flag);
+#define RESOLVE_REF_READING 0x01
+extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
+extern char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
 
 extern int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref);
 extern int dwim_log(const char *str, int len, unsigned char *sha1, char **ref);
diff --git a/http-backend.c b/http-backend.c
index d2c0a62..8f94f9b 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -417,7 +417,8 @@ static int show_head_ref(const char *refname, const unsigned char *sha1,
 
 	if (flag & REF_ISSYMREF) {
 		unsigned char unused[20];
-		const char *target = resolve_ref_unsafe(refname, unused, 1, NULL);
+		const char *target = resolve_ref_unsafe(refname, unused,
+						NULL, RESOLVE_REF_READING);
 		const char *target_nons = strip_namespace(target);
 
 		strbuf_addf(buf, "ref: %s\n", target_nons);
diff --git a/notes-merge.c b/notes-merge.c
index 94a1a8a..ffca602 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -549,7 +549,7 @@ int notes_merge(struct notes_merge_options *o,
 	       o->local_ref, o->remote_ref);
 
 	/* Dereference o->local_ref into local_sha1 */
-	if (read_ref_full(o->local_ref, local_sha1, 0, NULL))
+	if (read_ref_full(o->local_ref, local_sha1, NULL, 0))
 		die("Failed to resolve local notes ref '%s'", o->local_ref);
 	else if (!check_refname_format(o->local_ref, 0) &&
 		is_null_sha1(local_sha1))
diff --git a/reflog-walk.c b/reflog-walk.c
index 9ce8b53..feeaf0a 100644
--- a/reflog-walk.c
+++ b/reflog-walk.c
@@ -48,7 +48,8 @@ static struct complete_reflogs *read_complete_reflog(const char *ref)
 		unsigned char sha1[20];
 		const char *name;
 		void *name_to_free;
-		name = name_to_free = resolve_refdup(ref, sha1, 1, NULL);
+		name = name_to_free = resolve_refdup(ref, sha1, NULL,
+						     RESOLVE_REF_READING);
 		if (name) {
 			for_each_reflog_ent(name, read_one_reflog, reflogs);
 			free(name_to_free);
@@ -174,7 +175,7 @@ int add_reflog_for_walk(struct reflog_walk_info *info,
 		if (*branch == '\0') {
 			unsigned char sha1[20];
 			free(branch);
-			branch = resolve_refdup("HEAD", sha1, 0, NULL);
+			branch = resolve_refdup("HEAD", sha1, NULL, 0);
 			if (!branch)
 				die ("No current branch");
 
diff --git a/refs.c b/refs.c
index c2dab4a..a5f9734 100644
--- a/refs.c
+++ b/refs.c
@@ -1192,7 +1192,8 @@ static void read_loose_refs(const char *dirname, struct ref_dir *dir)
 					hashclr(sha1);
 					flag |= REF_ISBROKEN;
 				}
-			} else if (read_ref_full(refname.buf, sha1, 1, &flag)) {
+			} else if (read_ref_full(refname.buf, sha1, &flag,
+						 RESOLVE_REF_READING)) {
 				hashclr(sha1);
 				flag |= REF_ISBROKEN;
 			}
@@ -1344,21 +1345,20 @@ static const char *handle_missing_loose_ref(const char *refname,
 }
 
 /* This function needs to return a meaningful errno on failure */
-const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int reading, int *flag)
+const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *flags, int resolve_flags)
 {
 	int depth = MAXDEPTH;
 	ssize_t len;
 	char buffer[256];
 	static char refname_buffer[256];
 
-	if (flag)
-		*flag = 0;
+	if (flags)
+		*flags = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		errno = EINVAL;
 		return NULL;
 	}
-
 	for (;;) {
 		char path[PATH_MAX];
 		struct stat st;
@@ -1385,7 +1385,8 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 		if (lstat(path, &st) < 0) {
 			if (errno == ENOENT)
 				return handle_missing_loose_ref(refname, sha1,
-								reading, flag);
+					resolve_flags & RESOLVE_REF_READING,
+					flags);
 			else
 				return NULL;
 		}
@@ -1405,8 +1406,8 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 					!check_refname_format(buffer, 0)) {
 				strcpy(refname_buffer, buffer);
 				refname = refname_buffer;
-				if (flag)
-					*flag |= REF_ISSYMREF;
+				if (flags)
+					*flags |= REF_ISSYMREF;
 				continue;
 			}
 		}
@@ -1451,21 +1452,21 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 			 */
 			if (get_sha1_hex(buffer, sha1) ||
 			    (buffer[40] != '\0' && !isspace(buffer[40]))) {
-				if (flag)
-					*flag |= REF_ISBROKEN;
+				if (flags)
+					*flags |= REF_ISBROKEN;
 				errno = EINVAL;
 				return NULL;
 			}
 			return refname;
 		}
-		if (flag)
-			*flag |= REF_ISSYMREF;
+		if (flags)
+			*flags |= REF_ISSYMREF;
 		buf = buffer + 4;
 		while (isspace(*buf))
 			buf++;
 		if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
-			if (flag)
-				*flag |= REF_ISBROKEN;
+			if (flags)
+				*flags |= REF_ISBROKEN;
 			errno = EINVAL;
 			return NULL;
 		}
@@ -1473,9 +1474,9 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 	}
 }
 
-char *resolve_refdup(const char *ref, unsigned char *sha1, int reading, int *flag)
+char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags)
 {
-	const char *ret = resolve_ref_unsafe(ref, sha1, reading, flag);
+	const char *ret = resolve_ref_unsafe(ref, sha1, flags, resolve_flags);
 	return ret ? xstrdup(ret) : NULL;
 }
 
@@ -1486,22 +1487,22 @@ struct ref_filter {
 	void *cb_data;
 };
 
-int read_ref_full(const char *refname, unsigned char *sha1, int reading, int *flags)
+int read_ref_full(const char *refname, unsigned char *sha1, int *flags, int resolve_flags)
 {
-	if (resolve_ref_unsafe(refname, sha1, reading, flags))
+	if (resolve_ref_unsafe(refname, sha1, flags, resolve_flags))
 		return 0;
 	return -1;
 }
 
 int read_ref(const char *refname, unsigned char *sha1)
 {
-	return read_ref_full(refname, sha1, 1, NULL);
+	return read_ref_full(refname, sha1, NULL, RESOLVE_REF_READING);
 }
 
 int ref_exists(const char *refname)
 {
 	unsigned char sha1[20];
-	return !!resolve_ref_unsafe(refname, sha1, 1, NULL);
+	return !!resolve_ref_unsafe(refname, sha1, NULL, RESOLVE_REF_READING);
 }
 
 static int filter_refs(const char *refname, const unsigned char *sha1, int flags,
@@ -1615,7 +1616,7 @@ int peel_ref(const char *refname, unsigned char *sha1)
 		return 0;
 	}
 
-	if (read_ref_full(refname, base, 1, &flag))
+	if (read_ref_full(refname, base, &flag, RESOLVE_REF_READING))
 		return -1;
 
 	/*
@@ -1656,7 +1657,7 @@ static int warn_if_dangling_symref(const char *refname, const unsigned char *sha
 	if (!(flags & REF_ISSYMREF))
 		return 0;
 
-	resolves_to = resolve_ref_unsafe(refname, junk, 0, NULL);
+	resolves_to = resolve_ref_unsafe(refname, junk, NULL, 0);
 	if (!resolves_to
 	    || (d->refname
 		? strcmp(resolves_to, d->refname)
@@ -1781,7 +1782,7 @@ static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
 		return 0;
 	}
 
-	if (!read_ref_full("HEAD", sha1, 1, &flag))
+	if (!read_ref_full("HEAD", sha1, &flag, RESOLVE_REF_READING))
 		return fn("HEAD", sha1, flag, cb_data);
 
 	return 0;
@@ -1861,7 +1862,7 @@ int head_ref_namespaced(each_ref_fn fn, void *cb_data)
 	int flag;
 
 	strbuf_addf(&buf, "%sHEAD", get_git_namespace());
-	if (!read_ref_full(buf.buf, sha1, 1, &flag))
+	if (!read_ref_full(buf.buf, sha1, &flag, RESOLVE_REF_READING))
 		ret = fn(buf.buf, sha1, flag, cb_data);
 	strbuf_release(&buf);
 
@@ -1956,7 +1957,8 @@ int refname_match(const char *abbrev_name, const char *full_name)
 static struct ref_lock *verify_lock(struct ref_lock *lock,
 	const unsigned char *old_sha1, int mustexist)
 {
-	if (read_ref_full(lock->ref_name, lock->old_sha1, mustexist, NULL)) {
+	if (read_ref_full(lock->ref_name, lock->old_sha1, NULL,
+			  mustexist ? RESOLVE_REF_READING : 0)) {
 		int save_errno = errno;
 		error("Can't verify ref %s", lock->ref_name);
 		unlock_ref(lock);
@@ -2029,7 +2031,8 @@ int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
 
 		this_result = refs_found ? sha1_from_ref : sha1;
 		mksnpath(fullref, sizeof(fullref), *p, len, str);
-		r = resolve_ref_unsafe(fullref, this_result, 1, &flag);
+		r = resolve_ref_unsafe(fullref, this_result, &flag,
+				       RESOLVE_REF_READING);
 		if (r) {
 			if (!refs_found++)
 				*ref = xstrdup(r);
@@ -2058,7 +2061,7 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 		const char *ref, *it;
 
 		mksnpath(path, sizeof(path), *p, len, str);
-		ref = resolve_ref_unsafe(path, hash, 1, NULL);
+		ref = resolve_ref_unsafe(path, hash, NULL, RESOLVE_REF_READING);
 		if (!ref)
 			continue;
 		if (reflog_exists(path))
@@ -2093,6 +2096,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	int last_errno = 0;
 	int type, lflags;
 	int mustexist = (old_sha1 && !is_null_sha1(old_sha1));
+	int resolve_flags = 0;
 	int missing = 0;
 	int attempts_remaining = 3;
 
@@ -2104,7 +2108,11 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
-	refname = resolve_ref_unsafe(refname, lock->old_sha1, mustexist, &type);
+	if (mustexist)
+		resolve_flags |= RESOLVE_REF_READING;
+
+	refname = resolve_ref_unsafe(refname, lock->old_sha1, &type,
+				     resolve_flags);
 	if (!refname && errno == EISDIR) {
 		/* we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -2117,7 +2125,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 			error("there are still refs under '%s'", orig_refname);
 			goto error_return;
 		}
-		refname = resolve_ref_unsafe(orig_refname, lock->old_sha1, mustexist, &type);
+		refname = resolve_ref_unsafe(orig_refname, lock->old_sha1,
+					     &type, resolve_flags);
 	}
 	if (type_p)
 	    *type_p = type;
@@ -2470,7 +2479,7 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 		unsigned char sha1[20];
 		int flags;
 
-		if (read_ref_full(entry->name, sha1, 0, &flags))
+		if (read_ref_full(entry->name, sha1, &flags, 0))
 			/* We should at least have found the packed ref. */
 			die("Internal error");
 		if ((flags & REF_ISSYMREF) || !(flags & REF_ISPACKED)) {
@@ -2662,7 +2671,8 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	if (log && S_ISLNK(loginfo.st_mode))
 		return error("reflog for %s is a symlink", oldrefname);
 
-	symref = resolve_ref_unsafe(oldrefname, orig_sha1, 1, &flag);
+	symref = resolve_ref_unsafe(oldrefname, orig_sha1, &flag,
+				    RESOLVE_REF_READING);
 	if (flag & REF_ISSYMREF)
 		return error("refname %s is a symbolic ref, renaming it is not supported",
 			oldrefname);
@@ -2691,7 +2701,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 		goto rollback;
 	}
 
-	if (!read_ref_full(newrefname, sha1, 1, NULL) &&
+	if (!read_ref_full(newrefname, sha1, NULL, RESOLVE_REF_READING) &&
 	    delete_ref(newrefname, sha1, REF_NODEREF)) {
 		if (errno==EISDIR) {
 			if (remove_empty_directories(git_path("%s", newrefname))) {
@@ -2969,7 +2979,8 @@ static int write_ref_sha1(struct ref_lock *lock,
 		unsigned char head_sha1[20];
 		int head_flag;
 		const char *head_ref;
-		head_ref = resolve_ref_unsafe("HEAD", head_sha1, 1, &head_flag);
+		head_ref = resolve_ref_unsafe("HEAD", head_sha1, &head_flag,
+					      RESOLVE_REF_READING);
 		if (head_ref && (head_flag & REF_ISSYMREF) &&
 		    !strcmp(head_ref, lock->ref_name))
 			log_ref_write("HEAD", lock->old_sha1, sha1, logmsg);
@@ -3336,7 +3347,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data
 				retval = do_for_each_reflog(name, fn, cb_data);
 			} else {
 				unsigned char sha1[20];
-				if (read_ref_full(name->buf, sha1, 0, NULL))
+				if (read_ref_full(name->buf, sha1, NULL, 0))
 					retval = error("bad ref for %s", name->buf);
 				else
 					retval = fn(name->buf, sha1, 0, cb_data);
diff --git a/remote.c b/remote.c
index 0e9459c..67c375d 100644
--- a/remote.c
+++ b/remote.c
@@ -486,7 +486,7 @@ static void read_config(void)
 		return;
 	default_remote_name = "origin";
 	current_branch = NULL;
-	head_ref = resolve_ref_unsafe("HEAD", sha1, 0, &flag);
+	head_ref = resolve_ref_unsafe("HEAD", sha1, &flag, 0);
 	if (head_ref && (flag & REF_ISSYMREF) &&
 	    starts_with(head_ref, "refs/heads/")) {
 		current_branch =
@@ -1121,7 +1121,8 @@ static char *guess_ref(const char *name, struct ref *peer)
 	struct strbuf buf = STRBUF_INIT;
 	unsigned char sha1[20];
 
-	const char *r = resolve_ref_unsafe(peer->name, sha1, 1, NULL);
+	const char *r = resolve_ref_unsafe(peer->name, sha1, NULL,
+					   RESOLVE_REF_READING);
 	if (!r)
 		return NULL;
 
@@ -1182,7 +1183,8 @@ static int match_explicit(struct ref *src, struct ref *dst,
 		unsigned char sha1[20];
 		int flag;
 
-		dst_value = resolve_ref_unsafe(matched_src->name, sha1, 1, &flag);
+		dst_value = resolve_ref_unsafe(matched_src->name, sha1, &flag,
+					       RESOLVE_REF_READING);
 		if (!dst_value ||
 		    ((flag & REF_ISSYMREF) &&
 		     !starts_with(dst_value, "refs/heads/")))
@@ -1656,7 +1658,7 @@ static int ignore_symref_update(const char *refname)
 	unsigned char sha1[20];
 	int flag;
 
-	if (!resolve_ref_unsafe(refname, sha1, 0, &flag))
+	if (!resolve_ref_unsafe(refname, sha1, &flag, 0))
 		return 0; /* non-existing refs are OK */
 	return (flag & REF_ISSYMREF);
 }
diff --git a/sequencer.c b/sequencer.c
index c5b7b8a..6a05ad4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -366,7 +366,7 @@ static int is_index_unchanged(void)
 	unsigned char head_sha1[20];
 	struct commit *head_commit;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, 1, NULL))
+	if (!resolve_ref_unsafe("HEAD", head_sha1, NULL, RESOLVE_REF_READING))
 		return error(_("Could not resolve HEAD commit\n"));
 
 	head_commit = lookup_commit(head_sha1);
@@ -912,7 +912,7 @@ static int rollback_single_pick(void)
 	if (!file_exists(git_path("CHERRY_PICK_HEAD")) &&
 	    !file_exists(git_path("REVERT_HEAD")))
 		return error(_("no cherry-pick or revert in progress"));
-	if (read_ref_full("HEAD", head_sha1, 0, NULL))
+	if (read_ref_full("HEAD", head_sha1, NULL, 0))
 		return error(_("cannot resolve HEAD"));
 	if (is_null_sha1(head_sha1))
 		return error(_("cannot abort from a branch yet to be born"));
diff --git a/transport-helper.c b/transport-helper.c
index 84c616f..8365441 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -889,7 +889,9 @@ static int push_refs_with_export(struct transport *transport,
 					int flag;
 
 					/* Follow symbolic refs (mainly for HEAD). */
-					name = resolve_ref_unsafe(ref->peer_ref->name, sha1, 1, &flag);
+					name = resolve_ref_unsafe(
+						 ref->peer_ref->name, sha1,
+						 &flag, RESOLVE_REF_READING);
 					if (!name || !(flag & REF_ISSYMREF))
 						name = ref->peer_ref->name;
 
diff --git a/transport.c b/transport.c
index 325f03e..3ba7bbf 100644
--- a/transport.c
+++ b/transport.c
@@ -168,7 +168,8 @@ static void set_upstreams(struct transport *transport, struct ref *refs,
 		/* Follow symbolic refs (mainly for HEAD). */
 		localname = ref->peer_ref->name;
 		remotename = ref->name;
-		tmp = resolve_ref_unsafe(localname, sha, 1, &flag);
+		tmp = resolve_ref_unsafe(localname, sha, &flag,
+					 RESOLVE_REF_READING);
 		if (tmp && flag & REF_ISSYMREF &&
 			starts_with(tmp, "refs/heads/"))
 			localname = tmp;
@@ -753,7 +754,7 @@ void transport_print_push_status(const char *dest, struct ref *refs,
 	unsigned char head_sha1[20];
 	char *head;
 
-	head = resolve_refdup("HEAD", head_sha1, 1, NULL);
+	head = resolve_refdup("HEAD", head_sha1, NULL, RESOLVE_REF_READING);
 
 	if (verbose) {
 		for (ref = refs; ref; ref = ref->next)
diff --git a/upload-pack.c b/upload-pack.c
index 01de944..3b51ccb 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -743,7 +743,7 @@ static int find_symref(const char *refname, const unsigned char *sha1, int flag,
 
 	if ((flag & REF_ISSYMREF) == 0)
 		return 0;
-	symref_target = resolve_ref_unsafe(refname, unused, 0, &flag);
+	symref_target = resolve_ref_unsafe(refname, unused, &flag, 0);
 	if (!symref_target || (flag & REF_ISSYMREF) == 0)
 		die("'%s' is a symref but it is not?", refname);
 	item = string_list_append(cb_data, refname);
diff --git a/wt-status.c b/wt-status.c
index 318a191..6819e2b 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -128,7 +128,7 @@ void wt_status_prepare(struct wt_status *s)
 	s->show_untracked_files = SHOW_NORMAL_UNTRACKED_FILES;
 	s->use_color = -1;
 	s->relative_paths = 1;
-	s->branch = resolve_refdup("HEAD", sha1, 0, NULL);
+	s->branch = resolve_refdup("HEAD", sha1, NULL, 0);
 	s->reference = "HEAD";
 	s->fp = stdout;
 	s->index_file = get_index_file();
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 14/19] branch -d: avoid repeated symref resolution

[Jonathan Nieder]

If a repository gets in a broken state with too much symref nesting,
it cannot be repaired with "git branch -d":

 $ git symbolic-ref refs/heads/nonsense refs/heads/nonsense
 $ git branch -d nonsense
 error: branch 'nonsense' not found.

Worse, "git update-ref --no-deref -d" doesn't work for such repairs
either:

 $ git update-ref -d refs/heads/nonsense
 error: unable to resolve reference refs/heads/nonsense: Too many levels of symbolic links

Fix both by teaching resolve_ref_unsafe a new RESOLVE_REF_NODEREF flag
and passing it when appropriate.

Callers can still read the value of a symref (for example to print a
message about it) with that flag set --- resolve_ref_unsafe will
resolve one level of symrefs and stop there.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/branch.c      |  3 ++-
 cache.h               |  1 +
 refs.c                | 10 ++++++++++
 t/t1400-update-ref.sh | 16 ++++++++++++++++
 t/t3200-branch.sh     |  9 +++++++++
 5 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index f144808..7925660 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -234,7 +234,8 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, sha1, &flags, 0);
+		target = resolve_ref_unsafe(name, sha1,
+					    &flags, RESOLVE_REF_NODEREF);
 		if (!target ||
 		    (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {
 			error(remote_branch
diff --git a/cache.h b/cache.h
index 2d3c5ed..03ade12 100644
--- a/cache.h
+++ b/cache.h
@@ -982,6 +982,7 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  * errno is set to something meaningful on error.
  */
 #define RESOLVE_REF_READING 0x01
+#define RESOLVE_REF_NODEREF 0x02
 extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
 extern char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
 
diff --git a/refs.c b/refs.c
index a5f9734..f11df33 100644
--- a/refs.c
+++ b/refs.c
@@ -1408,6 +1408,10 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 				refname = refname_buffer;
 				if (flags)
 					*flags |= REF_ISSYMREF;
+				if (resolve_flags & RESOLVE_REF_NODEREF) {
+					hashclr(sha1);
+					return refname;
+				}
 				continue;
 			}
 		}
@@ -1471,6 +1475,10 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 			return NULL;
 		}
 		refname = strcpy(refname_buffer, buf);
+		if (resolve_flags & RESOLVE_REF_NODEREF) {
+			hashclr(sha1);
+			return refname;
+		}
 	}
 }
 
@@ -2110,6 +2118,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 
 	if (mustexist)
 		resolve_flags |= RESOLVE_REF_READING;
+	if (flags & REF_NODEREF)
+		resolve_flags |= RESOLVE_REF_NODEREF;
 
 	refname = resolve_ref_unsafe(refname, lock->old_sha1, &type,
 				     resolve_flags);
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 0218e96..ff4607b 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -110,6 +110,22 @@ test_expect_success "delete symref without dereference when the referred ref is
 cp -f .git/HEAD.orig .git/HEAD
 git update-ref -d $m
 
+test_expect_success 'update-ref -d is not confused by self-reference' '
+	git symbolic-ref refs/heads/self refs/heads/self &&
+	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_path_is_file .git/refs/heads/self &&
+	test_must_fail git update-ref -d refs/heads/self &&
+	test_path_is_file .git/refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete self-reference' '
+	git symbolic-ref refs/heads/self refs/heads/self &&
+	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_path_is_file .git/refs/heads/self &&
+	git update-ref --no-deref -d refs/heads/self &&
+	test_path_is_missing .git/refs/heads/self
+'
+
 test_expect_success '(not) create HEAD with old sha1' "
 	test_must_fail git update-ref HEAD $A $B
 "
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index ac31b71..432921b 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -285,6 +285,15 @@ test_expect_success 'deleting a dangling symref' '
 	test_i18ncmp expect actual
 '
 
+test_expect_success 'deleting a self-referential symref' '
+	git symbolic-ref refs/heads/self-reference refs/heads/self-reference &&
+	test_path_is_file .git/refs/heads/self-reference &&
+	echo "Deleted branch self-reference (was refs/heads/self-reference)." >expect &&
+	git branch -d self-reference >actual &&
+	test_path_is_missing .git/refs/heads/self-reference &&
+	test_i18ncmp expect actual
+'
+
 test_expect_success 'renaming a symref is not allowed' '
 	git symbolic-ref refs/heads/master2 refs/heads/master &&
 	test_must_fail git branch -m master2 master3 &&
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 15/19] refs.c: fix handling of badly named refs

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 3 Sep 2014 11:45:43 -0700

We currently do not handle badly named refs well:

  $ cp .git/refs/heads/master .git/refs/heads/master.....@\*@\\.
  $ git branch
    fatal: Reference has invalid format: 'refs/heads/master.....@*@\.'
  $ git branch -D master.....@\*@\\.
    error: branch 'master.....@*@\.' not found.

Currently we can not really recover from a badly named ref with less than
manually deleting the .git/refs/heads/<refname> file. This will also help
if change the naming rules in the future. For example if we decide to remove
`, " and such from the set of valid characters.

The purpose of this change is to allow git branch --list to show these refs
and to allow git branch -d/-D and update-ref -d to delete them.
Other functions will continue to not handle these refs but can be changed in
later patches.

Introduce two new internal flags: RESOLVE_REF_ALLOW_BAD_NAME and REF_BADNAMEOK.

In resolving functions, refuse to resolve badly named refs unless the new
RESOLVE_REF_ALLOW_BAD_NAME flag is passed. For these cases, if the badly named
ref exists then flag it as REF_ISBROKEN and resolve it to nullsha1.

In locking functions, refuse to act on badly named refs unless the new
REF_BADNAMEOK flag is passed. This is currently used only from branch.c and
update-ref.c when deleting a ref.

Change the internal functions that read loose and packed refs to allow
badly named refs but flag them as REF_ISBROKEN just like unresolvable refs
are.

During ref iteration only include these refs during for_each_rawref but
none of the other iterators. I.e. just like unresolvable refs, flag
inclusion of these refs based on DO_FOR_EACH_INCLUDE_BROKEN during
iteration.

In the transaction API, always refuse to create or update badly named refs
and refuse to delete them unless REF_BADNAMEOK flag is passed.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/branch.c            | 13 ++++++----
 builtin/update-ref.c        |  3 ++-
 cache.h                     |  8 +++++-
 refs.c                      | 63 ++++++++++++++++++++++++++++++++++++++-------
 refs.h                      |  2 ++
 t/t1402-check-ref-format.sh | 48 ++++++++++++++++++++++++++++++++++
 6 files changed, 121 insertions(+), 16 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index 7925660..5d5bc56 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -234,10 +234,13 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, sha1,
-					    &flags, RESOLVE_REF_NODEREF);
+		target = resolve_ref_unsafe(name, sha1, &flags,
+					    RESOLVE_REF_READING
+					    | RESOLVE_REF_NODEREF
+					    | RESOLVE_REF_ALLOW_BAD_NAME);
 		if (!target ||
-		    (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {
+		    (!(flags & (REF_ISSYMREF|REF_ISBROKEN)) &&
+		     is_null_sha1(sha1))) {
 			error(remote_branch
 			      ? _("remote branch '%s' not found.")
 			      : _("branch '%s' not found."), bname.buf);
@@ -245,14 +248,14 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 			continue;
 		}
 
-		if (!(flags & REF_ISSYMREF) &&
+		if (!(flags & (REF_ISSYMREF|REF_ISBROKEN)) &&
 		    check_branch_commit(bname.buf, name, sha1, head_rev, kinds,
 					force)) {
 			ret = 1;
 			continue;
 		}
 
-		if (delete_ref(name, sha1, REF_NODEREF)) {
+		if (delete_ref(name, sha1, REF_NODEREF|REF_BADNAMEOK)) {
 			error(remote_branch
 			      ? _("Error deleting remote branch '%s'")
 			      : _("Error deleting branch '%s'"),
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 6c9be05..e379fdd 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -419,7 +419,8 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (no_deref)
 		flags = REF_NODEREF;
 	if (delete)
-		return delete_ref(refname, oldval ? oldsha1 : NULL, flags);
+		return delete_ref(refname, oldval ? oldsha1 : NULL,
+				  flags|REF_BADNAMEOK);
 	else
 		return update_ref(msg, refname, sha1, oldval ? oldsha1 : NULL,
 				  flags, UPDATE_REFS_DIE_ON_ERR);
diff --git a/cache.h b/cache.h
index 03ade12..03a6144 100644
--- a/cache.h
+++ b/cache.h
@@ -979,10 +979,16 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  * NULL.  If more than MAXDEPTH recursive symbolic lookups are needed,
  * give up and return NULL.
  *
- * errno is set to something meaningful on error.
+ * RESOLVE_REF_ALLOW_BAD_NAME disables most of the ref name checking except
+ * for names that are absolute paths or contain ".." components. For both
+ * these cases the function will return NULL and set errno to EINVAL.
+ * If the name is bad then the function will set the REF_ISBROKEN flag and
+ * return the name, if the ref exists, or NULL, if it does not.
+ * When this flag is set, any badly named refs will resolve to nullsha1.
  */
 #define RESOLVE_REF_READING 0x01
 #define RESOLVE_REF_NODEREF 0x02
+#define RESOLVE_REF_ALLOW_BAD_NAME 0x04
 extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
 extern char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
 
diff --git a/refs.c b/refs.c
index f11df33..6faf0c8 100644
--- a/refs.c
+++ b/refs.c
@@ -1060,7 +1060,13 @@ static void read_packed_refs(FILE *f, struct ref_dir *dir)
 
 		refname = parse_ref_line(refline, sha1);
 		if (refname) {
-			last = create_ref_entry(refname, sha1, REF_ISPACKED, 1);
+			int flag = REF_ISPACKED;
+
+			if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT)) {
+				flag |= REF_ISBROKEN;
+				hashclr(sha1);
+			}
+			last = create_ref_entry(refname, sha1, flag, 0);
 			if (peeled == PEELED_FULLY ||
 			    (peeled == PEELED_TAGS && starts_with(refname, "refs/tags/")))
 				last->flag |= REF_KNOWS_PEELED;
@@ -1197,8 +1203,14 @@ static void read_loose_refs(const char *dirname, struct ref_dir *dir)
 				hashclr(sha1);
 				flag |= REF_ISBROKEN;
 			}
+			if (check_refname_format(refname.buf,
+						 REFNAME_ALLOW_ONELEVEL|
+						 REFNAME_DOT_COMPONENT)) {
+				hashclr(sha1);
+				flag |= REF_ISBROKEN;
+			}
 			add_entry_to_dir(dir,
-					 create_ref_entry(refname.buf, sha1, flag, 1));
+					 create_ref_entry(refname.buf, sha1, flag, 0));
 		}
 		strbuf_setlen(&refname, dirnamelen);
 	}
@@ -1344,6 +1356,18 @@ static const char *handle_missing_loose_ref(const char *refname,
 	}
 }
 
+static int escapes_cwd(const char *path) {
+	char *buf;
+	int result;
+
+	if (is_absolute_path(path))
+		return 1;
+	buf = xmalloc(strlen(path) + 1);
+	result = normalize_path_copy(buf, path);
+	free(buf);
+	return result;
+}
+
 /* This function needs to return a meaningful errno on failure */
 const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *flags, int resolve_flags)
 {
@@ -1351,13 +1375,21 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 	ssize_t len;
 	char buffer[256];
 	static char refname_buffer[256];
+	int bad_name = 0;
 
 	if (flags)
 		*flags = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
-		errno = EINVAL;
-		return NULL;
+		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+		    escapes_cwd(refname)) {
+			errno = EINVAL;
+			return NULL;
+		}
+		hashclr(sha1);
+		if (flags)
+			*flags |= REF_ISBROKEN;
+		bad_name = 1;
 	}
 	for (;;) {
 		char path[PATH_MAX];
@@ -1461,6 +1493,8 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 				errno = EINVAL;
 				return NULL;
 			}
+			if (bad_name)
+				hashclr(sha1);
 			return refname;
 		}
 		if (flags)
@@ -2108,14 +2142,12 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	int missing = 0;
 	int attempts_remaining = 3;
 
-	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
-		errno = EINVAL;
-		return NULL;
-	}
-
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
+	if (flags & REF_BADNAMEOK)
+		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
+
 	if (mustexist)
 		resolve_flags |= RESOLVE_REF_READING;
 	if (flags & REF_NODEREF)
@@ -3472,6 +3504,13 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	if (have_old && !old_sha1)
 		die("BUG: have_old is true but old_sha1 is NULL");
 
+	if (!is_null_sha1(new_sha1) &&
+	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		strbuf_addf(err, "refusing to update ref with bad name %s",
+			    refname);
+		return -1;
+	}
+
 	update = add_update(transaction, refname);
 	hashcpy(update->new_sha1, new_sha1);
 	update->flags = flags;
@@ -3497,6 +3536,12 @@ int ref_transaction_create(struct ref_transaction *transaction,
 	if (!new_sha1 || is_null_sha1(new_sha1))
 		die("BUG: create ref with null new_sha1");
 
+	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		strbuf_addf(err, "refusing to create ref with bad name %s",
+			    refname);
+		return -1;
+	}
+
 	update = add_update(transaction, refname);
 
 	hashcpy(update->new_sha1, new_sha1);
diff --git a/refs.h b/refs.h
index fafa493..a96e174 100644
--- a/refs.h
+++ b/refs.h
@@ -175,10 +175,12 @@ extern int peel_ref(const char *refname, unsigned char *sha1);
  * ref_transaction_create(), etc.
  * REF_NODEREF: act on the ref directly, instead of dereferencing
  *              symbolic references.
+ * REF_BADNAMEOK: allow locking a ref that has a bad name.
  *
  * Flags >= 0x100 are reserved for internal use.
  */
 #define REF_NODEREF	0x01
+#define REF_BADNAMEOK	0x02
 /*
  * This function sets errno to something meaningful on failure.
  */
diff --git a/t/t1402-check-ref-format.sh b/t/t1402-check-ref-format.sh
index 1a5a5f3..058fa37 100755
--- a/t/t1402-check-ref-format.sh
+++ b/t/t1402-check-ref-format.sh
@@ -196,4 +196,52 @@ invalid_ref_normalized 'heads///foo.lock'
 invalid_ref_normalized 'foo.lock/bar'
 invalid_ref_normalized 'foo.lock///bar'
 
+test_expect_success 'git branch shows badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch >output &&
+	grep -e "broken...ref" output
+'
+
+test_expect_success 'git branch -d can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -d broken...ref &&
+	git branch >output &&
+	! grep -e "broken...ref" output
+'
+
+test_expect_success 'git branch -D can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -D broken...ref &&
+	git branch >output &&
+	! grep -e "broken...ref" output
+'
+
+test_expect_success 'git update-ref -d can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git update-ref -d refs/heads/broken...ref &&
+	git branch >output &&
+	! grep -e "broken...ref" output
+'
+
+test_expect_success 'git branch can not create a badly named ref' '
+	test_must_fail git branch broken...ref
+'
+
+test_expect_success 'git branch can not rename to a bad ref name' '
+	git branch goodref &&
+	test_must_fail git branch -m goodref broken...ref
+'
+
+test_expect_failure 'git branch can rename from a bad ref name' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -m broken...ref renamed &&
+	test_must_fail git rev-parse broken...ref &&
+	test_cmp_rev master renamed
+'
+
 test_done
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 16/19] for-each-ref.c: improve message before aborting on broken ref

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Fri, 5 Sep 2014 14:35:17 -0700

Print a warning message for any badly named refs we find in the repo.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/for-each-ref.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c
index e193073..090390c 100644
--- a/builtin/for-each-ref.c
+++ b/builtin/for-each-ref.c
@@ -853,6 +853,12 @@ static int grab_single_ref(const char *refname, const unsigned char *sha1, int f
 	struct refinfo *ref;
 	int cnt;
 
+	if ((flag & REF_ISBROKEN) &&
+	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		  warning("ignoring ref with broken name %s", refname);
+		  return 0;
+	}
+
 	if (*cb->grab_pattern) {
 		const char **pattern;
 		int namelen = strlen(refname);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 17/19] refs.c: do not permit err == NULL

[Jonathan Nieder]

Some functions that take a strbuf argument to append an error to treat
!err as an indication that the message should be suppressed (e.g.,
ref_update_reject_duplicates).  Others write the message to stderr on
!err (e.g., repack_without_refs).  Others crash (e.g.,
ref_transaction_update).

Some of these behaviors are for historical reasons and others were
accidents.  Luckily no callers pass err == NULL any more.  Simplify
by consistently requiring the strbuf argument.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 refs.c | 46 +++++++++++++++++++++++++++-------------------
 1 file changed, 27 insertions(+), 19 deletions(-)

diff --git a/refs.c b/refs.c
index 6faf0c8..550223c 100644
--- a/refs.c
+++ b/refs.c
@@ -2560,6 +2560,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 	struct string_list_item *ref_to_delete;
 	int i, ret, removed = 0;
 
+	assert(err);
+
 	/* Look for a packed ref */
 	for (i = 0; i < n; i++)
 		if (get_packed_ref(refnames[i]))
@@ -2570,13 +2572,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
-		if (err) {
-			unable_to_lock_message(git_path("packed-refs"), errno,
-					       err);
-			return -1;
-		}
-		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refnames[i]);
+		unable_to_lock_message(git_path("packed-refs"), errno, err);
+		return -1;
 	}
 	packed = get_packed_refs(&ref_cache);
 
@@ -2602,7 +2599,7 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 
 	/* Write what remains */
 	ret = commit_packed_refs();
-	if (ret && err)
+	if (ret)
 		strbuf_addf(err, "unable to overwrite old ref-pack file: %s",
 			    strerror(errno));
 	return ret;
@@ -2610,6 +2607,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 
 static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
 {
+	assert(err);
+
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
 		/* loose */
 		int res, i = strlen(lock->lk->filename) - 5; /* .lock */
@@ -3459,6 +3458,8 @@ struct ref_transaction {
 
 struct ref_transaction *ref_transaction_begin(struct strbuf *err)
 {
+	assert(err);
+
 	return xcalloc(1, sizeof(struct ref_transaction));
 }
 
@@ -3498,6 +3499,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: update called for transaction that is not open");
 
@@ -3530,6 +3533,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: create called for transaction that is not open");
 
@@ -3561,6 +3566,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: delete called for transaction that is not open");
 
@@ -3623,13 +3630,14 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,
 					struct strbuf *err)
 {
 	int i;
+
+	assert(err);
+
 	for (i = 1; i < n; i++)
 		if (!strcmp(updates[i - 1]->refname, updates[i]->refname)) {
-			const char *str =
-				"Multiple updates for ref '%s' not allowed.";
-			if (err)
-				strbuf_addf(err, str, updates[i]->refname);
-
+			strbuf_addf(err,
+				    "Multiple updates for ref '%s' not allowed.",
+				    updates[i]->refname);
 			return 1;
 		}
 	return 0;
@@ -3643,6 +3651,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	int n = transaction->nr;
 	struct ref_update **updates = transaction->updates;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: commit called for transaction that is not open");
 
@@ -3675,9 +3685,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		if (!update->lock) {
 			int df_conflict = (errno == ENOTDIR);
 
-			if (err)
-				strbuf_addf(err, "Cannot lock the ref '%s'.",
-					    update->refname);
+			strbuf_addf(err, "Cannot lock the ref '%s'.",
+				    update->refname);
 			ret = df_conflict ? UPDATE_REFS_NAME_CONFLICT : -1;
 			goto cleanup;
 		}
@@ -3692,9 +3701,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 					     update->msg);
 			update->lock = NULL; /* freed by write_ref_sha1 */
 			if (ret) {
-				if (err)
-					strbuf_addf(err, "Cannot update the ref '%s'.",
-						    update->refname);
+				strbuf_addf(err, "Cannot update the ref '%s'.",
+					    update->refname);
 				ret = -1;
 				goto cleanup;
 			}
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 18/19] lockfile: remove unable_to_lock_error

[Jonathan Nieder]

The former caller uses unable_to_lock_message now.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
 cache.h    |  1 -
 lockfile.c | 10 ----------
 2 files changed, 11 deletions(-)

diff --git a/cache.h b/cache.h
index 03a6144..995729f 100644
--- a/cache.h
+++ b/cache.h
@@ -558,7 +558,6 @@ struct lock_file {
 };
 #define LOCK_DIE_ON_ERROR 1
 #define LOCK_NODEREF 2
-extern int unable_to_lock_error(const char *path, int err);
 extern void unable_to_lock_message(const char *path, int err,
 				   struct strbuf *buf);
 extern NORETURN void unable_to_lock_index_die(const char *path, int err);
diff --git a/lockfile.c b/lockfile.c
index a921d77..dbd4101 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -176,16 +176,6 @@ void unable_to_lock_message(const char *path, int err, struct strbuf *buf)
 			    absolute_path(path), strerror(err));
 }
 
-int unable_to_lock_error(const char *path, int err)
-{
-	struct strbuf buf = STRBUF_INIT;
-
-	unable_to_lock_message(path, err, &buf);
-	error("%s", buf.buf);
-	strbuf_release(&buf);
-	return -1;
-}
-
 NORETURN void unable_to_lock_index_die(const char *path, int err)
 {
 	struct strbuf buf = STRBUF_INIT;
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 19/19] ref_transaction_commit: bail out on failure to remove a ref

[Jonathan Nieder]

When removal of a loose or packed ref fails, bail out instead of
trying to finish the transaction.  This way, a single error message
can be printed (instead of multiple messages being concatenated by
mistake) and the operator can try to solve the underlying problem
before there is a chance to muck things up even more.

In particular, when git fails to remove a ref, git goes on to try to
delete the reflog.  Exiting early lets us keep the reflog.

When git succeeds in deleting a ref A and fails to remove a ref B, it
goes on to try to delete both reflogs.  It would be better to just
remove the reflog for A, but that would be a more invasive change.
Failing early means we keep both reflogs, which puts the operator in a
good position to understand the problem and recover.

A long term goal is to avoid these problems altogether and roll back
the transaction on failure.  That kind of transactionality will have
to wait for a later series (the plan for which is to make all
destructive work happen in a single update of the packed-refs file).

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
Thanks for reading.

 refs.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index 550223c..3b27758 100644
--- a/refs.c
+++ b/refs.c
@@ -3714,16 +3714,20 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			if (delete_ref_loose(update->lock, update->type, err))
+			if (delete_ref_loose(update->lock, update->type, err)) {
 				ret = -1;
+				goto cleanup;
+			}
 
 			if (!(update->flags & REF_ISPRUNING))
 				delnames[delnum++] = update->lock->ref_name;
 		}
 	}
 
-	if (repack_without_refs(delnames, delnum, err))
+	if (repack_without_refs(delnames, delnum, err)) {
 		ret = -1;
+		goto cleanup;
+	}
 	for (i = 0; i < delnum; i++)
 		unlink_or_warn(git_path("logs/%s", delnames[i]));
 	clear_loose_ref_cache(&ref_cache);
-- 
2.1.0.rc2.206.gedb03e5

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Jonathan Nieder wrote:
>
>> The next series from Ronnie's collection is available at
>> https://code-review.googlesource.com/#/q/topic:ref-transaction in case
>> someone wants a fresh series to look at.
>
> Here is the outcome of that review.  It could use another set of eyes
> (hint, hint) but should be mostly ready.  Interdiff below.  This is meant
> to replace rs/ref-transaction in 'pu' and I'd prefer to wait a little
> for more comments before merging to 'next'.

Alright.  I'd assume that all the other rs/ref-transaction* topics
that depends on rs/ref-transaction series will be rerolled on top of
this new round when ready.

Thanks.

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> These patches are also available from the git repository at
>
>   git://repo.or.cz/git/jrn.git tags/rs/ref-transaction

The tag fetched and built as-is seems to break 5514 among other
things ("git remote rm" segfaults).

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Jonathan Nieder]

Junio C Hamano wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> These patches are also available from the git repository at
>>
>>   git://repo.or.cz/git/jrn.git tags/rs/ref-transaction
>
> The tag fetched and built as-is seems to break 5514 among other
> things ("git remote rm" segfaults).

Yeah, I noticed that right after sending the series out. :/

The patch below fixes it[1].

-- >8 --
From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 11 Sep 2014 08:42:57 -0700
Subject: remote rm/prune: print a message when writing packed-refs fails

Until v2.1.0-rc0~22^2~11 (refs.c: add an err argument to
repack_without_refs, 2014-06-20), repack_without_refs forgot to
provide an error message when commit_packed_refs fails.  Even today,
it only provides a message for callers that pass a non-NULL err
parameter.  Internal callers in refs.c pass non-NULL err but
"git remote" does not.

That means that "git remote rm" and "git remote prune" can fail
without printing a message about why.  Fix them by passing in a
non-NULL err parameter and printing the returned message.

This is the last caller to a ref handling function passing err ==
NULL.  A later patch can drop support for err == NULL, avoiding such
problems in the future.

Change-Id: Ifb8a726ef03d0aa282a25a102313064d2e8ec283
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
[1] https://code-review.googlesource.com/1110
    https://code-review.googlesource.com/1060

 builtin/remote.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/builtin/remote.c b/builtin/remote.c
index 6eaeee7..ef1ffc3 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -750,13 +750,16 @@ static int mv(int argc, const char **argv)
 
 static int remove_branches(struct string_list *branches)
 {
+	struct strbuf err = STRBUF_INIT;
 	const char **branch_names;
 	int i, result = 0;
 
 	branch_names = xmalloc(branches->nr * sizeof(*branch_names));
 	for (i = 0; i < branches->nr; i++)
 		branch_names[i] = branches->items[i].string;
-	result |= repack_without_refs(branch_names, branches->nr, NULL);
+	if (repack_without_refs(branch_names, branches->nr, &err))
+		result |= error("%s", err.buf);
+	strbuf_release(&err);
 	free(branch_names);
 
 	for (i = 0; i < branches->nr; i++) {
@@ -1333,9 +1336,13 @@ static int prune_remote(const char *remote, int dry_run)
 		delete_refs = xmalloc(states.stale.nr * sizeof(*delete_refs));
 		for (i = 0; i < states.stale.nr; i++)
 			delete_refs[i] = states.stale.items[i].util;
-		if (!dry_run)
-			result |= repack_without_refs(delete_refs,
-						      states.stale.nr, NULL);
+		if (!dry_run) {
+			struct strbuf err = STRBUF_INIT;
+			if (repack_without_refs(delete_refs, states.stale.nr,
+						&err))
+				result |= error("%s", err.buf);
+			strbuf_release(&err);
+		}
 		free(delete_refs);
 	}
 
-- 
2.1.0.rc2.206.gedb03e5

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Junio C Hamano wrote:
>> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>>> These patches are also available from the git repository at
>>>
>>>   git://repo.or.cz/git/jrn.git tags/rs/ref-transaction
>>
>> The tag fetched and built as-is seems to break 5514 among other
>> things ("git remote rm" segfaults).
>
> Yeah, I noticed that right after sending the series out. :/
>
> The patch below fixes it[1].

Is this meant to replace anything, or is it "Oops, the previous ones
are broken, and this is to patch it up on top"?

> -- >8 --
> From: Ronnie Sahlberg <sahlberg@google.com>
> Date: Thu, 11 Sep 2014 08:42:57 -0700
> Subject: remote rm/prune: print a message when writing packed-refs fails
>
> Until v2.1.0-rc0~22^2~11 (refs.c: add an err argument to
> repack_without_refs, 2014-06-20), repack_without_refs forgot to
> provide an error message when commit_packed_refs fails.  Even today,
> it only provides a message for callers that pass a non-NULL err
> parameter.  Internal callers in refs.c pass non-NULL err but
> "git remote" does not.
>
> That means that "git remote rm" and "git remote prune" can fail
> without printing a message about why.  Fix them by passing in a
> non-NULL err parameter and printing the returned message.
>
> This is the last caller to a ref handling function passing err ==
> NULL.  A later patch can drop support for err == NULL, avoiding such
> problems in the future.
>
> Change-Id: Ifb8a726ef03d0aa282a25a102313064d2e8ec283
> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
> ---
> [1] https://code-review.googlesource.com/1110
>     https://code-review.googlesource.com/1060
>
>  builtin/remote.c | 15 +++++++++++----
>  1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/builtin/remote.c b/builtin/remote.c
> index 6eaeee7..ef1ffc3 100644
> --- a/builtin/remote.c
> +++ b/builtin/remote.c
> @@ -750,13 +750,16 @@ static int mv(int argc, const char **argv)
>  
>  static int remove_branches(struct string_list *branches)
>  {
> +	struct strbuf err = STRBUF_INIT;
>  	const char **branch_names;
>  	int i, result = 0;
>  
>  	branch_names = xmalloc(branches->nr * sizeof(*branch_names));
>  	for (i = 0; i < branches->nr; i++)
>  		branch_names[i] = branches->items[i].string;
> -	result |= repack_without_refs(branch_names, branches->nr, NULL);
> +	if (repack_without_refs(branch_names, branches->nr, &err))
> +		result |= error("%s", err.buf);
> +	strbuf_release(&err);
>  	free(branch_names);
>  
>  	for (i = 0; i < branches->nr; i++) {
> @@ -1333,9 +1336,13 @@ static int prune_remote(const char *remote, int dry_run)
>  		delete_refs = xmalloc(states.stale.nr * sizeof(*delete_refs));
>  		for (i = 0; i < states.stale.nr; i++)
>  			delete_refs[i] = states.stale.items[i].util;
> -		if (!dry_run)
> -			result |= repack_without_refs(delete_refs,
> -						      states.stale.nr, NULL);
> +		if (!dry_run) {
> +			struct strbuf err = STRBUF_INIT;
> +			if (repack_without_refs(delete_refs, states.stale.nr,
> +						&err))
> +				result |= error("%s", err.buf);
> +			strbuf_release(&err);
> +		}
>  		free(delete_refs);
>  	}

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Jonathan Nieder]

Junio C Hamano wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
> > Junio C Hamano wrote:

>>> The tag fetched and built as-is seems to break 5514 among other
>>> things ("git remote rm" segfaults).
>>
>> Yeah, I noticed that right after sending the series out. :/
>>
>> The patch below fixes it[1].
>
> Is this meant to replace anything, or is it "Oops, the previous ones
> are broken, and this is to patch it up on top"?

It's "Oops, the next one (refs.c: do not permit err == NULL) is broken,
and this is to patch it up in advance". :)

But it should apply on top, too.

There were a few other minor changes, and I think with them the series
should be ready for "next".  My "send and hope that more reviewers
appear" strategy didn't really work, so I'll send a reroll of the
series as-is in an hour or so.

Here's an interdiff as a preview.

diff --git a/builtin/branch.c b/builtin/branch.c
index 5d5bc56..4bf931e 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -238,9 +238,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 					    RESOLVE_REF_READING
 					    | RESOLVE_REF_NODEREF
 					    | RESOLVE_REF_ALLOW_BAD_NAME);
-		if (!target ||
-		    (!(flags & (REF_ISSYMREF|REF_ISBROKEN)) &&
-		     is_null_sha1(sha1))) {
+		if (!target) {
 			error(remote_branch
 			      ? _("remote branch '%s' not found.")
 			      : _("branch '%s' not found."), bname.buf);
@@ -268,8 +266,8 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 			       ? _("Deleted remote branch %s (was %s).\n")
 			       : _("Deleted branch %s (was %s).\n"),
 			       bname.buf,
-			       (flags & REF_ISSYMREF)
-			       ? target
+			       (flags & REF_ISBROKEN) ? "broken"
+			       : (flags & REF_ISSYMREF) ? target
 			       : find_unique_abbrev(sha1, DEFAULT_ABBREV));
 		}
 		delete_branch_config(bname.buf);
diff --git a/builtin/remote.c b/builtin/remote.c
index 6eaeee7..ef1ffc3 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -750,13 +750,16 @@ static int mv(int argc, const char **argv)
 
 static int remove_branches(struct string_list *branches)
 {
+	struct strbuf err = STRBUF_INIT;
 	const char **branch_names;
 	int i, result = 0;
 
 	branch_names = xmalloc(branches->nr * sizeof(*branch_names));
 	for (i = 0; i < branches->nr; i++)
 		branch_names[i] = branches->items[i].string;
-	result |= repack_without_refs(branch_names, branches->nr, NULL);
+	if (repack_without_refs(branch_names, branches->nr, &err))
+		result |= error("%s", err.buf);
+	strbuf_release(&err);
 	free(branch_names);
 
 	for (i = 0; i < branches->nr; i++) {
@@ -1333,9 +1336,13 @@ static int prune_remote(const char *remote, int dry_run)
 		delete_refs = xmalloc(states.stale.nr * sizeof(*delete_refs));
 		for (i = 0; i < states.stale.nr; i++)
 			delete_refs[i] = states.stale.items[i].util;
-		if (!dry_run)
-			result |= repack_without_refs(delete_refs,
-						      states.stale.nr, NULL);
+		if (!dry_run) {
+			struct strbuf err = STRBUF_INIT;
+			if (repack_without_refs(delete_refs, states.stale.nr,
+						&err))
+				result |= error("%s", err.buf);
+			strbuf_release(&err);
+		}
 		free(delete_refs);
 	}
 

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> It's "Oops, the next one (refs.c: do not permit err == NULL) is broken,
> and this is to patch it up in advance". :)
>
> But it should apply on top, too.

I think "in advance" makes sense for this one, too.

> There were a few other minor changes, and I think with them the series
> should be ready for "next".  My "send and hope that more reviewers
> appear" strategy didn't really work,...

You sent them just a few days ago.  Expecting anybody to have a
solid time to sit thru a 19-patch series inside that timeframe is
not so realistic.

> so I'll send a reroll of the series as-is in an hour or so.

OK.  Thanks.

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> It's "Oops, the next one (refs.c: do not permit err == NULL) is broken,
>> and this is to patch it up in advance". :)
>>
>> But it should apply on top, too.
>
> I think "in advance" makes sense for this one, too.
>
>> There were a few other minor changes, and I think with them the series
>> should be ready for "next".  My "send and hope that more reviewers
>> appear" strategy didn't really work,...
>
> You sent them just a few days ago.  Expecting anybody to have a
> solid time to sit thru a 19-patch series inside that timeframe is
> not so realistic.
>
>> so I'll send a reroll of the series as-is in an hour or so.
>
> OK.  Thanks.

I do not think I have enough time to pick a reroll up to redo the
day's integration, so please take time to make it perfect.

I noticed that with this series merged to the version I use daily,
detaching HEAD (i.e. "git checkout HEAD^0") breaks my HEAD reflog,
which made me redo the integration ejecting the series out of 'pu'.

Not happy, as my workflow relies fairly heavily on detached HEAD
X-<.

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Michael Haggerty]

On 09/12/2014 09:56 PM, Junio C Hamano wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
> [...]
>> There were a few other minor changes, and I think with them the series
>> should be ready for "next".  My "send and hope that more reviewers
>> appear" strategy didn't really work,...
> 
> You sent them just a few days ago.  Expecting anybody to have a
> solid time to sit thru a 19-patch series inside that timeframe is
> not so realistic.

It's hard to tell from my glacial (tectonic?) pace, but I really do plan
to work through all of Ronnie's ref-related patches. Of course it's up
to you whether to wait for me. I really hope to get through the third
series by the end of next week.

>> so I'll send a reroll of the series as-is in an hour or so.

Jonathan: Is a current version of this patch series set up for review in
Gerrit?

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Jonathan Nieder]

Michael Haggerty wrote:
>> Jonathan Nieder <jrnieder@gmail.com> writes:

>>> so I'll send a reroll of the series as-is in an hour or so.
>
> Jonathan: Is a current version of this patch series set up for review in
> Gerrit?

Yes.
(https://code-review.googlesource.com/#/q/project:git+topic:ref-transaction)

Thanks,
Jonathan

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> I noticed that with this series merged to the version I use daily,
> detaching HEAD (i.e. "git checkout HEAD^0") breaks my HEAD reflog,
> which made me redo the integration ejecting the series out of 'pu'.
>
> Not happy, as my workflow relies fairly heavily on detached HEAD
> X-<.

Just FYI.

Bisecting the series using the attached as a test script points
"branch -d: avoid repeated symref resolution" as a possible culprit.
Perhaps these tests may want to be added to t3200 which is touched
by the said commit (or add them earlier in the series)?

-- >8 --

#!/bin/sh

test_description='reflog not nuked with co HEAD^0'
. ./test-lib.sh

check_reflog () {
	while read name
	do
		git rev-parse --verify "$name"
	done >expect &&
	if test -f "$2"
	then
		while read object rest
		do
			git rev-parse --verify "$object"
		done >>expect <"$2"
	fi &&
	while read object rest
	do
		git rev-parse --verify "$object"
	done >actual <"$1" &&
	test_cmp expect actual
}

test_expect_success setup '
	test_tick &&
	git commit --allow-empty -m initial &&
	git branch side &&
	test_tick &&
	git commit --allow-empty -m second &&
	git log -g --oneline >baseline &&
	check_reflog baseline <<-\EOF
	master
	master^
	EOF
'

test_expect_success 'switch to branch' '
	git checkout side &&
	git log -g --oneline >switched &&
	check_reflog switched baseline <<-\EOF
	side
	EOF
'

test_expect_success 'detach to other' '
	git checkout master^0 &&
	git log -g --oneline >detach-1 &&
	check_reflog detach-1 switched <<-\EOF
	master
	EOF
'

test_expect_success 'attach to self' '
	git checkout master &&
	git log -g --oneline >detach-2 &&
	check_reflog detach-2 detach-1 <<-\EOF
	master
	EOF
'

test_expect_success 'detach to self' '
	git checkout master^0 &&
	git log -g --oneline >detach-3 &&
	check_reflog detach-3 detach-2 <<-\EOF
	master
	EOF
'

test_expect_success 'attach to other' '
	git checkout HEAD^0 &&
	git checkout side &&
	git log -g --oneline >detach-4 &&
	check_reflog detach-4 detach-3 <<-\EOF
	side
	EOF
'

test_done

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Michael Haggerty]

On 09/13/2014 01:57 AM, Jonathan Nieder wrote:
> Michael Haggerty wrote:
>>> Jonathan Nieder <jrnieder@gmail.com> writes:
> 
>>>> so I'll send a reroll of the series as-is in an hour or so.
>>
>> Jonathan: Is a current version of this patch series set up for review in
>> Gerrit?
> 
> Yes.
> (https://code-review.googlesource.com/#/q/project:git+topic:ref-transaction)

I just worked through the patch series, leaving lots of comments in
Gerrit. Overall it looks pretty good and makes a lot of very worthwhile
progress. The only patch that gives me a bit of heartburn is

    [PATCH 15/19] refs.c: fix handling of badly named refs

not because it is necessarily wrong, but because it has a lot of
non-local effects that are hard to evaluate. I made a bunch of comments
in Gerrit about that patch, too, and will wait for a response before
having another go at it.

Thanks for all your hard and detailed work, Ronnie and Jonathan!

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Michael Haggerty <mhagger@alum.mit.edu> writes:

> On 09/13/2014 01:57 AM, Jonathan Nieder wrote:
>> Michael Haggerty wrote:
>>>> Jonathan Nieder <jrnieder@gmail.com> writes:
>> 
>>>>> so I'll send a reroll of the series as-is in an hour or so.
>>>
>>> Jonathan: Is a current version of this patch series set up for review in
>>> Gerrit?
>> 
>> Yes.
>> (https://code-review.googlesource.com/#/q/project:git+topic:ref-transaction)
>
> I just worked through the patch series, leaving lots of comments in
> Gerrit. Overall it looks pretty good and makes a lot of very worthwhile
> progress. The only patch that gives me a bit of heartburn is
>
>     [PATCH 15/19] refs.c: fix handling of badly named refs
>
> not because it is necessarily wrong, but because it has a lot of
> non-local effects that are hard to evaluate. I made a bunch of comments
> in Gerrit about that patch, too, and will wait for a response before
> having another go at it.
>
> Thanks for all your hard and detailed work, Ronnie and Jonathan!
>
> Michael

Jonathan: Is a current version of this patch series set up to be
fetched so that it can be reviewed outside Gerrit?

Running ls-remote against https://code.googlesource.com/git shows
many refs under refs/changes/* but it is unclear to me if there is a
coherent single "here is the latest and greatest, dependents rebased
on dependeds in the right order" thing that I can fetch and look at
with "log -p master..".

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Jonathan Nieder]

Junio C Hamano wrote:

> Jonathan: Is a current version of this patch series set up to be
> fetched so that it can be reviewed outside Gerrit?

The current tip is 06d707cb63e34fc55a18ecc47e668f3c44acae57 from
https://code.googlesource.com/git (fetch-by-sha1 should work).  Each
reroll gets its own refname of the form refs/changes/62/1062/<number>
with <number> increasing.  The "Download" widget in the top-right
corner of https://code-review.googlesource.com/1062 shows the refname.

There are plenty of unaddressed comments from Michael, so I'd hold off
on picking up the latest for pu for now.

Thanks,
Jonathan

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Junio C Hamano wrote:
>
>> Jonathan: Is a current version of this patch series set up to be
>> fetched so that it can be reviewed outside Gerrit?
>
> The current tip is 06d707cb63e34fc55a18ecc47e668f3c44acae57 from
> https://code.googlesource.com/git (fetch-by-sha1 should work).  Each
> reroll gets its own refname of the form refs/changes/62/1062/<number>
> with <number> increasing.  The "Download" widget in the top-right
> corner of https://code-review.googlesource.com/1062 shows the refname.

While I am showing my naiveté, how does one figure out that 1062 is
the last one in the series?

Does the order of changes that appear in

https://code-review.googlesource.com/#/q/project:git+branch:master+topic:ref-transaction

have any significance?  e.g. is a "topic" supposed to be a single
strand of pearls on top of the "branch", and the top one is the tip,
or something?

> There are plenty of unaddressed comments from Michael, so I'd hold off
> on picking up the latest for pu for now.

Oh, the request was not for that.  I simply cannot read the patch
with only limited context in the web browser without having a ready
access to a coherent whole, i.e. a full tree with history, to review
a change like this.

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Jonathan Nieder]

Junio C Hamano wrote:

> Does the order of changes that appear in
>
> https://code-review.googlesource.com/#/q/project:git+branch:master+topic:ref-transaction
>
> have any significance?  e.g. is a "topic" supposed to be a single
> strand of pearls on top of the "branch", and the top one is the tip,
> or something?

Alas no, though that would be a good feature.

Search results are ordered by which was last updated (for example when
someone adds a comment).

The 'Related Changes' shown on the change screen are in topological
order, with the tip at the top.

Jonathan

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

I know that a review-update cycle is still going in the dark at

  https://code-review.googlesource.com/#/q/topic:ref-transaction

for this series.  Are we almost there for v22 which hopefully be the
final before we merge it to 'next' and go incremental?  

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Jonathan Nieder]

Junio C Hamano wrote:

> I know that a review-update cycle is still going in the dark at
>
>   https://code-review.googlesource.com/#/q/topic:ref-transaction
>
> for this series.

Eh, it's at least public and doesn't flood the list with rebased
versions of the series.

Would you prefer if there were some list archived on gmane with the
automated mails from gerrit, to make it easier to look back at later?

>                  Are we almost there for v22 which hopefully be the
> final before we merge it to 'next' and go incremental?

The patch "fix handling of badly named refs"[1] is still undergoing
heavy churn.

I think it's worth getting that one right.

Thanks,
Jonathan

[1] https://code-review.googlesource.com/1070

Re: [PATCH v21 0/19] rs/ref-transaction (Re: Transaction patch series overview)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> The patch "fix handling of badly named refs"[1] is still undergoing
> heavy churn.
>
> I think it's worth getting that one right.

Oh, no question about it.  I was only making sure that I didn't miss
availability of updates for larger series we saw during this cycle.

[PATCH v22 0/24] rs/ref-transaction

[Jonathan Nieder]

Jonathan Nieder wrote:
> Jonathan Nieder wrote:

>> The next series from Ronnie's collection is available at
>> https://code-review.googlesource.com/#/q/topic:ref-transaction in case
>> someone wants a fresh series to look at.
>
> Here is the outcome of that review.  It could use another set of eyes
> (hint, hint)

Another set of eyes arrived and helped.  Here's a reroll.

Jonathan Nieder (6):
  mv test: recreate mod/ directory instead of relying on stale copy
  branch -d: avoid repeated symref resolution
  packed-ref cache: forbid dot-components in refnames
  refs.c: do not permit err == NULL
  lockfile: remove unable_to_lock_error
  ref_transaction_commit: bail out on failure to remove a ref

Junio C Hamano (1):
  reflog test: test interaction with detached HEAD

Ronnie Sahlberg (17):
  wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as success
  wrapper.c: add a new function unlink_or_msg
  refs.c: add an err argument to delete_ref_loose
  refs.c: pass the ref log message to _create/delete/update instead of
    _commit
  rename_ref: don't ask read_ref_full where the ref came from
  refs.c: refuse to lock badly named refs in lock_ref_sha1_basic
  refs.c: call lock_ref_sha1_basic directly from commit
  refs.c: pass a list of names to skip to is_refname_available
  refs.c: ref_transaction_commit: distinguish name conflicts from other
    errors
  fetch.c: change s_update_ref to use a ref transaction
  refs.c: make write_ref_sha1 static
  refs.c: change resolve_ref_unsafe reading argument to be a flags field
  branch -d: simplify by using RESOLVE_REF_READING flag
  test: put tests for handling of bad ref names in one place
  refs.c: allow listing and deleting badly named refs
  for-each-ref.c: improve message before aborting on broken ref
  remote rm/prune: print a message when writing packed-refs fails

 branch.c                 |   6 +-
 builtin/blame.c          |   2 +-
 builtin/branch.c         |  22 ++-
 builtin/checkout.c       |   6 +-
 builtin/clone.c          |   2 +-
 builtin/commit.c         |   6 +-
 builtin/fetch.c          |  34 ++--
 builtin/fmt-merge-msg.c  |   2 +-
 builtin/for-each-ref.c   |  11 +-
 builtin/fsck.c           |   2 +-
 builtin/log.c            |   3 +-
 builtin/merge.c          |   2 +-
 builtin/notes.c          |   2 +-
 builtin/receive-pack.c   |   9 +-
 builtin/remote.c         |  20 ++-
 builtin/replace.c        |   5 +-
 builtin/show-branch.c    |   7 +-
 builtin/symbolic-ref.c   |   2 +-
 builtin/tag.c            |   4 +-
 builtin/update-ref.c     |  13 +-
 bundle.c                 |   2 +-
 cache.h                  |  42 +++--
 fast-import.c            |   8 +-
 git-compat-util.h        |  16 +-
 http-backend.c           |   4 +-
 lockfile.c               |  10 --
 notes-merge.c            |   2 +-
 reflog-walk.c            |   5 +-
 refs.c                   | 438 ++++++++++++++++++++++++++++++++---------------
 refs.h                   |  40 +++--
 remote.c                 |  11 +-
 sequencer.c              |   8 +-
 t/t1400-update-ref.sh    |  62 +++----
 t/t1413-reflog-detach.sh |  70 ++++++++
 t/t1430-bad-ref-name.sh  | 207 ++++++++++++++++++++++
 t/t3200-branch.sh        |   9 +
 t/t7001-mv.sh            |  15 +-
 t/t9300-fast-import.sh   |  30 ----
 transport-helper.c       |   5 +-
 transport.c              |   5 +-
 upload-pack.c            |   2 +-
 walker.c                 |   5 +-
 wrapper.c                |  28 ++-
 wt-status.c              |   2 +-
 44 files changed, 838 insertions(+), 348 deletions(-)
 create mode 100755 t/t1413-reflog-detach.sh
 create mode 100755 t/t1430-bad-ref-name.sh

-- 
2.0.0.450.ga793d96

---
Changes since v21:

 branch.c                    |   2 +-
 builtin/blame.c             |   2 +-
 builtin/branch.c            |  25 ++---
 builtin/checkout.c          |   6 +-
 builtin/clone.c             |   2 +-
 builtin/commit.c            |   2 +-
 builtin/fetch.c             |   6 +-
 builtin/fmt-merge-msg.c     |   2 +-
 builtin/for-each-ref.c      |  11 +-
 builtin/fsck.c              |   2 +-
 builtin/log.c               |   4 +-
 builtin/merge.c             |   2 +-
 builtin/notes.c             |   2 +-
 builtin/receive-pack.c      |   4 +-
 builtin/remote.c            |  21 ++--
 builtin/show-branch.c       |   9 +-
 builtin/symbolic-ref.c      |   2 +-
 builtin/update-ref.c        |   3 +-
 bundle.c                    |   2 +-
 cache.h                     |  33 ++++--
 http-backend.c              |   5 +-
 notes-merge.c               |   2 +-
 reflog-walk.c               |   6 +-
 refs.c                      | 263 +++++++++++++++++++++++++-------------------
 refs.h                      |  34 +++---
 remote.c                    |  13 ++-
 sequencer.c                 |   4 +-
 t/t1400-update-ref.sh       |  46 ++------
 t/t1402-check-ref-format.sh |  48 --------
 t/t1413-reflog-detach.sh    |  70 ++++++++++++
 t/t1430-bad-ref-name.sh     | 207 ++++++++++++++++++++++++++++++++++
 t/t9300-fast-import.sh      |  30 -----
 transport-helper.c          |   5 +-
 transport.c                 |   6 +-
 upload-pack.c               |   2 +-
 wt-status.c                 |   2 +-
 36 files changed, 557 insertions(+), 328 deletions(-)

diff --git c/branch.c w/branch.c
index ba3e1c1..adb07c6 100644
--- c/branch.c
+++ w/branch.c
@@ -186,7 +186,7 @@ int validate_new_branchname(const char *name, struct strbuf *ref,
 		const char *head;
 		unsigned char sha1[20];
 
-		head = resolve_ref_unsafe("HEAD", sha1, NULL, 0);
+		head = resolve_ref_unsafe("HEAD", 0, sha1, NULL);
 		if (!is_bare_repository() && head && !strcmp(head, ref->buf))
 			die(_("Cannot force update the current branch."));
 	}
diff --git c/builtin/blame.c w/builtin/blame.c
index b8bec0a..5cbd38f 100644
--- c/builtin/blame.c
+++ w/builtin/blame.c
@@ -2292,7 +2292,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 	commit->object.type = OBJ_COMMIT;
 	parent_tail = &commit->parents;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, NULL, RESOLVE_REF_READING))
+	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
 		die("no such ref: HEAD");
 
 	parent_tail = append_parent(parent_tail, head_sha1);
diff --git c/builtin/branch.c w/builtin/branch.c
index 5d5bc56..94aaea1 100644
--- c/builtin/branch.c
+++ w/builtin/branch.c
@@ -129,8 +129,8 @@ static int branch_merged(int kind, const char *name,
 		    branch->merge[0] &&
 		    branch->merge[0]->dst &&
 		    (reference_name = reference_name_to_free =
-		     resolve_refdup(branch->merge[0]->dst, sha1,
-				    NULL, RESOLVE_REF_READING)) != NULL)
+		     resolve_refdup(branch->merge[0]->dst, RESOLVE_REF_READING,
+				    sha1, NULL)) != NULL)
 			reference_rev = lookup_commit_reference(sha1);
 	}
 	if (!reference_rev)
@@ -234,13 +234,12 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, sha1, &flags,
+		target = resolve_ref_unsafe(name,
 					    RESOLVE_REF_READING
-					    | RESOLVE_REF_NODEREF
-					    | RESOLVE_REF_ALLOW_BAD_NAME);
-		if (!target ||
-		    (!(flags & (REF_ISSYMREF|REF_ISBROKEN)) &&
-		     is_null_sha1(sha1))) {
+					    | RESOLVE_REF_NO_RECURSE
+					    | RESOLVE_REF_ALLOW_BAD_NAME,
+					    sha1, &flags);
+		if (!target) {
 			error(remote_branch
 			      ? _("remote branch '%s' not found.")
 			      : _("branch '%s' not found."), bname.buf);
@@ -255,7 +254,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 			continue;
 		}
 
-		if (delete_ref(name, sha1, REF_NODEREF|REF_BADNAMEOK)) {
+		if (delete_ref(name, sha1, REF_NODEREF)) {
 			error(remote_branch
 			      ? _("Error deleting remote branch '%s'")
 			      : _("Error deleting branch '%s'"),
@@ -268,8 +267,8 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 			       ? _("Deleted remote branch %s (was %s).\n")
 			       : _("Deleted branch %s (was %s).\n"),
 			       bname.buf,
-			       (flags & REF_ISSYMREF)
-			       ? target
+			       (flags & REF_ISBROKEN) ? "broken"
+			       : (flags & REF_ISSYMREF) ? target
 			       : find_unique_abbrev(sha1, DEFAULT_ABBREV));
 		}
 		delete_branch_config(bname.buf);
@@ -301,7 +300,7 @@ static char *resolve_symref(const char *src, const char *prefix)
 	int flag;
 	const char *dst, *cp;
 
-	dst = resolve_ref_unsafe(src, sha1, &flag, 0);
+	dst = resolve_ref_unsafe(src, 0, sha1, &flag);
 	if (!(dst && (flag & REF_ISSYMREF)))
 		return NULL;
 	if (prefix && (cp = skip_prefix(dst, prefix)))
@@ -867,7 +866,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	track = git_branch_track;
 
-	head = resolve_refdup("HEAD", head_sha1, NULL, 0);
+	head = resolve_refdup("HEAD", 0, head_sha1, NULL);
 	if (!head)
 		die(_("Failed to resolve HEAD as a valid ref."));
 	if (!strcmp(head, "HEAD")) {
diff --git c/builtin/checkout.c w/builtin/checkout.c
index 64af1f7..a5fef2d 100644
--- c/builtin/checkout.c
+++ w/builtin/checkout.c
@@ -356,7 +356,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 	    commit_locked_index(lock_file))
 		die(_("unable to write new index file"));
 
-	read_ref_full("HEAD", rev, &flag, 0);
+	read_ref_full("HEAD", 0, rev, &flag);
 	head = lookup_commit_reference_gently(rev, 1);
 
 	errs |= post_checkout_hook(head, head, 0);
@@ -771,7 +771,7 @@ static int switch_branches(const struct checkout_opts *opts,
 	unsigned char rev[20];
 	int flag, writeout_error = 0;
 	memset(&old, 0, sizeof(old));
-	old.path = path_to_free = resolve_refdup("HEAD", rev, &flag, 0);
+	old.path = path_to_free = resolve_refdup("HEAD", 0, rev, &flag);
 	old.commit = lookup_commit_reference_gently(rev, 1);
 	if (!(flag & REF_ISSYMREF))
 		old.path = NULL;
@@ -1068,7 +1068,7 @@ static int checkout_branch(struct checkout_opts *opts,
 		unsigned char rev[20];
 		int flag;
 
-		if (!read_ref_full("HEAD", rev, &flag, 0) &&
+		if (!read_ref_full("HEAD", 0, rev, &flag) &&
 		    (flag & REF_ISSYMREF) && is_null_sha1(rev))
 			return switch_unborn_to_new_branch(opts);
 	}
diff --git c/builtin/clone.c w/builtin/clone.c
index 12a78e1..0f5c880 100644
--- c/builtin/clone.c
+++ w/builtin/clone.c
@@ -622,7 +622,7 @@ static int checkout(void)
 	if (option_no_checkout)
 		return 0;
 
-	head = resolve_refdup("HEAD", sha1, NULL, RESOLVE_REF_READING);
+	head = resolve_refdup("HEAD", RESOLVE_REF_READING, sha1, NULL);
 	if (!head) {
 		warning(_("remote HEAD refers to nonexistent ref, "
 			  "unable to checkout.\n"));
diff --git c/builtin/commit.c w/builtin/commit.c
index 3536330..9ccc78b 100644
--- c/builtin/commit.c
+++ w/builtin/commit.c
@@ -1468,7 +1468,7 @@ static void print_summary(const char *prefix, const unsigned char *sha1,
 	rev.diffopt.break_opt = 0;
 	diff_setup_done(&rev.diffopt);
 
-	head = resolve_ref_unsafe("HEAD", junk_sha1, NULL, 0);
+	head = resolve_ref_unsafe("HEAD", 0, junk_sha1, NULL);
 	printf("[%s%s ",
 		starts_with(head, "refs/heads/") ?
 			head + 11 :
diff --git c/builtin/fetch.c w/builtin/fetch.c
index 2e3bc73..30b40f6 100644
--- c/builtin/fetch.c
+++ w/builtin/fetch.c
@@ -392,10 +392,10 @@ static int s_update_ref(const char *action,
 		goto fail;
 
 	ret = ref_transaction_commit(transaction, &err);
-	if (ret == UPDATE_REFS_NAME_CONFLICT)
-		df_conflict = 1;
-	if (ret)
+	if (ret) {
+		df_conflict = (ret == TRANSACTION_NAME_CONFLICT);
 		goto fail;
+	}
 
 	ref_transaction_free(transaction);
 	strbuf_release(&err);
diff --git c/builtin/fmt-merge-msg.c w/builtin/fmt-merge-msg.c
index b2355ad..afe05dc 100644
--- c/builtin/fmt-merge-msg.c
+++ w/builtin/fmt-merge-msg.c
@@ -602,7 +602,7 @@ int fmt_merge_msg(struct strbuf *in, struct strbuf *out,
 
 	/* get current branch */
 	current_branch = current_branch_to_free =
-		resolve_refdup("HEAD", head_sha1, NULL, RESOLVE_REF_READING);
+		resolve_refdup("HEAD", RESOLVE_REF_READING, head_sha1, NULL);
 	if (!current_branch)
 		die("No current branch");
 	if (starts_with(current_branch, "refs/heads/"))
diff --git c/builtin/for-each-ref.c w/builtin/for-each-ref.c
index 090390c..a88d681 100644
--- c/builtin/for-each-ref.c
+++ w/builtin/for-each-ref.c
@@ -649,8 +649,8 @@ static void populate_value(struct refinfo *ref)
 
 	if (need_symref && (ref->flag & REF_ISSYMREF) && !ref->symref) {
 		unsigned char unused1[20];
-		ref->symref = resolve_refdup(ref->refname, unused1,
-					     NULL, RESOLVE_REF_READING);
+		ref->symref = resolve_refdup(ref->refname, RESOLVE_REF_READING,
+					     unused1, NULL);
 		if (!ref->symref)
 			ref->symref = "";
 	}
@@ -708,8 +708,8 @@ static void populate_value(struct refinfo *ref)
 			const char *head;
 			unsigned char sha1[20];
 
-			head = resolve_ref_unsafe("HEAD", sha1,
-						  NULL, RESOLVE_REF_READING);
+			head = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+						  sha1, NULL);
 			if (!strcmp(ref->refname, head))
 				v->s = "*";
 			else
@@ -853,8 +853,7 @@ static int grab_single_ref(const char *refname, const unsigned char *sha1, int f
 	struct refinfo *ref;
 	int cnt;
 
-	if ((flag & REF_ISBROKEN) &&
-	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+	if (flag & REF_BAD_NAME) {
 		  warning("ignoring ref with broken name %s", refname);
 		  return 0;
 	}
diff --git c/builtin/fsck.c w/builtin/fsck.c
index 506e32b..7cd109a 100644
--- c/builtin/fsck.c
+++ w/builtin/fsck.c
@@ -560,7 +560,7 @@ static int fsck_head_link(void)
 	if (verbose)
 		fprintf(stderr, "Checking HEAD link\n");
 
-	head_points_at = resolve_ref_unsafe("HEAD", head_sha1, &flag, 0);
+	head_points_at = resolve_ref_unsafe("HEAD", 0, head_sha1, &flag);
 	if (!head_points_at)
 		return error("Invalid HEAD");
 	if (!strcmp(head_points_at, "HEAD"))
diff --git c/builtin/log.c w/builtin/log.c
index 230a9ef..493440a 100644
--- c/builtin/log.c
+++ w/builtin/log.c
@@ -1395,8 +1395,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (check_head) {
 			unsigned char sha1[20];
 			const char *ref;
-			ref = resolve_ref_unsafe("HEAD", sha1, NULL,
-						 RESOLVE_REF_READING);
+			ref = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+						 sha1, NULL);
 			if (ref && starts_with(ref, "refs/heads/"))
 				branch_name = xstrdup(ref + strlen("refs/heads/"));
 			else
diff --git c/builtin/merge.c w/builtin/merge.c
index d262279..6f56967 100644
--- c/builtin/merge.c
+++ w/builtin/merge.c
@@ -1108,7 +1108,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
 	 * Check if we are _not_ on a detached HEAD, i.e. if there is a
 	 * current branch.
 	 */
-	branch = branch_to_free = resolve_refdup("HEAD", head_sha1, &flag, 0);
+	branch = branch_to_free = resolve_refdup("HEAD", 0, head_sha1, &flag);
 	if (branch && starts_with(branch, "refs/heads/"))
 		branch += 11;
 	if (!branch || is_null_sha1(head_sha1))
diff --git c/builtin/notes.c w/builtin/notes.c
index 16df78c..eaf297d 100644
--- c/builtin/notes.c
+++ w/builtin/notes.c
@@ -703,7 +703,7 @@ static int merge_commit(struct notes_merge_options *o)
 	init_notes(t, "NOTES_MERGE_PARTIAL", combine_notes_overwrite, 0);
 
 	o->local_ref = local_ref_to_free =
-		resolve_refdup("NOTES_MERGE_REF", sha1, NULL, 0);
+		resolve_refdup("NOTES_MERGE_REF", 0, sha1, NULL);
 	if (!o->local_ref)
 		die("Failed to resolve NOTES_MERGE_REF");
 
diff --git c/builtin/receive-pack.c w/builtin/receive-pack.c
index 555a4a6..8a6e7e3 100644
--- c/builtin/receive-pack.c
+++ w/builtin/receive-pack.c
@@ -656,7 +656,7 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)
 	int flag;
 
 	strbuf_addf(&buf, "%s%s", get_git_namespace(), cmd->ref_name);
-	dst_name = resolve_ref_unsafe(buf.buf, sha1, &flag, 0);
+	dst_name = resolve_ref_unsafe(buf.buf, 0, sha1, &flag);
 	strbuf_release(&buf);
 
 	if (!(flag & REF_ISSYMREF))
@@ -817,7 +817,7 @@ static void execute_commands(struct command *commands,
 	check_aliased_updates(commands);
 
 	free(head_name_to_free);
-	head_name = head_name_to_free = resolve_refdup("HEAD", sha1, NULL, 0);
+	head_name = head_name_to_free = resolve_refdup("HEAD", 0, sha1, NULL);
 
 	checked_connectivity = 1;
 	for (cmd = commands; cmd; cmd = cmd->next) {
diff --git c/builtin/remote.c w/builtin/remote.c
index 6eaeee7..8517cfa 100644
--- c/builtin/remote.c
+++ w/builtin/remote.c
@@ -568,8 +568,8 @@ static int read_remote_branches(const char *refname,
 	strbuf_addf(&buf, "refs/remotes/%s/", rename->old);
 	if (starts_with(refname, buf.buf)) {
 		item = string_list_append(rename->remote_branches, xstrdup(refname));
-		symref = resolve_ref_unsafe(refname, orig_sha1, &flag,
-					    RESOLVE_REF_READING);
+		symref = resolve_ref_unsafe(refname, RESOLVE_REF_READING,
+					    orig_sha1, &flag);
 		if (flag & REF_ISSYMREF)
 			item->util = xstrdup(symref);
 		else
@@ -705,7 +705,7 @@ static int mv(int argc, const char **argv)
 		int flag = 0;
 		unsigned char sha1[20];
 
-		read_ref_full(item->string, sha1, &flag, RESOLVE_REF_READING);
+		read_ref_full(item->string, RESOLVE_REF_READING, sha1, &flag);
 		if (!(flag & REF_ISSYMREF))
 			continue;
 		if (delete_ref(item->string, NULL, REF_NODEREF))
@@ -750,13 +750,16 @@ static int mv(int argc, const char **argv)
 
 static int remove_branches(struct string_list *branches)
 {
+	struct strbuf err = STRBUF_INIT;
 	const char **branch_names;
 	int i, result = 0;
 
 	branch_names = xmalloc(branches->nr * sizeof(*branch_names));
 	for (i = 0; i < branches->nr; i++)
 		branch_names[i] = branches->items[i].string;
-	result |= repack_without_refs(branch_names, branches->nr, NULL);
+	if (repack_without_refs(branch_names, branches->nr, &err))
+		result |= error("%s", err.buf);
+	strbuf_release(&err);
 	free(branch_names);
 
 	for (i = 0; i < branches->nr; i++) {
@@ -1333,9 +1336,13 @@ static int prune_remote(const char *remote, int dry_run)
 		delete_refs = xmalloc(states.stale.nr * sizeof(*delete_refs));
 		for (i = 0; i < states.stale.nr; i++)
 			delete_refs[i] = states.stale.items[i].util;
-		if (!dry_run)
-			result |= repack_without_refs(delete_refs,
-						      states.stale.nr, NULL);
+		if (!dry_run) {
+			struct strbuf err = STRBUF_INIT;
+			if (repack_without_refs(delete_refs, states.stale.nr,
+						&err))
+				result |= error("%s", err.buf);
+			strbuf_release(&err);
+		}
 		free(delete_refs);
 	}
 
diff --git c/builtin/show-branch.c w/builtin/show-branch.c
index ef6ea52..acc8dc1 100644
--- c/builtin/show-branch.c
+++ w/builtin/show-branch.c
@@ -727,8 +727,9 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		if (ac == 0) {
 			static const char *fake_av[2];
 
-			fake_av[0] = resolve_refdup("HEAD", sha1, NULL,
-						    RESOLVE_REF_READING);
+			fake_av[0] = resolve_refdup("HEAD",
+						    RESOLVE_REF_READING,
+						    sha1, NULL);
 			fake_av[1] = NULL;
 			av = fake_av;
 			ac = 1;
@@ -790,8 +791,8 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		}
 	}
 
-	head_p = resolve_ref_unsafe("HEAD", head_sha1, NULL,
-				    RESOLVE_REF_READING);
+	head_p = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+				    head_sha1, NULL);
 	if (head_p) {
 		head_len = strlen(head_p);
 		memcpy(head, head_p, head_len + 1);
diff --git c/builtin/symbolic-ref.c w/builtin/symbolic-ref.c
index 1dd04af..29fb3f1 100644
--- c/builtin/symbolic-ref.c
+++ w/builtin/symbolic-ref.c
@@ -13,7 +13,7 @@ static int check_symref(const char *HEAD, int quiet, int shorten, int print)
 {
 	unsigned char sha1[20];
 	int flag;
-	const char *refname = resolve_ref_unsafe(HEAD, sha1, &flag, 0);
+	const char *refname = resolve_ref_unsafe(HEAD, 0, sha1, &flag);
 
 	if (!refname)
 		die("No such ref: %s", HEAD);
diff --git c/builtin/update-ref.c w/builtin/update-ref.c
index e379fdd..6c9be05 100644
--- c/builtin/update-ref.c
+++ w/builtin/update-ref.c
@@ -419,8 +419,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (no_deref)
 		flags = REF_NODEREF;
 	if (delete)
-		return delete_ref(refname, oldval ? oldsha1 : NULL,
-				  flags|REF_BADNAMEOK);
+		return delete_ref(refname, oldval ? oldsha1 : NULL, flags);
 	else
 		return update_ref(msg, refname, sha1, oldval ? oldsha1 : NULL,
 				  flags, UPDATE_REFS_DIE_ON_ERR);
diff --git c/bundle.c w/bundle.c
index 32dd2f7..d92e49c 100644
--- c/bundle.c
+++ w/bundle.c
@@ -311,7 +311,7 @@ int create_bundle(struct bundle_header *header, const char *path,
 			continue;
 		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
 			continue;
-		if (read_ref_full(e->name, sha1, &flag, RESOLVE_REF_READING))
+		if (read_ref_full(e->name, RESOLVE_REF_READING, sha1, &flag))
 			flag = 0;
 		display_ref = (flag & REF_ISSYMREF) ? e->name : ref;
 
diff --git c/cache.h w/cache.h
index 995729f..f582b6c 100644
--- c/cache.h
+++ w/cache.h
@@ -946,8 +946,8 @@ extern int for_each_abbrev(const char *prefix, each_abbrev_fn, void *);
 extern int get_sha1_hex(const char *hex, unsigned char *sha1);
 
 extern char *sha1_to_hex(const unsigned char *sha1);	/* static buffer result! */
-extern int read_ref_full(const char *refname, unsigned char *sha1,
-			 int *flags, int resolve_flags);
+extern int read_ref_full(const char *refname, int resolve_flags,
+			 unsigned char *sha1, int *flags);
 extern int read_ref(const char *refname, unsigned char *sha1);
 
 /*
@@ -969,27 +969,36 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  *   "writing" to the ref, the return value is the name of the ref
  *   that will actually be created or changed.
  *
+ * If the RESOLVE_REF_NO_RECURSE flag is passed, only resolves one
+ * level of symbolic reference.  The value stored in sha1 for a symbolic
+ * reference will always be null_sha1 in this case, and the return
+ * value is the reference that the symref refers to directly.
+ *
  * If flags is non-NULL, set the value that it points to the
  * combination of REF_ISPACKED (if the reference was found among the
  * packed references), REF_ISSYMREF (if the initial reference was a
- * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
+ * symbolic reference), REF_BAD_NAME (if the reference name is ill
+ * formed --- see RESOLVE_REF_ALLOW_BAD_NAME below), and REF_ISBROKEN
+ * (if the ref is malformed).
  *
  * If ref is not a properly-formatted, normalized reference, return
  * NULL.  If more than MAXDEPTH recursive symbolic lookups are needed,
  * give up and return NULL.
  *
- * RESOLVE_REF_ALLOW_BAD_NAME disables most of the ref name checking except
- * for names that are absolute paths or contain ".." components. For both
- * these cases the function will return NULL and set errno to EINVAL.
- * If the name is bad then the function will set the REF_ISBROKEN flag and
- * return the name, if the ref exists, or NULL, if it does not.
- * When this flag is set, any badly named refs will resolve to nullsha1.
+ * RESOLVE_REF_ALLOW_BAD_NAME allows resolving refs even when their
+ * name is invalid according to git-check-ref-format(1).  If the name
+ * is bad then the value stored in sha1 will be null_sha1 and the
+ * REF_ISBROKEN and REF_BAD_NAME flags will be set.
+ *
+ * Even with RESOLVE_REF_ALLOW_BAD_NAME, names that escape the refs/
+ * directory and do not consist of all caps and underscores cannot be
+ * resolved.  The function returns NULL for such ref names.
  */
 #define RESOLVE_REF_READING 0x01
-#define RESOLVE_REF_NODEREF 0x02
+#define RESOLVE_REF_NO_RECURSE 0x02
 #define RESOLVE_REF_ALLOW_BAD_NAME 0x04
-extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
-extern char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags);
+extern const char *resolve_ref_unsafe(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
+extern char *resolve_refdup(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
 
 extern int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref);
 extern int dwim_log(const char *str, int len, unsigned char *sha1, char **ref);
diff --git c/http-backend.c w/http-backend.c
index 8f94f9b..e172886 100644
--- c/http-backend.c
+++ w/http-backend.c
@@ -417,8 +417,9 @@ static int show_head_ref(const char *refname, const unsigned char *sha1,
 
 	if (flag & REF_ISSYMREF) {
 		unsigned char unused[20];
-		const char *target = resolve_ref_unsafe(refname, unused,
-						NULL, RESOLVE_REF_READING);
+		const char *target = resolve_ref_unsafe(refname,
+							RESOLVE_REF_READING,
+							unused, NULL);
 		const char *target_nons = strip_namespace(target);
 
 		strbuf_addf(buf, "ref: %s\n", target_nons);
diff --git c/notes-merge.c w/notes-merge.c
index ffca602..3c88d17 100644
--- c/notes-merge.c
+++ w/notes-merge.c
@@ -549,7 +549,7 @@ int notes_merge(struct notes_merge_options *o,
 	       o->local_ref, o->remote_ref);
 
 	/* Dereference o->local_ref into local_sha1 */
-	if (read_ref_full(o->local_ref, local_sha1, NULL, 0))
+	if (read_ref_full(o->local_ref, 0, local_sha1, NULL))
 		die("Failed to resolve local notes ref '%s'", o->local_ref);
 	else if (!check_refname_format(o->local_ref, 0) &&
 		is_null_sha1(local_sha1))
diff --git c/reflog-walk.c w/reflog-walk.c
index feeaf0a..23345ea 100644
--- c/reflog-walk.c
+++ w/reflog-walk.c
@@ -48,8 +48,8 @@ static struct complete_reflogs *read_complete_reflog(const char *ref)
 		unsigned char sha1[20];
 		const char *name;
 		void *name_to_free;
-		name = name_to_free = resolve_refdup(ref, sha1, NULL,
-						     RESOLVE_REF_READING);
+		name = name_to_free = resolve_refdup(ref, RESOLVE_REF_READING,
+						     sha1, NULL);
 		if (name) {
 			for_each_reflog_ent(name, read_one_reflog, reflogs);
 			free(name_to_free);
@@ -175,7 +175,7 @@ int add_reflog_for_walk(struct reflog_walk_info *info,
 		if (*branch == '\0') {
 			unsigned char sha1[20];
 			free(branch);
-			branch = resolve_refdup("HEAD", sha1, NULL, 0);
+			branch = resolve_refdup("HEAD", 0, sha1, NULL);
 			if (!branch)
 				die ("No current branch");
 
diff --git c/refs.c w/refs.c
index 3b27758..d9d327d 100644
--- c/refs.c
+++ w/refs.c
@@ -69,16 +69,8 @@ static int check_refname_component(const char *refname, int flags)
 out:
 	if (cp == refname)
 		return 0; /* Component has zero length. */
-	if (refname[0] == '.') {
-		if (!(flags & REFNAME_DOT_COMPONENT))
-			return -1; /* Component starts with '.'. */
-		/*
-		 * Even if leading dots are allowed, don't allow "."
-		 * as a component (".." is prevented by a rule above).
-		 */
-		if (refname[1] == '\0')
-			return -1; /* Component equals ".". */
-	}
+	if (refname[0] == '.')
+		return -1; /* Component starts with '.'. */
 	if (cp - refname >= 5 && !memcmp(cp - 5, ".lock", 5))
 		return -1; /* Refname ends with ".lock". */
 	return cp - refname;
@@ -193,8 +185,8 @@ struct ref_dir {
 
 /*
  * Bit values for ref_entry::flag.  REF_ISSYMREF=0x01,
- * REF_ISPACKED=0x02, and REF_ISBROKEN=0x04 are public values; see
- * refs.h.
+ * REF_ISPACKED=0x02, REF_ISBROKEN=0x04 and REF_BAD_NAME=0x08 are
+ * public values; see refs.h.
  */
 
 /*
@@ -202,16 +194,16 @@ struct ref_dir {
  * the correct peeled value for the reference, which might be
  * null_sha1 if the reference is not a tag or if it is broken.
  */
-#define REF_KNOWS_PEELED 0x08
+#define REF_KNOWS_PEELED 0x10
 
 /* ref_entry represents a directory of references */
-#define REF_DIR 0x10
+#define REF_DIR 0x20
 
 /*
  * Entry has not yet been read from disk (used only for REF_DIR
  * entries representing loose references)
  */
-#define REF_INCOMPLETE 0x20
+#define REF_INCOMPLETE 0x40
 
 /*
  * A ref_entry represents either a reference or a "subdirectory" of
@@ -280,6 +272,37 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
 	return dir;
 }
 
+static int escapes_cwd(const char *path) {
+	char *buf;
+	int result;
+
+	if (is_absolute_path(path))
+		return 1;
+	buf = xmalloc(strlen(path) + 1);
+	result = !!normalize_path_copy(buf, path);
+	free(buf);
+	return result;
+}
+
+/*
+ * Check if a refname is safe.
+ * For refs that start with "refs/" we consider it safe as long as the rest
+ * of the path components does not allow it to escape from this directory.
+ * For all other refs we only consider them safe iff they only contain
+ * upper case characters and '_'.
+ */
+static int refname_is_safe(const char *refname)
+{
+	if (starts_with(refname, "refs/"))
+		return !escapes_cwd(refname + strlen("refs/"));
+	while (*refname) {
+		if (!isupper(*refname) && *refname != '_')
+			return 0;
+		refname++;
+	}
+	return 1;
+}
+
 static struct ref_entry *create_ref_entry(const char *refname,
 					  const unsigned char *sha1, int flag,
 					  int check_name)
@@ -288,8 +311,10 @@ static struct ref_entry *create_ref_entry(const char *refname,
 	struct ref_entry *ref;
 
 	if (check_name &&
-	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT))
+	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		die("Reference has invalid format: '%s'", refname);
+	if (!check_name && !refname_is_safe(refname))
+		die("Reference has invalid name: '%s'", refname);
 	len = strlen(refname) + 1;
 	ref = xmalloc(sizeof(struct ref_entry) + len);
 	hashcpy(ref->u.value.sha1, sha1);
@@ -822,10 +847,9 @@ static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
 /*
  * Return true iff a reference named refname could be created without
  * conflicting with the name of an existing reference in dir.  If
- * oldrefname is non-NULL, ignore potential conflicts with oldrefname
- * (e.g., because oldrefname is scheduled for deletion in the same
- * operation). skiplist contains a list of refs we want to skip checking for
- * conflicts with. skiplist must be sorted.
+ * skiplist is non-NULL, ignore potential conflicts with names in
+ * skiplist (e.g., because those refs are scheduled for deletion in
+ * the same operation).  skiplist must be sorted.
  */
 static int is_refname_available(const char *refname,
 				struct ref_dir *dir,
@@ -1062,9 +1086,9 @@ static void read_packed_refs(FILE *f, struct ref_dir *dir)
 		if (refname) {
 			int flag = REF_ISPACKED;
 
-			if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT)) {
-				flag |= REF_ISBROKEN;
+			if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 				hashclr(sha1);
+				flag |= REF_BAD_NAME | REF_ISBROKEN;
 			}
 			last = create_ref_entry(refname, sha1, flag, 0);
 			if (peeled == PEELED_FULLY ||
@@ -1198,16 +1222,16 @@ static void read_loose_refs(const char *dirname, struct ref_dir *dir)
 					hashclr(sha1);
 					flag |= REF_ISBROKEN;
 				}
-			} else if (read_ref_full(refname.buf, sha1, &flag,
-						 RESOLVE_REF_READING)) {
+			} else if (read_ref_full(refname.buf,
+						 RESOLVE_REF_READING,
+						 sha1, &flag)) {
 				hashclr(sha1);
 				flag |= REF_ISBROKEN;
 			}
 			if (check_refname_format(refname.buf,
-						 REFNAME_ALLOW_ONELEVEL|
-						 REFNAME_DOT_COMPONENT)) {
+						 REFNAME_ALLOW_ONELEVEL)) {
 				hashclr(sha1);
-				flag |= REF_ISBROKEN;
+				flag |= REF_BAD_NAME | REF_ISBROKEN;
 			}
 			add_entry_to_dir(dir,
 					 create_ref_entry(refname.buf, sha1, flag, 0));
@@ -1329,10 +1353,10 @@ static struct ref_entry *get_packed_ref(const char *refname)
  * A loose ref file doesn't exist; check for a packed ref.  The
  * options are forwarded from resolve_safe_unsafe().
  */
-static const char *handle_missing_loose_ref(const char *refname,
-					    unsigned char *sha1,
-					    int reading,
-					    int *flag)
+static int resolve_missing_loose_ref(const char *refname,
+				     int resolve_flags,
+				     unsigned char *sha1,
+				     int *flags)
 {
 	struct ref_entry *entry;
 
@@ -1343,33 +1367,22 @@ static const char *handle_missing_loose_ref(const char *refname,
 	entry = get_packed_ref(refname);
 	if (entry) {
 		hashcpy(sha1, entry->u.value.sha1);
-		if (flag)
-			*flag |= REF_ISPACKED;
-		return refname;
+		if (flags)
+			*flags |= REF_ISPACKED;
+		return 0;
 	}
 	/* The reference is not a packed reference, either. */
-	if (reading) {
-		return NULL;
+	if (resolve_flags & RESOLVE_REF_READING) {
+		errno = ENOENT;
+		return -1;
 	} else {
 		hashclr(sha1);
-		return refname;
+		return 0;
 	}
 }
 
-static int escapes_cwd(const char *path) {
-	char *buf;
-	int result;
-
-	if (is_absolute_path(path))
-		return 1;
-	buf = xmalloc(strlen(path) + 1);
-	result = normalize_path_copy(buf, path);
-	free(buf);
-	return result;
-}
-
 /* This function needs to return a meaningful errno on failure */
-const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *flags, int resolve_flags)
+const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned char *sha1, int *flags)
 {
 	int depth = MAXDEPTH;
 	ssize_t len;
@@ -1381,14 +1394,22 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 		*flags = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		if (flags)
+			*flags |= REF_BAD_NAME;
+
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
-		    escapes_cwd(refname)) {
+		    !refname_is_safe(refname)) {
 			errno = EINVAL;
 			return NULL;
 		}
-		hashclr(sha1);
-		if (flags)
-			*flags |= REF_ISBROKEN;
+		/*
+		 * dwim_ref() uses REF_ISBROKEN to distinguish between
+		 * missing refs and refs that were present but invalid,
+		 * to complain about the latter to stderr.
+		 *
+		 * We don't know whether the ref exists, so don't set
+		 * REF_ISBROKEN yet.
+		 */
 		bad_name = 1;
 	}
 	for (;;) {
@@ -1415,12 +1436,17 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 		 */
 	stat_ref:
 		if (lstat(path, &st) < 0) {
-			if (errno == ENOENT)
-				return handle_missing_loose_ref(refname, sha1,
-					resolve_flags & RESOLVE_REF_READING,
-					flags);
-			else
+			if (errno != ENOENT)
 				return NULL;
+			if (resolve_missing_loose_ref(refname, resolve_flags,
+						      sha1, flags))
+				return NULL;
+			if (bad_name) {
+				hashclr(sha1);
+				if (flags)
+					*flags |= REF_ISBROKEN;
+			}
+			return refname;
 		}
 
 		/* Follow "normalized" - ie "refs/.." symlinks by hand */
@@ -1440,7 +1466,7 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 				refname = refname_buffer;
 				if (flags)
 					*flags |= REF_ISSYMREF;
-				if (resolve_flags & RESOLVE_REF_NODEREF) {
+				if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
 					hashclr(sha1);
 					return refname;
 				}
@@ -1493,8 +1519,11 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 				errno = EINVAL;
 				return NULL;
 			}
-			if (bad_name)
+			if (bad_name) {
 				hashclr(sha1);
+				if (flags)
+					*flags |= REF_ISBROKEN;
+			}
 			return refname;
 		}
 		if (flags)
@@ -1502,23 +1531,28 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int *fl
 		buf = buffer + 4;
 		while (isspace(*buf))
 			buf++;
-		if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
-			if (flags)
-				*flags |= REF_ISBROKEN;
-			errno = EINVAL;
-			return NULL;
-		}
 		refname = strcpy(refname_buffer, buf);
-		if (resolve_flags & RESOLVE_REF_NODEREF) {
+		if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
 			hashclr(sha1);
 			return refname;
 		}
+		if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
+			if (flags)
+				*flags |= REF_BAD_NAME | REF_ISBROKEN;
+
+			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+			    !refname_is_safe(buf)) {
+				errno = EINVAL;
+				return NULL;
+			}
+			bad_name = 1;
+		}
 	}
 }
 
-char *resolve_refdup(const char *ref, unsigned char *sha1, int *flags, int resolve_flags)
+char *resolve_refdup(const char *ref, int resolve_flags, unsigned char *sha1, int *flags)
 {
-	const char *ret = resolve_ref_unsafe(ref, sha1, flags, resolve_flags);
+	const char *ret = resolve_ref_unsafe(ref, resolve_flags, sha1, flags);
 	return ret ? xstrdup(ret) : NULL;
 }
 
@@ -1529,22 +1563,22 @@ struct ref_filter {
 	void *cb_data;
 };
 
-int read_ref_full(const char *refname, unsigned char *sha1, int *flags, int resolve_flags)
+int read_ref_full(const char *refname, int resolve_flags, unsigned char *sha1, int *flags)
 {
-	if (resolve_ref_unsafe(refname, sha1, flags, resolve_flags))
+	if (resolve_ref_unsafe(refname, resolve_flags, sha1, flags))
 		return 0;
 	return -1;
 }
 
 int read_ref(const char *refname, unsigned char *sha1)
 {
-	return read_ref_full(refname, sha1, NULL, RESOLVE_REF_READING);
+	return read_ref_full(refname, RESOLVE_REF_READING, sha1, NULL);
 }
 
 int ref_exists(const char *refname)
 {
 	unsigned char sha1[20];
-	return !!resolve_ref_unsafe(refname, sha1, NULL, RESOLVE_REF_READING);
+	return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
 }
 
 static int filter_refs(const char *refname, const unsigned char *sha1, int flags,
@@ -1658,7 +1692,7 @@ int peel_ref(const char *refname, unsigned char *sha1)
 		return 0;
 	}
 
-	if (read_ref_full(refname, base, &flag, RESOLVE_REF_READING))
+	if (read_ref_full(refname, RESOLVE_REF_READING, base, &flag))
 		return -1;
 
 	/*
@@ -1699,7 +1733,7 @@ static int warn_if_dangling_symref(const char *refname, const unsigned char *sha
 	if (!(flags & REF_ISSYMREF))
 		return 0;
 
-	resolves_to = resolve_ref_unsafe(refname, junk, NULL, 0);
+	resolves_to = resolve_ref_unsafe(refname, 0, junk, NULL);
 	if (!resolves_to
 	    || (d->refname
 		? strcmp(resolves_to, d->refname)
@@ -1824,7 +1858,7 @@ static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
 		return 0;
 	}
 
-	if (!read_ref_full("HEAD", sha1, &flag, RESOLVE_REF_READING))
+	if (!read_ref_full("HEAD", RESOLVE_REF_READING, sha1, &flag))
 		return fn("HEAD", sha1, flag, cb_data);
 
 	return 0;
@@ -1904,7 +1938,7 @@ int head_ref_namespaced(each_ref_fn fn, void *cb_data)
 	int flag;
 
 	strbuf_addf(&buf, "%sHEAD", get_git_namespace());
-	if (!read_ref_full(buf.buf, sha1, &flag, RESOLVE_REF_READING))
+	if (!read_ref_full(buf.buf, RESOLVE_REF_READING, sha1, &flag))
 		ret = fn(buf.buf, sha1, flag, cb_data);
 	strbuf_release(&buf);
 
@@ -1999,8 +2033,9 @@ int refname_match(const char *abbrev_name, const char *full_name)
 static struct ref_lock *verify_lock(struct ref_lock *lock,
 	const unsigned char *old_sha1, int mustexist)
 {
-	if (read_ref_full(lock->ref_name, lock->old_sha1, NULL,
-			  mustexist ? RESOLVE_REF_READING : 0)) {
+	if (read_ref_full(lock->ref_name,
+			  mustexist ? RESOLVE_REF_READING : 0,
+			  lock->old_sha1, NULL)) {
 		int save_errno = errno;
 		error("Can't verify ref %s", lock->ref_name);
 		unlock_ref(lock);
@@ -2073,8 +2108,8 @@ int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
 
 		this_result = refs_found ? sha1_from_ref : sha1;
 		mksnpath(fullref, sizeof(fullref), *p, len, str);
-		r = resolve_ref_unsafe(fullref, this_result, &flag,
-				       RESOLVE_REF_READING);
+		r = resolve_ref_unsafe(fullref, RESOLVE_REF_READING,
+				       this_result, &flag);
 		if (r) {
 			if (!refs_found++)
 				*ref = xstrdup(r);
@@ -2103,7 +2138,8 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 		const char *ref, *it;
 
 		mksnpath(path, sizeof(path), *p, len, str);
-		ref = resolve_ref_unsafe(path, hash, NULL, RESOLVE_REF_READING);
+		ref = resolve_ref_unsafe(path, RESOLVE_REF_READING,
+					 hash, NULL);
 		if (!ref)
 			continue;
 		if (reflog_exists(path))
@@ -2145,16 +2181,16 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
-	if (flags & REF_BADNAMEOK)
-		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
-
 	if (mustexist)
 		resolve_flags |= RESOLVE_REF_READING;
-	if (flags & REF_NODEREF)
-		resolve_flags |= RESOLVE_REF_NODEREF;
+	if (flags & REF_DELETING) {
+		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
+		if (flags & REF_NODEREF)
+			resolve_flags |= RESOLVE_REF_NO_RECURSE;
+	}
 
-	refname = resolve_ref_unsafe(refname, lock->old_sha1, &type,
-				     resolve_flags);
+	refname = resolve_ref_unsafe(refname, resolve_flags,
+				     lock->old_sha1, &type);
 	if (!refname && errno == EISDIR) {
 		/* we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -2167,8 +2203,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 			error("there are still refs under '%s'", orig_refname);
 			goto error_return;
 		}
-		refname = resolve_ref_unsafe(orig_refname, lock->old_sha1,
-					     &type, resolve_flags);
+		refname = resolve_ref_unsafe(orig_refname, resolve_flags,
+					     lock->old_sha1, &type);
 	}
 	if (type_p)
 	    *type_p = type;
@@ -2521,7 +2557,7 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 		unsigned char sha1[20];
 		int flags;
 
-		if (read_ref_full(entry->name, sha1, &flags, 0))
+		if (read_ref_full(entry->name, 0, sha1, &flags))
 			/* We should at least have found the packed ref. */
 			die("Internal error");
 		if ((flags & REF_ISSYMREF) || !(flags & REF_ISPACKED)) {
@@ -2712,8 +2748,8 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	if (log && S_ISLNK(loginfo.st_mode))
 		return error("reflog for %s is a symlink", oldrefname);
 
-	symref = resolve_ref_unsafe(oldrefname, orig_sha1, &flag,
-				    RESOLVE_REF_READING);
+	symref = resolve_ref_unsafe(oldrefname, RESOLVE_REF_READING,
+				    orig_sha1, &flag);
 	if (flag & REF_ISSYMREF)
 		return error("refname %s is a symbolic ref, renaming it is not supported",
 			oldrefname);
@@ -2742,7 +2778,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 		goto rollback;
 	}
 
-	if (!read_ref_full(newrefname, sha1, NULL, RESOLVE_REF_READING) &&
+	if (!read_ref_full(newrefname, RESOLVE_REF_READING, sha1, NULL) &&
 	    delete_ref(newrefname, sha1, REF_NODEREF)) {
 		if (errno==EISDIR) {
 			if (remove_empty_directories(git_path("%s", newrefname))) {
@@ -2956,7 +2992,7 @@ static int is_branch(const char *refname)
 }
 
 /*
- * Writes sha1 into the ref specified by the lock. Makes sure that errno
+ * Write sha1 into the ref specified by the lock. Make sure that errno
  * is sane on error.
  */
 static int write_ref_sha1(struct ref_lock *lock,
@@ -3020,8 +3056,8 @@ static int write_ref_sha1(struct ref_lock *lock,
 		unsigned char head_sha1[20];
 		int head_flag;
 		const char *head_ref;
-		head_ref = resolve_ref_unsafe("HEAD", head_sha1, &head_flag,
-					      RESOLVE_REF_READING);
+		head_ref = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+					      head_sha1, &head_flag);
 		if (head_ref && (head_flag & REF_ISSYMREF) &&
 		    !strcmp(head_ref, lock->ref_name))
 			log_ref_write("HEAD", lock->old_sha1, sha1, logmsg);
@@ -3388,7 +3424,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data
 				retval = do_for_each_reflog(name, fn, cb_data);
 			} else {
 				unsigned char sha1[20];
-				if (read_ref_full(name->buf, sha1, NULL, 0))
+				if (read_ref_full(name->buf, 0, sha1, NULL))
 					retval = error("bad ref for %s", name->buf);
 				else
 					retval = fn(name->buf, sha1, 0, cb_data);
@@ -3667,27 +3703,30 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	/* Copy, sort, and reject duplicate refs */
 	qsort(updates, n, sizeof(*updates), ref_update_compare);
 	if (ref_update_reject_duplicates(updates, n, err)) {
-		ret = -1;
+		ret = TRANSACTION_GENERIC_ERROR;
 		goto cleanup;
 	}
 
 	/* Acquire all locks while verifying old values */
 	for (i = 0; i < n; i++) {
 		struct ref_update *update = updates[i];
+		int flags = update->flags;
 
+		if (is_null_sha1(update->new_sha1))
+			flags |= REF_DELETING;
 		update->lock = lock_ref_sha1_basic(update->refname,
 						   (update->have_old ?
 						    update->old_sha1 :
 						    NULL),
 						   NULL,
-						   update->flags,
+						   flags,
 						   &update->type);
 		if (!update->lock) {
-			int df_conflict = (errno == ENOTDIR);
-
+			ret = (errno == ENOTDIR)
+				? TRANSACTION_NAME_CONFLICT
+				: TRANSACTION_GENERIC_ERROR;
 			strbuf_addf(err, "Cannot lock the ref '%s'.",
 				    update->refname);
-			ret = df_conflict ? UPDATE_REFS_NAME_CONFLICT : -1;
 			goto cleanup;
 		}
 	}
@@ -3697,15 +3736,15 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (!is_null_sha1(update->new_sha1)) {
-			ret = write_ref_sha1(update->lock, update->new_sha1,
-					     update->msg);
-			update->lock = NULL; /* freed by write_ref_sha1 */
-			if (ret) {
+			if (write_ref_sha1(update->lock, update->new_sha1,
+					   update->msg)) {
+				update->lock = NULL; /* freed by write_ref_sha1 */
 				strbuf_addf(err, "Cannot update the ref '%s'.",
 					    update->refname);
-				ret = -1;
+				ret = TRANSACTION_GENERIC_ERROR;
 				goto cleanup;
 			}
+			update->lock = NULL; /* freed by write_ref_sha1 */
 		}
 	}
 
@@ -3715,7 +3754,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 
 		if (update->lock) {
 			if (delete_ref_loose(update->lock, update->type, err)) {
-				ret = -1;
+				ret = TRANSACTION_GENERIC_ERROR;
 				goto cleanup;
 			}
 
@@ -3725,7 +3764,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	}
 
 	if (repack_without_refs(delnames, delnum, err)) {
-		ret = -1;
+		ret = TRANSACTION_GENERIC_ERROR;
 		goto cleanup;
 	}
 	for (i = 0; i < delnum; i++)
diff --git c/refs.h w/refs.h
index a96e174..3b35387 100644
--- c/refs.h
+++ w/refs.h
@@ -56,11 +56,15 @@ struct ref_transaction;
 
 /*
  * Reference cannot be resolved to an object name: dangling symbolic
- * reference (directly or indirectly), corrupt reference file, or
- * symbolic reference refers to ill-formatted reference name.
+ * reference (directly or indirectly), corrupt reference file,
+ * reference exists but name is bad, or symbolic reference refers to
+ * ill-formatted reference name.
  */
 #define REF_ISBROKEN 0x04
 
+/* Reference name is not well formed (see git-check-ref-format(1)). */
+#define REF_BAD_NAME 0x08
+
 /*
  * The signature for the callback function for the for_each_*()
  * functions below.  The memory pointed to by the refname and sha1
@@ -175,12 +179,12 @@ extern int peel_ref(const char *refname, unsigned char *sha1);
  * ref_transaction_create(), etc.
  * REF_NODEREF: act on the ref directly, instead of dereferencing
  *              symbolic references.
- * REF_BADNAMEOK: allow locking a ref that has a bad name.
+ * REF_DELETING: tolerate broken refs
  *
  * Flags >= 0x100 are reserved for internal use.
  */
 #define REF_NODEREF	0x01
-#define REF_BADNAMEOK	0x02
+#define REF_DELETING	0x02
 /*
  * This function sets errno to something meaningful on failure.
  */
@@ -226,7 +230,6 @@ extern int for_each_reflog(each_ref_fn, void *);
 
 #define REFNAME_ALLOW_ONELEVEL 1
 #define REFNAME_REFSPEC_PATTERN 2
-#define REFNAME_DOT_COMPONENT 4
 
 /*
  * Return 0 iff refname has the correct format for a refname according
@@ -234,10 +237,7 @@ extern int for_each_reflog(each_ref_fn, void *);
  * If REFNAME_ALLOW_ONELEVEL is set in flags, then accept one-level
  * reference names.  If REFNAME_REFSPEC_PATTERN is set in flags, then
  * allow a "*" wildcard character in place of one of the name
- * components.  No leading or repeated slashes are accepted.  If
- * REFNAME_DOT_COMPONENT is set in flags, then allow refname
- * components to start with "." (but not a whole component equal to
- * "." or "..").
+ * components.  No leading or repeated slashes are accepted.
  */
 extern int check_refname_format(const char *refname, int flags);
 
@@ -270,8 +270,8 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * The following functions add a reference check or update to a
  * ref_transaction.  In all of them, refname is the name of the
  * reference to be affected.  The functions make internal copies of
- * refname, so the caller retains ownership of the parameter.  flags
- * can be REF_NODEREF; it is passed to update_ref_lock().
+ * refname and msg, so the caller retains ownership of these parameters.
+ * flags can be REF_NODEREF; it is passed to update_ref_lock().
  */
 
 /*
@@ -322,14 +322,14 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 
 /*
  * Commit all of the changes that have been queued in transaction, as
- * atomically as possible.  Return a nonzero value if there is a
- * problem.
+ * atomically as possible.
  *
- * Function returns 0 on success, -1 for generic failures and
- * UPDATE_REFS_NAME_CONFLICT (-2) if the failure was due to a naming conflict.
- * For example, the ref names A and A/B conflict.
+ * Returns 0 for success, or one of the below error codes for errors.
  */
-#define UPDATE_REFS_NAME_CONFLICT -2
+/* Naming conflict (for example, the ref names A and A/B conflict). */
+#define TRANSACTION_NAME_CONFLICT -1
+/* All other errors. */
+#define TRANSACTION_GENERIC_ERROR -2
 int ref_transaction_commit(struct ref_transaction *transaction,
 			   struct strbuf *err);
 
diff --git c/remote.c w/remote.c
index 67c375d..25b07ac 100644
--- c/remote.c
+++ w/remote.c
@@ -486,7 +486,7 @@ static void read_config(void)
 		return;
 	default_remote_name = "origin";
 	current_branch = NULL;
-	head_ref = resolve_ref_unsafe("HEAD", sha1, &flag, 0);
+	head_ref = resolve_ref_unsafe("HEAD", 0, sha1, &flag);
 	if (head_ref && (flag & REF_ISSYMREF) &&
 	    starts_with(head_ref, "refs/heads/")) {
 		current_branch =
@@ -1121,8 +1121,8 @@ static char *guess_ref(const char *name, struct ref *peer)
 	struct strbuf buf = STRBUF_INIT;
 	unsigned char sha1[20];
 
-	const char *r = resolve_ref_unsafe(peer->name, sha1, NULL,
-					   RESOLVE_REF_READING);
+	const char *r = resolve_ref_unsafe(peer->name, RESOLVE_REF_READING,
+					   sha1, NULL);
 	if (!r)
 		return NULL;
 
@@ -1183,8 +1183,9 @@ static int match_explicit(struct ref *src, struct ref *dst,
 		unsigned char sha1[20];
 		int flag;
 
-		dst_value = resolve_ref_unsafe(matched_src->name, sha1, &flag,
-					       RESOLVE_REF_READING);
+		dst_value = resolve_ref_unsafe(matched_src->name,
+					       RESOLVE_REF_READING,
+					       sha1, &flag);
 		if (!dst_value ||
 		    ((flag & REF_ISSYMREF) &&
 		     !starts_with(dst_value, "refs/heads/")))
@@ -1658,7 +1659,7 @@ static int ignore_symref_update(const char *refname)
 	unsigned char sha1[20];
 	int flag;
 
-	if (!resolve_ref_unsafe(refname, sha1, &flag, 0))
+	if (!resolve_ref_unsafe(refname, 0, sha1, &flag))
 		return 0; /* non-existing refs are OK */
 	return (flag & REF_ISSYMREF);
 }
diff --git c/sequencer.c w/sequencer.c
index 6a05ad4..70fb7a8 100644
--- c/sequencer.c
+++ w/sequencer.c
@@ -366,7 +366,7 @@ static int is_index_unchanged(void)
 	unsigned char head_sha1[20];
 	struct commit *head_commit;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, NULL, RESOLVE_REF_READING))
+	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
 		return error(_("Could not resolve HEAD commit\n"));
 
 	head_commit = lookup_commit(head_sha1);
@@ -912,7 +912,7 @@ static int rollback_single_pick(void)
 	if (!file_exists(git_path("CHERRY_PICK_HEAD")) &&
 	    !file_exists(git_path("REVERT_HEAD")))
 		return error(_("no cherry-pick or revert in progress"));
-	if (read_ref_full("HEAD", head_sha1, NULL, 0))
+	if (read_ref_full("HEAD", 0, head_sha1, NULL))
 		return error(_("cannot resolve HEAD"));
 	if (is_null_sha1(head_sha1))
 		return error(_("cannot abort from a branch yet to be born"));
diff --git c/t/t1400-update-ref.sh w/t/t1400-update-ref.sh
index ff4607b..7b4707b 100755
--- c/t/t1400-update-ref.sh
+++ w/t/t1400-update-ref.sh
@@ -126,6 +126,16 @@ test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	test_path_is_missing .git/refs/heads/self
 '
 
+test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
+	>.git/refs/heads/bad &&
+	test_when_finished "rm -f .git/refs/heads/bad" &&
+	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
+	test_when_finished "rm -f .git/refs/heads/ref-to-bad" &&
+	test_path_is_file .git/refs/heads/ref-to-bad &&
+	git update-ref --no-deref -d refs/heads/ref-to-bad &&
+	test_path_is_missing .git/refs/heads/ref-to-bad
+'
+
 test_expect_success '(not) create HEAD with old sha1' "
 	test_must_fail git update-ref HEAD $A $B
 "
@@ -390,12 +400,6 @@ test_expect_success 'stdin fails create with no ref' '
 	grep "fatal: create: missing <ref>" err
 '
 
-test_expect_success 'stdin fails create with bad ref name' '
-	echo "create ~a $m" >stdin &&
-	test_must_fail git update-ref --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin fails create with no new value' '
 	echo "create $a" >stdin &&
 	test_must_fail git update-ref --stdin <stdin 2>err &&
@@ -414,12 +418,6 @@ test_expect_success 'stdin fails update with no ref' '
 	grep "fatal: update: missing <ref>" err
 '
 
-test_expect_success 'stdin fails update with bad ref name' '
-	echo "update ~a $m" >stdin &&
-	test_must_fail git update-ref --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin fails update with no new value' '
 	echo "update $a" >stdin &&
 	test_must_fail git update-ref --stdin <stdin 2>err &&
@@ -438,12 +436,6 @@ test_expect_success 'stdin fails delete with no ref' '
 	grep "fatal: delete: missing <ref>" err
 '
 
-test_expect_success 'stdin fails delete with bad ref name' '
-	echo "delete ~a $m" >stdin &&
-	test_must_fail git update-ref --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin fails delete with too many arguments' '
 	echo "delete $a $m $m" >stdin &&
 	test_must_fail git update-ref --stdin <stdin 2>err &&
@@ -716,12 +708,6 @@ test_expect_success 'stdin -z fails create with no ref' '
 	grep "fatal: create: missing <ref>" err
 '
 
-test_expect_success 'stdin -z fails create with bad ref name' '
-	printf $F "create ~a " "$m" >stdin &&
-	test_must_fail git update-ref -z --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a " err
-'
-
 test_expect_success 'stdin -z fails create with no new value' '
 	printf $F "create $a" >stdin &&
 	test_must_fail git update-ref -z --stdin <stdin 2>err &&
@@ -746,12 +732,6 @@ test_expect_success 'stdin -z fails update with too few args' '
 	grep "fatal: update $a: unexpected end of input when reading <oldvalue>" err
 '
 
-test_expect_success 'stdin -z fails update with bad ref name' '
-	printf $F "update ~a" "$m" "" >stdin &&
-	test_must_fail git update-ref -z --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin -z emits warning with empty new value' '
 	git update-ref $a $m &&
 	printf $F "update $a" "" "" >stdin &&
@@ -784,12 +764,6 @@ test_expect_success 'stdin -z fails delete with no ref' '
 	grep "fatal: delete: missing <ref>" err
 '
 
-test_expect_success 'stdin -z fails delete with bad ref name' '
-	printf $F "delete ~a" "$m" >stdin &&
-	test_must_fail git update-ref -z --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin -z fails delete with no old value' '
 	printf $F "delete $a" >stdin &&
 	test_must_fail git update-ref -z --stdin <stdin 2>err &&
diff --git c/t/t1402-check-ref-format.sh w/t/t1402-check-ref-format.sh
index 058fa37..1a5a5f3 100755
--- c/t/t1402-check-ref-format.sh
+++ w/t/t1402-check-ref-format.sh
@@ -196,52 +196,4 @@ invalid_ref_normalized 'heads///foo.lock'
 invalid_ref_normalized 'foo.lock/bar'
 invalid_ref_normalized 'foo.lock///bar'
 
-test_expect_success 'git branch shows badly named ref' '
-	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
-	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
-	git branch >output &&
-	grep -e "broken...ref" output
-'
-
-test_expect_success 'git branch -d can delete badly named ref' '
-	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
-	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
-	git branch -d broken...ref &&
-	git branch >output &&
-	! grep -e "broken...ref" output
-'
-
-test_expect_success 'git branch -D can delete badly named ref' '
-	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
-	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
-	git branch -D broken...ref &&
-	git branch >output &&
-	! grep -e "broken...ref" output
-'
-
-test_expect_success 'git update-ref -d can delete badly named ref' '
-	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
-	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
-	git update-ref -d refs/heads/broken...ref &&
-	git branch >output &&
-	! grep -e "broken...ref" output
-'
-
-test_expect_success 'git branch can not create a badly named ref' '
-	test_must_fail git branch broken...ref
-'
-
-test_expect_success 'git branch can not rename to a bad ref name' '
-	git branch goodref &&
-	test_must_fail git branch -m goodref broken...ref
-'
-
-test_expect_failure 'git branch can rename from a bad ref name' '
-	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
-	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
-	git branch -m broken...ref renamed &&
-	test_must_fail git rev-parse broken...ref &&
-	test_cmp_rev master renamed
-'
-
 test_done
diff --git c/t/t1413-reflog-detach.sh w/t/t1413-reflog-detach.sh
new file mode 100755
index 0000000..c730600
--- /dev/null
+++ w/t/t1413-reflog-detach.sh
@@ -0,0 +1,70 @@
+#!/bin/sh
+
+test_description='Test reflog interaction with detached HEAD'
+. ./test-lib.sh
+
+reset_state () {
+	git checkout master &&
+	cp saved_reflog .git/logs/HEAD
+}
+
+test_expect_success setup '
+	test_tick &&
+	git commit --allow-empty -m initial &&
+	git branch side &&
+	test_tick &&
+	git commit --allow-empty -m second &&
+	cat .git/logs/HEAD >saved_reflog
+'
+
+test_expect_success baseline '
+	reset_state &&
+	git rev-parse master master^ >expect &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'switch to branch' '
+	reset_state &&
+	git rev-parse side master master^ >expect &&
+	git checkout side &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'detach to other' '
+	reset_state &&
+	git rev-parse master side master master^ >expect &&
+	git checkout side &&
+	git checkout master^0 &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'detach to self' '
+	reset_state &&
+	git rev-parse master master master^ >expect &&
+	git checkout master^0 &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'attach to self' '
+	reset_state &&
+	git rev-parse master master master master^ >expect &&
+	git checkout master^0 &&
+	git checkout master &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'attach to other' '
+	reset_state &&
+	git rev-parse side master master master^ >expect &&
+	git checkout master^0 &&
+	git checkout side &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_done
diff --git c/t/t1430-bad-ref-name.sh w/t/t1430-bad-ref-name.sh
new file mode 100755
index 0000000..468e856
--- /dev/null
+++ w/t/t1430-bad-ref-name.sh
@@ -0,0 +1,207 @@
+#!/bin/sh
+
+test_description='Test handling of ref names that check-ref-format rejects'
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit one &&
+	test_commit two
+'
+
+test_expect_success 'fast-import: fail on invalid branch name ".badbranchname"' '
+	test_when_finished "rm -f .git/objects/pack_* .git/objects/index_*" &&
+	cat >input <<-INPUT_END &&
+		commit .badbranchname
+		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+		data <<COMMIT
+		corrupt
+		COMMIT
+
+		from refs/heads/master
+
+	INPUT_END
+	test_must_fail git fast-import <input
+'
+
+test_expect_success 'fast-import: fail on invalid branch name "bad[branch]name"' '
+	test_when_finished "rm -f .git/objects/pack_* .git/objects/index_*" &&
+	cat >input <<-INPUT_END &&
+		commit bad[branch]name
+		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+		data <<COMMIT
+		corrupt
+		COMMIT
+
+		from refs/heads/master
+
+	INPUT_END
+	test_must_fail git fast-import <input
+'
+
+test_expect_success 'git branch shows badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch >output &&
+	grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -d can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -d broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -D can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -D broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -D cannot delete non-ref in .git dir' '
+	echo precious >.git/my-private-file &&
+	echo precious >expect &&
+	test_must_fail git branch -D ../../my-private-file &&
+	test_cmp expect .git/my-private-file
+'
+
+test_expect_success 'branch -D cannot delete absolute path' '
+	git branch -f extra &&
+	test_must_fail git branch -D "$(pwd)/.git/refs/heads/extra" &&
+	test_cmp_rev HEAD extra
+'
+
+test_expect_success 'git branch cannot create a badly named ref' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	test_must_fail git branch broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -m cannot rename to a bad ref name' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	test_might_fail git branch -D goodref &&
+	git branch goodref &&
+	test_must_fail git branch -m goodref broken...ref &&
+	test_cmp_rev master goodref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_failure 'branch -m can rename from a bad ref name' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -m broken...ref renamed &&
+	test_cmp_rev master renamed &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'push cannot create a badly named ref' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	test_must_fail git push "file://$(pwd)" HEAD:refs/heads/broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_failure 'push --mirror can delete badly named ref' '
+	top=$(pwd) &&
+	git init src &&
+	git init dest &&
+
+	(
+		cd src &&
+		test_commit one
+	) &&
+	(
+		cd dest &&
+		test_commit two &&
+		git checkout --detach &&
+		cp .git/refs/heads/master .git/refs/heads/broken...ref
+	) &&
+	git -C src push --mirror "file://$top/dest" &&
+	git -C dest branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'rev-parse skips symref pointing to broken name' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch shadow one &&
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	git symbolic-ref refs/tags/shadow refs/heads/broken...ref &&
+
+	git rev-parse --verify one >expect &&
+	git rev-parse --verify shadow >actual 2>err &&
+	test_cmp expect actual &&
+	test_i18ngrep "ignoring.*refs/tags/shadow" err
+'
+
+test_expect_success 'update-ref --no-deref -d can delete reference to broken name' '
+	git symbolic-ref refs/heads/badname refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/badname" &&
+	test_path_is_file .git/refs/heads/badname &&
+	git update-ref --no-deref -d refs/heads/badname &&
+	test_path_is_missing .git/refs/heads/badname
+'
+
+test_expect_success 'update-ref -d can delete broken name' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git update-ref -d refs/heads/broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'update-ref -d cannot delete non-ref in .git dir' '
+	echo precious >.git/my-private-file &&
+	echo precious >expect &&
+	test_must_fail git update-ref -d my-private-file &&
+	test_cmp expect .git/my-private-file
+'
+
+test_expect_success 'update-ref -d cannot delete absolute path' '
+	git branch -f extra &&
+	test_must_fail git update-ref -d "$(pwd)/.git/refs/heads/extra" &&
+	test_cmp_rev HEAD extra
+'
+
+test_expect_success 'update-ref --stdin fails create with bad ref name' '
+	echo "create ~a refs/heads/master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin fails update with bad ref name' '
+	echo "update ~a refs/heads/master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin fails delete with bad ref name' '
+	echo "delete ~a refs/heads/master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin -z fails create with bad ref name' '
+	printf "%s\0" "create ~a " refs/heads/master >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a " err
+'
+
+test_expect_success 'update-ref --stdin -z fails update with bad ref name' '
+	printf "%s\0" "update ~a" refs/heads/master "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin -z fails delete with bad ref name' '
+	printf "%s\0" "delete ~a" refs/heads/master >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_done
diff --git c/t/t9300-fast-import.sh w/t/t9300-fast-import.sh
index 5fc9ef2..3d156f9 100755
--- c/t/t9300-fast-import.sh
+++ w/t/t9300-fast-import.sh
@@ -347,36 +347,6 @@ test_expect_success 'B: fail on invalid blob sha1' '
 rm -f .git/objects/pack_* .git/objects/index_*
 
 cat >input <<INPUT_END
-commit .badbranchname
-committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
-data <<COMMIT
-corrupt
-COMMIT
-
-from refs/heads/master
-
-INPUT_END
-test_expect_success 'B: fail on invalid branch name ".badbranchname"' '
-    test_must_fail git fast-import <input
-'
-rm -f .git/objects/pack_* .git/objects/index_*
-
-cat >input <<INPUT_END
-commit bad[branch]name
-committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
-data <<COMMIT
-corrupt
-COMMIT
-
-from refs/heads/master
-
-INPUT_END
-test_expect_success 'B: fail on invalid branch name "bad[branch]name"' '
-    test_must_fail git fast-import <input
-'
-rm -f .git/objects/pack_* .git/objects/index_*
-
-cat >input <<INPUT_END
 commit TEMP_TAG
 committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 data <<COMMIT
diff --git c/transport-helper.c w/transport-helper.c
index 8365441..3497a5e 100644
--- c/transport-helper.c
+++ w/transport-helper.c
@@ -890,8 +890,9 @@ static int push_refs_with_export(struct transport *transport,
 
 					/* Follow symbolic refs (mainly for HEAD). */
 					name = resolve_ref_unsafe(
-						 ref->peer_ref->name, sha1,
-						 &flag, RESOLVE_REF_READING);
+						 ref->peer_ref->name,
+						 RESOLVE_REF_READING,
+						 sha1, &flag);
 					if (!name || !(flag & REF_ISSYMREF))
 						name = ref->peer_ref->name;
 
diff --git c/transport.c w/transport.c
index 3ba7bbf..76e0a9a 100644
--- c/transport.c
+++ w/transport.c
@@ -168,8 +168,8 @@ static void set_upstreams(struct transport *transport, struct ref *refs,
 		/* Follow symbolic refs (mainly for HEAD). */
 		localname = ref->peer_ref->name;
 		remotename = ref->name;
-		tmp = resolve_ref_unsafe(localname, sha, &flag,
-					 RESOLVE_REF_READING);
+		tmp = resolve_ref_unsafe(localname, RESOLVE_REF_READING,
+					 sha, &flag);
 		if (tmp && flag & REF_ISSYMREF &&
 			starts_with(tmp, "refs/heads/"))
 			localname = tmp;
@@ -754,7 +754,7 @@ void transport_print_push_status(const char *dest, struct ref *refs,
 	unsigned char head_sha1[20];
 	char *head;
 
-	head = resolve_refdup("HEAD", head_sha1, NULL, RESOLVE_REF_READING);
+	head = resolve_refdup("HEAD", RESOLVE_REF_READING, head_sha1, NULL);
 
 	if (verbose) {
 		for (ref = refs; ref; ref = ref->next)
diff --git c/upload-pack.c w/upload-pack.c
index 3b51ccb..4542565 100644
--- c/upload-pack.c
+++ w/upload-pack.c
@@ -743,7 +743,7 @@ static int find_symref(const char *refname, const unsigned char *sha1, int flag,
 
 	if ((flag & REF_ISSYMREF) == 0)
 		return 0;
-	symref_target = resolve_ref_unsafe(refname, unused, &flag, 0);
+	symref_target = resolve_ref_unsafe(refname, 0, unused, &flag);
 	if (!symref_target || (flag & REF_ISSYMREF) == 0)
 		die("'%s' is a symref but it is not?", refname);
 	item = string_list_append(cb_data, refname);
diff --git c/wt-status.c w/wt-status.c
index 6819e2b..c3cbf50 100644
--- c/wt-status.c
+++ w/wt-status.c
@@ -128,7 +128,7 @@ void wt_status_prepare(struct wt_status *s)
 	s->show_untracked_files = SHOW_NORMAL_UNTRACKED_FILES;
 	s->use_color = -1;
 	s->relative_paths = 1;
-	s->branch = resolve_refdup("HEAD", sha1, NULL, 0);
+	s->branch = resolve_refdup("HEAD", 0, sha1, NULL);
 	s->reference = "HEAD";
 	s->fp = stdout;
 	s->index_file = get_index_file();

[PATCH 01/24] mv test: recreate mod/ directory instead of relying on stale copy

[Jonathan Nieder]

Date: Wed, 10 Sep 2014 14:01:46 -0700

The tests for 'git mv moves a submodule' functionality often run
commands like

	git mv sub mod/sub

to move a submodule into a subdirectory.  Just like plain /bin/mv,
this is supposed to succeed if the mod/ parent directory exists
and fail if it doesn't exist.

Usually these tests mkdir the parent directory beforehand, but some
instead rely on it being left behind by previous tests.

More precisely, when 'git reset --hard' tries to move to a state where
mod/sub is not present any more, it would perform the following
operations:

	rmdir("mod/sub")
	rmdir("mod")

The first fails with ENOENT because the test script removed mod/sub
with "rm -rf" already, so 'reset --hard' doesn't bother to move on to
the second, and the mod/ directory is kept around.

Better to explicitly remove and re-create the mod/ directory so later
tests don't have to depend on the directory left behind by the earlier
ones at all (making it easier to rearrange or skip some tests in the
file or to tweak 'reset --hard' behavior without breaking unrelated
tests).

Noticed while testing a patch that fixes the reset --hard behavior
described above.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
As before.

 t/t7001-mv.sh | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/t/t7001-mv.sh b/t/t7001-mv.sh
index 54d7807..69f11bd 100755
--- a/t/t7001-mv.sh
+++ b/t/t7001-mv.sh
@@ -350,10 +350,11 @@ test_expect_success 'git mv moves a submodule with a .git directory and .gitmodu
 '
 
 test_expect_success 'git mv moves a submodule with gitfile' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	(
 		cd mod &&
 		git mv ../sub/ .
@@ -372,11 +373,12 @@ test_expect_success 'git mv moves a submodule with gitfile' '
 '
 
 test_expect_success 'mv does not complain when no .gitmodules file is found' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git rm .gitmodules &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	git mv sub mod/sub 2>actual.err &&
 	! test -s actual.err &&
 	! test -e sub &&
@@ -390,11 +392,12 @@ test_expect_success 'mv does not complain when no .gitmodules file is found' '
 '
 
 test_expect_success 'mv will error out on a modified .gitmodules file unless staged' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git config -f .gitmodules foo.bar true &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
+	mkdir mod &&
 	test_must_fail git mv sub mod/sub 2>actual.err &&
 	test -s actual.err &&
 	test -e sub &&
@@ -413,13 +416,14 @@ test_expect_success 'mv will error out on a modified .gitmodules file unless sta
 '
 
 test_expect_success 'mv issues a warning when section is not found in .gitmodules' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
 	git config -f .gitmodules --remove-section submodule.sub &&
 	git add .gitmodules &&
 	entry="$(git ls-files --stage sub | cut -f 1)" &&
 	echo "warning: Could not find section in .gitmodules where path=sub" >expect.err &&
+	mkdir mod &&
 	git mv sub mod/sub 2>actual.err &&
 	test_i18ncmp expect.err actual.err &&
 	! test -e sub &&
@@ -433,9 +437,10 @@ test_expect_success 'mv issues a warning when section is not found in .gitmodule
 '
 
 test_expect_success 'mv --dry-run does not touch the submodule or .gitmodules' '
-	rm -rf mod/sub &&
+	rm -rf mod &&
 	git reset --hard &&
 	git submodule update &&
+	mkdir mod &&
 	git mv -n sub mod/sub 2>actual.err &&
 	test -f sub/.git &&
 	git diff-index --exit-code HEAD &&
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 02/24] wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as success

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Jul 2014 11:01:18 -0700

Simplify the function warn_if_unremovable slightly. Additionally, change
behaviour slightly. If we failed to remove the object because the object
does not exist, we can still return success back to the caller since none of
the callers depend on "fail if the file did not exist".

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Change since v21:
- adjust caller to remove redundant errno check

 git-compat-util.h |  7 +++++--
 refs.c            |  2 +-
 wrapper.c         | 14 ++++++--------
 3 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/git-compat-util.h b/git-compat-util.h
index b6f03b3..611e77b 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -706,11 +706,14 @@ void git_qsort(void *base, size_t nmemb, size_t size,
 
 /*
  * Preserves errno, prints a message, but gives no warning for ENOENT.
- * Always returns the return value of unlink(2).
+ * Returns 0 on success which includes trying to unlink an object that does
+ * not exist.
  */
 int unlink_or_warn(const char *path);
 /*
- * Likewise for rmdir(2).
+ * Preserves errno, prints a message, but gives no warning for ENOENT.
+ * Returns 0 on success which includes trying to remove a directory that does
+ * not exist.
  */
 int rmdir_or_warn(const char *path);
 /*
diff --git a/refs.c b/refs.c
index 7235574..d0565b1 100644
--- a/refs.c
+++ b/refs.c
@@ -2557,7 +2557,7 @@ static int delete_ref_loose(struct ref_lock *lock, int flag)
 		lock->lk->filename[i] = 0;
 		err = unlink_or_warn(lock->lk->filename);
 		lock->lk->filename[i] = '.';
-		if (err && errno != ENOENT)
+		if (err)
 			return 1;
 	}
 	return 0;
diff --git a/wrapper.c b/wrapper.c
index bc1bfb8..c9605cd 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -429,14 +429,12 @@ int xmkstemp_mode(char *template, int mode)
 
 static int warn_if_unremovable(const char *op, const char *file, int rc)
 {
-	if (rc < 0) {
-		int err = errno;
-		if (ENOENT != err) {
-			warning("unable to %s %s: %s",
-				op, file, strerror(errno));
-			errno = err;
-		}
-	}
+	int err;
+	if (!rc || errno == ENOENT)
+		return 0;
+	err = errno;
+	warning("unable to %s %s: %s", op, file, strerror(errno));
+	errno = err;
 	return rc;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 03/24] wrapper.c: add a new function unlink_or_msg

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 16 Jul 2014 11:20:36 -0700

This behaves like unlink_or_warn except that on failure it writes the message
to its 'err' argument, which the caller can display in an appropriate way or
ignore.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
---
As before.

 git-compat-util.h |  9 +++++++++
 wrapper.c         | 14 ++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index 611e77b..5ee140c 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -307,6 +307,8 @@ extern char *gitbasename(char *);
 
 #include "wildmatch.h"
 
+struct strbuf;
+
 /* General helper functions */
 extern void vreportf(const char *prefix, const char *err, va_list params);
 extern void vwritef(int fd, const char *prefix, const char *err, va_list params);
@@ -710,6 +712,13 @@ void git_qsort(void *base, size_t nmemb, size_t size,
  * not exist.
  */
 int unlink_or_warn(const char *path);
+ /*
+  * Tries to unlink file.  Returns 0 if unlink succeeded
+  * or the file already didn't exist.  Returns -1 and
+  * appends a message to err suitable for
+  * 'error("%s", err->buf)' on error.
+  */
+int unlink_or_msg(const char *file, struct strbuf *err);
 /*
  * Preserves errno, prints a message, but gives no warning for ENOENT.
  * Returns 0 on success which includes trying to remove a directory that does
diff --git a/wrapper.c b/wrapper.c
index c9605cd..ec7a08b 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -438,6 +438,20 @@ static int warn_if_unremovable(const char *op, const char *file, int rc)
 	return rc;
 }
 
+int unlink_or_msg(const char *file, struct strbuf *err)
+{
+	int rc = unlink(file);
+
+	assert(err);
+
+	if (!rc || errno == ENOENT)
+		return 0;
+
+	strbuf_addf(err, "unable to unlink %s: %s",
+		    file, strerror(errno));
+	return -1;
+}
+
 int unlink_or_warn(const char *file)
 {
 	return warn_if_unremovable("unlink", file, unlink(file));
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 04/24] refs.c: add an err argument to delete_ref_loose

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 15 May 2014 08:25:23 -0700

Add an err argument to delete_loose_ref so that we can pass a descriptive
error string back to the caller. Pass the err argument from transaction
commit to this function so that transaction users will have a nice error
string if the transaction failed due to delete_ref_loose.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Changes since v21:
- s/delete_loose_ref/delete_ref_loose/ once in commit message (but
  the other one still needs fixing)

 refs.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index d0565b1..5609622 100644
--- a/refs.c
+++ b/refs.c
@@ -2548,16 +2548,16 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 	return ret;
 }
 
-static int delete_ref_loose(struct ref_lock *lock, int flag)
+static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
 		/* loose */
-		int err, i = strlen(lock->lk->filename) - 5; /* .lock */
+		int res, i = strlen(lock->lk->filename) - 5; /* .lock */
 
 		lock->lk->filename[i] = 0;
-		err = unlink_or_warn(lock->lk->filename);
+		res = unlink_or_msg(lock->lk->filename, err);
 		lock->lk->filename[i] = '.';
-		if (err)
+		if (res)
 			return 1;
 	}
 	return 0;
@@ -3604,7 +3604,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			ret |= delete_ref_loose(update->lock, update->type);
+			ret |= delete_ref_loose(update->lock, update->type,
+						err);
 			if (!(update->flags & REF_ISPRUNING))
 				delnames[delnum++] = update->lock->ref_name;
 		}
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 05/24] refs.c: pass the ref log message to _create/delete/update instead of _commit

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 12:22:42 -0700

Change the ref transaction API so that we pass the reflog message to the
create/delete/update functions instead of to ref_transaction_commit.
This allows different reflog messages for each ref update in a multi-ref
transaction.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Changes since v21:
- cleaned up commit message
- clarified ownership of msg in comment in refs.h

 branch.c               |  4 ++--
 builtin/commit.c       |  4 ++--
 builtin/receive-pack.c |  5 +++--
 builtin/replace.c      |  5 +++--
 builtin/tag.c          |  4 ++--
 builtin/update-ref.c   | 13 +++++++------
 fast-import.c          |  8 ++++----
 refs.c                 | 34 +++++++++++++++++++++-------------
 refs.h                 | 12 ++++++------
 sequencer.c            |  4 ++--
 walker.c               |  5 ++---
 11 files changed, 54 insertions(+), 44 deletions(-)

diff --git a/branch.c b/branch.c
index 37ac555..76a8ec9 100644
--- a/branch.c
+++ b/branch.c
@@ -301,8 +301,8 @@ void create_branch(const char *head,
 		transaction = ref_transaction_begin(&err);
 		if (!transaction ||
 		    ref_transaction_update(transaction, ref.buf, sha1,
-					   null_sha1, 0, !forcing, &err) ||
-		    ref_transaction_commit(transaction, msg, &err))
+					   null_sha1, 0, !forcing, msg, &err) ||
+		    ref_transaction_commit(transaction, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
 		strbuf_release(&err);
diff --git a/builtin/commit.c b/builtin/commit.c
index 9bf1003..d23e876 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1767,8 +1767,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	    ref_transaction_update(transaction, "HEAD", sha1,
 				   current_head
 				   ? current_head->object.sha1 : NULL,
-				   0, !!current_head, &err) ||
-	    ref_transaction_commit(transaction, sb.buf, &err)) {
+				   0, !!current_head, sb.buf, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		rollback_index_files();
 		die("%s", err.buf);
 	}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 224fadc..d1f4cf7 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -585,8 +585,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		transaction = ref_transaction_begin(&err);
 		if (!transaction ||
 		    ref_transaction_update(transaction, namespaced_name,
-					   new_sha1, old_sha1, 0, 1, &err) ||
-		    ref_transaction_commit(transaction, "push", &err)) {
+					   new_sha1, old_sha1, 0, 1, "push",
+					   &err) ||
+		    ref_transaction_commit(transaction, &err)) {
 			ref_transaction_free(transaction);
 
 			rp_error("%s", err.buf);
diff --git a/builtin/replace.c b/builtin/replace.c
index 1fcd06d..9d03b84 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -169,8 +169,9 @@ static int replace_object_sha1(const char *object_ref,
 
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref, repl, prev, 0, 1, &err) ||
-	    ref_transaction_commit(transaction, NULL, &err))
+	    ref_transaction_update(transaction, ref, repl, prev,
+				   0, 1, NULL, &err) ||
+	    ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 
 	ref_transaction_free(transaction);
diff --git a/builtin/tag.c b/builtin/tag.c
index f3f172f..70d28c5 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -705,8 +705,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, object, prev,
-				   0, 1, &err) ||
-	    ref_transaction_commit(transaction, NULL, &err))
+				   0, 1, NULL, &err) ||
+	    ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
 	if (force && !is_null_sha1(prev) && hashcmp(prev, object))
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 54a48c0..6c9be05 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -14,6 +14,7 @@ static const char * const git_update_ref_usage[] = {
 
 static char line_termination = '\n';
 static int update_flags;
+static const char *msg;
 
 /*
  * Parse one whitespace- or NUL-terminated, possibly C-quoted argument
@@ -198,7 +199,7 @@ static const char *parse_cmd_update(struct ref_transaction *transaction,
 		die("update %s: extra input: %s", refname, next);
 
 	if (ref_transaction_update(transaction, refname, new_sha1, old_sha1,
-				   update_flags, have_old, &err))
+				   update_flags, have_old, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -229,7 +230,7 @@ static const char *parse_cmd_create(struct ref_transaction *transaction,
 		die("create %s: extra input: %s", refname, next);
 
 	if (ref_transaction_create(transaction, refname, new_sha1,
-				   update_flags, &err))
+				   update_flags, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -264,7 +265,7 @@ static const char *parse_cmd_delete(struct ref_transaction *transaction,
 		die("delete %s: extra input: %s", refname, next);
 
 	if (ref_transaction_delete(transaction, refname, old_sha1,
-				   update_flags, have_old, &err))
+				   update_flags, have_old, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -300,7 +301,7 @@ static const char *parse_cmd_verify(struct ref_transaction *transaction,
 		die("verify %s: extra input: %s", refname, next);
 
 	if (ref_transaction_update(transaction, refname, new_sha1, old_sha1,
-				   update_flags, have_old, &err))
+				   update_flags, have_old, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = 0;
@@ -354,7 +355,7 @@ static void update_refs_stdin(struct ref_transaction *transaction)
 
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
-	const char *refname, *oldval, *msg = NULL;
+	const char *refname, *oldval;
 	unsigned char sha1[20], oldsha1[20];
 	int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;
 	struct option options[] = {
@@ -385,7 +386,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 		if (end_null)
 			line_termination = '\0';
 		update_refs_stdin(transaction);
-		if (ref_transaction_commit(transaction, msg, &err))
+		if (ref_transaction_commit(transaction, &err))
 			die("%s", err.buf);
 		ref_transaction_free(transaction);
 		strbuf_release(&err);
diff --git a/fast-import.c b/fast-import.c
index e7f6e37..51dfaad 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1708,8 +1708,8 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, b->sha1, old_sha1,
-				   0, 1, &err) ||
-	    ref_transaction_commit(transaction, msg, &err)) {
+				   0, 1, msg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
 		strbuf_release(&err);
@@ -1749,12 +1749,12 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf, t->sha1,
-					   NULL, 0, 0, &err)) {
+					   NULL, 0, 0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
 	}
-	if (ref_transaction_commit(transaction, msg, &err))
+	if (ref_transaction_commit(transaction, &err))
 		failure |= error("%s", err.buf);
 
  cleanup:
diff --git a/refs.c b/refs.c
index 5609622..99a9b86 100644
--- a/refs.c
+++ b/refs.c
@@ -2396,8 +2396,8 @@ static void prune_ref(struct ref_to_prune *r)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, r->name, r->sha1,
-				   REF_ISPRUNING, 1, &err) ||
-	    ref_transaction_commit(transaction, NULL, &err)) {
+				   REF_ISPRUNING, 1, NULL, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
 		strbuf_release(&err);
@@ -2571,8 +2571,8 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, refname, sha1, delopt,
-				   sha1 && !is_null_sha1(sha1), &err) ||
-	    ref_transaction_commit(transaction, NULL, &err)) {
+				   sha1 && !is_null_sha1(sha1), NULL, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ref_transaction_free(transaction);
 		strbuf_release(&err);
@@ -3350,6 +3350,7 @@ struct ref_update {
 	int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
 	struct ref_lock *lock;
 	int type;
+	char *msg;
 	const char refname[FLEX_ARRAY];
 };
 
@@ -3392,9 +3393,10 @@ void ref_transaction_free(struct ref_transaction *transaction)
 	if (!transaction)
 		return;
 
-	for (i = 0; i < transaction->nr; i++)
+	for (i = 0; i < transaction->nr; i++) {
+		free(transaction->updates[i]->msg);
 		free(transaction->updates[i]);
-
+	}
 	free(transaction->updates);
 	free(transaction);
 }
@@ -3415,7 +3417,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err)
 {
 	struct ref_update *update;
@@ -3432,13 +3434,15 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	update->have_old = have_old;
 	if (have_old)
 		hashcpy(update->old_sha1, old_sha1);
+	if (msg)
+		update->msg = xstrdup(msg);
 	return 0;
 }
 
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
-			   int flags,
+			   int flags, const char *msg,
 			   struct strbuf *err)
 {
 	struct ref_update *update;
@@ -3455,13 +3459,15 @@ int ref_transaction_create(struct ref_transaction *transaction,
 	hashclr(update->old_sha1);
 	update->flags = flags;
 	update->have_old = 1;
+	if (msg)
+		update->msg = xstrdup(msg);
 	return 0;
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err)
 {
 	struct ref_update *update;
@@ -3479,6 +3485,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		assert(!is_null_sha1(old_sha1));
 		hashcpy(update->old_sha1, old_sha1);
 	}
+	if (msg)
+		update->msg = xstrdup(msg);
 	return 0;
 }
 
@@ -3492,8 +3500,8 @@ int update_ref(const char *action, const char *refname,
 	t = ref_transaction_begin(&err);
 	if (!t ||
 	    ref_transaction_update(t, refname, sha1, oldval, flags,
-				   !!oldval, &err) ||
-	    ref_transaction_commit(t, action, &err)) {
+				   !!oldval, action, &err) ||
+	    ref_transaction_commit(t, &err)) {
 		const char *str = "update_ref failed for ref '%s': %s";
 
 		ref_transaction_free(t);
@@ -3539,7 +3547,7 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,
 }
 
 int ref_transaction_commit(struct ref_transaction *transaction,
-			   const char *msg, struct strbuf *err)
+			   struct strbuf *err)
 {
 	int ret = 0, delnum = 0, i;
 	const char **delnames;
@@ -3588,7 +3596,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 
 		if (!is_null_sha1(update->new_sha1)) {
 			ret = write_ref_sha1(update->lock, update->new_sha1,
-					     msg);
+					     update->msg);
 			update->lock = NULL; /* freed by write_ref_sha1 */
 			if (ret) {
 				if (err)
diff --git a/refs.h b/refs.h
index 69ef28c..aded545 100644
--- a/refs.h
+++ b/refs.h
@@ -271,8 +271,8 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * The following functions add a reference check or update to a
  * ref_transaction.  In all of them, refname is the name of the
  * reference to be affected.  The functions make internal copies of
- * refname, so the caller retains ownership of the parameter.  flags
- * can be REF_NODEREF; it is passed to update_ref_lock().
+ * refname and msg, so the caller retains ownership of these parameters.
+ * flags can be REF_NODEREF; it is passed to update_ref_lock().
  */
 
 /*
@@ -289,7 +289,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err);
 
 /*
@@ -304,7 +304,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
-			   int flags,
+			   int flags, const char *msg,
 			   struct strbuf *err);
 
 /*
@@ -318,7 +318,7 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const unsigned char *old_sha1,
-			   int flags, int have_old,
+			   int flags, int have_old, const char *msg,
 			   struct strbuf *err);
 
 /*
@@ -327,7 +327,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
  * problem.
  */
 int ref_transaction_commit(struct ref_transaction *transaction,
-			   const char *msg, struct strbuf *err);
+			   struct strbuf *err);
 
 /*
  * Free an existing transaction and all associated data.
diff --git a/sequencer.c b/sequencer.c
index 5e93b6a..c5b7b8a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -286,8 +286,8 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn ? null_sha1 : from,
-				   0, 1, &err) ||
-	    ref_transaction_commit(transaction, sb.buf, &err)) {
+				   0, 1, sb.buf, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
 		strbuf_release(&sb);
diff --git a/walker.c b/walker.c
index b8a5441..3c6fb8a 100644
--- a/walker.c
+++ b/walker.c
@@ -298,14 +298,13 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
 					   &sha1[20 * i], NULL, 0, 0,
+					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
 			goto done;
 		}
 	}
-	if (ref_transaction_commit(transaction,
-				   msg ? msg : "fetch (unknown)",
-				   &err)) {
+	if (ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		goto done;
 	}
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 06/24] rename_ref: don't ask read_ref_full where the ref came from

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 30 Apr 2014 12:41:04 -0700

We call read_ref_full with a pointer to flags from rename_ref but since
we never actually use the returned flags we can just pass NULL here instead.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
---
As before.

 refs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 99a9b86..39571f5 100644
--- a/refs.c
+++ b/refs.c
@@ -2671,7 +2671,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 		goto rollback;
 	}
 
-	if (!read_ref_full(newrefname, sha1, 1, &flag) &&
+	if (!read_ref_full(newrefname, sha1, 1, NULL) &&
 	    delete_ref(newrefname, sha1, REF_NODEREF)) {
 		if (errno==EISDIR) {
 			if (remove_empty_directories(git_path("%s", newrefname))) {
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 07/24] refs.c: refuse to lock badly named refs in lock_ref_sha1_basic

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 1 May 2014 10:40:10 -0700

Move the check for check_refname_format from lock_any_ref_for_update to
lock_ref_sha1_basic.  At some later stage we will get rid of
lock_any_ref_for_update completely.  This has no visible impact to callers
except for the inability to lock badly named refs, which is not possible
today already for other reasons.(*)

Keep lock_any_ref_for_update as a no-op wrapper.  It is the public facing
version of this interface and keeping it as a separate function will make
it easier to experiment with the internal lock_ref_sha1_basic signature.

(*) For example, if lock_ref_sha1_basic checks the refname format and
refuses to lock badly named refs, it will not be possible to delete
such refs because the first step of deletion is to lock the ref.  We
currently already fail in that case because these refs are not recognized
to exist:

 $ cp .git/refs/heads/master .git/refs/heads/echo...\*\*
 $ git branch -D .git/refs/heads/echo...\*\*
 error: branch '.git/refs/heads/echo...**' not found.

This has been broken for a while.  Later patches in the series will start
repairing the handling of badly named refs.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Change since v21:
- clarified commit message

 refs.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index 39571f5..3c2ce57 100644
--- a/refs.c
+++ b/refs.c
@@ -2091,6 +2091,11 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	int missing = 0;
 	int attempts_remaining = 3;
 
+	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		errno = EINVAL;
+		return NULL;
+	}
+
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
@@ -2182,8 +2187,6 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
 {
-	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
-		return NULL;
 	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
 }
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 08/24] refs.c: call lock_ref_sha1_basic directly from commit

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 1 May 2014 10:43:39 -0700

Skip using the lock_any_ref_for_update wrapper and call lock_ref_sha1_basic
directly from the commit function.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
---
As before.

 refs.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index 3c2ce57..f124c2b 100644
--- a/refs.c
+++ b/refs.c
@@ -3578,12 +3578,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	for (i = 0; i < n; i++) {
 		struct ref_update *update = updates[i];
 
-		update->lock = lock_any_ref_for_update(update->refname,
-						       (update->have_old ?
-							update->old_sha1 :
-							NULL),
-						       update->flags,
-						       &update->type);
+		update->lock = lock_ref_sha1_basic(update->refname,
+						   (update->have_old ?
+						    update->old_sha1 :
+						    NULL),
+						   update->flags,
+						   &update->type);
 		if (!update->lock) {
 			if (err)
 				strbuf_addf(err, "Cannot lock the ref '%s'.",
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 09/24] refs.c: pass a list of names to skip to is_refname_available

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 1 May 2014 11:16:07 -0700

Change is_refname_available to take a list of strings to exclude when
checking for conflicts instead of just one single name. We can already
exclude a single name for the sake of renames. This generalizes that support.

ref_transaction_commit already tracks a set of refs that are being deleted
in an array.  This array is then used to exclude refs from being written to
the packed-refs file.  At some stage we will want to change this array to a
struct string_list and then we can pass it to is_refname_available via the
call to lock_ref_sha1_basic.  That will allow us to perform transactions
that perform multiple renames as long as there are no conflicts within the
starting or ending state.

For example, that would allow a single transaction that contains two
renames that are both individually conflicting:

   m -> n/n
   n -> m/m

No functional change intended yet.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Since v21:
- clarified commit message
- clarified comments

 refs.c | 44 +++++++++++++++++++++++++++++---------------
 1 file changed, 29 insertions(+), 15 deletions(-)

diff --git a/refs.c b/refs.c
index f124c2b..6820c93 100644
--- a/refs.c
+++ b/refs.c
@@ -801,14 +801,16 @@ static int names_conflict(const char *refname1, const char *refname2)
 
 struct name_conflict_cb {
 	const char *refname;
-	const char *oldrefname;
 	const char *conflicting_refname;
+	struct string_list *skiplist;
 };
 
 static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
 {
 	struct name_conflict_cb *data = (struct name_conflict_cb *)cb_data;
-	if (data->oldrefname && !strcmp(data->oldrefname, entry->name))
+
+	if (data->skiplist &&
+	    string_list_has_string(data->skiplist, entry->name))
 		return 0;
 	if (names_conflict(data->refname, entry->name)) {
 		data->conflicting_refname = entry->name;
@@ -820,17 +822,18 @@ static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
 /*
  * Return true iff a reference named refname could be created without
  * conflicting with the name of an existing reference in dir.  If
- * oldrefname is non-NULL, ignore potential conflicts with oldrefname
- * (e.g., because oldrefname is scheduled for deletion in the same
- * operation).
+ * skiplist is non-NULL, ignore potential conflicts with names in
+ * skiplist (e.g., because those refs are scheduled for deletion in
+ * the same operation).  skiplist must be sorted.
  */
-static int is_refname_available(const char *refname, const char *oldrefname,
-				struct ref_dir *dir)
+static int is_refname_available(const char *refname,
+				struct ref_dir *dir,
+				struct string_list *skiplist)
 {
 	struct name_conflict_cb data;
 	data.refname = refname;
-	data.oldrefname = oldrefname;
 	data.conflicting_refname = NULL;
+	data.skiplist = skiplist;
 
 	sort_ref_dir(dir);
 	if (do_for_each_entry_in_dir(dir, 0, name_conflict_fn, &data)) {
@@ -2080,6 +2083,7 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
  */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
+					    struct string_list *skiplist,
 					    int flags, int *type_p)
 {
 	char *ref_file;
@@ -2129,7 +2133,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	 * name is a proper prefix of our refname.
 	 */
 	if (missing &&
-	     !is_refname_available(refname, NULL, get_packed_refs(&ref_cache))) {
+	     !is_refname_available(refname, get_packed_refs(&ref_cache),
+				   skiplist)) {
 		last_errno = ENOTDIR;
 		goto error_return;
 	}
@@ -2187,7 +2192,7 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
 {
-	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
+	return lock_ref_sha1_basic(refname, old_sha1, NULL, flags, type_p);
 }
 
 /*
@@ -2648,6 +2653,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	struct stat loginfo;
 	int log = !lstat(git_path("logs/%s", oldrefname), &loginfo);
 	const char *symref = NULL;
+	struct string_list skiplist = STRING_LIST_INIT_NODUP;
 
 	if (log && S_ISLNK(loginfo.st_mode))
 		return error("reflog for %s is a symlink", oldrefname);
@@ -2659,11 +2665,18 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	if (!symref)
 		return error("refname %s not found", oldrefname);
 
-	if (!is_refname_available(newrefname, oldrefname, get_packed_refs(&ref_cache)))
+	string_list_insert(&skiplist, oldrefname);
+	if (!is_refname_available(newrefname, get_packed_refs(&ref_cache),
+				  &skiplist)) {
+		string_list_clear(&skiplist, 0);
 		return 1;
-
-	if (!is_refname_available(newrefname, oldrefname, get_loose_refs(&ref_cache)))
+	}
+	if (!is_refname_available(newrefname, get_loose_refs(&ref_cache),
+				  &skiplist)) {
+		string_list_clear(&skiplist, 0);
 		return 1;
+	}
+	string_list_clear(&skiplist, 0);
 
 	if (log && rename(git_path("logs/%s", oldrefname), git_path(TMP_RENAMED_LOG)))
 		return error("unable to move logfile logs/%s to "TMP_RENAMED_LOG": %s",
@@ -2692,7 +2705,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 
 	logmoved = log;
 
-	lock = lock_ref_sha1_basic(newrefname, NULL, 0, NULL);
+	lock = lock_ref_sha1_basic(newrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for update", newrefname);
 		goto rollback;
@@ -2707,7 +2720,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	return 0;
 
  rollback:
-	lock = lock_ref_sha1_basic(oldrefname, NULL, 0, NULL);
+	lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for rollback", oldrefname);
 		goto rollbacklog;
@@ -3582,6 +3595,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 						   (update->have_old ?
 						    update->old_sha1 :
 						    NULL),
+						   NULL,
 						   update->flags,
 						   &update->type);
 		if (!update->lock) {
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 10/24] refs.c: ref_transaction_commit: distinguish name conflicts from other errors

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Fri, 16 May 2014 14:14:38 -0700

In _commit, ENOTDIR can happen in the call to lock_ref_sha1_basic, either
when we lstat the new refname or if the name checking function reports that
the same type of conflict happened.  In both cases, it means that we can not
create the new ref due to a name conflict.

Start defining specific return codes for _commit.  TRANSACTION_NAME_CONFLICT
refers to a failure to create a ref due to a name conflict with another ref.
TRANSACTION_GENERIC_ERROR is for all other errors.

When "git fetch" is creating refs, name conflicts differ from other errors in
that they are likely to be resolved by running "git remote prune <remote>".
"git fetch" currently inspects errno to decide whether to give that advice.
Once it switches to the transaction API, it can check for
TRANSACTION_NAME_CONFLICT instead.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Since v21:
- clarified commit message and updated to match code
- small code cleanups
- clarified API doc, introduced TRANSACTION_GENERIC_ERROR so both error
  codes have names

 refs.c | 26 ++++++++++++++++----------
 refs.h |  9 +++++++--
 2 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/refs.c b/refs.c
index 6820c93..623a1ae 100644
--- a/refs.c
+++ b/refs.c
@@ -3583,9 +3583,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 
 	/* Copy, sort, and reject duplicate refs */
 	qsort(updates, n, sizeof(*updates), ref_update_compare);
-	ret = ref_update_reject_duplicates(updates, n, err);
-	if (ret)
+	if (ref_update_reject_duplicates(updates, n, err)) {
+		ret = TRANSACTION_GENERIC_ERROR;
 		goto cleanup;
+	}
 
 	/* Acquire all locks while verifying old values */
 	for (i = 0; i < n; i++) {
@@ -3599,10 +3600,12 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 						   update->flags,
 						   &update->type);
 		if (!update->lock) {
+			ret = (errno == ENOTDIR)
+				? TRANSACTION_NAME_CONFLICT
+				: TRANSACTION_GENERIC_ERROR;
 			if (err)
 				strbuf_addf(err, "Cannot lock the ref '%s'.",
 					    update->refname);
-			ret = 1;
 			goto cleanup;
 		}
 	}
@@ -3612,15 +3615,16 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (!is_null_sha1(update->new_sha1)) {
-			ret = write_ref_sha1(update->lock, update->new_sha1,
-					     update->msg);
-			update->lock = NULL; /* freed by write_ref_sha1 */
-			if (ret) {
+			if (write_ref_sha1(update->lock, update->new_sha1,
+					   update->msg)) {
+				update->lock = NULL; /* freed by write_ref_sha1 */
 				if (err)
 					strbuf_addf(err, "Cannot update the ref '%s'.",
 						    update->refname);
+				ret = TRANSACTION_GENERIC_ERROR;
 				goto cleanup;
 			}
+			update->lock = NULL; /* freed by write_ref_sha1 */
 		}
 	}
 
@@ -3629,14 +3633,16 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			ret |= delete_ref_loose(update->lock, update->type,
-						err);
+			if (delete_ref_loose(update->lock, update->type, err))
+				ret = TRANSACTION_GENERIC_ERROR;
+
 			if (!(update->flags & REF_ISPRUNING))
 				delnames[delnum++] = update->lock->ref_name;
 		}
 	}
 
-	ret |= repack_without_refs(delnames, delnum, err);
+	if (repack_without_refs(delnames, delnum, err))
+		ret = TRANSACTION_GENERIC_ERROR;
 	for (i = 0; i < delnum; i++)
 		unlink_or_warn(git_path("logs/%s", delnames[i]));
 	clear_loose_ref_cache(&ref_cache);
diff --git a/refs.h b/refs.h
index aded545..fd63b47 100644
--- a/refs.h
+++ b/refs.h
@@ -323,9 +323,14 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 
 /*
  * Commit all of the changes that have been queued in transaction, as
- * atomically as possible.  Return a nonzero value if there is a
- * problem.
+ * atomically as possible.
+ *
+ * Returns 0 for success, or one of the below error codes for errors.
  */
+/* Naming conflict (for example, the ref names A and A/B conflict). */
+#define TRANSACTION_NAME_CONFLICT -1
+/* All other errors. */
+#define TRANSACTION_GENERIC_ERROR -2
 int ref_transaction_commit(struct ref_transaction *transaction,
 			   struct strbuf *err);
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 11/24] fetch.c: change s_update_ref to use a ref transaction

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 13:49:07 -0700

Change s_update_ref to use a ref transaction for the ref update.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Since v21:
- tweaked handling of ref_transaction_commit result (no functional
  change)

 builtin/fetch.c | 34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 55f457c..30b40f6 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -375,23 +375,37 @@ static int s_update_ref(const char *action,
 {
 	char msg[1024];
 	char *rla = getenv("GIT_REFLOG_ACTION");
-	static struct ref_lock *lock;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret, df_conflict = 0;
 
 	if (dry_run)
 		return 0;
 	if (!rla)
 		rla = default_rla.buf;
 	snprintf(msg, sizeof(msg), "%s: %s", rla, action);
-	lock = lock_any_ref_for_update(ref->name,
-				       check_old ? ref->old_sha1 : NULL,
-				       0, NULL);
-	if (!lock)
-		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
-					  STORE_REF_ERROR_OTHER;
-	if (write_ref_sha1(lock, ref->new_sha1, msg) < 0)
-		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
-					  STORE_REF_ERROR_OTHER;
+
+	transaction = ref_transaction_begin(&err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref->name, ref->new_sha1,
+				   ref->old_sha1, 0, check_old, msg, &err))
+		goto fail;
+
+	ret = ref_transaction_commit(transaction, &err);
+	if (ret) {
+		df_conflict = (ret == TRANSACTION_NAME_CONFLICT);
+		goto fail;
+	}
+
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
 	return 0;
+fail:
+	ref_transaction_free(transaction);
+	error("%s", err.buf);
+	strbuf_release(&err);
+	return df_conflict ? STORE_REF_ERROR_DF_CONFLICT
+			   : STORE_REF_ERROR_OTHER;
 }
 
 #define REFCOL_WIDTH  10
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 12/24] refs.c: make write_ref_sha1 static

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Mon, 28 Apr 2014 15:36:58 -0700

No external users call write_ref_sha1 any more so let's declare it static.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Since v21:
- punctuation fix in commit message
- grammar tweak in doc comment

 refs.c | 10 ++++++++--
 refs.h |  3 ---
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index 623a1ae..f596a9f 100644
--- a/refs.c
+++ b/refs.c
@@ -2645,6 +2645,9 @@ static int rename_tmp_log(const char *newrefname)
 	return 0;
 }
 
+static int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1,
+			  const char *logmsg);
+
 int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)
 {
 	unsigned char sha1[20], orig_sha1[20];
@@ -2900,8 +2903,11 @@ static int is_branch(const char *refname)
 	return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
 }
 
-/* This function must return a meaningful errno */
-int write_ref_sha1(struct ref_lock *lock,
+/*
+ * Write sha1 into the ref specified by the lock. Make sure that errno
+ * is sane on error.
+ */
+static int write_ref_sha1(struct ref_lock *lock,
 	const unsigned char *sha1, const char *logmsg)
 {
 	static char term = '\n';
diff --git a/refs.h b/refs.h
index fd63b47..3bb16db 100644
--- a/refs.h
+++ b/refs.h
@@ -195,9 +195,6 @@ extern int commit_ref(struct ref_lock *lock);
 /** Release any lock taken but not written. **/
 extern void unlock_ref(struct ref_lock *lock);
 
-/** Writes sha1 into the ref specified by the lock. **/
-extern int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1, const char *msg);
-
 /*
  * Setup reflog before using. Set errno to something meaningful on failure.
  */
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 13/24] refs.c: change resolve_ref_unsafe reading argument to be a flags field

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Tue, 15 Jul 2014 12:59:36 -0700

resolve_ref_unsafe takes a boolean argument for reading (a nonexistent ref
resolves successfully for writing but not for reading).  Change this to be
a flags field instead, and pass the new constant RESOLVE_REF_READING when
we want this behaviour.

While at it, swap two of the arguments in the function to put output
arguments at the end.  As a nice side effect, this ensures that we can
catch callers that were unaware of the new API so they can be audited.

Give the wrapper functions resolve_refdup and read_ref_full the same
treatment for consistency.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Since v21:
- clarified commit message
- put output parameters last

 branch.c                |  2 +-
 builtin/blame.c         |  2 +-
 builtin/branch.c        |  9 ++---
 builtin/checkout.c      |  6 ++--
 builtin/clone.c         |  2 +-
 builtin/commit.c        |  2 +-
 builtin/fmt-merge-msg.c |  2 +-
 builtin/for-each-ref.c  |  6 ++--
 builtin/fsck.c          |  2 +-
 builtin/log.c           |  3 +-
 builtin/merge.c         |  2 +-
 builtin/notes.c         |  2 +-
 builtin/receive-pack.c  |  4 +--
 builtin/remote.c        |  5 +--
 builtin/show-branch.c   |  7 ++--
 builtin/symbolic-ref.c  |  2 +-
 bundle.c                |  2 +-
 cache.h                 | 23 ++++++------
 http-backend.c          |  4 ++-
 notes-merge.c           |  2 +-
 reflog-walk.c           |  5 +--
 refs.c                  | 93 ++++++++++++++++++++++++++++---------------------
 remote.c                | 11 +++---
 sequencer.c             |  4 +--
 transport-helper.c      |  5 ++-
 transport.c             |  5 +--
 upload-pack.c           |  2 +-
 wt-status.c             |  2 +-
 28 files changed, 124 insertions(+), 92 deletions(-)

diff --git a/branch.c b/branch.c
index 76a8ec9..adb07c6 100644
--- a/branch.c
+++ b/branch.c
@@ -186,7 +186,7 @@ int validate_new_branchname(const char *name, struct strbuf *ref,
 		const char *head;
 		unsigned char sha1[20];
 
-		head = resolve_ref_unsafe("HEAD", sha1, 0, NULL);
+		head = resolve_ref_unsafe("HEAD", 0, sha1, NULL);
 		if (!is_bare_repository() && head && !strcmp(head, ref->buf))
 			die(_("Cannot force update the current branch."));
 	}
diff --git a/builtin/blame.c b/builtin/blame.c
index a52a279..5cbd38f 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -2292,7 +2292,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 	commit->object.type = OBJ_COMMIT;
 	parent_tail = &commit->parents;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, 1, NULL))
+	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
 		die("no such ref: HEAD");
 
 	parent_tail = append_parent(parent_tail, head_sha1);
diff --git a/builtin/branch.c b/builtin/branch.c
index 652b1d2..e5d1377 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -129,7 +129,8 @@ static int branch_merged(int kind, const char *name,
 		    branch->merge[0] &&
 		    branch->merge[0]->dst &&
 		    (reference_name = reference_name_to_free =
-		     resolve_refdup(branch->merge[0]->dst, sha1, 1, NULL)) != NULL)
+		     resolve_refdup(branch->merge[0]->dst, RESOLVE_REF_READING,
+				    sha1, NULL)) != NULL)
 			reference_rev = lookup_commit_reference(sha1);
 	}
 	if (!reference_rev)
@@ -233,7 +234,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, sha1, 0, &flags);
+		target = resolve_ref_unsafe(name, 0, sha1, &flags);
 		if (!target ||
 		    (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {
 			error(remote_branch
@@ -296,7 +297,7 @@ static char *resolve_symref(const char *src, const char *prefix)
 	int flag;
 	const char *dst, *cp;
 
-	dst = resolve_ref_unsafe(src, sha1, 0, &flag);
+	dst = resolve_ref_unsafe(src, 0, sha1, &flag);
 	if (!(dst && (flag & REF_ISSYMREF)))
 		return NULL;
 	if (prefix && (cp = skip_prefix(dst, prefix)))
@@ -862,7 +863,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	track = git_branch_track;
 
-	head = resolve_refdup("HEAD", head_sha1, 0, NULL);
+	head = resolve_refdup("HEAD", 0, head_sha1, NULL);
 	if (!head)
 		die(_("Failed to resolve HEAD as a valid ref."));
 	if (!strcmp(head, "HEAD")) {
diff --git a/builtin/checkout.c b/builtin/checkout.c
index f1dc56e..a5fef2d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -356,7 +356,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 	    commit_locked_index(lock_file))
 		die(_("unable to write new index file"));
 
-	read_ref_full("HEAD", rev, 0, &flag);
+	read_ref_full("HEAD", 0, rev, &flag);
 	head = lookup_commit_reference_gently(rev, 1);
 
 	errs |= post_checkout_hook(head, head, 0);
@@ -771,7 +771,7 @@ static int switch_branches(const struct checkout_opts *opts,
 	unsigned char rev[20];
 	int flag, writeout_error = 0;
 	memset(&old, 0, sizeof(old));
-	old.path = path_to_free = resolve_refdup("HEAD", rev, 0, &flag);
+	old.path = path_to_free = resolve_refdup("HEAD", 0, rev, &flag);
 	old.commit = lookup_commit_reference_gently(rev, 1);
 	if (!(flag & REF_ISSYMREF))
 		old.path = NULL;
@@ -1068,7 +1068,7 @@ static int checkout_branch(struct checkout_opts *opts,
 		unsigned char rev[20];
 		int flag;
 
-		if (!read_ref_full("HEAD", rev, 0, &flag) &&
+		if (!read_ref_full("HEAD", 0, rev, &flag) &&
 		    (flag & REF_ISSYMREF) && is_null_sha1(rev))
 			return switch_unborn_to_new_branch(opts);
 	}
diff --git a/builtin/clone.c b/builtin/clone.c
index b12989d..0f5c880 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -622,7 +622,7 @@ static int checkout(void)
 	if (option_no_checkout)
 		return 0;
 
-	head = resolve_refdup("HEAD", sha1, 1, NULL);
+	head = resolve_refdup("HEAD", RESOLVE_REF_READING, sha1, NULL);
 	if (!head) {
 		warning(_("remote HEAD refers to nonexistent ref, "
 			  "unable to checkout.\n"));
diff --git a/builtin/commit.c b/builtin/commit.c
index d23e876..9ccc78b 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1468,7 +1468,7 @@ static void print_summary(const char *prefix, const unsigned char *sha1,
 	rev.diffopt.break_opt = 0;
 	diff_setup_done(&rev.diffopt);
 
-	head = resolve_ref_unsafe("HEAD", junk_sha1, 0, NULL);
+	head = resolve_ref_unsafe("HEAD", 0, junk_sha1, NULL);
 	printf("[%s%s ",
 		starts_with(head, "refs/heads/") ?
 			head + 11 :
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index 3906eda..afe05dc 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -602,7 +602,7 @@ int fmt_merge_msg(struct strbuf *in, struct strbuf *out,
 
 	/* get current branch */
 	current_branch = current_branch_to_free =
-		resolve_refdup("HEAD", head_sha1, 1, NULL);
+		resolve_refdup("HEAD", RESOLVE_REF_READING, head_sha1, NULL);
 	if (!current_branch)
 		die("No current branch");
 	if (starts_with(current_branch, "refs/heads/"))
diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c
index 3e1d5c3..20949b7 100644
--- a/builtin/for-each-ref.c
+++ b/builtin/for-each-ref.c
@@ -649,7 +649,8 @@ static void populate_value(struct refinfo *ref)
 
 	if (need_symref && (ref->flag & REF_ISSYMREF) && !ref->symref) {
 		unsigned char unused1[20];
-		ref->symref = resolve_refdup(ref->refname, unused1, 1, NULL);
+		ref->symref = resolve_refdup(ref->refname, RESOLVE_REF_READING,
+					     unused1, NULL);
 		if (!ref->symref)
 			ref->symref = "";
 	}
@@ -707,7 +708,8 @@ static void populate_value(struct refinfo *ref)
 			const char *head;
 			unsigned char sha1[20];
 
-			head = resolve_ref_unsafe("HEAD", sha1, 1, NULL);
+			head = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+						  sha1, NULL);
 			if (!strcmp(ref->refname, head))
 				v->s = "*";
 			else
diff --git a/builtin/fsck.c b/builtin/fsck.c
index fc150c8..7cd109a 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -560,7 +560,7 @@ static int fsck_head_link(void)
 	if (verbose)
 		fprintf(stderr, "Checking HEAD link\n");
 
-	head_points_at = resolve_ref_unsafe("HEAD", head_sha1, 0, &flag);
+	head_points_at = resolve_ref_unsafe("HEAD", 0, head_sha1, &flag);
 	if (!head_points_at)
 		return error("Invalid HEAD");
 	if (!strcmp(head_points_at, "HEAD"))
diff --git a/builtin/log.c b/builtin/log.c
index a7ba211..493440a 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1395,7 +1395,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (check_head) {
 			unsigned char sha1[20];
 			const char *ref;
-			ref = resolve_ref_unsafe("HEAD", sha1, 1, NULL);
+			ref = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+						 sha1, NULL);
 			if (ref && starts_with(ref, "refs/heads/"))
 				branch_name = xstrdup(ref + strlen("refs/heads/"));
 			else
diff --git a/builtin/merge.c b/builtin/merge.c
index 428ca24..6f56967 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -1108,7 +1108,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
 	 * Check if we are _not_ on a detached HEAD, i.e. if there is a
 	 * current branch.
 	 */
-	branch = branch_to_free = resolve_refdup("HEAD", head_sha1, 0, &flag);
+	branch = branch_to_free = resolve_refdup("HEAD", 0, head_sha1, &flag);
 	if (branch && starts_with(branch, "refs/heads/"))
 		branch += 11;
 	if (!branch || is_null_sha1(head_sha1))
diff --git a/builtin/notes.c b/builtin/notes.c
index 820c341..eaf297d 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -703,7 +703,7 @@ static int merge_commit(struct notes_merge_options *o)
 	init_notes(t, "NOTES_MERGE_PARTIAL", combine_notes_overwrite, 0);
 
 	o->local_ref = local_ref_to_free =
-		resolve_refdup("NOTES_MERGE_REF", sha1, 0, NULL);
+		resolve_refdup("NOTES_MERGE_REF", 0, sha1, NULL);
 	if (!o->local_ref)
 		die("Failed to resolve NOTES_MERGE_REF");
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index d1f4cf7..8a6e7e3 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -656,7 +656,7 @@ static void check_aliased_update(struct command *cmd, struct string_list *list)
 	int flag;
 
 	strbuf_addf(&buf, "%s%s", get_git_namespace(), cmd->ref_name);
-	dst_name = resolve_ref_unsafe(buf.buf, sha1, 0, &flag);
+	dst_name = resolve_ref_unsafe(buf.buf, 0, sha1, &flag);
 	strbuf_release(&buf);
 
 	if (!(flag & REF_ISSYMREF))
@@ -817,7 +817,7 @@ static void execute_commands(struct command *commands,
 	check_aliased_updates(commands);
 
 	free(head_name_to_free);
-	head_name = head_name_to_free = resolve_refdup("HEAD", sha1, 0, NULL);
+	head_name = head_name_to_free = resolve_refdup("HEAD", 0, sha1, NULL);
 
 	checked_connectivity = 1;
 	for (cmd = commands; cmd; cmd = cmd->next) {
diff --git a/builtin/remote.c b/builtin/remote.c
index 401feb3..c7f82f4 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -568,7 +568,8 @@ static int read_remote_branches(const char *refname,
 	strbuf_addf(&buf, "refs/remotes/%s/", rename->old);
 	if (starts_with(refname, buf.buf)) {
 		item = string_list_append(rename->remote_branches, xstrdup(refname));
-		symref = resolve_ref_unsafe(refname, orig_sha1, 1, &flag);
+		symref = resolve_ref_unsafe(refname, RESOLVE_REF_READING,
+					    orig_sha1, &flag);
 		if (flag & REF_ISSYMREF)
 			item->util = xstrdup(symref);
 		else
@@ -704,7 +705,7 @@ static int mv(int argc, const char **argv)
 		int flag = 0;
 		unsigned char sha1[20];
 
-		read_ref_full(item->string, sha1, 1, &flag);
+		read_ref_full(item->string, RESOLVE_REF_READING, sha1, &flag);
 		if (!(flag & REF_ISSYMREF))
 			continue;
 		if (delete_ref(item->string, NULL, REF_NODEREF))
diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index d873172..acc8dc1 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -727,7 +727,9 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		if (ac == 0) {
 			static const char *fake_av[2];
 
-			fake_av[0] = resolve_refdup("HEAD", sha1, 1, NULL);
+			fake_av[0] = resolve_refdup("HEAD",
+						    RESOLVE_REF_READING,
+						    sha1, NULL);
 			fake_av[1] = NULL;
 			av = fake_av;
 			ac = 1;
@@ -789,7 +791,8 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		}
 	}
 
-	head_p = resolve_ref_unsafe("HEAD", head_sha1, 1, NULL);
+	head_p = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+				    head_sha1, NULL);
 	if (head_p) {
 		head_len = strlen(head_p);
 		memcpy(head, head_p, head_len + 1);
diff --git a/builtin/symbolic-ref.c b/builtin/symbolic-ref.c
index b6a711d..29fb3f1 100644
--- a/builtin/symbolic-ref.c
+++ b/builtin/symbolic-ref.c
@@ -13,7 +13,7 @@ static int check_symref(const char *HEAD, int quiet, int shorten, int print)
 {
 	unsigned char sha1[20];
 	int flag;
-	const char *refname = resolve_ref_unsafe(HEAD, sha1, 0, &flag);
+	const char *refname = resolve_ref_unsafe(HEAD, 0, sha1, &flag);
 
 	if (!refname)
 		die("No such ref: %s", HEAD);
diff --git a/bundle.c b/bundle.c
index 1222952..d92e49c 100644
--- a/bundle.c
+++ b/bundle.c
@@ -311,7 +311,7 @@ int create_bundle(struct bundle_header *header, const char *path,
 			continue;
 		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
 			continue;
-		if (read_ref_full(e->name, sha1, 1, &flag))
+		if (read_ref_full(e->name, RESOLVE_REF_READING, sha1, &flag))
 			flag = 0;
 		display_ref = (flag & REF_ISSYMREF) ? e->name : ref;
 
diff --git a/cache.h b/cache.h
index e7ec626..5b54911 100644
--- a/cache.h
+++ b/cache.h
@@ -947,8 +947,8 @@ extern int for_each_abbrev(const char *prefix, each_abbrev_fn, void *);
 extern int get_sha1_hex(const char *hex, unsigned char *sha1);
 
 extern char *sha1_to_hex(const unsigned char *sha1);	/* static buffer result! */
-extern int read_ref_full(const char *refname, unsigned char *sha1,
-			 int reading, int *flags);
+extern int read_ref_full(const char *refname, int resolve_flags,
+			 unsigned char *sha1, int *flags);
 extern int read_ref(const char *refname, unsigned char *sha1);
 
 /*
@@ -960,20 +960,20 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  * or the input ref.
  *
  * If the reference cannot be resolved to an object, the behavior
- * depends on the "reading" argument:
+ * depends on the RESOLVE_REF_READING flag:
  *
- * - If reading is set, return NULL.
+ * - If RESOLVE_REF_READING is set, return NULL.
  *
- * - If reading is not set, clear sha1 and return the name of the last
- *   reference name in the chain, which will either be a non-symbolic
+ * - If RESOLVE_REF_READING is not set, clear sha1 and return the name of
+ *   the last reference name in the chain, which will either be a non-symbolic
  *   reference or an undefined reference.  If this is a prelude to
  *   "writing" to the ref, the return value is the name of the ref
  *   that will actually be created or changed.
  *
- * If flag is non-NULL, set the value that it points to the
+ * If flags is non-NULL, set the value that it points to the
  * combination of REF_ISPACKED (if the reference was found among the
- * packed references) and REF_ISSYMREF (if the initial reference was a
- * symbolic reference).
+ * packed references), REF_ISSYMREF (if the initial reference was a
+ * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
  *
  * If ref is not a properly-formatted, normalized reference, return
  * NULL.  If more than MAXDEPTH recursive symbolic lookups are needed,
@@ -981,8 +981,9 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  *
  * errno is set to something meaningful on error.
  */
-extern const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1, int reading, int *flag);
-extern char *resolve_refdup(const char *ref, unsigned char *sha1, int reading, int *flag);
+#define RESOLVE_REF_READING 0x01
+extern const char *resolve_ref_unsafe(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
+extern char *resolve_refdup(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
 
 extern int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref);
 extern int dwim_log(const char *str, int len, unsigned char *sha1, char **ref);
diff --git a/http-backend.c b/http-backend.c
index d2c0a62..e172886 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -417,7 +417,9 @@ static int show_head_ref(const char *refname, const unsigned char *sha1,
 
 	if (flag & REF_ISSYMREF) {
 		unsigned char unused[20];
-		const char *target = resolve_ref_unsafe(refname, unused, 1, NULL);
+		const char *target = resolve_ref_unsafe(refname,
+							RESOLVE_REF_READING,
+							unused, NULL);
 		const char *target_nons = strip_namespace(target);
 
 		strbuf_addf(buf, "ref: %s\n", target_nons);
diff --git a/notes-merge.c b/notes-merge.c
index 94a1a8a..3c88d17 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -549,7 +549,7 @@ int notes_merge(struct notes_merge_options *o,
 	       o->local_ref, o->remote_ref);
 
 	/* Dereference o->local_ref into local_sha1 */
-	if (read_ref_full(o->local_ref, local_sha1, 0, NULL))
+	if (read_ref_full(o->local_ref, 0, local_sha1, NULL))
 		die("Failed to resolve local notes ref '%s'", o->local_ref);
 	else if (!check_refname_format(o->local_ref, 0) &&
 		is_null_sha1(local_sha1))
diff --git a/reflog-walk.c b/reflog-walk.c
index 9ce8b53..23345ea 100644
--- a/reflog-walk.c
+++ b/reflog-walk.c
@@ -48,7 +48,8 @@ static struct complete_reflogs *read_complete_reflog(const char *ref)
 		unsigned char sha1[20];
 		const char *name;
 		void *name_to_free;
-		name = name_to_free = resolve_refdup(ref, sha1, 1, NULL);
+		name = name_to_free = resolve_refdup(ref, RESOLVE_REF_READING,
+						     sha1, NULL);
 		if (name) {
 			for_each_reflog_ent(name, read_one_reflog, reflogs);
 			free(name_to_free);
@@ -174,7 +175,7 @@ int add_reflog_for_walk(struct reflog_walk_info *info,
 		if (*branch == '\0') {
 			unsigned char sha1[20];
 			free(branch);
-			branch = resolve_refdup("HEAD", sha1, 0, NULL);
+			branch = resolve_refdup("HEAD", 0, sha1, NULL);
 			if (!branch)
 				die ("No current branch");
 
diff --git a/refs.c b/refs.c
index f596a9f..4916d16 100644
--- a/refs.c
+++ b/refs.c
@@ -1191,7 +1191,9 @@ static void read_loose_refs(const char *dirname, struct ref_dir *dir)
 					hashclr(sha1);
 					flag |= REF_ISBROKEN;
 				}
-			} else if (read_ref_full(refname.buf, sha1, 1, &flag)) {
+			} else if (read_ref_full(refname.buf,
+						 RESOLVE_REF_READING,
+						 sha1, &flag)) {
 				hashclr(sha1);
 				flag |= REF_ISBROKEN;
 			}
@@ -1316,9 +1318,9 @@ static struct ref_entry *get_packed_ref(const char *refname)
  * options are forwarded from resolve_safe_unsafe().
  */
 static const char *handle_missing_loose_ref(const char *refname,
+					    int resolve_flags,
 					    unsigned char *sha1,
-					    int reading,
-					    int *flag)
+					    int *flags)
 {
 	struct ref_entry *entry;
 
@@ -1329,12 +1331,12 @@ static const char *handle_missing_loose_ref(const char *refname,
 	entry = get_packed_ref(refname);
 	if (entry) {
 		hashcpy(sha1, entry->u.value.sha1);
-		if (flag)
-			*flag |= REF_ISPACKED;
+		if (flags)
+			*flags |= REF_ISPACKED;
 		return refname;
 	}
 	/* The reference is not a packed reference, either. */
-	if (reading) {
+	if (resolve_flags & RESOLVE_REF_READING) {
 		return NULL;
 	} else {
 		hashclr(sha1);
@@ -1343,21 +1345,20 @@ static const char *handle_missing_loose_ref(const char *refname,
 }
 
 /* This function needs to return a meaningful errno on failure */
-const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int reading, int *flag)
+const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned char *sha1, int *flags)
 {
 	int depth = MAXDEPTH;
 	ssize_t len;
 	char buffer[256];
 	static char refname_buffer[256];
 
-	if (flag)
-		*flag = 0;
+	if (flags)
+		*flags = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		errno = EINVAL;
 		return NULL;
 	}
-
 	for (;;) {
 		char path[PATH_MAX];
 		struct stat st;
@@ -1383,8 +1384,8 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 	stat_ref:
 		if (lstat(path, &st) < 0) {
 			if (errno == ENOENT)
-				return handle_missing_loose_ref(refname, sha1,
-								reading, flag);
+				return handle_missing_loose_ref(refname,
+						resolve_flags, sha1, flags);
 			else
 				return NULL;
 		}
@@ -1404,8 +1405,8 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 					!check_refname_format(buffer, 0)) {
 				strcpy(refname_buffer, buffer);
 				refname = refname_buffer;
-				if (flag)
-					*flag |= REF_ISSYMREF;
+				if (flags)
+					*flags |= REF_ISSYMREF;
 				continue;
 			}
 		}
@@ -1450,21 +1451,21 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 			 */
 			if (get_sha1_hex(buffer, sha1) ||
 			    (buffer[40] != '\0' && !isspace(buffer[40]))) {
-				if (flag)
-					*flag |= REF_ISBROKEN;
+				if (flags)
+					*flags |= REF_ISBROKEN;
 				errno = EINVAL;
 				return NULL;
 			}
 			return refname;
 		}
-		if (flag)
-			*flag |= REF_ISSYMREF;
+		if (flags)
+			*flags |= REF_ISSYMREF;
 		buf = buffer + 4;
 		while (isspace(*buf))
 			buf++;
 		if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
-			if (flag)
-				*flag |= REF_ISBROKEN;
+			if (flags)
+				*flags |= REF_ISBROKEN;
 			errno = EINVAL;
 			return NULL;
 		}
@@ -1472,9 +1473,9 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int rea
 	}
 }
 
-char *resolve_refdup(const char *ref, unsigned char *sha1, int reading, int *flag)
+char *resolve_refdup(const char *ref, int resolve_flags, unsigned char *sha1, int *flags)
 {
-	const char *ret = resolve_ref_unsafe(ref, sha1, reading, flag);
+	const char *ret = resolve_ref_unsafe(ref, resolve_flags, sha1, flags);
 	return ret ? xstrdup(ret) : NULL;
 }
 
@@ -1485,22 +1486,22 @@ struct ref_filter {
 	void *cb_data;
 };
 
-int read_ref_full(const char *refname, unsigned char *sha1, int reading, int *flags)
+int read_ref_full(const char *refname, int resolve_flags, unsigned char *sha1, int *flags)
 {
-	if (resolve_ref_unsafe(refname, sha1, reading, flags))
+	if (resolve_ref_unsafe(refname, resolve_flags, sha1, flags))
 		return 0;
 	return -1;
 }
 
 int read_ref(const char *refname, unsigned char *sha1)
 {
-	return read_ref_full(refname, sha1, 1, NULL);
+	return read_ref_full(refname, RESOLVE_REF_READING, sha1, NULL);
 }
 
 int ref_exists(const char *refname)
 {
 	unsigned char sha1[20];
-	return !!resolve_ref_unsafe(refname, sha1, 1, NULL);
+	return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
 }
 
 static int filter_refs(const char *refname, const unsigned char *sha1, int flags,
@@ -1614,7 +1615,7 @@ int peel_ref(const char *refname, unsigned char *sha1)
 		return 0;
 	}
 
-	if (read_ref_full(refname, base, 1, &flag))
+	if (read_ref_full(refname, RESOLVE_REF_READING, base, &flag))
 		return -1;
 
 	/*
@@ -1655,7 +1656,7 @@ static int warn_if_dangling_symref(const char *refname, const unsigned char *sha
 	if (!(flags & REF_ISSYMREF))
 		return 0;
 
-	resolves_to = resolve_ref_unsafe(refname, junk, 0, NULL);
+	resolves_to = resolve_ref_unsafe(refname, 0, junk, NULL);
 	if (!resolves_to
 	    || (d->refname
 		? strcmp(resolves_to, d->refname)
@@ -1780,7 +1781,7 @@ static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
 		return 0;
 	}
 
-	if (!read_ref_full("HEAD", sha1, 1, &flag))
+	if (!read_ref_full("HEAD", RESOLVE_REF_READING, sha1, &flag))
 		return fn("HEAD", sha1, flag, cb_data);
 
 	return 0;
@@ -1860,7 +1861,7 @@ int head_ref_namespaced(each_ref_fn fn, void *cb_data)
 	int flag;
 
 	strbuf_addf(&buf, "%sHEAD", get_git_namespace());
-	if (!read_ref_full(buf.buf, sha1, 1, &flag))
+	if (!read_ref_full(buf.buf, RESOLVE_REF_READING, sha1, &flag))
 		ret = fn(buf.buf, sha1, flag, cb_data);
 	strbuf_release(&buf);
 
@@ -1955,7 +1956,9 @@ int refname_match(const char *abbrev_name, const char *full_name)
 static struct ref_lock *verify_lock(struct ref_lock *lock,
 	const unsigned char *old_sha1, int mustexist)
 {
-	if (read_ref_full(lock->ref_name, lock->old_sha1, mustexist, NULL)) {
+	if (read_ref_full(lock->ref_name,
+			  mustexist ? RESOLVE_REF_READING : 0,
+			  lock->old_sha1, NULL)) {
 		int save_errno = errno;
 		error("Can't verify ref %s", lock->ref_name);
 		unlock_ref(lock);
@@ -2028,7 +2031,8 @@ int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
 
 		this_result = refs_found ? sha1_from_ref : sha1;
 		mksnpath(fullref, sizeof(fullref), *p, len, str);
-		r = resolve_ref_unsafe(fullref, this_result, 1, &flag);
+		r = resolve_ref_unsafe(fullref, RESOLVE_REF_READING,
+				       this_result, &flag);
 		if (r) {
 			if (!refs_found++)
 				*ref = xstrdup(r);
@@ -2057,7 +2061,8 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 		const char *ref, *it;
 
 		mksnpath(path, sizeof(path), *p, len, str);
-		ref = resolve_ref_unsafe(path, hash, 1, NULL);
+		ref = resolve_ref_unsafe(path, RESOLVE_REF_READING,
+					 hash, NULL);
 		if (!ref)
 			continue;
 		if (reflog_exists(path))
@@ -2092,6 +2097,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	int last_errno = 0;
 	int type, lflags;
 	int mustexist = (old_sha1 && !is_null_sha1(old_sha1));
+	int resolve_flags = 0;
 	int missing = 0;
 	int attempts_remaining = 3;
 
@@ -2103,7 +2109,11 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
-	refname = resolve_ref_unsafe(refname, lock->old_sha1, mustexist, &type);
+	if (mustexist)
+		resolve_flags |= RESOLVE_REF_READING;
+
+	refname = resolve_ref_unsafe(refname, resolve_flags,
+				     lock->old_sha1, &type);
 	if (!refname && errno == EISDIR) {
 		/* we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -2116,7 +2126,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 			error("there are still refs under '%s'", orig_refname);
 			goto error_return;
 		}
-		refname = resolve_ref_unsafe(orig_refname, lock->old_sha1, mustexist, &type);
+		refname = resolve_ref_unsafe(orig_refname, resolve_flags,
+					     lock->old_sha1, &type);
 	}
 	if (type_p)
 	    *type_p = type;
@@ -2469,7 +2480,7 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 		unsigned char sha1[20];
 		int flags;
 
-		if (read_ref_full(entry->name, sha1, 0, &flags))
+		if (read_ref_full(entry->name, 0, sha1, &flags))
 			/* We should at least have found the packed ref. */
 			die("Internal error");
 		if ((flags & REF_ISSYMREF) || !(flags & REF_ISPACKED)) {
@@ -2661,7 +2672,8 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	if (log && S_ISLNK(loginfo.st_mode))
 		return error("reflog for %s is a symlink", oldrefname);
 
-	symref = resolve_ref_unsafe(oldrefname, orig_sha1, 1, &flag);
+	symref = resolve_ref_unsafe(oldrefname, RESOLVE_REF_READING,
+				    orig_sha1, &flag);
 	if (flag & REF_ISSYMREF)
 		return error("refname %s is a symbolic ref, renaming it is not supported",
 			oldrefname);
@@ -2690,7 +2702,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 		goto rollback;
 	}
 
-	if (!read_ref_full(newrefname, sha1, 1, NULL) &&
+	if (!read_ref_full(newrefname, RESOLVE_REF_READING, sha1, NULL) &&
 	    delete_ref(newrefname, sha1, REF_NODEREF)) {
 		if (errno==EISDIR) {
 			if (remove_empty_directories(git_path("%s", newrefname))) {
@@ -2968,7 +2980,8 @@ static int write_ref_sha1(struct ref_lock *lock,
 		unsigned char head_sha1[20];
 		int head_flag;
 		const char *head_ref;
-		head_ref = resolve_ref_unsafe("HEAD", head_sha1, 1, &head_flag);
+		head_ref = resolve_ref_unsafe("HEAD", RESOLVE_REF_READING,
+					      head_sha1, &head_flag);
 		if (head_ref && (head_flag & REF_ISSYMREF) &&
 		    !strcmp(head_ref, lock->ref_name))
 			log_ref_write("HEAD", lock->old_sha1, sha1, logmsg);
@@ -3335,7 +3348,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data
 				retval = do_for_each_reflog(name, fn, cb_data);
 			} else {
 				unsigned char sha1[20];
-				if (read_ref_full(name->buf, sha1, 0, NULL))
+				if (read_ref_full(name->buf, 0, sha1, NULL))
 					retval = error("bad ref for %s", name->buf);
 				else
 					retval = fn(name->buf, sha1, 0, cb_data);
diff --git a/remote.c b/remote.c
index 0e9459c..25b07ac 100644
--- a/remote.c
+++ b/remote.c
@@ -486,7 +486,7 @@ static void read_config(void)
 		return;
 	default_remote_name = "origin";
 	current_branch = NULL;
-	head_ref = resolve_ref_unsafe("HEAD", sha1, 0, &flag);
+	head_ref = resolve_ref_unsafe("HEAD", 0, sha1, &flag);
 	if (head_ref && (flag & REF_ISSYMREF) &&
 	    starts_with(head_ref, "refs/heads/")) {
 		current_branch =
@@ -1121,7 +1121,8 @@ static char *guess_ref(const char *name, struct ref *peer)
 	struct strbuf buf = STRBUF_INIT;
 	unsigned char sha1[20];
 
-	const char *r = resolve_ref_unsafe(peer->name, sha1, 1, NULL);
+	const char *r = resolve_ref_unsafe(peer->name, RESOLVE_REF_READING,
+					   sha1, NULL);
 	if (!r)
 		return NULL;
 
@@ -1182,7 +1183,9 @@ static int match_explicit(struct ref *src, struct ref *dst,
 		unsigned char sha1[20];
 		int flag;
 
-		dst_value = resolve_ref_unsafe(matched_src->name, sha1, 1, &flag);
+		dst_value = resolve_ref_unsafe(matched_src->name,
+					       RESOLVE_REF_READING,
+					       sha1, &flag);
 		if (!dst_value ||
 		    ((flag & REF_ISSYMREF) &&
 		     !starts_with(dst_value, "refs/heads/")))
@@ -1656,7 +1659,7 @@ static int ignore_symref_update(const char *refname)
 	unsigned char sha1[20];
 	int flag;
 
-	if (!resolve_ref_unsafe(refname, sha1, 0, &flag))
+	if (!resolve_ref_unsafe(refname, 0, sha1, &flag))
 		return 0; /* non-existing refs are OK */
 	return (flag & REF_ISSYMREF);
 }
diff --git a/sequencer.c b/sequencer.c
index c5b7b8a..70fb7a8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -366,7 +366,7 @@ static int is_index_unchanged(void)
 	unsigned char head_sha1[20];
 	struct commit *head_commit;
 
-	if (!resolve_ref_unsafe("HEAD", head_sha1, 1, NULL))
+	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
 		return error(_("Could not resolve HEAD commit\n"));
 
 	head_commit = lookup_commit(head_sha1);
@@ -912,7 +912,7 @@ static int rollback_single_pick(void)
 	if (!file_exists(git_path("CHERRY_PICK_HEAD")) &&
 	    !file_exists(git_path("REVERT_HEAD")))
 		return error(_("no cherry-pick or revert in progress"));
-	if (read_ref_full("HEAD", head_sha1, 0, NULL))
+	if (read_ref_full("HEAD", 0, head_sha1, NULL))
 		return error(_("cannot resolve HEAD"));
 	if (is_null_sha1(head_sha1))
 		return error(_("cannot abort from a branch yet to be born"));
diff --git a/transport-helper.c b/transport-helper.c
index 84c616f..3497a5e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -889,7 +889,10 @@ static int push_refs_with_export(struct transport *transport,
 					int flag;
 
 					/* Follow symbolic refs (mainly for HEAD). */
-					name = resolve_ref_unsafe(ref->peer_ref->name, sha1, 1, &flag);
+					name = resolve_ref_unsafe(
+						 ref->peer_ref->name,
+						 RESOLVE_REF_READING,
+						 sha1, &flag);
 					if (!name || !(flag & REF_ISSYMREF))
 						name = ref->peer_ref->name;
 
diff --git a/transport.c b/transport.c
index 325f03e..76e0a9a 100644
--- a/transport.c
+++ b/transport.c
@@ -168,7 +168,8 @@ static void set_upstreams(struct transport *transport, struct ref *refs,
 		/* Follow symbolic refs (mainly for HEAD). */
 		localname = ref->peer_ref->name;
 		remotename = ref->name;
-		tmp = resolve_ref_unsafe(localname, sha, 1, &flag);
+		tmp = resolve_ref_unsafe(localname, RESOLVE_REF_READING,
+					 sha, &flag);
 		if (tmp && flag & REF_ISSYMREF &&
 			starts_with(tmp, "refs/heads/"))
 			localname = tmp;
@@ -753,7 +754,7 @@ void transport_print_push_status(const char *dest, struct ref *refs,
 	unsigned char head_sha1[20];
 	char *head;
 
-	head = resolve_refdup("HEAD", head_sha1, 1, NULL);
+	head = resolve_refdup("HEAD", RESOLVE_REF_READING, head_sha1, NULL);
 
 	if (verbose) {
 		for (ref = refs; ref; ref = ref->next)
diff --git a/upload-pack.c b/upload-pack.c
index 01de944..4542565 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -743,7 +743,7 @@ static int find_symref(const char *refname, const unsigned char *sha1, int flag,
 
 	if ((flag & REF_ISSYMREF) == 0)
 		return 0;
-	symref_target = resolve_ref_unsafe(refname, unused, 0, &flag);
+	symref_target = resolve_ref_unsafe(refname, 0, unused, &flag);
 	if (!symref_target || (flag & REF_ISSYMREF) == 0)
 		die("'%s' is a symref but it is not?", refname);
 	item = string_list_append(cb_data, refname);
diff --git a/wt-status.c b/wt-status.c
index 318a191..c3cbf50 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -128,7 +128,7 @@ void wt_status_prepare(struct wt_status *s)
 	s->show_untracked_files = SHOW_NORMAL_UNTRACKED_FILES;
 	s->use_color = -1;
 	s->relative_paths = 1;
-	s->branch = resolve_refdup("HEAD", sha1, 0, NULL);
+	s->branch = resolve_refdup("HEAD", 0, sha1, NULL);
 	s->reference = "HEAD";
 	s->fp = stdout;
 	s->index_file = get_index_file();
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 14/24] reflog test: test interaction with detached HEAD

[Jonathan Nieder]

From: Junio C Hamano <gitster@pobox.com>

A proposed patch produced broken HEAD reflog entries when checking out
anything other than a branch.  The testsuite still passed, so it took
a few days for the bug to be noticed.

Add tests checking the content of the reflog after detaching and
reattaching HEAD so we don't have to rely on manual testing to catch
such problems in the future.

[jn: using 'log -g --format=%H' instead of parsing --oneline output,
 resetting state in each test so they can be safely reordered or
 skipped]

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
New since v21.  Thanks to Junio for noticing the bug.

 t/t1413-reflog-detach.sh | 70 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100755 t/t1413-reflog-detach.sh

diff --git a/t/t1413-reflog-detach.sh b/t/t1413-reflog-detach.sh
new file mode 100755
index 0000000..c730600
--- /dev/null
+++ b/t/t1413-reflog-detach.sh
@@ -0,0 +1,70 @@
+#!/bin/sh
+
+test_description='Test reflog interaction with detached HEAD'
+. ./test-lib.sh
+
+reset_state () {
+	git checkout master &&
+	cp saved_reflog .git/logs/HEAD
+}
+
+test_expect_success setup '
+	test_tick &&
+	git commit --allow-empty -m initial &&
+	git branch side &&
+	test_tick &&
+	git commit --allow-empty -m second &&
+	cat .git/logs/HEAD >saved_reflog
+'
+
+test_expect_success baseline '
+	reset_state &&
+	git rev-parse master master^ >expect &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'switch to branch' '
+	reset_state &&
+	git rev-parse side master master^ >expect &&
+	git checkout side &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'detach to other' '
+	reset_state &&
+	git rev-parse master side master master^ >expect &&
+	git checkout side &&
+	git checkout master^0 &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'detach to self' '
+	reset_state &&
+	git rev-parse master master master^ >expect &&
+	git checkout master^0 &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'attach to self' '
+	reset_state &&
+	git rev-parse master master master master^ >expect &&
+	git checkout master^0 &&
+	git checkout master &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'attach to other' '
+	reset_state &&
+	git rev-parse side master master master^ >expect &&
+	git checkout master^0 &&
+	git checkout side &&
+	git log -g --format=%H >actual &&
+	test_cmp expect actual
+'
+
+test_done
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 15/24] branch -d: avoid repeated symref resolution

[Jonathan Nieder]

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Wed, 10 Sep 2014 18:22:48 -0700

If a repository gets in a broken state with too much symref nesting,
it cannot be repaired with "git branch -d":

 $ git symbolic-ref refs/heads/nonsense refs/heads/nonsense
 $ git branch -d nonsense
 error: branch 'nonsense' not found.

Worse, "git update-ref --no-deref -d" doesn't work for such repairs
either:

 $ git update-ref -d refs/heads/nonsense
 error: unable to resolve reference refs/heads/nonsense: Too many levels of symbolic links

Fix both by teaching resolve_ref_unsafe a new RESOLVE_REF_NO_RECURSE
flag and passing it when appropriate.

Callers can still read the value of a symref (for example to print a
message about it) with that flag set --- resolve_ref_unsafe will
resolve one level of symrefs and stop there.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
Since v21:
- renamed flag from RESOLVE_REF_NO_DEREF to _NO_RECURSE
- more detail in API docs
- only set NO_RECURSE when deleting refs.  Locking refs for non-deletion
  updates needs to recurse to get old_sha1 for the reflog.
- add more tests (for symrefs to refs with bad names, which should also
  be deletable now)

 builtin/branch.c      |  3 ++-
 cache.h               |  6 ++++++
 refs.c                | 17 +++++++++++++++--
 refs.h                |  2 ++
 t/t1400-update-ref.sh | 34 ++++++++++++++++++++++++++++++++++
 t/t3200-branch.sh     |  9 +++++++++
 6 files changed, 68 insertions(+), 3 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index e5d1377..a334380 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -234,7 +234,8 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, 0, sha1, &flags);
+		target = resolve_ref_unsafe(name, RESOLVE_REF_NO_RECURSE,
+					    sha1, &flags);
 		if (!target ||
 		    (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {
 			error(remote_branch
diff --git a/cache.h b/cache.h
index 5b54911..5ca7f2b 100644
--- a/cache.h
+++ b/cache.h
@@ -970,6 +970,11 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  *   "writing" to the ref, the return value is the name of the ref
  *   that will actually be created or changed.
  *
+ * If the RESOLVE_REF_NO_RECURSE flag is passed, only resolves one
+ * level of symbolic reference.  The value stored in sha1 for a symbolic
+ * reference will always be null_sha1 in this case, and the return
+ * value is the reference that the symref refers to directly.
+ *
  * If flags is non-NULL, set the value that it points to the
  * combination of REF_ISPACKED (if the reference was found among the
  * packed references), REF_ISSYMREF (if the initial reference was a
@@ -982,6 +987,7 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  * errno is set to something meaningful on error.
  */
 #define RESOLVE_REF_READING 0x01
+#define RESOLVE_REF_NO_RECURSE 0x02
 extern const char *resolve_ref_unsafe(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
 extern char *resolve_refdup(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
 
diff --git a/refs.c b/refs.c
index 4916d16..490e788 100644
--- a/refs.c
+++ b/refs.c
@@ -1407,6 +1407,10 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned
 				refname = refname_buffer;
 				if (flags)
 					*flags |= REF_ISSYMREF;
+				if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
+					hashclr(sha1);
+					return refname;
+				}
 				continue;
 			}
 		}
@@ -1463,13 +1467,17 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned
 		buf = buffer + 4;
 		while (isspace(*buf))
 			buf++;
+		refname = strcpy(refname_buffer, buf);
+		if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
+			hashclr(sha1);
+			return refname;
+		}
 		if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
 			if (flags)
 				*flags |= REF_ISBROKEN;
 			errno = EINVAL;
 			return NULL;
 		}
-		refname = strcpy(refname_buffer, buf);
 	}
 }
 
@@ -2111,6 +2119,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 
 	if (mustexist)
 		resolve_flags |= RESOLVE_REF_READING;
+	if (flags & REF_NODEREF && flags & REF_DELETING)
+		resolve_flags |= RESOLVE_REF_NO_RECURSE;
 
 	refname = resolve_ref_unsafe(refname, resolve_flags,
 				     lock->old_sha1, &type);
@@ -3610,13 +3620,16 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	/* Acquire all locks while verifying old values */
 	for (i = 0; i < n; i++) {
 		struct ref_update *update = updates[i];
+		int flags = update->flags;
 
+		if (is_null_sha1(update->new_sha1))
+			flags |= REF_DELETING;
 		update->lock = lock_ref_sha1_basic(update->refname,
 						   (update->have_old ?
 						    update->old_sha1 :
 						    NULL),
 						   NULL,
-						   update->flags,
+						   flags,
 						   &update->type);
 		if (!update->lock) {
 			ret = (errno == ENOTDIR)
diff --git a/refs.h b/refs.h
index 3bb16db..b62c4c4 100644
--- a/refs.h
+++ b/refs.h
@@ -175,10 +175,12 @@ extern int peel_ref(const char *refname, unsigned char *sha1);
  * ref_transaction_create(), etc.
  * REF_NODEREF: act on the ref directly, instead of dereferencing
  *              symbolic references.
+ * REF_DELETING: tolerate broken refs
  *
  * Flags >= 0x100 are reserved for internal use.
  */
 #define REF_NODEREF	0x01
+#define REF_DELETING	0x02
 /*
  * This function sets errno to something meaningful on failure.
  */
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 0218e96..7c8c41a 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -110,6 +110,40 @@ test_expect_success "delete symref without dereference when the referred ref is
 cp -f .git/HEAD.orig .git/HEAD
 git update-ref -d $m
 
+test_expect_success 'update-ref -d is not confused by self-reference' '
+	git symbolic-ref refs/heads/self refs/heads/self &&
+	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_path_is_file .git/refs/heads/self &&
+	test_must_fail git update-ref -d refs/heads/self &&
+	test_path_is_file .git/refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete self-reference' '
+	git symbolic-ref refs/heads/self refs/heads/self &&
+	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_path_is_file .git/refs/heads/self &&
+	git update-ref --no-deref -d refs/heads/self &&
+	test_path_is_missing .git/refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
+	>.git/refs/heads/bad &&
+	test_when_finished "rm -f .git/refs/heads/bad" &&
+	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
+	test_when_finished "rm -f .git/refs/heads/ref-to-bad" &&
+	test_path_is_file .git/refs/heads/ref-to-bad &&
+	git update-ref --no-deref -d refs/heads/ref-to-bad &&
+	test_path_is_missing .git/refs/heads/ref-to-bad
+'
+
+test_expect_success 'update-ref --no-deref -d can delete reference to broken name' '
+	git symbolic-ref refs/heads/badname refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/badname" &&
+	test_path_is_file .git/refs/heads/badname &&
+	git update-ref --no-deref -d refs/heads/badname &&
+	test_path_is_missing .git/refs/heads/badname
+'
+
 test_expect_success '(not) create HEAD with old sha1' "
 	test_must_fail git update-ref HEAD $A $B
 "
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index ac31b71..432921b 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -285,6 +285,15 @@ test_expect_success 'deleting a dangling symref' '
 	test_i18ncmp expect actual
 '
 
+test_expect_success 'deleting a self-referential symref' '
+	git symbolic-ref refs/heads/self-reference refs/heads/self-reference &&
+	test_path_is_file .git/refs/heads/self-reference &&
+	echo "Deleted branch self-reference (was refs/heads/self-reference)." >expect &&
+	git branch -d self-reference >actual &&
+	test_path_is_missing .git/refs/heads/self-reference &&
+	test_i18ncmp expect actual
+'
+
 test_expect_success 'renaming a symref is not allowed' '
 	git symbolic-ref refs/heads/master2 refs/heads/master &&
 	test_must_fail git branch -m master2 master3 &&
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 16/24] branch -d: simplify by using RESOLVE_REF_READING flag

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 11 Sep 2014 10:34:36 -0700

When "git branch -d" reads the branch it is about to delete, it used
to avoid passing the RESOLVE_REF_READING ('treat missing ref as
error') flag because a symref pointing to a nonexistent ref would show
up as missing instead of as something that could be deleted.  To check
if a ref is actually missing, we then check

 - is it a symref?
 - if not, did it resolve to null_sha1?

Now we pass RESOLVE_REF_NO_RECURSE and the correct information is
returned for a symref even when it points to a missing ref.  Simplify
by relying on RESOLVE_REF_READING.

No functional change intended.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Split out from the 'fix handling of badly named refs' patch.

 builtin/branch.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index a334380..a0c5601 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -234,10 +234,11 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		free(name);
 
 		name = mkpathdup(fmt, bname.buf);
-		target = resolve_ref_unsafe(name, RESOLVE_REF_NO_RECURSE,
+		target = resolve_ref_unsafe(name,
+					    RESOLVE_REF_READING
+					    | RESOLVE_REF_NO_RECURSE,
 					    sha1, &flags);
-		if (!target ||
-		    (!(flags & REF_ISSYMREF) && is_null_sha1(sha1))) {
+		if (!target) {
 			error(remote_branch
 			      ? _("remote branch '%s' not found.")
 			      : _("branch '%s' not found."), bname.buf);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 17/24] packed-ref cache: forbid dot-components in refnames

[Jonathan Nieder]

Since v1.7.9-rc1~10^2 (write_head_info(): handle "extra refs" locally,
2012-01-06), this trick to keep track of ".have" refs that are only
valid on the wire and not on the filesystem is not needed any more.

Simplify by removing support for the REFNAME_DOT_COMPONENT flag.

This means we'll be slightly stricter with invalid refs found in a
packed-refs file or during clone.  read_loose_refs() already checks
for and skips refnames with .components so it is not affected.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
Noticed while reviewing other patches.

 refs.c | 14 +++-----------
 refs.h |  6 +-----
 2 files changed, 4 insertions(+), 16 deletions(-)

diff --git a/refs.c b/refs.c
index 490e788..94d6d89 100644
--- a/refs.c
+++ b/refs.c
@@ -69,16 +69,8 @@ static int check_refname_component(const char *refname, int flags)
 out:
 	if (cp == refname)
 		return 0; /* Component has zero length. */
-	if (refname[0] == '.') {
-		if (!(flags & REFNAME_DOT_COMPONENT))
-			return -1; /* Component starts with '.'. */
-		/*
-		 * Even if leading dots are allowed, don't allow "."
-		 * as a component (".." is prevented by a rule above).
-		 */
-		if (refname[1] == '\0')
-			return -1; /* Component equals ".". */
-	}
+	if (refname[0] == '.')
+		return -1; /* Component starts with '.'. */
 	if (cp - refname >= 5 && !memcmp(cp - 5, ".lock", 5))
 		return -1; /* Refname ends with ".lock". */
 	return cp - refname;
@@ -288,7 +280,7 @@ static struct ref_entry *create_ref_entry(const char *refname,
 	struct ref_entry *ref;
 
 	if (check_name &&
-	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL|REFNAME_DOT_COMPONENT))
+	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		die("Reference has invalid format: '%s'", refname);
 	len = strlen(refname) + 1;
 	ref = xmalloc(sizeof(struct ref_entry) + len);
diff --git a/refs.h b/refs.h
index b62c4c4..e48d8b9 100644
--- a/refs.h
+++ b/refs.h
@@ -226,7 +226,6 @@ extern int for_each_reflog(each_ref_fn, void *);
 
 #define REFNAME_ALLOW_ONELEVEL 1
 #define REFNAME_REFSPEC_PATTERN 2
-#define REFNAME_DOT_COMPONENT 4
 
 /*
  * Return 0 iff refname has the correct format for a refname according
@@ -234,10 +233,7 @@ extern int for_each_reflog(each_ref_fn, void *);
  * If REFNAME_ALLOW_ONELEVEL is set in flags, then accept one-level
  * reference names.  If REFNAME_REFSPEC_PATTERN is set in flags, then
  * allow a "*" wildcard character in place of one of the name
- * components.  No leading or repeated slashes are accepted.  If
- * REFNAME_DOT_COMPONENT is set in flags, then allow refname
- * components to start with "." (but not a whole component equal to
- * "." or "..").
+ * components.  No leading or repeated slashes are accepted.
  */
 extern int check_refname_format(const char *refname, int flags);
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 18/24] test: put tests for handling of bad ref names in one place

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>

There's no straightforward way to grep for all tests dealing with
invalid refs.  Put them in a single test script so it is easy to see
what functionality has not been exercised with bad ref names yet.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
New.

 t/t1400-update-ref.sh   | 44 --------------------------
 t/t1430-bad-ref-name.sh | 84 +++++++++++++++++++++++++++++++++++++++++++++++++
 t/t9300-fast-import.sh  | 30 ------------------
 3 files changed, 84 insertions(+), 74 deletions(-)
 create mode 100755 t/t1430-bad-ref-name.sh

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 7c8c41a..7b4707b 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -136,14 +136,6 @@ test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
 	test_path_is_missing .git/refs/heads/ref-to-bad
 '
 
-test_expect_success 'update-ref --no-deref -d can delete reference to broken name' '
-	git symbolic-ref refs/heads/badname refs/heads/broken...ref &&
-	test_when_finished "rm -f .git/refs/heads/badname" &&
-	test_path_is_file .git/refs/heads/badname &&
-	git update-ref --no-deref -d refs/heads/badname &&
-	test_path_is_missing .git/refs/heads/badname
-'
-
 test_expect_success '(not) create HEAD with old sha1' "
 	test_must_fail git update-ref HEAD $A $B
 "
@@ -408,12 +400,6 @@ test_expect_success 'stdin fails create with no ref' '
 	grep "fatal: create: missing <ref>" err
 '
 
-test_expect_success 'stdin fails create with bad ref name' '
-	echo "create ~a $m" >stdin &&
-	test_must_fail git update-ref --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin fails create with no new value' '
 	echo "create $a" >stdin &&
 	test_must_fail git update-ref --stdin <stdin 2>err &&
@@ -432,12 +418,6 @@ test_expect_success 'stdin fails update with no ref' '
 	grep "fatal: update: missing <ref>" err
 '
 
-test_expect_success 'stdin fails update with bad ref name' '
-	echo "update ~a $m" >stdin &&
-	test_must_fail git update-ref --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin fails update with no new value' '
 	echo "update $a" >stdin &&
 	test_must_fail git update-ref --stdin <stdin 2>err &&
@@ -456,12 +436,6 @@ test_expect_success 'stdin fails delete with no ref' '
 	grep "fatal: delete: missing <ref>" err
 '
 
-test_expect_success 'stdin fails delete with bad ref name' '
-	echo "delete ~a $m" >stdin &&
-	test_must_fail git update-ref --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin fails delete with too many arguments' '
 	echo "delete $a $m $m" >stdin &&
 	test_must_fail git update-ref --stdin <stdin 2>err &&
@@ -734,12 +708,6 @@ test_expect_success 'stdin -z fails create with no ref' '
 	grep "fatal: create: missing <ref>" err
 '
 
-test_expect_success 'stdin -z fails create with bad ref name' '
-	printf $F "create ~a " "$m" >stdin &&
-	test_must_fail git update-ref -z --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a " err
-'
-
 test_expect_success 'stdin -z fails create with no new value' '
 	printf $F "create $a" >stdin &&
 	test_must_fail git update-ref -z --stdin <stdin 2>err &&
@@ -764,12 +732,6 @@ test_expect_success 'stdin -z fails update with too few args' '
 	grep "fatal: update $a: unexpected end of input when reading <oldvalue>" err
 '
 
-test_expect_success 'stdin -z fails update with bad ref name' '
-	printf $F "update ~a" "$m" "" >stdin &&
-	test_must_fail git update-ref -z --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin -z emits warning with empty new value' '
 	git update-ref $a $m &&
 	printf $F "update $a" "" "" >stdin &&
@@ -802,12 +764,6 @@ test_expect_success 'stdin -z fails delete with no ref' '
 	grep "fatal: delete: missing <ref>" err
 '
 
-test_expect_success 'stdin -z fails delete with bad ref name' '
-	printf $F "delete ~a" "$m" >stdin &&
-	test_must_fail git update-ref -z --stdin <stdin 2>err &&
-	grep "fatal: invalid ref format: ~a" err
-'
-
 test_expect_success 'stdin -z fails delete with no old value' '
 	printf $F "delete $a" >stdin &&
 	test_must_fail git update-ref -z --stdin <stdin 2>err &&
diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
new file mode 100755
index 0000000..c7b19b0
--- /dev/null
+++ b/t/t1430-bad-ref-name.sh
@@ -0,0 +1,84 @@
+#!/bin/sh
+
+test_description='Test handling of ref names that check-ref-format rejects'
+. ./test-lib.sh
+
+test_expect_success setup '
+	test_commit one
+'
+
+test_expect_success 'fast-import: fail on invalid branch name ".badbranchname"' '
+	test_when_finished "rm -f .git/objects/pack_* .git/objects/index_*" &&
+	cat >input <<-INPUT_END &&
+		commit .badbranchname
+		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+		data <<COMMIT
+		corrupt
+		COMMIT
+
+		from refs/heads/master
+
+	INPUT_END
+	test_must_fail git fast-import <input
+'
+
+test_expect_success 'fast-import: fail on invalid branch name "bad[branch]name"' '
+	test_when_finished "rm -f .git/objects/pack_* .git/objects/index_*" &&
+	cat >input <<-INPUT_END &&
+		commit bad[branch]name
+		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+		data <<COMMIT
+		corrupt
+		COMMIT
+
+		from refs/heads/master
+
+	INPUT_END
+	test_must_fail git fast-import <input
+'
+
+test_expect_success 'update-ref --no-deref -d can delete reference to broken name' '
+	git symbolic-ref refs/heads/badname refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/badname" &&
+	test_path_is_file .git/refs/heads/badname &&
+	git update-ref --no-deref -d refs/heads/badname &&
+	test_path_is_missing .git/refs/heads/badname
+'
+
+test_expect_success 'update-ref --stdin fails create with bad ref name' '
+	echo "create ~a refs/heads/master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin fails update with bad ref name' '
+	echo "update ~a refs/heads/master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin fails delete with bad ref name' '
+	echo "delete ~a refs/heads/master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin -z fails create with bad ref name' '
+	printf "%s\0" "create ~a " refs/heads/master >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a " err
+'
+
+test_expect_success 'update-ref --stdin -z fails update with bad ref name' '
+	printf "%s\0" "update ~a" refs/heads/master "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'update-ref --stdin -z fails delete with bad ref name' '
+	printf "%s\0" "delete ~a" refs/heads/master >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_done
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index 5fc9ef2..3d156f9 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -347,36 +347,6 @@ test_expect_success 'B: fail on invalid blob sha1' '
 rm -f .git/objects/pack_* .git/objects/index_*
 
 cat >input <<INPUT_END
-commit .badbranchname
-committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
-data <<COMMIT
-corrupt
-COMMIT
-
-from refs/heads/master
-
-INPUT_END
-test_expect_success 'B: fail on invalid branch name ".badbranchname"' '
-    test_must_fail git fast-import <input
-'
-rm -f .git/objects/pack_* .git/objects/index_*
-
-cat >input <<INPUT_END
-commit bad[branch]name
-committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
-data <<COMMIT
-corrupt
-COMMIT
-
-from refs/heads/master
-
-INPUT_END
-test_expect_success 'B: fail on invalid branch name "bad[branch]name"' '
-    test_must_fail git fast-import <input
-'
-rm -f .git/objects/pack_* .git/objects/index_*
-
-cat >input <<INPUT_END
 commit TEMP_TAG
 committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 data <<COMMIT
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 19/24] refs.c: allow listing and deleting badly named refs

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Wed, 3 Sep 2014 11:45:43 -0700

We currently do not handle badly named refs well:

  $ cp .git/refs/heads/master .git/refs/heads/master.....@\*@\\.
  $ git branch
    fatal: Reference has invalid format: 'refs/heads/master.....@*@\.'
  $ git branch -D master.....@\*@\\.
    error: branch 'master.....@*@\.' not found.

Users cannot recover from a badly named ref without manually finding
and deleting the loose ref file or appropriate line in packed-refs.
Making that easier will make it easier to tweak the ref naming rules
in the future, for example to forbid shell metacharacters like '`'
and '"', without putting people in a state that is hard to get out of.

So allow "git branch --list" to show these refs and to allow "git
branch -d/-D" and "git update-ref -d" to delete them.  Other commands
(for example to rename refs) will continue to not handle these refs
but can be changed in later patches.

Details:

In resolving functions, refuse to resolve refs that don't pass the
check-ref-format(1) check unless the new RESOLVE_REF_ALLOW_BAD_NAME
flag is passed.  Even with RESOLVE_REF_ALLOW_BAD_NAME, refuse to
resolve refs that escape the refs/ directory and do not match the
pattern [A-Z_]* (think "HEAD" and "MERGE_HEAD").

In locking functions, refuse to act on badly named refs unless they
are being deleted and either are in the refs/ directory or match [A-Z_]*.

Just like other invalid refs, flag resolved, badly named refs with the
REF_ISBROKEN flag, treat them as resolving to null_sha1, and skip them
in all iteration functions except for for_each_rawref.

Flag badly named refs with a REF_BAD_NAME flag to make it easier for
future callers to notice and handle them specially.

In the transaction API, refuse to create or update badly named refs,
but allow deleting them (unless they escape refs/ and don't match
[A-Z_]*).

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Since v21:
- clarified change description
- handle REF_ISBROKEN case too when printing "Deleted branch" message
- tighten fallback check, so commands like 'git branch -d ../../config'
  won't work
- use REF_BAD_NAME so it's easier for callers (like for-each-ref in a
  later patch) to notice that we decided a ref's name was bad
- set errno when failing to find ref in the packed-refs file
- consistently don't set REF_ISBROKEN for missing refs
- consistently clear sha1 when successfully resolving a badly named ref
- treat refs with bad names encountered as part of symref resolution the
  same as such refs explicitly named
- always allow safe bad names when trying to lock a ref for deletion
  (though some callers don't get that far yet)
- clearer API docs, more tests

 builtin/branch.c        |   9 +--
 cache.h                 |  14 ++++-
 refs.c                  | 148 ++++++++++++++++++++++++++++++++++++++----------
 refs.h                  |   8 ++-
 t/t1430-bad-ref-name.sh | 125 +++++++++++++++++++++++++++++++++++++++-
 5 files changed, 265 insertions(+), 39 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index a0c5601..94aaea1 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -236,7 +236,8 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		name = mkpathdup(fmt, bname.buf);
 		target = resolve_ref_unsafe(name,
 					    RESOLVE_REF_READING
-					    | RESOLVE_REF_NO_RECURSE,
+					    | RESOLVE_REF_NO_RECURSE
+					    | RESOLVE_REF_ALLOW_BAD_NAME,
 					    sha1, &flags);
 		if (!target) {
 			error(remote_branch
@@ -246,7 +247,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 			continue;
 		}
 
-		if (!(flags & REF_ISSYMREF) &&
+		if (!(flags & (REF_ISSYMREF|REF_ISBROKEN)) &&
 		    check_branch_commit(bname.buf, name, sha1, head_rev, kinds,
 					force)) {
 			ret = 1;
@@ -266,8 +267,8 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 			       ? _("Deleted remote branch %s (was %s).\n")
 			       : _("Deleted branch %s (was %s).\n"),
 			       bname.buf,
-			       (flags & REF_ISSYMREF)
-			       ? target
+			       (flags & REF_ISBROKEN) ? "broken"
+			       : (flags & REF_ISSYMREF) ? target
 			       : find_unique_abbrev(sha1, DEFAULT_ABBREV));
 		}
 		delete_branch_config(bname.buf);
diff --git a/cache.h b/cache.h
index 5ca7f2b..0c0ac60 100644
--- a/cache.h
+++ b/cache.h
@@ -978,16 +978,26 @@ extern int read_ref(const char *refname, unsigned char *sha1);
  * If flags is non-NULL, set the value that it points to the
  * combination of REF_ISPACKED (if the reference was found among the
  * packed references), REF_ISSYMREF (if the initial reference was a
- * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
+ * symbolic reference), REF_BAD_NAME (if the reference name is ill
+ * formed --- see RESOLVE_REF_ALLOW_BAD_NAME below), and REF_ISBROKEN
+ * (if the ref is malformed).
  *
  * If ref is not a properly-formatted, normalized reference, return
  * NULL.  If more than MAXDEPTH recursive symbolic lookups are needed,
  * give up and return NULL.
  *
- * errno is set to something meaningful on error.
+ * RESOLVE_REF_ALLOW_BAD_NAME allows resolving refs even when their
+ * name is invalid according to git-check-ref-format(1).  If the name
+ * is bad then the value stored in sha1 will be null_sha1 and the
+ * REF_ISBROKEN and REF_BAD_NAME flags will be set.
+ *
+ * Even with RESOLVE_REF_ALLOW_BAD_NAME, names that escape the refs/
+ * directory and do not consist of all caps and underscores cannot be
+ * resolved.  The function returns NULL for such ref names.
  */
 #define RESOLVE_REF_READING 0x01
 #define RESOLVE_REF_NO_RECURSE 0x02
+#define RESOLVE_REF_ALLOW_BAD_NAME 0x04
 extern const char *resolve_ref_unsafe(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
 extern char *resolve_refdup(const char *ref, int resolve_flags, unsigned char *sha1, int *flags);
 
diff --git a/refs.c b/refs.c
index 94d6d89..0d3abb1 100644
--- a/refs.c
+++ b/refs.c
@@ -185,8 +185,8 @@ struct ref_dir {
 
 /*
  * Bit values for ref_entry::flag.  REF_ISSYMREF=0x01,
- * REF_ISPACKED=0x02, and REF_ISBROKEN=0x04 are public values; see
- * refs.h.
+ * REF_ISPACKED=0x02, REF_ISBROKEN=0x04 and REF_BAD_NAME=0x08 are
+ * public values; see refs.h.
  */
 
 /*
@@ -194,16 +194,16 @@ struct ref_dir {
  * the correct peeled value for the reference, which might be
  * null_sha1 if the reference is not a tag or if it is broken.
  */
-#define REF_KNOWS_PEELED 0x08
+#define REF_KNOWS_PEELED 0x10
 
 /* ref_entry represents a directory of references */
-#define REF_DIR 0x10
+#define REF_DIR 0x20
 
 /*
  * Entry has not yet been read from disk (used only for REF_DIR
  * entries representing loose references)
  */
-#define REF_INCOMPLETE 0x20
+#define REF_INCOMPLETE 0x40
 
 /*
  * A ref_entry represents either a reference or a "subdirectory" of
@@ -272,6 +272,37 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
 	return dir;
 }
 
+static int escapes_cwd(const char *path) {
+	char *buf;
+	int result;
+
+	if (is_absolute_path(path))
+		return 1;
+	buf = xmalloc(strlen(path) + 1);
+	result = !!normalize_path_copy(buf, path);
+	free(buf);
+	return result;
+}
+
+/*
+ * Check if a refname is safe.
+ * For refs that start with "refs/" we consider it safe as long as the rest
+ * of the path components does not allow it to escape from this directory.
+ * For all other refs we only consider them safe iff they only contain
+ * upper case characters and '_'.
+ */
+static int refname_is_safe(const char *refname)
+{
+	if (starts_with(refname, "refs/"))
+		return !escapes_cwd(refname + strlen("refs/"));
+	while (*refname) {
+		if (!isupper(*refname) && *refname != '_')
+			return 0;
+		refname++;
+	}
+	return 1;
+}
+
 static struct ref_entry *create_ref_entry(const char *refname,
 					  const unsigned char *sha1, int flag,
 					  int check_name)
@@ -282,6 +313,8 @@ static struct ref_entry *create_ref_entry(const char *refname,
 	if (check_name &&
 	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		die("Reference has invalid format: '%s'", refname);
+	if (!check_name && !refname_is_safe(refname))
+		die("Reference has invalid name: '%s'", refname);
 	len = strlen(refname) + 1;
 	ref = xmalloc(sizeof(struct ref_entry) + len);
 	hashcpy(ref->u.value.sha1, sha1);
@@ -1051,7 +1084,13 @@ static void read_packed_refs(FILE *f, struct ref_dir *dir)
 
 		refname = parse_ref_line(refline, sha1);
 		if (refname) {
-			last = create_ref_entry(refname, sha1, REF_ISPACKED, 1);
+			int flag = REF_ISPACKED;
+
+			if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+				hashclr(sha1);
+				flag |= REF_BAD_NAME | REF_ISBROKEN;
+			}
+			last = create_ref_entry(refname, sha1, flag, 0);
 			if (peeled == PEELED_FULLY ||
 			    (peeled == PEELED_TAGS && starts_with(refname, "refs/tags/")))
 				last->flag |= REF_KNOWS_PEELED;
@@ -1189,8 +1228,13 @@ static void read_loose_refs(const char *dirname, struct ref_dir *dir)
 				hashclr(sha1);
 				flag |= REF_ISBROKEN;
 			}
+			if (check_refname_format(refname.buf,
+						 REFNAME_ALLOW_ONELEVEL)) {
+				hashclr(sha1);
+				flag |= REF_BAD_NAME | REF_ISBROKEN;
+			}
 			add_entry_to_dir(dir,
-					 create_ref_entry(refname.buf, sha1, flag, 1));
+					 create_ref_entry(refname.buf, sha1, flag, 0));
 		}
 		strbuf_setlen(&refname, dirnamelen);
 	}
@@ -1309,10 +1353,10 @@ static struct ref_entry *get_packed_ref(const char *refname)
  * A loose ref file doesn't exist; check for a packed ref.  The
  * options are forwarded from resolve_safe_unsafe().
  */
-static const char *handle_missing_loose_ref(const char *refname,
-					    int resolve_flags,
-					    unsigned char *sha1,
-					    int *flags)
+static int resolve_missing_loose_ref(const char *refname,
+				     int resolve_flags,
+				     unsigned char *sha1,
+				     int *flags)
 {
 	struct ref_entry *entry;
 
@@ -1325,14 +1369,15 @@ static const char *handle_missing_loose_ref(const char *refname,
 		hashcpy(sha1, entry->u.value.sha1);
 		if (flags)
 			*flags |= REF_ISPACKED;
-		return refname;
+		return 0;
 	}
 	/* The reference is not a packed reference, either. */
 	if (resolve_flags & RESOLVE_REF_READING) {
-		return NULL;
+		errno = ENOENT;
+		return -1;
 	} else {
 		hashclr(sha1);
-		return refname;
+		return 0;
 	}
 }
 
@@ -1343,13 +1388,29 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned
 	ssize_t len;
 	char buffer[256];
 	static char refname_buffer[256];
+	int bad_name = 0;
 
 	if (flags)
 		*flags = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
-		errno = EINVAL;
-		return NULL;
+		if (flags)
+			*flags |= REF_BAD_NAME;
+
+		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+		    !refname_is_safe(refname)) {
+			errno = EINVAL;
+			return NULL;
+		}
+		/*
+		 * dwim_ref() uses REF_ISBROKEN to distinguish between
+		 * missing refs and refs that were present but invalid,
+		 * to complain about the latter to stderr.
+		 *
+		 * We don't know whether the ref exists, so don't set
+		 * REF_ISBROKEN yet.
+		 */
+		bad_name = 1;
 	}
 	for (;;) {
 		char path[PATH_MAX];
@@ -1375,11 +1436,17 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned
 		 */
 	stat_ref:
 		if (lstat(path, &st) < 0) {
-			if (errno == ENOENT)
-				return handle_missing_loose_ref(refname,
-						resolve_flags, sha1, flags);
-			else
+			if (errno != ENOENT)
 				return NULL;
+			if (resolve_missing_loose_ref(refname, resolve_flags,
+						      sha1, flags))
+				return NULL;
+			if (bad_name) {
+				hashclr(sha1);
+				if (flags)
+					*flags |= REF_ISBROKEN;
+			}
+			return refname;
 		}
 
 		/* Follow "normalized" - ie "refs/.." symlinks by hand */
@@ -1452,6 +1519,11 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned
 				errno = EINVAL;
 				return NULL;
 			}
+			if (bad_name) {
+				hashclr(sha1);
+				if (flags)
+					*flags |= REF_ISBROKEN;
+			}
 			return refname;
 		}
 		if (flags)
@@ -1466,9 +1538,14 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags, unsigned
 		}
 		if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
 			if (flags)
-				*flags |= REF_ISBROKEN;
-			errno = EINVAL;
-			return NULL;
+				*flags |= REF_BAD_NAME | REF_ISBROKEN;
+
+			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+			    !refname_is_safe(buf)) {
+				errno = EINVAL;
+				return NULL;
+			}
+			bad_name = 1;
 		}
 	}
 }
@@ -2101,18 +2178,16 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	int missing = 0;
 	int attempts_remaining = 3;
 
-	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
-		errno = EINVAL;
-		return NULL;
-	}
-
 	lock = xcalloc(1, sizeof(struct ref_lock));
 	lock->lock_fd = -1;
 
 	if (mustexist)
 		resolve_flags |= RESOLVE_REF_READING;
-	if (flags & REF_NODEREF && flags & REF_DELETING)
-		resolve_flags |= RESOLVE_REF_NO_RECURSE;
+	if (flags & REF_DELETING) {
+		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
+		if (flags & REF_NODEREF)
+			resolve_flags |= RESOLVE_REF_NO_RECURSE;
+	}
 
 	refname = resolve_ref_unsafe(refname, resolve_flags,
 				     lock->old_sha1, &type);
@@ -3465,6 +3540,13 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	if (have_old && !old_sha1)
 		die("BUG: have_old is true but old_sha1 is NULL");
 
+	if (!is_null_sha1(new_sha1) &&
+	    check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		strbuf_addf(err, "refusing to update ref with bad name %s",
+			    refname);
+		return -1;
+	}
+
 	update = add_update(transaction, refname);
 	hashcpy(update->new_sha1, new_sha1);
 	update->flags = flags;
@@ -3490,6 +3572,12 @@ int ref_transaction_create(struct ref_transaction *transaction,
 	if (!new_sha1 || is_null_sha1(new_sha1))
 		die("BUG: create ref with null new_sha1");
 
+	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+		strbuf_addf(err, "refusing to create ref with bad name %s",
+			    refname);
+		return -1;
+	}
+
 	update = add_update(transaction, refname);
 
 	hashcpy(update->new_sha1, new_sha1);
diff --git a/refs.h b/refs.h
index e48d8b9..3b35387 100644
--- a/refs.h
+++ b/refs.h
@@ -56,11 +56,15 @@ struct ref_transaction;
 
 /*
  * Reference cannot be resolved to an object name: dangling symbolic
- * reference (directly or indirectly), corrupt reference file, or
- * symbolic reference refers to ill-formatted reference name.
+ * reference (directly or indirectly), corrupt reference file,
+ * reference exists but name is bad, or symbolic reference refers to
+ * ill-formatted reference name.
  */
 #define REF_ISBROKEN 0x04
 
+/* Reference name is not well formed (see git-check-ref-format(1)). */
+#define REF_BAD_NAME 0x08
+
 /*
  * The signature for the callback function for the for_each_*()
  * functions below.  The memory pointed to by the refname and sha1
diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
index c7b19b0..468e856 100755
--- a/t/t1430-bad-ref-name.sh
+++ b/t/t1430-bad-ref-name.sh
@@ -4,7 +4,8 @@ test_description='Test handling of ref names that check-ref-format rejects'
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_commit one
+	test_commit one &&
+	test_commit two
 '
 
 test_expect_success 'fast-import: fail on invalid branch name ".badbranchname"' '
@@ -37,6 +38,107 @@ test_expect_success 'fast-import: fail on invalid branch name "bad[branch]name"'
 	test_must_fail git fast-import <input
 '
 
+test_expect_success 'git branch shows badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch >output &&
+	grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -d can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -d broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -D can delete badly named ref' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -D broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -D cannot delete non-ref in .git dir' '
+	echo precious >.git/my-private-file &&
+	echo precious >expect &&
+	test_must_fail git branch -D ../../my-private-file &&
+	test_cmp expect .git/my-private-file
+'
+
+test_expect_success 'branch -D cannot delete absolute path' '
+	git branch -f extra &&
+	test_must_fail git branch -D "$(pwd)/.git/refs/heads/extra" &&
+	test_cmp_rev HEAD extra
+'
+
+test_expect_success 'git branch cannot create a badly named ref' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	test_must_fail git branch broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'branch -m cannot rename to a bad ref name' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	test_might_fail git branch -D goodref &&
+	git branch goodref &&
+	test_must_fail git branch -m goodref broken...ref &&
+	test_cmp_rev master goodref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_failure 'branch -m can rename from a bad ref name' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch -m broken...ref renamed &&
+	test_cmp_rev master renamed &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'push cannot create a badly named ref' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	test_must_fail git push "file://$(pwd)" HEAD:refs/heads/broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_failure 'push --mirror can delete badly named ref' '
+	top=$(pwd) &&
+	git init src &&
+	git init dest &&
+
+	(
+		cd src &&
+		test_commit one
+	) &&
+	(
+		cd dest &&
+		test_commit two &&
+		git checkout --detach &&
+		cp .git/refs/heads/master .git/refs/heads/broken...ref
+	) &&
+	git -C src push --mirror "file://$top/dest" &&
+	git -C dest branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'rev-parse skips symref pointing to broken name' '
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git branch shadow one &&
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	git symbolic-ref refs/tags/shadow refs/heads/broken...ref &&
+
+	git rev-parse --verify one >expect &&
+	git rev-parse --verify shadow >actual 2>err &&
+	test_cmp expect actual &&
+	test_i18ngrep "ignoring.*refs/tags/shadow" err
+'
+
 test_expect_success 'update-ref --no-deref -d can delete reference to broken name' '
 	git symbolic-ref refs/heads/badname refs/heads/broken...ref &&
 	test_when_finished "rm -f .git/refs/heads/badname" &&
@@ -45,6 +147,27 @@ test_expect_success 'update-ref --no-deref -d can delete reference to broken nam
 	test_path_is_missing .git/refs/heads/badname
 '
 
+test_expect_success 'update-ref -d can delete broken name' '
+	cp .git/refs/heads/master .git/refs/heads/broken...ref &&
+	test_when_finished "rm -f .git/refs/heads/broken...ref" &&
+	git update-ref -d refs/heads/broken...ref &&
+	git branch >output &&
+	! grep -e "broken\.\.\.ref" output
+'
+
+test_expect_success 'update-ref -d cannot delete non-ref in .git dir' '
+	echo precious >.git/my-private-file &&
+	echo precious >expect &&
+	test_must_fail git update-ref -d my-private-file &&
+	test_cmp expect .git/my-private-file
+'
+
+test_expect_success 'update-ref -d cannot delete absolute path' '
+	git branch -f extra &&
+	test_must_fail git update-ref -d "$(pwd)/.git/refs/heads/extra" &&
+	test_cmp_rev HEAD extra
+'
+
 test_expect_success 'update-ref --stdin fails create with bad ref name' '
 	echo "create ~a refs/heads/master" >stdin &&
 	test_must_fail git update-ref --stdin <stdin 2>err &&
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 20/24] for-each-ref.c: improve message before aborting on broken ref

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Fri, 5 Sep 2014 14:35:17 -0700

Print a warning message for any badly named refs we find in the repo.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
As before.

 builtin/for-each-ref.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c
index 20949b7..a88d681 100644
--- a/builtin/for-each-ref.c
+++ b/builtin/for-each-ref.c
@@ -853,6 +853,11 @@ static int grab_single_ref(const char *refname, const unsigned char *sha1, int f
 	struct refinfo *ref;
 	int cnt;
 
+	if (flag & REF_BAD_NAME) {
+		  warning("ignoring ref with broken name %s", refname);
+		  return 0;
+	}
+
 	if (*cb->grab_pattern) {
 		const char **pattern;
 		int namelen = strlen(refname);
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 21/24] remote rm/prune: print a message when writing packed-refs fails

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 11 Sep 2014 08:42:57 -0700

Until v2.1.0-rc0~22^2~11 (refs.c: add an err argument to
repack_without_refs, 2014-06-20), repack_without_refs forgot to
provide an error message when commit_packed_refs fails.  Even today,
it only provides a message for callers that pass a non-NULL err
parameter.  Internal callers in refs.c pass non-NULL err but
"git remote" does not.

That means that "git remote rm" and "git remote prune" can fail
without printing a message about why.  Fix them by passing in a
non-NULL err parameter and printing the returned message.

This is the last caller to a ref handling function passing err ==
NULL.  A later patch can drop support for err == NULL, avoiding such
problems in the future.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
---
Since v21:
- s/without/about/ in the commit message

 builtin/remote.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/builtin/remote.c b/builtin/remote.c
index c7f82f4..8517cfa 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -750,13 +750,16 @@ static int mv(int argc, const char **argv)
 
 static int remove_branches(struct string_list *branches)
 {
+	struct strbuf err = STRBUF_INIT;
 	const char **branch_names;
 	int i, result = 0;
 
 	branch_names = xmalloc(branches->nr * sizeof(*branch_names));
 	for (i = 0; i < branches->nr; i++)
 		branch_names[i] = branches->items[i].string;
-	result |= repack_without_refs(branch_names, branches->nr, NULL);
+	if (repack_without_refs(branch_names, branches->nr, &err))
+		result |= error("%s", err.buf);
+	strbuf_release(&err);
 	free(branch_names);
 
 	for (i = 0; i < branches->nr; i++) {
@@ -1333,9 +1336,13 @@ static int prune_remote(const char *remote, int dry_run)
 		delete_refs = xmalloc(states.stale.nr * sizeof(*delete_refs));
 		for (i = 0; i < states.stale.nr; i++)
 			delete_refs[i] = states.stale.items[i].util;
-		if (!dry_run)
-			result |= repack_without_refs(delete_refs,
-						      states.stale.nr, NULL);
+		if (!dry_run) {
+			struct strbuf err = STRBUF_INIT;
+			if (repack_without_refs(delete_refs, states.stale.nr,
+						&err))
+				result |= error("%s", err.buf);
+			strbuf_release(&err);
+		}
 		free(delete_refs);
 	}
 
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 22/24] refs.c: do not permit err == NULL

[Jonathan Nieder]

Date: Thu, 28 Aug 2014 16:42:37 -0700

Some functions that take a strbuf argument to append an error treat
!err as an indication that the message should be suppressed (e.g.,
ref_update_reject_duplicates).  Others write the message to stderr on
!err (e.g., repack_without_refs).  Others crash (e.g.,
ref_transaction_update).

Some of these behaviors are for historical reasons and others were
accidents.  Luckily no callers pass err == NULL any more.  Simplify
by consistently requiring the strbuf argument.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Since v21:
- dropped spurious 'to' from commit message

 refs.c | 46 +++++++++++++++++++++++++++-------------------
 1 file changed, 27 insertions(+), 19 deletions(-)

diff --git a/refs.c b/refs.c
index 0d3abb1..e13f843 100644
--- a/refs.c
+++ b/refs.c
@@ -2596,6 +2596,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 	struct string_list_item *ref_to_delete;
 	int i, ret, removed = 0;
 
+	assert(err);
+
 	/* Look for a packed ref */
 	for (i = 0; i < n; i++)
 		if (get_packed_ref(refnames[i]))
@@ -2606,13 +2608,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
-		if (err) {
-			unable_to_lock_message(git_path("packed-refs"), errno,
-					       err);
-			return -1;
-		}
-		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refnames[i]);
+		unable_to_lock_message(git_path("packed-refs"), errno, err);
+		return -1;
 	}
 	packed = get_packed_refs(&ref_cache);
 
@@ -2638,7 +2635,7 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 
 	/* Write what remains */
 	ret = commit_packed_refs();
-	if (ret && err)
+	if (ret)
 		strbuf_addf(err, "unable to overwrite old ref-pack file: %s",
 			    strerror(errno));
 	return ret;
@@ -2646,6 +2643,8 @@ int repack_without_refs(const char **refnames, int n, struct strbuf *err)
 
 static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
 {
+	assert(err);
+
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
 		/* loose */
 		int res, i = strlen(lock->lk->filename) - 5; /* .lock */
@@ -3495,6 +3494,8 @@ struct ref_transaction {
 
 struct ref_transaction *ref_transaction_begin(struct strbuf *err)
 {
+	assert(err);
+
 	return xcalloc(1, sizeof(struct ref_transaction));
 }
 
@@ -3534,6 +3535,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: update called for transaction that is not open");
 
@@ -3566,6 +3569,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: create called for transaction that is not open");
 
@@ -3597,6 +3602,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 {
 	struct ref_update *update;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: delete called for transaction that is not open");
 
@@ -3659,13 +3666,14 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,
 					struct strbuf *err)
 {
 	int i;
+
+	assert(err);
+
 	for (i = 1; i < n; i++)
 		if (!strcmp(updates[i - 1]->refname, updates[i]->refname)) {
-			const char *str =
-				"Multiple updates for ref '%s' not allowed.";
-			if (err)
-				strbuf_addf(err, str, updates[i]->refname);
-
+			strbuf_addf(err,
+				    "Multiple updates for ref '%s' not allowed.",
+				    updates[i]->refname);
 			return 1;
 		}
 	return 0;
@@ -3679,6 +3687,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 	int n = transaction->nr;
 	struct ref_update **updates = transaction->updates;
 
+	assert(err);
+
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: commit called for transaction that is not open");
 
@@ -3715,9 +3725,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 			ret = (errno == ENOTDIR)
 				? TRANSACTION_NAME_CONFLICT
 				: TRANSACTION_GENERIC_ERROR;
-			if (err)
-				strbuf_addf(err, "Cannot lock the ref '%s'.",
-					    update->refname);
+			strbuf_addf(err, "Cannot lock the ref '%s'.",
+				    update->refname);
 			goto cleanup;
 		}
 	}
@@ -3730,9 +3739,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 			if (write_ref_sha1(update->lock, update->new_sha1,
 					   update->msg)) {
 				update->lock = NULL; /* freed by write_ref_sha1 */
-				if (err)
-					strbuf_addf(err, "Cannot update the ref '%s'.",
-						    update->refname);
+				strbuf_addf(err, "Cannot update the ref '%s'.",
+					    update->refname);
 				ret = TRANSACTION_GENERIC_ERROR;
 				goto cleanup;
 			}
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 23/24] lockfile: remove unable_to_lock_error

[Jonathan Nieder]

Date: Thu, 28 Aug 2014 16:41:34 -0700

The former caller uses unable_to_lock_message now.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
As before.

 cache.h    |  1 -
 lockfile.c | 10 ----------
 2 files changed, 11 deletions(-)

diff --git a/cache.h b/cache.h
index 0c0ac60..f582b6c 100644
--- a/cache.h
+++ b/cache.h
@@ -558,7 +558,6 @@ struct lock_file {
 };
 #define LOCK_DIE_ON_ERROR 1
 #define LOCK_NODEREF 2
-extern int unable_to_lock_error(const char *path, int err);
 extern void unable_to_lock_message(const char *path, int err,
 				   struct strbuf *buf);
 extern NORETURN void unable_to_lock_index_die(const char *path, int err);
diff --git a/lockfile.c b/lockfile.c
index a921d77..dbd4101 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -176,16 +176,6 @@ void unable_to_lock_message(const char *path, int err, struct strbuf *buf)
 			    absolute_path(path), strerror(err));
 }
 
-int unable_to_lock_error(const char *path, int err)
-{
-	struct strbuf buf = STRBUF_INIT;
-
-	unable_to_lock_message(path, err, &buf);
-	error("%s", buf.buf);
-	strbuf_release(&buf);
-	return -1;
-}
-
 NORETURN void unable_to_lock_index_die(const char *path, int err)
 {
 	struct strbuf buf = STRBUF_INIT;
-- 
2.1.0.rc2.206.gedb03e5

[PATCH 24/24] ref_transaction_commit: bail out on failure to remove a ref

[Jonathan Nieder]

Date: Thu, 28 Aug 2014 17:01:35 -0700

When removal of a loose or packed ref fails, bail out instead of
trying to finish the transaction.  This way, a single error message
can be printed (instead of multiple messages being concatenated by
mistake) and the operator can try to solve the underlying problem
before there is a chance to muck things up even more.

In particular, when git fails to remove a ref, git goes on to try to
delete the reflog.  Exiting early lets us keep the reflog.

When git succeeds in deleting a ref A and fails to remove a ref B, it
goes on to try to delete both reflogs.  It would be better to just
remove the reflog for A, but that would be a more invasive change.
Failing early means we keep both reflogs, which puts the operator in a
good position to understand the problem and recover.

A long term goal is to avoid these problems altogether and roll back
the transaction on failure.  That kind of transactionality will have
to wait for a later series (the plan for which is to make all
destructive work happen in a single update of the packed-refs file).

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Ronnie Sahlberg <sahlberg@google.com>
---
That's the end of the series.  Thanks for reading.

 refs.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index e13f843..d9d327d 100644
--- a/refs.c
+++ b/refs.c
@@ -3753,16 +3753,20 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (update->lock) {
-			if (delete_ref_loose(update->lock, update->type, err))
+			if (delete_ref_loose(update->lock, update->type, err)) {
 				ret = TRANSACTION_GENERIC_ERROR;
+				goto cleanup;
+			}
 
 			if (!(update->flags & REF_ISPRUNING))
 				delnames[delnum++] = update->lock->ref_name;
 		}
 	}
 
-	if (repack_without_refs(delnames, delnum, err))
+	if (repack_without_refs(delnames, delnum, err)) {
 		ret = TRANSACTION_GENERIC_ERROR;
+		goto cleanup;
+	}
 	for (i = 0; i < delnum; i++)
 		unlink_or_warn(git_path("logs/%s", delnames[i]));
 	clear_loose_ref_cache(&ref_cache);
-- 
2.1.0.rc2.206.gedb03e5

Re: [PATCH 19/24] refs.c: allow listing and deleting badly named refs

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> From: Ronnie Sahlberg <sahlberg@google.com>
> ...
> In resolving functions, refuse to resolve refs that don't pass the
> check-ref-format(1) check unless the new RESOLVE_REF_ALLOW_BAD_NAME
> flag is passed.  Even with RESOLVE_REF_ALLOW_BAD_NAME, refuse to
> resolve refs that escape the refs/ directory and do not match the
> pattern [A-Z_]* (think "HEAD" and "MERGE_HEAD").
>
> In locking functions, refuse to act on badly named refs unless they
> are being deleted and either are in the refs/ directory or match [A-Z_]*.
>
> Just like other invalid refs, flag resolved, badly named refs with the
> REF_ISBROKEN flag, treat them as resolving to null_sha1, and skip them
> in all iteration functions except for for_each_rawref.
>
> Flag badly named refs with a REF_BAD_NAME flag to make it easier for
> future callers to notice and handle them specially.
>
> In the transaction API, refuse to create or update badly named refs,
> but allow deleting them (unless they escape refs/ and don't match
> [A-Z_]*).
>
> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>

Thanks.  We originally threw all the different kind of breakages
into ISBROKEN, but a ref can have a malformed name or can contain a
bad/non value and allowing us to tell them apart is a good direction
to go.

> diff --git a/cache.h b/cache.h
> index 5ca7f2b..0c0ac60 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -978,16 +978,26 @@ extern int read_ref(const char *refname, unsigned char *sha1);
>   * If flags is non-NULL, set the value that it points to the
>   * combination of REF_ISPACKED (if the reference was found among the
>   * packed references), REF_ISSYMREF (if the initial reference was a
> - * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
> + * symbolic reference), REF_BAD_NAME (if the reference name is ill
> + * formed --- see RESOLVE_REF_ALLOW_BAD_NAME below), and REF_ISBROKEN
> + * (if the ref is malformed).

You want to define "is malformed" here.

The original defines REF_ISBROKEN as "malformed" because

 (1) resolve_ref_unsafe() uses get_sha1_hex() and read_loose_refs()
     uses read_ref_full(), both to catch "malformed values" stored;

 (2) resolve_ref_unsafe() uses check_refname_format() and catches
     "malformed names" stored as a symref target.

I _think_ you are introducing ALLOW_BAD_NAME to allow add the third
class ".git/refs/remotes/origin/mal..formed..name".  I do not know
if they should be the same class as a symref with a good name
".git/refs/remotes/origin/HEAD" that points at a bad name
"mal..formed..name", which is (2) above).  Perhaps not.  (2) is
still above what is stored in the ref, and the ref in question may
or may not have a well-formed name, which is orthogonal.

So probably you left only the "value stored in the ref is malformed"
(in other words, "we expected to find 40-hex object name but didn't
find one") case for REF_ISBROKEN?

Do we want to separate "value is not 40-hex" and "a symref points at
a malformed refname" as separate "malformed value" errors?

> + * RESOLVE_REF_ALLOW_BAD_NAME allows resolving refs even when their
> + * name is invalid according to git-check-ref-format(1).  If the name
> + * is bad then the value stored in sha1 will be null_sha1 and the
> + * REF_ISBROKEN and REF_BAD_NAME flags will be set.

Viewed with that light, I am not sure if a badly named ref should
yield null_sha1[] (REF_ISBROKEN, which I am assuming is about a
value that is badly formatted and cannot be read, should keep
yielding it as before).  Wouldn't it make it harder for the user if
you give null_sha1[] back to somebody who is trying to recover by
reading "refs/heads/mal..formed", creating "refs/heads/sensible" to
point at the same value and then removing the former?

Note that I am not saying we should give back the parsed value at
this step in the series.  Perhaps there are some existing callers
that do not check for ISBROKEN flag and instead says "null_sha1[]
ref is to be rejected/ignored", in which case they may need to be
corrected before that happens.  Or there may be some reason I
overlooked that makes it not so useful if we returned the object
name stored in a ref whose name is malformed.  Just wondering.

> @@ -272,6 +272,37 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
>  	return dir;
>  }
>  
> +static int escapes_cwd(const char *path) {
> +	char *buf;
> +	int result;
> +
> +	if (is_absolute_path(path))
> +		return 1;
> +	buf = xmalloc(strlen(path) + 1);
> +	result = !!normalize_path_copy(buf, path);
> +	free(buf);
> +	return result;
> +}

I think this function is misnamed for two reasons.

 - It does not have anything to do with cwd; it does not make any
   difference to the outcome of this function given the same input,
   if 'pwd' says "/u/jc/git" or "/u/jc/git/Documentation", no?

 - Even if this had something to do with cwd, I would expect a
   function whose name is escapes_cwd("/u/jc/git/Documentation") to
   yield false when 'pwd' says "/u/jc/git", but the implementation
   unconditionally rejects absolute path.  In the context of the
   (sole) caller of this function, which deals with a refname "refs/...",
   it makes no sense to see an absolute path, but that does not have
   anything to do with "does this path escape cwd?", no?

> +/*
> + * Check if a refname is safe.
> + * For refs that start with "refs/" we consider it safe as long as the rest
> + * of the path components does not allow it to escape from this directory.
> + * For all other refs we only consider them safe iff they only contain
> + * upper case characters and '_'.
> + */

I presume that the exception is to accomodate for "HEAD", "ORIG_HEAD",
"MERGE_HEAD" and friends, but you probably do not want the readers to
guess.

> +static int refname_is_safe(const char *refname)
> +{
> +	if (starts_with(refname, "refs/"))
> +		return !escapes_cwd(refname + strlen("refs/"));
> +	while (*refname) {
> +		if (!isupper(*refname) && *refname != '_')
> +			return 0;
> +		refname++;
> +	}
> +	return 1;
> +}

Re: [PATCH 09/24] refs.c: pass a list of names to skip to is_refname_available

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> diff --git a/refs.c b/refs.c
> index f124c2b..6820c93 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -801,14 +801,16 @@ static int names_conflict(const char *refname1, const char *refname2)
>  
>  struct name_conflict_cb {
>  	const char *refname;
> -	const char *oldrefname;
>  	const char *conflicting_refname;
> +	struct string_list *skiplist;
>  };

As cbe73331 (refs: speed up is_refname_available, 2014-09-10)
touches the same area and is now in 'master', the logic around here
in this series needs to be reworked.

Thanks.

Re: [PATCH 09/24] refs.c: pass a list of names to skip to is_refname_available

[Jonathan Nieder]

Junio C Hamano wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> diff --git a/refs.c b/refs.c
>> index f124c2b..6820c93 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -801,14 +801,16 @@ static int names_conflict(const char *refname1, const char *refname2)
>> 
>>  struct name_conflict_cb {
>>  	const char *refname;
>> -	const char *oldrefname;
>>  	const char *conflicting_refname;
>> +	struct string_list *skiplist;
>>  };
>
> As cbe73331 (refs: speed up is_refname_available, 2014-09-10)
> touches the same area and is now in 'master', the logic around here
> in this series needs to be reworked.

Thanks for the heads up.  (I hadn't realized the ref-transaction-1 was
part of 'master' --- yay!)  Rebased and reworked:
https://code-review.googlesource.com/1027

I'll give a week or so for review comments on this version of the
series and then post a hopefully final version.

Re: [PATCH 09/24] refs.c: pass a list of names to skip to is_refname_available

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Junio C Hamano wrote:
> ...
>> As cbe73331 (refs: speed up is_refname_available, 2014-09-10)
>> touches the same area and is now in 'master', the logic around here
>> in this series needs to be reworked.
>
> Thanks for the heads up.  (I hadn't realized the ref-transaction-1 was
> part of 'master' --- yay!)  Rebased and reworked:
> https://code-review.googlesource.com/1027

Heh, I was sort of expecting that heavy-weight contributors that
throw many and/or large patch series would at least be paying
attention to "What's cooking" reports.

Will take a look over there (I really hate having to context switch
only to review this series, though).

> I'll give a week or so for review comments on this version of the
> series and then post a hopefully final version.

Thanks.

Re: [PATCH 19/24] refs.c: allow listing and deleting badly named refs

[Ronnie Sahlberg]

On Thu, Oct 2, 2014 at 11:55 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> From: Ronnie Sahlberg <sahlberg@google.com>
>> ...
>> In resolving functions, refuse to resolve refs that don't pass the
>> check-ref-format(1) check unless the new RESOLVE_REF_ALLOW_BAD_NAME
>> flag is passed.  Even with RESOLVE_REF_ALLOW_BAD_NAME, refuse to
>> resolve refs that escape the refs/ directory and do not match the
>> pattern [A-Z_]* (think "HEAD" and "MERGE_HEAD").
>>
>> In locking functions, refuse to act on badly named refs unless they
>> are being deleted and either are in the refs/ directory or match [A-Z_]*.
>>
>> Just like other invalid refs, flag resolved, badly named refs with the
>> REF_ISBROKEN flag, treat them as resolving to null_sha1, and skip them
>> in all iteration functions except for for_each_rawref.
>>
>> Flag badly named refs with a REF_BAD_NAME flag to make it easier for
>> future callers to notice and handle them specially.
>>
>> In the transaction API, refuse to create or update badly named refs,
>> but allow deleting them (unless they escape refs/ and don't match
>> [A-Z_]*).
>>
>> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
>> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
>
> Thanks.  We originally threw all the different kind of breakages
> into ISBROKEN, but a ref can have a malformed name or can contain a
> bad/non value and allowing us to tell them apart is a good direction
> to go.
>
>> diff --git a/cache.h b/cache.h
>> index 5ca7f2b..0c0ac60 100644
>> --- a/cache.h
>> +++ b/cache.h
>> @@ -978,16 +978,26 @@ extern int read_ref(const char *refname, unsigned char *sha1);
>>   * If flags is non-NULL, set the value that it points to the
>>   * combination of REF_ISPACKED (if the reference was found among the
>>   * packed references), REF_ISSYMREF (if the initial reference was a
>> - * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
>> + * symbolic reference), REF_BAD_NAME (if the reference name is ill
>> + * formed --- see RESOLVE_REF_ALLOW_BAD_NAME below), and REF_ISBROKEN
>> + * (if the ref is malformed).
>
> You want to define "is malformed" here.
>
> The original defines REF_ISBROKEN as "malformed" because
>
>  (1) resolve_ref_unsafe() uses get_sha1_hex() and read_loose_refs()
>      uses read_ref_full(), both to catch "malformed values" stored;
>
>  (2) resolve_ref_unsafe() uses check_refname_format() and catches
>      "malformed names" stored as a symref target.
>
> I _think_ you are introducing ALLOW_BAD_NAME to allow add the third
> class ".git/refs/remotes/origin/mal..formed..name".  I do not know
> if they should be the same class as a symref with a good name
> ".git/refs/remotes/origin/HEAD" that points at a bad name
> "mal..formed..name", which is (2) above).  Perhaps not.  (2) is
> still above what is stored in the ref, and the ref in question may
> or may not have a well-formed name, which is orthogonal.
>
> So probably you left only the "value stored in the ref is malformed"
> (in other words, "we expected to find 40-hex object name but didn't
> find one") case for REF_ISBROKEN?

I updated cache.h to try to clarify it better.
The intention here is to expand the use of REF_ISBROKEN.

For all cases  REF_ISBROKEN will be set. This includes both "the sha1
value is bad" as well as "a name is bad".

For those cases where a name is bad then REF_BAD_NAME is also set. Bad
names are when either the ref itself has a bad name or when a bad name
is encountered while resolving a chain of symbilic refs.


I.e.  REF_BAD_NAME is a special case of broken ref. All REF_BAD_NAME
refs are also REF_ISBROKEN but not the reverse.


>
> Do we want to separate "value is not 40-hex" and "a symref points at
> a malformed refname" as separate "malformed value" errors?
>
>> + * RESOLVE_REF_ALLOW_BAD_NAME allows resolving refs even when their
>> + * name is invalid according to git-check-ref-format(1).  If the name
>> + * is bad then the value stored in sha1 will be null_sha1 and the
>> + * REF_ISBROKEN and REF_BAD_NAME flags will be set.
>
> Viewed with that light, I am not sure if a badly named ref should
> yield null_sha1[] (REF_ISBROKEN, which I am assuming is about a
> value that is badly formatted and cannot be read, should keep
> yielding it as before).  Wouldn't it make it harder for the user if
> you give null_sha1[] back to somebody who is trying to recover by
> reading "refs/heads/mal..formed", creating "refs/heads/sensible" to
> point at the same value and then removing the former?
>
> Note that I am not saying we should give back the parsed value at
> this step in the series.  Perhaps there are some existing callers
> that do not check for ISBROKEN flag and instead says "null_sha1[]
> ref is to be rejected/ignored", in which case they may need to be
> corrected before that happens.  Or there may be some reason I
> overlooked that makes it not so useful if we returned the object
> name stored in a ref whose name is malformed.  Just wondering.

The reason for these malformed refs resolving to null_sha1 is exactly
that. There may be callers that do not check ISBROKEN, so all those
callers need to be carefully audited before we start returning
potentially valid non null_sha1 here.
Right now I only want to do the minimal changes needed to open up only
those few paths I need to in order to allow the refs to be deleted.
For the delete case, returning the null_sha1 is sufficient.


Now, IF we add support to "rename a bad-ref-name to a good-ref-name",
then for that case we would need to start allowing resolve_ref_unsafe
to actually return the sha1 for the ref.
And that would require that we do those changes and audits. Given how
rare this should be, I am not convinced we need "rename bad to good"
at this point and would thus prefer to hold off adding that.

Thus, returning the object name is not so useful for the problem I try
to solve at this stage, namely "make delete work".


>
>> @@ -272,6 +272,37 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
>>       return dir;
>>  }
>>
>> +static int escapes_cwd(const char *path) {
>> +     char *buf;
>> +     int result;
>> +
>> +     if (is_absolute_path(path))
>> +             return 1;
>> +     buf = xmalloc(strlen(path) + 1);
>> +     result = !!normalize_path_copy(buf, path);
>> +     free(buf);
>> +     return result;
>> +}
>
> I think this function is misnamed for two reasons.
>
>  - It does not have anything to do with cwd; it does not make any
>    difference to the outcome of this function given the same input,
>    if 'pwd' says "/u/jc/git" or "/u/jc/git/Documentation", no?
>
>  - Even if this had something to do with cwd, I would expect a
>    function whose name is escapes_cwd("/u/jc/git/Documentation") to
>    yield false when 'pwd' says "/u/jc/git", but the implementation
>    unconditionally rejects absolute path.  In the context of the
>    (sole) caller of this function, which deals with a refname "refs/...",
>    it makes no sense to see an absolute path, but that does not have
>    anything to do with "does this path escape cwd?", no?
>

Agree.
I removed this function completely and just inline the !normalize_path_copy()
checks to make sure that refs/* paths remain within refs/.


>> +/*
>> + * Check if a refname is safe.
>> + * For refs that start with "refs/" we consider it safe as long as the rest
>> + * of the path components does not allow it to escape from this directory.
>> + * For all other refs we only consider them safe iff they only contain
>> + * upper case characters and '_'.
>> + */
>
> I presume that the exception is to accomodate for "HEAD", "ORIG_HEAD",
> "MERGE_HEAD" and friends, but you probably do not want the readers to
> guess.
>

I updated the commend to highlight that this is for "HEAD" and friends.

>> +static int refname_is_safe(const char *refname)
>> +{
>> +     if (starts_with(refname, "refs/"))
>> +             return !escapes_cwd(refname + strlen("refs/"));
>> +     while (*refname) {
>> +             if (!isupper(*refname) && *refname != '_')
>> +                     return 0;
>> +             refname++;
>> +     }
>> +     return 1;
>> +}

Re: [PATCH 19/24] refs.c: allow listing and deleting badly named refs

[Ronnie Sahlberg]

On Fri, Oct 3, 2014 at 1:25 PM, Ronnie Sahlberg <sahlberg@google.com> wrote:
> On Thu, Oct 2, 2014 at 11:55 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jonathan Nieder <jrnieder@gmail.com> writes:
>>
>>> From: Ronnie Sahlberg <sahlberg@google.com>
>>> ...
>>> In resolving functions, refuse to resolve refs that don't pass the
>>> check-ref-format(1) check unless the new RESOLVE_REF_ALLOW_BAD_NAME
>>> flag is passed.  Even with RESOLVE_REF_ALLOW_BAD_NAME, refuse to
>>> resolve refs that escape the refs/ directory and do not match the
>>> pattern [A-Z_]* (think "HEAD" and "MERGE_HEAD").
>>>
>>> In locking functions, refuse to act on badly named refs unless they
>>> are being deleted and either are in the refs/ directory or match [A-Z_]*.
>>>
>>> Just like other invalid refs, flag resolved, badly named refs with the
>>> REF_ISBROKEN flag, treat them as resolving to null_sha1, and skip them
>>> in all iteration functions except for for_each_rawref.
>>>
>>> Flag badly named refs with a REF_BAD_NAME flag to make it easier for
>>> future callers to notice and handle them specially.
>>>
>>> In the transaction API, refuse to create or update badly named refs,
>>> but allow deleting them (unless they escape refs/ and don't match
>>> [A-Z_]*).
>>>
>>> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
>>> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
>>
>> Thanks.  We originally threw all the different kind of breakages
>> into ISBROKEN, but a ref can have a malformed name or can contain a
>> bad/non value and allowing us to tell them apart is a good direction
>> to go.
>>
>>> diff --git a/cache.h b/cache.h
>>> index 5ca7f2b..0c0ac60 100644
>>> --- a/cache.h
>>> +++ b/cache.h
>>> @@ -978,16 +978,26 @@ extern int read_ref(const char *refname, unsigned char *sha1);
>>>   * If flags is non-NULL, set the value that it points to the
>>>   * combination of REF_ISPACKED (if the reference was found among the
>>>   * packed references), REF_ISSYMREF (if the initial reference was a
>>> - * symbolic reference) and REF_ISBROKEN (if the ref is malformed).
>>> + * symbolic reference), REF_BAD_NAME (if the reference name is ill
>>> + * formed --- see RESOLVE_REF_ALLOW_BAD_NAME below), and REF_ISBROKEN
>>> + * (if the ref is malformed).
>>
>> You want to define "is malformed" here.
>>
>> The original defines REF_ISBROKEN as "malformed" because
>>
>>  (1) resolve_ref_unsafe() uses get_sha1_hex() and read_loose_refs()
>>      uses read_ref_full(), both to catch "malformed values" stored;
>>
>>  (2) resolve_ref_unsafe() uses check_refname_format() and catches
>>      "malformed names" stored as a symref target.
>>
>> I _think_ you are introducing ALLOW_BAD_NAME to allow add the third
>> class ".git/refs/remotes/origin/mal..formed..name".  I do not know
>> if they should be the same class as a symref with a good name
>> ".git/refs/remotes/origin/HEAD" that points at a bad name
>> "mal..formed..name", which is (2) above).  Perhaps not.  (2) is
>> still above what is stored in the ref, and the ref in question may
>> or may not have a well-formed name, which is orthogonal.
>>
>> So probably you left only the "value stored in the ref is malformed"
>> (in other words, "we expected to find 40-hex object name but didn't
>> find one") case for REF_ISBROKEN?
>
> I updated cache.h to try to clarify it better.
> The intention here is to expand the use of REF_ISBROKEN.
>
> For all cases  REF_ISBROKEN will be set. This includes both "the sha1
> value is bad" as well as "a name is bad".
>
> For those cases where a name is bad then REF_BAD_NAME is also set. Bad
> names are when either the ref itself has a bad name or when a bad name
> is encountered while resolving a chain of symbilic refs.
>
>
> I.e.  REF_BAD_NAME is a special case of broken ref. All REF_BAD_NAME
> refs are also REF_ISBROKEN but not the reverse.

Let me add some rationale why REF_BAD_NAME and REF_ISBROKEN are
notorthogonal properties.

I think almost all callers of these APIs are only concerned about "is
the ref good or bad" and they today only check REF_ISBROKEN.
I think that is a reasonable API and it allows the majority of callers
to just "check this single flag".
(The alternative would be to keep all callers in sync and use a set of
flags for all "bad conditions".)

A very small subset of callers are actually interested in knowing why
the ref was bad, and in particular if the ref was bad due to a name
component.
Those callers, that are aware that there are different types of
ISBROKEN can then inspect the REF_BAD_NAME flag in order to decide
"is the ref broken due to the ref name?".


This is why I made REF_BAD_NAME a special case of REF_ISBROKEN.



>
>
>>
>> Do we want to separate "value is not 40-hex" and "a symref points at
>> a malformed refname" as separate "malformed value" errors?
>>
>>> + * RESOLVE_REF_ALLOW_BAD_NAME allows resolving refs even when their
>>> + * name is invalid according to git-check-ref-format(1).  If the name
>>> + * is bad then the value stored in sha1 will be null_sha1 and the
>>> + * REF_ISBROKEN and REF_BAD_NAME flags will be set.
>>
>> Viewed with that light, I am not sure if a badly named ref should
>> yield null_sha1[] (REF_ISBROKEN, which I am assuming is about a
>> value that is badly formatted and cannot be read, should keep
>> yielding it as before).  Wouldn't it make it harder for the user if
>> you give null_sha1[] back to somebody who is trying to recover by
>> reading "refs/heads/mal..formed", creating "refs/heads/sensible" to
>> point at the same value and then removing the former?
>>
>> Note that I am not saying we should give back the parsed value at
>> this step in the series.  Perhaps there are some existing callers
>> that do not check for ISBROKEN flag and instead says "null_sha1[]
>> ref is to be rejected/ignored", in which case they may need to be
>> corrected before that happens.  Or there may be some reason I
>> overlooked that makes it not so useful if we returned the object
>> name stored in a ref whose name is malformed.  Just wondering.
>
> The reason for these malformed refs resolving to null_sha1 is exactly
> that. There may be callers that do not check ISBROKEN, so all those
> callers need to be carefully audited before we start returning
> potentially valid non null_sha1 here.
> Right now I only want to do the minimal changes needed to open up only
> those few paths I need to in order to allow the refs to be deleted.
> For the delete case, returning the null_sha1 is sufficient.
>
>
> Now, IF we add support to "rename a bad-ref-name to a good-ref-name",
> then for that case we would need to start allowing resolve_ref_unsafe
> to actually return the sha1 for the ref.
> And that would require that we do those changes and audits. Given how
> rare this should be, I am not convinced we need "rename bad to good"
> at this point and would thus prefer to hold off adding that.
>
> Thus, returning the object name is not so useful for the problem I try
> to solve at this stage, namely "make delete work".
>
>
>>
>>> @@ -272,6 +272,37 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
>>>       return dir;
>>>  }
>>>
>>> +static int escapes_cwd(const char *path) {
>>> +     char *buf;
>>> +     int result;
>>> +
>>> +     if (is_absolute_path(path))
>>> +             return 1;
>>> +     buf = xmalloc(strlen(path) + 1);
>>> +     result = !!normalize_path_copy(buf, path);
>>> +     free(buf);
>>> +     return result;
>>> +}
>>
>> I think this function is misnamed for two reasons.
>>
>>  - It does not have anything to do with cwd; it does not make any
>>    difference to the outcome of this function given the same input,
>>    if 'pwd' says "/u/jc/git" or "/u/jc/git/Documentation", no?
>>
>>  - Even if this had something to do with cwd, I would expect a
>>    function whose name is escapes_cwd("/u/jc/git/Documentation") to
>>    yield false when 'pwd' says "/u/jc/git", but the implementation
>>    unconditionally rejects absolute path.  In the context of the
>>    (sole) caller of this function, which deals with a refname "refs/...",
>>    it makes no sense to see an absolute path, but that does not have
>>    anything to do with "does this path escape cwd?", no?
>>
>
> Agree.
> I removed this function completely and just inline the !normalize_path_copy()
> checks to make sure that refs/* paths remain within refs/.
>
>
>>> +/*
>>> + * Check if a refname is safe.
>>> + * For refs that start with "refs/" we consider it safe as long as the rest
>>> + * of the path components does not allow it to escape from this directory.
>>> + * For all other refs we only consider them safe iff they only contain
>>> + * upper case characters and '_'.
>>> + */
>>
>> I presume that the exception is to accomodate for "HEAD", "ORIG_HEAD",
>> "MERGE_HEAD" and friends, but you probably do not want the readers to
>> guess.
>>
>
> I updated the commend to highlight that this is for "HEAD" and friends.
>
>>> +static int refname_is_safe(const char *refname)
>>> +{
>>> +     if (starts_with(refname, "refs/"))
>>> +             return !escapes_cwd(refname + strlen("refs/"));
>>> +     while (*refname) {
>>> +             if (!isupper(*refname) && *refname != '_')
>>> +                     return 0;
>>> +             refname++;
>>> +     }
>>> +     return 1;
>>> +}

Re: [PATCH 19/24] refs.c: allow listing and deleting badly named refs

[Junio C Hamano]

Ronnie Sahlberg <sahlberg@google.com> writes:

> On Thu, Oct 2, 2014 at 11:55 AM, Junio C Hamano <gitster@pobox.com> wrote:
> ...
>> Thanks.  We originally threw all the different kind of breakages
>> into ISBROKEN, but a ref can have a malformed name or can contain a
>> bad/non value and allowing us to tell them apart is a good direction
>> to go.
>> ...
>
> I updated cache.h to try to clarify it better.
> The intention here is to expand the use of REF_ISBROKEN.
>
> For all cases  REF_ISBROKEN will be set. This includes both "the sha1
> value is bad" as well as "a name is bad".

OK.  As long as it is documented in a way to help other people who
touch the code later to tell what these REF_* mean, either way is
fine between "BADNAME is just one of the possible ways a ref is
ISBROKEN" and "BADNAME is a way and ISBROKEN is another way for a
ref to be bad", and I agree that the former (i.e. the way you
defined) is easier on the existing code.

Thanks.

[PATCH v22.5 09/24] refs.c: pass a list of names to skip to is_refname_available

[Jonathan Nieder]

From: Ronnie Sahlberg <sahlberg@google.com>
Date: Thu, 1 May 2014 11:16:07 -0700

commit 23acf975d74825789112a3a7ba97dbbdc76904f4 upstream.

Change is_refname_available to take a list of strings to exclude when
checking for conflicts instead of just one single name. We can already
exclude a single name for the sake of renames. This generalizes that support.

ref_transaction_commit already tracks a set of refs that are being deleted
in an array.  This array is then used to exclude refs from being written to
the packed-refs file.  At some stage we will want to change this array to a
struct string_list and then we can pass it to is_refname_available via the
call to lock_ref_sha1_basic.  That will allow us to perform transactions
that perform multiple renames as long as there are no conflicts within the
starting or ending state.

For example, that would allow a single transaction that contains two
renames that are both individually conflicting:

   m -> n/n
   n -> m/m

No functional change intended yet.

Change-Id: I8c68f0a47eef25759d572436ba683cb7b1ccb7eb
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Junio C Hamano wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> Thanks for the heads up.  (I hadn't realized the ref-transaction-1 was
>> part of 'master' --- yay!)  Rebased and reworked:
>> https://code-review.googlesource.com/1027
>
> Heh, I was sort of expecting that heavy-weight contributors that
> throw many and/or large patch series would at least be paying
> attention to "What's cooking" reports.

That's fair.  I don't pay enough attention to the 'graduated' section.

> Will take a look over there (I really hate having to context switch
> only to review this series, though).

Here's a copy to save a trip.

 refs.c | 50 ++++++++++++++++++++++++++++++++------------------
 1 file changed, 32 insertions(+), 18 deletions(-)

diff --git a/refs.c b/refs.c
index 83b2c77..05e42a7 100644
--- a/refs.c
+++ b/refs.c
@@ -785,13 +785,13 @@ static void prime_ref_dir(struct ref_dir *dir)
 	}
 }
 
-static int entry_matches(struct ref_entry *entry, const char *refname)
+static int entry_matches(struct ref_entry *entry, const struct string_list *list)
 {
-	return refname && !strcmp(entry->name, refname);
+	return list && string_list_has_string(list, entry->name);
 }
 
 struct nonmatching_ref_data {
-	const char *skip;
+	const struct string_list *skip;
 	struct ref_entry *found;
 };
 
@@ -815,16 +815,19 @@ static void report_refname_conflict(struct ref_entry *entry,
 /*
  * Return true iff a reference named refname could be created without
  * conflicting with the name of an existing reference in dir.  If
- * oldrefname is non-NULL, ignore potential conflicts with oldrefname
- * (e.g., because oldrefname is scheduled for deletion in the same
+ * skip is non-NULL, ignore potential conflicts with refs in skip
+ * (e.g., because they are scheduled for deletion in the same
  * operation).
  *
  * Two reference names conflict if one of them exactly matches the
  * leading components of the other; e.g., "foo/bar" conflicts with
  * both "foo" and with "foo/bar/baz" but not with "foo/bar" or
  * "foo/barbados".
+ *
+ * skip must be sorted.
  */
-static int is_refname_available(const char *refname, const char *oldrefname,
+static int is_refname_available(const char *refname,
+				const struct string_list *skip,
 				struct ref_dir *dir)
 {
 	const char *slash;
@@ -838,12 +841,12 @@ static int is_refname_available(const char *refname, const char *oldrefname,
 		 * looking for a conflict with a leaf entry.
 		 *
 		 * If we find one, we still must make sure it is
-		 * not "oldrefname".
+		 * not in "skip".
 		 */
 		pos = search_ref_dir(dir, refname, slash - refname);
 		if (pos >= 0) {
 			struct ref_entry *entry = dir->entries[pos];
-			if (entry_matches(entry, oldrefname))
+			if (entry_matches(entry, skip))
 				return 1;
 			report_refname_conflict(entry, refname);
 			return 0;
@@ -876,13 +879,13 @@ static int is_refname_available(const char *refname, const char *oldrefname,
 		/*
 		 * We found a directory named "refname". It is a
 		 * problem iff it contains any ref that is not
-		 * "oldrefname".
+		 * in "skip".
 		 */
 		struct ref_entry *entry = dir->entries[pos];
 		struct ref_dir *dir = get_ref_dir(entry);
 		struct nonmatching_ref_data data;
 
-		data.skip = oldrefname;
+		data.skip = skip;
 		sort_ref_dir(dir);
 		if (!do_for_each_entry_in_dir(dir, 0, nonmatching_ref_fn, &data))
 			return 1;
@@ -2137,6 +2140,7 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
  */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
+					    const struct string_list *skip,
 					    int flags, int *type_p)
 {
 	char *ref_file;
@@ -2186,7 +2190,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	 * name is a proper prefix of our refname.
 	 */
 	if (missing &&
-	     !is_refname_available(refname, NULL, get_packed_refs(&ref_cache))) {
+	     !is_refname_available(refname, skip, get_packed_refs(&ref_cache))) {
 		last_errno = ENOTDIR;
 		goto error_return;
 	}
@@ -2244,7 +2248,7 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,
 					 const unsigned char *old_sha1,
 					 int flags, int *type_p)
 {
-	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
+	return lock_ref_sha1_basic(refname, old_sha1, NULL, flags, type_p);
 }
 
 /*
@@ -2690,6 +2694,18 @@ static int rename_tmp_log(const char *newrefname)
 	return 0;
 }
 
+static int rename_ref_available(const char *oldname, const char *newname)
+{
+	struct string_list skip = STRING_LIST_INIT_NODUP;
+	int ret;
+
+	string_list_insert(&skip, oldname);
+	ret = is_refname_available(newname, &skip, get_packed_refs(&ref_cache))
+	    && is_refname_available(newname, &skip, get_loose_refs(&ref_cache));
+	string_list_clear(&skip, 0);
+	return ret;
+}
+
 int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)
 {
 	unsigned char sha1[20], orig_sha1[20];
@@ -2709,10 +2725,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	if (!symref)
 		return error("refname %s not found", oldrefname);
 
-	if (!is_refname_available(newrefname, oldrefname, get_packed_refs(&ref_cache)))
-		return 1;
-
-	if (!is_refname_available(newrefname, oldrefname, get_loose_refs(&ref_cache)))
+	if (!rename_ref_available(oldrefname, newrefname))
 		return 1;
 
 	if (log && rename(git_path("logs/%s", oldrefname), git_path(TMP_RENAMED_LOG)))
@@ -2742,7 +2755,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 
 	logmoved = log;
 
-	lock = lock_ref_sha1_basic(newrefname, NULL, 0, NULL);
+	lock = lock_ref_sha1_basic(newrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for update", newrefname);
 		goto rollback;
@@ -2757,7 +2770,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	return 0;
 
  rollback:
-	lock = lock_ref_sha1_basic(oldrefname, NULL, 0, NULL);
+	lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, 0, NULL);
 	if (!lock) {
 		error("unable to lock %s for rollback", oldrefname);
 		goto rollbacklog;
@@ -3636,6 +3649,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 						   (update->have_old ?
 						    update->old_sha1 :
 						    NULL),
+						   NULL,
 						   update->flags,
 						   &update->type);
 		if (!update->lock) {
-- 
2.1.0.rc2.206.gedb03e5

Re: [PATCH v22.5 09/24] refs.c: pass a list of names to skip to is_refname_available

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

>> Will take a look over there (I really hate having to context switch
>> only to review this series, though).
>
> Here's a copy to save a trip.

The patch itself looks fine, but Gerrit really needs a way to convey
"these changes make a single topic and here is the tip" somehow.  It
could be that there is and it is not well advertised, but the end
result is I visit the URL,

  https://code-review.googlesource.com/#/q/project:git+topic:ref-transaction

pick and view a few patches at random to view the changes in pretty
side-by-side diff output (which by itself is fine), check "Download"
to see the fetch URL, when it is followed it often gives me only an
early half of the topic, get confused and scratch my head, give up
and abandon and the whole time doing so ends up being wasted.