git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "SZEDER Gábor" <szeder.dev@gmail.com>
To: phillip.wood@dunelm.org.uk
Cc: Junio C Hamano <gitster@pobox.com>,
	git@vger.kernel.org, Brian Norris <briannorris@chromium.org>,
	Johannes Schindelin <Johannes.Schindelin@gmx.de>
Subject: Re: [PATCH] t3429: try to protect against a potential racy todo file problem
Date: Mon, 25 Nov 2019 14:18:33 +0100	[thread overview]
Message-ID: <20191125131833.GD23183@szeder.dev> (raw)
In-Reply-To: <20191124211021.GB23183@szeder.dev>

On Sun, Nov 24, 2019 at 10:10:21PM +0100, SZEDER Gábor wrote:
> To notice a changed todo file the sequencer stores the file's stat
> data in its 'struct todo_list' instance, and compares it with the
> file's current stat data after 'reword', 'squash' and 'exec'
> instructions.  If the two stat data doesn't match, it re-reads the
> todo file.
> 
> Sounds simple, but there are some subtleties going on here:
> 
>   - The 'struct todo_list' holds the stat data from the time when the
>     todo file was last read.
> 
>   - This stat data in 'struct todo_list' is not updated when the
>     sequencer itself writes the todo file.
> 
>   - Before executing each instruction during an interactive rebase,
>     the sequencer always updates the todo file by removing the
>     just-about-to-be-executed instruction.  This changes the file's
>     size and inode [1].
> 
> Consequently, when the sequencer looks at the stat data after a
> 'reword', 'squash' or 'exec' instruction, it most likely finds that
> they differ, even when the user didn't modify the todo list at all!
> This is not an issue in practice, it just wastes a few cycles on
> re-reading the todo list that matches what the sequencer already has
> in memory anyway.

It can be much more than just a few cycles, because the total number
of parsed instructions from all the todo file re-reads can go
quadratic with the number of rebased commits.

The simple test below runs 'git rebase -i -x' on 1000 commits, which
takes over 14seconds to run.  If it doesn't re-read the todo file at
all (I simply deleted the whole condition block checking the stat data
and re-reading) it runs for only ~2.5secs.

Just another angle to consider...


  ---  >8  ---

test_expect_success 'test' '
	num_commits=1000 &&
	test_commit_bulk --filename=file $num_commits &&

	/usr/bin/time -f "Elapsed time: %e" \
		git rebase -i --root -x true 2>out &&

	grep "Executing: true" out >actual &&
	test_line_count = $num_commits actual &&

	# show the elapsed time
	tail -n2 out
'

  ---  >8  ---


  parent reply	other threads:[~2019-11-25 13:18 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-22 23:10 git 2.24: git revert <commit1> <commit2> requires extra '--continue'? Brian Norris
2019-11-23  0:34 ` SZEDER Gábor
2019-11-23  9:53   ` Phillip Wood
2019-11-23 17:20     ` [PATCH] sequencer: don't re-read todo for revert and cherry-pick SZEDER Gábor
2019-11-23 21:14       ` Johannes Schindelin
2019-11-24  4:49       ` Junio C Hamano
2019-11-24 10:44         ` Phillip Wood
2019-11-24 21:10           ` [PATCH] t3429: try to protect against a potential racy todo file problem SZEDER Gábor
2019-11-25  1:28             ` Junio C Hamano
2019-11-25  3:10             ` SZEDER Gábor
2019-11-25 13:18             ` SZEDER Gábor [this message]
2019-11-25 14:43               ` Phillip Wood
2019-11-25 15:15                 ` SZEDER Gábor
2019-11-25 16:40                   ` Phillip Wood
2019-11-25  1:10           ` [PATCH] sequencer: don't re-read todo for revert and cherry-pick Junio C Hamano
2019-11-25 10:47             ` Phillip Wood

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191125131833.GD23183@szeder.dev \
    --to=szeder.dev@gmail.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=briannorris@chromium.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=phillip.wood@dunelm.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).