Re: [yocto] [PATCH yocto-autobuilder-helper] scripts: run-docs-build: make the workdir pristine between builds

[Quentin Schulz <quentin.schulz@theobroma-systems.com>]

Hi Luka,

On 10/4/22 22:54, Luca Ceresoli wrote:
> Hi Quentin,
> 
> On Tue, 4 Oct 2022 10:15:20 +0200
> Quentin Schulz <quentin.schulz@theobroma-systems.com> wrote:
> 
>> Hi Luca,
>>
>> On 10/3/22 23:15, Luca Ceresoli wrote:
>>> Hi Quentin,
>>>
>>> On Mon,  3 Oct 2022 19:04:01 +0200
>>> "Quentin Schulz" <foss@0leil.net> wrote:
>>>    
>>>> From: Quentin Schulz <quentin.schulz@theobroma-systems.com>
>>>>
>>>> It happened that the git repositories were dirty and resulted in
>>>> incorrect files being used. Let's use git clean -ffdx to force a
>>>> completely clean git repositories before and after checking out a branch
>>>> so that nothing is left from or to another branch build
>>>>
>>>> Cc: Quentin Schulz <foss+yocto@0leil.net>
>>>> Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
>>>> ---
>>>>    scripts/run-docs-build | 10 +++++-----
>>>>    1 file changed, 5 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/scripts/run-docs-build b/scripts/run-docs-build
>>>> index c6b3965..69e3257 100755
>>>> --- a/scripts/run-docs-build
>>>> +++ b/scripts/run-docs-build
>>>> @@ -61,6 +61,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
>>>>    
>>>>        echo Building bitbake $branch branch
>>>>        git checkout $branch
>>>> +    git clean -ffdx
>>>>        git checkout origin/master releases.rst
>>>>        make clean
>>>>        SPHINXOPTS="-j auto" make publish
>>>> @@ -80,7 +81,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
>>>>        fi
>>>>    
>>>>        cp -r ./_build/final/* $outputdir/bitbake/$branch
>>>> -    git reset --hard
>>>> +    git clean -ffdx
>>>
>>> Sure this is correct? 'git clean -ffdx' does not revert changes to
>>> tracked files, be them staged or not.
>>>    
>>
>> Nope, not sure this is correct. I misread git clean manpage, we should
>> have a git reset --hard and git clean -ffdx. Now the question is when
>> those are necessary because with this patch we do it twice, before and
>> after the git checkout. I did this because I remember doing checkouts
>> between branches of U-Boot/kernel and while the pre-checkout branch was
>> not dirty, the after-checkout branch was dirty. I assume this might have
>> something to do with build artifacts of the pre-checkout build that
>> weren't .gitignored in the afer-checkout branch? Something that git
>> clean -ffdx should tackle I think.
>>
>> Sooo, I guess only having git reset --hard and git clean -ffdx before a
>> checkout should be enough and we don't need them both before and after
>> the checkout like I did in this patch?
> 
> I think 'reset --hard' + 'clean -ffdx' only before the checkout should
> be enough. However I'm not sure whether there are corner cases such as
> a file that is .gitignored in commit A and versioned in commit B or
> similar. Perhaps worth trying with reset+clean only before, and see

I guess it does not hurt to be on the safe side by having them before 
and after the git checkout then? Since the current issue went unnoticed 
for months...

> what happens. However I don't know exactly the initial problem you're
> trying to fix.
> 

https://lore.kernel.org/yocto-docs/e50abe3c777e4a23a752a3ec25ad0b2a@axis.com/T/#t

Cheers,
Quentin