Upcoming sparse release RC5

Upcoming sparse release RC5

[Christopher Li]

All the code I want is already in sparse-next. A little bit more testing
I will soon release the RC5.

I want to add two documents.

1) Something similar to the linux kernel submiting-patches.rst
     Especially we need to have some document for the purpose of the
     Signed-off-by: tag. Sparse has different open source license t(MIT) han
     linux kernel (GPL). Any one want to submit a patch?

2) I want to have a document for small project can be done on sparse.
    Thing we want to happen is sparse. The goal of this document is to
     invite more developer to join sparse development.

    We can discuss what we want to see in sparse in the future.
    I especially want some beginner friendly projects. Project that
    don't require too much detail knowledge of internal sparse to work on.

    A little bit of motion speech why you should try sparse if you are
    looking for compiler hacking or teaching compilers in course.
    - sparse is small. The full project compiles in less than 10 seconds on my
      old and not performing laptop.
    - sparse is fast. Typically, sparse can check on C file 1/10 of
time it take for
      gcc to generate object files.
    - sparse can digest  the full kernel source file. With sparse-llvm, sparse
       use llvm as back end to emit real machine code.

     Here is some brain dump, you are welcome to add yours or discuss/disagree
     with some of the projects.

     Project that are relative easy, does not require a lot of back
ground to work on:

     - Sparse all sparse warning message should include the option how
to disable it.
       e.g. "pre-process.c:2019:28: warning: Variable length array is used."
       should be something like
        "pre-process.c:2019:28: warning: Variable length array is
used. (-Wno-vla)"
     - extend test-inspect to inspect more AST fields.
     - Addd inspect to inspect instructions.
     - adding architecture handling in sparse similar to cgcc
     - parallel processing of test-suit
     - Howto: fix the kernel rcu related checker warnings
     - option to disable AST level inline.

    The harder project for more experienced sparse developers:
     - incremental declare on C type and function prototype.
     - move attribute out of ctype to allow easier to add new attribute.
     - serialize, general object walking driven by data structures.
     - serialize, write sparse byte code into file
     - serialize, load sparse byte code from file.
     - symbol index/linker, know which symbol in which byte code file.
     - inline function in instruction level
     - cross function checking
     - checker error output database
     - debug version of sparse do all the verification double
check.(might belong to easy)
     - debug: optimization step by step log
     - debug: fancy animation of CFG

     - phi node location (Luc has patch)
     - revisit crazy programmer warning, invalid ssa form.
     - ptrlist, looping while modify inside the loop.
     - dead code eliminate using ssa
     - constant propagation using ssa.
     - x86/arm back end instruction set define
     - register allocation.
     - emit x86/arm machine level code.

I think that should keep us busy for a while.

Chris

Fwd: Upcoming sparse release RC5

[Christopher Li]

Adding the sparse mailing list.

Agree. We can add the testsuilt validate IR and LLVM IR to the project list.

If we do more complicate validation, we might consider using a more text
processing friendly  language to do the validation.  I would pick python
if I am going to code it, but that is just me.

Chris


---------- Forwarded message ----------
From: Dibyendu Majumdar <mobile@majumdar.org.uk>
Date: Sat, Aug 5, 2017 at 12:09 PM
Subject: Re: Upcoming sparse release RC5
To: Christopher Li <sparse@chrisli.org>


Hi Chris,

On 5 August 2017 at 16:46, Christopher Li <sparse@chrisli.org> wrote:
>     We can discuss what we want to see in sparse in the future.

I think that the automated test suite needs to be enhanced to a)
validate IR, and b) validate LLVM output. At the moment most tests do
not appear to actually verify that the generated IR / simplifications
are correct. This makes it risky and error prone to make changes. The
LLVM verification can be an additional validation step to ensure that
the generated code actually runs correctly.

Regards
Dibyendu

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Sat, Aug 5, 2017 at 5:46 PM, Christopher Li <sparse@chrisli.org> wrote:
> All the code I want is already in sparse-next. A little bit more testing
> I will soon release the RC5.

sparse-next is currently not fast-forwardable from master.

-- Luc

Re: Upcoming sparse release RC5

[Christopher Li]

On Sat, Aug 5, 2017 at 6:32 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sat, Aug 5, 2017 at 5:46 PM, Christopher Li <sparse@chrisli.org> wrote:
>> All the code I want is already in sparse-next. A little bit more testing
>> I will soon release the RC5.
>
> sparse-next is currently not fast-forwardable from master.

I think I use one of your branch as base. I will fix it now.
I think just rebase sparse-next to master now.
It should fix the problem.

Chris

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Sun, Aug 6, 2017 at 12:50 AM, Christopher Li <sparse@chrisli.org> wrote:
>>> I will soon release the RC5.
>>
>> sparse-next is currently not fast-forwardable from master.
>
> I think I use one of your branch as base. I will fix it now.

You have now rebased an existing branch,
the one for "fix crash with sym->bb_target == NULL".
There was no reasons for that.

Please make good use of 'git pull' instead of rebasing external
branches, it's a pain.
Thanks.

-- Luc

Re: Upcoming sparse release RC5

[Christopher Li]

On Mon, Aug 7, 2017 at 12:41 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
>> I think I use one of your branch as base. I will fix it now.
>
> You have now rebased an existing branch,
> the one for "fix crash with sym->bb_target == NULL".
> There was no reasons for that.

I just do git rebase master from the sparse-next.

I think sparse-next was design to allow to rebase.

It is not about git pull any more, because I rebase the sparse-next
from your branch, I need rebase because I need to throw out the
patch replaced by yours in sparse-next.

Your branch is not fast forward to master.

If your branch is fast forward to master then I don't need to
rebase sparse-next to master again.

That being said, what is your recommended way to fix it instead
of rebase?

We can still fix up sparse-next as I haven't push to master.

Chris

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Mon, Aug 07, 2017 at 07:32:37PM -0400, Christopher Li wrote:
> That being said, what is your recommended way to fix it instead
> of rebase?
> 
> We can still fix up sparse-next as I haven't push to master.

There was something like:
	.. rc4 - fix-crashes-v3 - some others patches - sparse-next
               \ noipa = master

Where master must be stable and fix-crashes-v3 is the external branch.
The external branch is fast-forwardable to master but not always
based on the top of it (because it was developed & tested earlier).

So late in the release and with sparse-next having received
quite a bit testing I would expect to be stable (in the git meaning)
and master to be fast-forwardable to it.

So, IMO the thing to do in this case would have been to merge 
the external branch with master and only rebase the others patches
on top of this merge point. So we would have something like this:
	.. rc4 - fix-crashes-v3  - some others patches - sparse-next
               \ noipa = master /

but instead, because of the rebasing, we have:
	.. rc4 - fix-crashes-v3
	       \ master - rebased-fix-crashes-v3  - some others patches - sparse-next

and this create problems for development done on top of the external branch
(basically you force them to also rebase their work or to have duplicated
commits in the tree).

The correct workflow is to never rebase anything you pulled.


That said, I think that part of the problem is that things stay for too long
in this sparse-next. As soon as something is tested well enough, it should
move to master. 

-- Luc

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Mon, Aug 07, 2017 at 07:32:37PM -0400, Christopher Li wrote:
> We can still fix up sparse-next as I haven't push to master.

If you don't mind, it would be nice if you could use the pull request
I sent here: http://marc.info/?l=linux-sparse&m=150188507213269&w=4
(you never saw it because I copy-pasted "[PATCH v3]" in the title of
the cover page instead of using "[PATCH v4]" and so used the v3 and
corrected the s/list/__list/ by hand).
But it's not very important.

Thanks,
-- Luc

Re: Upcoming sparse release RC5

[Christopher Li]

On Mon, Aug 7, 2017 at 8:23 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> If you don't mind, it would be nice if you could use the pull request
> I sent here: http://marc.info/?l=linux-sparse&m=150188507213269&w=4
> (you never saw it because I copy-pasted "[PATCH v3]" in the title of
> the cover page instead of using "[PATCH v4]" and so used the v3 and
> corrected the s/list/__list/ by hand).
> But it's not very important.

I did not find that git pull url due to the V3 title. I did look for it.

When I look at the patch, there is some compile error on one of the
patch (patch 4?). I just fix it by hand. I assume the git pull has the
same error because the patch are generated from git.

I just checked, that compile error still on the git url you send out
in the email. I still can't use it

I actually prefer to use git pull if it is available to use.

Also, can you please give the pull request a dedicate branch,
like "for-chris",  not a tag? I don't want to pull from random branch
and end up merging between your topic branches.

I often need to seek clarify if this patches series is mean for merge.
If you make that in the dedicate pull branch, I will know for sure.
I can see that branch get updated from git fetch before I saw the
email.

Chris

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Tue, Aug 8, 2017 at 2:42 AM, Christopher Li <sparse@chrisli.org> wrote:
> On Mon, Aug 7, 2017 at 8:23 PM, Luc Van Oostenryck
> <luc.vanoostenryck@gmail.com> wrote:
>> If you don't mind, it would be nice if you could use the pull request
>> I sent here: http://marc.info/?l=linux-sparse&m=150188507213269&w=4
>> (you never saw it because I copy-pasted "[PATCH v3]" in the title of
>> the cover page instead of using "[PATCH v4]" and so used the v3 and
>> corrected the s/list/__list/ by hand).
>> But it's not very important.
>
> I did not find that git pull url due to the V3 title. I did look for it.
>
> When I look at the patch, there is some compile error on one of the
> patch (patch 4?). I just fix it by hand. I assume the git pull has the
> same error because the patch are generated from git.
>
> I just checked, that compile error still on the git url you send out
> in the email. I still can't use it

Are you sure?
in the link I pasted here above, the git branch is:

    git://github.com/lucvoo/sparse.git fix-nested-pseudo-users-deletion-v4

    aka: bddf7d1e4ed81c3def7c6de2796a5489ca678399


> I actually prefer to use git pull if it is available to use.

Good.

> Also, can you please give the pull request a dedicate branch,
> like "for-chris",  not a tag? I don't want to pull from random branch
> and end up merging between your topic branches.

It won't change anything, in both case it's just a name for something
(and using specific name for specific thing is a good thing I think)
but no problem, it's fine for me.

I'll sent a proper pull request.

> I often need to seek clarify if this patches series is mean for merge.
> If you make that in the dedicate pull branch, I will know for sure.

Make sense.

-- Luc

Re: Upcoming sparse release RC5

[Christopher Li]

On Mon, Aug 7, 2017 at 8:58 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Tue, Aug 8, 2017 at 2:42 AM, Christopher Li <sparse@chrisli.org> wrote:
>> On Mon, Aug 7, 2017 at 8:23 PM, Luc Van Oostenryck
>> <luc.vanoostenryck@gmail.com> wrote:
>>> If you don't mind, it would be nice if you could use the pull request
>>> I sent here: http://marc.info/?l=linux-sparse&m=150188507213269&w=4
>>> (you never saw it because I copy-pasted "[PATCH v3]" in the title of
>>> the cover page instead of using "[PATCH v4]" and so used the v3 and
>>> corrected the s/list/__list/ by hand).
>>> But it's not very important.
>>
>> I did not find that git pull url due to the V3 title. I did look for it.
>>
>> When I look at the patch, there is some compile error on one of the
>> patch (patch 4?). I just fix it by hand. I assume the git pull has the
>> same error because the patch are generated from git.
>>
>> I just checked, that compile error still on the git url you send out
>> in the email. I still can't use it
>
> Are you sure?
> in the link I pasted here above, the git branch is:
>
>     git://github.com/lucvoo/sparse.git fix-nested-pseudo-users-deletion-v4
>
>     aka: bddf7d1e4ed81c3def7c6de2796a5489ca678399


Yes, see the git log -p bddf7d1e4ed81c3def7c6de2796a5489ca678399


commit bddf7d1e4ed81c3def7c6de2796a5489ca678399
(luc/fix-nested-pseudo-users-deletion-v4)
Author: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Date:   Fri Aug 4 21:50:20 2017 +0200

    mark pseudo users as deleted instead of removing them

    This will fix the (rare) problems with deletion while doing
    nested ptrlist walking that occurs when doing recursive
    kill_instruction() - remove_usage() - kill_instruction()

    Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>

diff --git a/ptrlist.h b/ptrlist.h
index 1839b0f..78625c8 100644
--- a/ptrlist.h
+++ b/ptrlist.h
@@ -162,7 +162,7 @@ static inline void *last_ptr_list(struct ptr_list *list)
                        for (__nr = 0; __nr < __list->nr; __nr++) {
                 \
                                do {
                 \
                                        ptr = PTR_ENTRY(__list,__nr);
                 \
-                                       if (list->rm && !ptr)
                 \
+                                       if (__list->rm && !ptr)
                 \
                                                continue;
                 \
                                        do {

@@ -185,7 +185,7 @@ static inline void *last_ptr_list(struct ptr_list *list)
                        while (--__nr >= 0) {
                 \
                                do {
                 \
                                        ptr = PTR_ENTRY(__list,__nr);
                 \
-                                       if (list->rm && !ptr)
                 \
+                                       if (__list->rm && !ptr)
                 \
                                                continue;
                 \
                                        do {

That change is your later patch fixing the compile error in the previous
patch. I have to move this change into the previous patch.


>> Also, can you please give the pull request a dedicate branch,
>> like "for-chris",  not a tag? I don't want to pull from random branch
>> and end up merging between your topic branches.
>
> It won't change anything, in both case it's just a name for something
> (and using specific name for specific thing is a good thing I think)
> but no problem, it's fine for me.

You can still have your specific topic branch. I want a unified view
to pull from you.

It does change some thing. I find out that your like to just assign
the tag to the topic branch you want me to pull from. The last
"for-chris" tag might not be fast forwardable to the next "for-chris"
tag.

Think that "for-chris" as if your own luc-next. You can still maintain
a lot of topic branches. You aggregate the stable enough bits as luc-next
for others to try out. That is where I pull from, and I expect that
to be fast forwardable from master. The difference will be, you will
merge between your own branches before let me pull.

I don't need to keep track of what is the updated version of
different topic branches I have merged from you.

Currently, rebase the sparse-next with your topic branch pulled
is very tricky. I need to create a stable point for your aggregated
topic branches. That is equivalent of the "for-chris" branch on my
side. Then I rebase sparse-next on that.

If you maintain that "for-chris" to aggregate your own topic branch.
I can just use "for-chris" as a base to rease the sparse-next,
I only need to rebase patches not from you.

I do want to have the flexibility to rebase spase-next.
It is just how to do it in a way impact you less.

>> I often need to seek clarify if this patches series is mean for merge.
>> If you make that in the dedicate pull branch, I will know for sure.

Let me clarify that does not stop you from send out review with
specific topic branch.

I want a merged unified view from you to use as my base.

Chris

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Tue, Aug 8, 2017 at 3:34 AM, Christopher Li <sparse@chrisli.org> wrote:
>>> I just checked, that compile error still on the git url you send out
>>> in the email. I still can't use it
>>
>> Are you sure?
>> in the link I pasted here above, the git branch is:
>>
>>     git://github.com/lucvoo/sparse.git fix-nested-pseudo-users-deletion-v4
>>
>>     aka: bddf7d1e4ed81c3def7c6de2796a5489ca678399
>
> Yes, see the git log -p bddf7d1e4ed81c3def7c6de2796a5489ca678399

Yes, I see now.
It compiles but not the previous commit.

please ignore the pull request I just sent.

--Luc

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Tue, Aug 8, 2017 at 3:34 AM, Christopher Li <sparse@chrisli.org> wrote:

> If you maintain that "for-chris" to aggregate your own topic branch.
> I can just use "for-chris" as a base to rease the sparse-next,
> I only need to rebase patches not from you.

Yes, exactly.

> I do want to have the flexibility to rebase spase-next.
> It is just how to do it in a way impact you less.

Just don't rebase my series. I maintain a few aggregation
branches with all the topics branches I need and often
do new dev on top of them because I need to.
When I don't need it I base new dev on some stable point
like master.

If it is a small thing I did the day before, it won't probably be a big
problem but for older stuff it is.

>>> I often need to seek clarify if this patches series is mean for merge.
>>> If you make that in the dedicate pull branch, I will know for sure.

Let's do as we had agreed: real pull request will have "[GIT PULL]"
in the email and will always be from the 'for-chris'. I also guarantee
that the pull will not need a non-trivial merge when done from
the current sparse-next.

-- Luc

Re: Upcoming sparse release RC5

[Christopher Li]

On Mon, Aug 7, 2017 at 10:13 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:

>> I do want to have the flexibility to rebase spase-next.
>> It is just how to do it in a way impact you less.
>
> Just don't rebase my series. I maintain a few aggregation
> branches with all the topics branches I need and often
> do new dev on top of them because I need to.
> When I don't need it I base new dev on some stable point
> like master.

Do as I suggested, I always use "for-chris" as base.
You always put all the bits you want me to pull in "for-chris".
I will not rebase your series at all. Why? because your
"for-chris" are the base for me.

You need to make "for-chris" fast forwardable from master.

> If it is a small thing I did the day before, it won't probably be a big
> problem but for older stuff it is.

The beauty in my model is that. It does not matter how long does
sparse-next merge back to master. The you provide your own
stable point which I rebase on top of.

> Let's do as we had agreed: real pull request will have "[GIT PULL]"
> in the email and will always be from the 'for-chris'. I also guarantee
> that the pull will not need a non-trivial merge when done from
> the current sparse-next.

That will make your harder to make some V4 -> V5 change.

Your "for-chris" contain all your patches you want me to pull. Then
sparse-next only rebase on your "for-chris". Let me due with the
merges (not from your patch).

Chris

Re: Upcoming sparse release RC5

[Linus Torvalds]

On Mon, Aug 7, 2017 at 7:38 PM, Christopher Li <sparse@chrisli.org> wrote:
>
> You need to make "for-chris" fast forwardable from master.

Chris, that makes no sense at all.

If you ever apply any other patches than the patches from Luc,
suddenly his "for-chris" is no longer fast-forwardeable.

Why do you want that stupid linear history in the first place? Why
can't you just merge Luc's git tree? That's how git is supposed to
work, and it makes it easier for everybody.

This whole rebasing is a disease. It is basically *impossible* to work
sanely with other people if you keep rebasing.

You mustn't rebase other peoples work (because that causes tons of
extra work for them) when you pull, and if you apply any other patches
(or commit any of your own work), then Luc's branch by definition will
need merging.

Nobody else works like you do. Yes, other people have unstable
branches (eg git's own "pu" branch), but those are for experimental
stuff, not for working with anybody else.

Rebasing really is wrong for any pubic tree. Its' fine for your own
_private_ pacthes, as you develop and clean up a branch that you're
still working on, but it shouldn't be an integration model.

                   Linus

Re: Upcoming sparse release RC5

[Christopher Li]

On Mon, Aug 7, 2017 at 11:31 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Aug 7, 2017 at 7:38 PM, Christopher Li <sparse@chrisli.org> wrote:
>>
>> You need to make "for-chris" fast forwardable from master.
>
> Chris, that makes no sense at all.
>
> If you ever apply any other patches than the patches from Luc,
> suddenly his "for-chris" is no longer fast-forwardeable.

You mean "for-chris" is not fast-forwardable to sparse-next.
master will stable able to fast forward to "for-chris".

That is fine. Because in my model, this "for-chris" is the stable
point. When he send out pull request, "for-chris" contain some
new bits which is not on sparse-next, sparse-next can not
fastforward to, sparse-next will *rebase* on "for-chris".

In other words, sparse-next will adept to solve that not
fast-forwardable problem.

In this case, sparse-next is the floating point. The stable
point is "for-chris". Any thing merge into "for-chris", I am not able to
rebase any more.

> Why do you want that stupid linear history in the first place? Why

Basically I want to have some place to test temporary patches.
Like the revert for the wine bug and later we drop the revert
when we have better patches.

> can't you just merge Luc's git tree? That's how git is supposed to
> work, and it makes it easier for everybody.

That is effective make Luc's git tree the stable branch. My sparse-next
is floating base on that. If I directly merge Luc's git tree from spase-next,
those unstable change will not be able to drop from the sparse-next history.

> This whole rebasing is a disease. It is basically *impossible* to work
> sanely with other people if you keep rebasing.
>
> You mustn't rebase other peoples work (because that causes tons of
> extra work for them) when you pull, and if you apply any other patches
> (or commit any of your own work), then Luc's branch by definition will
> need merging.

That is exactly what I am trying to avoid.  Not rebase Luc's change.
My purpose is that always use Luc's "for-chris" branch as base.
The spase-next only contain the unstable part of the experiment change.

> Nobody else works like you do. Yes, other people have unstable
> branches (eg git's own "pu" branch), but those are for experimental
> stuff, not for working with anybody else.

I think I have make it clear that sparse-next is the unstable experiment
branch. Before I use sparse-next, I also ask on the sparse mailing list if
I can have have a branch for experiment that I rebase. The feed back
I got back then was fine as long as I make it clear it is a branch will rebase.

> Rebasing really is wrong for any pubic tree. Its' fine for your own
> _private_ pacthes, as you develop and clean up a branch that you're
> still working on, but it shouldn't be an integration model.

Will it be better I keep using sparse-next-xxxxx as the experiment
branch and use "sparse-next" as a tag point to sparse-next?

If we don't rebase, then we will keep the revert and revert of the revert
in the history. I can change to that too. In that case, we might as well
just do it on the master branch, because we never rebase on it any way.

What is the recommended way to use experiment branch in git any
way?

Chris

Re: Upcoming sparse release RC5

[Christopher Li]

On Mon, Aug 7, 2017 at 11:31 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> Why do you want that stupid linear history in the first place? Why
> can't you just merge Luc's git tree? That's how git is supposed to
> work, and it makes it easier for everybody.
>
> This whole rebasing is a disease. It is basically *impossible* to work
> sanely with other people if you keep rebasing.

I carefully think over it. You are right. Try to clean up history actually
make the whole integration a lot worse. I will take your advice and just
do the simple git merge on master from now on.

>
> You mustn't rebase other peoples work (because that causes tons of
> extra work for them) when you pull, and if you apply any other patches
> (or commit any of your own work), then Luc's branch by definition will
> need merging.

I will remember no to rebase other peoples work..

Thanks.

Chris

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Tue, Aug 8, 2017 at 4:38 AM, Christopher Li <sparse@chrisli.org> wrote:
>
> You need to make "for-chris" fast forwardable from master.
>

It's possible to work like this too if you wish so.
It means that this 'for-chris' becomes the integration tree and
all patches need to go through this tree first.

It would be fine for me.

-- Luc

Re: Upcoming sparse release RC5

[Christopher Li]

On Tue, Aug 8, 2017 at 8:02 AM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Tue, Aug 8, 2017 at 4:38 AM, Christopher Li <sparse@chrisli.org> wrote:
>>
>> You need to make "for-chris" fast forwardable from master.
>>
>
> It's possible to work like this too if you wish so.
> It means that this 'for-chris' becomes the integration tree and
> all patches need to go through this tree first.

More precisely, the "for-chris" will be come the proposal of the  integration
tree. If I am happy with "for-chris" and Luc don't intend to rebase "for-chris"
any more. The master can just sync up to the "for-chris". Everybody is happy.

If I have some feed back for "for-chris" V3 and Luc decide to incorporate the
feed back and come back with "for-chris" V4. Notice the "for-chris" V4 does
not need to be fast forwardable from V3. If Luc wants the V4 contain a "clean"
history, he is free to do so. Of course he want to have incremental change
from V3 -> V4, V3 will show up in master history. That is of fine too.
It is Luc's
call.

That effective make Luc control what become the master branch. It does mean
that patch in sparse-next will go through Luc's "for-chris" to become part of
master. The net effect is that, if Luc and I want to clean up the experimental
history, we can do so. Those V1...V3 internal version don't have to show up
on master.

From my point of view, I think that is the model Luc and I already agree on
about two month back, when we have the discussion how to do git pull properly
for sparse using pull request.  I kind of see Luc doing that for
sparse up to RC3,
He is already integrating the static assert patches etc. Luc's is not using a
"for-chris" branch though, I did not insist either.

I did not realized that there is an understand gap how this git pull request
will work on Luc's side. I did not see the proper git pull request I
assume those
patch on mailing list are for discussion/experimental only. I have send email
to clarify those are for git pull or not.

I hope that clear up the git pull situations on sparse-next. If it helps, I can
also rename spase-next to "unstable". I am also open to suggestion for how
this integration should be done. The original intend for having this complex
model is to get rid of the dirty history. If that turn out to be the
bigger evil.
We can do the other way around as well.

Chris

Re: Upcoming sparse release RC5

[Dibyendu Majumdar]

Hi Chris,

On 8 August 2017 at 14:44, Christopher Li <sparse@chrisli.org> wrote:

> I hope that clear up the git pull situations on sparse-next. If it helps, I can
> also rename spase-next to "unstable". I am also open to suggestion for how
> this integration should be done. The original intend for having this complex
> model is to get rid of the dirty history. If that turn out to be the
> bigger evil.

I am not sure why you consider some history "dirty". I think all
history is good as it tells people how the change evolved, If you
apply a patch and a bug is found, or you want to amend the patch
somehow, in my view there should simply be another patch or change
done following the original rather than trying to backout history and
redo the patch. I am not a git expert so I cannot comment on the git
processes but conceptually I am saying that all history is good.

Regards
Dibyendu

Re: Upcoming sparse release RC5

[Christopher Li]

On Tue, Aug 8, 2017 at 12:10 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
> I am not sure why you consider some history "dirty". I think all
> history is good as it tells people how the change evolved, If you
> apply a patch and a bug is found, or you want to amend the patch

For example, there is some internal version of patches has some compile error.
That definitely don't want to be part of the history. It will hurt the
binary search.
Some patch in the early version might be totally wrong, that does not
need to go into history. e.g. the revert of the change and revert of the revert
later on.

I consider a patch will take some time and experiment to mature.
We don't have to submit every generation of the patch into history.

> somehow, in my view there should simply be another patch or change
> done following the original rather than trying to backout history and
> redo the patch. I am not a git expert so I cannot comment on the git
> processes but conceptually I am saying that all history is good.

If follow the process I suggested, that ultimately give Luc the flexibility to
use the linear incremental history or the clean up version of the history.
He has the most pending changes.

I prefer the clean up histories, but I can do the all detail dirty history
as well if that is we all agree to do.

Chris

Re: Upcoming sparse release RC5

[Dibyendu Majumdar]

Hi Chris,

On 8 August 2017 at 20:03, Christopher Li <sparse@chrisli.org> wrote:
> On Tue, Aug 8, 2017 at 12:10 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>> I am not sure why you consider some history "dirty". I think all
>> history is good as it tells people how the change evolved, If you
>> apply a patch and a bug is found, or you want to amend the patch
>
> For example, there is some internal version of patches has some compile error.
> That definitely don't want to be part of the history. It will hurt the
> binary search.
> Some patch in the early version might be totally wrong, that does not
> need to go into history. e.g. the revert of the change and revert of the revert
> later on.
>
> I consider a patch will take some time and experiment to mature.
> We don't have to submit every generation of the patch into history.
>

I think that compile error should not be possible? Because before you
merge any patch presumably you build Sparse and run the tests?

As for other changes that were later removed - in my view this is
valuable history as it tells people what was tried and discarded and
why. I still feel all history is good.

Regards
Dibyendu

Re: Upcoming sparse release RC5

[Christopher Li]

On Wed, Aug 9, 2017 at 5:10 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> I think that compile error should not be possible? Because before you
> merge any patch presumably you build Sparse and run the tests?
>
> As for other changes that were later removed - in my view this is
> valuable history as it tells people what was tried and discarded and
> why. I still feel all history is good.
>

OK. I think my obsession of "clean up the history" is wrong and not  worthy
doing. Linus's criticism of sparse-next rebase is correct.  I will just got back
to the master doing the normal git pull and merges. That actually simplify
my own work flow as we.

Chris

Re: Upcoming sparse release RC5

[Dibyendu Majumdar]

Hi Chris,

On 10 August 2017 at 01:18, Christopher Li <sparse@chrisli.org> wrote:
> On Wed, Aug 9, 2017 at 5:10 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>
>> I think that compile error should not be possible? Because before you
>> merge any patch presumably you build Sparse and run the tests?
>>
>> As for other changes that were later removed - in my view this is
>> valuable history as it tells people what was tried and discarded and
>> why. I still feel all history is good.
>>
>
> OK. I think my obsession of "clean up the history" is wrong and not  worthy
> doing. Linus's criticism of sparse-next rebase is correct.  I will just got back
> to the master doing the normal git pull and merges. That actually simplify
> my own work flow as we.
>

It will also help me if I can assume that a patch will not be re-done.
At the moment I cannot apply any patch - because I do not know if a
new version of the patch will be published soon. Instead if changes
always go forward - i.e. you apply fixes to existing patches - then I
can safely apply a patch, and when a fix is published, apply the fix.

I learnt from database technology that it is better for changes to
always go forward - trying to revert past changes by undoing them is
troublesome as it means you may need to undo all changes that follow
the one you want to undo. Progress can become very difficult in this
approach. That is why in databases, often undo is just another forward
change - there is no going back to remove a past change. A really good
description of this approach is given in the ARIES approach to
database recovery.

Regards
Dibyendu

Re: Upcoming sparse release RC5

[Christopher Li]

On Wed, Aug 9, 2017 at 8:42 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> It will also help me if I can assume that a patch will not be re-done.
> At the moment I cannot apply any patch - because I do not know if a
> new version of the patch will be published soon. Instead if changes
> always go forward - i.e. you apply fixes to existing patches - then I
> can safely apply a patch, and when a fix is published, apply the fix.

Well, I can promise any thing apply to master, which will be the new
integration happening on, will not be rebase or revert.

If there is improvement of already applied patches. It will be a
delta change apply to master.  Master will not roll back, ever.
It actually haven't. However, if the patch is not yet apply to master, the
developer can come up with new versions. That I will not able to stop,
there is no reason to.
>
> I learnt from database technology that it is better for changes to
> always go forward - trying to revert past changes by undoing them is
> troublesome as it means you may need to undo all changes that follow

Yes, master will be always forwarding. It will also solve another problem,
patches takes too long from sparse-next to reach master. So master will
be where integration happens. There is no safe period like sparse-next
any more. The topic branches will still control by their original developer,
I don't have direct control unless that is my topic branch.

> the one you want to undo. Progress can become very difficult in this
> approach. That is why in databases, often undo is just another forward
> change - there is no going back to remove a past change. A really good
> description of this approach is given in the ARIES approach to
> database recovery.

I hope that answer your questions.

Chris

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Tue, Aug 8, 2017 at 3:44 PM, Christopher Li <sparse@chrisli.org> wrote:
> On Tue, Aug 8, 2017 at 8:02 AM, Luc Van Oostenryck <luc.vanoostenryck@gmail.com> wrote:
>> On Tue, Aug 8, 2017 at 4:38 AM, Christopher Li <sparse@chrisli.org> wrote:
>>>
>>> You need to make "for-chris" fast forwardable from master.
>>>
>>
>> It's possible to work like this too if you wish so.
>> It means that this 'for-chris' becomes the integration tree and
>> all patches need to go through this tree first.
>
> That effective make Luc control what become the master branch. It does mean
> that patch in sparse-next will go through Luc's "for-chris" to become part of
> master.

I just find that in this case 'for-chris' doesn't convey the right intention.
So for now, I'll use 'tip', it's a branch, will never be rebased and will be
fast-forwardable from master (and its name won't change anymore).

> I hope that clear up the git pull situations on sparse-next. If it helps, I can
> also rename spase-next to "unstable".

I think that those sparse-next-XXXX branches have no place on the master
repository, they never had. Even as tags they shouldn't be there.

Review and testing can and should be done on topic branches in separate
repositories. We can have a sparse/sparse-dev for that if needed. Once
acked & tested, as agreed here above, I'll do the integration.
Since integration will be done through my tree there won't be a need
for such branches on the main repo.

I don't see the need for an unstable branch on the main repository,
on the contrary, it's a nuisance for everybody.

Regards,
-- Luc

Re: Upcoming sparse release RC5

[Christopher Li]

On Sat, Aug 5, 2017 at 6:32 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
>
> sparse-next is currently not fast-forwardable from master.
>

BTW, do you have any thing to add for RC5? If not I am going
tag the RC5 and hope that is the last RC for this release.

I still need to add two document for sparse. I assume I can
add it after RC5 into the release because there is no any
code changes. Maybe I should check in two place holder
file now so it does not mess up with the packaging for the
finial release.

Chris

Re: Upcoming sparse release RC5

[Luc Van Oostenryck]

On Mon, Aug 7, 2017 at 4:06 AM, Christopher Li <sparse@chrisli.org> wrote:
> On Sat, Aug 5, 2017 at 6:32 PM, Luc Van Oostenryck wrote:
>
> BTW, do you have any thing to add for RC5?

Not at all.

-- Luc