Google Summer of Code 2009: GIT

Google Summer of Code 2009: GIT

[Saurabh Gupta]

hello everyone,

/*First, a brief introduction about me: */
I am Saurabh Gupta from India. I am a senior engineering student. I
have been working for open source projects for a long time and I was a
*GSoC student in 2008* where I successfully completed my project under
the organization Openmoko (www.openmoko.org
<http://www.openmoko.org/>). In 2008 winters, I had my internship at
*Adobe Inc.* where I worked on the package  management system using
C++ and sqlite library. I have a long experience in programming in C,
C++,  Python and other languages. Currently, my final year project at
university is about the automatic data logging system in which I am
working upon socket programming in C, C++,   MYSQL and setting up an
intra-college website using LAMP. I am a user of svn extensively and
use git sometimes.


/*About GSoC GIT ideas;
*/Here are the ideas which I found to be interested in. Although, I
would like to discuss any other idea than these in GIT organization.

*1) Domain specific merge helpers*
Intelligence in the merger can be put which modifies the source file
according the format. Different file formats can be put in the merger
to support.


I am looking forward to get feedback and suggestions for this and to
discuss the ideas. I would like to contribute GIT this summer if I get
a chance through GSoC. If not, then still, I would remain in contact
with the community to contribute in the best possible manner I can.

Re: Google Summer of Code 2009: GIT

[Daniel Barkalow]

On Wed, 11 Mar 2009, Saurabh Gupta wrote:

> hello everyone,
> 
> /*First, a brief introduction about me: */
> I am Saurabh Gupta from India. I am a senior engineering student. I
> have been working for open source projects for a long time and I was a
> *GSoC student in 2008* where I successfully completed my project under
> the organization Openmoko (www.openmoko.org
> <http://www.openmoko.org/>). In 2008 winters, I had my internship at
> *Adobe Inc.* where I worked on the package  management system using
> C++ and sqlite library. I have a long experience in programming in C,
> C++,  Python and other languages. Currently, my final year project at
> university is about the automatic data logging system in which I am
> working upon socket programming in C, C++,   MYSQL and setting up an
> intra-college website using LAMP. I am a user of svn extensively and
> use git sometimes.
> 
> 
> /*About GSoC GIT ideas;
> */Here are the ideas which I found to be interested in. Although, I
> would like to discuss any other idea than these in GIT organization.
> 
> *1) Domain specific merge helpers*
> Intelligence in the merger can be put which modifies the source file
> according the format. Different file formats can be put in the merger
> to support.

This is something I've thought would be really handy for making git more 
widely useful. The hard part, of course, is coming up with useful 
alternative merge algorithms which work for formats that the usual merge 
doesn't handle. The infrastructure is mostly there (git supports declaring 
the types of files), but there's no point in support for running a 
type-specific merger until there are actually type-specific mergers to 
run.

Do you have ideas on what formats you'd try to merge, and how?

	-Daniel
*This .sig left intentionally blank*

Re: Google Summer of Code 2009: GIT

[David Symonds]

On Wed, Mar 11, 2009 at 3:52 PM, Saurabh Gupta
<saurabhgupta1403@gmail.com> wrote:

> *1) Domain specific merge helpers*
> Intelligence in the merger can be put which modifies the source file
> according the format. Different file formats can be put in the merger
> to support.

I don't think that's Git specific, though it still might be reasonable
to be done as a Git GSoC project. I'm sure other VCSs would find that
useful, and even folk who aren't using VCSs might benefit from it.


Dave.

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 2:29 PM, David Symonds <dsymonds@gmail.com> wrote:
>
> On Wed, Mar 11, 2009 at 3:52 PM, Saurabh Gupta
> <saurabhgupta1403@gmail.com> wrote:
>
> > *1) Domain specific merge helpers*
> > Intelligence in the merger can be put which modifies the source file
> > according the format. Different file formats can be put in the merger
> > to support.
>
> I don't think that's Git specific, though it still might be reasonable
> to be done as a Git GSoC project. I'm sure other VCSs would find that
> useful, and even folk who aren't using VCSs might benefit from it.

All right.
Can you suggest me another idea or any aim which GIT will mostly be
interested in. I am also looking over other ideas and areas where more
improvement can be put.

Thanks...


>
> Dave.



--
Saurabh Gupta
Senior,
Electronics and Communication Engg.
NSIT,New Delhi, India

Google Summer of Code 2009: GIT

[saurabh gupta]

hello,

On Wed, Mar 11, 2009 at 11:26 AM, Daniel Barkalow <barkalow@iabervon.org> wrote:
>
> On Wed, 11 Mar 2009, Saurabh Gupta wrote:
>
> >
> > /*About GSoC GIT ideas;
> > */Here are the ideas which I found to be interested in. Although, I
> > would like to discuss any other idea than these in GIT organization.
> >
> > *1) Domain specific merge helpers*
> > Intelligence in the merger can be put which modifies the source file
> > according the format. Different file formats can be put in the merger
> > to support.
>
> This is something I've thought would be really handy for making git more
> widely useful. The hard part, of course, is coming up with useful
> alternative merge algorithms which work for formats that the usual merge
> doesn't handle. The infrastructure is mostly there (git supports declaring
> the types of files), but there's no point in support for running a
> type-specific merger until there are actually type-specific mergers to
> run.
>
> Do you have ideas on what formats you'd try to merge, and how?

Well, What I think is to implement file formats other than text like
that written on wiki i.e. latex, xml, or even any database file (db
file).  Another idea (although it can be weired also) is to implement
the new file formats in the plug-in formats. For example, to
incorporate the merger engine for a new file format, a plug-in is
created and can be integrated with the present merger in the git.
However, I am not sure how much valid is this idea to make the present
merger in git to be compatible with the plug-ins for enabling newer
file formats.

Any new idea or suggestions are welcome.



-- 
Saurabh Gupta
Senior,
Electronics and Communication Engg.
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[David Symonds]

On Wed, Mar 11, 2009 at 10:55 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:

> It has been suggested on the GSoC mentor list, and IMHO Git is as good as
> any organization to host that project.

Yeah, I think Git would be a good host for such a project. I'd still
like to see it VCS-neutral, though.


Dave.

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, David Symonds wrote:

> On Wed, Mar 11, 2009 at 3:52 PM, Saurabh Gupta
> <saurabhgupta1403@gmail.com> wrote:
> 
> > *1) Domain specific merge helpers* Intelligence in the merger can be 
> > put which modifies the source file according the format. Different 
> > file formats can be put in the merger to support.
> 
> I don't think that's Git specific, though it still might be reasonable 
> to be done as a Git GSoC project. I'm sure other VCSs would find that 
> useful, and even folk who aren't using VCSs might benefit from it.

It has been suggested on the GSoC mentor list, and IMHO Git is as good as 
any organization to host that project.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

Welcome, Saurabh!

On Wed, 11 Mar 2009, Saurabh Gupta wrote:

> /*About GSoC GIT ideas; */Here are the ideas which I found to be 
> interested in. Although, I would like to discuss any other idea than 
> these in GIT organization.
> 
> *1) Domain specific merge helpers* Intelligence in the merger can be put 
> which modifies the source file according the format. Different file 
> formats can be put in the merger to support.

You said that you are interested in this project, but from your mails I do 
not see what are the specific reasons why.

IMHO this project can only fly if you have a specific file format that you 
absolutely want to be able to merge; otherwise, it will be an uphill 
fight.

Personally, I would _love_ to see a good graphical tool (maybe written 
in Tcl/Tk) to help merging conflicts in LaTeX files, but I just do not 
have the time...

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 5:28 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi,
>
> Welcome, Saurabh!
>
> On Wed, 11 Mar 2009, Saurabh Gupta wrote:
>
> > /*About GSoC GIT ideas; */Here are the ideas which I found to be
> > interested in. Although, I would like to discuss any other idea than
> > these in GIT organization.
> >
> > *1) Domain specific merge helpers* Intelligence in the merger can be put
> > which modifies the source file according the format. Different file
> > formats can be put in the merger to support.
>
> You said that you are interested in this project, but from your mails I do
> not see what are the specific reasons why.
>

All right. May be I lacked in my mail to specify the reason for my
interest. The reason is that from my past experience, I got the notion
that this project is according to my interest and is doable in the
three months time period.
Another reason is that I have been using the versioning tools like svn
and now perforce for a long time and this added up to my interest.


> IMHO this project can only fly if you have a specific file format that you
> absolutely want to be able to merge; otherwise, it will be an uphill
> fight.
>

Well, as suggested on the wiki, I would like to work on the xml file
formats as I have quite experience of working with xml files and
parsing them using msxml and nsxml libraries and some of personal
wrappers.

How about my idea of making the support of new file formats in the
plug-ins (suggested in my last post). I would like to discuss more on
this and any new suggestions or methods are welcome.

> Personally, I would _love_ to see a good graphical tool (maybe written
> in Tcl/Tk) to help merging conflicts in LaTeX files, but I just do not
> have the time...

Ok. What I am thinking is to implement something  like that of
graphical *diff* command output but in these special file formats, it
ought to have intelligence to bring out the difference of two files
(like latex or xml) in a readable manner. For example, in case of xml
files, if one file contains an inner tag block , then merger GUI
should notify the user in a readable manner about this added tag
rather than only the difference in lines.

>
> Ciao,
> Dscho
>

Regards...

--
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, David Symonds wrote:

> On Wed, Mar 11, 2009 at 10:55 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> 
> > It has been suggested on the GSoC mentor list, and IMHO Git is as good 
> > as any organization to host that project.
> 
> Yeah, I think Git would be a good host for such a project. I'd still
> like to see it VCS-neutral, though.

Yeah, I cannot see this begin Git-specific, either.  But it may be 
integrated with Git so that it works best using Git...

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, saurabh gupta wrote:

> On Wed, Mar 11, 2009 at 5:28 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>
> > On Wed, 11 Mar 2009, Saurabh Gupta wrote:
> >
> > > /*About GSoC GIT ideas; */Here are the ideas which I found to be 
> > > interested in. Although, I would like to discuss any other idea than 
> > > these in GIT organization.
> > >
> > > *1) Domain specific merge helpers* Intelligence in the merger can be 
> > > put which modifies the source file according the format. Different 
> > > file formats can be put in the merger to support.
> >
> > You said that you are interested in this project, but from your mails 
> > I do not see what are the specific reasons why.
> 
> All right. May be I lacked in my mail to specify the reason for my 
> interest.

Oh, sorry, I did not mean to imply any offense...

> The reason is that from my past experience, I got the notion that this 
> project is according to my interest and is doable in the three months 
> time period.
>
> Another reason is that I have been using the versioning tools like svn 
> and now perforce for a long time and this added up to my interest.

Sounds good!

> > IMHO this project can only fly if you have a specific file format that 
> > you absolutely want to be able to merge; otherwise, it will be an 
> > uphill fight.
> 
> Well, as suggested on the wiki, I would like to work on the xml file 
> formats as I have quite experience of working with xml files and parsing 
> them using msxml and nsxml libraries and some of personal wrappers.

As I am known to not exactly like Microsoft's products, if you wanted to 
have me as a mentor, you'd need to use Open Source libraries to do the 
parsing.

> How about my idea of making the support of new file formats in the 
> plug-ins (suggested in my last post).

Sorry, I missed that idea.  Could you describe it again?

> > Personally, I would _love_ to see a good graphical tool (maybe written 
> > in Tcl/Tk) to help merging conflicts in LaTeX files, but I just do not 
> > have the time...
> 
> Ok. What I am thinking is to implement something  like that of
> graphical *diff* command output but in these special file formats, it
> ought to have intelligence to bring out the difference of two files
> (like latex or xml) in a readable manner. For example, in case of xml
> files, if one file contains an inner tag block , then merger GUI
> should notify the user in a readable manner about this added tag
> rather than only the difference in lines.

A diff would be a first step, but the real issue are the merge helpers.  
And they need first and foremost a thought-through user interface design.  
The technical issues are all solveable, I am sure.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 6:28 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Wed, 11 Mar 2009, saurabh gupta wrote:
>
>> On Wed, Mar 11, 2009 at 5:28 PM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>
>> > On Wed, 11 Mar 2009, Saurabh Gupta wrote:
>> >
>> > > /*About GSoC GIT ideas; */Here are the ideas which I found to be
>> > > interested in. Although, I would like to discuss any other idea than
>> > > these in GIT organization.
>> > >
>> > > *1) Domain specific merge helpers* Intelligence in the merger can be
>> > > put which modifies the source file according the format. Different
>> > > file formats can be put in the merger to support.
>> >
>> > You said that you are interested in this project, but from your mails
>> > I do not see what are the specific reasons why.
>>
>> All right. May be I lacked in my mail to specify the reason for my
>> interest.
>
> Oh, sorry, I did not mean to imply any offense...

No, I didn;t mean that either too :-D

>
>> The reason is that from my past experience, I got the notion that this
>> project is according to my interest and is doable in the three months
>> time period.
>>
>> Another reason is that I have been using the versioning tools like svn
>> and now perforce for a long time and this added up to my interest.
>
> Sounds good!
>
>> > IMHO this project can only fly if you have a specific file format that
>> > you absolutely want to be able to merge; otherwise, it will be an
>> > uphill fight.
>>
>> Well, as suggested on the wiki, I would like to work on the xml file
>> formats as I have quite experience of working with xml files and parsing
>> them using msxml and nsxml libraries and some of personal wrappers.
>
> As I am known to not exactly like Microsoft's products, if you wanted to
> have me as a mentor, you'd need to use Open Source libraries to do the
> parsing.

Yeah, of course. I mentioned msxml and nsxml just to indicate that I
am comfortable with the xml parsing. I will, no doubt, go for open
source libraries to parse xml and write or reuse some of my own
wrappers to parse xml.



>> How about my idea of making the support of new file formats in the
>> plug-ins (suggested in my last post).
>
> Sorry, I missed that idea.  Could you describe it again?
>

All right. This is the quote which I said in my last posts.

=>What I think is to implement file formats other than text like
that written on wiki i.e. latex, xml, or even any database file (db
file).  Another idea (although it can be weired also) is to implement
the new file formats in the plug-in formats. For example, to
incorporate the merger engine for a new file format, a plug-in is
created and can be integrated with the present merger in the git.
However, I am not sure how much valid is this idea to make the present
merger in git to be compatible with the plug-ins for enabling newer
file formats.


>> > Personally, I would _love_ to see a good graphical tool (maybe written
>> > in Tcl/Tk) to help merging conflicts in LaTeX files, but I just do not
>> > have the time...
>>
>> Ok. What I am thinking is to implement something  like that of
>> graphical *diff* command output but in these special file formats, it
>> ought to have intelligence to bring out the difference of two files
>> (like latex or xml) in a readable manner. For example, in case of xml
>> files, if one file contains an inner tag block , then merger GUI
>> should notify the user in a readable manner about this added tag
>> rather than only the difference in lines.
>
> A diff would be a first step, but the real issue are the merge helpers.
> And they need first and foremost a thought-through user interface design.
> The technical issues are all solveable, I am sure.

I got your point. I am thinking of using gtk+ libraries to implement
the GUI part (I am quite comfortable with gtk+). However, I think in
merging and notifying about the conflicts in the xml files, other
things can also be put forward. Like the GUI will show the number of
tags differing and what are the new tags added and even if any tag is
renamed with the content unchanged. If possible, how about showing a
tree like structure (just like DOM model) to compare (or diff) the two
xml files.


> Ciao,
> Dscho
>
>



-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, saurabh gupta wrote:

> What I think is to implement file formats other than text like that 
> written on wiki i.e. latex, xml, or even any database file (db file).  
> Another idea (although it can be weired also) is to implement the new 
> file formats in the plug-in formats. For example, to incorporate the 
> merger engine for a new file format, a plug-in is created and can be 
> integrated with the present merger in the git. However, I am not sure 
> how much valid is this idea to make the present merger in git to be 
> compatible with the plug-ins for enabling newer file formats.

I am not sure that a plugin structure is needed.  Take, for example, three 
different .xml based formats: OpenOffice documents, .svg files and Ant 
build.xml files.  They need very different user interfaces.

> I am thinking of using gtk+ libraries to implement the GUI part (I am 
> quite comfortable with gtk+).

I mentioned Tcl/Tk, because it is portable, but I'll also take gtk-based 
stuff ;-)

> However, I think in merging and notifying about the conflicts in the xml 
> files, other things can also be put forward. Like the GUI will show the 
> number of tags differing and what are the new tags added and even if any 
> tag is renamed with the content unchanged. If possible, how about 
> showing a tree like structure (just like DOM model) to compare (or diff) 
> the two xml files.

This is a little bit too low-level for my liking.  Taking the OpenOffice 
example again, the GUI should not expose XML at all...

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 7:32 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Wed, 11 Mar 2009, saurabh gupta wrote:
>
>> What I think is to implement file formats other than text like that
>> written on wiki i.e. latex, xml, or even any database file (db file).
>> Another idea (although it can be weired also) is to implement the new
>> file formats in the plug-in formats. For example, to incorporate the
>> merger engine for a new file format, a plug-in is created and can be
>> integrated with the present merger in the git. However, I am not sure
>> how much valid is this idea to make the present merger in git to be
>> compatible with the plug-ins for enabling newer file formats.
>
> I am not sure that a plugin structure is needed.  Take, for example, three
> different .xml based formats: OpenOffice documents, .svg files and Ant
> build.xml files.  They need very different user interfaces.

okay. In that case, if they have  a different user interfaces then
separate plug-in would be needed for each of these. May be this will
get more messy.

>
>> I am thinking of using gtk+ libraries to implement the GUI part (I am
>> quite comfortable with gtk+).
>
> I mentioned Tcl/Tk, because it is portable, but I'll also take gtk-based
> stuff ;-)

All right. We can think over this issue. Being honest, I have not
worked on Tcl/Tk yet but I have no problem in learning it and I am
sure that I will get my hands on Tcl/Tk within no time.

>> However, I think in merging and notifying about the conflicts in the xml
>> files, other things can also be put forward. Like the GUI will show the
>> number of tags differing and what are the new tags added and even if any
>> tag is renamed with the content unchanged. If possible, how about
>> showing a tree like structure (just like DOM model) to compare (or diff)
>> the two xml files.
>
> This is a little bit too low-level for my liking.  Taking the OpenOffice
> example again, the GUI should not expose XML at all...

hmmmm.....I think I get your point somewhat. Let me do some research
over the formats and the background formats in which tools like
OpenOffice store the data in xml files. May be for docbooks by
OpenOffice, the best thing would be to give the *diff* output in terms
of lines.
I would also appreciate to know what you think and would like to see
the output in such case.

>
> Ciao,
> Dscho
>
>



-- 
Saurabh Gupta
Senior,
Electronics and Communication Engg.
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Rogan Dawes]

saurabh gupta wrote:
>>> However, I think in merging and notifying about the conflicts in the xml
>>> files, other things can also be put forward. Like the GUI will show the
>>> number of tags differing and what are the new tags added and even if any
>>> tag is renamed with the content unchanged. If possible, how about
>>> showing a tree like structure (just like DOM model) to compare (or diff)
>>> the two xml files.
>>
>> This is a little bit too low-level for my liking.  Taking the OpenOffice
>> example again, the GUI should not expose XML at all...
> 
> hmmmm.....I think I get your point somewhat. Let me do some research
> over the formats and the background formats in which tools like
> OpenOffice store the data in xml files. May be for docbooks by
> OpenOffice, the best thing would be to give the *diff* output in terms
> of lines.
> I would also appreciate to know what you think and would like to see
> the output in such case.

I think that the implementation may make use of features inherent in the
file format where possible. e.g. I suspect that OpenOffice has the
ability to show "Tracked changes", and then allow the user to view the
changes using the actual OpenOffice implementation.

I suspect that that will get a lot more difficult with e.g. conflicts
and merges, because I doubt that OOo has the ability to show changes
from multiple versions.

But I have to agree with Dscho, that the output would have to depend on
the file type (OOo document), not just the data structure (e.g. XML)
inside the file.

A regular XML file diff could choose to ignore/collapse whitespace
(pretty printing) when doing the comparison, to show things like moving
a branch further down the tree.

e.g.

<i>text</i>

vs

<b><i>text</i></b>

vs

<b>
  <i>text</i>
</b>

For plain XML, a textual diff might choose to show it with each element
un-indented, and a standard text diff output:

+ <b>
  <i>
  text
  </i>
+ </b>

while a GUI diff might show the new element highlighted in a tree:

#green#<b>#/green#
  <i>
   text

I think that where reasonable that you should aim to have a text-only
version that could be wrapped by a GUI. Obviously, this would be
meaningless when diffing a JPG, for instance.

Ok, that was a bit rambling. I hope it helped more than it confused.

Rogan

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, saurabh gupta wrote:

> On Wed, Mar 11, 2009 at 7:32 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>
> > On Wed, 11 Mar 2009, saurabh gupta wrote:
> >
> >> What I think is to implement file formats other than text like that 
> >> written on wiki i.e. latex, xml, or even any database file (db file). 
> >> Another idea (although it can be weired also) is to implement the new 
> >> file formats in the plug-in formats. For example, to incorporate the 
> >> merger engine for a new file format, a plug-in is created and can be 
> >> integrated with the present merger in the git. However, I am not sure 
> >> how much valid is this idea to make the present merger in git to be 
> >> compatible with the plug-ins for enabling newer file formats.
> >
> > I am not sure that a plugin structure is needed.  Take, for example, 
> > three different .xml based formats: OpenOffice documents, .svg files 
> > and Ant build.xml files.  They need very different user interfaces.
> 
> okay. In that case, if they have a different user interfaces then 
> separate plug-in would be needed for each of these. May be this will get 
> more messy.

The thing is: "plugin" is an architecture issue, which I think we will 
have plenty of time hashing out.  "GUI" is the bigger problem, because if 
we cannot come up with something that is worth implementing, we can stop 
the project early.

> >> However, I think in merging and notifying about the conflicts in the 
> >> xml files, other things can also be put forward. Like the GUI will 
> >> show the number of tags differing and what are the new tags added and 
> >> even if any tag is renamed with the content unchanged. If possible, 
> >> how about showing a tree like structure (just like DOM model) to 
> >> compare (or diff) the two xml files.
> >
> > This is a little bit too low-level for my liking.  Taking the 
> > OpenOffice example again, the GUI should not expose XML at all...
> 
> hmmmm.....I think I get your point somewhat. Let me do some research 
> over the formats and the background formats in which tools like 
> OpenOffice store the data in xml files. May be for docbooks by 
> OpenOffice, the best thing would be to give the *diff* output in terms 
> of lines.

Actually, I think that the diff is not the issue.  What is needed is a way 
that is both intuitive and versatile enough that all kinds of merge 
conflicts in OpenOffice documents can be resolved by a total computer 
illiterate.

The same problem applies for SVG files, but the user interface would look 
completely different.

As such, it might not be wise to have a common framework at all, but to 
make the first an extension for OpenOffice, and the second a modification 
of, say, inkscape.

Of course, David will then come and say: "But that is more appropriate a 
project for OpenOffice and inkscape, then!".

The good thing is that this is Open Source, and we'll just ask them to 
co-mentor this project.

> I would also appreciate to know what you think and would like to see the 
> output in such case.

That's the thing: I do not know yet how it should look like.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 8:57 PM, Rogan Dawes <lists@dawes.za.net> wrote:
> saurabh gupta wrote:
>>>> However, I think in merging and notifying about the conflicts in the xml
>>>> files, other things can also be put forward. Like the GUI will show the
>>>> number of tags differing and what are the new tags added and even if any
>>>> tag is renamed with the content unchanged. If possible, how about
>>>> showing a tree like structure (just like DOM model) to compare (or diff)
>>>> the two xml files.
>>>
>>> This is a little bit too low-level for my liking.  Taking the OpenOffice
>>> example again, the GUI should not expose XML at all...
>>
>> hmmmm.....I think I get your point somewhat. Let me do some research
>> over the formats and the background formats in which tools like
>> OpenOffice store the data in xml files. May be for docbooks by
>> OpenOffice, the best thing would be to give the *diff* output in terms
>> of lines.
>> I would also appreciate to know what you think and would like to see
>> the output in such case.
>
> I think that the implementation may make use of features inherent in the
> file format where possible. e.g. I suspect that OpenOffice has the
> ability to show "Tracked changes", and then allow the user to view the
> changes using the actual OpenOffice implementation.
>
> I suspect that that will get a lot more difficult with e.g. conflicts
> and merges, because I doubt that OOo has the ability to show changes
> from multiple versions.
>
> But I have to agree with Dscho, that the output would have to depend on
> the file type (OOo document), not just the data structure (e.g. XML)
> inside the file.
>
> A regular XML file diff could choose to ignore/collapse whitespace
> (pretty printing) when doing the comparison, to show things like moving
> a branch further down the tree.
>
> e.g.
>
> <i>text</i>
>
> vs
>
> <b><i>text</i></b>
>
> vs
>
> <b>
>  <i>text</i>
> </b>
>
> For plain XML, a textual diff might choose to show it with each element
> un-indented, and a standard text diff output:
>
> + <b>
>  <i>
>  text
>  </i>
> + </b>
>
> while a GUI diff might show the new element highlighted in a tree:
>
> #green#<b>#/green#
>  <i>
>   text
>
> I think that where reasonable that you should aim to have a text-only
> version that could be wrapped by a GUI. Obviously, this would be
> meaningless when diffing a JPG, for instance.

All right. I got what you mean to say.
>
> Ok, that was a bit rambling. I hope it helped more than it confused.
>
> Rogan
>



-- 
Saurabh Gupta
Senior,
Electronics and Communication Engg.
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 9:08 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Wed, 11 Mar 2009, saurabh gupta wrote:
>
>> On Wed, Mar 11, 2009 at 7:32 PM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>
>> > On Wed, 11 Mar 2009, saurabh gupta wrote:
>> >
>> >> What I think is to implement file formats other than text like that
>> >> written on wiki i.e. latex, xml, or even any database file (db file).
>> >> Another idea (although it can be weired also) is to implement the new
>> >> file formats in the plug-in formats. For example, to incorporate the
>> >> merger engine for a new file format, a plug-in is created and can be
>> >> integrated with the present merger in the git. However, I am not sure
>> >> how much valid is this idea to make the present merger in git to be
>> >> compatible with the plug-ins for enabling newer file formats.
>> >
>> > I am not sure that a plugin structure is needed.  Take, for example,
>> > three different .xml based formats: OpenOffice documents, .svg files
>> > and Ant build.xml files.  They need very different user interfaces.
>>
>> okay. In that case, if they have a different user interfaces then
>> separate plug-in would be needed for each of these. May be this will get
>> more messy.
>
> The thing is: "plugin" is an architecture issue, which I think we will
> have plenty of time hashing out.  "GUI" is the bigger problem, because if
> we cannot come up with something that is worth implementing, we can stop
> the project early.

We can decide for the GUI part. RIght now, I am going through the
background for Tcl/Tk. I am sure that using this tool will serve the
purpose in the better way and portability issue can also be kept in
mind.


>> >> However, I think in merging and notifying about the conflicts in the
>> >> xml files, other things can also be put forward. Like the GUI will
>> >> show the number of tags differing and what are the new tags added and
>> >> even if any tag is renamed with the content unchanged. If possible,
>> >> how about showing a tree like structure (just like DOM model) to
>> >> compare (or diff) the two xml files.
>> >
>> > This is a little bit too low-level for my liking.  Taking the
>> > OpenOffice example again, the GUI should not expose XML at all...
>>
>> hmmmm.....I think I get your point somewhat. Let me do some research
>> over the formats and the background formats in which tools like
>> OpenOffice store the data in xml files. May be for docbooks by
>> OpenOffice, the best thing would be to give the *diff* output in terms
>> of lines.
>
> Actually, I think that the diff is not the issue.  What is needed is a way
> that is both intuitive and versatile enough that all kinds of merge
> conflicts in OpenOffice documents can be resolved by a total computer
> illiterate.
>
> The same problem applies for SVG files, but the user interface would look
> completely different.
>
> As such, it might not be wise to have a common framework at all, but to
> make the first an extension for OpenOffice, and the second a modification
> of, say, inkscape.
>
> Of course, David will then come and say: "But that is more appropriate a
> project for OpenOffice and inkscape, then!".
>
> The good thing is that this is Open Source, and we'll just ask them to
> co-mentor this project.

All right. I agree with this and We can come up with a plan to
implement the thing in an organized way. I agree with you that one
common platform for these will not work because of the way they are
represented and is comfortable for a normal computer user.

>> I would also appreciate to know what you think and would like to see the
>> output in such case.
>
> That's the thing: I do not know yet how it should look like.
>
> Ciao,
> Dscho
>
>



-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Daniel Barkalow]

On Wed, 11 Mar 2009, saurabh gupta wrote:

> On Wed, Mar 11, 2009 at 7:32 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > Hi,
> >
> > On Wed, 11 Mar 2009, saurabh gupta wrote:
> >
> >> What I think is to implement file formats other than text like that
> >> written on wiki i.e. latex, xml, or even any database file (db file).
> >> Another idea (although it can be weired also) is to implement the new
> >> file formats in the plug-in formats. For example, to incorporate the
> >> merger engine for a new file format, a plug-in is created and can be
> >> integrated with the present merger in the git. However, I am not sure
> >> how much valid is this idea to make the present merger in git to be
> >> compatible with the plug-ins for enabling newer file formats.
> >
> > I am not sure that a plugin structure is needed.  Take, for example, three
> > different .xml based formats: OpenOffice documents, .svg files and Ant
> > build.xml files.  They need very different user interfaces.
> 
> okay. In that case, if they have  a different user interfaces then
> separate plug-in would be needed for each of these. May be this will
> get more messy.

One thing that I think would be good whenever possible is to have the 
merge program generate a file in the same format which is easily 
recognizable as having conflict markers. For example, I think it should be 
possible to show conflicts in the text of office documents by having 
styles for each side of the merge, and show each side's content in the 
appropriate style. Then the user opens the document with their choice of 
office software, finds the things in the conflict styles, and decides what 
the result should be.

Of course, if the two sides conflict over something that isn't text, it 
gets harder.

Also remember that, for a merge, there are two important cases: (1) the 
two sides changed things that aren't related at all; (2) the two sides 
changed things that might affect each other. In case (1), the tool should 
take care of everything automatically and report that it took care of it; 
in case (2), it should reliably determine that user assistance is 
required.

	-Daniel
*This .sig left intentionally blank*

Re: Google Summer of Code 2009: GIT

[david]

On Wed, 11 Mar 2009, Johannes Schindelin wrote:

> Hi,
>
> On Wed, 11 Mar 2009, saurabh gupta wrote:
>
>> What I think is to implement file formats other than text like that
>> written on wiki i.e. latex, xml, or even any database file (db file).
>> Another idea (although it can be weired also) is to implement the new
>> file formats in the plug-in formats. For example, to incorporate the
>> merger engine for a new file format, a plug-in is created and can be
>> integrated with the present merger in the git. However, I am not sure
>> how much valid is this idea to make the present merger in git to be
>> compatible with the plug-ins for enabling newer file formats.
>
> I am not sure that a plugin structure is needed.  Take, for example, three
> different .xml based formats: OpenOffice documents, .svg files and Ant
> build.xml files.  They need very different user interfaces.
>
>> I am thinking of using gtk+ libraries to implement the GUI part (I am
>> quite comfortable with gtk+).
>
> I mentioned Tcl/Tk, because it is portable, but I'll also take gtk-based
> stuff ;-)
>
>> However, I think in merging and notifying about the conflicts in the xml
>> files, other things can also be put forward. Like the GUI will show the
>> number of tags differing and what are the new tags added and even if any
>> tag is renamed with the content unchanged. If possible, how about
>> showing a tree like structure (just like DOM model) to compare (or diff)
>> the two xml files.
>
> This is a little bit too low-level for my liking.  Taking the OpenOffice
> example again, the GUI should not expose XML at all...

don't assume that you have a GUI just to handle a filetype. if you have 
one, good, make use of it. but have a fallback for how to deal with things 
if all you have is a text terminal.

David Lang

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, Daniel Barkalow wrote:

> One thing that I think would be good whenever possible is to have the 
> merge program generate a file in the same format which is easily 
> recognizable as having conflict markers. For example, I think it should 
> be possible to show conflicts in the text of office documents by having 
> styles for each side of the merge, and show each side's content in the 
> appropriate style. Then the user opens the document with their choice of 
> office software, finds the things in the conflict styles, and decides 
> what the result should be.

That's a very good idea!  (Except for LaTeX, maybe...)

For SVG, you could add both versions of a modified object, for 
example, maybe with some visual effect to show the version...

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 9:59 PM, Daniel Barkalow <barkalow@iabervon.org> wrote:
> On Wed, 11 Mar 2009, saurabh gupta wrote:
>
>> On Wed, Mar 11, 2009 at 7:32 PM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> > Hi,
>> >
>> > On Wed, 11 Mar 2009, saurabh gupta wrote:
>> >
>> >> What I think is to implement file formats other than text like that
>> >> written on wiki i.e. latex, xml, or even any database file (db file).
>> >> Another idea (although it can be weired also) is to implement the new
>> >> file formats in the plug-in formats. For example, to incorporate the
>> >> merger engine for a new file format, a plug-in is created and can be
>> >> integrated with the present merger in the git. However, I am not sure
>> >> how much valid is this idea to make the present merger in git to be
>> >> compatible with the plug-ins for enabling newer file formats.
>> >
>> > I am not sure that a plugin structure is needed.  Take, for example, three
>> > different .xml based formats: OpenOffice documents, .svg files and Ant
>> > build.xml files.  They need very different user interfaces.
>>
>> okay. In that case, if they have  a different user interfaces then
>> separate plug-in would be needed for each of these. May be this will
>> get more messy.
>
> One thing that I think would be good whenever possible is to have the
> merge program generate a file in the same format which is easily
> recognizable as having conflict markers. For example, I think it should be
> possible to show conflicts in the text of office documents by having
> styles for each side of the merge, and show each side's content in the
> appropriate style. Then the user opens the document with their choice of
> office software, finds the things in the conflict styles, and decides what
> the result should be.

Well, I think this is what which is done in case of normal text files
also. The conflicts put the markers in the file to indicate the
changes and the modification part. However, in the case of OO
documents, we have to change the content for the xml file and when it
is opened in the office software, the user will get the modified
contents.


> Of course, if the two sides conflict over something that isn't text, it
> gets harder.
>
> Also remember that, for a merge, there are two important cases: (1) the
> two sides changed things that aren't related at all; (2) the two sides
> changed things that might affect each other. In case (1), the tool should
> take care of everything automatically and report that it took care of it;
> in case (2), it should reliably determine that user assistance is
> required.
>
>        -Daniel
> *This .sig left intentionally blank*
>



-- 
Saurabh Gupta
Senior,
Electronics and Communication Engg.
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, david@lang.hm wrote:

> On Wed, 11 Mar 2009, Johannes Schindelin wrote:
> 
> > On Wed, 11 Mar 2009, saurabh gupta wrote:
> >
> > > What I think is to implement file formats other than text like that
> > > written on wiki i.e. latex, xml, or even any database file (db file).
> > > Another idea (although it can be weired also) is to implement the new
> > > file formats in the plug-in formats. For example, to incorporate the
> > > merger engine for a new file format, a plug-in is created and can be
> > > integrated with the present merger in the git. However, I am not sure
> > > how much valid is this idea to make the present merger in git to be
> > > compatible with the plug-ins for enabling newer file formats.
> >
> > I am not sure that a plugin structure is needed.  Take, for example, three
> > different .xml based formats: OpenOffice documents, .svg files and Ant
> > build.xml files.  They need very different user interfaces.
> >
> > > I am thinking of using gtk+ libraries to implement the GUI part (I am
> > > quite comfortable with gtk+).
> >
> > I mentioned Tcl/Tk, because it is portable, but I'll also take gtk-based
> > stuff ;-)
> >
> > > However, I think in merging and notifying about the conflicts in the xml
> > > files, other things can also be put forward. Like the GUI will show the
> > > number of tags differing and what are the new tags added and even if any
> > > tag is renamed with the content unchanged. If possible, how about
> > > showing a tree like structure (just like DOM model) to compare (or diff)
> > > the two xml files.
> >
> > This is a little bit too low-level for my liking.  Taking the OpenOffice
> > example again, the GUI should not expose XML at all...
> 
> don't assume that you have a GUI just to handle a filetype. if you have one,
> good, make use of it. but have a fallback for how to deal with things if all
> you have is a text terminal.

I do not think it makes sense to assume all you have at your hands is a 
terminal when you try to resolve a merge conflict in an .svg file.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Wed, Mar 11, 2009 at 10:02 PM,  <david@lang.hm> wrote:
> On Wed, 11 Mar 2009, Johannes Schindelin wrote:
>
>> Hi,
>>
>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>
>>> What I think is to implement file formats other than text like that
>>> written on wiki i.e. latex, xml, or even any database file (db file).
>>> Another idea (although it can be weired also) is to implement the new
>>> file formats in the plug-in formats. For example, to incorporate the
>>> merger engine for a new file format, a plug-in is created and can be
>>> integrated with the present merger in the git. However, I am not sure
>>> how much valid is this idea to make the present merger in git to be
>>> compatible with the plug-ins for enabling newer file formats.
>>
>> I am not sure that a plugin structure is needed.  Take, for example, three
>> different .xml based formats: OpenOffice documents, .svg files and Ant
>> build.xml files.  They need very different user interfaces.
>>
>>> I am thinking of using gtk+ libraries to implement the GUI part (I am
>>> quite comfortable with gtk+).
>>
>> I mentioned Tcl/Tk, because it is portable, but I'll also take gtk-based
>> stuff ;-)
>>
>>> However, I think in merging and notifying about the conflicts in the xml
>>> files, other things can also be put forward. Like the GUI will show the
>>> number of tags differing and what are the new tags added and even if any
>>> tag is renamed with the content unchanged. If possible, how about
>>> showing a tree like structure (just like DOM model) to compare (or diff)
>>> the two xml files.
>>
>> This is a little bit too low-level for my liking.  Taking the OpenOffice
>> example again, the GUI should not expose XML at all...
>
> don't assume that you have a GUI just to handle a filetype. if you have one,
> good, make use of it. but have a fallback for how to deal with things if all
> you have is a text terminal.

In case of only a terminal, It would be very difficult to show an OO
document to represent the *diff* output in both text as well in GUI.
For example, to indicate the changes in an OO document, we will have
to change the underlying XML file appropriately to show the markers
signs and other things in the conflict file. Now, if this file is
opened in terminal, it would not be at all comprehensible to see the
differences.

The main thing is that to create *diff* for different file formats, we
will have to write the parser code accordingly.

> David Lang
>



-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[david]

On Wed, 11 Mar 2009, saurabh gupta wrote:

> On Wed, Mar 11, 2009 at 10:02 PM,  <david@lang.hm> wrote:
>> On Wed, 11 Mar 2009, Johannes Schindelin wrote:
>>
>>> Hi,
>>>
>>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>>
>>>> What I think is to implement file formats other than text like that
>>>> written on wiki i.e. latex, xml, or even any database file (db file).
>>>> Another idea (although it can be weired also) is to implement the new
>>>> file formats in the plug-in formats. For example, to incorporate the
>>>> merger engine for a new file format, a plug-in is created and can be
>>>> integrated with the present merger in the git. However, I am not sure
>>>> how much valid is this idea to make the present merger in git to be
>>>> compatible with the plug-ins for enabling newer file formats.
>>>
>>> I am not sure that a plugin structure is needed.  Take, for example, three
>>> different .xml based formats: OpenOffice documents, .svg files and Ant
>>> build.xml files.  They need very different user interfaces.
>>>
>>>> I am thinking of using gtk+ libraries to implement the GUI part (I am
>>>> quite comfortable with gtk+).
>>>
>>> I mentioned Tcl/Tk, because it is portable, but I'll also take gtk-based
>>> stuff ;-)
>>>
>>>> However, I think in merging and notifying about the conflicts in the xml
>>>> files, other things can also be put forward. Like the GUI will show the
>>>> number of tags differing and what are the new tags added and even if any
>>>> tag is renamed with the content unchanged. If possible, how about
>>>> showing a tree like structure (just like DOM model) to compare (or diff)
>>>> the two xml files.
>>>
>>> This is a little bit too low-level for my liking.  Taking the OpenOffice
>>> example again, the GUI should not expose XML at all...
>>
>> don't assume that you have a GUI just to handle a filetype. if you have one,
>> good, make use of it. but have a fallback for how to deal with things if all
>> you have is a text terminal.
>
> In case of only a terminal, It would be very difficult to show an OO
> document to represent the *diff* output in both text as well in GUI.
> For example, to indicate the changes in an OO document, we will have
> to change the underlying XML file appropriately to show the markers
> signs and other things in the conflict file. Now, if this file is
> opened in terminal, it would not be at all comprehensible to see the
> differences.
>
> The main thing is that to create *diff* for different file formats, we
> will have to write the parser code accordingly.

correct, and in the case of an XML file, the meaningful diff can be 
substantially shorter than what a text diff of the two files would be 
(whitespace changes that don't matter, even some tag ordering changes 
may not matter)

I'm just asking that you don't get so fixated on what can be done in a GUI 
that you provide no benifit to people who don't have the GUI

there are a _lot_ of XML based formats out there, having a diff/merge 
capability to make dealing with them better than just treating them as 
text files would be a _very_ useful thing.

going beyond that and creating the ability to do the markup in 
application-specific ways, and present it to the user in a nice GUI would 
also be nice, but these are a step up after having the basic XML handling 
that isn't specific to a particular application.

David Lang

Re: Google Summer of Code 2009: GIT

[david]

On Wed, 11 Mar 2009, Johannes Schindelin wrote:

> Hi,
>
> On Wed, 11 Mar 2009, david@lang.hm wrote:
>
>> On Wed, 11 Mar 2009, Johannes Schindelin wrote:
>>
>>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>>
>>>> What I think is to implement file formats other than text like that
>>>> written on wiki i.e. latex, xml, or even any database file (db file).
>>>> Another idea (although it can be weired also) is to implement the new
>>>> file formats in the plug-in formats. For example, to incorporate the
>>>> merger engine for a new file format, a plug-in is created and can be
>>>> integrated with the present merger in the git. However, I am not sure
>>>> how much valid is this idea to make the present merger in git to be
>>>> compatible with the plug-ins for enabling newer file formats.
>>>
>>> I am not sure that a plugin structure is needed.  Take, for example, three
>>> different .xml based formats: OpenOffice documents, .svg files and Ant
>>> build.xml files.  They need very different user interfaces.
>>>
>>>> I am thinking of using gtk+ libraries to implement the GUI part (I am
>>>> quite comfortable with gtk+).
>>>
>>> I mentioned Tcl/Tk, because it is portable, but I'll also take gtk-based
>>> stuff ;-)
>>>
>>>> However, I think in merging and notifying about the conflicts in the xml
>>>> files, other things can also be put forward. Like the GUI will show the
>>>> number of tags differing and what are the new tags added and even if any
>>>> tag is renamed with the content unchanged. If possible, how about
>>>> showing a tree like structure (just like DOM model) to compare (or diff)
>>>> the two xml files.
>>>
>>> This is a little bit too low-level for my liking.  Taking the OpenOffice
>>> example again, the GUI should not expose XML at all...
>>
>> don't assume that you have a GUI just to handle a filetype. if you have one,
>> good, make use of it. but have a fallback for how to deal with things if all
>> you have is a text terminal.
>
> I do not think it makes sense to assume all you have at your hands is a
> terminal when you try to resolve a merge conflict in an .svg file.

I'm not saying that you assume that all you have is a terminal, I'm saying 
that you _support_ the case that all you have is a terminal.

David Lang

Re: Google Summer of Code 2009: GIT

[Junio C Hamano]

Saurabh Gupta <saurabhgupta1403@gmail.com> writes:

> *1) Domain specific merge helpers*
> Intelligence in the merger can be put which modifies the source file
> according the format. Different file formats can be put in the merger
> to support.

The way "merge" works is:

 - A tree-level 3-way merge is done (either inside merge-recursive backend
   or with "read-tree -m O A B" inside merge-resolve), and trivial merges
   are resolved at the whole-file level without need for any helper.
   Roughly speaking, the definition of "a trivially mergeable path" is a
   path that only one side modified while the other side didn't, or both
   sides modified identically.

 - The remaining paths need to be merged at file level.  The gitattributes
   mechanism is used to decide what exact algorithm to use based on what
   the merged file is; we have a plain-text "xdl" merge driver and "union"
   merge driver built-in, in addition to "binary" merge driver (which
   always says "the changes conflict; use ours as a tentative result).

   When a merge driver is called, it is given three blobs: original, ours
   and theirs (any one of them could be missing).  It is the driver's
   responsibility to come up with an automated merge result when the
   changes do not overlap and report success, or leave an intermediate
   "conflicted merge" and report conflicts.  In either case, the driver is
   expected to return _one_ single bytestream as its tentative result.

 - Cleanly merged paths are updated in the index and their results are
   written out to the work tree.  For paths the merge drivers reported
   conflicts, tentative results returned by the merge drivers are written
   out to the work tree but the index entries for them are left in an
   unmerged state.

 - If all paths are cleanly merged, "git merge" and friends write the
   index out as a tree and create a commit out of it (unless otherwise
   instructed) and report success.

 - When a merge left conflicts, the user can use external tools like "git
   mergetool" to resolve the conflict in the work tree, starting from the
   tentative result given by the merge driver, and "git add" to register
   the resolution to the index.

When people say a "merge helper" in the context of git, I think they think
about at least two kinds, that work at very different layers.  It is
unclear which one you are more interested in, or you are tackling both.

 - A group of new merge drivers that handle various structured text
   formats (e.g. XML based ones), on which the default plain-text merge is
   not suitable, would be a good addition to the git suite.  If you are
   interested in doing this as your GSoC project, it would very much be
   git specific.

 - Amerge helper that takes three files (original, ours and theirs) as its
   input and helps the end user (perhaps graphically) merge them can be
   used at a backend to the "git-mergetool", by registering a filetype as
   "binary" (so that the low-level merge driver won't even try merging the
   contents at the file level), and letting "git-mergetool" invoke the
   "helper" with these three files.  The development of this kind of
   "helper" would not be a git specific project.

Obviously it would help the users to have both, but which kind is more
important?

In a collaborative environment, people do not work in void without any
communication with each other, and they actively try not to step on each
other's toes.  Even when changes are made from both sides of a merge to
the same file (in other words, a file level merge is required), in the
majority of cases, the changes do not overlap, and being able to resolve
such merges cleanly most of the time, without having to resort to an
external "git mergetool", is a huge win in productivity.  I think a domain
specific "merge driver" project would benefit the git users a lot more
than a domain specific "merge helper" that can be used as a "git
mergetool" backend (and can also be used outside git).

Of course, you can do both.

But my point is it is unclear which one you meant when you said "merge
helper".

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, david@lang.hm wrote:

> On Wed, 11 Mar 2009, Johannes Schindelin wrote:
> 
> > On Wed, 11 Mar 2009, david@lang.hm wrote:
> >
> > > On Wed, 11 Mar 2009, Johannes Schindelin wrote:
> > >
> > > > On Wed, 11 Mar 2009, saurabh gupta wrote:
> > > >
> > > > > What I think is to implement file formats other than text like 
> > > > > that written on wiki i.e. latex, xml, or even any database file 
> > > > > (db file). Another idea (although it can be weired also) is to 
> > > > > implement the new file formats in the plug-in formats. For 
> > > > > example, to incorporate the merger engine for a new file format, 
> > > > > a plug-in is created and can be integrated with the present 
> > > > > merger in the git. However, I am not sure how much valid is this 
> > > > > idea to make the present merger in git to be compatible with the 
> > > > > plug-ins for enabling newer file formats.
> > > >
> > > > I am not sure that a plugin structure is needed.  Take, for 
> > > > example, three different .xml based formats: OpenOffice documents, 
> > > > .svg files and Ant build.xml files.  They need very different user 
> > > > interfaces.
> > > >
> > > > > I am thinking of using gtk+ libraries to implement the GUI part 
> > > > > (I am quite comfortable with gtk+).
> > > >
> > > > I mentioned Tcl/Tk, because it is portable, but I'll also take 
> > > > gtk-based stuff ;-)
> > > >
> > > > > However, I think in merging and notifying about the conflicts in 
> > > > > the xml files, other things can also be put forward. Like the 
> > > > > GUI will show the number of tags differing and what are the new 
> > > > > tags added and even if any tag is renamed with the content 
> > > > > unchanged. If possible, how about showing a tree like structure 
> > > > > (just like DOM model) to compare (or diff) the two xml files.
> > > >
> > > > This is a little bit too low-level for my liking.  Taking the 
> > > > OpenOffice example again, the GUI should not expose XML at all...
> > >
> > > don't assume that you have a GUI just to handle a filetype. if you 
> > > have one, good, make use of it. but have a fallback for how to deal 
> > > with things if all you have is a text terminal.
> >
> > I do not think it makes sense to assume all you have at your hands is 
> > a terminal when you try to resolve a merge conflict in an .svg file.
> 
> I'm not saying that you assume that all you have is a terminal, I'm 
> saying that you _support_ the case that all you have is a terminal.

Sorry, no, the GSoC idea was not about "merge helpers that run also in a 
terminal".  The idea was about "Domain specific merge helpers".

If I can choose, I'd rather have support for one more merge helper, even 
if it is all graphical, than an enhancement to support also a terminal.

While I am dreaming: this is the list of domains _I_ would like to see 
supported: LaTeX, OpenOffice documents, .svg files.

But that is not up to me to decide, just to suggest.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Thu, Mar 12, 2009 at 12:59 AM,  <david@lang.hm> wrote:
> On Wed, 11 Mar 2009, saurabh gupta wrote:
>
>> On Wed, Mar 11, 2009 at 10:02 PM,  <david@lang.hm> wrote:
>>>
>>> On Wed, 11 Mar 2009, Johannes Schindelin wrote:
>>>
>>>> Hi,
>>>>
>>>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>>>
>>
>> In case of only a terminal, It would be very difficult to show an OO
>> document to represent the *diff* output in both text as well in GUI.
>> For example, to indicate the changes in an OO document, we will have
>> to change the underlying XML file appropriately to show the markers
>> signs and other things in the conflict file. Now, if this file is
>> opened in terminal, it would not be at all comprehensible to see the
>> differences.
>>
>> The main thing is that to create *diff* for different file formats, we
>> will have to write the parser code accordingly.
>
> correct, and in the case of an XML file, the meaningful diff can be
> substantially shorter than what a text diff of the two files would be
> (whitespace changes that don't matter, even some tag ordering changes may
> not matter)
>
> I'm just asking that you don't get so fixated on what can be done in a GUI
> that you provide no benifit to people who don't have the GUI
>
> there are a _lot_ of XML based formats out there, having a diff/merge
> capability to make dealing with them better than just treating them as text
> files would be a _very_ useful thing.
>
> going beyond that and creating the ability to do the markup in
> application-specific ways, and present it to the user in a nice GUI would
> also be nice, but these are a step up after having the basic XML handling
> that isn't specific to a particular application.

Yes, but the thing is that the underlying codes and method will be
different for GUI part and terminal part to make it readable and
understandable. Like for OO Documents, if we aim to show the *diff*
output in the Office tool, then we have to change the xml file
accordingly. But the same xml file when used with terminal only, the
*diff* output is not clear.

As Johannes said in above post that for OO documents, while showing
the *diff* result, no xml data should be shown.

-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[david]

On Thu, 12 Mar 2009, saurabh gupta wrote:

> On Thu, Mar 12, 2009 at 12:59 AM,  <david@lang.hm> wrote:
>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>
>>> On Wed, Mar 11, 2009 at 10:02 PM,  <david@lang.hm> wrote:
>>>>
>>>> On Wed, 11 Mar 2009, Johannes Schindelin wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>>>>
>>>
>>> In case of only a terminal, It would be very difficult to show an OO
>>> document to represent the *diff* output in both text as well in GUI.
>>> For example, to indicate the changes in an OO document, we will have
>>> to change the underlying XML file appropriately to show the markers
>>> signs and other things in the conflict file. Now, if this file is
>>> opened in terminal, it would not be at all comprehensible to see the
>>> differences.
>>>
>>> The main thing is that to create *diff* for different file formats, we
>>> will have to write the parser code accordingly.
>>
>> correct, and in the case of an XML file, the meaningful diff can be
>> substantially shorter than what a text diff of the two files would be
>> (whitespace changes that don't matter, even some tag ordering changes may
>> not matter)
>>
>> I'm just asking that you don't get so fixated on what can be done in a GUI
>> that you provide no benifit to people who don't have the GUI
>>
>> there are a _lot_ of XML based formats out there, having a diff/merge
>> capability to make dealing with them better than just treating them as text
>> files would be a _very_ useful thing.
>>
>> going beyond that and creating the ability to do the markup in
>> application-specific ways, and present it to the user in a nice GUI would
>> also be nice, but these are a step up after having the basic XML handling
>> that isn't specific to a particular application.
>
> Yes, but the thing is that the underlying codes and method will be
> different for GUI part and terminal part to make it readable and
> understandable. Like for OO Documents, if we aim to show the *diff*
> output in the Office tool, then we have to change the xml file
> accordingly. But the same xml file when used with terminal only, the
> *diff* output is not clear.
>
> As Johannes said in above post that for OO documents, while showing
> the *diff* result, no xml data should be shown.

in part we are talking about different aspects of things, and we were all 
wrong.

see the e-mail a little bit ago by Junio

there are two types of helpers that can be written

1. a low-level part that does the simple merges automaticaly and leaves 
behind appropriate conflict markers when it can't

there is no GUI involved with this.

what 'appropriate conflict markers' are can vary from XML file to XML file


2. after a conflict has taken place, a helper to work with the user to 
resolve the conflict

this can have a GUI and/or a text UI and is tied to the 'appropriate 
conflict markers' as defined in #1, and can be _very_ tightly coupled to 
the specific use of the XML file.

I think it's very important to have a text UI tool that can be used for 
the conflict resolution step as well as supporting GUI tools.


besides XML-based formats, a couple other formats that I think would be 
useful to be smarter about

unordered config files
   files where config options can appear in any order

   optionally: comments are similar to whitespace (they can be ignored)

'paragraph' based config files
   files where config options are orginized into 'paragraphs' where the 
paragraphs can be re-ordered

   the definition of what's a paragraph may differ, support having 
different defintions
     examples:
      the git config file has a high level tag that starts on the left margin with the sub-tags indented
      the apache config file can have single entries or 'XML like' sections

   optionally: comments are similar to whitespace (they can be ignored)


David Lang

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 11 Mar 2009, david@lang.hm wrote:

> there are two types of helpers that can be written
> 
> 1. a low-level part that does the simple merges automaticaly and leaves 
>    behind appropriate conflict markers when it can't
> 
> [...]
> 
> 
> 2. after a conflict has taken place, a helper to work with the user to 
>    resolve the conflict

I thought that from my description on the wiki it was obvious that both 
are needed.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[david]

On Wed, 11 Mar 2009, Johannes Schindelin wrote:

> Hi,
>
> On Wed, 11 Mar 2009, david@lang.hm wrote:
>
>> there are two types of helpers that can be written
>>
>> 1. a low-level part that does the simple merges automaticaly and leaves
>>    behind appropriate conflict markers when it can't
>>
>> [...]
>>
>>
>> 2. after a conflict has taken place, a helper to work with the user to
>>    resolve the conflict
>
> I thought that from my description on the wiki it was obvious that both
> are needed.

first off, I'll admit that I am just going by what's been posted here, I 
haven't gone looking on the wiki.

secondly, I somewhat disagree with you. #1 is needed for any new formats 
that are goning to be handled, but #2 may not be.

take the case of OO documents, you may not need to write a conflict 
resolver helper. the 'appropriate conflict markers' may be something that 
shows up in your normal OO document editor similar to how the ====> shows 
up in a text editor for text conflicts

David Lang

Re: Google Summer of Code 2009: GIT

[Junio C Hamano]

david@lang.hm writes:

>>> there are two types of helpers that can be written
>>>
>>> 1. a low-level part that does the simple merges automaticaly and leaves
>>>    behind appropriate conflict markers when it can't
>>>
>>> 2. after a conflict has taken place, a helper to work with the user to
>>>    resolve the conflict
> ...
> secondly, I somewhat disagree with you. #1 is needed for any new
> formats that are goning to be handled, but #2 may not be.
>
> take the case of OO documents, you may not need to write a conflict
> resolver helper. the 'appropriate conflict markers' may be something
> that shows up in your normal OO document editor similar to how the
> ====> shows up in a text editor for text conflicts

You can cut it both ways.  For an OO document, you do not necessarily need
any file-level merger at the driver level, but just let the "binary"
driver declare conflicts and punt.  A merge helper can do all the work
starting from the "original, ours and theirs" that are not smudged with
conflict markers.

Between these two extremes, the discussion from other people in the thread
seemed to all focus too heavily on the "driver punts" approach, forgetting
that mergetool is useful only because most of the time we do not have to
even use it, thanks to the fact that "xdl" driver works reasonably well
for most trivial cases where branches being merged stayed away from each
other, which is the majority case.  It is a huge win from the productivity
point of view, and many people might be unaware of it because it is so
invisible.

Handling trivial cases safely and automatically at the driver level, if
you can figure out how, lets the user focus on hard cases that needs
manual intervention, and "merge helper" is about helping that manual
intervention step.

Being aware of these two distinct layers allows you to realize that there
is a third possibility.  The driver could notice the cases it can resolve
cleanly and return a cleanly merged result.  When it cannot autoresolve,
but there is no way to "mark" a tentative result with conflict markers, it
can do the same thing as the "binary" driver and let the mergetool backend
handle the "driver punted" case.

Re: Google Summer of Code 2009: GIT

[thestar]

<Entire conversation snipped>

Guys, I'm sure you're only using OpenOffice.org documents as an  
example to facilitate discussing merge helpers, but I feel the need to  
point out that one can already merge OpenOffice documents using  
OpenOffice to do so. (And I have done so in the past while reconciling  
differences between some spreadsheets).

However, the interface OpenOffice provides for that is awful and very  
confusing, but it is there. :)

The document merge provided by Microsoft Office 2003 is vastly  
superior imho, however it only provides 'change markers' - and doesn't  
help you do the actual merge - one must do a merge to get the change  
annotations, and then manually modify a new copy with those updates  
you feel are neccessary.
  - This is one feature that does not appear to be present in  
Microsoft Office 2007, by the way...

Re: Google Summer of Code 2009: GIT

[Junio C Hamano]

thestar@fussycoder.id.au writes:

> <Entire conversation snipped>

Which is not appreciated at all.

> Guys, I'm sure you're only using OpenOffice.org documents as an
> example to facilitate discussing merge helpers, but I feel the need to
> point out that one can already merge OpenOffice documents using
> OpenOffice to do so. (And I have done so in the past while reconciling
> differences between some spreadsheets).

It sounds like OOo itself can be counted as a "domain specific merge
helper" in the second sense of the word---it wouldn't work as a merge
driver but the end user can use it to merge the forked documents.

Which is good to know ;-)

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

hello,

On Thu, Mar 12, 2009 at 1:51 AM,  <david@lang.hm> wrote:
>
>> Yes, but the thing is that the underlying codes and method will be
>> different for GUI part and terminal part to make it readable and
>> understandable. Like for OO Documents, if we aim to show the *diff*
>> output in the Office tool, then we have to change the xml file
>> accordingly. But the same xml file when used with terminal only, the
>> *diff* output is not clear.
>>
>> As Johannes said in above post that for OO documents, while showing
>> the *diff* result, no xml data should be shown.
>
> in part we are talking about different aspects of things, and we were all
> wrong.
>
> see the e-mail a little bit ago by Junio
>
> there are two types of helpers that can be written
>
> 1. a low-level part that does the simple merges automaticaly and leaves
> behind appropriate conflict markers when it can't
>
> there is no GUI involved with this.
>
> what 'appropriate conflict markers' are can vary from XML file to XML file
>
>
> 2. after a conflict has taken place, a helper to work with the user to
> resolve the conflict
>
> this can have a GUI and/or a text UI and is tied to the 'appropriate
> conflict markers' as defined in #1, and can be _very_ tightly coupled to the
> specific use of the XML file.
>
> I think it's very important to have a text UI tool that can be used for the
> conflict resolution step as well as supporting GUI tools.

All right. What I can understand from the current situation is that
for merging and marking conflicts in xml (for example) files has
following things to do.

One, if the markers are put in the xml files like that of a text file,
one can see the difference using a text editor or a terminal. But if
the same xml file is to be opened in another editor which expects a
valid xml (as clearly mentioned on the wiki ideas for GIT), then a
merge helper is needed.

But if the conflict markers are put in a way to make the xml file
still valid which can be then opened in the appropriate editor, then
the marking will be different. The merge driver has to produce the
conflicted merged file in a manner which is still a valid xml file and
user has the choice to open it in his own editor to resolve the
conflicts.




-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Thu, Mar 12, 2009 at 3:17 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> You can cut it both ways.  For an OO document, you do not necessarily need
> any file-level merger at the driver level, but just let the "binary"
> driver declare conflicts and punt.  A merge helper can do all the work
> starting from the "original, ours and theirs" that are not smudged with
> conflict markers.
>
> Between these two extremes, the discussion from other people in the thread
> seemed to all focus too heavily on the "driver punts" approach, forgetting
> that mergetool is useful only because most of the time we do not have to
> even use it, thanks to the fact that "xdl" driver works reasonably well
> for most trivial cases where branches being merged stayed away from each
> other, which is the majority case.  It is a huge win from the productivity
> point of view, and many people might be unaware of it because it is so
> invisible.

If I am not wrong, then for merging two xml files, if we use a simple
xdl merge driver then it will mark the conflicts in the normal way as
it does for simple text files. As far as I can understand, the
following things are supposed to be aimed here taking an example of
xml file:


=>Merging of two xml files

=> existing merge driver (like xdl) is called which marks the
conflicts points just like a normal text file.

=> the conflicted file can be read through a text terminal and
conflicted lines can be seen.

=> suppose the xml file is from the domain of OO document. Then, a
merge helper for OO xml type file is called which takes input as the
conflicted file produced by xdl driver.

=> The merge helper creates a new file or changes the input file to
make it a valid xml file so that it can be opened in OpenOffice and
user can see the markers like "====" or "<<<<<"  in an appropriate
manner and can resolve the file manually.


> When it cannot autoresolve,
> but there is no way to "mark" a tentative result with conflict markers, it
> can do the same thing as the "binary" driver and let the mergetool backend
> handle the "driver punted" case.

I think you mean to say that in case, there is a conflict and the
changes don't overlap, then merge driver leaves the file as it is and
the merge helper will handle the file.


-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Michael J Gruber]

saurabh gupta venit, vidit, dixit 11.03.2009 17:58:
> On Wed, Mar 11, 2009 at 9:59 PM, Daniel Barkalow <barkalow@iabervon.org> wrote:
>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>
>>> On Wed, Mar 11, 2009 at 7:32 PM, Johannes Schindelin
>>> <Johannes.Schindelin@gmx.de> wrote:
>>>> Hi,
>>>>
>>>> On Wed, 11 Mar 2009, saurabh gupta wrote:
>>>>
>>>>> What I think is to implement file formats other than text like that
>>>>> written on wiki i.e. latex, xml, or even any database file (db file).
>>>>> Another idea (although it can be weired also) is to implement the new
>>>>> file formats in the plug-in formats. For example, to incorporate the
>>>>> merger engine for a new file format, a plug-in is created and can be
>>>>> integrated with the present merger in the git. However, I am not sure
>>>>> how much valid is this idea to make the present merger in git to be
>>>>> compatible with the plug-ins for enabling newer file formats.
>>>> I am not sure that a plugin structure is needed.  Take, for example, three
>>>> different .xml based formats: OpenOffice documents, .svg files and Ant
>>>> build.xml files.  They need very different user interfaces.
>>> okay. In that case, if they have  a different user interfaces then
>>> separate plug-in would be needed for each of these. May be this will
>>> get more messy.
>> One thing that I think would be good whenever possible is to have the
>> merge program generate a file in the same format which is easily
>> recognizable as having conflict markers. For example, I think it should be
>> possible to show conflicts in the text of office documents by having
>> styles for each side of the merge, and show each side's content in the
>> appropriate style. Then the user opens the document with their choice of
>> office software, finds the things in the conflict styles, and decides what
>> the result should be.
> Well, I think this is what which is done in case of normal text files
> also. The conflicts put the markers in the file to indicate the
> changes and the modification part. However, in the case of OO
> documents, we have to change the content for the xml file and when it
> is opened in the office software, the user will get the modified
> contents.

OO already knows versioned documents and recording of changes. It can
even merge documents which are different modifications of the same base
document (assuming all authors used recording of changes) and compare
possibly unrelated documents, merging them interactively. At least OO 3
can do that. So I guess for OO one mostly has to figure out how to call
that stuff from the command line. Heck, even MS Office can do that.
Remember those docs with recorded changes, where published documents
contained the deletions as well as the deleted passages?

Michael

Re: Google Summer of Code 2009: GIT

[Michael J Gruber]

Johannes Schindelin venit, vidit, dixit 11.03.2009 17:44:
> Hi,
> 
> On Wed, 11 Mar 2009, Daniel Barkalow wrote:
> 
>> One thing that I think would be good whenever possible is to have the 
>> merge program generate a file in the same format which is easily 
>> recognizable as having conflict markers. For example, I think it should 
>> be possible to show conflicts in the text of office documents by having 
>> styles for each side of the merge, and show each side's content in the 
>> appropriate style. Then the user opens the document with their choice of 
>> office software, finds the things in the conflict styles, and decides 
>> what the result should be.
> That's a very good idea!  (Except for LaTeX, maybe...)

latexdiff (in perl) may give you a head start (or ache, I dunno).

> For SVG, you could add both versions of a modified object, for 
> example, maybe with some visual effect to show the version...

Layers maybe?

I think for most formats, content changes could be handled well, while
changes in macro definitions or global settings are somewhat hopeless.

Michael

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Thu, 12 Mar 2009, Michael J Gruber wrote:

> Johannes Schindelin venit, vidit, dixit 11.03.2009 17:44:
> 
> > On Wed, 11 Mar 2009, Daniel Barkalow wrote:
> > 
> >> One thing that I think would be good whenever possible is to have the 
> >> merge program generate a file in the same format which is easily 
> >> recognizable as having conflict markers. For example, I think it 
> >> should be possible to show conflicts in the text of office documents 
> >> by having styles for each side of the merge, and show each side's 
> >> content in the appropriate style. Then the user opens the document 
> >> with their choice of office software, finds the things in the 
> >> conflict styles, and decides what the result should be.
> > That's a very good idea!  (Except for LaTeX, maybe...)
> 
> latexdiff (in perl) may give you a head start (or ache, I dunno).

No.  latexdiff is about diffing.  It does nothing to help me resolve a 
conflict.

Thanks,
Dscho

Re: Google Summer of Code 2009: GIT

[Michael J Gruber]

Johannes Schindelin venit, vidit, dixit 12.03.2009 14:07:
> Hi,
> 
> On Thu, 12 Mar 2009, Michael J Gruber wrote:
> 
>> Johannes Schindelin venit, vidit, dixit 11.03.2009 17:44:
>>
>>> On Wed, 11 Mar 2009, Daniel Barkalow wrote:
>>>
>>>> One thing that I think would be good whenever possible is to have the 
>>>> merge program generate a file in the same format which is easily 
>>>> recognizable as having conflict markers. For example, I think it 
>>>> should be possible to show conflicts in the text of office documents 
>>>> by having styles for each side of the merge, and show each side's 
>>>> content in the appropriate style. Then the user opens the document 
>>>> with their choice of office software, finds the things in the 
>>>> conflict styles, and decides what the result should be.
>>> That's a very good idea!  (Except for LaTeX, maybe...)
>> latexdiff (in perl) may give you a head start (or ache, I dunno).
> No.  latexdiff is about diffing.  It does nothing to help me resolve a 
> conflict.

Sure. If you want to merge you have to diff first (and display it). That
would be the "head start". Then you have to think about the merge.
That's where the "head ache" kicks in...

Michael

Re: Google Summer of Code 2009: GIT

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3377 bytes --]

On Thu, 12 Mar 2009, saurabh gupta wrote:

> On Thu, Mar 12, 2009 at 3:17 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> You can cut it both ways.  For an OO document, you do not necessarily need
>> any file-level merger at the driver level, but just let the "binary"
>> driver declare conflicts and punt.  A merge helper can do all the work
>> starting from the "original, ours and theirs" that are not smudged with
>> conflict markers.
>>
>> Between these two extremes, the discussion from other people in the thread
>> seemed to all focus too heavily on the "driver punts" approach, forgetting
>> that mergetool is useful only because most of the time we do not have to
>> even use it, thanks to the fact that "xdl" driver works reasonably well
>> for most trivial cases where branches being merged stayed away from each
>> other, which is the majority case.  It is a huge win from the productivity
>> point of view, and many people might be unaware of it because it is so
>> invisible.
>
> If I am not wrong, then for merging two xml files, if we use a simple
> xdl merge driver then it will mark the conflicts in the normal way as
> it does for simple text files. As far as I can understand, the
> following things are supposed to be aimed here taking an example of
> xml file:
>
>
> =>Merging of two xml files
>
> => existing merge driver (like xdl) is called which marks the
> conflicts points just like a normal text file.
>
> => the conflicted file can be read through a text terminal and
> conflicted lines can be seen.
>
> => suppose the xml file is from the domain of OO document. Then, a
> merge helper for OO xml type file is called which takes input as the
> conflicted file produced by xdl driver.
>
> => The merge helper creates a new file or changes the input file to
> make it a valid xml file so that it can be opened in OpenOffice and
> user can see the markers like "====" or "<<<<<"  in an appropriate
> manner and can resolve the file manually.

with XML files it's possible to be symanticly identical, but not identical 
as far as a text merge driver is concerned.

for example, the following two tags are identical in meaning, but would 
show up as different (and therefor in conflict) in a text merge


<tag attr1='foo' attr2='bar' />
<tag attr2='bar' attr1='foo' />

in many instances (such as config files), the order of items doesn't 
change the meaning. so the following two items would be identical

<tag1>
   stuff
</tag1>
<tag2>
   more stuff
</tag2>

vs

<tag2>
   more stuff
</tag2>
<tag1>
   stuff>
</tag1>

in addition whitespace may or may not be relavent (depending on how the 
XML is used) so the following may also be identical

<tag>stuff<tag>

vs
<tag>
stuff
</tag>

a good XML merge driver would have options that you could set for a 
particular file type to know about these sorts of things.


>>  When it cannot autoresolve,
>> but there is no way to "mark" a tentative result with conflict markers, it
>> can do the same thing as the "binary" driver and let the mergetool backend
>> handle the "driver punted" case.
>
> I think you mean to say that in case, there is a conflict and the
> changes don't overlap, then merge driver leaves the file as it is and
> the merge helper will handle the file.

if there is a conflict it should be because the changes do overlap. if 
they don't overlap why is it a conflict?

David Lang

Re: Google Summer of Code 2009: GIT

[david]

On Thu, 12 Mar 2009, saurabh gupta wrote:

> hello,
>
> On Thu, Mar 12, 2009 at 1:51 AM,  <david@lang.hm> wrote:
>>
>>> Yes, but the thing is that the underlying codes and method will be
>>> different for GUI part and terminal part to make it readable and
>>> understandable. Like for OO Documents, if we aim to show the *diff*
>>> output in the Office tool, then we have to change the xml file
>>> accordingly. But the same xml file when used with terminal only, the
>>> *diff* output is not clear.
>>>
>>> As Johannes said in above post that for OO documents, while showing
>>> the *diff* result, no xml data should be shown.
>>
>> in part we are talking about different aspects of things, and we were all
>> wrong.
>>
>> see the e-mail a little bit ago by Junio
>>
>> there are two types of helpers that can be written
>>
>> 1. a low-level part that does the simple merges automaticaly and leaves
>> behind appropriate conflict markers when it can't
>>
>> there is no GUI involved with this.
>>
>> what 'appropriate conflict markers' are can vary from XML file to XML file
>>
>>
>> 2. after a conflict has taken place, a helper to work with the user to
>> resolve the conflict
>>
>> this can have a GUI and/or a text UI and is tied to the 'appropriate
>> conflict markers' as defined in #1, and can be _very_ tightly coupled to the
>> specific use of the XML file.
>>
>> I think it's very important to have a text UI tool that can be used for the
>> conflict resolution step as well as supporting GUI tools.
>
> All right. What I can understand from the current situation is that
> for merging and marking conflicts in xml (for example) files has
> following things to do.
>
> One, if the markers are put in the xml files like that of a text file,
> one can see the difference using a text editor or a terminal. But if
> the same xml file is to be opened in another editor which expects a
> valid xml (as clearly mentioned on the wiki ideas for GIT), then a
> merge helper is needed.
>
> But if the conflict markers are put in a way to make the xml file
> still valid which can be then opened in the appropriate editor, then
> the marking will be different. The merge driver has to produce the
> conflicted merged file in a manner which is still a valid xml file and
> user has the choice to open it in his own editor to resolve the
> conflicts.

exactly. and how you mark the conflict to have it be valid XML is going to 
depend on details of the type of file. there are probably a few basic 
methods that will work the vast majority of the time, but with some 
details needing to be configurable.

for example, if the XML document is a ODF document, it may be possible to 
add 'revision' tags around the conflict that are already understood by the 
editor.

David Lang

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Thu, Mar 12, 2009 at 11:33 PM, <david@lang.hm> wrote:
>
> On Thu, 12 Mar 2009, saurabh gupta wrote:
>
>> hello,
>>
>> On Thu, Mar 12, 2009 at 1:51 AM,  <david@lang.hm> wrote:
>>>
>>>> Yes, but the thing is that the underlying codes and method will be
>>>> different for GUI part and terminal part to make it readable and
>>>> understandable. Like for OO Documents, if we aim to show the *diff*
>>>> output in the Office tool, then we have to change the xml file
>>>> accordingly. But the same xml file when used with terminal only, the
>>>> *diff* output is not clear.
>>>>
>>>> As Johannes said in above post that for OO documents, while showing
>>>> the *diff* result, no xml data should be shown.
>>>
>>> in part we are talking about different aspects of things, and we were all
>>> wrong.
>>>
>>> see the e-mail a little bit ago by Junio
>>>
>>> there are two types of helpers that can be written
>>>
>>> 1. a low-level part that does the simple merges automaticaly and leaves
>>> behind appropriate conflict markers when it can't
>>>
>>> there is no GUI involved with this.
>>>
>>> what 'appropriate conflict markers' are can vary from XML file to XML file
>>>
>>>
>>> 2. after a conflict has taken place, a helper to work with the user to
>>> resolve the conflict
>>>
>>> this can have a GUI and/or a text UI and is tied to the 'appropriate
>>> conflict markers' as defined in #1, and can be _very_ tightly coupled to the
>>> specific use of the XML file.
>>>
>>> I think it's very important to have a text UI tool that can be used for the
>>> conflict resolution step as well as supporting GUI tools.
>>
>> All right. What I can understand from the current situation is that
>> for merging and marking conflicts in xml (for example) files has
>> following things to do.
>>
>> One, if the markers are put in the xml files like that of a text file,
>> one can see the difference using a text editor or a terminal. But if
>> the same xml file is to be opened in another editor which expects a
>> valid xml (as clearly mentioned on the wiki ideas for GIT), then a
>> merge helper is needed.
>>
>> But if the conflict markers are put in a way to make the xml file
>> still valid which can be then opened in the appropriate editor, then
>> the marking will be different. The merge driver has to produce the
>> conflicted merged file in a manner which is still a valid xml file and
>> user has the choice to open it in his own editor to resolve the
>> conflicts.
>
> exactly. and how you mark the conflict to have it be valid XML is going to depend on details of the type of file. there are probably a few basic methods that will work the vast majority of the time, but with some details needing to be configurable.
>
> for example, if the XML document is a ODF document, it may be possible to add 'revision' tags around the conflict that are already understood by the editor.

Exactly. This includes the work to modify the xml tags and add
contents to represent marker in the best way.


--
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[david]

On Thu, 12 Mar 2009, Michael J Gruber wrote:

> Johannes Schindelin venit, vidit, dixit 12.03.2009 14:07:
>> Hi,
>>
>> On Thu, 12 Mar 2009, Michael J Gruber wrote:
>>
>>> Johannes Schindelin venit, vidit, dixit 11.03.2009 17:44:
>>>
>>>> On Wed, 11 Mar 2009, Daniel Barkalow wrote:
>>>>
>>>>> One thing that I think would be good whenever possible is to have the
>>>>> merge program generate a file in the same format which is easily
>>>>> recognizable as having conflict markers. For example, I think it
>>>>> should be possible to show conflicts in the text of office documents
>>>>> by having styles for each side of the merge, and show each side's
>>>>> content in the appropriate style. Then the user opens the document
>>>>> with their choice of office software, finds the things in the
>>>>> conflict styles, and decides what the result should be.
>>>> That's a very good idea!  (Except for LaTeX, maybe...)
>>> latexdiff (in perl) may give you a head start (or ache, I dunno).
>> No.  latexdiff is about diffing.  It does nothing to help me resolve a
>> conflict.
>
> Sure. If you want to merge you have to diff first (and display it). That
> would be the "head start". Then you have to think about the merge.
> That's where the "head ache" kicks in...

the idea is that the merge driver should do the diff and merge the simple 
things, only needing to display it if there is a conflict that it cannot 
resolve.

David Lang

Re: Google Summer of Code 2009: GIT

[Junio C Hamano]

david@lang.hm writes:

> On Thu, 12 Mar 2009, saurabh gupta wrote:
>
>> On Thu, Mar 12, 2009 at 3:17 AM, Junio C Hamano <gitster@pobox.com> wrote:
> ...
> with XML files it's possible to be symanticly identical, but not
> identical as far as a text merge driver is concerned.
> ...
> a good XML merge driver would have options that you could set for a
> particular file type to know about these sorts of things.

Correct.

>>>  When it cannot autoresolve,
>>> but there is no way to "mark" a tentative result with conflict markers, it
>>> can do the same thing as the "binary" driver and let the mergetool backend
>>> handle the "driver punted" case.
>>
>> I think you mean to say that in case, there is a conflict and the
>> changes don't overlap, then merge driver leaves the file as it is and
>> the merge helper will handle the file.
>
> if there is a conflict it should be because the changes do overlap. if
> they don't overlap why is it a conflict?

Correct.  In such a case when the "driver" can be sure that the result is
reasonable, "helper" should not even kick in.  That was the main point of
my suggestion, which you seem to have got right.

Thanks.

Re: Google Summer of Code 2009: GIT

[david]

On Thu, 12 Mar 2009, saurabh gupta wrote:

> On Thu, Mar 12, 2009 at 11:30 PM, <david@lang.hm> wrote:
>
>> On Thu, 12 Mar 2009, saurabh gupta wrote:
>>
>>>
>>> =>Merging of two xml files
>>>
>>> => existing merge driver (like xdl) is called which marks the
>>> conflicts points just like a normal text file.
>>>
>>> => the conflicted file can be read through a text terminal and
>>> conflicted lines can be seen.
>>>
>>> => suppose the xml file is from the domain of OO document. Then, a
>>> merge helper for OO xml type file is called which takes input as the
>>> conflicted file produced by xdl driver.
>>>
>>> => The merge helper creates a new file or changes the input file to
>>> make it a valid xml file so that it can be opened in OpenOffice and
>>> user can see the markers like "====" or "<<<<<"  in an appropriate
>>> manner and can resolve the file manually.
>>>
>>
>> with XML files it's possible to be symanticly identical, but not identical
>> as far as a text merge driver is concerned.
>>
<SNIPB>
> you are right. For xml merging, what I am thinking is to create the
> algorithm based on the document object model. Inside, any tag, all tags are
> compared only in terms of content and not in order. But again, this ordering
> option can be given to the user. If the user wants order to matter, then a
> conflict will be resulted if order mismatches.

right.

> But other issue is regarding the display of conflict markers. Either
> conflict markers should be put in xml format or like text merger. This is
> the main project idea for GSoC 2009.

this may need to be a configurable option, but I suspect that we could get 
away with always using something in XML format. exactly what the markers 
are needs to be configurable (the markers for OO will not be the same as 
for SVG for example)

building a library of 'this works especially well for this app' markers is 
something that needs to be started as part of the GSOC project, but 
possibly only far enough to show a couple of examples and have confidence 
that the tool is configurable enough.

David Lang

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Fri, Mar 13, 2009 at 12:23 AM,  <david@lang.hm> wrote:
> On Thu, 12 Mar 2009, saurabh gupta wrote:
>
>> On Thu, Mar 12, 2009 at 11:30 PM, <david@lang.hm> wrote:
>>
>>> On Thu, 12 Mar 2009, saurabh gupta wrote:
>>>
>>>>
>>>> =>Merging of two xml files
>>>>
>>>> => existing merge driver (like xdl) is called which marks the
>>>> conflicts points just like a normal text file.
>>>>
>>>> => the conflicted file can be read through a text terminal and
>>>> conflicted lines can be seen.
>>>>
>>>> => suppose the xml file is from the domain of OO document. Then, a
>>>> merge helper for OO xml type file is called which takes input as the
>>>> conflicted file produced by xdl driver.
>>>>
>>>> => The merge helper creates a new file or changes the input file to
>>>> make it a valid xml file so that it can be opened in OpenOffice and
>>>> user can see the markers like "====" or "<<<<<"  in an appropriate
>>>> manner and can resolve the file manually.
>>>>
>>>
>>> with XML files it's possible to be symanticly identical, but not
>>> identical
>>> as far as a text merge driver is concerned.
>>>
> <SNIPB>
>>
>> you are right. For xml merging, what I am thinking is to create the
>> algorithm based on the document object model. Inside, any tag, all tags
>> are
>> compared only in terms of content and not in order. But again, this
>> ordering
>> option can be given to the user. If the user wants order to matter, then a
>> conflict will be resulted if order mismatches.
>
> right.
>
>> But other issue is regarding the display of conflict markers. Either
>> conflict markers should be put in xml format or like text merger. This is
>> the main project idea for GSoC 2009.
>
> this may need to be a configurable option, but I suspect that we could get
> away with always using something in XML format. exactly what the markers are
> needs to be configurable (the markers for OO will not be the same as for SVG
> for example)

yeah.

> building a library of 'this works especially well for this app' markers is
> something that needs to be started as part of the GSOC project, but possibly
> only far enough to show a couple of examples and have confidence that the
> tool is configurable enough.
>

I think picking up some formats and then building libraries above that
is needed. In some sense, I talked about the plug-in architecture
also. Can;t it be possible that for different applications (like OO or
SVG), different merge helper plugins are created which can be
integrated with it. Or speaking in  other words, instead of plug-ins
now, libraries for merge helpers for different applications are
created.


-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4820 bytes --]

On Fri, 13 Mar 2009, saurabh gupta wrote:

> On Fri, Mar 13, 2009 at 12:23 AM,  <david@lang.hm> wrote:
>> On Thu, 12 Mar 2009, saurabh gupta wrote:
>>
>>> On Thu, Mar 12, 2009 at 11:30 PM, <david@lang.hm> wrote:
>>>
>>>> On Thu, 12 Mar 2009, saurabh gupta wrote:
>>>>
>>>>>
>>>>> =>Merging of two xml files
>>>>>
>>>>> => existing merge driver (like xdl) is called which marks the
>>>>> conflicts points just like a normal text file.
>>>>>
>>>>> => the conflicted file can be read through a text terminal and
>>>>> conflicted lines can be seen.
>>>>>
>>>>> => suppose the xml file is from the domain of OO document. Then, a
>>>>> merge helper for OO xml type file is called which takes input as the
>>>>> conflicted file produced by xdl driver.
>>>>>
>>>>> => The merge helper creates a new file or changes the input file to
>>>>> make it a valid xml file so that it can be opened in OpenOffice and
>>>>> user can see the markers like "====" or "<<<<<"  in an appropriate
>>>>> manner and can resolve the file manually.
>>>>>
>>>>
>>>> with XML files it's possible to be symanticly identical, but not
>>>> identical
>>>> as far as a text merge driver is concerned.
>>>>
>> <SNIPB>
>>>
>>> you are right. For xml merging, what I am thinking is to create the
>>> algorithm based on the document object model. Inside, any tag, all tags
>>> are
>>> compared only in terms of content and not in order. But again, this
>>> ordering
>>> option can be given to the user. If the user wants order to matter, then a
>>> conflict will be resulted if order mismatches.
>>
>> right.
>>
>>> But other issue is regarding the display of conflict markers. Either
>>> conflict markers should be put in xml format or like text merger. This is
>>> the main project idea for GSoC 2009.
>>
>> this may need to be a configurable option, but I suspect that we could get
>> away with always using something in XML format. exactly what the markers are
>> needs to be configurable (the markers for OO will not be the same as for SVG
>> for example)
>
> yeah.
>
>> building a library of 'this works especially well for this app' markers is
>> something that needs to be started as part of the GSOC project, but possibly
>> only far enough to show a couple of examples and have confidence that the
>> tool is configurable enough.
>>
>
> I think picking up some formats and then building libraries above that
> is needed. In some sense, I talked about the plug-in architecture
> also. Can;t it be possible that for different applications (like OO or
> SVG), different merge helper plugins are created which can be
> integrated with it. Or speaking in  other words, instead of plug-ins
> now, libraries for merge helpers for different applications are
> created.

defining terminology that was mentioned before

merge drivers are run by git to do the merges and create the conflict 
markers. git already has a 'plug-in architecture' for these drivers (you 
can define file types and tell git to use a particular merge driver for 
this file type)

merge helpers are run by the users if there is a conflict and make use of 
the markers. depending on what you end up using for conflict markers, you 
may not need to write a merge helper (for OO, if your conflict markers are 
good enough you can use OO to resolve conflicts easily, no need for a new 
tool)


with this terminology, you can't do merge helpers without doing the merge 
drivers first (what does the helper look for as an indicator of a 
conflict?)

I believe that there is a lot of potential for a configurable merge driver 
to support many similar formats.

using the example of XML-based files, configurable options could include

1. is the file stored compressed or not

2. does the order of the tags matter

3. does whitespace matter

   note: #2 and #3 may boil down to 'is this a document with XML markup, or 
are the XML tags the primary content'

4. how is the conflict marked

4a. wrap the conflicting tags in a set of tags that look like _

4b. if the conflict is in the content, not the tags, modify it similar to 
what we do with text today.

   note: this still requires the new driver to decide if there is a 
conflict or not

4c. other (potentially including calling out to other code for more 
drastic restructuring)


with a merge driver along these lines you can handle many different types 
of XML documents.

with SVG you may be able to put the offending tags in different layers

with OO you may be able to put in tags that indicate a merge conflict in a 
way that OO will directly handle

etc.

in many cases you may not even need to create a merge helper or library 
for other software you use. you just need to figure out what sort of 
manipulation would need to be done to to file to mark the conflict in a 
way that existing applications can understand.

David Lang

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Fri, Mar 13, 2009 at 12:59 AM,  <david@lang.hm> wrote:
> On Fri, 13 Mar 2009, saurabh gupta wrote:
>
> defining terminology that was mentioned before
>
> merge drivers are run by git to do the merges and create the conflict
> markers. git already has a 'plug-in architecture' for these drivers (you can
> define file types and tell git to use a particular merge driver for this
> file type)
>
> merge helpers are run by the users if there is a conflict and make use of
> the markers. depending on what you end up using for conflict markers, you
> may not need to write a merge helper (for OO, if your conflict markers are
> good enough you can use OO to resolve conflicts easily, no need for a new
> tool)
>
>
> with this terminology, you can't do merge helpers without doing the merge
> drivers first (what does the helper look for as an indicator of a conflict?)
>
> I believe that there is a lot of potential for a configurable merge driver
> to support many similar formats.
>
> using the example of XML-based files, configurable options could include
>
> 1. is the file stored compressed or not
>
> 2. does the order of the tags matter
>
> 3. does whitespace matter
>
>  note: #2 and #3 may boil down to 'is this a document with XML markup, or
> are the XML tags the primary content'
>
> 4. how is the conflict marked
>
> 4a. wrap the conflicting tags in a set of tags that look like _
>
> 4b. if the conflict is in the content, not the tags, modify it similar to
> what we do with text today.
>
>  note: this still requires the new driver to decide if there is a conflict
> or not
>
> 4c. other (potentially including calling out to other code for more drastic
> restructuring)
>
>
> with a merge driver along these lines you can handle many different types of
> XML documents.
>
> with SVG you may be able to put the offending tags in different layers
>
> with OO you may be able to put in tags that indicate a merge conflict in a
> way that OO will directly handle
>
> etc.
>
> in many cases you may not even need to create a merge helper or library for
> other software you use. you just need to figure out what sort of
> manipulation would need to be done to to file to mark the conflict in a way
> that existing applications can understand.

Very well described, David. I agree with you and providing these merge
options to the user, merge drivers can do the work and mark the
conflicts according to the option. The work to do is to modify the
merge driver. I think in this way, even people who have only a
terminal can also gain from it. They can choose the apt option to see
the conflict markers in their way. So, the aim is to make merge driver
configurable and create the merged/conflicted file according to the
options.


-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[david]

On Fri, 13 Mar 2009, saurabh gupta wrote:

> Very well described, David. I agree with you and providing these merge
> options to the user, merge drivers can do the work and mark the
> conflicts according to the option. The work to do is to modify the
> merge driver. I think in this way, even people who have only a
> terminal can also gain from it. They can choose the apt option to see
> the conflict markers in their way. So, the aim is to make merge driver
> configurable and create the merged/conflicted file according to the
> options.

for the GSOC I suspect that the right thing to do is the define one or 
more merge drivers to create, and list what applications are going to be 
used for testing these merges.

you and the mentor can decide what is a reasonable amount of work.

it may be just doing an XML merge driver is a summer's worth of work, or 
it may be that it's not really enough and you should try to do another one 
or two.

it also may be that there is a lot of overlap between different merge 
drivers, and once you have the XML driver the others become fairly trivial 
to do. (I'm thinking the config file examples I posted earlier in the 
thread)

David Lang

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Fri, Mar 13, 2009 at 1:29 AM,  <david@lang.hm> wrote:
> On Fri, 13 Mar 2009, saurabh gupta wrote:
>
>> Very well described, David. I agree with you and providing these merge
>> options to the user, merge drivers can do the work and mark the
>> conflicts according to the option. The work to do is to modify the
>> merge driver. I think in this way, even people who have only a
>> terminal can also gain from it. They can choose the apt option to see
>> the conflict markers in their way. So, the aim is to make merge driver
>> configurable and create the merged/conflicted file according to the
>> options.
>
> for the GSOC I suspect that the right thing to do is the define one or more
> merge drivers to create, and list what applications are going to be used for
> testing these merges.
>
> you and the mentor can decide what is a reasonable amount of work.
>

I will very glad to hear about this thing from the mentor (Johannes
Schindelin, according to wiki). I will try to plan out the things in a
proper way to carry out this project if I get a chance to work on this
for GSoC 2009.

> it may be just doing an XML merge driver is a summer's worth of work, or it
> may be that it's not really enough and you should try to do another one or
> two.
>
> it also may be that there is a lot of overlap between different merge
> drivers, and once you have the XML driver the others become fairly trivial
> to do. (I'm thinking the config file examples I posted earlier in the
> thread)

with the options given to the user, one can handle the config files
also where order doesn't matter and also the whitespaces problem can
also be handled in the similar way.


-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Junio C Hamano]

david@lang.hm writes:

> defining terminology that was mentioned before
>
> merge drivers are run by git to do the merges and create the conflict
> markers. git already has a 'plug-in architecture' for these drivers
> (you can define file types and tell git to use a particular merge
> driver for this file type)
>
> merge helpers are run by the users if there is a conflict and make use
> of the markers. depending on what you end up using for conflict
> markers, you may not need to write a merge helper (for OO, if your
> conflict markers are good enough you can use OO to resolve conflicts
> easily, no need for a new tool)

Not really.  A merge helper can look at the stages in the index to get the
(original, ours, theirs) tuple and start to work from there (and doing a
helper as a backend of mergetool will be one way to make it easier), and
for such helper the driver does not have to do anything.

> with this terminology, you can't do merge helpers without doing the
> merge drivers first (what does the helper look for as an indicator of
> a conflict?)

The answer to that question is "the index", and your "you can't" is
too strong.

I agree with you that the original "editor" for the specific type of
document (e.g. OOo, inkskape, ...) can be used to fix things up if a
driver can leave conflict markers in such a way that the helper does not
have to do three-file merge, and that would be a nice thing to have.  But
for a helper that can do a real three-file merge (and in this thread I
think somebody said OOo can do that), then it is Ok for a driver to punt.

But even then, *if* there is a driver *and* it can do trivial merges
cleanly and safely, it would be a huge productivity win, as you do not
have to run a helper every time you see two branching touching the same
document even they did so to edit totally unrelated parts of it.

Re: Google Summer of Code 2009: GIT

[david]

On Fri, 13 Mar 2009, saurabh gupta wrote:

> On Fri, Mar 13, 2009 at 1:29 AM,  <david@lang.hm> wrote:
>> On Fri, 13 Mar 2009, saurabh gupta wrote:
>>
>>> Very well described, David. I agree with you and providing these merge
>>> options to the user, merge drivers can do the work and mark the
>>> conflicts according to the option. The work to do is to modify the
>>> merge driver. I think in this way, even people who have only a
>>> terminal can also gain from it. They can choose the apt option to see
>>> the conflict markers in their way. So, the aim is to make merge driver
>>> configurable and create the merged/conflicted file according to the
>>> options.
>>
>> for the GSOC I suspect that the right thing to do is the define one or more
>> merge drivers to create, and list what applications are going to be used for
>> testing these merges.
>>
>> you and the mentor can decide what is a reasonable amount of work.
>>
>
> I will very glad to hear about this thing from the mentor (Johannes
> Schindelin, according to wiki). I will try to plan out the things in a
> proper way to carry out this project if I get a chance to work on this
> for GSoC 2009.
>
>> it may be just doing an XML merge driver is a summer's worth of work, or it
>> may be that it's not really enough and you should try to do another one or
>> two.
>>
>> it also may be that there is a lot of overlap between different merge
>> drivers, and once you have the XML driver the others become fairly trivial
>> to do. (I'm thinking the config file examples I posted earlier in the
>> thread)
>
> with the options given to the user, one can handle the config files
> also where order doesn't matter and also the whitespaces problem can
> also be handled in the similar way.

when I am mentioning config files here I'm thinking of ones that don't use 
XML (such as the git config file)

a 'paragraph' merge driver could also help with things like a maintainers 
file where the order of the paragaphs doesn't matter, just the content 
inside each one.

that's very similar to re-ordering XML tags, but with a slightly different 
syntax

David Lang

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Fri, Mar 13, 2009 at 2:15 AM,  <david@lang.hm> wrote:
>
> when I am mentioning config files here I'm thinking of ones that don't use
> XML (such as the git config file)
>
> a 'paragraph' merge driver could also help with things like a maintainers
> file where the order of the paragaphs doesn't matter, just the content
> inside each one.
>
> that's very similar to re-ordering XML tags, but with a slightly different
> syntax

yes, in that case, I think we can modify the existing text merge
driver somewhat and provide these configuration options to the user.
User can choose the option to configure the merge operation.


-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Rogan Dawes]

saurabh gupta wrote:
>> exactly. and how you mark the conflict to have it be valid XML is
>> going to depend on details of the type of file. there are probably
>> a few basic methods that will work the vast majority of the time,
>> but with some details needing to be configurable.
>> 
>> for example, if the XML document is a ODF document, it may be
>> possible to add 'revision' tags around the conflict that are
>> already understood by the editor.
> 
> Exactly. This includes the work to modify the xml tags and add 
> contents to represent marker in the best way.

On the XML topic, one last thing to keep in mind is the DTD/XSD which
governs the file.

Rogan

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Fri, Mar 13, 2009 at 3:11 PM, Rogan Dawes <lists@dawes.za.net> wrote:
> saurabh gupta wrote:
>>> exactly. and how you mark the conflict to have it be valid XML is
>>> going to depend on details of the type of file. there are probably
>>> a few basic methods that will work the vast majority of the time,
>>> but with some details needing to be configurable.
>>>
>>> for example, if the XML document is a ODF document, it may be
>>> possible to add 'revision' tags around the conflict that are
>>> already understood by the editor.
>>
>> Exactly. This includes the work to modify the xml tags and add
>> contents to represent marker in the best way.
>
> On the XML topic, one last thing to keep in mind is the DTD/XSD which
> governs the file.

This is another point of thinking. A merge helper changing an xml file
may need to modify the schema file also accordingly. Or, by proper
implementation, the need of changing the schema file can be escaped.
:-|


-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Fri, 13 Mar 2009, saurabh gupta wrote:

> On Fri, Mar 13, 2009 at 1:29 AM,  <david@lang.hm> wrote:
> > On Fri, 13 Mar 2009, saurabh gupta wrote:
> >
> >> Very well described, David. I agree with you and providing these 
> >> merge options to the user, merge drivers can do the work and mark the 
> >> conflicts according to the option. The work to do is to modify the 
> >> merge driver. I think in this way, even people who have only a 
> >> terminal can also gain from it. They can choose the apt option to see 
> >> the conflict markers in their way. So, the aim is to make merge 
> >> driver configurable and create the merged/conflicted file according 
> >> to the options.
> >
> > for the GSOC I suspect that the right thing to do is the define one or 
> > more merge drivers to create, and list what applications are going to 
> > be used for testing these merges.
> >
> > you and the mentor can decide what is a reasonable amount of work.
> 
> I will very glad to hear about this thing from the mentor (Johannes 
> Schindelin, according to wiki). I will try to plan out the things in a 
> proper way to carry out this project if I get a chance to work on this 
> for GSoC 2009.

Well, now that we have been accepted as an organization, we can move 
forward with this idea!

My main concern is that we define early on what should be the user 
interface, preferably with a quick sketch.

The technical details, we can hash them out later, I have no doubt that 
with the help of the complete Git community, we can overcome almost every 
problem handling XML data or some such.

> > it may be just doing an XML merge driver is a summer's worth of work, 
> > or it may be that it's not really enough and you should try to do 
> > another one or two.
> >
> > it also may be that there is a lot of overlap between different merge 
> > drivers, and once you have the XML driver the others become fairly 
> > trivial to do. (I'm thinking the config file examples I posted earlier 
> > in the thread)
> 
> with the options given to the user, one can handle the config files
> also where order doesn't matter and also the whitespaces problem can
> also be handled in the similar way.

In my humble opinion, we should focus on the data types we want to be 
able to support at the end of the summer first.

For example, if we decide that OOXML is a must (as it is a proper 
standard, and many people will benefit from it), we will most likely end 
up in having to write a merge _driver_ (to handle those .zip files), _and_ 
a merge _helper_, although we can avoid writing our own GUI, as we can 
create an OOXML that has its own version of conflict markers.

If we decide that SVG is something we want to support by the end of the 
summer, then we can probably avoid writing a merge _driver_, as plain text 
is handled reasonably well in Git.  OTOH it could turn out that there are 
_real_ conflicts in overlapping tag ids, and it would still be easier to 
write a merge driver, too.

IOW the details are not as important as

- knowing what data types we want to support _at the least_, and what data 
  types we keep for the free skate,

- a clear picture of the user interface we want to be able to provide,

- a timeline (weekly milestones should be fine, I guess) what should be 
  achieved when, and

- being flexible in how to support that (IOW if a merge driver appears 
  unnecessary first, but necessary later, we should be able to fit that 
  into both the design and the timeline).

How does that sound?

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[david]

On Thu, 19 Mar 2009, Johannes Schindelin wrote:

> Hi,
>
> On Fri, 13 Mar 2009, saurabh gupta wrote:
>
>> On Fri, Mar 13, 2009 at 1:29 AM,  <david@lang.hm> wrote:
>>> On Fri, 13 Mar 2009, saurabh gupta wrote:
>>>
>>> it may be just doing an XML merge driver is a summer's worth of work,
>>> or it may be that it's not really enough and you should try to do
>>> another one or two.
>>>
>>> it also may be that there is a lot of overlap between different merge
>>> drivers, and once you have the XML driver the others become fairly
>>> trivial to do. (I'm thinking the config file examples I posted earlier
>>> in the thread)
>>
>> with the options given to the user, one can handle the config files
>> also where order doesn't matter and also the whitespaces problem can
>> also be handled in the similar way.
>
> In my humble opinion, we should focus on the data types we want to be
> able to support at the end of the summer first.
>
> For example, if we decide that OOXML is a must (as it is a proper
> standard, and many people will benefit from it), we will most likely end
> up in having to write a merge _driver_ (to handle those .zip files), _and_
> a merge _helper_, although we can avoid writing our own GUI, as we can
> create an OOXML that has its own version of conflict markers.

do you mean OOXML (the microsoft format) or ODF (the open office format)?

> If we decide that SVG is something we want to support by the end of the
> summer, then we can probably avoid writing a merge _driver_, as plain text
> is handled reasonably well in Git.  OTOH it could turn out that there are
> _real_ conflicts in overlapping tag ids, and it would still be easier to
> write a merge driver, too.
>
> IOW the details are not as important as
>
> - knowing what data types we want to support _at the least_, and what data
>  types we keep for the free skate,
>
> - a clear picture of the user interface we want to be able to provide,
>
> - a timeline (weekly milestones should be fine, I guess) what should be
>  achieved when, and
>
> - being flexible in how to support that (IOW if a merge driver appears
>  unnecessary first, but necessary later, we should be able to fit that
>  into both the design and the timeline).

it's up to the student, but I suspect that the best approach would be to 
start with defining a merge driver to handle XML (with a minimum set of 
capabilities, and additional optional ones), and go from there.

David Lang

> How does that sound?
>
> Ciao,
> Dscho
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Wed, 18 Mar 2009, david@lang.hm wrote:

> On Thu, 19 Mar 2009, Johannes Schindelin wrote:
> 
> > On Fri, 13 Mar 2009, saurabh gupta wrote:
> >
> > > On Fri, Mar 13, 2009 at 1:29 AM,  <david@lang.hm> wrote:
> > > > On Fri, 13 Mar 2009, saurabh gupta wrote:
> > > >
> > > > it may be just doing an XML merge driver is a summer's worth of 
> > > > work, or it may be that it's not really enough and you should try 
> > > > to do another one or two.
> > > >
> > > > it also may be that there is a lot of overlap between different 
> > > > merge drivers, and once you have the XML driver the others become 
> > > > fairly trivial to do. (I'm thinking the config file examples I 
> > > > posted earlier in the thread)
> > >
> > > with the options given to the user, one can handle the config files 
> > > also where order doesn't matter and also the whitespaces problem can 
> > > also be handled in the similar way.
> >
> > In my humble opinion, we should focus on the data types we want to be 
> > able to support at the end of the summer first.
> >
> > For example, if we decide that OOXML is a must (as it is a proper 
> > standard, and many people will benefit from it), we will most likely 
> > end up in having to write a merge _driver_ (to handle those .zip 
> > files), _and_ a merge _helper_, although we can avoid writing our own 
> > GUI, as we can create an OOXML that has its own version of conflict 
> > markers.
> 
> do you mean OOXML (the microsoft format) or ODF (the open office 
> format)?

Oops.

EOVERLOAD

> > If we decide that SVG is something we want to support by the end of 
> > the summer, then we can probably avoid writing a merge _driver_, as 
> > plain text is handled reasonably well in Git.  OTOH it could turn out 
> > that there are _real_ conflicts in overlapping tag ids, and it would 
> > still be easier to write a merge driver, too.
> >
> > IOW the details are not as important as
> >
> > - knowing what data types we want to support _at the least_, and what 
> >   data types we keep for the free skate,
> >
> > - a clear picture of the user interface we want to be able to provide,
> >
> > - a timeline (weekly milestones should be fine, I guess) what should 
> >   be achieved when, and
> >
> > - being flexible in how to support that (IOW if a merge driver appears 
> >   unnecessary first, but necessary later, we should be able to fit 
> >   that into both the design and the timeline).
> 
> it's up to the student, but I suspect that the best approach would be to 
> start with defining a merge driver to handle XML (with a minimum set of 
> capabilities, and additional optional ones), and go from there.

Well, the thing is: if the student decides to have a go at an XML driver 
first and foremost, then I'll just flatly refuse to mentor that.  Because 
I sincerely believe that this project is best designed from top to bottom, 
not the other way round.

After all, the project is based on a user's request, not just a 
playthingie for an XML enthusiast (if such a thing exists).

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[Caleb Cushing]

On Wed, Mar 18, 2009 at 7:16 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> In my humble opinion, we should focus on the data types we want to be
> able to support at the end of the summer first.

my 2 cents don't support xml, support sgml start at the least common
denominator and refine from there.

-- 
Caleb Cushing

http://xenoterracide.blogspot.com

Re: Google Summer of Code 2009: GIT

[david]

On Thu, 19 Mar 2009, Johannes Schindelin wrote:

> On Wed, 18 Mar 2009, david@lang.hm wrote:
>
>> On Thu, 19 Mar 2009, Johannes Schindelin wrote:
>>
>>>
>>> In my humble opinion, we should focus on the data types we want to be
>>> able to support at the end of the summer first.
>>>
>>> For example, if we decide that OOXML is a must (as it is a proper
>>> standard, and many people will benefit from it), we will most likely
>>> end up in having to write a merge _driver_ (to handle those .zip
>>> files), _and_ a merge _helper_, although we can avoid writing our own
>>> GUI, as we can create an OOXML that has its own version of conflict
>>> markers.
>>
>> do you mean OOXML (the microsoft format) or ODF (the open office
>> format)?
>
> Oops.
>
> EOVERLOAD

it happens.

>>> If we decide that SVG is something we want to support by the end of
>>> the summer, then we can probably avoid writing a merge _driver_, as
>>> plain text is handled reasonably well in Git.  OTOH it could turn out
>>> that there are _real_ conflicts in overlapping tag ids, and it would
>>> still be easier to write a merge driver, too.
>>>
>>> IOW the details are not as important as
>>>
>>> - knowing what data types we want to support _at the least_, and what
>>>   data types we keep for the free skate,
>>>
>>> - a clear picture of the user interface we want to be able to provide,
>>>
>>> - a timeline (weekly milestones should be fine, I guess) what should
>>>   be achieved when, and
>>>
>>> - being flexible in how to support that (IOW if a merge driver appears
>>>   unnecessary first, but necessary later, we should be able to fit
>>>   that into both the design and the timeline).
>>
>> it's up to the student, but I suspect that the best approach would be to
>> start with defining a merge driver to handle XML (with a minimum set of
>> capabilities, and additional optional ones), and go from there.
>
> Well, the thing is: if the student decides to have a go at an XML driver
> first and foremost, then I'll just flatly refuse to mentor that.  Because
> I sincerely believe that this project is best designed from top to bottom,
> not the other way round.
>
> After all, the project is based on a user's request, not just a
> playthingie for an XML enthusiast (if such a thing exists).

all three formats mentioned here (OOXML, ODF, SVG) are XML-based formats 
and a single flexible XML merge driver could potentially handle all three 
(as well as other formats). for that matter, the ODF specs cover multiple 
types of data, and I suspect that appropriate conflict markers for text 
could well end up being different than the ones for spreadsheets.

that's not a 'plaything for an XML entusiast', it's making the tool 
slightly more general than it would need to be for any one of these 
formats to let it handle all of them.


but I'm not a mentor or a student, just an interested user.

David Lang

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Thu, 19 Mar 2009, Caleb Cushing wrote:

> On Wed, Mar 18, 2009 at 7:16 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>
> > In my humble opinion, we should focus on the data types we want to be 
> > able to support at the end of the summer first.
> 
> my 2 cents don't support xml, support sgml start at the least common 
> denominator and refine from there.

Sorry, by "data type" I tried to refer to the nature of the file.  I 
should have said "file type".

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Thu, 19 Mar 2009, david@lang.hm wrote:

> all three formats mentioned here (OOXML, ODF, SVG) are XML-based formats 
> and a single flexible XML merge driver could potentially handle all 
> three (as well as other formats). for that matter, the ODF specs cover 
> multiple types of data, and I suspect that appropriate conflict markers 
> for text could well end up being different than the ones for 
> spreadsheets.

You are misunderstanding me.

The fact that all three are XML based has nothing to do with the _real_ 
goal of the project.

IOW a user trying to 3-way-merge ODF files could not care less if the 
underlying technical details involve having an extra merge driver for XML 
files or not.

The user cares about the ease of use, about the user interface.  That is 
what I want to focus on.

And if we end up with a beautiful XML merge driver at the end of the 
summer that nobody uses, I will be not only a little disappointed.

So let's look at the _nature_ of the data at hand, i.e. text, marked-up 
text, images (we could include UML, which is also XML-based, and where the 
XML merge driver is as relevant for the user experience as for the 
others), and how to make it _easy_ to resolve merge conflicts there.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[david]

On Thu, 19 Mar 2009, Johannes Schindelin wrote:

> On Thu, 19 Mar 2009, david@lang.hm wrote:
>
>> all three formats mentioned here (OOXML, ODF, SVG) are XML-based formats
>> and a single flexible XML merge driver could potentially handle all
>> three (as well as other formats). for that matter, the ODF specs cover
>> multiple types of data, and I suspect that appropriate conflict markers
>> for text could well end up being different than the ones for
>> spreadsheets.
>
> You are misunderstanding me.
>
> The fact that all three are XML based has nothing to do with the _real_
> goal of the project.
>
> IOW a user trying to 3-way-merge ODF files could not care less if the
> underlying technical details involve having an extra merge driver for XML
> files or not.
>
> The user cares about the ease of use, about the user interface.  That is
> what I want to focus on.
>
> And if we end up with a beautiful XML merge driver at the end of the
> summer that nobody uses, I will be not only a little disappointed.
>
> So let's look at the _nature_ of the data at hand, i.e. text, marked-up
> text, images (we could include UML, which is also XML-based, and where the
> XML merge driver is as relevant for the user experience as for the
> others), and how to make it _easy_ to resolve merge conflicts there.

but don't you want to be able to auto-merge as much as possible before you 
have to go to _any_ user interaction? (the best user interface is one you 
don't need to use)

it's only after the merge drive decides that it can't do the merge that 
you would have to move on to manually resolving conflicts.

when you _do_ move on to resolving conflicts, it's not a good approach to 
write a GUI tool to deal with each datatype (git does not need it's own 
ODF text document editory, spreadsheed editor, graphics editor, etc). it 
may end up being nessasary for some document types, but that's a last 
resort. it's far better if the conflict markers can be inserted in such a 
way that the normal tools for dealing with that file type can be used.

git doesn't provide (or mandate) what editor is used to resolve conflicts 
in text files today, it should not do so for other file types either 
(again, except as a last resort)

the only way to start from the GUI and not create a merge driver first is 
to either have a custom GUI that accepts files 'corrupted' with the 
existing conflict markers, or work on a GUI that works with both of the 
sources as entire files, with no conflict markers or assistance from git.

David Lang

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

Hi all,

Sorry for replying so late as I was busy in my college's mid-semester exams :-|

On Thu, Mar 19, 2009 at 4:46 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Fri, 13 Mar 2009, saurabh gupta wrote:
>
>> On Fri, Mar 13, 2009 at 1:29 AM,  <david@lang.hm> wrote:
>> > On Fri, 13 Mar 2009, saurabh gupta wrote:
>> >
>> >> Very well described, David. I agree with you and providing these
>> >> merge options to the user, merge drivers can do the work and mark the
>> >> conflicts according to the option. The work to do is to modify the
>> >> merge driver. I think in this way, even people who have only a
>> >> terminal can also gain from it. They can choose the apt option to see
>> >> the conflict markers in their way. So, the aim is to make merge
>> >> driver configurable and create the merged/conflicted file according
>> >> to the options.
>> >
>> > for the GSOC I suspect that the right thing to do is the define one or
>> > more merge drivers to create, and list what applications are going to
>> > be used for testing these merges.
>> >
>> > you and the mentor can decide what is a reasonable amount of work.
>>
>> I will very glad to hear about this thing from the mentor (Johannes
>> Schindelin, according to wiki). I will try to plan out the things in a
>> proper way to carry out this project if I get a chance to work on this
>> for GSoC 2009.
>
> Well, now that we have been accepted as an organization, we can move
> forward with this idea!

Congrats for getting accepted in GSoC 2009.

> My main concern is that we define early on what should be the user
> interface, preferably with a quick sketch.
>
> The technical details, we can hash them out later, I have no doubt that
> with the help of the complete Git community, we can overcome almost every
> problem handling XML data or some such.
>
>> > it may be just doing an XML merge driver is a summer's worth of work,
>> > or it may be that it's not really enough and you should try to do
>> > another one or two.
>> >
>> > it also may be that there is a lot of overlap between different merge
>> > drivers, and once you have the XML driver the others become fairly
>> > trivial to do. (I'm thinking the config file examples I posted earlier
>> > in the thread)
>>
>> with the options given to the user, one can handle the config files
>> also where order doesn't matter and also the whitespaces problem can
>> also be handled in the similar way.
>
> In my humble opinion, we should focus on the data types we want to be
> able to support at the end of the summer first.
>
> For example, if we decide that OOXML is a must (as it is a proper
> standard, and many people will benefit from it), we will most likely end
> up in having to write a merge _driver_ (to handle those .zip files), _and_
> a merge _helper_, although we can avoid writing our own GUI, as we can
> create an OOXML that has its own version of conflict markers.

Well, for ODF type document, we can write a merge driver which will
change the xml file in an appropriate way that OO can understand it
and the user can see the merge result/conflict in a comfortable way.
As described by Junio, in this case, a dedicated merge helper is not
needed as OO can parse the markers made by merge-driver and provide
the user to resolve the conflict and register the changes to index.


>
> If we decide that SVG is something we want to support by the end of the
> summer, then we can probably avoid writing a merge _driver_, as plain text
> is handled reasonably well in Git.  OTOH it could turn out that there are
> _real_ conflicts in overlapping tag ids, and it would still be easier to
> write a merge driver, too.

>
> IOW the details are not as important as
>
> - knowing what data types we want to support _at the least_, and what data
>  types we keep for the free skate,

As of now, how about going for XML files. For this summer, we can go
for XML files and latex files can be handled later.

>
> - a clear picture of the user interface we want to be able to provide,

In my opinion, we have following things to do:

=> while merging an ODF document, merge-driver will merge the file at
file level. If changes don't overlap, then it returns the result with
a success. For example, if the file is changed only on one side, then
the driver will simply add the new content.

=> If conflicts appear, then the merge driver will put the markers in
an appropriate manner which the end-user application (e.g. open
office) can understand and show the user. For example, the XML file of
that ODF document will be modified and OO can show it  to user in its
way. We will have to study about the OO style of version marking.
Another method is to implement the marker style in our own way. For
example, to show any marker, the XML file is modified so that user can
see markers like ">>>> " or "====" in openoffice....In this case, we
will have to just change the xml content in this way.

>
> - a timeline (weekly milestones should be fine, I guess) what should be
>  achieved when, and

Timeline can be decided once we reach some conclusion and the work
which needs to be done become clear to us.

> - being flexible in how to support that (IOW if a merge driver appears
>  unnecessary first, but necessary later, we should be able to fit that
>  into both the design and the timeline).
>
> How does that sound?
>
> Ciao,
> Dscho
>
>



-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

On Thu, Mar 19, 2009 at 6:13 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Wed, 18 Mar 2009, david@lang.hm wrote:
>
>> On Thu, 19 Mar 2009, Johannes Schindelin wrote:
>>
>> > On Fri, 13 Mar 2009, saurabh gupta wrote:
>> >
>> > > On Fri, Mar 13, 2009 at 1:29 AM,  <david@lang.hm> wrote:
>> > > > On Fri, 13 Mar 2009, saurabh gupta wrote:
>> > > >
>> > > > it may be just doing an XML merge driver is a summer's worth of
>> > > > work, or it may be that it's not really enough and you should try
>> > > > to do another one or two.
>> > > >
>> > > > it also may be that there is a lot of overlap between different
>> > > > merge drivers, and once you have the XML driver the others become
>> > > > fairly trivial to do. (I'm thinking the config file examples I
>> > > > posted earlier in the thread)
>> > >
>> > > with the options given to the user, one can handle the config files
>> > > also where order doesn't matter and also the whitespaces problem can
>> > > also be handled in the similar way.
>> >
>> > In my humble opinion, we should focus on the data types we want to be
>> > able to support at the end of the summer first.
>> >
>> > For example, if we decide that OOXML is a must (as it is a proper
>> > standard, and many people will benefit from it), we will most likely
>> > end up in having to write a merge _driver_ (to handle those .zip
>> > files), _and_ a merge _helper_, although we can avoid writing our own
>> > GUI, as we can create an OOXML that has its own version of conflict
>> > markers.
>>
>> do you mean OOXML (the microsoft format) or ODF (the open office
>> format)?
>
> Oops.
>
> EOVERLOAD
>
>> > If we decide that SVG is something we want to support by the end of
>> > the summer, then we can probably avoid writing a merge _driver_, as
>> > plain text is handled reasonably well in Git.  OTOH it could turn out
>> > that there are _real_ conflicts in overlapping tag ids, and it would
>> > still be easier to write a merge driver, too.
>> >
>> > IOW the details are not as important as
>> >
>> > - knowing what data types we want to support _at the least_, and what
>> >   data types we keep for the free skate,
>> >
>> > - a clear picture of the user interface we want to be able to provide,
>> >
>> > - a timeline (weekly milestones should be fine, I guess) what should
>> >   be achieved when, and
>> >
>> > - being flexible in how to support that (IOW if a merge driver appears
>> >   unnecessary first, but necessary later, we should be able to fit
>> >   that into both the design and the timeline).
>>
>> it's up to the student, but I suspect that the best approach would be to
>> start with defining a merge driver to handle XML (with a minimum set of
>> capabilities, and additional optional ones), and go from there.
>
> Well, the thing is: if the student decides to have a go at an XML driver
> first and foremost, then I'll just flatly refuse to mentor that.  Because
> I sincerely believe that this project is best designed from top to bottom,
> not the other way round.
>
> After all, the project is based on a user's request, not just a
> playthingie for an XML enthusiast (if such a thing exists).

I do agree with you that unless an end user get to see the conflict
result in an appropriate manner, there is no use of having an xml
merger. But, once we decide as what will be the end file type which we
will aim this summer, we can then start working whether its about
making a GUI first, or creating a merge driver first.




> Ciao,
> Dscho
>
>



-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3554 bytes --]

Hi,

On Fri, 20 Mar 2009, saurabh gupta wrote:

> On Thu, Mar 19, 2009 at 4:46 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>
> > For example, if we decide that OOXML is a must (as it is a proper 
> > standard, and many people will benefit from it), we will most likely 
> > end up in having to write a merge _driver_ (to handle those .zip 
> > files), _and_ a merge _helper_, although we can avoid writing our own 
> > GUI, as we can create an OOXML that has its own version of conflict 
> > markers.
> 
> Well, for ODF type document, we can write a merge driver which will 
> change the xml file in an appropriate way that OO can understand it and 
> the user can see the merge result/conflict in a comfortable way. As 
> described by Junio, in this case, a dedicated merge helper is not needed 
> as OO can parse the markers made by merge-driver and provide the user to 
> resolve the conflict and register the changes to index.

There is also the idea that OOffice has building blocks in place to help 
resolving merge conflicts.  For a successful application, you will have to 
show that you researched that option, and describe how well/badly it fits 
with the goal of the project.

> > - knowing what data types we want to support _at the least_, and what 
> >   data  types we keep for the free skate,
> 
> As of now, how about going for XML files. For this summer, we can go for 
> XML files and latex files can be handled later.

If your goal is just XML files (without any more specific goal, like ODF 
or SVG), I am afraid that I think that project is not worth 4500 dollar 
from Google's pocket.  I mean, we are not talking peanuts here.

> > - a clear picture of the user interface we want to be able to provide,
> 
> In my opinion, we have following things to do:
> 
> => while merging an ODF document, merge-driver will merge the file at
> file level. If changes don't overlap, then it returns the result with
> a success. For example, if the file is changed only on one side, then
> the driver will simply add the new content.
> 
> => If conflicts appear, then the merge driver will put the markers in
> an appropriate manner which the end-user application (e.g. open
> office) can understand and show the user. For example, the XML file of
> that ODF document will be modified and OO can show it  to user in its
> way. We will have to study about the OO style of version marking.
> Another method is to implement the marker style in our own way. For
> example, to show any marker, the XML file is modified so that user can
> see markers like ">>>> " or "====" in openoffice....In this case, we
> will have to just change the xml content in this way.

That is correct, but I would appreciate a bit more definitive research 
_before_ the project proposal, as a sign that you are capable of working 
out the details of the project.

> > - a timeline (weekly milestones should be fine, I guess) what should 
> >   be  achieved when, and
> 
> Timeline can be decided once we reach some conclusion and the work which 
> needs to be done become clear to us.

Last year, most successful applications detailed a proposed timeline in 
their proposal...

Do not get me wrong, I want this project to succeed.

But on the other hand, I feel the obligation to be a bit demanding for the 
gracious donation of Google: we _do_ want to have something stunningly 
awesome at the end of the summer.

And that means that I have to get the impression from the student proposal 
that something like that is at least _possible_.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4504 bytes --]

On Fri, 20 Mar 2009, Johannes Schindelin wrote:

> Hi,
>
> On Fri, 20 Mar 2009, saurabh gupta wrote:
>
>> On Thu, Mar 19, 2009 at 4:46 AM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>
>>> For example, if we decide that OOXML is a must (as it is a proper
>>> standard, and many people will benefit from it), we will most likely
>>> end up in having to write a merge _driver_ (to handle those .zip
>>> files), _and_ a merge _helper_, although we can avoid writing our own
>>> GUI, as we can create an OOXML that has its own version of conflict
>>> markers.
>>
>> Well, for ODF type document, we can write a merge driver which will
>> change the xml file in an appropriate way that OO can understand it and
>> the user can see the merge result/conflict in a comfortable way. As
>> described by Junio, in this case, a dedicated merge helper is not needed
>> as OO can parse the markers made by merge-driver and provide the user to
>> resolve the conflict and register the changes to index.
>
> There is also the idea that OOffice has building blocks in place to help
> resolving merge conflicts.  For a successful application, you will have to
> show that you researched that option, and describe how well/badly it fits
> with the goal of the project.

true, although for the 'simple case' of an ODF text file you can use 
text strings exactly the same way you do with a text file. the difference 
is that when inserting the two versions of things into the 'conflict' 
version of the ODF file you need to make sure that you include the 
complete open/close set of tags in each version.

for example if file 1 has

<tag1 param='1'>
text
</tag1>

and file 2 has

<tag1 param='1'>
text2
</tag1>

you can do


<tag1 param='1'>
>>>>>>>>
text
========
text2
<<<<<<<<
</tag1>


but if file2 has


<tag1 param='2'>
text
</tag1>

your conflict would need to be

>>>>>>>>
<tag1 param='1'>
text
</tag1>
========
<tag1 param='1'>
text
</tag1>
<<<<<<<<

(although since < and > are special characters, they would really be &gt 
and &lt in the file)

if there are nicer ways to do this, supporting them would be good, but as 
long as the marker strings are configurable you can probably do so

you could change
the first string from >>>>>>> to <conflict option='1'>
the second string from ======== to </conflict><conflict option='2'>
the third string from <<<<<<< to </conflict>

and now instead of having to search for those special text strings, your 
ODF editor would 'magicly' identify them and remind you that you hadn't 
resolved all of them.

>>> - knowing what data types we want to support _at the least_, and what
>>>   data  types we keep for the free skate,
>>
>> As of now, how about going for XML files. For this summer, we can go for
>> XML files and latex files can be handled later.
>
> If your goal is just XML files (without any more specific goal, like ODF
> or SVG), I am afraid that I think that project is not worth 4500 dollar
> from Google's pocket.  I mean, we are not talking peanuts here.

I see good support for XML being a superset of what's needed to support 
ODF or SVG, not a subset.

or another way of putting it, the gitconfig definition for ODF would be a 
shortcut for a longer XML definition with a long list of options.

to be accepted by google, they will need to feel that the work is worth 
the money, so defining what file types you are going to support is an 
important item. This can include saying 'by handling this type of tweak to 
an XML file we can then handle file type Y instead of just file type X 
with the same merge driver'

as you are considering this list, please think about the items I mentioned 
earlier in the thread that would improve the support for config files and 
maintainers files (unordered lines/paragraphs)

>>> - a timeline (weekly milestones should be fine, I guess) what should
>>>   be  achieved when, and
>>
>> Timeline can be decided once we reach some conclusion and the work which
>> needs to be done become clear to us.
>
> Last year, most successful applications detailed a proposed timeline in
> their proposal...
>
> Do not get me wrong, I want this project to succeed.
>
> But on the other hand, I feel the obligation to be a bit demanding for the
> gracious donation of Google: we _do_ want to have something stunningly
> awesome at the end of the summer.
>
> And that means that I have to get the impression from the student proposal
> that something like that is at least _possible_.

sounds reasonable.

David Lang

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Thu, 19 Mar 2009, david@lang.hm wrote:

> I see good support for XML being a superset of what's needed to support 
> ODF or SVG, not a subset.

No, not at all.  If we can get away with the default 3-way merge of Git, 
the generic XML merge driver be damned.

I'd rather have more file types supported that are useful for the average 
user, than a generic XML merge driver that is useful to only a handful of 
people.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[david]

On Fri, 20 Mar 2009, Johannes Schindelin wrote:

> On Thu, 19 Mar 2009, david@lang.hm wrote:
>
>> I see good support for XML being a superset of what's needed to support
>> ODF or SVG, not a subset.
>
> No, not at all.  If we can get away with the default 3-way merge of Git,
> the generic XML merge driver be damned.

I would agree, but unless you don't do any auto-merging and punt 
everything to the 'conflict resolution tool' the existing merge drivers 
won't work for a structured file. And if you do want to do that, it's not 
a git project, it's a project for whatever tool you are working from to be 
the GUI plus (possibly) a smidge of scripting to call that tool from git.

> I'd rather have more file types supported that are useful for the average
> user, than a generic XML merge driver that is useful to only a handful of
> people.

we are both after the same thing, the most use to the average user.

you look at SVG, ODF word, ODF spreadsheet, OOXML, etc as completely 
seperate things that should have support developed seperatly.

I look at the same formats and am seeing a strong similarity between them. 
that being that they are all structured XML. so if you get the ability to 
handle XML in a configurable way (and define the appropriate 
configurations), you not only get the tools for these things, but many 
others as well.

I would be a little disappointed if the result of the summer only handled 
XML files (and more so if it only handled a handful of popular XML-based 
files). I think that there are a number of file types that aren't handled 
well by the current merge drivers. Saurabh has voiced the opinion that 
many of these have similar problems as the XML situations, so it may end 
up making sense to handle them in the same driver.

it would probably be a good thing to see suggestions from a bunch of 
people as to what file types they see being useful.

David Lang

Re: Google Summer of Code 2009: GIT

[Johannes Schindelin]

Hi,

On Thu, 19 Mar 2009, david@lang.hm wrote:

> On Fri, 20 Mar 2009, Johannes Schindelin wrote:
> 
> > I'd rather have more file types supported that are useful for the 
> > average user, than a generic XML merge driver that is useful to only a 
> > handful of people.
> 
> we are both after the same thing,

Apparently not...

> the most use to the average user.
> 
> you look at SVG, ODF word, ODF spreadsheet, OOXML, etc as completely 
> seperate things that should have support developed seperatly.

No.  I look at SVG, ODF text, ODF spreadsheet, etc as things with 
completely different user interfaces.

And likewise, the merge _helper_, the very thing the user will get to see, 
must have different user interfaces.

And I see much more potential for this project to fail in those different 
user interfaces than something as _trivial_ (in relation) as XML merging.

Ciao,
Dscho

Re: Google Summer of Code 2009: GIT

[david]

On Fri, 20 Mar 2009, Johannes Schindelin wrote:

> On Thu, 19 Mar 2009, david@lang.hm wrote:
>
>> On Fri, 20 Mar 2009, Johannes Schindelin wrote:
>>
>>> I'd rather have more file types supported that are useful for the
>>> average user, than a generic XML merge driver that is useful to only a
>>> handful of people.
>>
>> we are both after the same thing,
>
> Apparently not...
>
>> the most use to the average user.
>>
>> you look at SVG, ODF word, ODF spreadsheet, OOXML, etc as completely
>> seperate things that should have support developed seperatly.
>
> No.  I look at SVG, ODF text, ODF spreadsheet, etc as things with
> completely different user interfaces.
>
> And likewise, the merge _helper_, the very thing the user will get to see,
> must have different user interfaces.

I absolutly agree with this. the UI and merge _helper_ tools for these 
different file formats are completely different.

> And I see much more potential for this project to fail in those different
> user interfaces than something as _trivial_ (in relation) as XML merging.

and the key thing that I am saying is that a properly done XML merge may 
eliminate the need to do _any_ development of a merge helper tool.

so rather than focusing on what the merge helper tool is going to be and 
what the UI for that is, I am focusing on the potential to eliminate any 
need to hae a specific helper tool by making it so that the marked up 
files can be manipulated with the existing tools for that file type.

David Lang

Google Summer of Code 2009: GIT

[saurabh gupta]

hi,

On Fri, Mar 20, 2009 at 5:12 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Fri, 20 Mar 2009, saurabh gupta wrote:
>
>> On Thu, Mar 19, 2009 at 4:46 AM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>
>> > For example, if we decide that OOXML is a must (as it is a proper
>> > standard, and many people will benefit from it), we will most likely
>> > end up in having to write a merge _driver_ (to handle those .zip
>> > files), _and_ a merge _helper_, although we can avoid writing our own
>> > GUI, as we can create an OOXML that has its own version of conflict
>> > markers.
>>
>> Well, for ODF type document, we can write a merge driver which will
>> change the xml file in an appropriate way that OO can understand it and
>> the user can see the merge result/conflict in a comfortable way. As
>> described by Junio, in this case, a dedicated merge helper is not needed
>> as OO can parse the markers made by merge-driver and provide the user to
>> resolve the conflict and register the changes to index.
>
> There is also the idea that OOffice has building blocks in place to help
> resolving merge conflicts.  For a successful application, you will have to
> show that you researched that option, and describe how well/badly it fits
> with the goal of the project.

Exactly, I will have to do some research on it and I will come back to
you as I get over with my college's mid-semester exams (this week
more).

>> > - knowing what data types we want to support _at the least_, and what
>> >   data  types we keep for the free skate,
>>
>> As of now, how about going for XML files. For this summer, we can go for
>> XML files and latex files can be handled later.
>
> If your goal is just XML files (without any more specific goal, like ODF
> or SVG), I am afraid that I think that project is not worth 4500 dollar
> from Google's pocket.  I mean, we are not talking peanuts here.

Well, I didn;t mean to say that we will end up in only having a merge
driver for xml after the summer. We will definitely make the driver in
such a way to use the maximum power of xml manipulation so that the
end application can understand it and user can get the conflict
results in a user friendly manner because the end-user application wil
be able to parse the merged xml file and will present the conflict
markers in the GUI form.


>
>> > - a clear picture of the user interface we want to be able to provide,
>>
>> In my opinion, we have following things to do:
>>
>> => while merging an ODF document, merge-driver will merge the file at
>> file level. If changes don't overlap, then it returns the result with
>> a success. For example, if the file is changed only on one side, then
>> the driver will simply add the new content.
>>
>> => If conflicts appear, then the merge driver will put the markers in
>> an appropriate manner which the end-user application (e.g. open
>> office) can understand and show the user. For example, the XML file of
>> that ODF document will be modified and OO can show it  to user in its
>> way. We will have to study about the OO style of version marking.
>> Another method is to implement the marker style in our own way. For
>> example, to show any marker, the XML file is modified so that user can
>> see markers like ">>>> " or "====" in openoffice....In this case, we
>> will have to just change the xml content in this way.
>
> That is correct, but I would appreciate a bit more definitive research
> _before_ the project proposal, as a sign that you are capable of working
> out the details of the project.

I do understand your point and will work this.

>> > - a timeline (weekly milestones should be fine, I guess) what should
>> >   be  achieved when, and
>>
>> Timeline can be decided once we reach some conclusion and the work which
>> needs to be done become clear to us.

I meant to say that time line can decided after the list of works is
decided and discussed. Of course, I will present the timeline in my
student application in GSoC 2009. :)

> Do not get me wrong, I want this project to succeed.

I will try my best.

> But on the other hand, I feel the obligation to be a bit demanding for the
> gracious donation of Google: we _do_ want to have something stunningly
> awesome at the end of the summer.
>
> And that means that I have to get the impression from the student proposal
> that something like that is at least _possible_.
>
> Ciao,
> Dscho
>



--
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[saurabh gupta]

hi,

On Sat, Mar 21, 2009 at 2:20 AM,  <david@lang.hm> wrote:
> On Fri, 20 Mar 2009, Johannes Schindelin wrote:
>
>> On Thu, 19 Mar 2009, david@lang.hm wrote:
>>
>>> On Fri, 20 Mar 2009, Johannes Schindelin wrote:
>>>
>>>> I'd rather have more file types supported that are useful for the
>>>> average user, than a generic XML merge driver that is useful to only a
>>>> handful of people.
>>>
>>> we are both after the same thing,
>>
>> Apparently not...
>>
>>> the most use to the average user.
>>>
>>> you look at SVG, ODF word, ODF spreadsheet, OOXML, etc as completely
>>> seperate things that should have support developed seperatly.
>>
>> No.  I look at SVG, ODF text, ODF spreadsheet, etc as things with
>> completely different user interfaces.
>>
>> And likewise, the merge _helper_, the very thing the user will get to see,
>> must have different user interfaces.
>
> I absolutly agree with this. the UI and merge _helper_ tools for these
> different file formats are completely different.
>
>> And I see much more potential for this project to fail in those different
>> user interfaces than something as _trivial_ (in relation) as XML merging.
>
> and the key thing that I am saying is that a properly done XML merge may
> eliminate the need to do _any_ development of a merge helper tool.

I also think that if xml file merging is done in a proper way and
according to the end-user editor, then it will not be needed to work
on the GUI part. For example, to merge the ODF files, we need to study
the merge conflict structure of Openoffice and will have to modify the
xml file in the same way so that OO can understand it. Similarly for
other file types.

>
> so rather than focusing on what the merge helper tool is going to be and
> what the UI for that is, I am focusing on the potential to eliminate any
> need to hae a specific helper tool by making it so that the marked up files
> can be manipulated with the existing tools for that file type.
>
> David Lang
>



-- 
Saurabh Gupta
Senior,
NSIT,New Delhi, India

Re: Google Summer of Code 2009: GIT

[Caleb Cushing]

On Thu, Mar 19, 2009 at 6:19 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Sorry, by "data type" I tried to refer to the nature of the file.  I
> should have said "file type".

My concern is that xml will be focused on and html which is not xml
and yet similar and common will be left out.
-- 
Caleb Cushing

http://xenoterracide.blogspot.com