How to sync changes in GIT to ClearCase or How to get the list of files changed in GIT for a specific branch

How to sync changes in GIT to ClearCase or How to get the list of files changed in GIT for a specific branch

[ask4thunder]

Hi Everyone, 

Thanks for looking into this thread.

This is a help on GIT to ClearCase sync and am using UNIX session and not
Windows

I am intending to use ClearCase in the background and GIT in the front.
For syncing files from ClearCase to GIT i 
checkout the git branch
use rsync command to get the changed files in ClearCase
and then i do the 'git add' and 'git commit'

This works out well, but i want to get a list of files changed in GIt so
that i can bring that to ClearCase and checkout and checkin files.

Can someone provide me the commands to

1. Get list of files changed in GIT, after i do commit?
2. Get list of files changed in GIT after commit, but files changed after a
tag (baseline).
3. Sync these files to ClearCase?


Hope you gurus can help me....am in desperate need of these commands. 
Appreciate your help  in this.

Thanks
ask4thunder

-- 
View this message in context: http://www.nabble.com/How-to-sync-changes-in-GIT-to-ClearCase-or-How-to-get-the-list-of-files-changed-in-GIT-for-a-specific-branch-tp23197522p23197522.html
Sent from the git mailing list archive at Nabble.com.

Re: How to sync changes in GIT to ClearCase or How to get the list of files changed in GIT for a specific branch

[David Aguilar]

On  0, ask4thunder <ask4thunder@gmail.com> wrote:
> 
> I am intending to use ClearCase in the background and GIT in the front.
> For syncing files from ClearCase to GIT i 
> checkout the git branch
> use rsync command to get the changed files in ClearCase
> and then i do the 'git add' and 'git commit'

To handle files that have been removed make sure to use
	rsync --exclude=.git --delete ...

and
	git add -u

to remove any deleted files from git.


> This works out well, but i want to get a list of files changed in GIt so
> that i can bring that to ClearCase and checkout and checkin files.
> 
> Can someone provide me the commands to
> 
> 1. Get list of files changed in GIT, after i do commit?


One possibility:

move to the root of the clearcase project

set GIT_DIR to point to the .git directory
(which, presumably is outside the clearcase project)

and some combination of:
	git diff --name-only
	git diff-index --name-status HEAD

If clearcase wasn't so painful you would ideally just
run 'git checkout -f' to make clearcase match git,
but clearcase is horrible and you have to explicitly
'clearcase checkout' each file.


> 2. Get list of files changed in GIT after commit, but files changed after a
> tag (baseline).

	git diff --name-only <the-commit>^!

	git diff --name-only <the-tag>..HEAD

(or diff-index, etc)


> 3. Sync these files to ClearCase?

	clearcase checkout the <path>
	git checkout HEAD -- <path>

	(assuming you've set GIT_DIR accordingly
	 and are sitting at the root of the project)


> Hope you gurus can help me....am in desperate need of these commands. 
> Appreciate your help  in this.

If you get this stuff working with native clearcase commands,
then others could benefit from your work.

bi-directional git<->cc can be tricky (and importing all of
branches can be even trickier), but even being able to import
one branch is better than nothing, no?

The reason I replied is that I, too, once wrote a similar
tool to suck cc projects into git.  One project
was so huge (and cc sucks so bad) that it
literally took a few weeks for it to import.

Our organization also ran vob backups at 2am everynight,
so I couldn't just let it run continuously since clearcase
would lock the script out at 2am.

It ran incrementally, and lots of other site-specific
hacks.  I don't know clearcase at all (our site used a bunch
of in-house wrapper commands) so I never bothered to make
it work with raw cc commands. That's why I have nothing
to share.  It was also a total hack and once I got the stuff
I wanted out of it, you can bet that I never looked back
(which is why the bi-directional part was not important
 to me at all).


clearcase.. wow <shudders>

All it does is remind me of this horrible, horrible cc mvfs
kernel module bug that was going around deleting automounts...

okay.. enough scary thoughts.


kudos for fighting the good fight,

-- 

	David

Re: How to sync changes in GIT to ClearCase or How to get the list of files changed in GIT for a specific branch

[Peter Baumann]

On Thu, Apr 23, 2009 at 11:59:20AM -0700, ask4thunder wrote:
> 
> Hi Everyone, 
> 
> Thanks for looking into this thread.
> 
> This is a help on GIT to ClearCase sync and am using UNIX session and not
> Windows
> 
> I am intending to use ClearCase in the background and GIT in the front.
> For syncing files from ClearCase to GIT i 
> checkout the git branch
> use rsync command to get the changed files in ClearCase
> and then i do the 'git add' and 'git commit'
> 
> This works out well, but i want to get a list of files changed in GIt so
> that i can bring that to ClearCase and checkout and checkin files.
> 
> Can someone provide me the commands to
> 
> 1. Get list of files changed in GIT, after i do commit?
> 2. Get list of files changed in GIT after commit, but files changed after a
> tag (baseline).
> 3. Sync these files to ClearCase?
> 
> 
> Hope you gurus can help me....am in desperate need of these commands. 
> Appreciate your help  in this.
> 
> Thanks
> ask4thunder
> 

This is what I used to work with clearcase in a former job:

Use cc_import.sh to import your changes and later export your changes
back to CC with export2cc.pl <path_to_CC_repo> <LABEL> <from> <to>

Greetings,
Peter


cc_import.sh: Importing files from the clearcase checkout

#!/bin/sh

# ATTENTION: This will create a branch named 'cc' in your git import
# Make sure that your actual checkout is NEVER on that branch if
# running this script!!!!


###########################
# Adapt this to your needs

BRANCH=refs/heads/cc

export GIT_WORK_TREE="/place/where/your/clearcase/checkout/is"
MY_GIT_IMPORT="/place/to/your/git-import"

###########################



export GIT_INDEX_FILE="${MY_GIT_IMPORT}/.git/index.clearcase"

# save your HEAD
cp .git/HEAD .git/HEAD.local

# switch to the clearcase import branch
git symbolic-ref HEAD "$BRANCH"

git add -u
git add .


# use the date of the checkin as commit message
datum=$(date +'%Y.%m.%d %T')
git commit -m"$datum"

# switch HEAD back to your previous one
cp .git/HEAD.local .git/HEAD


========================

export2cc.pl: exporting git revisions to CC

#!/usr/bin/perl

# TODO's:
#  - The commit msg might be extracted multiple times
#  - it has a hardcoded path name (right now: /d/msg)
#  - There is a proplem adding a deep hierarchy of files
#    (e.g. newdir1/newdir2/newfile1}
#  - No rename support


use File::Basename;
use strict;

my $verbose = 1;
my $upstream;
my $from;

my $cc_repo;
my $CC_LABEL;

$cc_repo = shift @ARGV;
$CC_LABEL = shift @ARGV;

print "CC repo: $cc_repo\n";
print "Label:   $cc_repo\n";

sub run_cmd_pipe {
	if ($^O eq 'MSWin32' || $^O eq 'msys') {
		my @invalid = grep {m/[":*]/} @_;
		die "$^O does not support: @invalid\n" if @invalid;
		my @args = map { m/ /o ? "\"$_\"": $_ } @_;
		return qx{@args};
	} else {
		my $fh = undef;
		open($fh, '-|', @_) or die;
		return <$fh>;
	}
}

if (scalar(@ARGV) >= 1) {
	$upstream = $ARGV[0];

	$from = ((scalar(@ARGV) >= 2) ? "$ARGV[1]" : "HEAD");
}


my @revisions;
push @revisions, "$ARGV[0]";

doit($revisions[0]);

sub doit($) {
	my $rev = shift;

	my @difftree_result = qx(git diff-tree -r --name-status $rev^ $rev -- );

	my @added_files;	# A
	my @copied_files;	# C
	my @deleted_files;	# D
	my @renamed_files;	# R

	my @modified_files;	# M

	foreach (@difftree_result) {
		my ($status, $file) = split /\t/;
		chomp $file;


		if ($status =~ /A/) {
			print "adding file \'$file\'\n" if ($verbose);
			push @added_files, "$file";
		}

		if ($status =~ /C/) {
			print "copied file '$file'\n" if ($verbose);
			push @copied_files, "$file";
		}

		if ($status =~ /D/) {
			print "deleted file '$file'\n" if ($verbose);
			push @deleted_files, "$file";
		}

		if ($status =~ /R/) {
			print "renamed file '$file'\n" if ($verbose);
			push @renamed_files, "$file";
		}

		if ($status =~ /M/) {
			print "modify file '$file'\n" if ($verbose);
			push @modified_files, "$file";
		}
	}

	# sort the files
	@added_files   = sort @added_files;
	@copied_files  = sort @copied_files;
	@deleted_files = sort @deleted_files;
	@renamed_files = sort @renamed_files;
	@modified_files= sort @modified_files;

	my $commit_msg = extract_commitmsg_from_git($rev);
	print $commit_msg;

	foreach (@renamed_files, @copied_files) {
		print "- $_" . "\n";
		die "can't handle renames";
	}


	foreach (@deleted_files) {
		my ($filename, $dirname) = fileparse($_);

		die "The directory $cc_repo/$dirname doesn't exist" if (! -d "$cc_repo/$dirname");

		cc_checkout($cc_repo, $dirname, "") || die "Can't checkout $dirname\n";
		cc_rmname($cc_repo, $_, $commit_msg) || die "Can't delete $_\n";
		cc_checkin($cc_repo, $dirname, "") || die "Can't checkin $dirname\n";

		# lable file and directory
		cc_mklabel($cc_repo, $dirname, $CC_LABEL) || die "Can't label $_\n";
	}
	
	foreach (@added_files) {
		print "N $_" . "\t(add the file manually)\n";
		
		my ($filename, $dirname) = fileparse($_);

		die "The directory $cc_repo/$dirname doesn't exist" if (! -d "$cc_repo/$dirname");

		cc_checkout($cc_repo, $dirname, "") || die "Can't checkout $dirname\n";
		extract_file_from_git($cc_repo, $_, $rev);
		cc_mkelem($cc_repo, $_, $commit_msg);
		cc_checkin($cc_repo, $_, "") || die "Can't checkin $_\n";
		cc_checkin($cc_repo, $dirname, "") || die "Can't checkin $dirname\n";

		# lable file and directory
		cc_mklabel($cc_repo, $_, $CC_LABEL) || die "Can't label $_\n";
		cc_mklabel($cc_repo, $dirname, $CC_LABEL) || die "Can't label $_\n";
	}

	foreach (@modified_files) {
		print "M $_" . "\n";
		cc_checkout($cc_repo, $_, $commit_msg) || die "Can't checkout $_\n";
		extract_file_from_git($cc_repo, $_, $rev);
		cc_checkin($cc_repo, $_, $commit_msg) || die "Can't checkin $_\n";
		cc_mklabel($cc_repo, $_, $CC_LABEL);
	}
}



sub cc_checkout ($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $result;
	my $errcode = 0;

	if ($commit_msg eq "") {
		$result = qx(cleartool co -res -ncomment "$path_to_repo/$file");
		$errcode = ($? >> 8);
	} else {
		my $msg_file = "/d/msg.txt";
		open(MSGFILE, ">$msg_file")
			or die "can't export the commit msg";
	
		print MSGFILE $commit_msg;
		close(MSGFILE);
	
		#print "cleartool co -res -unr -cfi $msg_file $path_to_repo/$file\n";
		$result = qx(cleartool co -res -unr -cfi $msg_file "$path_to_repo/$file");
		$errcode = ($? >> 8);
	
		qx(rm $msg_file);
	}

	print $result;
	return !$errcode;
}

sub cc_checkin ($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $result;
	my $errcode = 0;

	if ($commit_msg eq "") {
		$result = qx(cleartool ci -nc "$path_to_repo/$file");
		$errcode = ($? >> 8);
	} else {
		my $msg_file = "/d/msg.txt";
		open(MSGFILE, ">$msg_file")
			or die "can't export the commit msg";

		print MSGFILE $commit_msg;
		close(MSGFILE);

		$result = qx(cleartool ci -cfi $msg_file "$path_to_repo/$file");
		$errcode = ($? >> 8);

		qx(rm $msg_file);
	}

	print $result;

	return !$errcode;
}

##
# Add a new file/directory to clearcase
#
# @preconditon: the underlying directory is already checked out
sub cc_mkelem($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $msg_file = "/d/msg.txt";
	open(MSGFILE, ">$msg_file")
		or die "can't export the commit msg";

	print MSGFILE $commit_msg;
	close(MSGFILE);

	my $result = qx(cleartool mkelem -cfi $msg_file "$path_to_repo/$file");
	die "mkelem $path_to_repo/$file failed" if ($?);
	
	print $result;

	qx(rm $msg_file);
}

##
# Delete a file from clearcase
#
# @precondition: the underlying directory is already checked out
sub cc_rmname($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $commit_msg = shift;

	my $errcode = 0;

	my $msg_file = "/d/msg.txt";
	open(MSGFILE, ">$msg_file")
		or die "can't export the commit msg";

	print MSGFILE $commit_msg;
	close(MSGFILE);

	my $result = qx(cleartool rmname -f -cfi $msg_file "$path_to_repo/$file");
	die "rmname $path_to_repo/$file failed" if ($?);
	$errcode = ($? >> 8);
	
	print $result;

	qx(rm $msg_file);
	return !$errcode;
}

##
#
sub cc_mklabel($$$) {
	my $path_to_repo = shift;
	my $file = shift;
	my $label = shift;
	
	my $result = qx(cleartool mklabel -replace "$label" "$path_to_repo/$file");
	die "mklabel $path_to_repo/$file failed" if ($?);
	print $result;
}

sub extract_file_from_git($$$) {
	my $cc_repo_path = shift;
	my $file = shift;
	my $rev = shift;

	my $result = qx(git show $rev:$file > $cc_repo_path/$file);
	die "extract_file_from_git $cc_repo_path/$file failed" if ($?);
}

sub extract_commitmsg_from_git($) {
	my $revision = shift;

	my $result = qx(git log -1 --pretty=format:'%s%n%b' $revision --);
	die "extract_commitmsg_from_git $revision failed" if ($?);
	return $result;
}

Re: How to sync changes in GIT to ClearCase or How to get the list of files changed in GIT for a specific branch

[Charles O'Farrell]

A Clearcase/Git bridge I wrote for my current job. It also pulls in history as
best it can.

http://github.com/charleso/git-cc/tree/master

Charles

Re: How to sync changes in GIT to ClearCase or How to get the list of files changed in GIT for a specific branch

[Chris Packham]

On Fri, May 8, 2009 at 7:04 AM, Senthil Kumar <ask4thunder@gmail.com> wrote:
> Hey Chris,
>
> First thanks for giving me the knowledge transfer and i really appreciate
> it. Am working on the sync stuff now, but i have an issue in a perl script
> in post-update hook.
>
>  Have a doubt here, am writing a post-update hook in perl, so that whenever
> a user pushes code from their local repo to the bare public repo, the
> post-update hook triggers so as to pull changes from the abre public repo to
> the actual parent data repo.
>
> My data repo: /snaps4/searumugam/spec-repo/mydodo.repo
> My bare repo: /snaps4/searumugam/spec-repo/mydodo.bare
> My local repo: /snaps4/searumugam/githome/searumugam_mydodo
>
> here is what i have in the script (left),  and the output i get is in blue
> (right)
>
> The concept: First when a user pushes to bare repo -
> /snaps4/searumugam/spec-repo/mydodo.bare, the post-update trigger starts, it
> shd cd into the data repo path  (/snaps4/searumugam/spec-repo/mydodo.repo)
> and initiate a 'git pull'
>
> The $data path has the correct path - my data repo path, where the pull will
> be initiated,
>
> print "\nData Path: $data_path\n";                 Data Path:
> /snaps4/searumugam/spec-repo/mydodo.repo/
> chdir ($data_path);
> system("pwd");
> /snaps4/searumugam/spec-repo/mydodo.repo
> system("git pull");                                      fatal: Not a git
> repository: '.'
> system("pwd");
> /snaps4/searumugam/spec-repo/mydodo.repo
>
> i even tried with system("/usr/libexec/git-core/git-pull"); but its saying
> the same error. But when i do it manually in the same direcotory it works
>
> Manual pull, when am in the same directory.
> rndhyper1:/snaps4/searumugam/spec-repo/mydodo.repo 30> pwd
> /snaps4/searumugam/spec-repo/mydodo.repo
> rndhyper1:/snaps4/searumugam/spec-repo/mydodo.repo 31> git pull
> remote: Counting objects: 5, done.
> remote: Compressing objects: 100% (4/4), done.
> remote: Total 4 (delta 2), reused 0 (delta 0)
> Unpacking objects: 100% (4/4), done.
> From /snaps4/searumugam/spec-repo/mydodo.bare
>    cb9f659..2c5bca8  master     -> origin/master
> Updating cb9f659..2c5bca8
> Fast forward
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 final111223
>  create mode 100644 final1112233
>
> Am i doing something wrong, or should i execute 'git pull' in a different
> manner?
> hope you could help mein this.
>
> Thanks & Regards
> Senthil A Kumar
>

Like I say not exactly an expert. But it looks like when run in this
manner 'git pull' is unable to figure out the location of the .git
directory. Try something like.

system("GIT_DIR=$data_path.git; git pull");

There were some other replies to your original post that promising for
the cc-to-git-and-back aspect. Looking into them may save you a lot of
time.

Good luck.