Merging Repositories¶
Question: Can we merge a repository into another without losing history? For merging two existing repositories a lot can be found on the internet. Eventually I merged two found methods into one method. The second part of the first method did not result in the wanted behavior. For that the second method was used.
The found methods:
Tip
Within the following steps repository A will be merged into repository B. It is wise to create a temporary directory in which both repositories will be cloned, so when a certain step does not have the wanted outcome it will be fairly easy to start all over again.
Note
Most of the steps below are shown from a script used to execute the merge between two repositories, see merge_git_repo_a_to repo_b
.
Steps¶
Create temporary directory:
mkdir $HOME/tmp/merge_git_repos
cd $HOME/tmp/merge_git_repos
Clone repository A into a temporary name tmp_repo_a:
git clone <url of repo a> tmp_repo_a
Clone repository B into a temporary name tmp_repo_b:
git clone <url of repo b> tmp_repo_b
Execute script for merging repo a into repo b:
merge_git_repo_a_to_repo_b
Note
The script can also be used with the locations of the two repositories (e.g. merge_git_repo_a_to_repo_b <location repo a> <location repo b>. If the location are omitted the defaults tmp_repo_a and tmp_repo_b are used.
Executed steps from the shell script:
Checkout all the branches you want to copy from repository A:
cd $REPO_A_PATH
git branch -a > $MERGE_PATH/repo_a_branches.txt
for branch in `cat $MERGE_PATH/repo_a_branches.txt`
do
git checkout $branch
done
Fetch all the tags from repository A:
git fetch --tags
Check if you have all the tags and branches:
git tag
git branch -a
Clear the link to the remote of repository A:
git remote rm origin
Do some filtering for rewriting the history:
git filter-branch -f --tree-filter 'mkdir -p toDelete;git mv -k docs/Makefile toDelete/.;git mv -k docs/_external_templates/conf/* toDelete/.;git mv -k docs/_external_templates/static/* toDelete/.;git mv -k docs/requirements.txt toDelete/.;git mv -k docs/doc/_generic.inc toDelete/.;git mv -k docs/doc/conf.py toDelete/.;git mv -k docs/doc/index.rst toDelete/docs_index.rst;git mv -k README.md toDelete/.' HEAD
git filter-branch -f --tree-filter 'mkdir -p docs/doc/Teams/0.Talk_like_pi;git mv -k docs/doc/teams/2018.1_Talk_like_pi/* docs/doc/Teams/0.Talk_like_pi;git mv -k docs/doc/teams/index.rst toDelete/teams_index.rst' HEAD
git filter-branch -f --tree-filter 'mkdir -p docs/doc/Demonstrators;git mv -k docs/doc/IPSC/* docs/doc/Demonstrators/.' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/MQTT/Bandwidth/Python;mkdir -p docs/doc/Demonstrators/ComparedToOtherProtocols/Bandwidth/MQTT;git mv -k IPSC/MQTT/py/Bandwidth/* src/demonstrators/ComparedToOtherProtocols/MQTT/Bandwidth/Python; git mv -k src/demonstrators/ComparedToOtherProtocols/MQTT/Bandwidth/Python/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/Bandwidth/MQTT/index.rst;git mv -k src/demonstrators/ComparedToOtherProtocols/MQTT/Bandwidth/Python/ipc/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/Bandwidth/MQTT/ipc.rst;git mv -k src/demonstrators/ComparedToOtherProtocols/MQTT/Bandwidth/Python/isc/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/Bandwidth/MQTT/isc.rst' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/MQTT/RoundTrip/Python;mkdir -p docs/doc/Demonstrators/ComparedToOtherProtocols/RoundTrip/MQTT;git mv -k IPSC/MQTT/py/RoundTrip/* src/demonstrators/ComparedToOtherProtocols/MQTT/RoundTrip/Python;git mv -k src/demonstrators/ComparedToOtherProtocols/MQTT/RoundTrip/Python/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/RoundTrip/MQTT/index.rst;git mv -k src/demonstrators/ComparedToOtherProtocols/MQTT/RoundTrip/Python/ipc/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/RoundTrip/MQTT/ipc.rst;git mv -k src/demonstrators/ComparedToOtherProtocols/MQTT/RoundTrip/Python/isc/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/RoundTrip/MQTT/isc.rst' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/MQTT/PingPong/Python;mkdir -p docs/doc/Demonstrators/ComparedToOtherProtocols/PingPong/MQTT;git mv -k IPSC/MQTT/py/* src/demonstrators/ComparedToOtherProtocols/MQTT/PingPong/Python;git mv -k src/demonstrators/ComparedToOtherProtocols/MQTT/PingPong/Python/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/PingPong/MQTT/index.rst' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/ZMQ/Bandwidth/Python;mkdir -p docs/doc/Demonstrators/ComparedToOtherProtocols/Bandwidth/ZMQ;git mv -k IPSC/ZMQ/py/Bandwidth/* src/demonstrators/ComparedToOtherProtocols/ZMQ/Bandwidth/Python;git mv -k src/demonstrators/ComparedToOtherProtocols/ZMQ/Bandwidth/Python/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/Bandwidth/ZMQ/index.rst' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/ZMQ/RoundTrip/Python;git mv -k IPSC/ZMQ/py/RoundTrip/* src/demonstrators/ComparedToOtherProtocols/ZMQ/RoundTrip/Python' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/ZMQ/Push-Pull/Python;git mv -k IPSC/ZMQ/py/Push-Pull/* src/demonstrators/ComparedToOtherProtocols/ZMQ/Push-Pull/Python' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/ZMQ/Request-Reply/PingPong/Python;git mv -k IPSC/ZMQ/py/Request-Reply/PingPong/* src/demonstrators/ComparedToOtherProtocols/ZMQ/Request-Reply/PingPong/Python' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/ZMQ/Request-Reply/RoundTrip/Python;git mv -k IPSC/ZMQ/py/Request-Reply/Roundtrip/* src/demonstrators/ComparedToOtherProtocols/ZMQ/Request-Reply/RoundTrip/Python' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/ComparedToOtherProtocols/ZMQ/PingPong/Python;mkdir -p docs/doc/Demonstrators/ComparedToOtherProtocols/PingPong/ZMQ;git mv -k IPSC/ZMQ/py/* src/demonstrators/ComparedToOtherProtocols/ZMQ/PingPong/Python;git mv -k src/demonstrators/ComparedToOtherProtocols/ZMQ/PingPong/Python/readme.rst docs/doc/Demonstrators/ComparedToOtherProtocols/PingPong/ZMQ/index.rst' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/PingPong/Python;mkdir -p docs/doc/Demonstrators/PingPong;git mv -k IPSC/DDS/CycloneDDS/py/PingPong/* src/demonstrators/PingPong/Python;git mv -k src/demonstrators/PingPong/Python/readme.rst docs/doc/Demonstrators/PingPong/index.rst' HEAD
git filter-branch -f --tree-filter 'mkdir -p src/demonstrators/RoundTrip/Python;mkdir -p docs/doc/Demonstrators/RoundTrip;git mv -k IPSC/DDS/CycloneDDS/py/Loop/* src/demonstrators/RoundTrip/Python;git mv -k src/demonstrators/RoundTrip/Python/readme.rst docs/doc/Demonstrators/RoundTrip/index.rst' HEAD
git filter-branch -f --index-filter 'git rm -r --cached --ignore-unmatch toDelete' HEAD
Tip
The index-filter takes less time in comparison to the tree-filter and can be used for removing files/directories. Also sometimes removing files/directories using the tree-filter does not always rewrite history.
For changing the directory structure the following commands can be used within a tree-filter
> mkdir -p <new directory>;git mv -k <file to move> <new directory>
For removing a file/directory the following command can be used within a index-filter
> git rm -r –cached –ignore-unmatch <file or directory>
Note
The order of the actions within the commands or executed filtering is very important. It is best to first move sources and finally removing files/directories, since the second filtering must use option -f (force) to be executed.
Removing garbage from filtering:
git reset --hard
git gc --aggressive
git prune
git clean -fd
Perfrom any necessary changes and commit these:
git status
git add .
git commit
Create a new branch in repository B:
cd $REPO_B_PATH
# create a new branch
git checkout -b feature/merge_git_repos
Create a remote connection to repository A as a branch in repository B:
git remote add repo-a $REPO_A_PATH
Note
repo-a can be anything - it’s just a random name
Pull files and history from branches into repository B:
git pull repo-a master --allow-unrelated-histories
Remove the remote connection to repository A:
git remote rm repo-a
Finally, push the changes:
git push origin <new branch name>