Updated on | #git, #gitolite, #sparkleshare | 0 User comments

Gitolite with Sparkleshare using Zentyal - Dropbox alternative

Distributed organizations find file sharing between offices challenging using traditional methods like Samba4. Modern infrastructures like Dropbox or Ubuntu One are providing the ability to store, share, access and backup files using an online fileserver (called: Cloud) that is automatically synchronized with an local storage. This Tutorial will show you how to install, configure and manage your own Cloud by using Gitolite, SparkleShare and Zentyal.

Gitolite

Create a user called gitolite

  1. sudo adduser \
  2.   --system \
  3.   --shell /bin/bash \
  4.   --gecos 'git version control' \
  5.   --group \
  6.   --disabled-password \
  7.   --home /home/gitolite \
  8.   gitolite

After the user is created, you have to add an admin (you ;-)) for Gitolite. Therefore, please use your Client-PC shell (not the Gitolite Server!!!) and generate an SSH-Key:

  1. cd ~/.ssh
  2. ssh-keygen -t rsa -f admin@gitserver
  3. chmod 700 admin@gitserver

Add the following enty to your ssh config

  1. nano ~/.ssh/config
  1. host gitserver
  2.         hostname {DOMAIN}.{TLD}
  3.         StrictHostKeyChecking no
  4.         User gitolite
  5.         Port 22
  6.         IdentityFile ~/.ssh/admin@gitserver.pub

Copy the public key to your Gitolite Server:

  1. scp admin@gitserver.pub admin@{DOMAIN}.{TLD}:/tmp/admin@gitserver.pub

And move the public key to your Gitolite home directory on the Gitolite Server:

  1. sudo mv /tmp/admin@gitserver.pub /home/gitolite/.ssh/admin@gitserver.pub
  2. sudo chown gitolite:gitolite /home/gitolite/.ssh/admin@gitserver.pub
  3. sudo chmod 0600 /home/gitolite/.ssh/admin@gitserver.pub

Install git on your Server

  1. sudo apt-get install git git-doc

Gitolite Version 2

  1. sudo apt-get install gitolite

Next step is to switch from admin user to gitolite

  1. sudo su gitolite

and to register your admin public key:

  1. gl-setup /home/gitolite/.ssh/admin@gitserver.pub

Gitolite Version 3

  1. mkdir /home/gitolite/bin && cd /tmp
  2. git clone git://github.com/sitaramc/gitolite
  3. gitolite/install -to /home/gitolite/bin
  4. sudo chown -R gitolite:gitolite /home/gitolite/

Next step is to switch from admin user to gitolite

  1. sudo su gitolite

Add the ~/bin directory to your path. Therefore, edit .profile in the gitolite home directory

  1. nano ~/.profile

and add the following at the end of the file:

  1. PATH="$HOME/bin:$PATH"

and register your admin public key in gitolite:

  1. /home/gitolite/bin/gitolite setup -pk /home/gitolite/.ssh/admin@gitserver.pub

If this step was sucessfully, you will get the following message:

  1. NOTE: the admin username is 'gitolite

Configure Gitolite

Now you are able to test the connection from your Client-PC:

  1. ssh gitserver

The answer should be look like this:

PTY allocation request failed on channel 0
hello gitolite, this is gitolite@{DOMAIN}.{TLD} running gitolite3 v3.5.3-0-gaae4162 on git 1.8.4

 R W    testing
Connection to {DOMAIN}.{TLD} closed.

You can manage very easly the new Gitolite Server by using gitolite-admin. In this case, you have to pull the gitolite-admin directory to your Client-PC:

  1. cd /tmp
  2. git clone gitolite@gitserver:gitolite-admin
  3. cd gitolite-admin

Copy your public key inside the keydir and delete the existing admin.pub. This helps to identify later your admin key.

  1. cp ~/.ssh/admin@gitserver.pub /tmp/gitolite-admin/keydir/
  2. rm /tmp/gitolite-admin/keydir/admin.pub

The config file /tmp/gitolite-admin/conf/gitolite.conf should be like this:

  1. repo gitolite-admin
  2.     RW+     =   admin
  3.  
  4. repo testing
  5.     RW+     =   @all

Change this file to your needs and upload it to the Gitolite Server:

  1. cd /tmp/gitolite-admin
  2.  
  3. git add -A
  4. git commit -m 'New config.'
  5. git push origin master

For GIT Admins

Do NOT add new repos or users manually on the server. Gitolite users, repos, and access rules are maintained by making changes to a special repo called 'gitolite-admin' and pushing those changes to the server. To administer your gitolite installation, start by doing this on your workstation:

  1. cd /tmp
  2.     git clone --bare gitolite@{DOMAIN}.{TLD}:gitolite-admin
  3.     cd gitolite-admin
  4.     git add keydir/*;
  5.     git push -am "Added User XYZ";
  6.     git push

Now if you cd gitolite-admin, you will see two sub-directories in it: conf and keydir.

Adding a new user

Copy the user key file into the /tmp/gitolite-admin/keydir directory and do a push to your Gitolite Server

  1. git config --bool core.bare false
  2.     git config receive.denycurrentbranch ignore
  3.     git add .
  4.     git commit -m 'User XYZ added'
  5.     git push -f
  6.     git reset --hard HEAD
  7.     git checkout -f

Adding a new Repository

Modify /tmp/gitolite-admin/conf/gitolite.conf in order to add or to set permissions for an Repo.

  1. @repogroup = thomas gero
  2.  
  3.     repo    gitolite-admin
  4.             RW+     =   roger
  5.  
  6.     repo    testing
  7.             RW+     =   @repogroup
  8.  
  9.     repo    public
  10.         RW+ =   @all

After you have changed the file you need to commit and publish it to the server.

  1. git add gitolite.conf
  2.     git commit -m "gitolite Update"
  3.     git push

Gitolite and LDAP

Gitolite Version 2

Go to /usr/share/gitolite and create a file called ldap-query-groups

  1. sudo nano /usr/share/gitolite/ldap-query-groups

Insert the ldap-query-groups content below.

Gitolite Version 3

Go to /home/gitolite/bin and create a file called ldap-query-groups

  1. sudo nano /home/gitolite/bin/ldap-query-groups

Insert the ldap-query-groups content below.

ldap-query-groups

Insert the following syntax and please have a look at ldap_bindpw as well as dc={DOMAIN},dc={TLD}

  1. #!/bin/sh
  2. #
  3. # Copyright (c) 2010 Nokia Corporation
  4. # Modified by MoonMaker 2013
  5. #
  6. # This code is licensed to you under MIT-style license. License text for that
  7. # MIT-style license is as follows:
  8. #
  9. # Permission is hereby granted, free of charge, to any person obtaining a copy
  10. # of this software and associated documentation files (the "Software"), to deal
  11. # in the Software without restriction, including without limitation the rights
  12. # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  13. # copies of the Software, and to permit persons to whom the Software is
  14. # furnished to do so, subject to the following conditions:
  15. #
  16. # The above copyright notice and this permission notice shall be included in
  17. # all copies or substantial portions of the Software.
  18. #
  19. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  20. # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  21. # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  22. # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  23. # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  24. # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  25. # THE SOFTWARE.
  26. #
  27. # ldap-query-group.sh <arg1>
  28. #
  29. # this script is used to perform ldap querys by giving one argument:
  30. # - <arg1> the user UID for ldap search query
  31. #
  32. # NOTICE: This script requires ldap-utils to be installed to the system.
  33. #
  34. # Script requires user UID as the only parameter
  35. #
  36.  
  37. # Do you want to debug? (log=0 - No; log=1 - yes)
  38. log=1
  39.  
  40. # Choose the standard group: User has to be a member in order to connect to gitolite
  41. # If the User is a member of this group, we can look for all other gitolite groups
  42. gitolite_group="memberof=cn=gitolite,ou=Groups,dc={DOMAIN},dc={TLD}"
  43.  
  44. # Set needed LDAP search tool options for the query
  45. ldap_host="localhost"
  46. ldap_port=390
  47. ldap_binddn="cn=zentyalro,dc={DOMAIN},dc={TLD}"
  48. ldap_bindpw="{TLD}"
  49. ldap_searchbase="ou=Users,dc={DOMAIN},dc={TLD}"
  50. ldap_scope="subtree"
  51.  
  52. ##########################################################################################################################
  53.  
  54. if [ $# -ne 1 ]
  55. then
  56.         echo "ldap-query-group.sh requires one argument, user's email address"
  57.         exit 1
  58. fi
  59.  
  60. # Check if the argument is for Gitolite 2 or 3
  61. search="uid=${1}"               # Gitolite 2 - Username
  62. if echo "${1}" | egrep -q "@" ; then
  63.         search="mail=${1}"      # Gitolite 3 - Email
  64. fi
  65.  
  66. # Construct the search filter for the LDAP query for the given search term
  67. # User is activated and is in the gitolite group
  68. # Optimized for Zentyal
  69. ldap_filter="(&(objectClass=posixAccount)(objectClass=shadowAccount)(${gitolite_group})(!(givenName=-))(!(uidNumber=0))(!(uid=*$))(!(shadowExpire=0))(&(${search}))) memberOf"
  70.  
  71. # Construct the command line base with needed options for the LDAP query
  72. ldap_options="-h ${ldap_host} -p ${ldap_port} -x -D ${ldap_binddn} -w ${ldap_bindpw} -b ${ldap_searchbase} -s ${ldap_scope}"
  73.  
  74. # Execute the actual LDAP search to get groups for the given UID
  75. ldap_result=$(ldapsearch ${ldap_options} -LLL ${ldap_filter})
  76.  
  77. # Filter Groups
  78. result=''
  79. arr=$(echo "${ldap_result}")
  80. # | tr ":" "\n")
  81. for x in ${arr}
  82. do
  83.         if echo "${x}" | egrep -q ",ou=Groups," ; then
  84.                 sub=`expr match "${x}" '.*cn=\([^,]*\),.*' `
  85.                 result="${result} ${sub} "
  86.         fi
  87. done
  88.  
  89. # Logging
  90. if [ $log -eq 1 ]; then
  91.         LOGTIME=`date "+%Y-%m-%d %H:%M:%S"`
  92.         echo "${LOGTIME} | ${search} | ${result}" >> /var/log/gitolite_ldap-query-group.log
  93. fi
  94.  
  95. # Output
  96. echo $result

Save it and test it

  1. sudo chmod 0700 ldap-query-groups
  2. ./ldap-query-groups admin

You should get a list like this:

  1. __USERS__ gitolite gitolite_testing

After this you can change the /home/gitolite/.gitolite.rc file:

  1. sudo nano /home/gitolite/.gitolite.rc
Gitolite Version 2

Find $GL_GET_MEMBERSHIPS_PGM and change it to:

  1. $GL_GET_MEMBERSHIPS_PGM = "/usr/share/gitolite/ldap-query-groups";
Gitolite Version 3

Find UMASK and add after this entry:

  1. GROUPLIST_PGM => "/home/gitolite/bin/ldap-query-groups",
Create a log file
  1. sudo touch /var/log/gitolite_ldap-query-group.log
  2. sudo chmod 744 /var/log/gitolite_ldap-query-group.log
  3. sudo chown gitolite:gitolite /var/log/gitolite_ldap-query-group.log

Gitolite with git-annex

  1. sudo add-apt-repository ppa:fmarier/git-annex
  2. sudo apt-get update && sudo apt-get install git-annex

Gitweb

  1. sudo apt-get install highlight gitweb

Open the configuration file

  1. sudo nano /etc/gitweb.conf

and modify the following entries:

change $projectroot to $ENV{'GITWEB_PROJECTROOT'}; change $projects_list to $ENV{'GITWEB_PROJECTS_LIST'}; insert @git_base_url_list = qw(ssh://gitolite@{DOMAIN}.{TLD});

Modify your /home/gitolite/.gitolite.rc

change UMASK => 0027,

Apache2

In order to secure your server and to make the url's more pretty, please create the following file:

  1. sudo nano /etc/apache2/conf.d/gitweb.conf

And insert the following content:

  1. <Directory /usr/share/gitweb>
  2.         Options FollowSymLinks ExecCGI
  3.  
  4.         Allow from all
  5.         AllowOverride none
  6.         Order allow,deny
  7.  
  8.         SetEnv  GITWEB_CONFIG           /etc/gitweb.conf
  9.         SetEnv  GITWEB_PROJECTROOT      /home/gitolite/repositories
  10.         SetEnv  GITWEB_PROJECTS_LIST    /home/gitolite/projects.list
  11.  
  12.         <Files gitweb.cgi>
  13.                 AddHandler cgi-script cgi
  14.                 SetHandler cgi-script
  15.         </Files>
  16.  
  17.         <IfModule mod_rewrite.c>
  18.                 RewriteEngine on
  19.                 RewriteCond %{REQUEST_FILENAME} !-f
  20.                 RewriteCond %{REQUEST_FILENAME} !-d
  21.                 RewriteRule ^(gitweb/)(.*)$ /gitweb.cgi/$0 [L,PT]
  22.         </IfModule>
  23.  
  24. </Directory>
  25.  
  26. <Directory /home/gitolite/repositories>
  27.         Allow from all
  28.         AllowOverride none
  29. </Directory>

Next to this file, you have to create a server file. E.g. /etc/apache2/sites-enabled/gitweb

  1. Alias /gitweb /usr/share/gitweb

After this activate rewrite mod and restart the Apache Server

  1. sudo a2enmod rewrite
  2. sudo service apache2 restart

Gitweb configuration

  1. sudo nano /etc/gitweb.conf

You are free to configure gitweb for yourself. I add the following options at the of the file:

  1. ####### My Config #######
  2. $feature{'pathinfo'}{'default'} = [1];
  3.  
  4. $feature{'blame'}{'default'} = [1];
  5. $feature{'blame'}{'override'} = 1;
  6.  
  7. $feature{'pickaxe'}{'default'} = [1];
  8. $feature{'pickaxe'}{'override'} = 1;
  9.  
  10. $feature{'snapshot'}{'default'} = [1];
  11. $feature{'snapshot'}{'override'} = 1;
  12.  
  13. $feature{'search'}{'default'} = [1];
  14.  
  15. $feature{'grep'}{'default'} = [1];
  16. $feature{'grep'}{'override'} = 1;
  17.  
  18. $feature{'show-sizes'}{'default'} = [1];
  19. $feature{'show-sizes'}{'override'} = 1;
  20.  
  21. $feature{'avatar'}{'default'} = ['gravatar'];
  22. $feature{'avatar'}{'override'} = 1;
  23.  
  24. $feature{'highlight'}{'default'} = [1];
  25. $feature{'highlight'}{'override'} = 1;

Custom Theme

You can also use a custom theme. E.g. from Stefan Imhoff

  1. sudo mv /usr/share/gitweb/static/gitweb.js /usr/share/gitweb/static/gitweb.js.orig
  2. sudo mv /usr/share/gitweb/static/gitweb.css /usr/share/gitweb/static/gitweb.css.orig
  3.  
  4. cd /tmp
  5. git clone git://github.com/kogakure/gitweb-theme.git
  6.  
  7. cd gitweb-theme
  8. sudo cp gitweb.css gitweb.js git-logo.png git-favicon.png /usr/share/gitweb/static

Uncomment the following options in /etc/gitweb.conf

  1. # stylesheet to use
  2. @stylesheets = ("static/gitweb.css");
  3.  
  4. # javascript code for gitweb
  5. $javascript = "static/gitweb.js";
  6.  
  7. # logo to use
  8. $logo = "static/git-logo.png";
  9.  
  10. # the 'favicon'
  11. $favicon = "static/git-favicon.png";

SparkleShare

At first you need the latest SparkleShare Version. You can download the Software from SparkleShare.org. In case of Ubuntu/Debian etc. you can use the Launchpad Repository:

  1. sudo add-apt-repository ppa:rebuntu16/sparkleshare+unofficial
  2. sudo apt-get update && sudo apt-get install sparkleshare

Generate an SSH-Key

Generate your SSH-Key. Please change USER@PLACE (e.g. MAX@OFFICE):

  1. cd ~/.config/sparkleshare
  2. ssh-keygen -t rsa -f ~/.config/sparkleshare/sparkleshare.USER@PLACE.key
  3. chmod 700 ~/.config/sparkleshare/sparkleshare.USER@PLACE.key

Your public key: \~/.config/sparkleshare/sparkleshare.USER@PLACE.key.pub Your private key: \~/.config/sparkleshare/sparkleshare.USER@PLACE.key

Send your public key (\~/.config/sparkleshare/sparkleshare.USER@PLACE.key.pub) to the GIT Administrator. The Admin will insert it into Gitolite and he/she will set the permissions for you.

Register your generated SSH-Key to your SSH-Client

Please change {DOMAIN}.{TLD} as well as USER@PLACE:

  1. echo " Host {DOMAIN}.{TLD}
  2.    hostname {DOMAIN}.{TLD}
  3.    StrictHostKeyChecking no
  4.    User gitolite
  5.    IdentityFile ~/.config/sparkleshare/sparkleshare.USER@PLACE.key
  6.    " >> .ssh/config

Test your Account

There should be no password question when you connect to gitolite ssh-account:

  1. ssh -T gitolite@{DOMAIN}.{TLD}

And it should be written: > PTY allocation request failed on channel 0 > hello thomas, this is gitolite 2.3-1 (Debian) running on git 1.8.1.rc0 > the gitolite config gives you the following access: > @R_ @W_ testing > Connection to {DOMAIN}.{TLD} closed.

Add your Server to the default config

Create a file in your SparkleShare Plugin directory

  1. nano ~/.config/sparkleshare/plugins/{DOMAIN}.{TLD}.xml

and insert the following content:

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <sparkleshare>
  3.         <plugin>
  4.                 <info>
  5.                         <name>{DOMAIN}'s GitServer</name>
  6.                         <description>ssh://gitolite@{DOMAIN}.{TLD}</description>
  7.                         <icon>own-server.png</icon>
  8.                         <backend>Git</backend>
  9.                 </info>
  10.                 <address>
  11.                         <value>ssh://gitolite@{DOMAIN}.{TLD}</value>
  12.                 </address>
  13.                 <path>
  14.                         <value></value>
  15.                         <example>/project_name</example>
  16.                 </path>
  17.         </plugin>
  18. </sparkleshare>

You can download my plugin config here: LudwigHub

After restarting SparkleShare with

  1. sparkleshare restart

you can add a hosted project on your server very easily.

Fanout - Notification Server

SparkleShare uses a small script that handles update notifications between clients. By default it uses the one running on notifications.sparkleshare.org. The only information sent to this service is a hash of a folder identifier and a hash of the current revision. The service then tells the other connected clients that are subscribed to a folder that they can pull new changes from wherever your repository is hosted. This allows SparkleShare clients to sync new changes instantly, instead of polling with potentially long delays (up to 10 minutes). Source

You can download Travis Glenn Hansen's simple fanout pubsub message server in order to compile it:

  1. cd /opt
  2. git clone https://github.com/travisghansen/fanout.git
  3. cd /opt/fanout
  4. make

After compiling successfully this service, you can start to create the symlinks:

  1. ln -s /opt/fanout/fanout /usr/bin/sparkleshare_fanout
  2. ln -s /opt/fanout/debian/fanout.init /etc/init.d/sparkleshare_fanout
  3. ln -s /opt/fanout/debian/fanout.default /etc/default/sparkleshare_fanout
  4.  
  5. chmod +x /etc/init.d/sparkleshare_fanout
  6. chmod +x /usr/bin/sparkleshare_fanout

Change the 'SERVICE' variable in the /etc/init.d/sparkleshare_fanout script to

  1. SERVICE=sparkleshare_fanout

and change the 'default' parameters in /etc/default/sparkleshare_fanout to

  1. FANOUT_OPTS="--daemon --pidfile=/var/run/sparkleshare_fanout.pid --run-as=gitolite:gitolite --logfile=/var/log/sparkleshare_fanout.log --debug-level=0 --port=1986"

Now you are able to create the log file with:

  1. touch /var/log/sparkleshare_fanout.log
  2. chmod 666 /var/log/sparkleshare_fanout.log

Next to this, your have to create the scripts for start up as well as when the runlevel is changing:

  1. sudo update-rc.d sparkleshare_fanout defaults

If you finished all these steps successfully, you can start your own notification server with

  1. sudo service sparkleshare_fanout start

Last but not least, don't forget to configure your firewall and your router for Port 1986 (TCP).

SparkleShare

In order to configure the SparkleShare Server for the new Notification Server, you have to edit the config.xml file: * Windows:

%USERPROFILE%\AppData\Roaming\sparkleshare\config.xml
  • Ubuntu:
~/.config/sparkleshare/config.xml
  1. <sparkleshare>
  2.   // Use your personal notification service globally...
  3.   <announcements_url>tcp://{DOMAIN}.{TLD}:1986</announcements_url>
  4.  
  5.   // ...or for a single folder
  6.   <folder>
  7.     <name>Stuff</name>
  8.     <announcements_url>tcp://{DOMAIN}.{TLD}:1986</announcements_url>
  9.   </folder>
  10. </sparkleshare>

Gitolite

This Git hook sends update messages to a notification server, so that manual git pushes to a repository also will be noticed by SparkleShare clients. Source

  1. sudo su gitolite
  2. cd ~/.gitolite/hooks/common
  3. wget https://raw2.github.com/hbons/sparkleshare-git-hook/master/post-update
  4. chmod +x post-update
  5. chown gitolite:gitolite post-update

After this, you have to edit ~/.gitolite/hooks/common/post-update, in order to configure your Server:

  1. SERVER="{DOMAIN}.{TLD}"
  2. PORT="1986"

If you are done, you have to run gitolite setup again with: * Gitolite 2

  1. gl-setup
  • Gitolite 3
  1. /home/gitolite/bin/gitolite setup

Well, what do you think?

Comments powered by LudwigDisqus for ModX