GitLab + Redmine + WordPress + OpenLDAP in 5 Minutes (or so)

Someone wants to deploy a project management infrastructure on premise, luckily there's a large choice of open-source solutions out there: GitLab for version control and development, Redmine for team communication, WordPress for the website, OpenLDAP for authentication, etc. However, the cost of installing, configuring and managing all of those by himself didn't really seem worth it. Or is it?

Shasoco takes care of everything: easy installation, updates, backups, migration. With shasoco, you'll get a website with wordpress, an ldap server, a directory, gitlab, redmine and an authenticated https file server to share releases. It's not meant for large scale usage (thousands of users) but for 99% of us it is just fine.

In this tutorial, we'll setup an infrastructure solution with shasoco on a linux server, and a live backup on a second server.

Installation of shasoco

We’ll start from a freshly installed linux server, with 8GB of RAM (recommended). The only dependency to install is docker. Installation for all major distributions is documented here.

Make sure you're using the most up-to-date release, shasoco requires docker 1.9

Once docker is installed, you can install shasoco with the below command:

curl https://raw.githubusercontent.com/shasoco/shasoco/master/shasoco | sudo tee /usr/local/bin/shasoco && sudo chmod +x /usr/local/bin/shasoco

(Don't have curl? You can use wget -qO- instead of curl in the above one-liner)

Now check that shasoco works by typing:

shasoco

It should download the shasoco/shasoco image from dockerhub: this may take a while depending on your internet connection.

Declaration of the infrastructure

We want to install our infrastructure on the fovea.cc domain. GitLab will be reachable from gitlab.fovea.cc, wordpress from www.fovea.cc, redmine from redmine.fovea.cc, fusiondirectory from directory.fovea.cc, https files server from vault.fovea.cc. Make sure to setup your DNS entries to direct traffic to your server.

So let's declare our deployment. On the command line:

shasoco create v0 fovea.cc

This will declare a new deployment whose ID is v0 linked to the domain fovea.cc. There are some more options you can add, you can list them by typing:

shasoco help create

In particular, you most probably will have to use the --git-ssh-port, if your server already has ssh running on port 22 (and you don't want to change that).

What the create command did is generate a yaml configuration file in /var/lib/shasoco/deploys/v0/config.yml. We do not have a ssl certificate, so it also generated a self-signed one for us, as well as passwords for root and admin users.

Instead of looking for the config file, you can also learn about a deploy by running shasoco inspect v0. This will give you basically the same info, with some dynamic additional properties (is the deploy running and/or exposed on public ports).

Booting things up

To start the deploy services, use the following command:

shasoco up v0

The first deployment will probably take a long time, as it will pull quite a few images from dockerhub. Next deployments should just be a matter of 2-3 minutes.

Hopefully, everything should start: as docker tends to make things pretty predictable. The infrastructure is now up, but not yet public: you have to set this deploy as the active one.

shasoco activate v0

This will start a proxy that exposes the v0 deploy to the world. You can check this by running:

shasoco ps

You should see only v0 with the list of exposed ports. To stop incoming traffic, use shasoco deactivate fovea.cc. Why this extra layer? The idea behind it is to allow multiple deployments for the same domain to be up on the same machine at the same time. You can then clone a deploy, upgrade it, run some tests, and activate the upgraded version if you're happy with it. Doing so, users will only experience a few millis of downtime, you can also rollback instantly to the old version if there's any issue. But we'll see that later.

Usage

Now that you have an active deploy, what does it give you?

LDAP / Fusiondirectory

URL: https://directory.DOMAIN/fusiondirectory

All services will share the same users and password. Users can be created from fusiondirectory.

There's a bit of setup:

1. log-in with username admin and password found using the shasoco inspect v0 command.
2. in Systems, create a new server (Action → New Server), call it mail with IP 127.0.0.1 (or anything). Click OK. Edit the mail server to add the IMAP/POP service (on the second tab). Hit save then OK (why do we need to save twice, I don't know…)

Then you can create users. In Users, select Action → New User. Fill in the required informations, click OK. Then edit the user to add an email address (on the second tab) and save. This user can now access all services using his credentials.

Note that users may not be visible if you do not check Show subtrees on the right hand side.

Redmine

URL: https://redmine.DOMAIN

Administrator user is admin, password is the usual admin password (found using shasoco inspect v0). Other users are retrieved from the LDAP service.

GitLab

URL: https://gitlab.DOMAIN

Administrator user is root, password is the usual admin password. The trick with GitLab is that LDAP users that try to login for the first time on gitlab will encounter an error: "You've been blocked. Administrator need to unblock your account". The administrator has to login and unblock each new user.

WordPress

URL: http://www.DOMAIN

It's an empty install. I'd recommend using the same admin password (to make things simple). There's no connection with LDAP.

Some setup is required to have outgoing emails working from wordpress. Add the hectane plugin, use mail as the hostname and Ignore TLS errors.

Vault

URL: https://vault.DOMAIN

Files from the vault are located in /var/lib/shasoco/vault.DOMAIN. No special setup is required.

Backup / Restore

Shasoco provides commands to backup the whole infrastructure:

shasoco backup v0 bak0001

This will create a backup in/var/lib/shasoco/deploys/v1/backups/bak0001.tar.gz. However, as you may have noticed, this won't work if your deploy is up: this is to prevent having corrupted backups. So you have to stop your deploy first:

shasoco stop v0 && shasoco backup v0 bak0001

Then you can restart:

shasoco up v0 && shasoco activate v0

The above two command lines can be concatenated into a single one, launched from a cron job.

Backup can then be restored on the same deploy, or on a new one. You will often want to restore on a new deploy, because if something is wrong with the current one you may like to found out what it is, instead of erasing interesting evidences!

shasoco create v1 fovea.cc
shasoco restore v1 v0/bak0001
shasoco up v1
shasoco activate v1

The v0 deploy may still be running, but only the v1 one will be accessible to the outside. Time to inspect v0.

And the vault?

The only thing that isn't part of the backup is the vault. Files in /var/lib/shasoco/vault.DOMAIN are expected to be pretty heavy, so they are not backed up on the same host. We're good people, so we'll setup a remote backup of /var/lib/shasoco somewhere anyway.

Migration

Migrating a deploy (or whole infrastructure) is easy.

1. prepare a new host with docker and shasoco installed (see on top)
2. enter maintenance mode with shasoco deactivate DOMAIN
3. backup the deploys you want to migrate, eg: shasoco backup v0 mig
4. copy /var/lib/shasoco to the new host (or cherry-pick what you want to migrate)
5. from the new host restore backups, eg: shasoco restore v0 mig
6. up and activate as usual
7. DNS changes

Steps 1 to 7 can be automated in a script, that gives you a live backup of your infrastructure (in case of major failure).

Live backup

A live backup is the best way to be sure you always have a working backup of your data. Let's set this up.

First thing, prepare a second host by installing shasoco and docker (see on top). Make sure you can ssh without password from the backup host to the live server, by adding an entry in authorized_keys.

[email protected] # ssh-copy-id www.DOMAIN
[email protected] # ssh www.DOMAIN

Let's backup our main server every days at 3:00 am. Call this backup 0.

[email protected] # crontab -e

0 3 * * * shasoco deactivate fovea.cc && shasoco stop v0 && shasoco backup v0 0 && shasoco up v0 && shasoco activate v0

On the backup host, we'll create our clone-shasoco.sh script:

#!/bin/bash
rsync -a --delete www.DOMAIN:/var/lib/shasco/ /var/lib/shasoco
shasoco stop v0
shasoco restore v0 v0/0
shasoco up v0
shasoco activate v0

And schedule it every days at 5:00 am.

[email protected] # crontab -e

0 5 * * * /clone-shasoco.sh

After that, you should be able to test the clone by faking DNS in your /etc/hosts. A cool plus would be to have some automated tests on main and backup servers to be sure we're alerted if some service stops working on either the master or the clone (it's never too early to realize that your automated backup isn't working anymore).

We could also add some sugar to keep historical backups:

• renaming the previous backup before rsync in the clone-shasoco.sh script
• push the backups to a cloud storage service
• …

I will let this as an exercice!

Conclusion

I believed we covered almost everything there is to know about shasoco, if you followed this tutorial you should now have a nice setup to manage your projects, and some peace of mind that your data is in good hands.

Of course shasoco is a very young project, I'm looking forward for feedbacks (and contributions, let's dream) if you find this piece of software useful!