Someone wants to deploy a project management infrastructure on premise, luckily there's a large choice of open-source solutions out there: GitLab for version control and development, Redmine for team communication, WordPress for the website, OpenLDAP for authentication, etc. However, the cost of installing, configuring and managing all of those by himself didn't really seem worth it. Or is it?
Shasoco takes care of everything: easy installation, updates, backups, migration. With shasoco, you'll get a website with wordpress, an ldap server, a directory, gitlab, redmine and an authenticated https file server to share releases. It's not meant for large scale usage (thousands of users) but for 99% of us it is just fine.
In this tutorial, we'll setup an infrastructure solution with shasoco on a linux server, and a live backup on a second server.
Installation of shasoco
We’ll start from a freshly installed linux server, with 8GB of RAM (recommended). The only dependency to install is docker. Installation for all major distributions is documented here.
Make sure you're using the most up-to-date release, shasoco requires docker 1.9
Once docker is installed, you can install shasoco with the below command:
curl https://raw.githubusercontent.com/shasoco/shasoco/master/shasoco | sudo tee /usr/local/bin/shasoco && sudo chmod +x /usr/local/bin/shasoco
(Don't have curl? You can use
wget -qO- instead of
curl in the above one-liner)
Now check that shasoco works by typing:
It should download the
shasoco/shasoco image from dockerhub: this may take a while depending on your internet connection.
Declaration of the infrastructure
We want to install our infrastructure on the
fovea.cc domain. GitLab will be reachable from
gitlab.fovea.cc, wordpress from
www.fovea.cc, redmine from
redmine.fovea.cc, fusiondirectory from
directory.fovea.cc, https files server from
vault.fovea.cc. Make sure to setup your DNS entries to direct traffic to your server.
So let's declare our deployment. On the command line:
shasoco create v0 fovea.cc
This will declare a new deployment whose ID is
v0 linked to the domain
fovea.cc. There are some more options you can add, you can list them by typing:
shasoco help create
In particular, you most probably will have to use the
--git-ssh-port, if your server already has ssh running on port
22 (and you don't want to change that).
create command did is generate a yaml configuration file in
/var/lib/shasoco/deploys/v0/config.yml. We do not have a ssl certificate, so it also generated a self-signed one for us, as well as passwords for
Instead of looking for the config file, you can also learn about a deploy by running
shasoco inspect v0. This will give you basically the same info, with some dynamic additional properties (is the deploy running and/or exposed on public ports).
Booting things up
To start the deploy services, use the following command:
shasoco up v0
The first deployment will probably take a long time, as it will pull quite a few images from dockerhub. Next deployments should just be a matter of 2-3 minutes.
Hopefully, everything should start: as docker tends to make things pretty predictable. The infrastructure is now up, but not yet public: you have to set this deploy as the active one.
shasoco activate v0
This will start a proxy that exposes the
v0 deploy to the world. You can check this by running:
You should see only
v0 with the list of exposed ports. To stop incoming traffic, use
shasoco deactivate fovea.cc. Why this extra layer? The idea behind it is to allow multiple deployments for the same domain to be up on the same machine at the same time. You can then clone a deploy, upgrade it, run some tests, and activate the upgraded version if you're happy with it. Doing so, users will only experience a few millis of downtime, you can also rollback instantly to the old version if there's any issue. But we'll see that later.
Now that you have an active deploy, what does it give you?
LDAP / Fusiondirectory
All services will share the same users and password. Users can be created from fusiondirectory.
There's a bit of setup:
- log-in with username
adminand password found using the
shasoco inspect v0command.
Systems, create a new server (Action → New Server), call it
127.0.0.1(or anything). Click OK. Edit the
IMAP/POPservice (on the second tab). Hit
OK(why do we need to save twice, I don't know…)
Then you can create users. In
Action → New User. Fill in the required informations, click OK. Then edit the user to add an email address (on the second tab) and save. This user can now access all services using his credentials.
Note that users may not be visible if you do not check
Show subtrees on the right hand side.
Administrator user is
admin, password is the usual admin password (found using
shasoco inspect v0). Other users are retrieved from the LDAP service.
Administrator user is
root, password is the usual admin password. The trick with GitLab is that LDAP users that try to login for the first time on gitlab will encounter an error: "You've been blocked. Administrator need to unblock your account". The administrator has to login and unblock each new user.
It's an empty install. I'd recommend using the same admin password (to make things simple). There's no connection with LDAP.
Some setup is required to have outgoing emails working from wordpress. Add the
hectane plugin, use
Ignore TLS errors.
Files from the vault are located in
/var/lib/shasoco/vault.DOMAIN. No special setup is required.
Backup / Restore
Shasoco provides commands to backup the whole infrastructure:
shasoco backup v0 bak0001
This will create a backup in
/var/lib/shasoco/deploys/v1/backups/bak0001.tar.gz. However, as you may have noticed, this won't work if your deploy is up: this is to prevent having corrupted backups. So you have to stop your deploy first:
shasoco stop v0 && shasoco backup v0 bak0001
Then you can restart:
shasoco up v0 && shasoco activate v0
The above two command lines can be concatenated into a single one, launched from a cron job.
Backup can then be restored on the same deploy, or on a new one. You will often want to restore on a new deploy, because if something is wrong with the current one you may like to found out what it is, instead of erasing interesting evidences!
shasoco create v1 fovea.cc shasoco restore v1 v0/bak0001 shasoco up v1 shasoco activate v1
v0 deploy may still be running, but only the
v1 one will be accessible to the outside. Time to inspect
And the vault?
The only thing that isn't part of the backup is the vault. Files in
/var/lib/shasoco/vault.DOMAIN are expected to be pretty heavy, so they are not backed up on the same host. We're good people, so we'll setup a remote backup of
/var/lib/shasoco somewhere anyway.
Migrating a deploy (or whole infrastructure) is easy.
- prepare a new host with docker and shasoco installed (see on top)
- enter maintenance mode with
shasoco deactivate DOMAIN
- backup the deploys you want to migrate, eg:
shasoco backup v0 mig
/var/lib/shasocoto the new host (or cherry-pick what you want to migrate)
- from the new host restore backups, eg:
shasoco restore v0 mig
- DNS changes
Steps 1 to 7 can be automated in a script, that gives you a live backup of your infrastructure (in case of major failure).
A live backup is the best way to be sure you always have a working backup of your data. Let's set this up.
First thing, prepare a second host by installing shasoco and docker (see on top). Make sure you can ssh without password from the backup host to the live server, by adding an entry in
root@backup # ssh-copy-id www.DOMAIN root@backup # ssh www.DOMAIN
Let's backup our main server every days at 3:00 am. Call this backup
root@main # crontab -e 0 3 * * * shasoco deactivate fovea.cc && shasoco stop v0 && shasoco backup v0 0 && shasoco up v0 && shasoco activate v0
On the backup host, we'll create our
#!/bin/bash rsync -a --delete www.DOMAIN:/var/lib/shasco/ /var/lib/shasoco shasoco stop v0 shasoco restore v0 v0/0 shasoco up v0 shasoco activate v0
And schedule it every days at 5:00 am.
root@backup # crontab -e 0 5 * * * /clone-shasoco.sh
After that, you should be able to test the clone by faking DNS in your
/etc/hosts. A cool plus would be to have some automated tests on main and backup servers to be sure we're alerted if some service stops working on either the master or the clone (it's never too early to realize that your automated backup isn't working anymore).
We could also add some sugar to keep historical backups:
- renaming the previous backup before rsync in the
- push the backups to a cloud storage service
I will let this as an exercice!
I believed we covered almost everything there is to know about shasoco, if you followed this tutorial you should now have a nice setup to manage your projects, and some peace of mind that your data is in good hands.
Of course shasoco is a very young project, I'm looking forward for feedbacks (and contributions, let's dream) if you find this piece of software useful!