jenkins.plone.org Set Up Howto

This document describes how to set up the entire Jenkins infrastructure for jenkins.plone.org. Those are the main steps:

  • Set up Jenkins server (jenkins.plone.org, with Ansible)
  • Set up Jenkins nodes (node[1-x].jenkins.plone.org, with Ansible)
  • Set up the Jenkins jobs on the Jenkins server (with Jenkins Job Builder)

Prerequisites

Checkout this repository:

git clone git@github.com:plone/jenkins.plone.org.git
cd jenkins.plone.org

Create and activate a virtualenv:

python3.6 -m venv .
./bin/activate

Install all the tools needed (ansible, ansible roles and jenkins-job-builder):

pip install -r requirements.txt
ansible-galaxy install -r ansible/roles.yml
git submodule update --init

Note

For the roles that are downloaded from checkouts, plone.jenkins_server and plone.jenkins_node, you will need to remove them and clone them manually if you want to make changes on them.

cd ansible/roles
rm -rf plone.jenkins_server
rm -rf plone.jenkins_node
git clone git@github.com:plone/plone.jenkins_server
git clone git@github.com:plone/plone.jenkins_node

Check ansible/inventory.txt and make sure that you can connect to the machines listed there.

Copy your public ssh key to all servers:

ssh-copy-id -i ~/.ssh/<SSH-KEY>.pub root@<SERVER_IP>

Set Up Jenkins Server

./update_master.sh

Set Up Jenkins Nodes

./update_nodes.sh

Set Up Jenkins Jobs

Do the steps described above to clone, activate virtualenv and fetch submodules.

Put jenkins-job-builder in development mode:

cd src/jenkins-job-builder
pip install -r requirements.txt -c ../../requirements.txt
python setup.py develop

Test the jobs are properly setup:

jenkins-jobs --conf jobs/config.ini.in test jobs/jobs.yml -o output

Note

A folder named output should contain one file per each jenkins job configured on jobs.yml

Create your own jobs/config.ini by copying it from jobs/config.ini.in:

cp jobs/config.ini.in jobs/config.ini

Add your own credentials to jobs/config.ini. You can find them when you log into Jenkins and copy your API token (e.g. http://jenkins.plone.org/user/tisto/configure).

Create your own ansible/secrets.yml by copying it from ansible/secrets.yml.in:

$ cp ansible/secrets.yml.in ansible/secrets.yml

Add github API secrets that are needed for the github login functionality on jenkins.plone.org. You can find those settings on plone organization in github: https://github.com/organizations/plone/settings/applications

Look for the Plone Jenkins CI application name.

For the github_api_key you need a personal token (from https://github.com/jenkins-plone-org github user).

Now finally install the jobs on the server:

./update_jobs.sh

Manual Configuration

There are currently a few steps that we need to carry out manually. We will automate them later.

  1. Github post-commit hook for buildout.coredev:
  1. Manage Jenkins -> Configure System:
  • E-mail Notification:
  1. Manage Jenkins -> Manage Credentials -> Add Credentials: SSH Username with private key:
  • Scope: System
  • Username: jenkins
  • Description: jenkins.plone.org private ssh key
  • Private Key: From a file on Jenkins master: File: /var/lib/jenkins/jenkins.plone.org

=> Upload jenkins.plone.org private ssh key manually to /var/lib/jenkins => chown jenkins:jenkins jenkins.plone.org